rckrdmrd
f570727617
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
159 lines
4.4 KiB
Markdown
159 lines
4.4 KiB
Markdown
---
|
|
id: "RF-SCR-002"
|
|
title: "Gestion de Pool de Proxies"
|
|
type: "Functional Requirement"
|
|
epic: "IAI-007"
|
|
priority: "Alta"
|
|
status: "Draft"
|
|
project: "inmobiliaria-analytics"
|
|
created_date: "2026-01-04"
|
|
updated_date: "2026-01-04"
|
|
---
|
|
|
|
# RF-IA-007-002: Gestion de Pool de Proxies
|
|
|
|
---
|
|
|
|
## Descripcion
|
|
|
|
El sistema debe gestionar un pool de proxies residenciales para distribuir las solicitudes y evitar bloqueos por IP, incluyendo rotacion automatica, health checks y cooling periods.
|
|
|
|
---
|
|
|
|
## Justificacion
|
|
|
|
Cloudflare y los portales inmobiliarios rastrean IPs y bloquean aquellas con comportamiento sospechoso. Un pool de proxies residenciales permite distribuir la carga y simular trafico desde multiples ubicaciones geograficas legitimas.
|
|
|
|
---
|
|
|
|
## Requisitos Funcionales
|
|
|
|
### RF-002.1: Gestion de Pool
|
|
|
|
| ID | Requisito | Prioridad |
|
|
|----|-----------|-----------|
|
|
| RF-002.1.1 | El sistema debe mantener un pool de proxies residenciales | Alta |
|
|
| RF-002.1.2 | El sistema debe almacenar metadata de cada proxy (tipo, pais, status) | Alta |
|
|
| RF-002.1.3 | El sistema debe soportar multiples proveedores de proxy | Media |
|
|
| RF-002.1.4 | El sistema debe permitir agregar/remover proxies dinamicamente | Media |
|
|
|
|
### RF-002.2: Rotacion
|
|
|
|
| ID | Requisito | Prioridad |
|
|
|----|-----------|-----------|
|
|
| RF-002.2.1 | El sistema debe rotar proxies por sesion o por request | Alta |
|
|
| RF-002.2.2 | El sistema debe seleccionar proxies con mejor success rate | Alta |
|
|
| RF-002.2.3 | El sistema debe evitar proxies en cooling period | Alta |
|
|
| RF-002.2.4 | El sistema debe balancear carga entre proxies disponibles | Media |
|
|
|
|
### RF-002.3: Health Checks
|
|
|
|
| ID | Requisito | Prioridad |
|
|
|----|-----------|-----------|
|
|
| RF-002.3.1 | El sistema debe verificar conectividad de proxies periodicamente | Alta |
|
|
| RF-002.3.2 | El sistema debe marcar proxies como "banned" cuando detecte bloqueo | Alta |
|
|
| RF-002.3.3 | El sistema debe calcular y actualizar success rate por proxy | Alta |
|
|
| RF-002.3.4 | El sistema debe alertar cuando el pool este bajo umbral minimo | Media |
|
|
|
|
### RF-002.4: Cooling Periods
|
|
|
|
| ID | Requisito | Prioridad |
|
|
|----|-----------|-----------|
|
|
| RF-002.4.1 | El sistema debe poner proxies en cooling despues de rate limit | Alta |
|
|
| RF-002.4.2 | El sistema debe configurar duracion de cooling por tipo de error | Media |
|
|
| RF-002.4.3 | El sistema debe reactivar proxies automaticamente post-cooling | Alta |
|
|
|
|
---
|
|
|
|
## Modelo de Datos
|
|
|
|
```yaml
|
|
ProxyPool:
|
|
id: UUID
|
|
address: string
|
|
port: number
|
|
type: enum [residential, datacenter, mobile]
|
|
provider: string
|
|
country: string
|
|
city: string (opcional)
|
|
username: string (encrypted)
|
|
password: string (encrypted)
|
|
status: enum [active, cooling, banned, inactive]
|
|
success_rate: decimal (0-1)
|
|
total_requests: integer
|
|
successful_requests: integer
|
|
last_used_at: timestamp
|
|
last_success_at: timestamp
|
|
cooling_until: timestamp (nullable)
|
|
banned_at: timestamp (nullable)
|
|
created_at: timestamp
|
|
updated_at: timestamp
|
|
```
|
|
|
|
---
|
|
|
|
## Criterios de Aceptacion
|
|
|
|
- [ ] Pool almacena y gestiona 50+ proxies residenciales
|
|
- [ ] Rotacion selecciona proxies con mejor success rate
|
|
- [ ] Proxies con rate limit entran en cooling automaticamente
|
|
- [ ] Health check detecta proxies caidos en < 5 minutos
|
|
- [ ] Alertas se disparan cuando pool < 20 proxies activos
|
|
- [ ] Success rate se calcula correctamente por proxy
|
|
- [ ] Proxies banned no se seleccionan para nuevos requests
|
|
|
|
---
|
|
|
|
## Configuracion
|
|
|
|
```yaml
|
|
proxy_pool:
|
|
min_active_proxies: 20
|
|
health_check_interval_ms: 300000 # 5 minutos
|
|
|
|
rotation:
|
|
strategy: "weighted_random" # best_success, round_robin, weighted_random
|
|
change_every: "session" # request, session, n_requests
|
|
|
|
cooling:
|
|
rate_limit_duration_ms: 3600000 # 1 hora
|
|
error_duration_ms: 1800000 # 30 minutos
|
|
max_consecutive_failures: 3
|
|
|
|
providers:
|
|
- name: "brightdata"
|
|
priority: 1
|
|
- name: "iproyal"
|
|
priority: 2
|
|
```
|
|
|
|
---
|
|
|
|
## Dependencias
|
|
|
|
- Proveedor de proxies residenciales (Bright Data, IPRoyal, etc.)
|
|
- PostgreSQL para persistencia
|
|
- Redis para cache de status
|
|
|
|
---
|
|
|
|
## Costos Estimados
|
|
|
|
| Proveedor | Plan | Proxies | Costo/mes |
|
|
|-----------|------|---------|-----------|
|
|
| Bright Data | Residential | 5GB | $75 USD |
|
|
| IPRoyal | Residential | 5GB | $52 USD |
|
|
| Smartproxy | Residential | 5GB | $65 USD |
|
|
|
|
---
|
|
|
|
## Historias de Usuario Relacionadas
|
|
|
|
- US-SCR-001: Scraping de Inmuebles24
|
|
- US-SCR-002: Scraping de Vivanuncios
|
|
|
|
---
|
|
|
|
**Autor:** Tech Lead
|
|
**Fecha:** 2026-01-04
|