- Created TASK-2026-01-26-ANALYSIS-INTEGRATION-PLAN with complete CAPVED documentation - Orchestrated 5 specialized Explore agents in parallel (85% time reduction) - Identified 7 coherence gaps (DDL↔Backend↔Frontend) - Identified 4 P0 blockers preventing GO-LIVE - Documented 58 missing documentation items - Created detailed roadmap Q1-Q4 2026 (2,500h total) - Added 6 new ET specs for ML strategies (PVA, MRD, VBP, MSA, MTS, Backtesting) - Updated _INDEX.yml with new analysis task Hallazgos críticos: - E-COH-001 to E-COH-007: Coherence gaps (6.5h to fix) - BLOCKER-001 to 004: Token refresh, PCI-DSS, Video upload, MT4 Gateway (380h) - Documentation gaps: 8 ET specs, 8 US, 34 Swagger docs (47.5h) Roadmap phases: - Q1: Security & Blockers (249h) - Q2: Core Features + GO-LIVE (542h) - Q3: Scalability & Performance (380h) - Q4: Innovation & Advanced Features (1,514h) ROI: $223k investment → $750k revenue → $468k net profit (165% ROI) Next: Execute ST1 (Coherencia Fixes P0) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
939 lines
33 KiB
Markdown
939 lines
33 KiB
Markdown
---
|
|
id: "ET-ML-015"
|
|
title: "Backtesting Framework"
|
|
type: "Technical Specification"
|
|
status: "Approved"
|
|
priority: "Alta"
|
|
epic: "OQI-006"
|
|
project: "trading-platform"
|
|
version: "1.0.0"
|
|
created_date: "2026-01-25"
|
|
updated_date: "2026-01-25"
|
|
task_reference: "TASK-2026-01-25-ML-TRAINING-ENHANCEMENT"
|
|
---
|
|
|
|
# ET-ML-015: Backtesting Framework
|
|
|
|
## Metadata
|
|
|
|
| Campo | Valor |
|
|
|-------|-------|
|
|
| **ID** | ET-ML-015 |
|
|
| **Epica** | OQI-006 - Senales ML |
|
|
| **Tipo** | Especificacion Tecnica |
|
|
| **Version** | 1.0.0 |
|
|
| **Estado** | Aprobado |
|
|
| **Ultima actualizacion** | 2026-01-25 |
|
|
| **Tarea Referencia** | TASK-2026-01-25-ML-TRAINING-ENHANCEMENT |
|
|
|
|
---
|
|
|
|
## Resumen
|
|
|
|
El Backtesting Framework proporciona una infraestructura completa para evaluar estrategias ML en datos historicos. Incluye **BacktestEngine**, **PositionManager**, **MetricsCalculator**, validacion walk-forward, y position sizing basado en **Kelly Criterion**.
|
|
|
|
### Componentes Principales
|
|
|
|
- **BacktestEngine**: Motor principal de backtesting
|
|
- **PositionManager**: Gestion de posiciones y ordenes
|
|
- **MetricsCalculator**: Calculo de metricas de performance
|
|
- **EffectivenessValidator**: Validacion de efectividad (target 80%)
|
|
- **Walk-Forward Validator**: Validacion temporal robusta
|
|
|
|
---
|
|
|
|
## Arquitectura
|
|
|
|
### Diagrama General
|
|
|
|
```
|
|
┌─────────────────────────────────────────────────────────────────────────┐
|
|
│ BACKTESTING FRAMEWORK │
|
|
├─────────────────────────────────────────────────────────────────────────┤
|
|
│ │
|
|
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
|
│ │ BACKTEST ENGINE │ │
|
|
│ │ │ │
|
|
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ DATA HANDLER │ │ │
|
|
│ │ │ - Load historical data │ │ │
|
|
│ │ │ - Manage data streams │ │ │
|
|
│ │ │ - Handle multiple timeframes │ │ │
|
|
│ │ └─────────────────────────────────────────────────────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ▼ │ │
|
|
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ STRATEGY EXECUTOR │ │ │
|
|
│ │ │ - Execute ML predictions │ │ │
|
|
│ │ │ - Generate signals │ │ │
|
|
│ │ │ - Manage signal queue │ │ │
|
|
│ │ └─────────────────────────────────────────────────────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ▼ │ │
|
|
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ POSITION MANAGER │ │ │
|
|
│ │ │ - Open/close positions │ │ │
|
|
│ │ │ - Kelly position sizing │ │ │
|
|
│ │ │ - TP/SL management │ │ │
|
|
│ │ │ - Risk controls │ │ │
|
|
│ │ └─────────────────────────────────────────────────────────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ▼ │ │
|
|
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
|
|
│ │ │ METRICS CALCULATOR │ │ │
|
|
│ │ │ - Calculate returns │ │ │
|
|
│ │ │ - Risk metrics (Sharpe, Sortino, Max DD) │ │ │
|
|
│ │ │ - Trade statistics │ │ │
|
|
│ │ │ - ML-specific metrics │ │ │
|
|
│ │ └─────────────────────────────────────────────────────────────┘ │ │
|
|
│ └───────────────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ ┌───────────────────────────────────────────────────────────────────┐ │
|
|
│ │ WALK-FORWARD VALIDATOR │ │
|
|
│ │ │ │
|
|
│ │ ┌───────────┐ ┌───────────┐ ┌───────────┐ ┌───────────┐ │ │
|
|
│ │ │ Fold 1 │ │ Fold 2 │ │ Fold 3 │ │ Fold N │ │ │
|
|
│ │ │ Train/Test│ │ Train/Test│ │ Train/Test│ │ Train/Test│ │ │
|
|
│ │ └───────────┘ └───────────┘ └───────────┘ └───────────┘ │ │
|
|
│ │ │ │ │
|
|
│ │ ▼ │ │
|
|
│ │ ┌─────────────────────────────────────────────────┐ │ │
|
|
│ │ │ EFFECTIVENESS VALIDATOR │ │ │
|
|
│ │ │ Target: 80% Effectiveness │ │ │
|
|
│ │ └─────────────────────────────────────────────────┘ │ │
|
|
│ └───────────────────────────────────────────────────────────────────┘ │
|
|
│ │ │
|
|
│ ▼ │
|
|
│ Output: BacktestResults │
|
|
│ - Performance metrics │
|
|
│ - Trade history │
|
|
│ - Equity curve │
|
|
│ - Walk-forward analysis │
|
|
│ - Effectiveness score │
|
|
└─────────────────────────────────────────────────────────────────────────┘
|
|
```
|
|
|
|
---
|
|
|
|
## BacktestEngine
|
|
|
|
### Configuracion
|
|
|
|
```python
|
|
@dataclass
|
|
class BacktestConfig:
|
|
# Data
|
|
symbol: str
|
|
start_date: str
|
|
end_date: str
|
|
timeframe: str = '5m'
|
|
|
|
# Capital
|
|
initial_capital: float = 10000.0
|
|
currency: str = 'USD'
|
|
|
|
# Position sizing
|
|
position_sizing: str = 'kelly' # 'fixed', 'kelly', 'volatility'
|
|
max_position_pct: float = 0.1 # Max 10% per position
|
|
kelly_fraction: float = 0.5 # Half-Kelly
|
|
|
|
# Risk management
|
|
max_drawdown_pct: float = 0.2 # Stop trading at 20% DD
|
|
max_daily_loss_pct: float = 0.05
|
|
max_open_positions: int = 3
|
|
|
|
# Execution
|
|
slippage_pct: float = 0.001 # 0.1% slippage
|
|
commission_pct: float = 0.001 # 0.1% commission
|
|
|
|
# Walk-forward
|
|
walk_forward_folds: int = 5
|
|
train_ratio: float = 0.8
|
|
```
|
|
|
|
### Clase Principal
|
|
|
|
```python
|
|
class BacktestEngine:
|
|
"""Motor principal de backtesting"""
|
|
|
|
def __init__(self, config: BacktestConfig):
|
|
self.config = config
|
|
self.data_handler = DataHandler(config)
|
|
self.position_manager = PositionManager(config)
|
|
self.metrics_calculator = MetricsCalculator()
|
|
self.trade_log: List[Trade] = []
|
|
self.equity_curve: List[float] = []
|
|
|
|
def run(
|
|
self,
|
|
strategy: MLStrategy,
|
|
data: pd.DataFrame
|
|
) -> BacktestResults:
|
|
"""Ejecutar backtest completo"""
|
|
|
|
self.equity_curve = [self.config.initial_capital]
|
|
|
|
for i in range(len(data)):
|
|
bar = data.iloc[i]
|
|
|
|
# 1. Update positions with current prices
|
|
self._update_positions(bar)
|
|
|
|
# 2. Check exit conditions (TP/SL)
|
|
self._check_exits(bar)
|
|
|
|
# 3. Generate signal from strategy
|
|
signal = strategy.generate_signal(data.iloc[:i+1])
|
|
|
|
# 4. Execute signal if valid
|
|
if signal and self._can_trade():
|
|
self._execute_signal(signal, bar)
|
|
|
|
# 5. Record equity
|
|
self.equity_curve.append(self._calculate_equity(bar))
|
|
|
|
# 6. Check risk limits
|
|
if self._check_risk_limits():
|
|
break
|
|
|
|
return self._compile_results()
|
|
|
|
def _execute_signal(self, signal: Signal, bar: pd.Series):
|
|
"""Ejecutar senal de trading"""
|
|
# Calculate position size
|
|
size = self.position_manager.calculate_size(
|
|
signal=signal,
|
|
current_price=bar['close'],
|
|
account_value=self.equity_curve[-1]
|
|
)
|
|
|
|
if size > 0:
|
|
# Apply slippage
|
|
entry_price = self._apply_slippage(
|
|
bar['close'],
|
|
signal.direction
|
|
)
|
|
|
|
# Open position
|
|
position = self.position_manager.open_position(
|
|
direction=signal.direction,
|
|
size=size,
|
|
entry_price=entry_price,
|
|
stop_loss=signal.stop_loss,
|
|
take_profit=signal.take_profit,
|
|
timestamp=bar.name
|
|
)
|
|
|
|
self.trade_log.append(position)
|
|
```
|
|
|
|
---
|
|
|
|
## PositionManager
|
|
|
|
### Kelly Criterion Position Sizing
|
|
|
|
```python
|
|
class PositionManager:
|
|
"""Gestion de posiciones con Kelly sizing"""
|
|
|
|
def __init__(self, config: BacktestConfig):
|
|
self.config = config
|
|
self.positions: List[Position] = []
|
|
self.closed_positions: List[Position] = []
|
|
self.win_rate: float = 0.5 # Updated dynamically
|
|
self.avg_win: float = 0.01
|
|
self.avg_loss: float = 0.01
|
|
|
|
def calculate_kelly_fraction(self) -> float:
|
|
"""Calcular fraccion de Kelly optima"""
|
|
# Kelly formula: f* = (p * b - q) / b
|
|
# p = win probability
|
|
# q = loss probability (1 - p)
|
|
# b = win/loss ratio
|
|
|
|
p = self.win_rate
|
|
q = 1 - p
|
|
b = self.avg_win / self.avg_loss if self.avg_loss > 0 else 1
|
|
|
|
kelly = (p * b - q) / b
|
|
|
|
# Apply fraction (half-Kelly recommended)
|
|
kelly *= self.config.kelly_fraction
|
|
|
|
# Cap at max position
|
|
kelly = min(kelly, self.config.max_position_pct)
|
|
kelly = max(kelly, 0) # No negative
|
|
|
|
return kelly
|
|
|
|
def calculate_size(
|
|
self,
|
|
signal: Signal,
|
|
current_price: float,
|
|
account_value: float
|
|
) -> float:
|
|
"""Calcular tamano de posicion"""
|
|
|
|
if self.config.position_sizing == 'fixed':
|
|
position_value = account_value * self.config.max_position_pct
|
|
|
|
elif self.config.position_sizing == 'kelly':
|
|
kelly_fraction = self.calculate_kelly_fraction()
|
|
position_value = account_value * kelly_fraction
|
|
|
|
# Adjust for signal confidence
|
|
position_value *= signal.confidence
|
|
|
|
elif self.config.position_sizing == 'volatility':
|
|
# Risk parity based on volatility
|
|
volatility = signal.expected_volatility or 0.02
|
|
target_risk = 0.02 # 2% target volatility contribution
|
|
position_value = (target_risk / volatility) * account_value
|
|
position_value = min(position_value, account_value * self.config.max_position_pct)
|
|
|
|
# Convert to units
|
|
units = position_value / current_price
|
|
|
|
return units
|
|
|
|
def update_statistics(self):
|
|
"""Actualizar estadisticas de trading"""
|
|
if len(self.closed_positions) < 10:
|
|
return
|
|
|
|
wins = [p for p in self.closed_positions if p.pnl > 0]
|
|
losses = [p for p in self.closed_positions if p.pnl <= 0]
|
|
|
|
self.win_rate = len(wins) / len(self.closed_positions)
|
|
|
|
if wins:
|
|
self.avg_win = np.mean([p.pnl_pct for p in wins])
|
|
if losses:
|
|
self.avg_loss = abs(np.mean([p.pnl_pct for p in losses]))
|
|
```
|
|
|
|
### Position Class
|
|
|
|
```python
|
|
@dataclass
|
|
class Position:
|
|
id: str
|
|
symbol: str
|
|
direction: str # 'LONG' or 'SHORT'
|
|
size: float # Units
|
|
entry_price: float
|
|
entry_time: datetime
|
|
stop_loss: Optional[float] = None
|
|
take_profit: Optional[float] = None
|
|
exit_price: Optional[float] = None
|
|
exit_time: Optional[datetime] = None
|
|
exit_reason: Optional[str] = None # 'TP', 'SL', 'SIGNAL', 'MANUAL'
|
|
|
|
@property
|
|
def is_open(self) -> bool:
|
|
return self.exit_price is None
|
|
|
|
@property
|
|
def pnl(self) -> float:
|
|
if self.exit_price is None:
|
|
return 0
|
|
if self.direction == 'LONG':
|
|
return (self.exit_price - self.entry_price) * self.size
|
|
else:
|
|
return (self.entry_price - self.exit_price) * self.size
|
|
|
|
@property
|
|
def pnl_pct(self) -> float:
|
|
if self.exit_price is None:
|
|
return 0
|
|
if self.direction == 'LONG':
|
|
return (self.exit_price - self.entry_price) / self.entry_price
|
|
else:
|
|
return (self.entry_price - self.exit_price) / self.entry_price
|
|
|
|
@property
|
|
def duration(self) -> Optional[timedelta]:
|
|
if self.exit_time is None:
|
|
return None
|
|
return self.exit_time - self.entry_time
|
|
```
|
|
|
|
---
|
|
|
|
## MetricsCalculator
|
|
|
|
### Metricas Implementadas
|
|
|
|
```python
|
|
class MetricsCalculator:
|
|
"""Calculo de metricas de performance"""
|
|
|
|
def calculate_all_metrics(
|
|
self,
|
|
equity_curve: List[float],
|
|
trades: List[Position],
|
|
benchmark: Optional[pd.Series] = None
|
|
) -> BacktestMetrics:
|
|
"""Calcular todas las metricas"""
|
|
|
|
returns = self._calculate_returns(equity_curve)
|
|
|
|
return BacktestMetrics(
|
|
# Return metrics
|
|
total_return=self.total_return(equity_curve),
|
|
cagr=self.cagr(equity_curve),
|
|
annualized_return=self.annualized_return(returns),
|
|
|
|
# Risk metrics
|
|
volatility=self.volatility(returns),
|
|
max_drawdown=self.max_drawdown(equity_curve),
|
|
avg_drawdown=self.avg_drawdown(equity_curve),
|
|
calmar_ratio=self.calmar_ratio(equity_curve),
|
|
|
|
# Risk-adjusted returns
|
|
sharpe_ratio=self.sharpe_ratio(returns),
|
|
sortino_ratio=self.sortino_ratio(returns),
|
|
|
|
# Trade metrics
|
|
n_trades=len(trades),
|
|
win_rate=self.win_rate(trades),
|
|
profit_factor=self.profit_factor(trades),
|
|
avg_trade_return=self.avg_trade_return(trades),
|
|
avg_win=self.avg_win(trades),
|
|
avg_loss=self.avg_loss(trades),
|
|
largest_win=self.largest_win(trades),
|
|
largest_loss=self.largest_loss(trades),
|
|
avg_trade_duration=self.avg_trade_duration(trades),
|
|
|
|
# ML-specific
|
|
prediction_accuracy=self.prediction_accuracy(trades),
|
|
signal_quality=self.signal_quality(trades)
|
|
)
|
|
```
|
|
|
|
### Formulas de Metricas
|
|
|
|
#### Sharpe Ratio
|
|
|
|
```python
|
|
def sharpe_ratio(
|
|
self,
|
|
returns: np.ndarray,
|
|
risk_free_rate: float = 0.02,
|
|
periods_per_year: int = 252 * 24 * 12 # 5-min candles
|
|
) -> float:
|
|
"""Calcular Sharpe Ratio anualizado"""
|
|
if len(returns) < 2:
|
|
return 0
|
|
|
|
excess_returns = returns - (risk_free_rate / periods_per_year)
|
|
if np.std(excess_returns) == 0:
|
|
return 0
|
|
|
|
sharpe = np.mean(excess_returns) / np.std(excess_returns)
|
|
sharpe_annualized = sharpe * np.sqrt(periods_per_year)
|
|
|
|
return sharpe_annualized
|
|
```
|
|
|
|
#### Sortino Ratio
|
|
|
|
```python
|
|
def sortino_ratio(
|
|
self,
|
|
returns: np.ndarray,
|
|
risk_free_rate: float = 0.02,
|
|
periods_per_year: int = 252 * 24 * 12
|
|
) -> float:
|
|
"""Calcular Sortino Ratio (solo downside risk)"""
|
|
if len(returns) < 2:
|
|
return 0
|
|
|
|
excess_returns = returns - (risk_free_rate / periods_per_year)
|
|
downside_returns = excess_returns[excess_returns < 0]
|
|
|
|
if len(downside_returns) == 0 or np.std(downside_returns) == 0:
|
|
return 0
|
|
|
|
sortino = np.mean(excess_returns) / np.std(downside_returns)
|
|
sortino_annualized = sortino * np.sqrt(periods_per_year)
|
|
|
|
return sortino_annualized
|
|
```
|
|
|
|
#### Maximum Drawdown
|
|
|
|
```python
|
|
def max_drawdown(self, equity_curve: List[float]) -> float:
|
|
"""Calcular maximo drawdown"""
|
|
equity = np.array(equity_curve)
|
|
peak = np.maximum.accumulate(equity)
|
|
drawdown = (peak - equity) / peak
|
|
|
|
return np.max(drawdown)
|
|
```
|
|
|
|
#### Profit Factor
|
|
|
|
```python
|
|
def profit_factor(self, trades: List[Position]) -> float:
|
|
"""Calcular profit factor (gross profit / gross loss)"""
|
|
gross_profit = sum(t.pnl for t in trades if t.pnl > 0)
|
|
gross_loss = abs(sum(t.pnl for t in trades if t.pnl < 0))
|
|
|
|
if gross_loss == 0:
|
|
return float('inf') if gross_profit > 0 else 0
|
|
|
|
return gross_profit / gross_loss
|
|
```
|
|
|
|
---
|
|
|
|
## Walk-Forward Validation
|
|
|
|
### Implementacion
|
|
|
|
```python
|
|
class WalkForwardValidator:
|
|
"""Validacion walk-forward para backtesting"""
|
|
|
|
def __init__(
|
|
self,
|
|
n_folds: int = 5,
|
|
train_ratio: float = 0.8,
|
|
gap_periods: int = 0
|
|
):
|
|
self.n_folds = n_folds
|
|
self.train_ratio = train_ratio
|
|
self.gap_periods = gap_periods
|
|
|
|
def validate(
|
|
self,
|
|
strategy_class: Type[MLStrategy],
|
|
data: pd.DataFrame,
|
|
backtest_config: BacktestConfig
|
|
) -> WalkForwardResults:
|
|
"""Ejecutar validacion walk-forward"""
|
|
|
|
n_samples = len(data)
|
|
fold_size = n_samples // (self.n_folds + 1)
|
|
|
|
fold_results = []
|
|
|
|
for fold in range(self.n_folds):
|
|
# Define train/test split
|
|
train_end = (fold + 1) * fold_size
|
|
test_start = train_end + self.gap_periods
|
|
test_end = min(test_start + int(fold_size * (1 - self.train_ratio)), n_samples)
|
|
|
|
train_data = data.iloc[:train_end]
|
|
test_data = data.iloc[test_start:test_end]
|
|
|
|
# Train strategy
|
|
strategy = strategy_class()
|
|
strategy.fit(train_data)
|
|
|
|
# Backtest on test period
|
|
engine = BacktestEngine(backtest_config)
|
|
results = engine.run(strategy, test_data)
|
|
|
|
fold_results.append({
|
|
'fold': fold + 1,
|
|
'train_size': len(train_data),
|
|
'test_size': len(test_data),
|
|
'sharpe': results.metrics.sharpe_ratio,
|
|
'return': results.metrics.total_return,
|
|
'max_dd': results.metrics.max_drawdown,
|
|
'win_rate': results.metrics.win_rate,
|
|
'n_trades': results.metrics.n_trades
|
|
})
|
|
|
|
return self._aggregate_results(fold_results)
|
|
|
|
def _aggregate_results(
|
|
self,
|
|
fold_results: List[Dict]
|
|
) -> WalkForwardResults:
|
|
"""Agregar resultados de todos los folds"""
|
|
|
|
sharpes = [f['sharpe'] for f in fold_results]
|
|
returns = [f['return'] for f in fold_results]
|
|
max_dds = [f['max_dd'] for f in fold_results]
|
|
win_rates = [f['win_rate'] for f in fold_results]
|
|
|
|
return WalkForwardResults(
|
|
n_folds=len(fold_results),
|
|
fold_results=fold_results,
|
|
avg_sharpe=np.mean(sharpes),
|
|
std_sharpe=np.std(sharpes),
|
|
avg_return=np.mean(returns),
|
|
std_return=np.std(returns),
|
|
avg_max_dd=np.mean(max_dds),
|
|
avg_win_rate=np.mean(win_rates),
|
|
stability_score=1 - np.std(sharpes) / (np.mean(sharpes) + 1e-6)
|
|
)
|
|
```
|
|
|
|
### Visualizacion Walk-Forward
|
|
|
|
```
|
|
Time ──────────────────────────────────────────────────────────────▶
|
|
|
|
Fold 1: [========== TRAIN ==========][gap][===TEST===]
|
|
▼
|
|
Fold 2: [============== TRAIN ==============][gap][===TEST===]
|
|
▼
|
|
Fold 3: [================== TRAIN ==================][gap][===TEST===]
|
|
▼
|
|
Fold 4: [====================== TRAIN ======================][gap][===TEST===]
|
|
▼
|
|
Fold 5: [========================== TRAIN ==========================][gap][===TEST===]
|
|
```
|
|
|
|
---
|
|
|
|
## Effectiveness Validator
|
|
|
|
### Target: 80% Effectiveness
|
|
|
|
```python
|
|
class EffectivenessValidator:
|
|
"""Validar efectividad de estrategia ML"""
|
|
|
|
EFFECTIVENESS_TARGET = 0.80 # 80% target
|
|
|
|
def __init__(self, min_trades: int = 30):
|
|
self.min_trades = min_trades
|
|
|
|
def validate(
|
|
self,
|
|
results: BacktestResults,
|
|
predictions: List[Prediction]
|
|
) -> EffectivenessScore:
|
|
"""Calcular score de efectividad"""
|
|
|
|
if len(results.trades) < self.min_trades:
|
|
return EffectivenessScore(
|
|
score=0,
|
|
is_valid=False,
|
|
message=f"Insufficient trades: {len(results.trades)} < {self.min_trades}"
|
|
)
|
|
|
|
# Component scores
|
|
prediction_accuracy = self._prediction_accuracy(predictions)
|
|
directional_accuracy = self._directional_accuracy(results.trades)
|
|
risk_adjusted = self._risk_adjusted_score(results.metrics)
|
|
consistency = self._consistency_score(results)
|
|
|
|
# Weighted final score
|
|
weights = {
|
|
'prediction': 0.25,
|
|
'direction': 0.25,
|
|
'risk_adjusted': 0.30,
|
|
'consistency': 0.20
|
|
}
|
|
|
|
final_score = (
|
|
prediction_accuracy * weights['prediction'] +
|
|
directional_accuracy * weights['direction'] +
|
|
risk_adjusted * weights['risk_adjusted'] +
|
|
consistency * weights['consistency']
|
|
)
|
|
|
|
return EffectivenessScore(
|
|
score=final_score,
|
|
is_valid=final_score >= self.EFFECTIVENESS_TARGET,
|
|
components={
|
|
'prediction_accuracy': prediction_accuracy,
|
|
'directional_accuracy': directional_accuracy,
|
|
'risk_adjusted': risk_adjusted,
|
|
'consistency': consistency
|
|
},
|
|
message=self._generate_message(final_score)
|
|
)
|
|
|
|
def _prediction_accuracy(self, predictions: List[Prediction]) -> float:
|
|
"""Accuracy de predicciones ML"""
|
|
correct = sum(1 for p in predictions if p.was_correct)
|
|
return correct / len(predictions) if predictions else 0
|
|
|
|
def _directional_accuracy(self, trades: List[Position]) -> float:
|
|
"""Accuracy direccional de trades"""
|
|
correct = sum(1 for t in trades if t.pnl > 0)
|
|
return correct / len(trades) if trades else 0
|
|
|
|
def _risk_adjusted_score(self, metrics: BacktestMetrics) -> float:
|
|
"""Score basado en metricas de riesgo"""
|
|
sharpe_score = min(1.0, max(0, metrics.sharpe_ratio) / 2.0)
|
|
sortino_score = min(1.0, max(0, metrics.sortino_ratio) / 2.5)
|
|
dd_score = 1.0 - min(1.0, metrics.max_drawdown * 2)
|
|
|
|
return (sharpe_score + sortino_score + dd_score) / 3
|
|
|
|
def _consistency_score(self, results: BacktestResults) -> float:
|
|
"""Score de consistencia"""
|
|
# Variabilidad de returns mensuales
|
|
monthly_returns = self._calculate_monthly_returns(results.equity_curve)
|
|
if len(monthly_returns) < 3:
|
|
return 0.5
|
|
|
|
positive_months = sum(1 for r in monthly_returns if r > 0)
|
|
consistency = positive_months / len(monthly_returns)
|
|
|
|
return consistency
|
|
```
|
|
|
|
### Criterios de Efectividad
|
|
|
|
| Componente | Peso | Target | Descripcion |
|
|
|------------|------|--------|-------------|
|
|
| **Prediction Accuracy** | 25% | >= 55% | ML predictions correctas |
|
|
| **Directional Accuracy** | 25% | >= 55% | Trades en direccion correcta |
|
|
| **Risk-Adjusted** | 30% | Sharpe >= 1.0 | Retorno ajustado por riesgo |
|
|
| **Consistency** | 20% | >= 60% meses positivos | Estabilidad temporal |
|
|
|
|
---
|
|
|
|
## API y Uso
|
|
|
|
### Uso Basico
|
|
|
|
```python
|
|
from backtesting import BacktestEngine, BacktestConfig, WalkForwardValidator
|
|
|
|
# Configuracion
|
|
config = BacktestConfig(
|
|
symbol='XAUUSD',
|
|
start_date='2024-01-01',
|
|
end_date='2024-12-31',
|
|
initial_capital=10000,
|
|
position_sizing='kelly',
|
|
kelly_fraction=0.5
|
|
)
|
|
|
|
# Cargar datos
|
|
data = load_historical_data(config.symbol, config.start_date, config.end_date)
|
|
|
|
# Crear estrategia
|
|
strategy = MyMLStrategy()
|
|
strategy.load_model('models/pva/XAUUSD/v1.0.0')
|
|
|
|
# Ejecutar backtest
|
|
engine = BacktestEngine(config)
|
|
results = engine.run(strategy, data)
|
|
|
|
# Mostrar resultados
|
|
print(f"Total Return: {results.metrics.total_return:.2%}")
|
|
print(f"Sharpe Ratio: {results.metrics.sharpe_ratio:.2f}")
|
|
print(f"Max Drawdown: {results.metrics.max_drawdown:.2%}")
|
|
print(f"Win Rate: {results.metrics.win_rate:.2%}")
|
|
print(f"Profit Factor: {results.metrics.profit_factor:.2f}")
|
|
```
|
|
|
|
### Walk-Forward Validation
|
|
|
|
```python
|
|
# Configurar walk-forward
|
|
wf_validator = WalkForwardValidator(
|
|
n_folds=5,
|
|
train_ratio=0.8,
|
|
gap_periods=12 # 1 hour gap
|
|
)
|
|
|
|
# Ejecutar validacion
|
|
wf_results = wf_validator.validate(
|
|
strategy_class=PVAStrategy,
|
|
data=data,
|
|
backtest_config=config
|
|
)
|
|
|
|
print(f"Average Sharpe: {wf_results.avg_sharpe:.2f} +/- {wf_results.std_sharpe:.2f}")
|
|
print(f"Stability Score: {wf_results.stability_score:.2f}")
|
|
|
|
# Fold breakdown
|
|
for fold in wf_results.fold_results:
|
|
print(f"Fold {fold['fold']}: Sharpe={fold['sharpe']:.2f}, Return={fold['return']:.2%}")
|
|
```
|
|
|
|
### Effectiveness Validation
|
|
|
|
```python
|
|
# Validar efectividad
|
|
eff_validator = EffectivenessValidator(min_trades=30)
|
|
eff_score = eff_validator.validate(results, predictions)
|
|
|
|
print(f"Effectiveness Score: {eff_score.score:.2%}")
|
|
print(f"Target Achieved: {eff_score.is_valid}")
|
|
|
|
for component, value in eff_score.components.items():
|
|
print(f" {component}: {value:.2%}")
|
|
```
|
|
|
|
---
|
|
|
|
## Estructura de Archivos
|
|
|
|
```
|
|
apps/ml-engine/src/backtesting/
|
|
├── __init__.py
|
|
├── engine.py # BacktestEngine
|
|
├── position_manager.py # PositionManager, Position
|
|
├── metrics.py # MetricsCalculator, BacktestMetrics
|
|
├── walk_forward.py # WalkForwardValidator
|
|
├── effectiveness.py # EffectivenessValidator
|
|
├── kelly.py # Kelly Criterion implementation
|
|
├── data_handler.py # DataHandler for historical data
|
|
└── visualization.py # Plotting utilities
|
|
```
|
|
|
|
---
|
|
|
|
## Visualizaciones
|
|
|
|
### Equity Curve
|
|
|
|
```python
|
|
def plot_equity_curve(results: BacktestResults):
|
|
"""Graficar curva de equity"""
|
|
fig, axes = plt.subplots(3, 1, figsize=(12, 10))
|
|
|
|
# Equity
|
|
axes[0].plot(results.equity_curve)
|
|
axes[0].set_title('Equity Curve')
|
|
axes[0].set_ylabel('Account Value ($)')
|
|
|
|
# Drawdown
|
|
drawdown = calculate_drawdown_series(results.equity_curve)
|
|
axes[1].fill_between(range(len(drawdown)), drawdown, alpha=0.3, color='red')
|
|
axes[1].set_title('Drawdown')
|
|
axes[1].set_ylabel('Drawdown %')
|
|
|
|
# Monthly returns
|
|
monthly_returns = calculate_monthly_returns(results.equity_curve)
|
|
colors = ['green' if r > 0 else 'red' for r in monthly_returns]
|
|
axes[2].bar(range(len(monthly_returns)), monthly_returns, color=colors)
|
|
axes[2].set_title('Monthly Returns')
|
|
axes[2].set_ylabel('Return %')
|
|
|
|
plt.tight_layout()
|
|
return fig
|
|
```
|
|
|
|
### Trade Distribution
|
|
|
|
```python
|
|
def plot_trade_distribution(results: BacktestResults):
|
|
"""Distribucion de trades"""
|
|
fig, axes = plt.subplots(2, 2, figsize=(12, 10))
|
|
|
|
trades = results.trades
|
|
returns = [t.pnl_pct for t in trades]
|
|
|
|
# Return distribution
|
|
axes[0, 0].hist(returns, bins=50, edgecolor='black')
|
|
axes[0, 0].axvline(0, color='red', linestyle='--')
|
|
axes[0, 0].set_title('Trade Return Distribution')
|
|
|
|
# Win/Loss ratio
|
|
wins = sum(1 for r in returns if r > 0)
|
|
losses = len(returns) - wins
|
|
axes[0, 1].pie([wins, losses], labels=['Wins', 'Losses'], autopct='%1.1f%%')
|
|
axes[0, 1].set_title('Win Rate')
|
|
|
|
# Cumulative returns
|
|
cumulative = np.cumsum(returns)
|
|
axes[1, 0].plot(cumulative)
|
|
axes[1, 0].set_title('Cumulative Trade Returns')
|
|
|
|
# Trade duration
|
|
durations = [t.duration.total_seconds() / 3600 for t in trades if t.duration]
|
|
axes[1, 1].hist(durations, bins=30, edgecolor='black')
|
|
axes[1, 1].set_title('Trade Duration (hours)')
|
|
|
|
plt.tight_layout()
|
|
return fig
|
|
```
|
|
|
|
---
|
|
|
|
## Consideraciones de Produccion
|
|
|
|
### Performance Optimization
|
|
|
|
```python
|
|
# Usar numba para calculos intensivos
|
|
from numba import jit
|
|
|
|
@jit(nopython=True)
|
|
def fast_max_drawdown(equity: np.ndarray) -> float:
|
|
"""Calculo optimizado de max drawdown"""
|
|
peak = equity[0]
|
|
max_dd = 0.0
|
|
|
|
for i in range(len(equity)):
|
|
if equity[i] > peak:
|
|
peak = equity[i]
|
|
dd = (peak - equity[i]) / peak
|
|
if dd > max_dd:
|
|
max_dd = dd
|
|
|
|
return max_dd
|
|
```
|
|
|
|
### Parallel Backtesting
|
|
|
|
```python
|
|
from concurrent.futures import ProcessPoolExecutor
|
|
|
|
def parallel_backtest(
|
|
strategies: List[MLStrategy],
|
|
data: pd.DataFrame,
|
|
config: BacktestConfig,
|
|
n_workers: int = 4
|
|
) -> List[BacktestResults]:
|
|
"""Ejecutar backtests en paralelo"""
|
|
|
|
def run_single(strategy):
|
|
engine = BacktestEngine(config)
|
|
return engine.run(strategy, data)
|
|
|
|
with ProcessPoolExecutor(max_workers=n_workers) as executor:
|
|
results = list(executor.map(run_single, strategies))
|
|
|
|
return results
|
|
```
|
|
|
|
### Result Persistence
|
|
|
|
```python
|
|
def save_results(results: BacktestResults, path: str):
|
|
"""Guardar resultados de backtest"""
|
|
output = {
|
|
'metrics': results.metrics.to_dict(),
|
|
'trades': [t.to_dict() for t in results.trades],
|
|
'equity_curve': results.equity_curve,
|
|
'config': results.config.to_dict(),
|
|
'timestamp': datetime.now().isoformat()
|
|
}
|
|
|
|
with open(path, 'w') as f:
|
|
json.dump(output, f, indent=2)
|
|
```
|
|
|
|
---
|
|
|
|
## Referencias
|
|
|
|
- [ET-ML-001: Arquitectura ML Engine](./ET-ML-001-arquitectura.md)
|
|
- [ET-ML-010 to ET-ML-014: Strategy Specifications](./ET-ML-010-pva-strategy.md)
|
|
- [Kelly Criterion (Wikipedia)](https://en.wikipedia.org/wiki/Kelly_criterion)
|
|
- [Walk-Forward Analysis (Pardo)](https://www.amazon.com/Evaluation-Optimization-Trading-Strategies-Pardo/dp/0470128011)
|
|
|
|
---
|
|
|
|
**Autor:** ML-Specialist (NEXUS v4.0)
|
|
**Fecha:** 2026-01-25
|