Commit Graph

8 Commits

Author SHA1 Message Date
Adrian Flores Cortes
d015e2b0f3 feat(ml-engine): Phase 4 - PostgreSQL migration, dynamic OOS, data pipeline
- Fix database.py: Add DatabaseConnection alias for backward compat
- Fix train_symbol_timeframe_models.py: Use PostgreSQLConnection + native queries
- Fix run_oos_backtest.py: Fix broken import + add dynamic OOS support
- Update data_splitter.py: split_dynamic_oos() method (from previous session)
- Update validation_oos.yaml: Dynamic OOS config + all 6 symbols enabled
- Create ingest_ohlcv_polygon.py: Standalone Polygon→PostgreSQL ingestion script
- Fix .gitignore: /data/ instead of data/ to not ignore src/data/
- Add untracked src/ modules: backtesting, data, llm, models (attention/metamodel/strategies)
- Add aiohttp, sqlalchemy, psycopg2-binary to requirements.txt

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-27 04:39:05 -06:00
Adrian Flores Cortes
775252c872 [GAP-5] feat: Implement historical signals query system
- Created SignalStore service (src/services/signal_store.py)
  - In-memory storage with FIFO eviction (max 10,000 signals)
  - 7-day retention policy
  - Query by symbol, direction, date range, confidence, AMD phase
  - Pagination support
  - Optional JSON persistence
  - Fast lookup via indexed structure
- Integrated signal storage into PredictionService
  - Signals auto-stored on generation
  - Query methods exposed to API
- Added API endpoints for historical signals:
  - GET /api/signals/history - Query with filters and pagination
  - GET /api/signals/latest - Get most recent signals
  - GET /api/signals/valid - Get currently valid signals
  - GET /api/signals/{signal_id} - Get specific signal
  - GET /api/signals/stats - Get store statistics

StoredSignal dataclass includes:
- Core signal data (direction, prices, confidence)
- Attention info (score, flow class, high flow flag)
- Metadata (timeframe, AMD phase, volatility)
- Validity window (timestamp, valid_until)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 07:22:50 -06:00
Adrian Flores Cortes
a80aeea3c7 [GAP-4] feat: Implement real-time WebSocket signal streaming
- Created SignalStreamManager class for connection management
  - Per-symbol subscriptions support
  - Connection state tracking
  - Targeted and broadcast messaging
- Added background signal generation task
  - Generates signals every 60 seconds for subscribed symbols
  - Includes attention scores in signals
- Enhanced /ws/signals endpoint with full protocol:
  - subscribe: Subscribe to symbols
  - unsubscribe: Unsubscribe from symbols
  - get_signal: Request immediate signal
  - get_attention: Request attention scores
  - get_hierarchical: Request full L0→L1→L2 prediction
  - ping/pong: Connection keepalive
- Added GET /ws/status endpoint for monitoring
- Added startup/shutdown handlers for background task

WebSocket Protocol:
1. Connect to /ws/signals
2. Send: {"action": "subscribe", "symbols": ["XAUUSD"]}
3. Receive: {"type": "signal", "symbol": "XAUUSD", "data": {...}}

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 07:17:45 -06:00
Adrian Flores Cortes
a1e606c21a [GAP-3] feat: Wire RRBacktester into prediction service and API
- Added RRBacktester import and initialization in PredictionService
- Added BacktestSummary dataclass for typed responses
- Added run_backtest() method with signal generation
- Added _generate_backtest_signals() helper for historical signals
- Added backtester_available property
- Updated POST /api/backtest endpoint to use real backtester
- Added GET /api/backtest/status endpoint for availability check
- Added losing_trades, avg_trade_duration, metrics_by_rr, metrics_by_amd to response

The backtester now uses:
- RRBacktester from src/backtesting/rr_backtester.py
- MetricsCalculator from src/backtesting/metrics.py
- AMD phase filtering and R:R configuration support

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 07:13:26 -06:00
Adrian Flores Cortes
a22fb11968 [GAP-2] feat: Wire HierarchicalPipeline into prediction service
- Added HierarchicalPipeline initialization in PredictionService._load_models()
- Added HierarchicalResult dataclass for typed responses
- Added get_hierarchical_prediction() method for full L0→L1→L2 predictions
- Added get_hierarchical_model_info() method for model introspection
- Added hierarchical_available property
- Added API endpoints:
  - GET /api/hierarchical/{symbol} - Full 3-level prediction
  - GET /api/hierarchical/{symbol}/models - Model info per symbol
  - GET /api/hierarchical/status - Pipeline status
- Updated health endpoint to include hierarchical_available flag

This completes GAP #2: L1→L2 Hierarchical Predictor integration.
The pipeline now uses symbol-specific metamodels (XGBoost or Neural Gating)
to synthesize 5m and 15m base model predictions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 07:05:59 -06:00
Adrian Flores Cortes
b0b4a712eb feat: Integrate Level 0 attention models into prediction pipeline
- Add AttentionProvider service to load and serve attention scores
- Integrate attention scoring into PredictionService.generate_signal()
- Add AttentionInfo dataclass to TradingSignal
- Boost confidence by 15% when high flow detected
- Add /api/attention/{symbol} endpoint for direct attention queries
- Add /api/attention/models endpoint to list loaded models
- Add attention field to SignalResponse API model

This completes the L0→L1 integration of the hierarchical ML architecture.
Attention models identify high-flow market moments to improve signal quality.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 06:57:59 -06:00
Adrian Flores Cortes
475e913e3c config: Deprecate MySQL and update PostgreSQL credentials
- PostgreSQL credentials updated to trading_user/trading_dev_2026
- MySQL section marked as deprecated (remote server no longer accessible)
- Table mappings updated to use market_data PostgreSQL schema
- Password redacted from deprecated MySQL config

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-25 05:52:50 -06:00
75c4d07690 feat: Initial commit - ML Engine codebase
Hierarchical ML Pipeline for trading predictions:
- Level 0: Attention Models (volatility/flow classification)
- Level 1: Base Models (XGBoost per symbol/timeframe)
- Level 2: Metamodels (XGBoost Stacking + Neural Gating)

Key components:
- src/pipelines/hierarchical_pipeline.py - Main prediction pipeline
- src/models/ - All ML model classes
- src/training/ - Training utilities
- src/api/ - FastAPI endpoints
- scripts/ - Training and evaluation scripts
- config/ - YAML configurations

Note: Trained models (*.joblib, *.pt) are gitignored.
      Regenerate with training scripts.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 04:27:40 -06:00