rckrdmrd
62a9f3e1d9
Data aggregation and distribution service: - Market data collection - OHLCV aggregation - Real-time data feeds - Data API endpoints Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
152 lines
3.3 KiB
Markdown
152 lines
3.3 KiB
Markdown
# Data Service
|
|
|
|
Market data service for the Trading Platform.
|
|
|
|
## Features
|
|
|
|
- **REST API**: FastAPI-based endpoints for market data
|
|
- **WebSocket Streaming**: Real-time price updates
|
|
- **Multi-Provider Support**: Polygon.io, Binance, MT4/MT5
|
|
- **Historical Data**: OHLCV candles with multiple timeframes
|
|
- **Spread Tracking**: Broker spread monitoring and statistics
|
|
- **Price Adjustment**: ML-based price adjustment models
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# Install dependencies
|
|
pip install -r requirements.txt
|
|
|
|
# Set environment variables
|
|
cp .env.example .env
|
|
|
|
# Run development server
|
|
python -m uvicorn src.app:app --reload --port 8001
|
|
|
|
# Or with Docker
|
|
docker-compose up -d
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Health
|
|
- `GET /health` - Service health status
|
|
- `GET /ready` - Kubernetes readiness probe
|
|
- `GET /live` - Kubernetes liveness probe
|
|
|
|
### Symbols
|
|
- `GET /api/v1/symbols` - List trading symbols
|
|
- `GET /api/v1/symbols/{symbol}` - Get symbol info
|
|
|
|
### Market Data
|
|
- `GET /api/v1/ticker/{symbol}` - Current price
|
|
- `GET /api/v1/tickers` - Multiple tickers
|
|
- `GET /api/v1/candles/{symbol}` - Historical OHLCV
|
|
- `GET /api/v1/orderbook/{symbol}` - Order book snapshot
|
|
- `GET /api/v1/trades/{symbol}` - Recent trades
|
|
|
|
### Admin
|
|
- `POST /api/v1/admin/backfill/{symbol}` - Trigger data backfill
|
|
- `POST /api/v1/admin/sync` - Trigger sync
|
|
|
|
## WebSocket
|
|
|
|
Connect to `/ws/stream` for real-time data.
|
|
|
|
```javascript
|
|
const ws = new WebSocket('ws://localhost:8001/ws/stream');
|
|
|
|
ws.onopen = () => {
|
|
// Subscribe to ticker updates
|
|
ws.send(JSON.stringify({
|
|
action: 'subscribe',
|
|
channel: 'ticker',
|
|
symbols: ['EURUSD', 'BTCUSD']
|
|
}));
|
|
};
|
|
|
|
ws.onmessage = (event) => {
|
|
const data = JSON.parse(event.data);
|
|
console.log(data);
|
|
};
|
|
```
|
|
|
|
### Channels
|
|
- `ticker` - Real-time price updates
|
|
- `candles` - OHLCV candle updates (specify timeframe)
|
|
- `orderbook` - Order book snapshots
|
|
- `trades` - Recent trades
|
|
- `signals` - ML trading signals
|
|
|
|
## Architecture
|
|
|
|
```
|
|
src/
|
|
├── app.py # FastAPI application
|
|
├── main.py # Scheduler-based service
|
|
├── config.py # Configuration
|
|
├── api/
|
|
│ ├── routes.py # REST endpoints
|
|
│ └── dependencies.py # FastAPI dependencies
|
|
├── websocket/
|
|
│ ├── manager.py # Connection management
|
|
│ └── handlers.py # WebSocket routes
|
|
├── models/
|
|
│ └── market.py # Pydantic models
|
|
├── providers/
|
|
│ ├── polygon_client.py # Polygon.io client
|
|
│ ├── binance_client.py # Binance client
|
|
│ └── mt4_client.py # MT4/MetaAPI client
|
|
└── services/
|
|
└── price_adjustment.py # Price adjustment service
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
```env
|
|
# Database
|
|
DB_HOST=localhost
|
|
DB_PORT=5432
|
|
DB_NAME=trading_data
|
|
DB_USER=trading_user
|
|
DB_PASSWORD=trading_dev_2025
|
|
|
|
# Polygon.io
|
|
POLYGON_API_KEY=your_api_key
|
|
POLYGON_TIER=basic
|
|
|
|
# Binance
|
|
BINANCE_API_KEY=your_api_key
|
|
BINANCE_API_SECRET=your_secret
|
|
BINANCE_TESTNET=false
|
|
|
|
# MetaAPI (MT4/MT5)
|
|
METAAPI_TOKEN=your_token
|
|
METAAPI_ACCOUNT_ID=your_account_id
|
|
|
|
# Service
|
|
SYNC_INTERVAL_MINUTES=5
|
|
BACKFILL_DAYS=30
|
|
LOG_LEVEL=INFO
|
|
```
|
|
|
|
## Development
|
|
|
|
```bash
|
|
# Run tests
|
|
pytest
|
|
|
|
# Code formatting
|
|
black src/
|
|
isort src/
|
|
|
|
# Type checking
|
|
mypy src/
|
|
```
|
|
|
|
## API Documentation
|
|
|
|
When running, visit:
|
|
- Swagger UI: http://localhost:8001/docs
|
|
- ReDoc: http://localhost:8001/redoc
|