ML Engine Updates: - Updated BTCUSD with Polygon API data (2024-2025): 215,699 new records - Re-trained all ML models: Attention (R²: 0.223), Base, Metamodel (87.3% confidence) - Backtest results: +176.71R profit with aggressive_filter strategy Documentation Consolidation: - Created docs/99-analisis/_MAP.md index with 13 new analysis documents - Consolidated inventories: removed duplicates from orchestration/inventarios/ - Updated ML_INVENTORY.yml with BTCUSD metrics and training results - Added execution reports: FASE11-BTCUSD, correction issues, alignment validation Architecture & Integration: - Updated all module documentation with NEXUS v3.4 frontmatter - Fixed _MAP.md indexes across all folders - Updated orchestration plans and traces Files: 229 changed, 5064 insertions(+), 1872 deletions(-) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
28 KiB
28 KiB
| id | title | type | status | priority | epic | project | version | created_date | updated_date |
|---|---|---|---|---|---|---|---|---|---|
| ET-LLM-006 | Gestión de Contexto y Memoria | Technical Specification | Done | Alta | OQI-007 | trading-platform | 1.0.0 | 2025-12-05 | 2026-01-04 |
ET-LLM-006: Gestión de Contexto y Memoria
Épica: OQI-007 - LLM Strategy Agent Versión: 1.0 Fecha: 2025-12-05 Estado: Planificado Prioridad: P0 - Crítico
Resumen
Esta especificación define el sistema de gestión de contexto y memoria del agente LLM, incluyendo el manejo de conversaciones, memoria a largo plazo, enriquecimiento automático de contexto y compresión de tokens.
Arquitectura del Sistema
┌─────────────────────────────────────────────────────────────────────────┐
│ CONTEXT & MEMORY SYSTEM │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ Context Builder │ │
│ │ │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ User │ │ Market │ │ Conversation│ │ │
│ │ │ Context │ │ Context │ │ History │ │ │
│ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │
│ │ └─────────────────┼───────────────┘ │ │
│ │ ↓ │ │
│ │ ┌─────────────────────────────────────────────────────────────┐ │ │
│ │ │ Context Assembler │ │ │
│ │ └─────────────────────────────────────────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ↓ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ Token Manager │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │ │
│ │ │ Token │ │ Context │ │ Summarizer │ │ │
│ │ │ Counter │ │ Compressor │ │ │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │ │
│ ↓ │
│ ┌───────────────────────────────────────────────────────────────────┐ │
│ │ Memory Store │ │
│ │ │ │
│ │ ┌──────────────┐ ┌──────────────┐ ┌──────────────────────────┐ │ │
│ │ │ Short-term │ │ Long-term │ │ Preferences │ │ │
│ │ │ (Redis) │ │ (Postgres) │ │ (Postgres) │ │ │
│ │ └──────────────┘ └──────────────┘ └──────────────────────────┘ │ │
│ └───────────────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
Context Builder
Estructura del Contexto
// src/modules/copilot/context/context.types.ts
interface FullContext {
// User information
user: UserContext;
// Market information
market: MarketContext;
// Conversation history
conversation: ConversationContext;
// Long-term memory
memory: MemoryContext;
// System information
system: SystemContext;
}
interface UserContext {
id: string;
name: string;
plan: 'free' | 'pro' | 'premium';
riskProfile: 'conservative' | 'moderate' | 'aggressive';
experience: 'beginner' | 'intermediate' | 'advanced';
language: string;
timezone: string;
preferences: UserPreferences;
}
interface MarketContext {
status: 'open' | 'closed' | 'pre_market' | 'after_hours';
currentTime: string;
nextOpen?: string;
nextClose?: string;
upcomingEvents: MarketEvent[];
relevantSymbols: SymbolSnapshot[];
}
interface ConversationContext {
id: string;
startedAt: string;
messageCount: number;
recentMessages: Message[];
summary?: string;
recentTopics: string[];
mentionedSymbols: string[];
}
interface MemoryContext {
favoriteSymbols: string[];
tradingStyle: string;
frequentQuestions: string[];
recentStrategies: StrategyRecord[];
userNotes: string[];
}
interface SystemContext {
currentDate: string;
currentTime: string;
agentVersion: string;
availableTools: string[];
}
Context Builder Service
// src/modules/copilot/context/context-builder.ts
@Injectable()
export class ContextBuilder {
constructor(
private readonly userService: UserService,
private readonly marketService: MarketService,
private readonly conversationService: ConversationService,
private readonly memoryService: MemoryService,
private readonly portfolioService: PortfolioService,
private readonly alertService: AlertService,
) {}
async build(params: BuildContextParams): Promise<FullContext> {
const { userId, conversationId, currentMessage } = params;
// Fetch all context data in parallel
const [
user,
marketStatus,
conversation,
memory,
portfolio,
alerts,
] = await Promise.all([
this.userService.getProfile(userId),
this.marketService.getMarketStatus(),
this.conversationService.getContext(conversationId),
this.memoryService.getUserMemory(userId),
this.portfolioService.getPositions(userId),
this.alertService.getActive(userId),
]);
// Extract mentioned symbols from current message
const mentionedSymbols = this.extractSymbols(currentMessage);
// Get relevant symbol snapshots
const relevantSymbols = await this.getRelevantSymbols(
mentionedSymbols,
memory.favoriteSymbols,
portfolio.map(p => p.symbol),
);
// Build upcoming events
const upcomingEvents = await this.marketService.getUpcomingEvents(
relevantSymbols.map(s => s.symbol),
);
return {
user: {
id: user.id,
name: user.firstName,
plan: user.plan,
riskProfile: user.riskProfile,
experience: user.experience,
language: user.language || 'es',
timezone: user.timezone || 'America/Mexico_City',
preferences: user.preferences,
},
market: {
status: marketStatus.status,
currentTime: new Date().toISOString(),
nextOpen: marketStatus.nextOpen,
nextClose: marketStatus.nextClose,
upcomingEvents,
relevantSymbols,
},
conversation: {
id: conversationId,
startedAt: conversation.startedAt,
messageCount: conversation.messageCount,
recentMessages: conversation.recentMessages,
summary: conversation.summary,
recentTopics: conversation.topics,
mentionedSymbols,
},
memory: {
favoriteSymbols: memory.favoriteSymbols,
tradingStyle: memory.tradingStyle,
frequentQuestions: memory.frequentQuestions,
recentStrategies: memory.recentStrategies,
userNotes: memory.userNotes,
},
system: {
currentDate: new Date().toISOString().split('T')[0],
currentTime: new Date().toISOString(),
agentVersion: '1.0.0',
availableTools: this.getAvailableTools(user.plan),
},
};
}
private extractSymbols(message: string): string[] {
// Pattern for stock symbols (uppercase, 1-5 chars)
const stockPattern = /\b[A-Z]{1,5}\b/g;
// Pattern for crypto pairs
const cryptoPattern = /\b(BTC|ETH|SOL|ADA|XRP)\/?(USD|USDT)?\b/gi;
const matches = [
...(message.match(stockPattern) || []),
...(message.match(cryptoPattern) || []),
];
// Filter out common words that look like symbols
const commonWords = new Set(['I', 'A', 'THE', 'AND', 'OR', 'FOR', 'IS', 'IT', 'MY']);
return [...new Set(matches)]
.filter(s => !commonWords.has(s.toUpperCase()))
.map(s => s.toUpperCase());
}
private async getRelevantSymbols(
mentioned: string[],
favorites: string[],
portfolio: string[],
): Promise<SymbolSnapshot[]> {
// Combine and dedupe symbols
const allSymbols = [...new Set([...mentioned, ...favorites.slice(0, 3), ...portfolio])];
// Limit to 10 symbols max
const limitedSymbols = allSymbols.slice(0, 10);
// Get snapshots
return Promise.all(
limitedSymbols.map(symbol => this.marketService.getSnapshot(symbol))
);
}
}
Token Manager
Token Counter
// src/modules/copilot/context/token-counter.ts
import { encoding_for_model, TiktokenModel } from 'tiktoken';
@Injectable()
export class TokenCounter {
private encoder: any;
constructor() {
// Use cl100k_base for GPT-4 and Claude
this.encoder = encoding_for_model('gpt-4' as TiktokenModel);
}
count(text: string): number {
return this.encoder.encode(text).length;
}
countMessages(messages: Message[]): number {
let total = 0;
for (const msg of messages) {
// Each message has overhead of ~4 tokens
total += 4;
total += this.count(msg.content);
if (msg.role) {
total += this.count(msg.role);
}
}
return total;
}
estimateContextTokens(context: FullContext): TokenEstimate {
const systemPromptTokens = this.count(this.formatSystemPrompt(context));
const messagesTokens = this.countMessages(context.conversation.recentMessages);
const toolsTokens = context.system.availableTools.length * 150; // ~150 tokens per tool
return {
systemPrompt: systemPromptTokens,
messages: messagesTokens,
tools: toolsTokens,
total: systemPromptTokens + messagesTokens + toolsTokens,
remaining: 8000 - (systemPromptTokens + messagesTokens + toolsTokens),
};
}
private formatSystemPrompt(context: FullContext): string {
// Template for system prompt (simplified)
return `User: ${context.user.name}, Plan: ${context.user.plan}...`;
}
}
Context Compressor
// src/modules/copilot/context/context-compressor.ts
interface CompressionResult {
context: FullContext;
compressed: boolean;
originalTokens: number;
compressedTokens: number;
strategies: string[];
}
@Injectable()
export class ContextCompressor {
private readonly maxTokens = 6000; // Leave room for response
private readonly targetTokens = 4000;
constructor(
private readonly tokenCounter: TokenCounter,
private readonly summarizer: ConversationSummarizer,
) {}
async compress(context: FullContext): Promise<CompressionResult> {
const estimate = this.tokenCounter.estimateContextTokens(context);
const strategies: string[] = [];
if (estimate.total <= this.maxTokens) {
return {
context,
compressed: false,
originalTokens: estimate.total,
compressedTokens: estimate.total,
strategies: [],
};
}
let compressedContext = { ...context };
// Strategy 1: Reduce market context
if (estimate.total > this.maxTokens) {
compressedContext = this.reduceMarketContext(compressedContext);
strategies.push('reduce_market_context');
}
// Strategy 2: Trim conversation history
const newEstimate = this.tokenCounter.estimateContextTokens(compressedContext);
if (newEstimate.total > this.maxTokens) {
compressedContext = await this.trimConversation(compressedContext);
strategies.push('trim_conversation');
}
// Strategy 3: Summarize old messages
const finalEstimate = this.tokenCounter.estimateContextTokens(compressedContext);
if (finalEstimate.total > this.maxTokens) {
compressedContext = await this.summarizeConversation(compressedContext);
strategies.push('summarize_conversation');
}
return {
context: compressedContext,
compressed: true,
originalTokens: estimate.total,
compressedTokens: this.tokenCounter.estimateContextTokens(compressedContext).total,
strategies,
};
}
private reduceMarketContext(context: FullContext): FullContext {
return {
...context,
market: {
...context.market,
// Keep only essential symbols (mentioned + portfolio)
relevantSymbols: context.market.relevantSymbols.slice(0, 5),
// Keep only high-impact events
upcomingEvents: context.market.upcomingEvents.filter(e => e.impact === 'high'),
},
};
}
private async trimConversation(context: FullContext): Promise<FullContext> {
const { recentMessages } = context.conversation;
// Keep last 10 messages
const trimmedMessages = recentMessages.slice(-10);
return {
...context,
conversation: {
...context.conversation,
recentMessages: trimmedMessages,
},
};
}
private async summarizeConversation(context: FullContext): Promise<FullContext> {
const { recentMessages } = context.conversation;
if (recentMessages.length <= 5) {
return context;
}
// Summarize older messages
const oldMessages = recentMessages.slice(0, -5);
const recentMessages5 = recentMessages.slice(-5);
const summary = await this.summarizer.summarize(oldMessages);
return {
...context,
conversation: {
...context.conversation,
summary: summary,
recentMessages: recentMessages5,
},
};
}
}
Conversation Summarizer
// src/modules/copilot/context/summarizer.ts
@Injectable()
export class ConversationSummarizer {
constructor(private readonly llmProvider: LLMProviderService) {}
async summarize(messages: Message[]): Promise<string> {
if (messages.length === 0) return '';
const formattedMessages = messages.map(m =>
`${m.role}: ${m.content}`
).join('\n');
const response = await this.llmProvider.createChatCompletion({
model: 'gpt-4o-mini', // Use smaller model for summarization
messages: [
{
role: 'system',
content: `Eres un asistente que resume conversaciones de trading.
Crea un resumen conciso (máximo 100 palabras) que capture:
- Símbolos mencionados
- Análisis o estrategias discutidas
- Decisiones o conclusiones importantes
- Preguntas pendientes del usuario
Formato: Lista de puntos clave.`,
},
{
role: 'user',
content: `Resume esta conversación:\n\n${formattedMessages}`,
},
],
max_tokens: 200,
});
return response.choices[0].message.content;
}
}
Memory Store
Short-term Memory (Redis)
// src/modules/copilot/memory/short-term-memory.ts
interface ShortTermData {
activeConversation: string;
recentSymbols: string[];
lastActivity: string;
pendingActions: PendingAction[];
}
@Injectable()
export class ShortTermMemory {
private readonly ttl = 3600; // 1 hour
constructor(private readonly redis: RedisService) {}
async get(userId: string): Promise<ShortTermData | null> {
const key = `stm:${userId}`;
const data = await this.redis.get(key);
return data ? JSON.parse(data) : null;
}
async set(userId: string, data: Partial<ShortTermData>): Promise<void> {
const key = `stm:${userId}`;
const existing = await this.get(userId);
const merged = { ...existing, ...data };
await this.redis.setex(key, this.ttl, JSON.stringify(merged));
}
async addRecentSymbol(userId: string, symbol: string): Promise<void> {
const data = await this.get(userId);
const recentSymbols = data?.recentSymbols || [];
// Add to front, dedupe, limit to 10
const updated = [symbol, ...recentSymbols.filter(s => s !== symbol)].slice(0, 10);
await this.set(userId, { recentSymbols: updated });
}
async addPendingAction(userId: string, action: PendingAction): Promise<void> {
const data = await this.get(userId);
const pendingActions = data?.pendingActions || [];
await this.set(userId, {
pendingActions: [...pendingActions, action],
});
}
async clearPendingActions(userId: string): Promise<void> {
await this.set(userId, { pendingActions: [] });
}
}
Long-term Memory (PostgreSQL)
// src/modules/copilot/memory/long-term-memory.ts
@Entity('user_memory')
export class UserMemory {
@PrimaryColumn()
userId: string;
@Column({ type: 'jsonb', default: [] })
favoriteSymbols: string[];
@Column({ nullable: true })
tradingStyle: string;
@Column({ type: 'jsonb', default: [] })
frequentQuestions: string[];
@Column({ type: 'jsonb', default: [] })
recentStrategies: StrategyRecord[];
@Column({ type: 'jsonb', default: [] })
userNotes: string[];
@Column({ type: 'jsonb', default: {} })
preferences: Record<string, any>;
@UpdateDateColumn()
updatedAt: Date;
}
@Injectable()
export class LongTermMemory {
constructor(
@InjectRepository(UserMemory)
private readonly memoryRepo: Repository<UserMemory>,
) {}
async get(userId: string): Promise<UserMemory> {
let memory = await this.memoryRepo.findOne({ where: { userId } });
if (!memory) {
memory = this.memoryRepo.create({
userId,
favoriteSymbols: [],
frequentQuestions: [],
recentStrategies: [],
userNotes: [],
preferences: {},
});
await this.memoryRepo.save(memory);
}
return memory;
}
async update(userId: string, updates: Partial<UserMemory>): Promise<void> {
await this.memoryRepo.update({ userId }, updates);
}
async addFavoriteSymbol(userId: string, symbol: string): Promise<void> {
const memory = await this.get(userId);
if (!memory.favoriteSymbols.includes(symbol)) {
memory.favoriteSymbols = [symbol, ...memory.favoriteSymbols].slice(0, 20);
await this.memoryRepo.save(memory);
}
}
async addStrategy(userId: string, strategy: StrategyRecord): Promise<void> {
const memory = await this.get(userId);
memory.recentStrategies = [strategy, ...memory.recentStrategies].slice(0, 10);
await this.memoryRepo.save(memory);
}
async learnFromConversation(
userId: string,
conversation: Message[],
): Promise<void> {
// Extract patterns from conversation
const patterns = this.extractPatterns(conversation);
const memory = await this.get(userId);
// Update frequent questions
if (patterns.questions.length > 0) {
const updated = this.mergeFrequentQuestions(
memory.frequentQuestions,
patterns.questions,
);
memory.frequentQuestions = updated;
}
// Update trading style
if (patterns.tradingStyle) {
memory.tradingStyle = patterns.tradingStyle;
}
// Update preferences
if (patterns.preferences) {
memory.preferences = { ...memory.preferences, ...patterns.preferences };
}
await this.memoryRepo.save(memory);
}
private extractPatterns(conversation: Message[]): ExtractedPatterns {
const patterns: ExtractedPatterns = {
questions: [],
tradingStyle: null,
preferences: {},
};
for (const msg of conversation) {
if (msg.role === 'user') {
// Detect questions
if (msg.content.includes('?')) {
const question = this.normalizeQuestion(msg.content);
patterns.questions.push(question);
}
// Detect trading style mentions
if (msg.content.match(/swing|day trading|scalping|largo plazo/i)) {
patterns.tradingStyle = this.detectTradingStyle(msg.content);
}
// Detect preferences
if (msg.content.match(/prefiero|me gusta|siempre/i)) {
const prefs = this.extractPreferences(msg.content);
patterns.preferences = { ...patterns.preferences, ...prefs };
}
}
}
return patterns;
}
private normalizeQuestion(question: string): string {
// Remove specific symbols/values to get question template
return question
.replace(/\$[\d,]+/g, '[AMOUNT]')
.replace(/\b[A-Z]{1,5}\b/g, '[SYMBOL]')
.substring(0, 100);
}
private mergeFrequentQuestions(
existing: string[],
newQuestions: string[],
): string[] {
const merged = [...existing];
for (const q of newQuestions) {
// Check for similar question
const similar = merged.find(eq =>
this.questionSimilarity(eq, q) > 0.8
);
if (!similar) {
merged.push(q);
}
}
return merged.slice(0, 20);
}
private questionSimilarity(q1: string, q2: string): number {
// Simple word overlap similarity
const words1 = new Set(q1.toLowerCase().split(/\s+/));
const words2 = new Set(q2.toLowerCase().split(/\s+/));
const intersection = [...words1].filter(w => words2.has(w)).length;
const union = new Set([...words1, ...words2]).size;
return intersection / union;
}
}
System Prompt Builder
// src/modules/copilot/context/system-prompt-builder.ts
@Injectable()
export class SystemPromptBuilder {
build(context: FullContext): string {
const sections = [
this.buildHeader(),
this.buildUserSection(context.user),
this.buildMarketSection(context.market),
this.buildMemorySection(context.memory),
this.buildConversationSection(context.conversation),
this.buildInstructions(),
];
return sections.join('\n\n');
}
private buildHeader(): string {
return `# Trading Platform Trading Copilot
Eres un asistente especializado en trading e inversiones.
Tu objetivo es ayudar al usuario con análisis de mercado,
estrategias de trading, educación financiera y gestión de portfolio.`;
}
private buildUserSection(user: UserContext): string {
return `## Usuario
- **Nombre:** ${user.name}
- **Plan:** ${user.plan}
- **Perfil de riesgo:** ${user.riskProfile}
- **Experiencia:** ${user.experience}
- **Idioma:** ${user.language}
- **Zona horaria:** ${user.timezone}`;
}
private buildMarketSection(market: MarketContext): string {
let section = `## Contexto de Mercado
- **Estado:** ${this.formatMarketStatus(market.status)}
- **Hora actual:** ${market.currentTime}`;
if (market.nextOpen) {
section += `\n- **Próxima apertura:** ${market.nextOpen}`;
}
if (market.relevantSymbols.length > 0) {
section += `\n\n### Símbolos Relevantes`;
for (const symbol of market.relevantSymbols.slice(0, 5)) {
section += `\n- ${symbol.symbol}: $${symbol.price} (${symbol.changePercent > 0 ? '+' : ''}${symbol.changePercent}%)`;
}
}
if (market.upcomingEvents.length > 0) {
section += `\n\n### Eventos Próximos`;
for (const event of market.upcomingEvents.slice(0, 3)) {
section += `\n- ${event.date}: ${event.name} (${event.impact})`;
}
}
return section;
}
private buildMemorySection(memory: MemoryContext): string {
if (!memory.favoriteSymbols.length && !memory.tradingStyle) {
return '';
}
let section = `## Lo que sé del usuario`;
if (memory.favoriteSymbols.length > 0) {
section += `\n- **Símbolos favoritos:** ${memory.favoriteSymbols.slice(0, 5).join(', ')}`;
}
if (memory.tradingStyle) {
section += `\n- **Estilo de trading:** ${memory.tradingStyle}`;
}
if (memory.recentStrategies.length > 0) {
section += `\n- **Estrategia reciente:** ${memory.recentStrategies[0].name}`;
}
return section;
}
private buildConversationSection(conversation: ConversationContext): string {
if (!conversation.summary && conversation.recentTopics.length === 0) {
return '';
}
let section = `## Contexto de Conversación`;
if (conversation.summary) {
section += `\n### Resumen previo\n${conversation.summary}`;
}
if (conversation.recentTopics.length > 0) {
section += `\n- **Temas recientes:** ${conversation.recentTopics.join(', ')}`;
}
if (conversation.mentionedSymbols.length > 0) {
section += `\n- **Símbolos mencionados:** ${conversation.mentionedSymbols.join(', ')}`;
}
return section;
}
private buildInstructions(): string {
return `## Instrucciones
1. Usa las herramientas disponibles para obtener datos actualizados
2. Siempre incluye disclaimers en análisis financieros
3. Adapta tu lenguaje al nivel de experiencia del usuario
4. Si el usuario pregunta algo que requiere un plan superior, informa educadamente
5. Sé conciso pero completo en tus respuestas
6. Si no tienes datos suficientes, dilo claramente
7. Recuerda el contexto de la conversación`;
}
private formatMarketStatus(status: string): string {
const statusMap = {
open: 'Mercado Abierto',
closed: 'Mercado Cerrado',
pre_market: 'Pre-mercado',
after_hours: 'After Hours',
};
return statusMap[status] || status;
}
}
Dependencias
Base de Datos
- PostgreSQL: user_memory, conversations, messages
- Redis: short-term memory, cache
Bibliotecas
- tiktoken (conteo de tokens)
- @anthropic-ai/sdk / openai (LLM calls)
Referencias
Especificación técnica - Sistema NEXUS Trading Platform