erp-core-backend/src/modules/financial/MIGRATION_GUIDE.md

Financial Module TypeORM Migration Guide

Overview

This guide documents the migration of the Financial module from raw SQL queries to TypeORM. The migration maintains backwards compatibility while introducing modern ORM patterns.

Completed Tasks

1. Entity Creation ✅

All TypeORM entities have been created in /src/modules/financial/entities/:

• account-type.entity.ts - Chart of account types catalog
• account.entity.ts - Accounts with hierarchy support
• journal.entity.ts - Accounting journals
• journal-entry.entity.ts - Journal entries (header)
• journal-entry-line.entity.ts - Journal entry lines (detail)
• invoice.entity.ts - Customer and supplier invoices
• invoice-line.entity.ts - Invoice line items
• payment.entity.ts - Payment transactions
• tax.entity.ts - Tax configuration
• fiscal-year.entity.ts - Fiscal years
• fiscal-period.entity.ts - Fiscal periods (months/quarters)
• index.ts - Barrel export file

2. Entity Registration ✅

All financial entities have been registered in /src/config/typeorm.ts:

• Import statements added
• Entities added to the entities array in AppDataSource configuration

3. Service Refactoring ✅

accounts.service.ts - COMPLETED

The accounts service has been fully migrated to TypeORM with the following features:

Key Changes:

• Uses Repository<Account> and Repository<AccountType>
• Implements QueryBuilder for complex queries with joins
• Supports both snake_case (DB) and camelCase (TS) through decorators
• Maintains all original functionality including:
  • Account hierarchy with cycle detection
  • Soft delete with validation
  • Balance calculations
  • Full CRUD operations

Pattern to Follow:

import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { Entity } from './entities/index.js';

class MyService {
  private repository: Repository<Entity>;

  constructor() {
    this.repository = AppDataSource.getRepository(Entity);
  }

  async findAll(tenantId: string, filters = {}) {
    const queryBuilder = this.repository
      .createQueryBuilder('alias')
      .leftJoin('alias.relation', 'relation')
      .addSelect(['relation.field'])
      .where('alias.tenantId = :tenantId', { tenantId });

    // Apply filters
    // Get count and results
    return { data, total };
  }
}

Remaining Tasks

Services to Migrate

1. journals.service.ts - PRIORITY HIGH

Current State: Uses raw SQL queries Target Pattern: Same as accounts.service.ts

Migration Steps:

1. Import Journal entity and Repository
2. Replace all query() and queryOne() calls with Repository methods
3. Use QueryBuilder for complex queries with joins (company, account, currency)
4. Update return types to use entity types instead of interfaces
5. Maintain validation logic for:
   • Unique code per company
   • Journal entry existence check before delete
6. Test endpoints thoroughly

Key Relationships:

• Journal → Company (ManyToOne)
• Journal → Account (default account, ManyToOne, optional)

---

2. taxes.service.ts - PRIORITY HIGH

Current State: Uses raw SQL queries Special Feature: Tax calculation logic

Migration Steps:

1. Import Tax entity and Repository
2. Migrate CRUD operations to Repository
3. IMPORTANT: Keep calculateTaxes() and calculateDocumentTaxes() logic intact
4. These calculation methods can still use raw queries if needed
5. Update filters to use QueryBuilder

Tax Calculation Logic:

• Located in lines 224-354 of current service
• Critical for invoice and payment processing
• DO NOT modify calculation algorithms
• Only update data access layer

---

3. journal-entries.service.ts - PRIORITY MEDIUM

Current State: Uses raw SQL with transactions Complexity: HIGH - Multi-table operations

Migration Steps:

1. Import JournalEntry, JournalEntryLine entities
2. Use TypeORM QueryRunner for transactions:

const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // Operations
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}

3. Double-Entry Balance Validation:
   • Keep validation logic lines 172-177
   • Validate debit = credit before saving
4. Use cascade operations for lines:
   • cascade: true is already set in entity
   • Can save entry with lines in single operation

Critical Features:

• Transaction management (BEGIN/COMMIT/ROLLBACK)
• Balance validation (debits must equal credits)
• Status transitions (draft → posted → cancelled)
• Fiscal period validation

---

4. invoices.service.ts - PRIORITY MEDIUM

Current State: Uses raw SQL with complex line management Complexity: HIGH - Invoice lines, tax calculations

Migration Steps:

1. Import Invoice, InvoiceLine entities
2. Use transactions for multi-table operations
3. Tax Integration:
   • Line 331-340: Uses taxesService.calculateTaxes()
   • Keep this integration intact
   • Only migrate data access
4. Amount Calculations:
   • updateTotals() method (lines 525-543)
   • Can use QueryBuilder aggregation or raw SQL
5. Number Generation:
   • Lines 472-478: Sequential invoice numbering
   • Keep this logic, migrate to Repository

Relationships:

• Invoice → Company
• Invoice → Journal (optional)
• Invoice → JournalEntry (optional, for accounting integration)
• Invoice → InvoiceLine[] (one-to-many, cascade)
• InvoiceLine → Account (optional)

---

5. payments.service.ts - PRIORITY MEDIUM

Current State: Uses raw SQL with invoice reconciliation Complexity: MEDIUM-HIGH - Payment-Invoice linking

Migration Steps:

1. Import Payment entity
2. Payment-Invoice Junction:
   • Table: financial.payment_invoice
   • Not modeled as entity (junction table)
   • Can use raw SQL for this or create entity
3. Use transactions for reconciliation
4. Invoice Status Updates:
   • Lines 373-380: Updates invoice amounts
   • Must coordinate with Invoice entity

Critical Logic:

• Reconciliation workflow (lines 314-401)
• Invoice amount updates
• Transaction rollback on errors

---

6. fiscalPeriods.service.ts - PRIORITY LOW

Current State: Uses raw SQL + database functions Complexity: MEDIUM - Database function calls

Migration Steps:

1. Import FiscalYear, FiscalPeriod entities
2. Basic CRUD can use Repository
3. Database Functions:
   • Line 242: financial.close_fiscal_period()
   • Line 265: financial.reopen_fiscal_period()
   • Keep these as raw SQL calls:

   await this.repository.query(
     'SELECT * FROM financial.close_fiscal_period($1, $2)',
     [periodId, userId]
   );
   

4. Date Overlap Validation:
   • Lines 102-107, 207-212
   • Use QueryBuilder with date range checks

---

Controller Updates

Accept Both snake_case and camelCase

The controller currently only accepts snake_case. Update to support both:

Current:

const createAccountSchema = z.object({
  company_id: z.string().uuid(),
  code: z.string(),
  // ...
});

Updated:

const createAccountSchema = z.object({
  companyId: z.string().uuid().optional(),
  company_id: z.string().uuid().optional(),
  code: z.string(),
  // ...
}).refine(
  (data) => data.companyId || data.company_id,
  { message: "Either companyId or company_id is required" }
);

// Then normalize before service call:
const dto = {
  companyId: parseResult.data.companyId || parseResult.data.company_id,
  // ... rest of fields
};

Simpler Approach: Transform incoming data before validation:

// Add utility function
function toCamelCase(obj: any): any {
  const camelObj: any = {};
  for (const key in obj) {
    const camelKey = key.replace(/_([a-z])/g, (g) => g[1].toUpperCase());
    camelObj[camelKey] = obj[key];
  }
  return camelObj;
}

// Use in controller
const normalizedBody = toCamelCase(req.body);
const parseResult = createAccountSchema.safeParse(normalizedBody);

---

Migration Patterns

1. Repository Setup

import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { MyEntity } from './entities/index.js';

class MyService {
  private repository: Repository<MyEntity>;

  constructor() {
    this.repository = AppDataSource.getRepository(MyEntity);
  }
}

2. Simple Find Operations

Before (Raw SQL):

const result = await queryOne<Entity>(
  `SELECT * FROM schema.table WHERE id = $1 AND tenant_id = $2`,
  [id, tenantId]
);

After (TypeORM):

const result = await this.repository.findOne({
  where: { id, tenantId, deletedAt: IsNull() }
});

3. Complex Queries with Joins

Before:

const data = await query<Entity>(
  `SELECT e.*, r.name as relation_name
   FROM schema.entities e
   LEFT JOIN schema.relations r ON e.relation_id = r.id
   WHERE e.tenant_id = $1`,
  [tenantId]
);

After:

const data = await this.repository
  .createQueryBuilder('entity')
  .leftJoin('entity.relation', 'relation')
  .addSelect(['relation.name'])
  .where('entity.tenantId = :tenantId', { tenantId })
  .getMany();

4. Transactions

Before:

const client = await getClient();
try {
  await client.query('BEGIN');
  // operations
  await client.query('COMMIT');
} catch (error) {
  await client.query('ROLLBACK');
  throw error;
} finally {
  client.release();
}

After:

const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // operations using queryRunner.manager
  await queryRunner.manager.save(entity);
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}

5. Soft Deletes

Pattern:

await this.repository.update(
  { id, tenantId },
  {
    deletedAt: new Date(),
    deletedBy: userId,
  }
);

6. Pagination

const skip = (page - 1) * limit;

const [data, total] = await this.repository.findAndCount({
  where: { tenantId, deletedAt: IsNull() },
  skip,
  take: limit,
  order: { createdAt: 'DESC' },
});

return { data, total };

---

Testing Strategy

1. Unit Tests

For each refactored service:

describe('AccountsService', () => {
  let service: AccountsService;
  let repository: Repository<Account>;

  beforeEach(() => {
    repository = AppDataSource.getRepository(Account);
    service = new AccountsService();
  });

  it('should create account with valid data', async () => {
    const dto = { /* ... */ };
    const result = await service.create(dto, tenantId, userId);
    expect(result.id).toBeDefined();
    expect(result.code).toBe(dto.code);
  });
});

2. Integration Tests

Test with actual database:

# Run tests
npm test src/modules/financial/__tests__/

3. API Tests

Test HTTP endpoints:

# Test accounts endpoints
curl -X GET http://localhost:3000/api/financial/accounts?companyId=xxx
curl -X POST http://localhost:3000/api/financial/accounts -d '{"companyId":"xxx",...}'

---

Rollback Plan

If migration causes issues:

1. Restore Old Services:

cd src/modules/financial
mv accounts.service.ts accounts.service.new.ts
mv accounts.service.old.ts accounts.service.ts

2. Remove Entity Imports: Edit /src/config/typeorm.ts and remove financial entity imports

3. Restart Application:

npm run dev

---

Database Schema Notes

Schema: financial

All tables use the financial schema as specified in entities.

Important Columns:

• tenant_id: Multi-tenancy isolation (UUID, NOT NULL)
• company_id: Company isolation (UUID, NOT NULL)
• deleted_at: Soft delete timestamp (NULL = active)
• created_at: Audit timestamp
• created_by: User ID who created (UUID)
• updated_at: Audit timestamp
• updated_by: User ID who updated (UUID)

Decimal Precision:

• Amounts: DECIMAL(15, 2) - invoices, payments
• Quantity: DECIMAL(15, 4) - invoice lines
• Tax Rate: DECIMAL(5, 2) - tax percentage

---

Common Issues and Solutions

Issue 1: Column Name Mismatch

Error: column "companyId" does not exist

Solution: Entity decorators map camelCase to snake_case:

@Column({ name: 'company_id' })
companyId: string;

Issue 2: Soft Deletes Not Working

Solution: Always include deletedAt: IsNull() in where clauses:

where: { id, tenantId, deletedAt: IsNull() }

Issue 3: Transaction Not Rolling Back

Solution: Always use try-catch-finally with queryRunner:

finally {
  await queryRunner.release(); // MUST release
}

Issue 4: Relations Not Loading

Solution: Use leftJoin or relations option:

// Option 1: Query Builder
.leftJoin('entity.relation', 'relation')
.addSelect(['relation.field'])

// Option 2: Find options
findOne({
  where: { id },
  relations: ['relation'],
})

---

Performance Considerations

1. Query Optimization

• Use leftJoin + addSelect instead of relations option for better control
• Add indexes on frequently queried columns (already in entities)
• Use pagination for large result sets

2. Connection Pooling

TypeORM pool configuration (in typeorm.ts):

extra: {
  max: 10,  // Conservative to not compete with pg pool
  min: 2,
  idleTimeoutMillis: 30000,
}

3. Caching

Currently disabled:

cache: false

Can enable later for read-heavy operations.

---

Next Steps

1. Complete service migrations in this order:

   • taxes.service.ts (High priority, simple)
   • journals.service.ts (High priority, simple)
   • journal-entries.service.ts (Medium, complex transactions)
   • invoices.service.ts (Medium, tax integration)
   • payments.service.ts (Medium, reconciliation)
   • fiscalPeriods.service.ts (Low, DB functions)

2. Update controller to accept both snake_case and camelCase

3. Write tests for each migrated service

4. Update API documentation to reflect camelCase support

5. Monitor performance after deployment

---

Support and Questions

For questions about this migration:

• Check existing patterns in accounts.service.ts
• Review TypeORM documentation: https://typeorm.io
• Check entity definitions in /entities/ folder

---

Changelog

2024-12-14

• Created all TypeORM entities
• Registered entities in AppDataSource
• Completed accounts.service.ts migration
• Created this migration guide