erp-core/backend/src/modules/financial/MIGRATION_GUIDE.md

Financial Module TypeORM Migration Guide

Overview

This guide documents the migration of the Financial module from raw SQL queries to TypeORM. The migration maintains backwards compatibility while introducing modern ORM patterns.

Completed Tasks

1. Entity Creation ✅

All TypeORM entities have been created in /src/modules/financial/entities/:

• account-type.entity.ts - Chart of account types catalog
• account.entity.ts - Accounts with hierarchy support
• journal.entity.ts - Accounting journals
• journal-entry.entity.ts - Journal entries (header)
• journal-entry-line.entity.ts - Journal entry lines (detail)
• invoice.entity.ts - Customer and supplier invoices
• invoice-line.entity.ts - Invoice line items
• payment.entity.ts - Payment transactions
• tax.entity.ts - Tax configuration
• fiscal-year.entity.ts - Fiscal years
• fiscal-period.entity.ts - Fiscal periods (months/quarters)
• index.ts - Barrel export file

2. Entity Registration ✅

All financial entities have been registered in /src/config/typeorm.ts:

• Import statements added
• Entities added to the entities array in AppDataSource configuration

3. Service Refactoring ✅

accounts.service.ts - COMPLETED

The accounts service has been fully migrated to TypeORM with the following features:

journals.service.ts - COMPLETED (2025-01-04)

The journals service has been fully migrated to TypeORM with the following features:

Key Changes:

• Uses Repository<Journal> from TypeORM
• Implements QueryBuilder for complex queries with joins (company, defaultAccount)
• Uses camelCase properties matching entity definitions (companyId, journalType, etc.)
• Maintains all original functionality including:
  • Unique code validation per company
  • Journal entry existence check before delete
  • Soft delete with deletedAt/deletedBy
  • Full CRUD operations with pagination

API Changes (DTOs now use camelCase):

• company_id -> companyId
• journal_type -> journalType
• default_account_id -> defaultAccountId
• sequence_id -> sequenceId
• currency_id -> currencyId

taxes.service.ts - COMPLETED (2025-01-04)

The taxes service has been fully migrated to TypeORM with the following features:

Key Changes:

• Uses Repository<Tax> from TypeORM
• Uses In() operator for batch tax lookups in calculateTaxes()
• Implements QueryBuilder for complex queries with company join
• Uses TaxType enum from entity for type safety
• Maintains all original functionality including:
  • Unique code validation per tenant
  • Tax usage check before delete
  • PRESERVED: calculateTaxes() and calculateDocumentTaxes() logic unchanged

API Changes (DTOs now use camelCase):

• company_id -> companyId
• tax_type -> taxType
• included_in_price -> includedInPrice

Critical Preserved Logic:

• Tax calculation algorithms (lines 321-423 in new file)
• Odoo-style VAT calculation for included/excluded prices
• Document tax consolidation

Key Changes:

• Uses Repository<Account> and Repository<AccountType>
• Implements QueryBuilder for complex queries with joins
• Supports both snake_case (DB) and camelCase (TS) through decorators
• Maintains all original functionality including:
  • Account hierarchy with cycle detection
  • Soft delete with validation
  • Balance calculations
  • Full CRUD operations

Pattern to Follow:

import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { Entity } from './entities/index.js';

class MyService {
  private repository: Repository<Entity>;

  constructor() {
    this.repository = AppDataSource.getRepository(Entity);
  }

  async findAll(tenantId: string, filters = {}) {
    const queryBuilder = this.repository
      .createQueryBuilder('alias')
      .leftJoin('alias.relation', 'relation')
      .addSelect(['relation.field'])
      .where('alias.tenantId = :tenantId', { tenantId });

    // Apply filters
    // Get count and results
    return { data, total };
  }
}

Remaining Tasks

Services to Migrate

1. journals.service.ts - ✅ COMPLETED (2025-01-04)

Current State: Uses raw SQL queries Status: Migrated to TypeORM Repository pattern

---

2. taxes.service.ts - ✅ COMPLETED (2025-01-04)

Current State: Uses raw SQL queries Status: Migrated to TypeORM Repository pattern Note: Tax calculation logic preserved exactly as specified

---

3. journal-entries.service.ts - COMPLETED (2025-01-04)

Current State: Uses raw SQL with transactions Status: Migrated to TypeORM Repository pattern with QueryRunner transactions

Key Changes:

• Uses Repository<JournalEntry> and Repository<JournalEntryLine> from TypeORM
• Uses QueryRunner for transaction management (create, update operations)
• Implements QueryBuilder for complex queries with joins (company, journal)
• Uses camelCase properties matching entity definitions

Critical Preserved Logic:

• Double-entry balance validation (debits must equal credits with 0.01 tolerance)
• Minimum 2 lines validation
• Status transitions (draft -> posted -> cancelled)
• Only draft entries can be modified/deleted
• Multi-tenant security with tenantId on all operations

API Changes (DTOs now use camelCase):

• company_id -> companyId
• journal_id -> journalId
• account_id -> accountId
• partner_id -> partnerId
• date_from -> dateFrom
• date_to -> dateTo
• total_debit -> totalDebit
• total_credit -> totalCredit

Transaction Pattern Used:

const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // Operations using queryRunner.manager
  await queryRunner.manager.save(JournalEntry, entry);
  await queryRunner.manager.save(JournalEntryLine, line);
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}

---

4. invoices.service.ts - ✅ COMPLETED (2025-01-04)

Current State: Uses raw SQL with complex line management Status: Migrated to TypeORM Repository pattern

Key Changes:

• Uses Repository<Invoice> and Repository<InvoiceLine> from TypeORM
• Implements QueryBuilder for main queries, raw SQL for cross-schema joins
• Uses camelCase properties matching entity definitions
• Preserved all critical business logic:
  • Tax calculation integration with taxesService.calculateTaxes()
  • Invoice status workflow (draft -> open -> paid/cancelled)
  • Sequential number generation (INV-XXXXXX / BILL-XXXXXX)
  • Line management with automatic total recalculation
  • Only draft invoices can be modified/deleted
  • Payment amount tracking (amountPaid, amountResidual)

API Changes (DTOs now use camelCase):

• company_id -> companyId
• partner_id -> partnerId
• invoice_type -> invoiceType
• invoice_date -> invoiceDate
• due_date -> dueDate
• currency_id -> currencyId
• payment_term_id -> paymentTermId
• journal_id -> journalId
• journal_entry_id -> journalEntryId
• amount_untaxed -> amountUntaxed
• amount_tax -> amountTax
• amount_total -> amountTotal
• amount_paid -> amountPaid
• amount_residual -> amountResidual
• product_id -> productId
• price_unit -> priceUnit
• tax_ids -> taxIds
• uom_id -> uomId
• account_id -> accountId
• date_from -> dateFrom
• date_to -> dateTo

Critical Preserved Logic:

• Tax calculation for invoice lines using taxesService
• Invoice totals recalculation from lines (updateTotals method)
• Transaction type determination (sales vs purchase) for tax applicability
• Cascade delete for invoice lines
• Multi-tenant security with tenantId on all operations

Cross-Schema Joins:

• Used raw SQL queries for joins with core.partners, core.currencies, inventory.products
• TypeORM QueryBuilder used for financial schema relations only

---

5. payments.service.ts - ✅ COMPLETED (2025-01-04)

Current State: Uses raw SQL with invoice reconciliation Status: Migrated to TypeORM Repository pattern with QueryRunner transactions

Key Changes:

• Uses Repository<Payment>, Repository<PaymentInvoice>, and Repository<Invoice> from TypeORM
• Created PaymentInvoice entity for payment-invoice junction table
• Uses QueryRunner for transaction management (reconcile, cancel operations)
• Implements QueryBuilder for complex queries with joins (company, journal)
• Uses raw SQL for cross-schema joins (partners, currencies)
• Uses camelCase properties matching entity definitions

Critical Preserved Logic:

• Payment reconciliation workflow with invoice validation
• Invoice amount updates (amountPaid, amountResidual, status)
• Partner validation (invoice must belong to same partner as payment)
• Amount validation (reconciled amount cannot exceed payment amount or invoice residual)
• Transaction rollback on errors
• Status transitions (draft -> posted -> reconciled -> cancelled)
• Only draft payments can be modified/deleted
• Reverse reconciliations on cancel
• Multi-tenant security with tenantId on all operations

API Changes (DTOs now use camelCase):

• company_id -> companyId
• partner_id -> partnerId
• payment_type -> paymentType
• payment_method -> paymentMethod
• currency_id -> currencyId
• payment_date -> paymentDate
• journal_id -> journalId
• journal_entry_id -> journalEntryId
• date_from -> dateFrom
• date_to -> dateTo
• invoice_id -> invoiceId
• invoice_number -> invoiceNumber

Transaction Pattern Used:

const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // Remove existing payment-invoice links
  await queryRunner.manager.delete(PaymentInvoice, { paymentId: id });

  // Create new payment-invoice links
  await queryRunner.manager.save(PaymentInvoice, paymentInvoice);

  // Update invoice amounts
  await queryRunner.manager.update(Invoice, { id }, { amountPaid, amountResidual, status });

  // Update payment status
  await queryRunner.manager.update(Payment, { id }, { status });

  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}

---

6. fiscalPeriods.service.ts - COMPLETED (2025-01-04)

Current State: Uses raw SQL + database functions Status: Migrated to TypeORM Repository pattern

Key Changes:

• Uses Repository<FiscalYear> and Repository<FiscalPeriod> from TypeORM
• Implements QueryBuilder for complex queries with joins (fiscalYear, company)
• Uses camelCase properties matching entity definitions
• Maintains all original functionality including:
  • Date overlap validation for years and periods
  • Database function calls for close/reopen operations (preserved as raw SQL)
  • Monthly period generation
  • Period statistics calculation
  • User name lookup for closedBy field

API Changes (DTOs now use camelCase):

• company_id -> companyId
• fiscal_year_id -> fiscalYearId
• date_from -> dateFrom
• date_to -> dateTo
• closed_at -> closedAt
• closed_by -> closedBy

Critical Preserved Logic:

• Database functions for close/reopen (lines 443-499):

  await this.fiscalPeriodRepository.query(
    'SELECT * FROM financial.close_fiscal_period($1, $2)',
    [periodId, userId]
  );
  

• PostgreSQL OVERLAPS operator for date range validation
• Monthly period generation algorithm
• Period statistics using raw SQL (fiscal_period_id reference)
• Manual snake_case to camelCase mapping for DB function results

---

Controller Updates

Accept Both snake_case and camelCase

The controller currently only accepts snake_case. Update to support both:

Current:

const createAccountSchema = z.object({
  company_id: z.string().uuid(),
  code: z.string(),
  // ...
});

Updated:

const createAccountSchema = z.object({
  companyId: z.string().uuid().optional(),
  company_id: z.string().uuid().optional(),
  code: z.string(),
  // ...
}).refine(
  (data) => data.companyId || data.company_id,
  { message: "Either companyId or company_id is required" }
);

// Then normalize before service call:
const dto = {
  companyId: parseResult.data.companyId || parseResult.data.company_id,
  // ... rest of fields
};

Simpler Approach: Transform incoming data before validation:

// Add utility function
function toCamelCase(obj: any): any {
  const camelObj: any = {};
  for (const key in obj) {
    const camelKey = key.replace(/_([a-z])/g, (g) => g[1].toUpperCase());
    camelObj[camelKey] = obj[key];
  }
  return camelObj;
}

// Use in controller
const normalizedBody = toCamelCase(req.body);
const parseResult = createAccountSchema.safeParse(normalizedBody);

---

Migration Patterns

1. Repository Setup

import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { MyEntity } from './entities/index.js';

class MyService {
  private repository: Repository<MyEntity>;

  constructor() {
    this.repository = AppDataSource.getRepository(MyEntity);
  }
}

2. Simple Find Operations

Before (Raw SQL):

const result = await queryOne<Entity>(
  `SELECT * FROM schema.table WHERE id = $1 AND tenant_id = $2`,
  [id, tenantId]
);

After (TypeORM):

const result = await this.repository.findOne({
  where: { id, tenantId, deletedAt: IsNull() }
});

3. Complex Queries with Joins

Before:

const data = await query<Entity>(
  `SELECT e.*, r.name as relation_name
   FROM schema.entities e
   LEFT JOIN schema.relations r ON e.relation_id = r.id
   WHERE e.tenant_id = $1`,
  [tenantId]
);

After:

const data = await this.repository
  .createQueryBuilder('entity')
  .leftJoin('entity.relation', 'relation')
  .addSelect(['relation.name'])
  .where('entity.tenantId = :tenantId', { tenantId })
  .getMany();

4. Transactions

Before:

const client = await getClient();
try {
  await client.query('BEGIN');
  // operations
  await client.query('COMMIT');
} catch (error) {
  await client.query('ROLLBACK');
  throw error;
} finally {
  client.release();
}

After:

const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // operations using queryRunner.manager
  await queryRunner.manager.save(entity);
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}

5. Soft Deletes

Pattern:

await this.repository.update(
  { id, tenantId },
  {
    deletedAt: new Date(),
    deletedBy: userId,
  }
);

6. Pagination

const skip = (page - 1) * limit;

const [data, total] = await this.repository.findAndCount({
  where: { tenantId, deletedAt: IsNull() },
  skip,
  take: limit,
  order: { createdAt: 'DESC' },
});

return { data, total };

---

Testing Strategy

1. Unit Tests

For each refactored service:

describe('AccountsService', () => {
  let service: AccountsService;
  let repository: Repository<Account>;

  beforeEach(() => {
    repository = AppDataSource.getRepository(Account);
    service = new AccountsService();
  });

  it('should create account with valid data', async () => {
    const dto = { /* ... */ };
    const result = await service.create(dto, tenantId, userId);
    expect(result.id).toBeDefined();
    expect(result.code).toBe(dto.code);
  });
});

2. Integration Tests

Test with actual database:

# Run tests
npm test src/modules/financial/__tests__/

3. API Tests

Test HTTP endpoints:

# Test accounts endpoints
curl -X GET http://localhost:3000/api/financial/accounts?companyId=xxx
curl -X POST http://localhost:3000/api/financial/accounts -d '{"companyId":"xxx",...}'

---

Rollback Plan

If migration causes issues:

1. Restore Old Services:

cd src/modules/financial
mv accounts.service.ts accounts.service.new.ts
mv accounts.service.old.ts accounts.service.ts

2. Remove Entity Imports: Edit /src/config/typeorm.ts and remove financial entity imports

3. Restart Application:

npm run dev

---

Database Schema Notes

Schema: financial

All tables use the financial schema as specified in entities.

Important Columns:

• tenant_id: Multi-tenancy isolation (UUID, NOT NULL)
• company_id: Company isolation (UUID, NOT NULL)
• deleted_at: Soft delete timestamp (NULL = active)
• created_at: Audit timestamp
• created_by: User ID who created (UUID)
• updated_at: Audit timestamp
• updated_by: User ID who updated (UUID)

Decimal Precision:

• Amounts: DECIMAL(15, 2) - invoices, payments
• Quantity: DECIMAL(15, 4) - invoice lines
• Tax Rate: DECIMAL(5, 2) - tax percentage

---

Common Issues and Solutions

Issue 1: Column Name Mismatch

Error: column "companyId" does not exist

Solution: Entity decorators map camelCase to snake_case:

@Column({ name: 'company_id' })
companyId: string;

Issue 2: Soft Deletes Not Working

Solution: Always include deletedAt: IsNull() in where clauses:

where: { id, tenantId, deletedAt: IsNull() }

Issue 3: Transaction Not Rolling Back

Solution: Always use try-catch-finally with queryRunner:

finally {
  await queryRunner.release(); // MUST release
}

Issue 4: Relations Not Loading

Solution: Use leftJoin or relations option:

// Option 1: Query Builder
.leftJoin('entity.relation', 'relation')
.addSelect(['relation.field'])

// Option 2: Find options
findOne({
  where: { id },
  relations: ['relation'],
})

---

Performance Considerations

1. Query Optimization

• Use leftJoin + addSelect instead of relations option for better control
• Add indexes on frequently queried columns (already in entities)
• Use pagination for large result sets

2. Connection Pooling

TypeORM pool configuration (in typeorm.ts):

extra: {
  max: 10,  // Conservative to not compete with pg pool
  min: 2,
  idleTimeoutMillis: 30000,
}

3. Caching

Currently disabled:

cache: false

Can enable later for read-heavy operations.

---

Next Steps

1. Complete service migrations in this order:

   • taxes.service.ts (High priority, simple) ✅ DONE
   • journals.service.ts (High priority, simple) ✅ DONE
   • journal-entries.service.ts (Medium, complex transactions) ✅ DONE
   • payments.service.ts (Medium, reconciliation) ✅ DONE
   • invoices.service.ts (Medium, tax integration) ✅ DONE
   • fiscalPeriods.service.ts (Low, DB functions) ✅ DONE

2. Update controller to accept both snake_case and camelCase

3. Write tests for each migrated service

4. Update API documentation to reflect camelCase support

5. Monitor performance after deployment

---

Support and Questions

For questions about this migration:

• Check existing patterns in accounts.service.ts
• Review TypeORM documentation: https://typeorm.io
• Check entity definitions in /entities/ folder

---

Changelog

2025-01-04

• Completed fiscalPeriods.service.ts migration to TypeORM
  • Replaced raw SQL with Repository pattern for FiscalYear and FiscalPeriod
  • Implemented QueryBuilder for complex queries with joins (fiscalYear, company)
  • Preserved database function calls for close/reopen operations using repository.query()
  • Preserved all critical business logic:
    • Date overlap validation using PostgreSQL OVERLAPS operator
    • Monthly period generation algorithm
    • Period statistics calculation
    • User name lookup for closedBy field
  • Manual snake_case to camelCase mapping for database function results
  • Converted DTOs to camelCase
  • Added comprehensive logging
• Completed payments.service.ts migration to TypeORM
  • Created PaymentInvoice entity for payment-invoice junction table
  • Replaced raw SQL with Repository pattern for Payment, PaymentInvoice, and Invoice
  • Used QueryRunner for transaction management (reconcile, cancel operations)
  • Used QueryBuilder for main queries, raw SQL for cross-schema joins (partners, currencies)
  • Preserved all critical business logic:
    • Payment reconciliation workflow with invoice validation
    • Invoice amount updates (amountPaid, amountResidual, status)
    • Partner validation (invoice must belong to same partner as payment)
    • Amount validation (reconciled amount cannot exceed payment or invoice residual)
    • Status transitions (draft -> posted -> reconciled -> cancelled)
    • Only draft payments can be modified/deleted
    • Reverse reconciliations on cancel
  • Converted DTOs to camelCase
  • Added comprehensive logging
• Completed invoices.service.ts migration to TypeORM
  • Replaced raw SQL with Repository pattern for Invoice and InvoiceLine
  • Used QueryBuilder for main queries, raw SQL for cross-schema joins
  • Preserved all critical business logic:
    • Tax calculation integration with taxesService
    • Invoice status workflow (draft -> open -> paid/cancelled)
    • Sequential number generation (INV-XXXXXX / BILL-XXXXXX)
    • Line management with automatic total recalculation
    • Payment tracking (amountPaid, amountResidual)
  • Converted DTOs to camelCase
  • Added comprehensive logging
• Completed journal-entries.service.ts migration to TypeORM
  • Replaced raw SQL with Repository pattern
  • Used QueryRunner for transaction management (create, update)
  • Implemented QueryBuilder for complex queries with joins
  • Preserved all accounting logic:
    • Double-entry balance validation (debits = credits)
    • Minimum 2 lines validation
    • Status transitions (draft -> posted -> cancelled)
    • Only draft entries can be modified/deleted
  • Converted DTOs to camelCase
• Completed journals.service.ts migration to TypeORM
  • Replaced raw SQL with Repository pattern
  • Implemented QueryBuilder for joins
  • Converted DTOs to camelCase
• Completed taxes.service.ts migration to TypeORM
  • Replaced raw SQL CRUD with Repository pattern
  • Used In() operator for batch tax lookups
  • Preserved calculateTaxes() and calculateDocumentTaxes() logic exactly
  • Converted DTOs to camelCase
• Updated MIGRATION_GUIDE.md with progress

2024-12-14

• Created all TypeORM entities
• Registered entities in AppDataSource
• Completed accounts.service.ts migration
• Created this migration guide