# Financial Module TypeORM Migration Guide

## Overview

This guide documents the migration of the Financial module from raw SQL queries to TypeORM. The migration maintains backwards compatibility while introducing modern ORM patterns.

## Completed Tasks

### 1. Entity Creation ✅

All TypeORM entities have been created in `/src/modules/financial/entities/`:

- **account-type.entity.ts** - Chart of account types catalog
- **account.entity.ts** - Accounts with hierarchy support
- **journal.entity.ts** - Accounting journals
- **journal-entry.entity.ts** - Journal entries (header)
- **journal-entry-line.entity.ts** - Journal entry lines (detail)
- **invoice.entity.ts** - Customer and supplier invoices
- **invoice-line.entity.ts** - Invoice line items
- **payment.entity.ts** - Payment transactions
- **tax.entity.ts** - Tax configuration
- **fiscal-year.entity.ts** - Fiscal years
- **fiscal-period.entity.ts** - Fiscal periods (months/quarters)
- **index.ts** - Barrel export file

### 2. Entity Registration ✅

All financial entities have been registered in `/src/config/typeorm.ts`:
- Import statements added
- Entities added to the `entities` array in AppDataSource configuration

### 3. Service Refactoring ✅

#### accounts.service.ts - COMPLETED

The accounts service has been fully migrated to TypeORM with the following features:

#### journals.service.ts - COMPLETED (2025-01-04)

The journals service has been fully migrated to TypeORM with the following features:

**Key Changes:**
- Uses `Repository<Journal>` from TypeORM
- Implements QueryBuilder for complex queries with joins (company, defaultAccount)
- Uses camelCase properties matching entity definitions (companyId, journalType, etc.)
- Maintains all original functionality including:
  - Unique code validation per company
  - Journal entry existence check before delete
  - Soft delete with deletedAt/deletedBy
  - Full CRUD operations with pagination

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `journal_type` -> `journalType`
- `default_account_id` -> `defaultAccountId`
- `sequence_id` -> `sequenceId`
- `currency_id` -> `currencyId`

#### taxes.service.ts - COMPLETED (2025-01-04)

The taxes service has been fully migrated to TypeORM with the following features:

**Key Changes:**
- Uses `Repository<Tax>` from TypeORM
- Uses `In()` operator for batch tax lookups in calculateTaxes()
- Implements QueryBuilder for complex queries with company join
- Uses TaxType enum from entity for type safety
- Maintains all original functionality including:
  - Unique code validation per tenant
  - Tax usage check before delete
  - **PRESERVED: calculateTaxes() and calculateDocumentTaxes() logic unchanged**

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `tax_type` -> `taxType`
- `included_in_price` -> `includedInPrice`

**Critical Preserved Logic:**
- Tax calculation algorithms (lines 321-423 in new file)
- Odoo-style VAT calculation for included/excluded prices
- Document tax consolidation

**Key Changes:**
- Uses `Repository<Account>` and `Repository<AccountType>`
- Implements QueryBuilder for complex queries with joins
- Supports both snake_case (DB) and camelCase (TS) through decorators
- Maintains all original functionality including:
  - Account hierarchy with cycle detection
  - Soft delete with validation
  - Balance calculations
  - Full CRUD operations

**Pattern to Follow:**
```typescript
import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { Entity } from './entities/index.js';

class MyService {
  private repository: Repository<Entity>;

  constructor() {
    this.repository = AppDataSource.getRepository(Entity);
  }

  async findAll(tenantId: string, filters = {}) {
    const queryBuilder = this.repository
      .createQueryBuilder('alias')
      .leftJoin('alias.relation', 'relation')
      .addSelect(['relation.field'])
      .where('alias.tenantId = :tenantId', { tenantId });

    // Apply filters
    // Get count and results
    return { data, total };
  }
}
```

## Remaining Tasks

### Services to Migrate

#### 1. journals.service.ts - ✅ COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL queries~~
**Status:** Migrated to TypeORM Repository pattern

---

#### 2. taxes.service.ts - ✅ COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL queries~~
**Status:** Migrated to TypeORM Repository pattern
**Note:** Tax calculation logic preserved exactly as specified

---

#### 3. journal-entries.service.ts - COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL with transactions~~
**Status:** Migrated to TypeORM Repository pattern with QueryRunner transactions

**Key Changes:**
- Uses `Repository<JournalEntry>` and `Repository<JournalEntryLine>` from TypeORM
- Uses `QueryRunner` for transaction management (create, update operations)
- Implements QueryBuilder for complex queries with joins (company, journal)
- Uses camelCase properties matching entity definitions

**Critical Preserved Logic:**
- Double-entry balance validation (debits must equal credits with 0.01 tolerance)
- Minimum 2 lines validation
- Status transitions (draft -> posted -> cancelled)
- Only draft entries can be modified/deleted
- Multi-tenant security with tenantId on all operations

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `journal_id` -> `journalId`
- `account_id` -> `accountId`
- `partner_id` -> `partnerId`
- `date_from` -> `dateFrom`
- `date_to` -> `dateTo`
- `total_debit` -> `totalDebit`
- `total_credit` -> `totalCredit`

**Transaction Pattern Used:**
```typescript
const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // Operations using queryRunner.manager
  await queryRunner.manager.save(JournalEntry, entry);
  await queryRunner.manager.save(JournalEntryLine, line);
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}
```

---

#### 4. invoices.service.ts - ✅ COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL with complex line management~~
**Status:** Migrated to TypeORM Repository pattern

**Key Changes:**
- Uses `Repository<Invoice>` and `Repository<InvoiceLine>` from TypeORM
- Implements QueryBuilder for main queries, raw SQL for cross-schema joins
- Uses camelCase properties matching entity definitions
- Preserved all critical business logic:
  - Tax calculation integration with taxesService.calculateTaxes()
  - Invoice status workflow (draft -> open -> paid/cancelled)
  - Sequential number generation (INV-XXXXXX / BILL-XXXXXX)
  - Line management with automatic total recalculation
  - Only draft invoices can be modified/deleted
  - Payment amount tracking (amountPaid, amountResidual)

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `partner_id` -> `partnerId`
- `invoice_type` -> `invoiceType`
- `invoice_date` -> `invoiceDate`
- `due_date` -> `dueDate`
- `currency_id` -> `currencyId`
- `payment_term_id` -> `paymentTermId`
- `journal_id` -> `journalId`
- `journal_entry_id` -> `journalEntryId`
- `amount_untaxed` -> `amountUntaxed`
- `amount_tax` -> `amountTax`
- `amount_total` -> `amountTotal`
- `amount_paid` -> `amountPaid`
- `amount_residual` -> `amountResidual`
- `product_id` -> `productId`
- `price_unit` -> `priceUnit`
- `tax_ids` -> `taxIds`
- `uom_id` -> `uomId`
- `account_id` -> `accountId`
- `date_from` -> `dateFrom`
- `date_to` -> `dateTo`

**Critical Preserved Logic:**
- Tax calculation for invoice lines using taxesService
- Invoice totals recalculation from lines (updateTotals method)
- Transaction type determination (sales vs purchase) for tax applicability
- Cascade delete for invoice lines
- Multi-tenant security with tenantId on all operations

**Cross-Schema Joins:**
- Used raw SQL queries for joins with core.partners, core.currencies, inventory.products
- TypeORM QueryBuilder used for financial schema relations only

---

#### 5. payments.service.ts - ✅ COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL with invoice reconciliation~~
**Status:** Migrated to TypeORM Repository pattern with QueryRunner transactions

**Key Changes:**
- Uses `Repository<Payment>`, `Repository<PaymentInvoice>`, and `Repository<Invoice>` from TypeORM
- Created `PaymentInvoice` entity for payment-invoice junction table
- Uses `QueryRunner` for transaction management (reconcile, cancel operations)
- Implements QueryBuilder for complex queries with joins (company, journal)
- Uses raw SQL for cross-schema joins (partners, currencies)
- Uses camelCase properties matching entity definitions

**Critical Preserved Logic:**
- Payment reconciliation workflow with invoice validation
- Invoice amount updates (amountPaid, amountResidual, status)
- Partner validation (invoice must belong to same partner as payment)
- Amount validation (reconciled amount cannot exceed payment amount or invoice residual)
- Transaction rollback on errors
- Status transitions (draft -> posted -> reconciled -> cancelled)
- Only draft payments can be modified/deleted
- Reverse reconciliations on cancel
- Multi-tenant security with tenantId on all operations

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `partner_id` -> `partnerId`
- `payment_type` -> `paymentType`
- `payment_method` -> `paymentMethod`
- `currency_id` -> `currencyId`
- `payment_date` -> `paymentDate`
- `journal_id` -> `journalId`
- `journal_entry_id` -> `journalEntryId`
- `date_from` -> `dateFrom`
- `date_to` -> `dateTo`
- `invoice_id` -> `invoiceId`
- `invoice_number` -> `invoiceNumber`

**Transaction Pattern Used:**
```typescript
const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // Remove existing payment-invoice links
  await queryRunner.manager.delete(PaymentInvoice, { paymentId: id });

  // Create new payment-invoice links
  await queryRunner.manager.save(PaymentInvoice, paymentInvoice);

  // Update invoice amounts
  await queryRunner.manager.update(Invoice, { id }, { amountPaid, amountResidual, status });

  // Update payment status
  await queryRunner.manager.update(Payment, { id }, { status });

  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}
```

---

#### 6. fiscalPeriods.service.ts - COMPLETED (2025-01-04)

~~**Current State:** Uses raw SQL + database functions~~
**Status:** Migrated to TypeORM Repository pattern

**Key Changes:**
- Uses `Repository<FiscalYear>` and `Repository<FiscalPeriod>` from TypeORM
- Implements QueryBuilder for complex queries with joins (fiscalYear, company)
- Uses camelCase properties matching entity definitions
- Maintains all original functionality including:
  - Date overlap validation for years and periods
  - Database function calls for close/reopen operations (preserved as raw SQL)
  - Monthly period generation
  - Period statistics calculation
  - User name lookup for closedBy field

**API Changes (DTOs now use camelCase):**
- `company_id` -> `companyId`
- `fiscal_year_id` -> `fiscalYearId`
- `date_from` -> `dateFrom`
- `date_to` -> `dateTo`
- `closed_at` -> `closedAt`
- `closed_by` -> `closedBy`

**Critical Preserved Logic:**
- Database functions for close/reopen (lines 443-499):
  ```typescript
  await this.fiscalPeriodRepository.query(
    'SELECT * FROM financial.close_fiscal_period($1, $2)',
    [periodId, userId]
  );
  ```
- PostgreSQL OVERLAPS operator for date range validation
- Monthly period generation algorithm
- Period statistics using raw SQL (fiscal_period_id reference)
- Manual snake_case to camelCase mapping for DB function results

---

## Controller Updates

### Accept Both snake_case and camelCase

The controller currently only accepts snake_case. Update to support both:

**Current:**
```typescript
const createAccountSchema = z.object({
  company_id: z.string().uuid(),
  code: z.string(),
  // ...
});
```

**Updated:**
```typescript
const createAccountSchema = z.object({
  companyId: z.string().uuid().optional(),
  company_id: z.string().uuid().optional(),
  code: z.string(),
  // ...
}).refine(
  (data) => data.companyId || data.company_id,
  { message: "Either companyId or company_id is required" }
);

// Then normalize before service call:
const dto = {
  companyId: parseResult.data.companyId || parseResult.data.company_id,
  // ... rest of fields
};
```

**Simpler Approach:**
Transform incoming data before validation:
```typescript
// Add utility function
function toCamelCase(obj: any): any {
  const camelObj: any = {};
  for (const key in obj) {
    const camelKey = key.replace(/_([a-z])/g, (g) => g[1].toUpperCase());
    camelObj[camelKey] = obj[key];
  }
  return camelObj;
}

// Use in controller
const normalizedBody = toCamelCase(req.body);
const parseResult = createAccountSchema.safeParse(normalizedBody);
```

---

## Migration Patterns

### 1. Repository Setup

```typescript
import { Repository, IsNull } from 'typeorm';
import { AppDataSource } from '../../config/typeorm.js';
import { MyEntity } from './entities/index.js';

class MyService {
  private repository: Repository<MyEntity>;

  constructor() {
    this.repository = AppDataSource.getRepository(MyEntity);
  }
}
```

### 2. Simple Find Operations

**Before (Raw SQL):**
```typescript
const result = await queryOne<Entity>(
  `SELECT * FROM schema.table WHERE id = $1 AND tenant_id = $2`,
  [id, tenantId]
);
```

**After (TypeORM):**
```typescript
const result = await this.repository.findOne({
  where: { id, tenantId, deletedAt: IsNull() }
});
```

### 3. Complex Queries with Joins

**Before:**
```typescript
const data = await query<Entity>(
  `SELECT e.*, r.name as relation_name
   FROM schema.entities e
   LEFT JOIN schema.relations r ON e.relation_id = r.id
   WHERE e.tenant_id = $1`,
  [tenantId]
);
```

**After:**
```typescript
const data = await this.repository
  .createQueryBuilder('entity')
  .leftJoin('entity.relation', 'relation')
  .addSelect(['relation.name'])
  .where('entity.tenantId = :tenantId', { tenantId })
  .getMany();
```

### 4. Transactions

**Before:**
```typescript
const client = await getClient();
try {
  await client.query('BEGIN');
  // operations
  await client.query('COMMIT');
} catch (error) {
  await client.query('ROLLBACK');
  throw error;
} finally {
  client.release();
}
```

**After:**
```typescript
const queryRunner = AppDataSource.createQueryRunner();
await queryRunner.connect();
await queryRunner.startTransaction();

try {
  // operations using queryRunner.manager
  await queryRunner.manager.save(entity);
  await queryRunner.commitTransaction();
} catch (error) {
  await queryRunner.rollbackTransaction();
  throw error;
} finally {
  await queryRunner.release();
}
```

### 5. Soft Deletes

**Pattern:**
```typescript
await this.repository.update(
  { id, tenantId },
  {
    deletedAt: new Date(),
    deletedBy: userId,
  }
);
```

### 6. Pagination

```typescript
const skip = (page - 1) * limit;

const [data, total] = await this.repository.findAndCount({
  where: { tenantId, deletedAt: IsNull() },
  skip,
  take: limit,
  order: { createdAt: 'DESC' },
});

return { data, total };
```

---

## Testing Strategy

### 1. Unit Tests

For each refactored service:

```typescript
describe('AccountsService', () => {
  let service: AccountsService;
  let repository: Repository<Account>;

  beforeEach(() => {
    repository = AppDataSource.getRepository(Account);
    service = new AccountsService();
  });

  it('should create account with valid data', async () => {
    const dto = { /* ... */ };
    const result = await service.create(dto, tenantId, userId);
    expect(result.id).toBeDefined();
    expect(result.code).toBe(dto.code);
  });
});
```

### 2. Integration Tests

Test with actual database:

```bash
# Run tests
npm test src/modules/financial/__tests__/
```

### 3. API Tests

Test HTTP endpoints:

```bash
# Test accounts endpoints
curl -X GET http://localhost:3000/api/financial/accounts?companyId=xxx
curl -X POST http://localhost:3000/api/financial/accounts -d '{"companyId":"xxx",...}'
```

---

## Rollback Plan

If migration causes issues:

1. **Restore Old Services:**
```bash
cd src/modules/financial
mv accounts.service.ts accounts.service.new.ts
mv accounts.service.old.ts accounts.service.ts
```

2. **Remove Entity Imports:**
Edit `/src/config/typeorm.ts` and remove financial entity imports

3. **Restart Application:**
```bash
npm run dev
```

---

## Database Schema Notes

### Schema: `financial`

All tables use the `financial` schema as specified in entities.

### Important Columns:

- **tenant_id**: Multi-tenancy isolation (UUID, NOT NULL)
- **company_id**: Company isolation (UUID, NOT NULL)
- **deleted_at**: Soft delete timestamp (NULL = active)
- **created_at**: Audit timestamp
- **created_by**: User ID who created (UUID)
- **updated_at**: Audit timestamp
- **updated_by**: User ID who updated (UUID)

### Decimal Precision:

- **Amounts**: DECIMAL(15, 2) - invoices, payments
- **Quantity**: DECIMAL(15, 4) - invoice lines
- **Tax Rate**: DECIMAL(5, 2) - tax percentage

---

## Common Issues and Solutions

### Issue 1: Column Name Mismatch

**Error:** `column "companyId" does not exist`

**Solution:** Entity decorators map camelCase to snake_case:
```typescript
@Column({ name: 'company_id' })
companyId: string;
```

### Issue 2: Soft Deletes Not Working

**Solution:** Always include `deletedAt: IsNull()` in where clauses:
```typescript
where: { id, tenantId, deletedAt: IsNull() }
```

### Issue 3: Transaction Not Rolling Back

**Solution:** Always use try-catch-finally with queryRunner:
```typescript
finally {
  await queryRunner.release(); // MUST release
}
```

### Issue 4: Relations Not Loading

**Solution:** Use leftJoin or relations option:
```typescript
// Option 1: Query Builder
.leftJoin('entity.relation', 'relation')
.addSelect(['relation.field'])

// Option 2: Find options
findOne({
  where: { id },
  relations: ['relation'],
})
```

---

## Performance Considerations

### 1. Query Optimization

- Use `leftJoin` + `addSelect` instead of `relations` option for better control
- Add indexes on frequently queried columns (already in entities)
- Use pagination for large result sets

### 2. Connection Pooling

TypeORM pool configuration (in typeorm.ts):
```typescript
extra: {
  max: 10,  // Conservative to not compete with pg pool
  min: 2,
  idleTimeoutMillis: 30000,
}
```

### 3. Caching

Currently disabled:
```typescript
cache: false
```

Can enable later for read-heavy operations.

---

## Next Steps

1. **Complete service migrations** in this order:
   - ~~taxes.service.ts (High priority, simple)~~ ✅ DONE
   - ~~journals.service.ts (High priority, simple)~~ ✅ DONE
   - ~~journal-entries.service.ts (Medium, complex transactions)~~ ✅ DONE
   - ~~payments.service.ts (Medium, reconciliation)~~ ✅ DONE
   - ~~invoices.service.ts (Medium, tax integration)~~ ✅ DONE
   - ~~fiscalPeriods.service.ts (Low, DB functions)~~ ✅ DONE

2. **Update controller** to accept both snake_case and camelCase

3. **Write tests** for each migrated service

4. **Update API documentation** to reflect camelCase support

5. **Monitor performance** after deployment

---

## Support and Questions

For questions about this migration:
- Check existing patterns in `accounts.service.ts`
- Review TypeORM documentation: https://typeorm.io
- Check entity definitions in `/entities/` folder

---

## Changelog

### 2025-01-04
- Completed fiscalPeriods.service.ts migration to TypeORM
  - Replaced raw SQL with Repository pattern for FiscalYear and FiscalPeriod
  - Implemented QueryBuilder for complex queries with joins (fiscalYear, company)
  - Preserved database function calls for close/reopen operations using repository.query()
  - Preserved all critical business logic:
    - Date overlap validation using PostgreSQL OVERLAPS operator
    - Monthly period generation algorithm
    - Period statistics calculation
    - User name lookup for closedBy field
  - Manual snake_case to camelCase mapping for database function results
  - Converted DTOs to camelCase
  - Added comprehensive logging
- Completed payments.service.ts migration to TypeORM
  - Created PaymentInvoice entity for payment-invoice junction table
  - Replaced raw SQL with Repository pattern for Payment, PaymentInvoice, and Invoice
  - Used QueryRunner for transaction management (reconcile, cancel operations)
  - Used QueryBuilder for main queries, raw SQL for cross-schema joins (partners, currencies)
  - Preserved all critical business logic:
    - Payment reconciliation workflow with invoice validation
    - Invoice amount updates (amountPaid, amountResidual, status)
    - Partner validation (invoice must belong to same partner as payment)
    - Amount validation (reconciled amount cannot exceed payment or invoice residual)
    - Status transitions (draft -> posted -> reconciled -> cancelled)
    - Only draft payments can be modified/deleted
    - Reverse reconciliations on cancel
  - Converted DTOs to camelCase
  - Added comprehensive logging
- Completed invoices.service.ts migration to TypeORM
  - Replaced raw SQL with Repository pattern for Invoice and InvoiceLine
  - Used QueryBuilder for main queries, raw SQL for cross-schema joins
  - Preserved all critical business logic:
    - Tax calculation integration with taxesService
    - Invoice status workflow (draft -> open -> paid/cancelled)
    - Sequential number generation (INV-XXXXXX / BILL-XXXXXX)
    - Line management with automatic total recalculation
    - Payment tracking (amountPaid, amountResidual)
  - Converted DTOs to camelCase
  - Added comprehensive logging
- Completed journal-entries.service.ts migration to TypeORM
  - Replaced raw SQL with Repository pattern
  - Used QueryRunner for transaction management (create, update)
  - Implemented QueryBuilder for complex queries with joins
  - Preserved all accounting logic:
    - Double-entry balance validation (debits = credits)
    - Minimum 2 lines validation
    - Status transitions (draft -> posted -> cancelled)
    - Only draft entries can be modified/deleted
  - Converted DTOs to camelCase
- Completed journals.service.ts migration to TypeORM
  - Replaced raw SQL with Repository pattern
  - Implemented QueryBuilder for joins
  - Converted DTOs to camelCase
- Completed taxes.service.ts migration to TypeORM
  - Replaced raw SQL CRUD with Repository pattern
  - Used In() operator for batch tax lookups
  - Preserved calculateTaxes() and calculateDocumentTaxes() logic exactly
  - Converted DTOs to camelCase
- Updated MIGRATION_GUIDE.md with progress

### 2024-12-14
- Created all TypeORM entities
- Registered entities in AppDataSource
- Completed accounts.service.ts migration
- Created this migration guide