castle/MIGRATION_SQUASH_SUMMARY.md

# Castle Migration Squash Summary

**Date:** November 10, 2025  
**Action:** Squashed 16 incremental migrations into a single clean initial migration

## Overview

The Castle extension had accumulated 16 migrations (m001-m016) during development. Since the software has not been released yet, we safely squashed all migrations into a single clean `m001_initial` migration.

## Files Changed

- **migrations.py** - Replaced with squashed single migration (651 → 327 lines)
- **migrations_old.py.bak** - Backup of original 16 migrations for reference

## Final Database Schema

The squashed migration creates **7 tables**:

### 1. castle_accounts
- Core chart of accounts with hierarchical Beancount-style names
- Examples: "Assets:Bitcoin:Lightning", "Expenses:Food:Groceries"
- User-specific accounts: "Assets:Receivable:User-af983632"
- Includes comprehensive default account set (40+ accounts)

### 2. castle_extension_settings
- Castle-wide configuration
- Stores castle_wallet_id for Lightning payments

### 3. castle_user_wallet_settings
- Per-user wallet configuration
- Allows users to have separate wallet preferences

### 4. castle_manual_payment_requests
- User-submitted payment requests to Castle
- Reviewed by admins before processing
- Includes notes field for additional context

### 5. castle_balance_assertions
- Reconciliation and balance checking at specific dates
- Multi-currency support (satoshis + fiat)
- Tolerance checking for small discrepancies
- Includes notes field for reconciliation comments

### 6. castle_user_equity_status
- Manages equity contribution eligibility
- Equity-eligible users can convert expenses to equity
- Creates dynamic user-specific equity accounts: Equity:User-{user_id}

### 7. castle_account_permissions
- Granular access control for accounts
- Permission types: read, submit_expense, manage
- Supports hierarchical inheritance (parent permissions cascade)
- Time-based expiration support

## What Was Removed

The following tables were **intentionally NOT included** in the final schema (they were dropped in m016):

- **castle_journal_entries** - Journal entries now managed by Fava/Beancount (external source of truth)
- **castle_entry_lines** - Entry lines now managed by Fava/Beancount

Castle now uses Fava as the single source of truth for accounting data. Journal operations:
- **Write:** Submit to Fava via FavaClient.add_entry()
- **Read:** Query Fava via FavaClient.get_entries()

## Key Schema Decisions

1. **Hierarchical Account Names** - Beancount-style colon-separated hierarchy (e.g., "Assets:Bitcoin:Lightning")
2. **No Journal Tables** - Fava/Beancount is the source of truth for journal entries
3. **Dynamic User Accounts** - User-specific accounts created on-demand (Assets:Receivable:User-xxx, Equity:User-xxx)
4. **No Parent-Only Accounts** - Hierarchy is implicit in names (no "Assets:Bitcoin" parent account needed)
5. **Multi-Currency Support** - Balance assertions support both satoshis and fiat currencies
6. **Notes Fields** - Added notes to balance_assertions and manual_payment_requests for better documentation

## Migration History (Original 16 Migrations)

For reference, the original migration sequence (preserved in migrations_old.py.bak):

1. **m001** - Initial accounts, journal_entries, entry_lines tables
2. **m002** - Extension settings table
3. **m003** - User wallet settings table
4. **m004** - Manual payment requests table
5. **m005** - Added flag/meta columns to journal_entries
6. **m006** - Migrated to hierarchical account names
7. **m007** - Balance assertions table
8. **m008** - Renamed Lightning account (Assets:Lightning:Balance → Assets:Bitcoin:Lightning)
9. **m009** - Added OnChain Bitcoin account (Assets:Bitcoin:OnChain)
10. **m010** - User equity status table
11. **m011** - Account permissions table
12. **m012** - Updated default accounts with detailed hierarchy (40+ accounts)
13. **m013** - Removed parent-only accounts (Assets:Bitcoin, Equity)
14. **m014** - Removed legacy equity accounts (MemberEquity, RetainedEarnings)
15. **m015** - Converted entry_lines from debit/credit to single amount field
16. **m016** - Dropped journal_entries and entry_lines tables (Fava integration)

## Benefits of Squashing

1. **Cleaner Codebase** - Single 327-line migration vs 651 lines across 16 functions
2. **Easier to Understand** - New developers see final schema immediately
3. **Faster Fresh Installs** - One migration run instead of 16
4. **Better Documentation** - Comprehensive comments explain design decisions
5. **No Migration Artifacts** - No intermediate states, data conversions, or temporary columns

## Fresh Install Process

For new installations:

```bash
# Castle's migration system will run m001_initial automatically
# No manual intervention needed
```

The migration will:
1. Create all 7 tables with proper indexes and foreign keys
2. Insert 40+ default accounts with hierarchical names
3. Set up proper constraints and defaults
4. Complete in a single transaction

## Default Accounts Created

The migration automatically creates a comprehensive chart of accounts:

**Assets (12 accounts):**
- Assets:Bank
- Assets:Bitcoin:Lightning
- Assets:Bitcoin:OnChain
- Assets:Cash
- Assets:FixedAssets:Equipment
- Assets:FixedAssets:FarmEquipment
- Assets:FixedAssets:Network
- Assets:FixedAssets:ProductionFacility
- Assets:Inventory
- Assets:Livestock
- Assets:Receivable
- Assets:Tools

**Liabilities (1 account):**
- Liabilities:Payable

**Income (3 accounts):**
- Income:Accommodation:Guests
- Income:Service
- Income:Other

**Expenses (24 accounts):**
- Expenses:Administrative
- Expenses:Construction:Materials
- Expenses:Furniture
- Expenses:Garden
- Expenses:Gas:Kitchen
- Expenses:Gas:Vehicle
- Expenses:Groceries
- Expenses:Hardware
- Expenses:Housewares
- Expenses:Insurance
- Expenses:Kitchen
- Expenses:Maintenance:Car
- Expenses:Maintenance:Garden
- Expenses:Maintenance:Property
- Expenses:Membership
- Expenses:Supplies
- Expenses:Tools
- Expenses:Utilities:Electric
- Expenses:Utilities:Internet
- Expenses:WebHosting:Domain
- Expenses:WebHosting:Wix

**Equity:**
- Created dynamically as Equity:User-{user_id} when granting equity eligibility

## Testing

After squashing, verify the migration works:

```bash
# 1. Backup existing database (if any)
cp castle.sqlite3 castle.sqlite3.backup

# 2. Drop and recreate database to test fresh install
rm castle.sqlite3

# 3. Start LNbits - migration should run automatically
poetry run lnbits

# 4. Verify tables created
sqlite3 castle.sqlite3 ".tables"
# Should show: castle_accounts, castle_extension_settings, etc.

# 5. Verify default accounts
sqlite3 castle.sqlite3 "SELECT COUNT(*) FROM castle_accounts;"
# Should show: 40 (default accounts)
```

## Rollback Plan

If issues are discovered:

```bash
# Restore original migrations
cp migrations_old.py.bak migrations.py

# Restore database
cp castle.sqlite3.backup castle.sqlite3
```

## Notes

- This squash is safe because Castle has not been released yet
- No existing production databases need migration
- Historical migrations preserved in migrations_old.py.bak
- All functionality preserved in final schema
- No data loss concerns (no production data exists)

---

**Signed off by:** Claude Code  
**Reviewed by:** Human operator  
**Status:** Complete