castle/CLAUDE.md

# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

Castle Accounting is a double-entry bookkeeping extension for LNbits that enables collectives (co-living spaces, makerspaces, community projects) to track finances with proper accounting principles. It integrates Lightning Network payments with traditional accounting, supporting both cryptocurrency and fiat currency tracking.

## Architecture

### Core Design Principles

**Double-Entry Accounting**: Every transaction affects at least two accounts. Debits must equal credits. Five account types: Assets, Liabilities, Equity, Revenue (Income), Expenses.

**Fava/Beancount Backend**: Castle now uses [Fava](https://github.com/beancount/fava) as the primary accounting engine. Fava is a web interface for Beancount that provides a REST API for ledger operations. All accounting calculations (balance sheets, trial balances, account reports) are delegated to Fava/Beancount. Castle formats transactions as Beancount entries and submits them via Fava's API.

**Required External Dependency**: Fava must be running as a separate service. Configure `fava_url` and `fava_ledger_slug` in Castle settings (default: `http://localhost:3333` with slug `castle-accounting`). Castle will not function without Fava.

**Pure Functional Core**: The `core/` directory contains pure accounting logic independent of the database layer:
- `core/validation.py` - Entry validation rules

**Account Hierarchy**: Beancount-style hierarchical naming with `:` separators:
- `Assets:Lightning:Balance`
- `Assets:Receivable:User-af983632`
- `Liabilities:Payable:User-af983632`
- `Expenses:Food:Supplies`

**Amount Format**: Recent architecture change uses string-based amounts with currency codes:
- SATS amounts: `"200000 SATS"`
- Fiat amounts: `"100.00 EUR"` or `"250.00 USD"`
- Cost basis notation: `"200000 SATS {100.00 EUR}"` (200k sats acquired at 100 EUR)
- Parsing handles both formats via `parse_amount_string()` in views_api.py

**Metadata System**: Beancount metadata format stores original fiat amounts and exchange rates as key-value pairs. Critical: fiat balances are calculated by summing fiat amounts from journal entries, NOT by converting current satoshi balances. This prevents exchange rate fluctuations from affecting historical records.

### Key Files

- `models.py` - Pydantic models for API I/O and data structures
- `crud.py` - Database operations (create/read/update accounts, journal entries)
- `views_api.py` - FastAPI endpoints for all operations
- `views.py` - Web interface routing
- `services.py` - Settings management layer
- `migrations.py` - Database schema migrations
- `tasks.py` - Background tasks (invoice payment monitoring)
- `account_utils.py` - Hierarchical account naming utilities
- `fava_client.py` - HTTP client for Fava REST API (add_entry, query, balance_sheet)
- `beancount_format.py` - Converts Castle entries to Beancount transaction format
- `core/validation.py` - Pure validation functions for accounting rules

### Database Schema

**Note**: With Fava integration, Castle maintains a local cache of some data but delegates authoritative balance calculations to Beancount/Fava.

**journal_entries**: Transaction headers stored locally and synced to Fava
- `flag` field: `*` (cleared), `!` (pending), `#` (flagged), `x` (void)
- `meta` field: JSON storing source, tags, audit info
- `reference` field: Links to payment_hash, invoice numbers, etc.
- Enriched with `username` field when retrieved via API (added from LNbits user data)

**extension_settings**: Castle wallet configuration (admin-only)
- `castle_wallet_id` - The LNbits wallet used for Castle operations
- `fava_url` - Fava service URL (default: http://localhost:3333)
- `fava_ledger_slug` - Ledger identifier in Fava (default: castle-accounting)
- `fava_timeout` - API request timeout in seconds

**user_wallet_settings**: Per-user wallet configuration

**manual_payment_requests**: User requests for cash/manual payments

## Transaction Flows

### User Adds Expense (Liability)
User pays cash for groceries, Castle owes them:
```
DR Expenses:Food              39,669 sats
   CR Liabilities:Payable:User-af983632    39,669 sats
```
Metadata preserves: `{"fiat_currency": "EUR", "fiat_amount": "36.93", "fiat_rate": "1074.192"}`

### Castle Adds Receivable
User owes Castle for accommodation:
```
DR Assets:Receivable:User-af983632    268,548 sats
   CR Income:Accommodation                    268,548 sats
```

### User Pays with Lightning
Invoice generated on **Castle's wallet** (not user's). After payment:
```
DR Assets:Lightning:Balance           268,548 sats
   CR Assets:Receivable:User-af983632         268,548 sats
```

### Manual Payment Approval
User requests cash payment → Admin approves → Journal entry created:
```
DR Liabilities:Payable:User-af983632   39,669 sats
   CR Assets:Lightning:Balance                39,669 sats
```

## Balance Calculation Logic

**User Balance** (calculated by Beancount via Fava):
- Positive = Castle owes user (LIABILITY accounts have credit balance)
- Negative = User owes Castle (ASSET accounts have debit balance)
- Calculated by querying Fava for sum of all postings across user's accounts
- Fiat balances calculated by Beancount from cost basis annotations, NOT converted from current sats

**Perspective-Based UI**:
- **User View**: Green = Castle owes them, Red = They owe Castle
- **Castle Admin View**: Green = User owes Castle, Red = Castle owes user

**Balance Retrieval**: Use `GET /api/v1/balance` which queries Fava's balance sheet or account reports for accurate, Beancount-calculated balances.

## API Endpoints

### Accounts
- `GET /api/v1/accounts` - List all accounts
- `POST /api/v1/accounts` - Create account (admin)
- `GET /api/v1/accounts/{id}/balance` - Get account balance

### Journal Entries
- `POST /api/v1/entries/expense` - User adds expense (creates liability or equity)
- `POST /api/v1/entries/receivable` - Admin records what user owes (admin only)
- `POST /api/v1/entries/revenue` - Admin records direct revenue (admin only)
- `GET /api/v1/entries/user` - Get user's journal entries
- `POST /api/v1/entries` - Create raw journal entry (admin only)

### Payments & Balances
- `GET /api/v1/balance` - Get user balance (or Castle total if super user)
- `GET /api/v1/balances/all` - Get all user balances (admin, enriched with usernames)
- `POST /api/v1/generate-payment-invoice` - Generate invoice for user to pay Castle
- `POST /api/v1/record-payment` - Record Lightning payment from user to Castle
- `POST /api/v1/settle-receivable` - Manually settle receivable (cash/bank)
- `POST /api/v1/pay-user` - Castle pays user (cash/bank/lightning)

### Manual Payment Requests
- `POST /api/v1/manual-payment-requests` - User requests payment
- `GET /api/v1/manual-payment-requests` - User's requests
- `GET /api/v1/manual-payment-requests/all` - All requests (admin)
- `POST /api/v1/manual-payment-requests/{id}/approve` - Approve (admin)
- `POST /api/v1/manual-payment-requests/{id}/reject` - Reject (admin)

### Reconciliation
- `POST /api/v1/assertions/balance` - Create balance assertion
- `GET /api/v1/assertions/balance` - List balance assertions
- `POST /api/v1/assertions/balance/{id}/check` - Check assertion
- `POST /api/v1/tasks/daily-reconciliation` - Run daily reconciliation (admin)

### Settings
- `GET /api/v1/settings` - Get Castle settings (super user)
- `PUT /api/v1/settings` - Update Castle settings (super user)
- `GET /api/v1/user/wallet` - Get user wallet settings
- `PUT /api/v1/user/wallet` - Update user wallet settings

## Development Notes

### Testing Entry Creation

When creating journal entries programmatically, use the helper endpoints:
- `POST /api/v1/entries/expense` for user expenses (handles account creation automatically)
- `POST /api/v1/entries/receivable` for what users owe
- `POST /api/v1/entries/revenue` for direct revenue

For custom entries, use `POST /api/v1/entries` with properly balanced lines.

### User Account Management

User-specific accounts are created automatically with format:
- Assets: `Assets:Receivable:User-{user_id[:8]}`
- Liabilities: `Liabilities:Payable:User-{user_id[:8]}`
- Equity: `Equity:MemberEquity:User-{user_id[:8]}`

Use `get_or_create_user_account()` in crud.py to ensure consistency.

### Currency Handling

**CRITICAL**: Use `Decimal` for all fiat amounts, never `float`.

**New Amount String Format** (recent architecture change):
- Input format: `"100.00 EUR"` or `"200000 SATS"`
- Cost basis format: `"200000 SATS {100.00 EUR}"` (for recording acquisition cost)
- Parse using `parse_amount_string(amount_str)` in views_api.py
- Returns tuple: `(amount: Decimal, currency: str, cost_basis: Optional[tuple])`

**Beancount Metadata Format**:
```python
# Metadata attached to individual postings (legs of a transaction)
metadata = {
    "fiat_currency": "EUR",
    "fiat_amount": "250.00",  # String for precision
    "fiat_rate": "1074.192",   # Sats per fiat unit
}
```

**Important**: When creating entries to submit to Fava, use `beancount_format.format_transaction()` to ensure proper Beancount syntax.

### Fava Integration Patterns

**Adding a Transaction**:
```python
from .fava_client import get_fava_client
from .beancount_format import format_transaction
from datetime import date

# Format as Beancount transaction
entry = format_transaction(
    date_val=date.today(),
    flag="*",
    narration="Groceries purchase",
    postings=[
        {"account": "Expenses:Food", "amount": "50000 SATS {46.50 EUR}"},
        {"account": "Liabilities:Payable:User-abc123", "amount": "-50000 SATS"}
    ],
    tags=["groceries"],
    links=["castle-entry-123"]
)

# Submit to Fava
client = get_fava_client()
result = await client.add_entry(entry)
```

**Querying Balances**:
```python
# Query user balance from Fava
balance_result = await client.query(
    f"SELECT sum(position) WHERE account ~ 'User-{user_id_short}'"
)
```

**Important**: Always use `sanitize_link()` from beancount_format.py when creating links to ensure Beancount compatibility (only A-Z, a-z, 0-9, -, _, /, . allowed).

### Permission Model

- **Super User**: Full access (check via `wallet.wallet.user == lnbits_settings.super_user`)
- **Admin Key**: Required for creating receivables, approving payments, viewing all balances
- **Invoice Key**: Read access to user's own data
- **Users**: Can only see/manage their own accounts and transactions

### Extension as LNbits Module

This extension follows LNbits extension structure:
- Registered via `castle_ext` router in `__init__.py`
- Static files served from `static/` directory
- Templates in `templates/castle/`
- Database accessed via `db = Database("ext_castle")`

**Startup Requirements**:
- `castle_start()` initializes Fava client on extension load
- Background task `wait_for_paid_invoices()` monitors Lightning invoice payments
- Fava service MUST be running before starting LNbits with Castle extension

## Common Tasks

### Add New Account in Fava
```python
from .fava_client import get_fava_client
from datetime import date

# Create Open directive for new account
client = get_fava_client()
entry = {
    "t": "Open",
    "date": str(date.today()),
    "account": "Expenses:Internet",
    "currencies": ["SATS", "EUR"]
}
await client.add_entry(entry)
```

### Record Transaction to Fava
```python
from .beancount_format import format_transaction

entry = format_transaction(
    date_val=date.today(),
    flag="*",
    narration="Internet bill payment",
    postings=[
        {"account": "Expenses:Internet", "amount": "50000 SATS {46.50 EUR}"},
        {"account": "Assets:Lightning:Balance", "amount": "-50000 SATS"}
    ],
    tags=["utilities"],
    links=["castle-tx-123"]
)

client = get_fava_client()
await client.add_entry(entry)
```

### Query User Balance from Fava
```python
client = get_fava_client()

# Query all accounts for a user
user_short = user_id[:8]
query = f"SELECT account, sum(position) WHERE account ~ 'User-{user_short}' GROUP BY account"
result = await client.query(query)

# Parse result to calculate net balance
# (sum of all user accounts across Assets, Liabilities, Equity)
```

## Data Integrity

**Critical Invariants**:
1. Every transaction submitted to Fava MUST have balanced debits and credits (Beancount enforces this)
2. Fiat amounts tracked via cost basis notation: `"AMOUNT SATS {COST FIAT}"`
3. User accounts use `user_id` (NOT `wallet_id`) for consistency
4. All accounting calculations delegated to Beancount/Fava

**Validation** is performed in `core/validation.py`:
- Pure validation functions for entry correctness before submitting to Fava

**Beancount String Sanitization**:
- Links must match pattern: `[A-Za-z0-9\-_/.]`
- Use `sanitize_link()` from beancount_format.py for all links and tags

## Recent Architecture Changes

**Migration to Fava/Beancount** (2025):
- Removed local balance calculation logic (now handled by Beancount)
- Removed local `accounts` and `entry_lines` tables (Fava is source of truth)
- Added `fava_client.py` and `beancount_format.py` modules
- Changed amount format to string-based with currency codes
- Username enrichment added to journal entries for UI display

**Key Breaking Changes**:
- All balance queries now go through Fava API
- Account creation must use Fava's Open directive
- Transaction format must follow Beancount syntax
- Cost basis notation required for multi-currency tracking

## Development Setup

### Prerequisites

1. **LNbits**: This extension must be installed in the `lnbits/extensions/` directory
2. **Fava Service**: Must be running before starting LNbits with Castle enabled
   ```bash
   # Install Fava
   pip install fava

   # Create a basic Beancount file
   touch castle-ledger.beancount

   # Start Fava (default: http://localhost:3333)
   fava castle-ledger.beancount
   ```
3. **Configure Castle Settings**: Set `fava_url` and `fava_ledger_slug` via settings API or UI

### Running Castle Extension

Castle is loaded as part of LNbits. No separate build or test commands are needed for the extension itself. Development workflow:

1. Modify code in `lnbits/extensions/castle/`
2. Restart LNbits
3. Extension hot-reloads are supported by LNbits in development mode

### Testing Transactions

Use the web UI or API endpoints to create test transactions. For API testing:

```bash
# Create expense (user owes Castle)
curl -X POST http://localhost:5000/castle/api/v1/entries/expense \
  -H "X-Api-Key: YOUR_INVOICE_KEY" \
  -d '{"description": "Test expense", "amount": "100.00 EUR", "account_name": "Expenses:Test"}'

# Check user balance
curl http://localhost:5000/castle/api/v1/balance \
  -H "X-Api-Key: YOUR_INVOICE_KEY"
```

**Debugging Fava Connection**: Check logs for "Fava client initialized" message on startup. If missing, verify Fava is running and settings are correct.

## Related Documentation

- `docs/README.md` - User-facing overview
- `docs/DOCUMENTATION.md` - Comprehensive technical documentation
- `docs/BEANCOUNT_PATTERNS.md` - Beancount-inspired design patterns
- `docs/PHASE1_COMPLETE.md`, `PHASE2_COMPLETE.md`, `PHASE3_COMPLETE.md` - Development milestones
- `docs/EXPENSE_APPROVAL.md` - Manual payment request workflow
- `docs/DAILY_RECONCILIATION.md` - Automated reconciliation system