diff --git a/CLAUDE.md b/CLAUDE.md
index a80a185..3086441 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -12,9 +12,11 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
**Double-Entry Accounting**: Every transaction affects at least two accounts. Debits must equal credits. Five account types: Assets, Liabilities, Equity, Revenue (Income), Expenses.
+**Fava/Beancount Backend**: Castle now uses [Fava](https://github.com/beancount/fava) as the primary accounting engine. Fava is a web interface for Beancount that provides a REST API for ledger operations. All accounting calculations (balance sheets, trial balances, account reports) are delegated to Fava/Beancount. Castle formats transactions as Beancount entries and submits them via Fava's API.
+
+**Required External Dependency**: Fava must be running as a separate service. Configure `fava_url` and `fava_ledger_slug` in Castle settings (default: `http://localhost:3333` with slug `castle-accounting`). Castle will not function without Fava.
+
**Pure Functional Core**: The `core/` directory contains pure accounting logic independent of the database layer:
-- `core/balance.py` - Balance calculation from journal entries
-- `core/inventory.py` - Multi-currency position tracking (similar to Beancount's Inventory)
- `core/validation.py` - Entry validation rules
**Account Hierarchy**: Beancount-style hierarchical naming with `:` separators:
@@ -23,7 +25,13 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
- `Liabilities:Payable:User-af983632`
- `Expenses:Food:Supplies`
-**Metadata System**: Each `entry_line` stores JSON metadata preserving original fiat amounts. Critical: fiat balances are calculated by summing `fiat_amount` from metadata, NOT by converting current satoshi balances. This prevents exchange rate fluctuations from affecting historical records.
+**Amount Format**: Recent architecture change uses string-based amounts with currency codes:
+- SATS amounts: `"200000 SATS"`
+- Fiat amounts: `"100.00 EUR"` or `"250.00 USD"`
+- Cost basis notation: `"200000 SATS {100.00 EUR}"` (200k sats acquired at 100 EUR)
+- Parsing handles both formats via `parse_amount_string()` in views_api.py
+
+**Metadata System**: Beancount metadata format stores original fiat amounts and exchange rates as key-value pairs. Critical: fiat balances are calculated by summing fiat amounts from journal entries, NOT by converting current satoshi balances. This prevents exchange rate fluctuations from affecting historical records.
### Key Files
@@ -33,31 +41,27 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
- `views.py` - Web interface routing
- `services.py` - Settings management layer
- `migrations.py` - Database schema migrations
-- `tasks.py` - Background tasks (daily reconciliation checks)
+- `tasks.py` - Background tasks (invoice payment monitoring)
- `account_utils.py` - Hierarchical account naming utilities
+- `fava_client.py` - HTTP client for Fava REST API (add_entry, query, balance_sheet)
+- `beancount_format.py` - Converts Castle entries to Beancount transaction format
+- `core/validation.py` - Pure validation functions for accounting rules
### Database Schema
-**accounts**: Chart of accounts with hierarchical names
-- `user_id` field for per-user accounts (Receivable, Payable, Equity)
-- Indexed on `user_id` and `account_type`
+**Note**: With Fava integration, Castle maintains a local cache of some data but delegates authoritative balance calculations to Beancount/Fava.
-**journal_entries**: Transaction headers
+**journal_entries**: Transaction headers stored locally and synced to Fava
- `flag` field: `*` (cleared), `!` (pending), `#` (flagged), `x` (void)
- `meta` field: JSON storing source, tags, audit info
- `reference` field: Links to payment_hash, invoice numbers, etc.
-
-**entry_lines**: Individual debit/credit lines
-- Always balanced (sum of debits = sum of credits per entry)
-- `metadata` field stores fiat currency info as JSON
-- Indexed on `journal_entry_id` and `account_id`
-
-**balance_assertions**: Reconciliation checkpoints (Beancount-style)
-- Assert expected balance at a date
-- Status: pending, passed, failed
-- Used for daily reconciliation checks
+- Enriched with `username` field when retrieved via API (added from LNbits user data)
**extension_settings**: Castle wallet configuration (admin-only)
+- `castle_wallet_id` - The LNbits wallet used for Castle operations
+- `fava_url` - Fava service URL (default: http://localhost:3333)
+- `fava_ledger_slug` - Ledger identifier in Fava (default: castle-accounting)
+- `fava_timeout` - API request timeout in seconds
**user_wallet_settings**: Per-user wallet configuration
@@ -96,16 +100,18 @@ DR Liabilities:Payable:User-af983632 39,669 sats
## Balance Calculation Logic
-**User Balance**:
+**User Balance** (calculated by Beancount via Fava):
- Positive = Castle owes user (LIABILITY accounts have credit balance)
- Negative = User owes Castle (ASSET accounts have debit balance)
-- Calculated from sum of all entry lines across user's accounts
-- Fiat balances summed from metadata, NOT converted from sats
+- Calculated by querying Fava for sum of all postings across user's accounts
+- Fiat balances calculated by Beancount from cost basis annotations, NOT converted from current sats
**Perspective-Based UI**:
- **User View**: Green = Castle owes them, Red = They owe Castle
- **Castle Admin View**: Green = User owes Castle, Red = Castle owes user
+**Balance Retrieval**: Use `GET /api/v1/balance` which queries Fava's balance sheet or account reports for accurate, Beancount-calculated balances.
+
## API Endpoints
### Accounts
@@ -169,34 +175,61 @@ Use `get_or_create_user_account()` in crud.py to ensure consistency.
### Currency Handling
-**CRITICAL**: Use `Decimal` for all fiat amounts, never `float`. Fiat amounts are stored in metadata as strings to preserve precision:
-```python
-from decimal import Decimal
+**CRITICAL**: Use `Decimal` for all fiat amounts, never `float`.
+**New Amount String Format** (recent architecture change):
+- Input format: `"100.00 EUR"` or `"200000 SATS"`
+- Cost basis format: `"200000 SATS {100.00 EUR}"` (for recording acquisition cost)
+- Parse using `parse_amount_string(amount_str)` in views_api.py
+- Returns tuple: `(amount: Decimal, currency: str, cost_basis: Optional[tuple])`
+
+**Beancount Metadata Format**:
+```python
+# Metadata attached to individual postings (legs of a transaction)
metadata = {
"fiat_currency": "EUR",
- "fiat_amount": str(Decimal("250.00")),
- "fiat_rate": str(Decimal("1074.192")),
- "btc_rate": str(Decimal("0.000931"))
+ "fiat_amount": "250.00", # String for precision
+ "fiat_rate": "1074.192", # Sats per fiat unit
}
```
-When reading: `fiat_amount = Decimal(metadata["fiat_amount"])`
+**Important**: When creating entries to submit to Fava, use `beancount_format.format_transaction()` to ensure proper Beancount syntax.
-### Balance Assertions for Reconciliation
+### Fava Integration Patterns
-Create balance assertions to verify accounting accuracy:
+**Adding a Transaction**:
```python
-await create_balance_assertion(
- account_id="lightning_account_id",
- expected_balance_sats=1000000,
- expected_balance_fiat=Decimal("500.00"),
- fiat_currency="EUR",
- tolerance_sats=100
+from .fava_client import get_fava_client
+from .beancount_format import format_transaction
+from datetime import date
+
+# Format as Beancount transaction
+entry = format_transaction(
+ date_val=date.today(),
+ flag="*",
+ narration="Groceries purchase",
+ postings=[
+ {"account": "Expenses:Food", "amount": "50000 SATS {46.50 EUR}"},
+ {"account": "Liabilities:Payable:User-abc123", "amount": "-50000 SATS"}
+ ],
+ tags=["groceries"],
+ links=["castle-entry-123"]
+)
+
+# Submit to Fava
+client = get_fava_client()
+result = await client.add_entry(entry)
+```
+
+**Querying Balances**:
+```python
+# Query user balance from Fava
+balance_result = await client.query(
+ f"SELECT sum(position) WHERE account ~ 'User-{user_id_short}'"
)
```
-Run `POST /api/v1/tasks/daily-reconciliation` to check all assertions.
+**Important**: Always use `sanitize_link()` from beancount_format.py when creating links to ensure Beancount compatibility (only A-Z, a-z, 0-9, -, _, /, . allowed).
### Permission Model
@@ -213,62 +246,134 @@ This extension follows LNbits extension structure:
- Templates in `templates/castle/`
- Database accessed via `db = Database("ext_castle")`
+**Startup Requirements**:
+- `castle_start()` initializes Fava client on extension load
+- Background task `wait_for_paid_invoices()` monitors Lightning invoice payments
+- Fava service MUST be running before starting LNbits with Castle extension
+
## Common Tasks
-### Add New Expense Account
+### Add New Account in Fava
```python
-await create_account(CreateAccount(
- name="Expenses:Internet",
- account_type=AccountType.EXPENSE,
- description="Internet service costs"
-))
+from .fava_client import get_fava_client
+from datetime import date
+
+# Create Open directive for new account
+client = get_fava_client()
+entry = {
+ "t": "Open",
+ "date": str(date.today()),
+ "account": "Expenses:Internet",
+ "currencies": ["SATS", "EUR"]
+}
+await client.add_entry(entry)
```
-### Manually Record Cash Payment
+### Record Transaction to Fava
```python
-await create_journal_entry(CreateJournalEntry(
- description="Cash payment for groceries",
- lines=[
- CreateEntryLine(account_id=expense_account_id, amount=50000), # Positive = debit (expense increase)
- CreateEntryLine(account_id=cash_account_id, amount=-50000) # Negative = credit (asset decrease)
+from .beancount_format import format_transaction
+
+entry = format_transaction(
+ date_val=date.today(),
+ flag="*",
+ narration="Internet bill payment",
+ postings=[
+ {"account": "Expenses:Internet", "amount": "50000 SATS {46.50 EUR}"},
+ {"account": "Assets:Lightning:Balance", "amount": "-50000 SATS"}
],
- flag=JournalEntryFlag.CLEARED,
- meta={"source": "manual", "payment_method": "cash"}
-))
+ tags=["utilities"],
+ links=["castle-tx-123"]
+)
+
+client = get_fava_client()
+await client.add_entry(entry)
```
-### Check User Balance
+### Query User Balance from Fava
```python
-balance = await get_user_balance(user_id)
-print(f"Sats: {balance.balance}") # Positive = Castle owes user
-print(f"Fiat: {balance.fiat_balances}") # {"EUR": Decimal("36.93")}
-```
+client = get_fava_client()
-### Export to Beancount (Future)
-Follow patterns in `docs/BEANCOUNT_PATTERNS.md` for implementing Beancount export. Use hierarchical account names and preserve metadata in Beancount comments.
+# Query all accounts for a user
+user_short = user_id[:8]
+query = f"SELECT account, sum(position) WHERE account ~ 'User-{user_short}' GROUP BY account"
+result = await client.query(query)
+
+# Parse result to calculate net balance
+# (sum of all user accounts across Assets, Liabilities, Equity)
+```
## Data Integrity
**Critical Invariants**:
-1. Every journal entry MUST have balanced debits and credits
-2. Fiat balances calculated from metadata, not from converting sats
+1. Every transaction submitted to Fava MUST have balanced debits and credits (Beancount enforces this)
+2. Fiat amounts tracked via cost basis notation: `"AMOUNT SATS {COST FIAT}"`
3. User accounts use `user_id` (NOT `wallet_id`) for consistency
-4. Balance assertions checked daily via background task
+4. All accounting calculations delegated to Beancount/Fava
**Validation** is performed in `core/validation.py`:
-- `validate_journal_entry()` - Checks balance, minimum lines
-- `validate_balance()` - Verifies account balance calculation
-- `validate_receivable_entry()` - Ensures receivable entries are valid
-- `validate_expense_entry()` - Ensures expense entries are valid
+- Pure validation functions for entry correctness before submitting to Fava
-## Known Issues & Future Work
+**Beancount String Sanitization**:
+- Links must match pattern: `[A-Za-z0-9\-_/.]`
+- Use `sanitize_link()` from beancount_format.py for all links and tags
-See `docs/DOCUMENTATION.md` for comprehensive list. Key items:
-- No journal entry editing/deletion (use reversing entries)
-- No date range filtering on list endpoints (hardcoded limit of 100)
-- No batch operations for bulk imports
-- Plugin system architecture designed but not implemented
-- Beancount export endpoint not yet implemented
+## Recent Architecture Changes
+
+**Migration to Fava/Beancount** (2025):
+- Removed local balance calculation logic (now handled by Beancount)
+- Removed local `accounts` and `entry_lines` tables (Fava is source of truth)
+- Added `fava_client.py` and `beancount_format.py` modules
+- Changed amount format to string-based with currency codes
+- Username enrichment added to journal entries for UI display
+
+**Key Breaking Changes**:
+- All balance queries now go through Fava API
+- Account creation must use Fava's Open directive
+- Transaction format must follow Beancount syntax
+- Cost basis notation required for multi-currency tracking
+
+## Development Setup
+
+### Prerequisites
+
+1. **LNbits**: This extension must be installed in the `lnbits/extensions/` directory
+2. **Fava Service**: Must be running before starting LNbits with Castle enabled
+ ```bash
+ # Install Fava
+ pip install fava
+
+ # Create a basic Beancount file
+ touch castle-ledger.beancount
+
+ # Start Fava (default: http://localhost:3333)
+ fava castle-ledger.beancount
+ ```
+3. **Configure Castle Settings**: Set `fava_url` and `fava_ledger_slug` via settings API or UI
+
+### Running Castle Extension
+
+Castle is loaded as part of LNbits. No separate build or test commands are needed for the extension itself. Development workflow:
+
+1. Modify code in `lnbits/extensions/castle/`
+2. Restart LNbits
+3. Extension hot-reloads are supported by LNbits in development mode
+
+### Testing Transactions
+
+Use the web UI or API endpoints to create test transactions. For API testing:
+
+```bash
+# Create expense (user owes Castle)
+curl -X POST http://localhost:5000/castle/api/v1/entries/expense \
+ -H "X-Api-Key: YOUR_INVOICE_KEY" \
+ -d '{"description": "Test expense", "amount": "100.00 EUR", "account_name": "Expenses:Test"}'
+
+# Check user balance
+curl http://localhost:5000/castle/api/v1/balance \
+ -H "X-Api-Key: YOUR_INVOICE_KEY"
+```
+
+**Debugging Fava Connection**: Check logs for "Fava client initialized" message on startup. If missing, verify Fava is running and settings are correct.
## Related Documentation
diff --git a/templates/castle/index.html b/templates/castle/index.html
index 6aff4f5..367f54e 100644
--- a/templates/castle/index.html
+++ b/templates/castle/index.html
@@ -81,8 +81,8 @@
{% raw %}{{ formatDate(entry.entry_date) }}{% endraw %}
-
- User: {% raw %}{{ getUserName(entry.meta.user_id) }}{% endraw %}
+
+ User: {% raw %}{{ entry.username }}{% endraw %}
Ref: {% raw %}{{ entry.reference }}{% endraw %}
diff --git a/views_api.py b/views_api.py
index a020cfa..548daac 100644
--- a/views_api.py
+++ b/views_api.py
@@ -464,8 +464,8 @@ async def api_get_user_entries(
reference = link_clean
break
- # Get username from user ID (first 8 chars for display)
- username = f"User-{user_id_match[:8]}" if user_id_match else None
+ # Look up actual username using helper function
+ username = await _get_username_from_user_id(user_id_match) if user_id_match else None
entry_data = {
"id": entry_id or e.get("entry_hash", "unknown"),
@@ -506,6 +506,107 @@ async def api_get_user_entries(
}
+async def _get_username_from_user_id(user_id: str) -> str:
+ """
+ Helper function to get username from user_id, handling various formats.
+
+ Supports:
+ - Full UUID with dashes (36 chars): "375ec158-686c-4a21-b44d-a51cc90ef07d"
+ - Dashless UUID (32 chars): "375ec158686c4a21b44da51cc90ef07d"
+ - Partial ID (8 chars from account names): "375ec158"
+
+ Returns username or formatted fallback.
+ """
+ from lnbits.core.crud.users import get_user
+
+ logger.info(f"[USERNAME] Called with: '{user_id}' (len={len(user_id) if user_id else 0})")
+
+ if not user_id:
+ return None
+
+ # Case 1: Already in standard UUID format (36 chars with dashes)
+ if len(user_id) == 36 and user_id.count('-') == 4:
+ logger.info(f"[USERNAME] Case 1: Full UUID format")
+ user = await get_user(user_id)
+ result = user.username if user and user.username else f"User-{user_id[:8]}"
+ logger.info(f"[USERNAME] Case 1 result: '{result}'")
+ return result
+
+ # Case 2: Dashless 32-char UUID - lookup via Castle user settings
+ elif len(user_id) == 32 and '-' not in user_id:
+ logger.info(f"[USERNAME] Case 2: Dashless UUID format - looking up in Castle user settings")
+ try:
+ # Get all Castle users (which have full user_ids)
+ user_settings = await get_all_user_wallet_settings()
+
+ # Convert dashless to dashed format for comparison
+ user_id_with_dashes = f"{user_id[0:8]}-{user_id[8:12]}-{user_id[12:16]}-{user_id[16:20]}-{user_id[20:32]}"
+ logger.info(f"[USERNAME] Converted to dashed format: {user_id_with_dashes}")
+
+ # Find matching user
+ for setting in user_settings:
+ if setting.id == user_id_with_dashes:
+ logger.info(f"[USERNAME] Found matching user in Castle settings")
+ # Get username from LNbits
+ user = await get_user(setting.id)
+ result = user.username if user and user.username else f"User-{user_id[:8]}"
+ logger.info(f"[USERNAME] Case 2 result (found): '{result}'")
+ return result
+
+ # No matching user found
+ logger.info(f"[USERNAME] No matching user found in Castle settings")
+ result = f"User-{user_id[:8]}"
+ logger.info(f"[USERNAME] Case 2 result (not found): '{result}'")
+ return result
+
+ except Exception as e:
+ logger.error(f"Error looking up user by dashless UUID {user_id}: {e}")
+ result = f"User-{user_id[:8]}"
+ return result
+
+ # Case 3: Partial ID (8 chars from account name) - lookup via Castle user settings
+ elif len(user_id) == 8:
+ logger.info(f"[USERNAME] Case 3: Partial ID format - looking up in Castle user settings")
+ try:
+ # Get all Castle users (which have full user_ids)
+ user_settings = await get_all_user_wallet_settings()
+
+ # Find matching user by first 8 chars
+ for setting in user_settings:
+ if setting.id.startswith(user_id):
+ logger.info(f"[USERNAME] Found full user_id: {setting.id}")
+ # Now get username from LNbits with full ID
+ user = await get_user(setting.id)
+ result = user.username if user and user.username else f"User-{user_id}"
+ logger.info(f"[USERNAME] Case 3 result (found): '{result}'")
+ return result
+
+ # No matching user found in Castle settings
+ logger.info(f"[USERNAME] No matching user found in Castle settings")
+ result = f"User-{user_id}"
+ logger.info(f"[USERNAME] Case 3 result (not found): '{result}'")
+ return result
+
+ except Exception as e:
+ logger.error(f"Error looking up user by partial ID {user_id}: {e}")
+ result = f"User-{user_id}"
+ return result
+
+ # Case 4: Unknown format - try as-is and fall back
+ else:
+ logger.info(f"[USERNAME] Case 4: Unknown format - trying as-is")
+ try:
+ user = await get_user(user_id)
+ result = user.username if user and user.username else f"User-{user_id[:8]}"
+ logger.info(f"[USERNAME] Case 4 result: '{result}'")
+ return result
+ except Exception as e:
+ logger.info(f"[USERNAME] Case 4 exception: {e}")
+ result = f"User-{user_id[:8]}"
+ logger.info(f"[USERNAME] Case 4 fallback result: '{result}'")
+ return result
+
+
@castle_api_router.get("/api/v1/entries/pending")
async def api_get_pending_entries(
wallet: WalletTypeInfo = Depends(require_admin_key),
@@ -516,7 +617,6 @@ async def api_get_pending_entries(
Returns transactions with flag='!' from Fava/Beancount.
"""
from lnbits.settings import settings as lnbits_settings
- from lnbits.core.crud.users import get_user
from .fava_client import get_fava_client
if wallet.wallet.user != lnbits_settings.super_user:
@@ -552,9 +652,13 @@ async def api_get_pending_entries(
# Extract user ID from metadata or account names
user_id = None
entry_meta = e.get("meta", {})
+ logger.info(f"[EXTRACT] Entry metadata keys: {list(entry_meta.keys())}")
+ logger.info(f"[EXTRACT] Entry metadata: {entry_meta}")
if "user-id" in entry_meta:
user_id = entry_meta["user-id"]
+ logger.info(f"[EXTRACT] Found user-id in metadata: {user_id}")
else:
+ logger.info(f"[EXTRACT] No user-id in metadata, checking account names")
# Try to extract from account names in postings
for posting in e.get("postings", []):
account = posting.get("account", "")
@@ -563,13 +667,11 @@ async def api_get_pending_entries(
parts = account.split("User-")
if len(parts) > 1:
user_id = parts[1] # Short ID after User-
+ logger.info(f"[EXTRACT] Extracted user_id from account name: {user_id}")
break
- # Look up username
- username = None
- if user_id:
- user = await get_user(user_id)
- username = user.username if user and user.username else f"User-{user_id[:8]}"
+ # Look up username using helper function
+ username = await _get_username_from_user_id(user_id) if user_id else None
# Extract amount from postings (sum of absolute values / 2)
amount_sats = 0
@@ -1243,17 +1345,15 @@ async def api_get_all_balances(
wallet: WalletTypeInfo = Depends(require_admin_key),
) -> list[dict]:
"""Get all user balances (admin/super user only) from Fava/Beancount"""
- from lnbits.core.crud.users import get_user
from .fava_client import get_fava_client
fava = get_fava_client()
balances = await fava.get_all_user_balances()
- # Enrich with username information
+ # Enrich with username information using helper function
result = []
for balance in balances:
- user = await get_user(balance["user_id"])
- username = user.username if user and user.username else balance["user_id"][:16] + "..."
+ username = await _get_username_from_user_id(balance["user_id"])
result.append({
"user_id": balance["user_id"],