lnbits-exts/castle
1
0
Fork 0
Code Issues 2 Pull requests Projects Releases 2 Packages Wiki Activity Actions

update CLAUDE.md

Browse source
This commit is contained in:
padreug 2025-11-10 16:35:03 +01:00
parent 87a3505376
commit 1b1d066d07
3 changed files with 292 additions and 87 deletions
Download patch file Download diff file Expand all files Collapse all files

251
CLAUDE.md
View file

@ -12,9 +12,11 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
**Double-Entry Accounting**: Every transaction affects at least two accounts. Debits must equal credits. Five account types: Assets, Liabilities, Equity, Revenue (Income), Expenses.
**Fava/Beancount Backend**: Castle now uses [Fava](https://github.com/beancount/fava) as the primary accounting engine. Fava is a web interface for Beancount that provides a REST API for ledger operations. All accounting calculations (balance sheets, trial balances, account reports) are delegated to Fava/Beancount. Castle formats transactions as Beancount entries and submits them via Fava's API.
**Required External Dependency**: Fava must be running as a separate service. Configure `fava_url` and `fava_ledger_slug` in Castle settings (default: `http://localhost:3333` with slug `castle-accounting`). Castle will not function without Fava.
**Pure Functional Core**: The `core/` directory contains pure accounting logic independent of the database layer:
- `core/balance.py` - Balance calculation from journal entries
- `core/inventory.py` - Multi-currency position tracking (similar to Beancount's Inventory)
- `core/validation.py` - Entry validation rules
**Account Hierarchy**: Beancount-style hierarchical naming with `:` separators:
@ -23,7 +25,13 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
- `Liabilities:Payable:User-af983632`
- `Expenses:Food:Supplies`
**Metadata System**: Each `entry_line` stores JSON metadata preserving original fiat amounts. Critical: fiat balances are calculated by summing `fiat_amount` from metadata, NOT by converting current satoshi balances. This prevents exchange rate fluctuations from affecting historical records.
**Amount Format**: Recent architecture change uses string-based amounts with currency codes:
- SATS amounts: `"200000 SATS"`
- Fiat amounts: `"100.00 EUR"` or `"250.00 USD"`
- Cost basis notation: `"200000 SATS {100.00 EUR}"` (200k sats acquired at 100 EUR)
- Parsing handles both formats via `parse_amount_string()` in views_api.py
**Metadata System**: Beancount metadata format stores original fiat amounts and exchange rates as key-value pairs. Critical: fiat balances are calculated by summing fiat amounts from journal entries, NOT by converting current satoshi balances. This prevents exchange rate fluctuations from affecting historical records.
### Key Files
@ -33,31 +41,27 @@ Castle Accounting is a double-entry bookkeeping extension for LNbits that enable
- `views.py` - Web interface routing
- `services.py` - Settings management layer
- `migrations.py` - Database schema migrations
- `tasks.py` - Background tasks (daily reconciliation checks)
- `tasks.py` - Background tasks (invoice payment monitoring)
- `account_utils.py` - Hierarchical account naming utilities
- `fava_client.py` - HTTP client for Fava REST API (add_entry, query, balance_sheet)
- `beancount_format.py` - Converts Castle entries to Beancount transaction format
- `core/validation.py` - Pure validation functions for accounting rules
### Database Schema
**accounts**: Chart of accounts with hierarchical names
- `user_id` field for per-user accounts (Receivable, Payable, Equity)
- Indexed on `user_id` and `account_type`
**Note**: With Fava integration, Castle maintains a local cache of some data but delegates authoritative balance calculations to Beancount/Fava.
**journal_entries**: Transaction headers
**journal_entries**: Transaction headers stored locally and synced to Fava
- `flag` field: `*` (cleared), `!` (pending), `#` (flagged), `x` (void)
- `meta` field: JSON storing source, tags, audit info
- `reference` field: Links to payment_hash, invoice numbers, etc.
**entry_lines**: Individual debit/credit lines
- Always balanced (sum of debits = sum of credits per entry)
- `metadata` field stores fiat currency info as JSON
- Indexed on `journal_entry_id` and `account_id`
**balance_assertions**: Reconciliation checkpoints (Beancount-style)
- Assert expected balance at a date
- Status: pending, passed, failed
- Used for daily reconciliation checks
- Enriched with `username` field when retrieved via API (added from LNbits user data)
**extension_settings**: Castle wallet configuration (admin-only)
- `castle_wallet_id` - The LNbits wallet used for Castle operations
- `fava_url` - Fava service URL (default: http://localhost:3333)
- `fava_ledger_slug` - Ledger identifier in Fava (default: castle-accounting)
- `fava_timeout` - API request timeout in seconds
**user_wallet_settings**: Per-user wallet configuration
@ -96,16 +100,18 @@ DR Liabilities:Payable:User-af983632 39,669 sats
## Balance Calculation Logic
**User Balance**:
**User Balance** (calculated by Beancount via Fava):
- Positive = Castle owes user (LIABILITY accounts have credit balance)
- Negative = User owes Castle (ASSET accounts have debit balance)
- Calculated from sum of all entry lines across user's accounts
- Fiat balances summed from metadata, NOT converted from sats
- Calculated by querying Fava for sum of all postings across user's accounts
- Fiat balances calculated by Beancount from cost basis annotations, NOT converted from current sats
**Perspective-Based UI**:
- **User View**: Green = Castle owes them, Red = They owe Castle
- **Castle Admin View**: Green = User owes Castle, Red = Castle owes user
**Balance Retrieval**: Use `GET /api/v1/balance` which queries Fava's balance sheet or account reports for accurate, Beancount-calculated balances.
## API Endpoints
### Accounts
@ -169,34 +175,61 @@ Use `get_or_create_user_account()` in crud.py to ensure consistency.
### Currency Handling
**CRITICAL**: Use `Decimal` for all fiat amounts, never `float`. Fiat amounts are stored in metadata as strings to preserve precision:
```python
from decimal import Decimal
**CRITICAL**: Use `Decimal` for all fiat amounts, never `float`.
**New Amount String Format** (recent architecture change):
- Input format: `"100.00 EUR"` or `"200000 SATS"`
- Cost basis format: `"200000 SATS {100.00 EUR}"` (for recording acquisition cost)
- Parse using `parse_amount_string(amount_str)` in views_api.py
- Returns tuple: `(amount: Decimal, currency: str, cost_basis: Optional[tuple])`
**Beancount Metadata Format**:
```python
# Metadata attached to individual postings (legs of a transaction)
metadata = {
"fiat_currency": "EUR",
"fiat_amount": str(Decimal("250.00")),
"fiat_rate": str(Decimal("1074.192")),
"btc_rate": str(Decimal("0.000931"))
"fiat_amount": "250.00", # String for precision
"fiat_rate": "1074.192", # Sats per fiat unit
}
```
When reading: `fiat_amount = Decimal(metadata["fiat_amount"])`
**Important**: When creating entries to submit to Fava, use `beancount_format.format_transaction()` to ensure proper Beancount syntax.
### Balance Assertions for Reconciliation
### Fava Integration Patterns
Create balance assertions to verify accounting accuracy:
**Adding a Transaction**:
```python
await create_balance_assertion(
account_id="lightning_account_id",
expected_balance_sats=1000000,
expected_balance_fiat=Decimal("500.00"),
fiat_currency="EUR",
tolerance_sats=100
from .fava_client import get_fava_client
from .beancount_format import format_transaction
from datetime import date
# Format as Beancount transaction
entry = format_transaction(
date_val=date.today(),
flag="*",
narration="Groceries purchase",
postings=[
{"account": "Expenses:Food", "amount": "50000 SATS {46.50 EUR}"},
{"account": "Liabilities:Payable:User-abc123", "amount": "-50000 SATS"}
],
tags=["groceries"],
links=["castle-entry-123"]
)
# Submit to Fava
client = get_fava_client()
result = await client.add_entry(entry)
```
**Querying Balances**:
```python
# Query user balance from Fava
balance_result = await client.query(
f"SELECT sum(position) WHERE account ~ 'User-{user_id_short}'"
)
```
Run `POST /api/v1/tasks/daily-reconciliation` to check all assertions.
**Important**: Always use `sanitize_link()` from beancount_format.py when creating links to ensure Beancount compatibility (only A-Z, a-z, 0-9, -, _, /, . allowed).
### Permission Model
@ -213,62 +246,134 @@ This extension follows LNbits extension structure:
- Templates in `templates/castle/`
- Database accessed via `db = Database("ext_castle")`
**Startup Requirements**:
- `castle_start()` initializes Fava client on extension load
- Background task `wait_for_paid_invoices()` monitors Lightning invoice payments
- Fava service MUST be running before starting LNbits with Castle extension
## Common Tasks
### Add New Expense Account
### Add New Account in Fava
```python
await create_account(CreateAccount(
name="Expenses:Internet",
account_type=AccountType.EXPENSE,
description="Internet service costs"
))
from .fava_client import get_fava_client
from datetime import date
# Create Open directive for new account
client = get_fava_client()
entry = {
"t": "Open",
"date": str(date.today()),
"account": "Expenses:Internet",
"currencies": ["SATS", "EUR"]
}
await client.add_entry(entry)
```
### Manually Record Cash Payment
### Record Transaction to Fava
```python
await create_journal_entry(CreateJournalEntry(
description="Cash payment for groceries",
lines=[
CreateEntryLine(account_id=expense_account_id, amount=50000), # Positive = debit (expense increase)
CreateEntryLine(account_id=cash_account_id, amount=-50000) # Negative = credit (asset decrease)
from .beancount_format import format_transaction
entry = format_transaction(
date_val=date.today(),
flag="*",
narration="Internet bill payment",
postings=[
{"account": "Expenses:Internet", "amount": "50000 SATS {46.50 EUR}"},
{"account": "Assets:Lightning:Balance", "amount": "-50000 SATS"}
],
flag=JournalEntryFlag.CLEARED,
meta={"source": "manual", "payment_method": "cash"}
))
tags=["utilities"],
links=["castle-tx-123"]
)
client = get_fava_client()
await client.add_entry(entry)
```
### Check User Balance
### Query User Balance from Fava
```python
balance = await get_user_balance(user_id)
print(f"Sats: {balance.balance}") # Positive = Castle owes user
print(f"Fiat: {balance.fiat_balances}") # {"EUR": Decimal("36.93")}
```
client = get_fava_client()
### Export to Beancount (Future)
Follow patterns in `docs/BEANCOUNT_PATTERNS.md` for implementing Beancount export. Use hierarchical account names and preserve metadata in Beancount comments.
# Query all accounts for a user
user_short = user_id[:8]
query = f"SELECT account, sum(position) WHERE account ~ 'User-{user_short}' GROUP BY account"
result = await client.query(query)
# Parse result to calculate net balance
# (sum of all user accounts across Assets, Liabilities, Equity)
```
## Data Integrity
**Critical Invariants**:
1. Every journal entry MUST have balanced debits and credits
2. Fiat balances calculated from metadata, not from converting sats
1. Every transaction submitted to Fava MUST have balanced debits and credits (Beancount enforces this)
2. Fiat amounts tracked via cost basis notation: `"AMOUNT SATS {COST FIAT}"`
3. User accounts use `user_id` (NOT `wallet_id`) for consistency
4. Balance assertions checked daily via background task
4. All accounting calculations delegated to Beancount/Fava
**Validation** is performed in `core/validation.py`:
- `validate_journal_entry()` - Checks balance, minimum lines
- `validate_balance()` - Verifies account balance calculation
- `validate_receivable_entry()` - Ensures receivable entries are valid
- `validate_expense_entry()` - Ensures expense entries are valid
- Pure validation functions for entry correctness before submitting to Fava
## Known Issues & Future Work
**Beancount String Sanitization**:
- Links must match pattern: `[A-Za-z0-9\-_/.]`
- Use `sanitize_link()` from beancount_format.py for all links and tags
See `docs/DOCUMENTATION.md` for comprehensive list. Key items:
- No journal entry editing/deletion (use reversing entries)
- No date range filtering on list endpoints (hardcoded limit of 100)
- No batch operations for bulk imports
- Plugin system architecture designed but not implemented
- Beancount export endpoint not yet implemented
## Recent Architecture Changes
**Migration to Fava/Beancount** (2025):
- Removed local balance calculation logic (now handled by Beancount)
- Removed local `accounts` and `entry_lines` tables (Fava is source of truth)
- Added `fava_client.py` and `beancount_format.py` modules
- Changed amount format to string-based with currency codes
- Username enrichment added to journal entries for UI display
**Key Breaking Changes**:
- All balance queries now go through Fava API
- Account creation must use Fava's Open directive
- Transaction format must follow Beancount syntax
- Cost basis notation required for multi-currency tracking
## Development Setup
### Prerequisites
1. **LNbits**: This extension must be installed in the `lnbits/extensions/` directory
2. **Fava Service**: Must be running before starting LNbits with Castle enabled
```bash
# Install Fava
pip install fava
# Create a basic Beancount file
touch castle-ledger.beancount
# Start Fava (default: http://localhost:3333)
fava castle-ledger.beancount
```
3. **Configure Castle Settings**: Set `fava_url` and `fava_ledger_slug` via settings API or UI
### Running Castle Extension
Castle is loaded as part of LNbits. No separate build or test commands are needed for the extension itself. Development workflow:
1. Modify code in `lnbits/extensions/castle/`
2. Restart LNbits
3. Extension hot-reloads are supported by LNbits in development mode
### Testing Transactions
Use the web UI or API endpoints to create test transactions. For API testing:
```bash
# Create expense (user owes Castle)
curl -X POST http://localhost:5000/castle/api/v1/entries/expense \
-H "X-Api-Key: YOUR_INVOICE_KEY" \
-d '{"description": "Test expense", "amount": "100.00 EUR", "account_name": "Expenses:Test"}'
# Check user balance
curl http://localhost:5000/castle/api/v1/balance \
-H "X-Api-Key: YOUR_INVOICE_KEY"
```
**Debugging Fava Connection**: Check logs for "Fava client initialized" message on startup. If missing, verify Fava is running and settings are correct.
## Related Documentation

4
templates/castle/index.html
View file

@ -81,8 +81,8 @@
<q-item-label caption>
{% raw %}{{ formatDate(entry.entry_date) }}{% endraw %}
</q-item-label>
<q-item-label caption v-if="entry.meta && entry.meta.user_id">
User: {% raw %}{{ getUserName(entry.meta.user_id) }}{% endraw %}
<q-item-label caption v-if="entry.username">
User: {% raw %}{{ entry.username }}{% endraw %}
</q-item-label>
<q-item-label caption v-if="entry.reference" class="text-grey">
Ref: {% raw %}{{ entry.reference }}{% endraw %}

124
views_api.py
View file

@ -464,8 +464,8 @@ async def api_get_user_entries(
reference = link_clean
break
# Get username from user ID (first 8 chars for display)
username = f"User-{user_id_match[:8]}" if user_id_match else None
# Look up actual username using helper function
username = await _get_username_from_user_id(user_id_match) if user_id_match else None
entry_data = {
"id": entry_id or e.get("entry_hash", "unknown"),
@ -506,6 +506,107 @@ async def api_get_user_entries(
}
async def _get_username_from_user_id(user_id: str) -> str:
"""
Helper function to get username from user_id, handling various formats.
Supports:
- Full UUID with dashes (36 chars): "375ec158-686c-4a21-b44d-a51cc90ef07d"
- Dashless UUID (32 chars): "375ec158686c4a21b44da51cc90ef07d"
- Partial ID (8 chars from account names): "375ec158"
Returns username or formatted fallback.
"""
from lnbits.core.crud.users import get_user
logger.info(f"[USERNAME] Called with: '{user_id}' (len={len(user_id) if user_id else 0})")
if not user_id:
return None
# Case 1: Already in standard UUID format (36 chars with dashes)
if len(user_id) == 36 and user_id.count('-') == 4:
logger.info(f"[USERNAME] Case 1: Full UUID format")
user = await get_user(user_id)
result = user.username if user and user.username else f"User-{user_id[:8]}"
logger.info(f"[USERNAME] Case 1 result: '{result}'")
return result
# Case 2: Dashless 32-char UUID - lookup via Castle user settings
elif len(user_id) == 32 and '-' not in user_id:
logger.info(f"[USERNAME] Case 2: Dashless UUID format - looking up in Castle user settings")
try:
# Get all Castle users (which have full user_ids)
user_settings = await get_all_user_wallet_settings()
# Convert dashless to dashed format for comparison
user_id_with_dashes = f"{user_id[0:8]}-{user_id[8:12]}-{user_id[12:16]}-{user_id[16:20]}-{user_id[20:32]}"
logger.info(f"[USERNAME] Converted to dashed format: {user_id_with_dashes}")
# Find matching user
for setting in user_settings:
if setting.id == user_id_with_dashes:
logger.info(f"[USERNAME] Found matching user in Castle settings")
# Get username from LNbits
user = await get_user(setting.id)
result = user.username if user and user.username else f"User-{user_id[:8]}"
logger.info(f"[USERNAME] Case 2 result (found): '{result}'")
return result
# No matching user found
logger.info(f"[USERNAME] No matching user found in Castle settings")
result = f"User-{user_id[:8]}"
logger.info(f"[USERNAME] Case 2 result (not found): '{result}'")
return result
except Exception as e:
logger.error(f"Error looking up user by dashless UUID {user_id}: {e}")
result = f"User-{user_id[:8]}"
return result
# Case 3: Partial ID (8 chars from account name) - lookup via Castle user settings
elif len(user_id) == 8:
logger.info(f"[USERNAME] Case 3: Partial ID format - looking up in Castle user settings")
try:
# Get all Castle users (which have full user_ids)
user_settings = await get_all_user_wallet_settings()
# Find matching user by first 8 chars
for setting in user_settings:
if setting.id.startswith(user_id):
logger.info(f"[USERNAME] Found full user_id: {setting.id}")
# Now get username from LNbits with full ID
user = await get_user(setting.id)
result = user.username if user and user.username else f"User-{user_id}"
logger.info(f"[USERNAME] Case 3 result (found): '{result}'")
return result
# No matching user found in Castle settings
logger.info(f"[USERNAME] No matching user found in Castle settings")
result = f"User-{user_id}"
logger.info(f"[USERNAME] Case 3 result (not found): '{result}'")
return result
except Exception as e:
logger.error(f"Error looking up user by partial ID {user_id}: {e}")
result = f"User-{user_id}"
return result
# Case 4: Unknown format - try as-is and fall back
else:
logger.info(f"[USERNAME] Case 4: Unknown format - trying as-is")
try:
user = await get_user(user_id)
result = user.username if user and user.username else f"User-{user_id[:8]}"
logger.info(f"[USERNAME] Case 4 result: '{result}'")
return result
except Exception as e:
logger.info(f"[USERNAME] Case 4 exception: {e}")
result = f"User-{user_id[:8]}"
logger.info(f"[USERNAME] Case 4 fallback result: '{result}'")
return result
@castle_api_router.get("/api/v1/entries/pending")
async def api_get_pending_entries(
wallet: WalletTypeInfo = Depends(require_admin_key),
@ -516,7 +617,6 @@ async def api_get_pending_entries(
Returns transactions with flag='!' from Fava/Beancount.
"""
from lnbits.settings import settings as lnbits_settings
from lnbits.core.crud.users import get_user
from .fava_client import get_fava_client
if wallet.wallet.user != lnbits_settings.super_user:
@ -552,9 +652,13 @@ async def api_get_pending_entries(
# Extract user ID from metadata or account names
user_id = None
entry_meta = e.get("meta", {})
logger.info(f"[EXTRACT] Entry metadata keys: {list(entry_meta.keys())}")
logger.info(f"[EXTRACT] Entry metadata: {entry_meta}")
if "user-id" in entry_meta:
user_id = entry_meta["user-id"]
logger.info(f"[EXTRACT] Found user-id in metadata: {user_id}")
else:
logger.info(f"[EXTRACT] No user-id in metadata, checking account names")
# Try to extract from account names in postings
for posting in e.get("postings", []):
account = posting.get("account", "")
@ -563,13 +667,11 @@ async def api_get_pending_entries(
parts = account.split("User-")
if len(parts) > 1:
user_id = parts[1] # Short ID after User-
logger.info(f"[EXTRACT] Extracted user_id from account name: {user_id}")
break
# Look up username
username = None
if user_id:
user = await get_user(user_id)
username = user.username if user and user.username else f"User-{user_id[:8]}"
# Look up username using helper function
username = await _get_username_from_user_id(user_id) if user_id else None
# Extract amount from postings (sum of absolute values / 2)
amount_sats = 0
@ -1243,17 +1345,15 @@ async def api_get_all_balances(
wallet: WalletTypeInfo = Depends(require_admin_key),
) -> list[dict]:
"""Get all user balances (admin/super user only) from Fava/Beancount"""
from lnbits.core.crud.users import get_user
from .fava_client import get_fava_client
fava = get_fava_client()
balances = await fava.get_all_user_balances()
# Enrich with username information
# Enrich with username information using helper function
result = []
for balance in balances:
user = await get_user(balance["user_id"])
username = user.username if user and user.username else balance["user_id"][:16] + "..."
username = await _get_username_from_user_id(balance["user_id"])
result.append({
"user_id": balance["user_id"],