diff --git a/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.html b/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.html new file mode 100644 index 0000000..d0e9bfe --- /dev/null +++ b/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.html @@ -0,0 +1,953 @@ + + + + + + + ACCOUNTING-ANALYSIS-NET-SETTLEMENT + + + + + +

Accounting +Analysis: Net Settlement Entry Pattern

+

Date: 2025-01-12 Prepared By: +Senior Accounting Review Subject: Castle Extension - +Lightning Payment Settlement Entries Status: Technical +Review

+
+

Executive Summary

+

This document provides a professional accounting assessment of +Castle’s net settlement entry pattern used for recording Lightning +Network payments that settle fiat-denominated receivables. The analysis +identifies areas where the implementation deviates from traditional +accounting best practices and provides specific recommendations for +improvement.

+

Key Findings: - ✅ Double-entry integrity maintained +- ✅ Functional for intended purpose - ❌ Zero-amount postings violate +accounting principles - ❌ Redundant satoshi tracking - ❌ No exchange +gain/loss recognition - ⚠️ Mixed currency approach lacks clear +hierarchy

+
+

Background: The Technical +Challenge

+

Castle operates as a Lightning Network-integrated accounting system +for collectives (co-living spaces, makerspaces). It faces a unique +accounting challenge:

+

Scenario: User creates a receivable in EUR (e.g., +€200 for room rent), then pays via Lightning Network in satoshis +(225,033 sats).

+

Challenge: Record the payment while: 1. Clearing the +exact EUR receivable amount 2. Recording the exact satoshi amount +received 3. Handling cases where users have both receivables (owe +Castle) and payables (Castle owes them) 4. Maintaining Beancount +double-entry balance

+
+

Current Implementation

+

Transaction Example

+
; Step 1: Receivable Created
+2025-11-12 * "room (200.00 EUR)" #receivable-entry
+  user-id: "375ec158"
+  source: "castle-api"
+  sats-amount: "225033"
+  Assets:Receivable:User-375ec158     200.00 EUR
+    sats-equivalent: "225033"
+  Income:Accommodation:Guests        -200.00 EUR
+    sats-equivalent: "225033"
+
+; Step 2: Lightning Payment Received
+2025-11-12 * "Lightning payment settlement from user 375ec158"
+  #lightning-payment #net-settlement
+  user-id: "375ec158"
+  source: "lightning_payment"
+  payment-type: "net-settlement"
+  payment-hash: "8d080ec4cc4301715535004156085dd50c159185..."
+  Assets:Bitcoin:Lightning            225033 SATS @ 0.0008887585... EUR
+    payment-hash: "8d080ec4cc4301715535004156085dd50c159185..."
+  Assets:Receivable:User-375ec158    -200.00 EUR
+    sats-equivalent: "225033"
+  Liabilities:Payable:User-375ec158     0.00 EUR
+

Code Implementation

+

Location: +beancount_format.py:739-760

+
# Build postings for net settlement
+postings = [
+    {
+        "account": payment_account,
+        "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}",
+        "meta": {"payment-hash": payment_hash} if payment_hash else {}
+    },
+    {
+        "account": receivable_account,
+        "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}",
+        "meta": {"sats-equivalent": str(abs(amount_sats))}
+    },
+    {
+        "account": payable_account,
+        "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}",
+        "meta": {}
+    }
+]
+

Three-Posting Structure: 1. Lightning +Account: Records SATS received with @@ total price +notation 2. Receivable Account: Clears EUR receivable +with sats-equivalent metadata 3. Payable Account: +Clears any outstanding EUR payables (often 0.00)

+
+

Accounting Issues Identified

+

Issue 1: Zero-Amount Postings

+

Problem: The third posting often records +0.00 EUR when no payable exists.

+
Liabilities:Payable:User-375ec158     0.00 EUR
+

Why This Is Wrong: - Zero-amount postings have no +economic substance - Clutters the journal with non-events - Violates the +principle of materiality (GAAP Concept Statement 2) - Makes auditing +more difficult (reviewers must verify why zero amounts exist)

+

Accounting Principle Violated: > “Transactions +should only include postings that represent actual economic events or +changes in account balances.”

+

Impact: Low severity, but unprofessional +presentation

+

Recommendation:

+
# Make payable posting conditional
+postings = [
+    {"account": payment_account, "amount": ...},
+    {"account": receivable_account, "amount": ...}
+]
+
+# Only add payable posting if there's actually a payable
+if total_payable_fiat > 0:
+    postings.append({
+        "account": payable_account,
+        "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}",
+        "meta": {}
+    })
+
+

Issue 2: Redundant Satoshi +Tracking

+

Problem: Satoshis are tracked in TWO places in the +same transaction:

+
    +
  1. Position Amount (via @@ +notation):

    +
    Assets:Bitcoin:Lightning  225033 SATS @@ 200.00 EUR
  2. +
  3. Metadata (sats-equivalent):

    +
    Assets:Receivable:User-375ec158  -200.00 EUR
    +  sats-equivalent: "225033"
  4. +
+

Why This Is Problematic: - The @@ +notation already records the exact satoshi amount - Beancount’s price +database stores this relationship - Metadata becomes redundant for this +specific posting - Increases storage and potential for inconsistency

+

Technical Detail:

+

The @@ notation means “total price” and Beancount +converts it to per-unit price:

+
; You write:
+Assets:Bitcoin:Lightning  225033 SATS @@ 200.00 EUR
+
+; Beancount stores:
+Assets:Bitcoin:Lightning  225033 SATS @ 0.0008887585... EUR
+; (where 200.00 / 225033 = 0.0008887585...)
+

Beancount can query this:

+
SELECT account, sum(convert(position, SATS))
+WHERE account = 'Assets:Bitcoin:Lightning'
+

Recommendation:

+

Choose ONE approach consistently:

+

Option A - Use @ notation (Beancount standard):

+
Assets:Bitcoin:Lightning           225033 SATS @@ 200.00 EUR
+  payment-hash: "8d080ec4..."
+Assets:Receivable:User-375ec158   -200.00 EUR
+  ; No sats-equivalent needed here
+

Option B - Use EUR positions with metadata (Castle’s +current approach):

+
Assets:Bitcoin:Lightning           200.00 EUR
+  sats-received: "225033"
+  payment-hash: "8d080ec4..."
+Assets:Receivable:User-375ec158   -200.00 EUR
+  sats-cleared: "225033"
+

Don’t: Mix both in the same transaction (current +implementation)

+
+

Issue 3: No Exchange +Gain/Loss Recognition

+

Problem: When receivables are denominated in one +currency (EUR) and paid in another (SATS), exchange rate fluctuations +create gains or losses that should be recognized.

+

Example Scenario:

+
Day 1 - Receivable Created:
+  200 EUR = 225,033 SATS (rate: 1,125.165 sats/EUR)
+
+Day 5 - Payment Received:
+  225,033 SATS = 199.50 EUR (rate: 1,127.682 sats/EUR)
+  Exchange rate moved unfavorably
+
+Economic Reality: 0.50 EUR LOSS
+

Current Implementation: Forces balance by +calculating the @ rate to make it exactly 200 EUR:

+
Assets:Bitcoin:Lightning  225033 SATS @ 0.000888... EUR  ; = exactly 200.00 EUR
+

This hides the exchange variance by treating the +payment as if it was worth exactly the receivable amount.

+

GAAP/IFRS Requirement:

+

Under both US GAAP (ASC 830) and IFRS (IAS 21), exchange gains and +losses on monetary items (like receivables) should be recognized in the +period they occur.

+

Proper Accounting Treatment:

+
2025-11-12 * "Lightning payment with exchange loss"
+  Assets:Bitcoin:Lightning           225033 SATS @ 0.000886... EUR
+    ; Market rate at payment time = 199.50 EUR
+  Expenses:Foreign-Exchange-Loss     0.50 EUR
+  Assets:Receivable:User-375ec158   -200.00 EUR
+

Impact: Moderate severity - affects financial +statement accuracy

+

Why This Matters: - Tax reporting may require +exchange gain/loss recognition - Financial statements misstate true +economic results - Auditors would flag this as a compliance issue - +Cannot accurately calculate ROI or performance metrics

+
+

Issue 4: Semantic +Misuse of Price Notation

+

Problem: The @ notation in Beancount +represents acquisition cost, not settlement +value.

+

Current Usage:

+
Assets:Bitcoin:Lightning  225033 SATS @ 0.000888... EUR
+

What this notation means in accounting: “We +purchased 225,033 satoshis at a cost of 0.000888 EUR +per satoshi”

+

What actually happened: “We +received 225,033 satoshis as payment for a debt”

+

Economic Difference: - Purchase: +You exchange cash for an asset (buying Bitcoin) - Payment +Receipt: You receive an asset in settlement of a receivable

+

Accounting Substance vs. Form: - +Form: The transaction looks like a Bitcoin purchase - +Substance: The transaction is actually a receivable +collection

+

GAAP Principle (ASC 105-10-05): > “Accounting +should reflect the economic substance of transactions, not merely their +legal form.”

+

Why This Creates Issues:

+
    +
  1. Cost Basis Tracking: For tax purposes, the “cost” +of Bitcoin received as payment should be its fair market value at +receipt, not the receivable amount
  2. +
  3. Price Database Pollution: Beancount’s price +database now contains “prices” that aren’t real market prices
  4. +
  5. Auditor Confusion: An auditor reviewing this would +question why purchase prices don’t match market rates
  6. +
+

Proper Accounting Approach:

+
; Approach 1: Record at fair market value
+Assets:Bitcoin:Lightning           225033 SATS @ 0.000886... EUR
+  ; Using actual market price at time of receipt
+  acquisition-type: "payment-received"
+Revenue:Exchange-Gain              0.50 EUR
+Assets:Receivable:User-375ec158   -200.00 EUR
+
+; Approach 2: Don't use @ notation at all
+Assets:Bitcoin:Lightning           200.00 EUR
+  sats-received: "225033"
+  fmv-at-receipt: "199.50 EUR"
+Assets:Receivable:User-375ec158   -200.00 EUR
+
+

Issue 5: Misnamed +Function and Incorrect Usage

+

Problem: Function is called +format_net_settlement_entry, but it’s used for simple +payments that aren’t true net settlements.

+

Example from User’s Transaction: - Receivable: +200.00 EUR - Payable: 0.00 EUR - Net: 200.00 EUR (this is just a +payment, not a settlement)

+

Accounting Terminology:

+ +

When Net Settlement is Appropriate:

+
User owes Castle:    555.00 EUR (receivable)
+Castle owes User:     38.00 EUR (payable)
+Net amount due:      517.00 EUR (true settlement)
+

Proper three-posting entry:

+
Assets:Bitcoin:Lightning           565251 SATS @@ 517.00 EUR
+Assets:Receivable:User            -555.00 EUR
+Liabilities:Payable:User            38.00 EUR
+; Net: 517.00 = -555.00 + 38.00 ✓
+

When Two Postings Suffice:

+
User owes Castle:    200.00 EUR (receivable)
+Castle owes User:      0.00 EUR (no payable)
+Amount due:          200.00 EUR (simple payment)
+

Simpler two-posting entry:

+
Assets:Bitcoin:Lightning           225033 SATS @@ 200.00 EUR
+Assets:Receivable:User            -200.00 EUR
+

Best Practice: Use the simplest journal entry +structure that accurately represents the transaction.

+

Recommendation: 1. Rename function to +format_payment_entry or +format_receivable_payment_entry 2. Create separate +format_net_settlement_entry for true netting scenarios 3. +Use conditional logic to choose 2-posting vs 3-posting based on whether +both receivables AND payables exist

+
+

Traditional Accounting +Approaches

+

Approach +1: Record Bitcoin at Fair Market Value (Tax Compliant)

+
2025-11-12 * "Bitcoin payment from user 375ec158"
+  Assets:Bitcoin:Lightning           199.50 EUR
+    sats-received: "225033"
+    fmv-per-sat: "0.000886 EUR"
+    cost-basis: "199.50 EUR"
+    payment-hash: "8d080ec4..."
+  Revenue:Exchange-Gain              0.50 EUR
+    source: "cryptocurrency-receipt"
+  Assets:Receivable:User-375ec158   -200.00 EUR
+

Pros: - ✅ Tax compliant (establishes cost basis) - +✅ Recognizes exchange gain/loss - ✅ Uses actual market rates - ✅ +Audit trail for cryptocurrency receipts

+

Cons: - ❌ Requires real-time price feeds - ❌ +Creates taxable events

+
+

Approach 2: +Simplified EUR-Only Ledger (No SATS Positions)

+
2025-11-12 * "Bitcoin payment from user 375ec158"
+  Assets:Bitcoin:Lightning           200.00 EUR
+    sats-received: "225033"
+    sats-rate: "1125.165"
+    payment-hash: "8d080ec4..."
+  Assets:Receivable:User-375ec158   -200.00 EUR
+

Pros: - ✅ Simple and clean - ✅ EUR positions match +accounting reality - ✅ SATS tracked in metadata for reference - ✅ No +artificial price notation

+

Cons: - ❌ SATS not queryable via Beancount +positions - ❌ Requires metadata parsing for SATS balances

+
+

Approach +3: True Net Settlement (When Both Obligations Exist)

+
2025-11-12 * "Net settlement via Lightning"
+  ; User owes 555 EUR, Castle owes 38 EUR, net: 517 EUR
+  Assets:Bitcoin:Lightning           517.00 EUR
+    sats-received: "565251"
+  Assets:Receivable:User-375ec158   -555.00 EUR
+  Liabilities:Payable:User-375ec158   38.00 EUR
+

When to Use: Only when both +receivables and payables exist and you’re truly netting them.

+
+

Recommendations

+

Priority 1: Immediate +Fixes (Easy Wins)

+

1.1 Remove Zero-Amount +Postings

+

File: beancount_format.py:739-760

+

Current Code:

+
postings = [
+    {...},  # Lightning
+    {...},  # Receivable
+    {       # Payable (always included, even if 0.00)
+        "account": payable_account,
+        "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}",
+        "meta": {}
+    }
+]
+

Fixed Code:

+
postings = [
+    {
+        "account": payment_account,
+        "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}",
+        "meta": {"payment-hash": payment_hash} if payment_hash else {}
+    },
+    {
+        "account": receivable_account,
+        "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}",
+        "meta": {"sats-equivalent": str(abs(amount_sats))}
+    }
+]
+
+# Only add payable posting if there's actually a payable to clear
+if total_payable_fiat > 0:
+    postings.append({
+        "account": payable_account,
+        "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}",
+        "meta": {}
+    })
+

Impact: Cleaner journal, professional presentation, +easier auditing

+
+

1.2 Choose One SATS Tracking +Method

+

Decision Required: Select either position-based OR +metadata-based satoshi tracking.

+

Option A - Keep Metadata Approach (recommended for +Castle):

+
# In format_net_settlement_entry()
+postings = [
+    {
+        "account": payment_account,
+        "amount": f"{abs(net_fiat_amount):.2f} {fiat_currency}",  # EUR only
+        "meta": {
+            "sats-received": str(abs(amount_sats)),
+            "payment-hash": payment_hash
+        }
+    },
+    {
+        "account": receivable_account,
+        "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}",
+        "meta": {"sats-cleared": str(abs(amount_sats))}
+    }
+]
+

Option B - Use Position-Based Tracking:

+
# Remove sats-equivalent metadata entirely
+postings = [
+    {
+        "account": payment_account,
+        "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}",
+        "meta": {"payment-hash": payment_hash}
+    },
+    {
+        "account": receivable_account,
+        "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}",
+        # No sats-equivalent needed - queryable via price database
+    }
+]
+

Recommendation: Choose Option A (metadata) for +consistency with Castle’s architecture.

+
+

1.3 Rename Function for +Clarity

+

File: beancount_format.py

+

Current: +format_net_settlement_entry()

+

New: format_receivable_payment_entry() +or format_payment_settlement_entry()

+

Rationale: More accurately describes what the +function does (processes payments, not always net settlements)

+
+

Priority 2: +Medium-Term Improvements (Compliance)

+

2.1 Add Exchange Gain/Loss +Tracking

+

File: tasks.py:259-276 (get balance and +calculate settlement)

+

New Logic:

+
# Get user's current balance
+balance = await fava.get_user_balance(user_id)
+fiat_balances = balance.get("fiat_balances", {})
+total_fiat_balance = fiat_balances.get(fiat_currency, Decimal(0))
+
+# Calculate expected fiat value of SATS payment at current market rate
+market_rate = await get_current_sats_eur_rate()  # New function needed
+market_value = Decimal(amount_sats) * market_rate
+
+# Calculate exchange variance
+receivable_amount = abs(total_fiat_balance) if total_fiat_balance > 0 else Decimal(0)
+exchange_variance = market_value - receivable_amount
+
+# If variance is material (> 1 cent), create exchange gain/loss posting
+if abs(exchange_variance) > Decimal("0.01"):
+    # Add exchange gain/loss to postings
+    if exchange_variance > 0:
+        # Gain: payment worth more than receivable
+        exchange_account = "Revenue:Foreign-Exchange-Gain"
+    else:
+        # Loss: payment worth less than receivable
+        exchange_account = "Expenses:Foreign-Exchange-Loss"
+
+    # Include in entry creation
+    exchange_posting = {
+        "account": exchange_account,
+        "amount": f"{abs(exchange_variance):.2f} {fiat_currency}",
+        "meta": {
+            "sats-amount": str(amount_sats),
+            "market-rate": str(market_rate),
+            "receivable-amount": str(receivable_amount)
+        }
+    }
+

Benefits: - ✅ Tax compliance - ✅ Accurate +financial reporting - ✅ Audit trail for cryptocurrency gains/losses - +✅ Regulatory compliance (GAAP/IFRS)

+
+

2.2 +Implement True Net Settlement vs. Simple Payment Logic

+

File: tasks.py or new +payment_logic.py

+
async def create_payment_entry(
+    user_id: str,
+    amount_sats: int,
+    fiat_amount: Decimal,
+    fiat_currency: str,
+    payment_hash: str
+):
+    """
+    Create appropriate payment entry based on user's balance situation.
+    Uses 2-posting for simple payments, 3-posting for net settlements.
+    """
+    # Get user balance
+    balance = await fava.get_user_balance(user_id)
+    fiat_balances = balance.get("fiat_balances", {})
+    total_balance = fiat_balances.get(fiat_currency, Decimal(0))
+
+    receivable_amount = Decimal(0)
+    payable_amount = Decimal(0)
+
+    if total_balance > 0:
+        receivable_amount = total_balance
+    elif total_balance < 0:
+        payable_amount = abs(total_balance)
+
+    # Determine entry type
+    if receivable_amount > 0 and payable_amount > 0:
+        # TRUE NET SETTLEMENT: Both obligations exist
+        return await format_net_settlement_entry(
+            user_id=user_id,
+            amount_sats=amount_sats,
+            receivable_amount=receivable_amount,
+            payable_amount=payable_amount,
+            fiat_amount=fiat_amount,
+            fiat_currency=fiat_currency,
+            payment_hash=payment_hash
+        )
+    elif receivable_amount > 0:
+        # SIMPLE RECEIVABLE PAYMENT: Only receivable exists
+        return await format_receivable_payment_entry(
+            user_id=user_id,
+            amount_sats=amount_sats,
+            receivable_amount=receivable_amount,
+            fiat_amount=fiat_amount,
+            fiat_currency=fiat_currency,
+            payment_hash=payment_hash
+        )
+    else:
+        # PAYABLE PAYMENT: Castle paying user (different flow)
+        return await format_payable_payment_entry(...)
+
+

Priority 3: +Long-Term Architectural Decisions

+

3.1 Establish Primary +Currency Hierarchy

+

Current Issue: Mixed approach (EUR positions with +SATS metadata, but also SATS positions with @ notation)

+

Decision Required: Choose ONE of the following +architectures:

+

Architecture A - EUR Primary, SATS Secondary +(recommended):

+
; All positions in EUR, SATS in metadata
+2025-11-12 * "Payment"
+  Assets:Bitcoin:Lightning           200.00 EUR
+    sats-received: "225033"
+  Assets:Receivable:User            -200.00 EUR
+    sats-cleared: "225033"
+

Architecture B - SATS Primary, EUR Secondary:

+
; All positions in SATS, EUR in metadata
+2025-11-12 * "Payment"
+  Assets:Bitcoin:Lightning           225033 SATS
+    eur-value: "200.00"
+  Assets:Receivable:User            -225033 SATS
+    eur-cleared: "200.00"
+

Recommendation: Architecture A (EUR primary) +because: 1. Most receivables created in EUR 2. Financial reporting +requirements typically in fiat 3. Tax obligations calculated in fiat 4. +Aligns with current Castle metadata approach

+
+

3.2 +Consider Separate Ledger for Cryptocurrency Holdings

+

Advanced Approach: Separate cryptocurrency movements +from fiat accounting

+

Main Ledger (EUR-denominated):

+
2025-11-12 * "Payment received from user"
+  Assets:Bitcoin-Custody:User-375ec158  200.00 EUR
+  Assets:Receivable:User-375ec158      -200.00 EUR
+

Cryptocurrency Sub-Ledger (SATS-denominated):

+
2025-11-12 * "Lightning payment received"
+  Assets:Bitcoin:Lightning:Castle    225033 SATS
+  Assets:Bitcoin:Custody:User-375ec  225033 SATS
+

Benefits: - ✅ Clean separation of concerns - ✅ +Cryptocurrency movements tracked independently - ✅ Fiat accounting +unaffected by Bitcoin volatility - ✅ Can generate separate financial +statements

+

Drawbacks: - ❌ Increased complexity - ❌ +Reconciliation between ledgers required - ❌ Two sets of books to +maintain

+
+

Code Files Requiring Changes

+

High Priority (Immediate +Fixes)

+
    +
  1. beancount_format.py:739-760 +
  2. +
  3. beancount_format.py:692 +
  4. +
+

Medium Priority (Compliance)

+
    +
  1. tasks.py:235-310 +
  2. +
  3. New file: exchange_rates.py +
  4. +
  5. beancount_format.py +
  6. +
+
+

Testing Requirements

+

Test Case 1: +Simple Receivable Payment (No Payable)

+

Setup: - User has receivable: 200.00 EUR - User has +payable: 0.00 EUR - User pays: 225,033 SATS

+

Expected Entry (after fixes):

+
2025-11-12 * "Lightning payment from user"
+  Assets:Bitcoin:Lightning           200.00 EUR
+    sats-received: "225033"
+    payment-hash: "8d080ec4..."
+  Assets:Receivable:User            -200.00 EUR
+    sats-cleared: "225033"
+

Verify: - ✅ Only 2 postings (no zero-amount +payable) - ✅ Entry balances - ✅ SATS tracked in metadata - ✅ User +balance becomes 0 (both EUR and SATS)

+
+

Test Case 2: True Net +Settlement

+

Setup: - User has receivable: 555.00 EUR - User has +payable: 38.00 EUR - Net owed: 517.00 EUR - User pays: 565,251 SATS +(worth 517.00 EUR)

+

Expected Entry:

+
2025-11-12 * "Net settlement via Lightning"
+  Assets:Bitcoin:Lightning           517.00 EUR
+    sats-received: "565251"
+    payment-hash: "abc123..."
+  Assets:Receivable:User            -555.00 EUR
+    sats-portion: "565251"
+  Liabilities:Payable:User            38.00 EUR
+

Verify: - ✅ 3 postings (receivable + payable +cleared) - ✅ Net amount = receivable - payable - ✅ Both balances +become 0 - ✅ Mathematically balanced

+
+

Test Case 3: Exchange +Gain/Loss (Future)

+

Setup: - User has receivable: 200.00 EUR (created at +1,125 sats/EUR) - User pays: 225,033 SATS (now worth 199.50 EUR at +market) - Exchange loss: 0.50 EUR

+

Expected Entry (with exchange tracking):

+
2025-11-12 * "Lightning payment with exchange loss"
+  Assets:Bitcoin:Lightning           199.50 EUR
+    sats-received: "225033"
+    market-rate: "0.000886"
+  Expenses:Foreign-Exchange-Loss     0.50 EUR
+  Assets:Receivable:User            -200.00 EUR
+

Verify: - ✅ Bitcoin recorded at fair market value - +✅ Exchange loss recognized - ✅ Receivable cleared at book value - ✅ +Entry balances

+
+

Conclusion

+

Summary of Issues

+ ++++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IssueSeverityAccounting ImpactRecommended Action
Zero-amount postingsLowPresentation onlyRemove immediately
Redundant SATS trackingLowStorage/efficiencyChoose one method
No exchange gain/lossHighFinancial accuracyImplement for compliance
Semantic misuse of @MediumAudit clarityConsider EUR-only positions
Misnamed functionLowCode clarityRename function
+

Professional Assessment

+

Is this “best practice” accounting? +No, this implementation deviates from traditional +accounting standards in several ways.

+

Is it acceptable for Castle’s use case? Yes, +with modifications, it’s a reasonable pragmatic solution for a +novel problem (cryptocurrency payments of fiat debts).

+

Critical improvements needed: 1. ✅ Remove +zero-amount postings (easy fix, professional presentation) 2. ✅ +Implement exchange gain/loss tracking (required for compliance) 3. ✅ +Separate payment vs. settlement logic (accuracy and clarity)

+

The fundamental challenge: Traditional accounting +wasn’t designed for this scenario. There is no established “standard” +for recording cryptocurrency payments of fiat-denominated receivables. +Castle’s approach is functional, but should be refined to align better +with accounting principles where possible.

+

Next Steps

+
    +
  1. Week 1: Implement Priority 1 fixes (remove zero +postings, rename function)
  2. +
  3. Week 2-3: Design and implement exchange gain/loss +tracking
  4. +
  5. Week 4: Add payment vs. settlement logic
  6. +
  7. Ongoing: Monitor regulatory guidance on +cryptocurrency accounting
  8. +
+
+

References

+ +
+

Document Version: 1.0 Last Updated: +2025-01-12 Next Review: After Priority 1 fixes +implemented

+
+

This analysis was prepared for internal review and development +planning. It represents a professional accounting assessment of the +current implementation and should be used to guide improvements to +Castle’s payment recording system.

+ + diff --git a/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.md b/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.md new file mode 100644 index 0000000..b145128 --- /dev/null +++ b/docs/ACCOUNTING-ANALYSIS-NET-SETTLEMENT.md @@ -0,0 +1,861 @@ +# Accounting Analysis: Net Settlement Entry Pattern + +**Date**: 2025-01-12 +**Prepared By**: Senior Accounting Review +**Subject**: Castle Extension - Lightning Payment Settlement Entries +**Status**: Technical Review + +--- + +## Executive Summary + +This document provides a professional accounting assessment of Castle's net settlement entry pattern used for recording Lightning Network payments that settle fiat-denominated receivables. The analysis identifies areas where the implementation deviates from traditional accounting best practices and provides specific recommendations for improvement. + +**Key Findings**: +- ✅ Double-entry integrity maintained +- ✅ Functional for intended purpose +- ❌ Zero-amount postings violate accounting principles +- ❌ Redundant satoshi tracking +- ❌ No exchange gain/loss recognition +- ⚠️ Mixed currency approach lacks clear hierarchy + +--- + +## Background: The Technical Challenge + +Castle operates as a Lightning Network-integrated accounting system for collectives (co-living spaces, makerspaces). It faces a unique accounting challenge: + +**Scenario**: User creates a receivable in EUR (e.g., €200 for room rent), then pays via Lightning Network in satoshis (225,033 sats). + +**Challenge**: Record the payment while: +1. Clearing the exact EUR receivable amount +2. Recording the exact satoshi amount received +3. Handling cases where users have both receivables (owe Castle) and payables (Castle owes them) +4. Maintaining Beancount double-entry balance + +--- + +## Current Implementation + +### Transaction Example + +```beancount +; Step 1: Receivable Created +2025-11-12 * "room (200.00 EUR)" #receivable-entry + user-id: "375ec158" + source: "castle-api" + sats-amount: "225033" + Assets:Receivable:User-375ec158 200.00 EUR + sats-equivalent: "225033" + Income:Accommodation:Guests -200.00 EUR + sats-equivalent: "225033" + +; Step 2: Lightning Payment Received +2025-11-12 * "Lightning payment settlement from user 375ec158" + #lightning-payment #net-settlement + user-id: "375ec158" + source: "lightning_payment" + payment-type: "net-settlement" + payment-hash: "8d080ec4cc4301715535004156085dd50c159185..." + Assets:Bitcoin:Lightning 225033 SATS @ 0.0008887585... EUR + payment-hash: "8d080ec4cc4301715535004156085dd50c159185..." + Assets:Receivable:User-375ec158 -200.00 EUR + sats-equivalent: "225033" + Liabilities:Payable:User-375ec158 0.00 EUR +``` + +### Code Implementation + +**Location**: `beancount_format.py:739-760` + +```python +# Build postings for net settlement +postings = [ + { + "account": payment_account, + "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}", + "meta": {"payment-hash": payment_hash} if payment_hash else {} + }, + { + "account": receivable_account, + "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}", + "meta": {"sats-equivalent": str(abs(amount_sats))} + }, + { + "account": payable_account, + "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}", + "meta": {} + } +] +``` + +**Three-Posting Structure**: +1. **Lightning Account**: Records SATS received with `@@` total price notation +2. **Receivable Account**: Clears EUR receivable with sats-equivalent metadata +3. **Payable Account**: Clears any outstanding EUR payables (often 0.00) + +--- + +## Accounting Issues Identified + +### Issue 1: Zero-Amount Postings + +**Problem**: The third posting often records `0.00 EUR` when no payable exists. + +```beancount +Liabilities:Payable:User-375ec158 0.00 EUR +``` + +**Why This Is Wrong**: +- Zero-amount postings have no economic substance +- Clutters the journal with non-events +- Violates the principle of materiality (GAAP Concept Statement 2) +- Makes auditing more difficult (reviewers must verify why zero amounts exist) + +**Accounting Principle Violated**: +> "Transactions should only include postings that represent actual economic events or changes in account balances." + +**Impact**: Low severity, but unprofessional presentation + +**Recommendation**: +```python +# Make payable posting conditional +postings = [ + {"account": payment_account, "amount": ...}, + {"account": receivable_account, "amount": ...} +] + +# Only add payable posting if there's actually a payable +if total_payable_fiat > 0: + postings.append({ + "account": payable_account, + "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}", + "meta": {} + }) +``` + +--- + +### Issue 2: Redundant Satoshi Tracking + +**Problem**: Satoshis are tracked in TWO places in the same transaction: + +1. **Position Amount** (via `@@` notation): + ```beancount + Assets:Bitcoin:Lightning 225033 SATS @@ 200.00 EUR + ``` + +2. **Metadata** (sats-equivalent): + ```beancount + Assets:Receivable:User-375ec158 -200.00 EUR + sats-equivalent: "225033" + ``` + +**Why This Is Problematic**: +- The `@@` notation already records the exact satoshi amount +- Beancount's price database stores this relationship +- Metadata becomes redundant for this specific posting +- Increases storage and potential for inconsistency + +**Technical Detail**: + +The `@@` notation means "total price" and Beancount converts it to per-unit price: +```beancount +; You write: +Assets:Bitcoin:Lightning 225033 SATS @@ 200.00 EUR + +; Beancount stores: +Assets:Bitcoin:Lightning 225033 SATS @ 0.0008887585... EUR +; (where 200.00 / 225033 = 0.0008887585...) +``` + +Beancount can query this: +```sql +SELECT account, sum(convert(position, SATS)) +WHERE account = 'Assets:Bitcoin:Lightning' +``` + +**Recommendation**: + +Choose ONE approach consistently: + +**Option A - Use @ notation** (Beancount standard): +```beancount +Assets:Bitcoin:Lightning 225033 SATS @@ 200.00 EUR + payment-hash: "8d080ec4..." +Assets:Receivable:User-375ec158 -200.00 EUR + ; No sats-equivalent needed here +``` + +**Option B - Use EUR positions with metadata** (Castle's current approach): +```beancount +Assets:Bitcoin:Lightning 200.00 EUR + sats-received: "225033" + payment-hash: "8d080ec4..." +Assets:Receivable:User-375ec158 -200.00 EUR + sats-cleared: "225033" +``` + +**Don't**: Mix both in the same transaction (current implementation) + +--- + +### Issue 3: No Exchange Gain/Loss Recognition + +**Problem**: When receivables are denominated in one currency (EUR) and paid in another (SATS), exchange rate fluctuations create gains or losses that should be recognized. + +**Example Scenario**: + +``` +Day 1 - Receivable Created: + 200 EUR = 225,033 SATS (rate: 1,125.165 sats/EUR) + +Day 5 - Payment Received: + 225,033 SATS = 199.50 EUR (rate: 1,127.682 sats/EUR) + Exchange rate moved unfavorably + +Economic Reality: 0.50 EUR LOSS +``` + +**Current Implementation**: Forces balance by calculating the `@` rate to make it exactly 200 EUR: +```beancount +Assets:Bitcoin:Lightning 225033 SATS @ 0.000888... EUR ; = exactly 200.00 EUR +``` + +This **hides the exchange variance** by treating the payment as if it was worth exactly the receivable amount. + +**GAAP/IFRS Requirement**: + +Under both US GAAP (ASC 830) and IFRS (IAS 21), exchange gains and losses on monetary items (like receivables) should be recognized in the period they occur. + +**Proper Accounting Treatment**: + +```beancount +2025-11-12 * "Lightning payment with exchange loss" + Assets:Bitcoin:Lightning 225033 SATS @ 0.000886... EUR + ; Market rate at payment time = 199.50 EUR + Expenses:Foreign-Exchange-Loss 0.50 EUR + Assets:Receivable:User-375ec158 -200.00 EUR +``` + +**Impact**: Moderate severity - affects financial statement accuracy + +**Why This Matters**: +- Tax reporting may require exchange gain/loss recognition +- Financial statements misstate true economic results +- Auditors would flag this as a compliance issue +- Cannot accurately calculate ROI or performance metrics + +--- + +### Issue 4: Semantic Misuse of Price Notation + +**Problem**: The `@` notation in Beancount represents **acquisition cost**, not **settlement value**. + +**Current Usage**: +```beancount +Assets:Bitcoin:Lightning 225033 SATS @ 0.000888... EUR +``` + +**What this notation means in accounting**: "We **purchased** 225,033 satoshis at a cost of 0.000888 EUR per satoshi" + +**What actually happened**: "We **received** 225,033 satoshis as payment for a debt" + +**Economic Difference**: +- **Purchase**: You exchange cash for an asset (buying Bitcoin) +- **Payment Receipt**: You receive an asset in settlement of a receivable + +**Accounting Substance vs. Form**: +- **Form**: The transaction looks like a Bitcoin purchase +- **Substance**: The transaction is actually a receivable collection + +**GAAP Principle (ASC 105-10-05)**: +> "Accounting should reflect the economic substance of transactions, not merely their legal form." + +**Why This Creates Issues**: + +1. **Cost Basis Tracking**: For tax purposes, the "cost" of Bitcoin received as payment should be its fair market value at receipt, not the receivable amount +2. **Price Database Pollution**: Beancount's price database now contains "prices" that aren't real market prices +3. **Auditor Confusion**: An auditor reviewing this would question why purchase prices don't match market rates + +**Proper Accounting Approach**: + +```beancount +; Approach 1: Record at fair market value +Assets:Bitcoin:Lightning 225033 SATS @ 0.000886... EUR + ; Using actual market price at time of receipt + acquisition-type: "payment-received" +Revenue:Exchange-Gain 0.50 EUR +Assets:Receivable:User-375ec158 -200.00 EUR + +; Approach 2: Don't use @ notation at all +Assets:Bitcoin:Lightning 200.00 EUR + sats-received: "225033" + fmv-at-receipt: "199.50 EUR" +Assets:Receivable:User-375ec158 -200.00 EUR +``` + +--- + +### Issue 5: Misnamed Function and Incorrect Usage + +**Problem**: Function is called `format_net_settlement_entry`, but it's used for simple payments that aren't true net settlements. + +**Example from User's Transaction**: +- Receivable: 200.00 EUR +- Payable: 0.00 EUR +- Net: 200.00 EUR (this is just a **payment**, not a **settlement**) + +**Accounting Terminology**: + +- **Payment**: Settling a single obligation (receivable OR payable) +- **Net Settlement**: Offsetting multiple obligations (receivable AND payable) + +**When Net Settlement is Appropriate**: + +``` +User owes Castle: 555.00 EUR (receivable) +Castle owes User: 38.00 EUR (payable) +Net amount due: 517.00 EUR (true settlement) +``` + +Proper three-posting entry: +```beancount +Assets:Bitcoin:Lightning 565251 SATS @@ 517.00 EUR +Assets:Receivable:User -555.00 EUR +Liabilities:Payable:User 38.00 EUR +; Net: 517.00 = -555.00 + 38.00 ✓ +``` + +**When Two Postings Suffice**: + +``` +User owes Castle: 200.00 EUR (receivable) +Castle owes User: 0.00 EUR (no payable) +Amount due: 200.00 EUR (simple payment) +``` + +Simpler two-posting entry: +```beancount +Assets:Bitcoin:Lightning 225033 SATS @@ 200.00 EUR +Assets:Receivable:User -200.00 EUR +``` + +**Best Practice**: Use the simplest journal entry structure that accurately represents the transaction. + +**Recommendation**: +1. Rename function to `format_payment_entry` or `format_receivable_payment_entry` +2. Create separate `format_net_settlement_entry` for true netting scenarios +3. Use conditional logic to choose 2-posting vs 3-posting based on whether both receivables AND payables exist + +--- + +## Traditional Accounting Approaches + +### Approach 1: Record Bitcoin at Fair Market Value (Tax Compliant) + +```beancount +2025-11-12 * "Bitcoin payment from user 375ec158" + Assets:Bitcoin:Lightning 199.50 EUR + sats-received: "225033" + fmv-per-sat: "0.000886 EUR" + cost-basis: "199.50 EUR" + payment-hash: "8d080ec4..." + Revenue:Exchange-Gain 0.50 EUR + source: "cryptocurrency-receipt" + Assets:Receivable:User-375ec158 -200.00 EUR +``` + +**Pros**: +- ✅ Tax compliant (establishes cost basis) +- ✅ Recognizes exchange gain/loss +- ✅ Uses actual market rates +- ✅ Audit trail for cryptocurrency receipts + +**Cons**: +- ❌ Requires real-time price feeds +- ❌ Creates taxable events + +--- + +### Approach 2: Simplified EUR-Only Ledger (No SATS Positions) + +```beancount +2025-11-12 * "Bitcoin payment from user 375ec158" + Assets:Bitcoin:Lightning 200.00 EUR + sats-received: "225033" + sats-rate: "1125.165" + payment-hash: "8d080ec4..." + Assets:Receivable:User-375ec158 -200.00 EUR +``` + +**Pros**: +- ✅ Simple and clean +- ✅ EUR positions match accounting reality +- ✅ SATS tracked in metadata for reference +- ✅ No artificial price notation + +**Cons**: +- ❌ SATS not queryable via Beancount positions +- ❌ Requires metadata parsing for SATS balances + +--- + +### Approach 3: True Net Settlement (When Both Obligations Exist) + +```beancount +2025-11-12 * "Net settlement via Lightning" + ; User owes 555 EUR, Castle owes 38 EUR, net: 517 EUR + Assets:Bitcoin:Lightning 517.00 EUR + sats-received: "565251" + Assets:Receivable:User-375ec158 -555.00 EUR + Liabilities:Payable:User-375ec158 38.00 EUR +``` + +**When to Use**: Only when **both** receivables and payables exist and you're truly netting them. + +--- + +## Recommendations + +### Priority 1: Immediate Fixes (Easy Wins) + +#### 1.1 Remove Zero-Amount Postings + +**File**: `beancount_format.py:739-760` + +**Current Code**: +```python +postings = [ + {...}, # Lightning + {...}, # Receivable + { # Payable (always included, even if 0.00) + "account": payable_account, + "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}", + "meta": {} + } +] +``` + +**Fixed Code**: +```python +postings = [ + { + "account": payment_account, + "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}", + "meta": {"payment-hash": payment_hash} if payment_hash else {} + }, + { + "account": receivable_account, + "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}", + "meta": {"sats-equivalent": str(abs(amount_sats))} + } +] + +# Only add payable posting if there's actually a payable to clear +if total_payable_fiat > 0: + postings.append({ + "account": payable_account, + "amount": f"{abs(total_payable_fiat):.2f} {fiat_currency}", + "meta": {} + }) +``` + +**Impact**: Cleaner journal, professional presentation, easier auditing + +--- + +#### 1.2 Choose One SATS Tracking Method + +**Decision Required**: Select either position-based OR metadata-based satoshi tracking. + +**Option A - Keep Metadata Approach** (recommended for Castle): +```python +# In format_net_settlement_entry() +postings = [ + { + "account": payment_account, + "amount": f"{abs(net_fiat_amount):.2f} {fiat_currency}", # EUR only + "meta": { + "sats-received": str(abs(amount_sats)), + "payment-hash": payment_hash + } + }, + { + "account": receivable_account, + "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}", + "meta": {"sats-cleared": str(abs(amount_sats))} + } +] +``` + +**Option B - Use Position-Based Tracking**: +```python +# Remove sats-equivalent metadata entirely +postings = [ + { + "account": payment_account, + "amount": f"{abs(amount_sats)} SATS @@ {abs(net_fiat_amount):.2f} {fiat_currency}", + "meta": {"payment-hash": payment_hash} + }, + { + "account": receivable_account, + "amount": f"-{abs(total_receivable_fiat):.2f} {fiat_currency}", + # No sats-equivalent needed - queryable via price database + } +] +``` + +**Recommendation**: Choose Option A (metadata) for consistency with Castle's architecture. + +--- + +#### 1.3 Rename Function for Clarity + +**File**: `beancount_format.py` + +**Current**: `format_net_settlement_entry()` + +**New**: `format_receivable_payment_entry()` or `format_payment_settlement_entry()` + +**Rationale**: More accurately describes what the function does (processes payments, not always net settlements) + +--- + +### Priority 2: Medium-Term Improvements (Compliance) + +#### 2.1 Add Exchange Gain/Loss Tracking + +**File**: `tasks.py:259-276` (get balance and calculate settlement) + +**New Logic**: +```python +# Get user's current balance +balance = await fava.get_user_balance(user_id) +fiat_balances = balance.get("fiat_balances", {}) +total_fiat_balance = fiat_balances.get(fiat_currency, Decimal(0)) + +# Calculate expected fiat value of SATS payment at current market rate +market_rate = await get_current_sats_eur_rate() # New function needed +market_value = Decimal(amount_sats) * market_rate + +# Calculate exchange variance +receivable_amount = abs(total_fiat_balance) if total_fiat_balance > 0 else Decimal(0) +exchange_variance = market_value - receivable_amount + +# If variance is material (> 1 cent), create exchange gain/loss posting +if abs(exchange_variance) > Decimal("0.01"): + # Add exchange gain/loss to postings + if exchange_variance > 0: + # Gain: payment worth more than receivable + exchange_account = "Revenue:Foreign-Exchange-Gain" + else: + # Loss: payment worth less than receivable + exchange_account = "Expenses:Foreign-Exchange-Loss" + + # Include in entry creation + exchange_posting = { + "account": exchange_account, + "amount": f"{abs(exchange_variance):.2f} {fiat_currency}", + "meta": { + "sats-amount": str(amount_sats), + "market-rate": str(market_rate), + "receivable-amount": str(receivable_amount) + } + } +``` + +**Benefits**: +- ✅ Tax compliance +- ✅ Accurate financial reporting +- ✅ Audit trail for cryptocurrency gains/losses +- ✅ Regulatory compliance (GAAP/IFRS) + +--- + +#### 2.2 Implement True Net Settlement vs. Simple Payment Logic + +**File**: `tasks.py` or new `payment_logic.py` + +```python +async def create_payment_entry( + user_id: str, + amount_sats: int, + fiat_amount: Decimal, + fiat_currency: str, + payment_hash: str +): + """ + Create appropriate payment entry based on user's balance situation. + Uses 2-posting for simple payments, 3-posting for net settlements. + """ + # Get user balance + balance = await fava.get_user_balance(user_id) + fiat_balances = balance.get("fiat_balances", {}) + total_balance = fiat_balances.get(fiat_currency, Decimal(0)) + + receivable_amount = Decimal(0) + payable_amount = Decimal(0) + + if total_balance > 0: + receivable_amount = total_balance + elif total_balance < 0: + payable_amount = abs(total_balance) + + # Determine entry type + if receivable_amount > 0 and payable_amount > 0: + # TRUE NET SETTLEMENT: Both obligations exist + return await format_net_settlement_entry( + user_id=user_id, + amount_sats=amount_sats, + receivable_amount=receivable_amount, + payable_amount=payable_amount, + fiat_amount=fiat_amount, + fiat_currency=fiat_currency, + payment_hash=payment_hash + ) + elif receivable_amount > 0: + # SIMPLE RECEIVABLE PAYMENT: Only receivable exists + return await format_receivable_payment_entry( + user_id=user_id, + amount_sats=amount_sats, + receivable_amount=receivable_amount, + fiat_amount=fiat_amount, + fiat_currency=fiat_currency, + payment_hash=payment_hash + ) + else: + # PAYABLE PAYMENT: Castle paying user (different flow) + return await format_payable_payment_entry(...) +``` + +--- + +### Priority 3: Long-Term Architectural Decisions + +#### 3.1 Establish Primary Currency Hierarchy + +**Current Issue**: Mixed approach (EUR positions with SATS metadata, but also SATS positions with @ notation) + +**Decision Required**: Choose ONE of the following architectures: + +**Architecture A - EUR Primary, SATS Secondary** (recommended): +```beancount +; All positions in EUR, SATS in metadata +2025-11-12 * "Payment" + Assets:Bitcoin:Lightning 200.00 EUR + sats-received: "225033" + Assets:Receivable:User -200.00 EUR + sats-cleared: "225033" +``` + +**Architecture B - SATS Primary, EUR Secondary**: +```beancount +; All positions in SATS, EUR in metadata +2025-11-12 * "Payment" + Assets:Bitcoin:Lightning 225033 SATS + eur-value: "200.00" + Assets:Receivable:User -225033 SATS + eur-cleared: "200.00" +``` + +**Recommendation**: Architecture A (EUR primary) because: +1. Most receivables created in EUR +2. Financial reporting requirements typically in fiat +3. Tax obligations calculated in fiat +4. Aligns with current Castle metadata approach + +--- + +#### 3.2 Consider Separate Ledger for Cryptocurrency Holdings + +**Advanced Approach**: Separate cryptocurrency movements from fiat accounting + +**Main Ledger** (EUR-denominated): +```beancount +2025-11-12 * "Payment received from user" + Assets:Bitcoin-Custody:User-375ec158 200.00 EUR + Assets:Receivable:User-375ec158 -200.00 EUR +``` + +**Cryptocurrency Sub-Ledger** (SATS-denominated): +```beancount +2025-11-12 * "Lightning payment received" + Assets:Bitcoin:Lightning:Castle 225033 SATS + Assets:Bitcoin:Custody:User-375ec 225033 SATS +``` + +**Benefits**: +- ✅ Clean separation of concerns +- ✅ Cryptocurrency movements tracked independently +- ✅ Fiat accounting unaffected by Bitcoin volatility +- ✅ Can generate separate financial statements + +**Drawbacks**: +- ❌ Increased complexity +- ❌ Reconciliation between ledgers required +- ❌ Two sets of books to maintain + +--- + +## Code Files Requiring Changes + +### High Priority (Immediate Fixes) + +1. **`beancount_format.py:739-760`** + - Remove zero-amount postings + - Make payable posting conditional + +2. **`beancount_format.py:692`** + - Rename function to `format_receivable_payment_entry` + +### Medium Priority (Compliance) + +3. **`tasks.py:235-310`** + - Add exchange gain/loss calculation + - Implement payment vs. settlement logic + +4. **New file: `exchange_rates.py`** + - Create `get_current_sats_eur_rate()` function + - Implement price feed integration + +5. **`beancount_format.py`** + - Create new `format_net_settlement_entry()` for true netting + - Create `format_receivable_payment_entry()` for simple payments + +--- + +## Testing Requirements + +### Test Case 1: Simple Receivable Payment (No Payable) + +**Setup**: +- User has receivable: 200.00 EUR +- User has payable: 0.00 EUR +- User pays: 225,033 SATS + +**Expected Entry** (after fixes): +```beancount +2025-11-12 * "Lightning payment from user" + Assets:Bitcoin:Lightning 200.00 EUR + sats-received: "225033" + payment-hash: "8d080ec4..." + Assets:Receivable:User -200.00 EUR + sats-cleared: "225033" +``` + +**Verify**: +- ✅ Only 2 postings (no zero-amount payable) +- ✅ Entry balances +- ✅ SATS tracked in metadata +- ✅ User balance becomes 0 (both EUR and SATS) + +--- + +### Test Case 2: True Net Settlement + +**Setup**: +- User has receivable: 555.00 EUR +- User has payable: 38.00 EUR +- Net owed: 517.00 EUR +- User pays: 565,251 SATS (worth 517.00 EUR) + +**Expected Entry**: +```beancount +2025-11-12 * "Net settlement via Lightning" + Assets:Bitcoin:Lightning 517.00 EUR + sats-received: "565251" + payment-hash: "abc123..." + Assets:Receivable:User -555.00 EUR + sats-portion: "565251" + Liabilities:Payable:User 38.00 EUR +``` + +**Verify**: +- ✅ 3 postings (receivable + payable cleared) +- ✅ Net amount = receivable - payable +- ✅ Both balances become 0 +- ✅ Mathematically balanced + +--- + +### Test Case 3: Exchange Gain/Loss (Future) + +**Setup**: +- User has receivable: 200.00 EUR (created at 1,125 sats/EUR) +- User pays: 225,033 SATS (now worth 199.50 EUR at market) +- Exchange loss: 0.50 EUR + +**Expected Entry** (with exchange tracking): +```beancount +2025-11-12 * "Lightning payment with exchange loss" + Assets:Bitcoin:Lightning 199.50 EUR + sats-received: "225033" + market-rate: "0.000886" + Expenses:Foreign-Exchange-Loss 0.50 EUR + Assets:Receivable:User -200.00 EUR +``` + +**Verify**: +- ✅ Bitcoin recorded at fair market value +- ✅ Exchange loss recognized +- ✅ Receivable cleared at book value +- ✅ Entry balances + +--- + +## Conclusion + +### Summary of Issues + +| Issue | Severity | Accounting Impact | Recommended Action | +|-------|----------|-------------------|-------------------| +| Zero-amount postings | Low | Presentation only | Remove immediately | +| Redundant SATS tracking | Low | Storage/efficiency | Choose one method | +| No exchange gain/loss | **High** | Financial accuracy | Implement for compliance | +| Semantic misuse of @ | Medium | Audit clarity | Consider EUR-only positions | +| Misnamed function | Low | Code clarity | Rename function | + +### Professional Assessment + +**Is this "best practice" accounting?** +**No**, this implementation deviates from traditional accounting standards in several ways. + +**Is it acceptable for Castle's use case?** +**Yes, with modifications**, it's a reasonable pragmatic solution for a novel problem (cryptocurrency payments of fiat debts). + +**Critical improvements needed**: +1. ✅ Remove zero-amount postings (easy fix, professional presentation) +2. ✅ Implement exchange gain/loss tracking (required for compliance) +3. ✅ Separate payment vs. settlement logic (accuracy and clarity) + +**The fundamental challenge**: Traditional accounting wasn't designed for this scenario. There is no established "standard" for recording cryptocurrency payments of fiat-denominated receivables. Castle's approach is functional, but should be refined to align better with accounting principles where possible. + +### Next Steps + +1. **Week 1**: Implement Priority 1 fixes (remove zero postings, rename function) +2. **Week 2-3**: Design and implement exchange gain/loss tracking +3. **Week 4**: Add payment vs. settlement logic +4. **Ongoing**: Monitor regulatory guidance on cryptocurrency accounting + +--- + +## References + +- **FASB ASC 830**: Foreign Currency Matters +- **IAS 21**: The Effects of Changes in Foreign Exchange Rates +- **FASB Concept Statement No. 2**: Qualitative Characteristics of Accounting Information +- **ASC 105-10-05**: Substance Over Form +- **Beancount Documentation**: http://furius.ca/beancount/doc/index +- **Castle Extension**: `docs/SATS-EQUIVALENT-METADATA.md` +- **BQL Analysis**: `docs/BQL-BALANCE-QUERIES.md` + +--- + +**Document Version**: 1.0 +**Last Updated**: 2025-01-12 +**Next Review**: After Priority 1 fixes implemented + +--- + +*This analysis was prepared for internal review and development planning. It represents a professional accounting assessment of the current implementation and should be used to guide improvements to Castle's payment recording system.* diff --git a/docs/BQL-PRICE-NOTATION-SOLUTION.md b/docs/BQL-PRICE-NOTATION-SOLUTION.md new file mode 100644 index 0000000..24cd073 --- /dev/null +++ b/docs/BQL-PRICE-NOTATION-SOLUTION.md @@ -0,0 +1,529 @@ +# BQL Price Notation Solution for SATS Tracking + +**Date**: 2025-01-12 +**Status**: Testing +**Context**: Explore price notation as alternative to metadata for SATS tracking + +--- + +## Problem Recap + +Current approach stores SATS in metadata: +```beancount +2025-11-10 * "Groceries" + Expenses:Food -360.00 EUR + sats-equivalent: 337096 + Liabilities:Payable:User-abc 360.00 EUR + sats-equivalent: 337096 +``` + +**Issue**: BQL cannot access metadata, so balance queries require manual aggregation. + +--- + +## Solution: Use Price Notation + +### Proposed Format + +Post in actual transaction currency (EUR) with SATS as price: + +```beancount +2025-11-10 * "Groceries" + Expenses:Food -360.00 EUR @@ 337096 SATS + Liabilities:Payable:User-abc 360.00 EUR @@ 337096 SATS +``` + +**What this means**: +- Primary amount: `-360.00 EUR` (the actual transaction currency) +- Total price: `337096 SATS` (the bitcoin equivalent value) +- Transaction integrity preserved (posted in EUR as it occurred) +- SATS tracked as price (queryable by BQL) + +--- + +## Price Notation Options + +### Option 1: Per-Unit Price (`@`) + +```beancount +Expenses:Food -360.00 EUR @ 936.38 SATS +``` + +**What it means**: Each EUR is worth 936.38 SATS +**Total calculation**: 360 × 936.38 = 337,096.8 SATS +**Precision**: May introduce rounding (336,696.8 vs 337,096) + +### Option 2: Total Price (`@@`) ✅ RECOMMENDED + +```beancount +Expenses:Food -360.00 EUR @@ 337096 SATS +``` + +**What it means**: Total transaction value is 337,096 SATS +**Total calculation**: Exact 337,096 SATS (no rounding) +**Precision**: Preserves exact SATS amount from original calculation + +**Why `@@` is better for Castle:** +- ✅ Preserves exact SATS amount (no rounding errors) +- ✅ Matches current metadata storage exactly +- ✅ Clearer intent: "this transaction equals X SATS total" + +--- + +## How BQL Handles Prices + +### Available Price Columns + +From BQL schema: +- `price_number` - The numeric price amount (Decimal) +- `price_currency` - The currency of the price (str) +- `position` - Full posting (includes price) +- `WEIGHT(position)` - Function that returns balance weight + +### BQL Query Capabilities + +**Test Query 1: Access price directly** +```sql +SELECT account, number, currency, price_number, price_currency +WHERE account ~ 'User-375ec158' + AND price_currency = 'SATS'; +``` + +**Expected Result** (if price notation works): +```json +{ + "rows": [ + ["Liabilities:Payable:User-abc", "360.00", "EUR", "337096", "SATS"] + ] +} +``` + +**Test Query 2: Aggregate SATS from prices** +```sql +SELECT account, + SUM(price_number) as total_sats +WHERE account ~ 'User-' + AND price_currency = 'SATS' + AND flag != '!' +GROUP BY account; +``` + +**Expected Result**: +```json +{ + "rows": [ + ["Liabilities:Payable:User-abc", "337096"] + ] +} +``` + +--- + +## Testing Plan + +### Step 1: Run Metadata Test + +```bash +cd /home/padreug/projects/castle-beancounter +./test_metadata_simple.sh +``` + +**What to look for**: +- Does `meta` column exist in response? +- Is `sats-equivalent` accessible in the data? + +**If YES**: Metadata IS accessible, simpler solution available +**If NO**: Proceed with price notation approach + +### Step 2: Test Current Data Structure + +```bash +./test_bql_metadata.sh +``` + +This runs 6 tests: +1. Check metadata column +2. Check price columns +3. Basic position query +4. Test WEIGHT function +5. Aggregate positions +6. Aggregate weights + +**What to look for**: +- Which columns are available? +- What does `position` return for entries with prices? +- Can we access `price_number` and `price_currency`? + +### Step 3: Create Test Ledger Entry + +Add one test entry to your ledger: + +```beancount +2025-01-12 * "TEST: Price notation test" + Expenses:Test:PriceNotation -100.00 EUR @@ 93600 SATS + Liabilities:Payable:User-TEST 100.00 EUR @@ 93600 SATS +``` + +Then query: +```bash +curl -s "http://localhost:3333/castle-ledger/api/query" \ + -G \ + --data-urlencode "query_string=SELECT account, position, price_number, price_currency WHERE account ~ 'TEST'" \ + | jq '.' +``` + +**Expected if working**: +```json +{ + "data": { + "rows": [ + ["Expenses:Test:PriceNotation", "-100.00 EUR @@ 93600 SATS", "93600", "SATS"], + ["Liabilities:Payable:User-TEST", "100.00 EUR @@ 93600 SATS", "93600", "SATS"] + ], + "types": [ + {"name": "account", "type": "str"}, + {"name": "position", "type": "Position"}, + {"name": "price_number", "type": "Decimal"}, + {"name": "price_currency", "type": "str"} + ] + } +} +``` + +--- + +## Migration Strategy (If Price Notation Works) + +### Phase 1: Test on Sample Data + +1. Create test ledger with mix of formats +2. Verify BQL can query price_number +3. Verify aggregation accuracy +4. Compare with manual method results + +### Phase 2: Write Migration Script + +```python +#!/usr/bin/env python3 +""" +Migrate metadata sats-equivalent to price notation. + +Converts: + Expenses:Food -360.00 EUR + sats-equivalent: 337096 + +To: + Expenses:Food -360.00 EUR @@ 337096 SATS +""" + +import re +from pathlib import Path + +def migrate_entry(entry_lines): + """Migrate a single transaction entry.""" + result = [] + current_posting = None + sats_value = None + + for line in entry_lines: + # Check if this is a posting line + if re.match(r'^\s{2,}\w+:', line): + # If we have pending sats from previous posting, add it + if current_posting and sats_value: + # Add @@ notation to posting + posting = current_posting.rstrip() + posting += f" @@ {sats_value} SATS\n" + result.append(posting) + current_posting = None + sats_value = None + else: + if current_posting: + result.append(current_posting) + current_posting = line + + # Check if this is sats-equivalent metadata + elif 'sats-equivalent:' in line: + match = re.search(r'sats-equivalent:\s*(-?\d+)', line) + if match: + sats_value = match.group(1) + # Don't include metadata line in result + + else: + # Other lines (date, narration, other metadata) + if current_posting and sats_value: + posting = current_posting.rstrip() + posting += f" @@ {sats_value} SATS\n" + result.append(posting) + current_posting = None + sats_value = None + elif current_posting: + result.append(current_posting) + current_posting = None + + result.append(line) + + # Handle last posting + if current_posting and sats_value: + posting = current_posting.rstrip() + posting += f" @@ {sats_value} SATS\n" + result.append(posting) + elif current_posting: + result.append(current_posting) + + return result + +def migrate_ledger(input_file, output_file): + """Migrate entire ledger file.""" + with open(input_file, 'r') as f: + lines = f.readlines() + + result = [] + current_entry = [] + in_transaction = False + + for line in lines: + # Transaction start + if re.match(r'^\d{4}-\d{2}-\d{2}\s+[*!]', line): + in_transaction = True + current_entry = [line] + + # Empty line ends transaction + elif in_transaction and line.strip() == '': + current_entry.append(line) + migrated = migrate_entry(current_entry) + result.extend(migrated) + current_entry = [] + in_transaction = False + + # Inside transaction + elif in_transaction: + current_entry.append(line) + + # Outside transaction + else: + result.append(line) + + # Handle last entry if file doesn't end with blank line + if current_entry: + migrated = migrate_entry(current_entry) + result.extend(migrated) + + with open(output_file, 'w') as f: + f.writelines(result) + +if __name__ == '__main__': + import sys + if len(sys.argv) != 3: + print("Usage: migrate_ledger.py ") + sys.exit(1) + + migrate_ledger(sys.argv[1], sys.argv[2]) + print(f"Migrated {sys.argv[1]} -> {sys.argv[2]}") +``` + +### Phase 3: Update Balance Query Methods + +Replace `get_user_balance_bql()` with price-based version: + +```python +async def get_user_balance_bql(self, user_id: str) -> Dict[str, Any]: + """ + Get user balance using price notation (SATS stored as @@ price). + + Returns: + { + "balance": int (sats from price_number), + "fiat_balances": {"EUR": Decimal("100.50")}, + "accounts": [{"account": "...", "sats": 150000}] + } + """ + user_id_prefix = user_id[:8] + + # Query: Get EUR positions with SATS prices + query = f""" + SELECT + account, + number as eur_amount, + price_number as sats_amount + WHERE account ~ ':User-{user_id_prefix}' + AND (account ~ 'Payable' OR account ~ 'Receivable') + AND flag != '!' + AND price_currency = 'SATS' + """ + + result = await self.query_bql(query) + + total_sats = 0 + fiat_balances = {} + accounts_map = {} + + for row in result["rows"]: + account_name, eur_amount, sats_amount = row + + # Parse amounts + sats = int(Decimal(sats_amount)) if sats_amount else 0 + eur = Decimal(eur_amount) if eur_amount else Decimal(0) + + total_sats += sats + + # Aggregate fiat + if eur != 0: + if "EUR" not in fiat_balances: + fiat_balances["EUR"] = Decimal(0) + fiat_balances["EUR"] += eur + + # Track per account + if account_name not in accounts_map: + accounts_map[account_name] = {"account": account_name, "sats": 0} + accounts_map[account_name]["sats"] += sats + + return { + "balance": total_sats, + "fiat_balances": fiat_balances, + "accounts": list(accounts_map.values()) + } +``` + +### Phase 4: Validation + +1. Run both methods in parallel +2. Compare results for all users +3. Log any discrepancies +4. Investigate and fix differences +5. Once validated, switch to BQL method + +--- + +## Advantages of Price Notation Approach + +### 1. BQL Compatibility ✅ +- `price_number` is a standard BQL column +- Can aggregate: `SUM(price_number)` +- Can filter: `WHERE price_currency = 'SATS'` + +### 2. Transaction Integrity ✅ +- Post in actual transaction currency (EUR) +- SATS as secondary value (price) +- Proper accounting: source currency preserved + +### 3. Beancount Features ✅ +- Price database automatically updated +- Can query historical EUR/SATS rates +- Reports can show both EUR and SATS values + +### 4. Performance ✅ +- BQL filters at source (no fetching all entries) +- Direct column access (no metadata parsing) +- Efficient aggregation (database-level) + +### 5. Reporting Flexibility ✅ +- Show EUR amounts in reports +- Show SATS equivalents alongside +- Filter by either currency +- Calculate gains/losses if SATS price changes + +--- + +## Potential Issues and Solutions + +### Issue 1: Price vs Cost Confusion + +**Problem**: Beancount distinguishes between `@` price and `{}` cost +**Solution**: Always use price (`@` or `@@`), never cost (`{}`) + +**Why**: +- Cost is for tracking cost basis (investments, capital gains) +- Price is for conversion rates (what we need) + +### Issue 2: Precision Loss with `@` + +**Problem**: Per-unit price may have rounding +```beancount +360.00 EUR @ 936.38 SATS = 336,696.8 SATS (not 337,096) +``` + +**Solution**: Always use `@@` total price +```beancount +360.00 EUR @@ 337096 SATS = 337,096 SATS (exact) +``` + +### Issue 3: Negative Numbers + +**Problem**: How to handle negative EUR with positive SATS? +```beancount +-360.00 EUR @@ ??? SATS +``` + +**Solution**: Price is always positive (it's a rate, not an amount) +```beancount +-360.00 EUR @@ 337096 SATS ✅ Correct +``` + +The sign applies to the position, price is the conversion factor. + +### Issue 4: Historical Data + +**Problem**: Existing entries have metadata, not prices + +**Solution**: Migration script (see Phase 2) +- One-time conversion +- Validate with checksums +- Keep backup of original + +--- + +## Testing Checklist + +- [ ] Run `test_metadata_simple.sh` - Check if metadata is accessible +- [ ] Run `test_bql_metadata.sh` - Full BQL capabilities test +- [ ] Add test entry with `@@` notation to ledger +- [ ] Query test entry with BQL to verify price_number access +- [ ] Compare aggregation: metadata vs price notation +- [ ] Test negative amounts with prices +- [ ] Test zero amounts +- [ ] Test multi-currency scenarios (EUR, USD with SATS prices) +- [ ] Verify price database is populated correctly +- [ ] Check that WEIGHT() function returns SATS value +- [ ] Validate balances match current manual method + +--- + +## Decision Matrix + +| Criteria | Metadata | Price Notation | Winner | +|----------|----------|----------------|--------| +| BQL Queryable | ❌ No | ✅ Yes | Price | +| Transaction Integrity | ✅ EUR first | ✅ EUR first | Tie | +| SATS Precision | ✅ Exact int | ✅ Exact (with @@) | Tie | +| Migration Effort | ✅ None | ⚠️ Script needed | Metadata | +| Performance | ❌ Manual loop | ✅ BQL optimized | Price | +| Beancount Standard | ⚠️ Non-standard | ✅ Standard feature | Price | +| Reporting Flexibility | ⚠️ Limited | ✅ Both currencies | Price | +| Future Proof | ⚠️ Custom | ✅ Standard | Price | + +**Recommendation**: **Price Notation** if tests confirm BQL can access `price_number` + +--- + +## Next Steps + +1. **Run tests** (test_metadata_simple.sh and test_bql_metadata.sh) +2. **Review results** - Can BQL access price_number? +3. **Add test entry** with @@ notation +4. **Query test entry** - Verify aggregation works +5. **If successful**: + - Write full migration script + - Test on copy of production ledger + - Validate balances match + - Schedule migration (maintenance window) + - Update balance query methods + - Deploy and monitor +6. **If unsuccessful**: + - Document why price notation doesn't work + - Consider Beancount plugin approach + - Or accept manual aggregation with caching + +--- + +**Document Status**: Awaiting test results +**Next Action**: Run test scripts and report findings diff --git a/docs/SATS-EQUIVALENT-METADATA.md b/docs/SATS-EQUIVALENT-METADATA.md new file mode 100644 index 0000000..48ab36c --- /dev/null +++ b/docs/SATS-EQUIVALENT-METADATA.md @@ -0,0 +1,386 @@ +# SATS-Equivalent Metadata Field + +**Date**: 2025-01-12 +**Status**: Current Architecture +**Location**: Beancount posting metadata + +--- + +## Overview + +The `sats-equivalent` metadata field is Castle's solution for **dual-currency tracking** in a fiat-denominated ledger. It preserves Bitcoin (satoshi) amounts alongside fiat currency amounts without violating accounting principles or creating multi-currency complexity in position balances. + +### Quick Summary + +- **Purpose**: Track Bitcoin/Lightning amounts in a EUR-denominated ledger +- **Location**: Beancount posting metadata (not position amounts) +- **Format**: String containing absolute satoshi amount (e.g., `"337096"`) +- **Primary Use**: Calculate user balances in satoshis (Castle's primary currency) +- **Key Principle**: Satoshis are for reference; EUR is the actual transaction currency + +--- + +## The Problem: Dual-Currency Tracking + +Castle needs to track both: +1. **Fiat amounts** (EUR, USD) - The actual transaction currency +2. **Bitcoin amounts** (satoshis) - The Lightning Network settlement currency + +### Why Not Just Use SATS as Position Amounts? + +**Accounting Reality**: When a user pays €36.93 cash for groceries, the transaction is denominated in EUR, not Bitcoin. Recording it as Bitcoin would: +- ❌ Misrepresent the actual transaction +- ❌ Create exchange rate volatility issues +- ❌ Complicate traditional accounting reconciliation +- ❌ Make fiat-based reporting difficult + +**Castle's Philosophy**: Record transactions in their **actual currency**, with Bitcoin as supplementary data. + +--- + +## Architecture: EUR-Primary Format + +### Current Ledger Format + +```beancount +2025-11-10 * "Groceries (36.93 EUR)" #expense-entry + Expenses:Food:Supplies 36.93 EUR + sats-equivalent: "39669" + reference: "cash-payment-abc123" + Liabilities:Payable:User-5987ae95 -36.93 EUR + sats-equivalent: "39669" +``` + +**Key Components:** +- **Position Amount**: `36.93 EUR` - The actual transaction amount +- **Metadata**: `sats-equivalent: "39669"` - The Bitcoin equivalent at time of transaction +- **Sign**: The sign (debit/credit) is on the EUR amount; sats-equivalent is always absolute value + +### How It's Created + +In `views_api.py:839`: + +```python +# If fiat currency is provided, use EUR-based format +if fiat_currency and fiat_amount: + # EUR-based posting (current architecture) + posting_metadata["sats-equivalent"] = str(abs(line.amount)) + + # Apply the sign from line.amount to fiat_amount + signed_fiat_amount = fiat_amount if line.amount >= 0 else -fiat_amount + + posting = { + "account": account.name, + "amount": f"{signed_fiat_amount:.2f} {fiat_currency}", + "meta": posting_metadata if posting_metadata else None + } +``` + +**Critical Details:** +- `line.amount` is always in satoshis internally +- The sign (debit/credit) transfers to the fiat amount +- `sats-equivalent` stores the **absolute value** of the satoshi amount +- Sign interpretation depends on account type (Asset/Liability/etc.) + +--- + +## Usage: Balance Calculation + +### Primary Use Case: User Balances + +Castle's core function is tracking **who owes whom** in satoshis. The `sats-equivalent` metadata enables this. + +**Flow** (`fava_client.py:220-248`): + +```python +# Parse posting amount (EUR/USD) +fiat_match = re.match(r'^(-?[\d.]+)\s+([A-Z]{3})$', amount_str) +if fiat_match and fiat_match.group(2) in ('EUR', 'USD', 'GBP'): + fiat_amount = Decimal(fiat_match.group(1)) + fiat_currency = fiat_match.group(2) + + # Track fiat balance + fiat_balances[fiat_currency] += fiat_amount + + # Extract SATS equivalent from metadata + posting_meta = posting.get("meta", {}) + sats_equiv = posting_meta.get("sats-equivalent") + if sats_equiv: + # Apply the sign from fiat_amount to sats_equiv + sats_amount = int(sats_equiv) if fiat_amount > 0 else -int(sats_equiv) + total_sats += sats_amount +``` + +**Sign Interpretation:** +- EUR amount is `36.93` (positive/debit) → sats is `+39669` +- EUR amount is `-36.93` (negative/credit) → sats is `-39669` + +### Secondary Use: Journal Entry Display + +When displaying transactions to users (`views_api.py:747-751`): + +```python +# Extract sats equivalent from metadata +posting_meta = first_posting.get("meta", {}) +sats_equiv = posting_meta.get("sats-equivalent") +if sats_equiv: + amount_sats = abs(int(sats_equiv)) +``` + +This allows the UI to show both EUR and SATS amounts for each transaction. + +--- + +## Why Metadata Instead of Positions? + +### The BQL Limitation + +Beancount Query Language (BQL) **cannot access metadata**. This means: + +```sql +-- ✅ This works (queries position amounts): +SELECT account, sum(position) WHERE account ~ 'User-5987ae95' +-- Returns: EUR positions (not useful for satoshi balances) + +-- ❌ This is NOT possible: +SELECT account, sum(meta["sats-equivalent"]) WHERE account ~ 'User-5987ae95' +-- Error: BQL cannot access metadata +``` + +### Why Castle Accepts This Trade-off + +**Performance Analysis** (see `docs/BQL-BALANCE-QUERIES.md`): +1. **Caching solves the bottleneck**: 60-80% performance improvement from caching account/permission lookups +2. **Iteration is necessary anyway**: Even with BQL, we'd need to iterate postings to access metadata +3. **Manual aggregation is fast**: The actual summation is not the bottleneck +4. **Database queries are the bottleneck**: Solved by Phase 1 caching, not BQL + +**Architectural Correctness > Query Performance**: +- ✅ Transactions recorded in their actual currency +- ✅ No artificial multi-currency positions +- ✅ Clean accounting reconciliation +- ✅ Exchange rate changes don't affect historical records + +--- + +## Alternative Considered: Price Notation + +### Price Notation Format (Not Implemented) + +```beancount +2025-11-10 * "Groceries" + Expenses:Food -360.00 EUR @@ 337096 SATS + Liabilities:Payable:User-abc 360.00 EUR @@ 337096 SATS +``` + +**Pros:** +- ✅ BQL can query prices (enables BQL aggregation) +- ✅ Standard Beancount syntax +- ✅ SATS trackable via price database + +**Cons:** +- ❌ Semantically incorrect: `@@` means "total price paid", not "equivalent value" +- ❌ Implies currency conversion happened (it didn't) +- ❌ Confuses future readers about transaction nature +- ❌ Complicates Beancount's price database + +**Decision**: Metadata is more semantically correct for "reference value" than price notation. + +See `docs/BQL-PRICE-NOTATION-SOLUTION.md` for full analysis. + +--- + +## Data Flow Example + +### User Adds Expense + +**User Action**: "I paid €36.93 cash for groceries" + +**Castle's Internal Representation**: +```python +# User provides or Castle calculates: +fiat_amount = Decimal("36.93") # EUR +fiat_currency = "EUR" +amount_sats = 39669 # Calculated from exchange rate + +# Create journal entry line: +line = CreateEntryLine( + account_id=expense_account.id, + amount=amount_sats, # Internal: always satoshis + metadata={ + "fiat_currency": "EUR", + "fiat_amount": "36.93" + } +) +``` + +**Beancount Entry Created** (`views_api.py:835-849`): +```beancount +2025-11-10 * "Groceries (36.93 EUR)" #expense-entry + Expenses:Food:Supplies 36.93 EUR + sats-equivalent: "39669" + Liabilities:Payable:User-5987ae95 -36.93 EUR + sats-equivalent: "39669" +``` + +**Balance Calculation** (`fava_client.py:get_user_balance`): +```python +# Iterate all postings for user accounts +# For each posting: +# - Parse EUR amount: -36.93 EUR (credit to liability) +# - Extract sats-equivalent: "39669" +# - Apply sign: -36.93 is negative → sats = -39669 +# - Accumulate: user_balance_sats += -39669 + +# Result: negative balance = Castle owes user +``` + +**User Balance Response**: +```json +{ + "user_id": "5987ae95", + "balance": -39669, // Castle owes user 39,669 sats + "fiat_balances": { + "EUR": "-36.93" // Castle owes user €36.93 + } +} +``` + +--- + +## Implementation Details + +### Where It's Set + +**Primary Location**: `views_api.py:835-849` (Creating journal entries) + +All EUR-based postings get `sats-equivalent` metadata: +- Expense entries (user adds liability) +- Receivable entries (admin records what user owes) +- Revenue entries (direct income) +- Payment entries (settling balances) + +### Where It's Read + +**Primary Location**: `fava_client.py:239-247` (Balance calculation) + +Used in: +1. `get_user_balance()` - Calculate individual user balance +2. `get_all_user_balances()` - Calculate all user balances +3. `get_journal_entries()` - Display transaction amounts + +### Data Type and Format + +- **Type**: String (Beancount metadata values must be strings or numbers) +- **Format**: Absolute value, no sign, no decimal point +- **Examples**: + - ✅ `"39669"` (correct) + - ✅ `"1000000"` (1M sats) + - ❌ `"-39669"` (incorrect: sign goes on EUR amount) + - ❌ `"396.69"` (incorrect: satoshis are integers) + +--- + +## Key Principles + +### 1. Record in Transaction Currency + +```beancount +# ✅ CORRECT: User paid EUR, record in EUR +Expenses:Food 36.93 EUR + sats-equivalent: "39669" + +# ❌ WRONG: Recording Bitcoin when user paid cash +Expenses:Food 39669 SATS {36.93 EUR} +``` + +### 2. Preserve Historical Values + +The `sats-equivalent` is the **exact satoshi amount at transaction time**. It does NOT change when exchange rates change. + +**Example:** +- 2025-11-10: User pays €36.93 → 39,669 sats (rate: 1074.19 sats/EUR) +- 2025-11-15: Exchange rate changes to 1100 sats/EUR +- **Metadata stays**: `sats-equivalent: "39669"` ✅ +- **If we used current rate**: Would become 40,623 sats ❌ + +### 3. Separate Fiat and Sats Balances + +Castle tracks TWO independent balances: +- **Satoshi balance**: Sum of `sats-equivalent` metadata (primary) +- **Fiat balances**: Sum of EUR/USD position amounts (secondary) + +These are calculated independently and don't cross-convert. + +### 4. Absolute Values in Metadata + +The sign (debit/credit) lives on the position amount, NOT the metadata. + +```beancount +# Debit (expense increases): +Expenses:Food 36.93 EUR # Positive + sats-equivalent: "39669" # Absolute value + +# Credit (liability increases): +Liabilities:Payable -36.93 EUR # Negative + sats-equivalent: "39669" # Same absolute value +``` + +--- + +## Migration Path + +### Future: If We Change to SATS-Primary Format + +**Hypothetical future format:** +```beancount +; SATS as position, EUR as cost: +2025-11-10 * "Groceries" + Expenses:Food 39669 SATS {36.93 EUR} + Liabilities:Payable:User-abc -39669 SATS {36.93 EUR} +``` + +**Benefits:** +- ✅ BQL can query SATS directly +- ✅ No metadata parsing needed +- ✅ Standard Beancount cost accounting + +**Migration Script** (conceptual): +```python +# Read all postings with sats-equivalent metadata +# For each posting: +# - Extract sats from metadata +# - Extract EUR from position +# - Rewrite as: " SATS { EUR}" +``` + +**Decision**: Not implementing now because: +1. Current architecture is semantically correct +2. Performance is acceptable with caching +3. Migration would break existing tooling +4. EUR-primary aligns with accounting reality + +--- + +## Related Documentation + +- `docs/BQL-BALANCE-QUERIES.md` - Why BQL can't query metadata and performance analysis +- `docs/BQL-PRICE-NOTATION-SOLUTION.md` - Alternative using price notation (not implemented) +- `beancount_format.py` - Functions that create entries with sats-equivalent metadata +- `fava_client.py:get_user_balance()` - How metadata is parsed for balance calculation + +--- + +## Technical Summary + +**Field**: `sats-equivalent` +**Type**: Metadata (string) +**Location**: Beancount posting metadata +**Format**: Absolute satoshi amount as string (e.g., `"39669"`) +**Purpose**: Track Bitcoin equivalent of fiat transactions +**Primary Use**: Calculate user satoshi balances +**Sign Handling**: Inherits from position amount (EUR/USD) +**Queryable via BQL**: ❌ No (BQL cannot access metadata) +**Performance**: ✅ Acceptable with caching (60-80% improvement) +**Architectural Status**: ✅ Current production format +**Future Migration**: Possible to SATS-primary if needed