Implemented query_bql() method to enable efficient Beancount Query Language
(BQL) queries against Fava API. This is the foundation for replacing manual
balance aggregation (115 lines) with optimized BQL queries.
Benefits:
- Efficient server-side filtering and aggregation
- 5-10x expected performance improvement
- Cleaner, more maintainable code
Next: Implement get_user_balance_bql() using this method.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Since Castle extension has not been released yet, squash all database migrations
for cleaner initial deployments. This reduces migration complexity and improves
maintainability.
Changes:
- Squash migrations m001-m016 into single m001_initial migration
- Reduced from 651 lines (16 functions) to 327 lines (1 function)
- 50% code reduction, 93.75% fewer migration functions
Final database schema (7 tables):
- castle_accounts: Chart of accounts with 40+ default accounts
- castle_extension_settings: Castle configuration
- castle_user_wallet_settings: User wallet associations
- castle_manual_payment_requests: Payment approval workflow
- castle_balance_assertions: Reconciliation with Beancount integration
- castle_user_equity_status: Equity eligibility tracking
- castle_account_permissions: Granular access control
Tables removed (intentionally):
- castle_journal_entries: Now managed by Fava/Beancount (dropped in m016)
- castle_entry_lines: Now managed by Fava/Beancount (dropped in m016)
New migration includes:
- All 7 tables in their final state
- All indexes properly prefixed with idx_castle_
- All foreign key constraints
- 40+ default accounts with hierarchical names (Assets:Bitcoin:Lightning, etc.)
- Comprehensive documentation
Files:
- migrations.py: Single clean m001_initial migration
- migrations_old.py.bak: Backup of original 16 migrations for reference
- MIGRATION_SQUASH_SUMMARY.md: Complete documentation of squash process
Benefits:
- Simpler initial deployments (1 migration instead of 16)
- Easier to understand final schema
- Faster migration execution
- Cleaner codebase
See MIGRATION_SQUASH_SUMMARY.md for full details and testing instructions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit fixes two critical bugs in the user account creation flow:
1. **Always check/create in Fava regardless of Castle DB status**
- Previously, if an account existed in Castle DB, the function would
return early without checking if the Open directive existed in Fava
- This caused accounts to exist in Castle DB but not in Beancount
- Now we always check Fava and create Open directives if needed
2. **Fix Open directive insertion to preserve metadata**
- The insertion logic now skips over metadata lines when finding
the insertion point
- Prevents new Open directives from being inserted between existing
directives and their metadata, which was causing orphaned metadata
3. **Add comprehensive logging**
- Added detailed logging with [ACCOUNT CHECK], [FAVA CHECK],
[FAVA CREATE], [CASTLE DB], and [WALLET UPDATE] prefixes
- Makes it easier to trace account creation flow and debug issues
4. **Fix Fava filename handling**
- Now queries /api/options to get the Beancount file path dynamically
- Fixes "Parameter 'filename' is missing" errors with /api/source
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Balance assertions now use a hybrid architecture where Beancount is the source
of truth for validation, while Castle stores metadata for UI convenience.
Backend changes:
- Add format_balance() function to beancount_format.py for formatting balance directives
- Update POST /api/v1/assertions to write balance directive to Beancount first (via Fava)
- Store metadata in Castle DB (created_by, tolerance, notes) for UI features
- Validate assertions immediately by querying Fava for actual balance
Frontend changes:
- Update dialog description to explain Beancount validation
- Update button tooltip to clarify balance assertions are written to Beancount
- Update empty state message to mention Beancount checkpoints
Benefits:
- Single source of truth (Beancount ledger file)
- Automatic validation by Beancount
- Best of both worlds: robust validation + friendly UI
See misc-docs/BALANCE-ASSERTIONS-HYBRID-APPROACH.md for full documentation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update FavaClient.add_account() to use PUT /api/source instead of POST /api/add_entries
because Fava does not support Open directives via add_entries endpoint.
Changes:
- Fetch current Beancount source file via GET /api/source
- Check if account already exists to avoid duplicates
- Format Open directive as plain text (not JSON)
- Insert directive after existing Open directives
- Update source file via PUT /api/source with sha256sum validation
This fixes the issue where Open directives were not being written to the Beancount file.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Enhances the journal entries API to include username information.
This is achieved by extracting the user ID from transaction
metadata or account names and retrieving the corresponding
username. A default username is provided if the user is not found.
The pending entries API is updated with the same functionality.
Prioritizes parsing amount strings in the new EUR/USD format and introduces support for metadata containing sats equivalent.
Fallbacks to legacy parsing when the new format is not detected.
This ensures correct interpretation of amount data from different sources.
This change ensures that user-specific accounts are automatically created
in the Fava/Beancount ledger when they are first requested. It checks for
the existence of the account via a Fava query and creates it via an Open
directive if it's missing. This simplifies account management and
ensures that all necessary accounts are available for transactions.
This implementation adds a new `add_account` method to the `FavaClient`
class which makes use of the /add_entries endpoint to create an account
using an Open Directive.
Ensures that user-provided reference strings for expense,
receivable, and revenue entries are sanitized before being
included as Beancount links. This prevents issues caused by
invalid characters in the links, improving compatibility with
Beancount's link format. A new utility function is introduced
to handle the sanitization process.
Adds a function to sanitize strings for use as Beancount links,
ensuring compatibility with Beancount's link restrictions.
Refactors the journal entry creation process to use EUR-based
postings when fiat currency is provided, improving accuracy
and consistency. The legacy SATS-based fallback is retained for
cases without fiat currency information.
Adjusts reference generation for Beancount entries using the
sanitized description.
Improves the Beancount import process to send SATS amounts with fiat metadata to the API, enabling automatic conversion to EUR-based postings.
Updates the API to store entries in Fava instead of the Castle DB, simplifying the JournalEntry creation process.
Adds error handling to the upload entry function.
Includes a note about imported transactions being stored in EUR with SATS in metadata.
Removes the explicit call to the record-payment API when settling a receivable.
The webhook (on_invoice_paid in tasks.py) automatically handles recording the payment in Fava, making the API call redundant. This simplifies the frontend logic.
Also, in the `showPayUserDialog` function, it now correctly identifies users who are owed payments based on a negative balance instead of a positive balance.
Introduces a function to format fiat settlement entries for Beancount, handling cash, bank transfers, and other non-lightning payments.
This allows for recording transactions in fiat currency with sats as metadata.
Updates the API endpoint to use the new function when settling receivables with fiat currencies.
Reverses the condition for displaying the settle receivable dialog.
It now correctly shows the dialog only for users with a positive balance,
which indicates they owe the castle.
Improves payment recording logic by fetching recent entries and filtering using Python, replacing the BQL query.
This addresses issues with matching against set types in BQL, enhancing reliability.
Removes excessive logging to improve readability and reduce verbosity.
Streamlines balance processing and improves logging for settlement amounts.
Adds a note about Fava's internal normalization behavior to the beancount formatting.
Adds support for parsing direct EUR/USD amount strings in the format "37.22 EUR" or "12.34 USD".
It also retrieves the SATS equivalent from the metadata if present, for the new amount format. This ensures compatibility with both the old "SATS {EUR}" format and the newer, direct fiat formats.
Updates the amount parsing logic to support a new format where fiat amounts (EUR/USD) are specified directly.
Adds support for tracking SATS equivalents from metadata when the new format is used.
Also tracks fiat amounts specified in metadata as a fallback for backward compatibility.
Reverses the calculation of net balance to correctly reflect receivables and liabilities.
Updates the expense tracking system to store payables and receivables in fiat currency within Beancount.
This ensures accurate debt representation and simplifies balance calculations.
Changes include:
- Converting `format_expense_entry` and `format_receivable_entry` to use fiat amounts.
- Introducing `format_net_settlement_entry` for net settlement payments.
- Modifying `format_payment_entry` to use cost syntax for fiat tracking.
- Adjusting Fava client to correctly process new amount formats and metadata.
- Adding average cost basis posting format
The use of fiat amounts and cost basis aims to provide better accuracy and compatibility with existing Beancount workflows.
Refactors user balance calculation to directly parse journal
entries, enhancing accuracy and efficiency. This change
eliminates reliance on direct database queries and provides a
more reliable mechanism for determining user balances.
Adds logging for debugging purposes.
Also extracts and uses fiat metadata from invoice/payment extras.
Changes the displayed balance perspective to reflect the castle's point of view instead of the user's.
This involves:
- Displaying balances as positive when the user owes the castle
- Displaying balances as negative when the castle owes the user.
This change affects how balances are calculated and displayed in both the backend logic and the frontend templates.
Ensures unique identification for receivable and
revenue entries by generating a UUID and incorporating
it into a castle reference.
This enhances tracking and linking capabilities by
providing a consistent and easily identifiable
reference across the system.
Refactors user balance calculation to use journal entries
instead of querying Fava's query endpoint.
This change allows for exclusion of voided transactions
(tagged with #voided) in addition to pending transactions
when calculating user balances, providing more accurate
balance information.
Additionally the change improves parsing of the amounts in journal entries by using regular expressions.
Updates the pending entries API to correctly parse the amount and fiat values from the amount string, which can now contain both SATS and fiat information.
This change handles different formats of the amount string, including cases where the fiat amount is present within curly braces.
Improves handling of the amount field in user entries by parsing string formats that include both SATS and fiat currency information.
This change allows extracting the SATS amount and fiat amount/currency directly from the string, accommodating different display formats.
Switches to retrieving all journal entries from Fava and filtering in the application to allow filtering by account type and user.
This provides more flexibility and control over the data being presented to the user.
Also extracts and includes relevant metadata such as entry ID, fiat amounts, and references for improved frontend display.
Ensures that voided transactions are not included in the
list of pending entries. This prevents displaying
transactions that have been cancelled or reversed,
providing a more accurate view of truly pending items.
Removes redundant metadata from entries and postings.
The cost syntax already contains fiat/exchange rate information.
Metadata such as 'created-via', 'is-equity', and payer/payee
can be inferred from transaction direction, tags, and account names.
Instead of deleting pending expense entries, marks them as voided by adding a #voided tag.
This ensures an audit trail while excluding them from balances.
Updates the Fava client to use 'params' for the delete request.
Adds functionality to interact with Fava for managing
Beancount entries, including fetching, updating, and
deleting entries directly from the Beancount ledger.
This allows for approving/rejecting pending entries
via the API by modifying the source file through Fava.
The changes include:
- Adds methods to the Fava client for fetching all journal
entries, retrieving entry context (source and hash),
updating the entry source, and deleting entries.
- Updates the pending entries API to use the Fava journal
endpoint instead of querying transactions.
- Implements entry approval and rejection using the new
Fava client methods to modify the underlying Beancount file.
Modifies balance queries to exclude pending transactions (flag='!')
and only include cleared/completed transactions (flag='*'). This
ensures accurate balance calculations by reflecting only settled transactions.
Improves the handling of pending entries by extracting and deduplicating data from Fava's query results.
Adds support for displaying fiat amounts alongside entries and extracts them from the position data in Fava.
Streamlines receivables/payables/equity checks on the frontend by relying on BQL query to supply account type metadata and tags.
Inverts the sign of Beancount balances to represent the user's perspective, where liabilities are positive and receivables are negative.
This change ensures that user balances accurately reflect the amount the castle owes the user (positive) or the amount the user owes the castle (negative). It simplifies the logic by consistently negating the Beancount balance rather than using conditional checks based on account type.
Updates journal entry flags to align with Beancount's limited flag support.
Beancount only uses cleared (*) and pending (!) flags.
Removes the VOID and FLAGGED flags and recommends using tags instead
(e.g., "! + #voided" for voided entries, "! + #review" for flagged entries).
Updates the API to reflect this change, removing the ability to directly
"reject" an expense entry via the void flag. Instead, instructs users to
add the #voided tag in Fava.
Updates reconciliation summary to count entries with voided/review tags
instead of voided/flagged flags.
Replaces direct database queries for transactions with calls to the Fava API,
centralizing transaction logic and improving data consistency.
This change removes redundant code and simplifies the API by relying on Fava
for querying transactions based on account patterns and other criteria.
Specifically, the commit introduces new methods in the FavaClient class for
querying transactions, retrieving account transactions, and retrieving user
transactions. The API endpoints are updated to utilize these methods.
Migrates balance calculation and inventory tracking to
Fava/Beancount, leveraging Fava's query API for all
accounting calculations. This simplifies the core module
and centralizes accounting logic in Fava.
Removes direct journal entry creation in favor of using Fava for accounting.
This change centralizes accounting logic in Fava, improving auditability and consistency.
It replaces direct database interactions for recording payments and settlements with calls to the Fava client.
The changes also refactor balance retrieval to fetch data from Fava.
Integrates Fava/Beancount for managing journal entries.
This change introduces functions to format entries into Beancount
format and submit them to a Fava instance.
It replaces the previous direct database entry creation with Fava
submission for expense, receivable, and revenue entries. The existing
create_journal_entry function is also updated to submit generic
journal entries to Fava.
Refactors account balance retrieval to fetch data from Fava/Beancount
for improved accounting accuracy.
Updates user balance retrieval to use Fava/Beancount data source.
Updates Castle settings ledger slug name.
Refactors the payment processing logic to submit journal entries directly to
Fava/Beancount instead of storing them in the Castle database. It queries
Fava to prevent duplicate entries. The changes include extracting fiat
metadata from the invoice, formatting the data as a Beancount transaction
using a dedicated formatting function, and submitting it to the Fava API.
Implements a new handler to process Castle invoice payments by submitting them to Fava, which in turn writes them to a Beancount file.
This approach avoids storing payment data directly in the Castle database. The handler formats the payment as a Beancount transaction, includes fiat currency if available, and queries Fava to prevent duplicate entries.
The commit also updates documentation to reflect the changes to the invoice processing workflow.
Initializes the Fava client with default settings when the Castle extension starts.
This ensures the client is ready to interact with Fava immediately and provides feedback if Fava is not configured correctly.
The client is re-initialized if the admin updates settings later.
Introduces utilities to format Castle data models into Beancount
transactions for Fava API compatibility.
Provides functions to format transactions, postings with cost basis,
expense entries, receivable entries, and payment entries.
These functions ensure data is correctly formatted for Fava's
add_entries API, including cost basis, flags, and metadata.
Implements an asynchronous HTTP client to interact with the Fava accounting API.
This client provides methods for adding journal entries, retrieving account balances,
and querying user balances, allowing the application to delegate all accounting
logic to Fava and Beancount.
