Moved BQL-BALANCE-QUERIES.md from beancounter project to Castle docs/ folder.
Updated all code references from misc-docs/ to docs/ for proper documentation
location alongside implementation.
The document contains comprehensive BQL investigation results showing that
BQL is not feasible for Castle's current ledger format where SATS are stored
in posting metadata rather than position amounts.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added detailed comments to BQL methods explaining:
- Current limitation: cannot access posting metadata (sats-equivalent)
- BQL only queries position amounts (EUR/USD)
- Manual aggregation with caching remains the recommended approach
- Future consideration if ledger format changes
Changes:
- query_bql(): Added limitation warning and future consideration note
- get_user_balance_bql(): Added "NOT CURRENTLY USED" warning
- get_all_user_balances_bql(): Added "NOT CURRENTLY USED" warning
All methods kept as reference code for future architectural changes.
See: misc-docs/BQL-BALANCE-QUERIES.md for complete analysis.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Added get_user_balance_bql() and get_all_user_balances_bql() methods
that use Beancount Query Language for efficient balance queries.
Benefits:
- Replaces 115-line manual aggregation with ~100 lines of BQL queries
- Server-side filtering and aggregation
- Expected 5-10x performance improvement
- Handles multi-currency positions (SATS, EUR, USD, GBP)
- Returns same data structure as manual methods for compatibility
Implementation:
- get_user_balance_bql(): Query single user's Payable/Receivable accounts
- get_all_user_balances_bql(): Query all users in one efficient query
- Position parsing handles both dict and string formats
- Excludes pending transactions (flag != '!')
Next steps:
1. Test BQL queries against real Castle data
2. Compare results with manual aggregation methods
3. Update call sites to use BQL methods
4. Remove manual aggregation methods after validation
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implemented query_bql() method to enable efficient Beancount Query Language
(BQL) queries against Fava API. This is the foundation for replacing manual
balance aggregation (115 lines) with optimized BQL queries.
Benefits:
- Efficient server-side filtering and aggregation
- 5-10x expected performance improvement
- Cleaner, more maintainable code
Next: Implement get_user_balance_bql() using this method.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit fixes two critical bugs in the user account creation flow:
1. **Always check/create in Fava regardless of Castle DB status**
- Previously, if an account existed in Castle DB, the function would
return early without checking if the Open directive existed in Fava
- This caused accounts to exist in Castle DB but not in Beancount
- Now we always check Fava and create Open directives if needed
2. **Fix Open directive insertion to preserve metadata**
- The insertion logic now skips over metadata lines when finding
the insertion point
- Prevents new Open directives from being inserted between existing
directives and their metadata, which was causing orphaned metadata
3. **Add comprehensive logging**
- Added detailed logging with [ACCOUNT CHECK], [FAVA CHECK],
[FAVA CREATE], [CASTLE DB], and [WALLET UPDATE] prefixes
- Makes it easier to trace account creation flow and debug issues
4. **Fix Fava filename handling**
- Now queries /api/options to get the Beancount file path dynamically
- Fixes "Parameter 'filename' is missing" errors with /api/source
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update FavaClient.add_account() to use PUT /api/source instead of POST /api/add_entries
because Fava does not support Open directives via add_entries endpoint.
Changes:
- Fetch current Beancount source file via GET /api/source
- Check if account already exists to avoid duplicates
- Format Open directive as plain text (not JSON)
- Insert directive after existing Open directives
- Update source file via PUT /api/source with sha256sum validation
This fixes the issue where Open directives were not being written to the Beancount file.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This change ensures that user-specific accounts are automatically created
in the Fava/Beancount ledger when they are first requested. It checks for
the existence of the account via a Fava query and creates it via an Open
directive if it's missing. This simplifies account management and
ensures that all necessary accounts are available for transactions.
This implementation adds a new `add_account` method to the `FavaClient`
class which makes use of the /add_entries endpoint to create an account
using an Open Directive.
Removes excessive logging to improve readability and reduce verbosity.
Streamlines balance processing and improves logging for settlement amounts.
Adds a note about Fava's internal normalization behavior to the beancount formatting.
Updates the amount parsing logic to support a new format where fiat amounts (EUR/USD) are specified directly.
Adds support for tracking SATS equivalents from metadata when the new format is used.
Also tracks fiat amounts specified in metadata as a fallback for backward compatibility.
Reverses the calculation of net balance to correctly reflect receivables and liabilities.
Updates the expense tracking system to store payables and receivables in fiat currency within Beancount.
This ensures accurate debt representation and simplifies balance calculations.
Changes include:
- Converting `format_expense_entry` and `format_receivable_entry` to use fiat amounts.
- Introducing `format_net_settlement_entry` for net settlement payments.
- Modifying `format_payment_entry` to use cost syntax for fiat tracking.
- Adjusting Fava client to correctly process new amount formats and metadata.
- Adding average cost basis posting format
The use of fiat amounts and cost basis aims to provide better accuracy and compatibility with existing Beancount workflows.
Refactors user balance calculation to directly parse journal
entries, enhancing accuracy and efficiency. This change
eliminates reliance on direct database queries and provides a
more reliable mechanism for determining user balances.
Adds logging for debugging purposes.
Also extracts and uses fiat metadata from invoice/payment extras.
Changes the displayed balance perspective to reflect the castle's point of view instead of the user's.
This involves:
- Displaying balances as positive when the user owes the castle
- Displaying balances as negative when the castle owes the user.
This change affects how balances are calculated and displayed in both the backend logic and the frontend templates.
Refactors user balance calculation to use journal entries
instead of querying Fava's query endpoint.
This change allows for exclusion of voided transactions
(tagged with #voided) in addition to pending transactions
when calculating user balances, providing more accurate
balance information.
Additionally the change improves parsing of the amounts in journal entries by using regular expressions.
Instead of deleting pending expense entries, marks them as voided by adding a #voided tag.
This ensures an audit trail while excluding them from balances.
Updates the Fava client to use 'params' for the delete request.
Adds functionality to interact with Fava for managing
Beancount entries, including fetching, updating, and
deleting entries directly from the Beancount ledger.
This allows for approving/rejecting pending entries
via the API by modifying the source file through Fava.
The changes include:
- Adds methods to the Fava client for fetching all journal
entries, retrieving entry context (source and hash),
updating the entry source, and deleting entries.
- Updates the pending entries API to use the Fava journal
endpoint instead of querying transactions.
- Implements entry approval and rejection using the new
Fava client methods to modify the underlying Beancount file.
Modifies balance queries to exclude pending transactions (flag='!')
and only include cleared/completed transactions (flag='*'). This
ensures accurate balance calculations by reflecting only settled transactions.
Improves the handling of pending entries by extracting and deduplicating data from Fava's query results.
Adds support for displaying fiat amounts alongside entries and extracts them from the position data in Fava.
Streamlines receivables/payables/equity checks on the frontend by relying on BQL query to supply account type metadata and tags.
Inverts the sign of Beancount balances to represent the user's perspective, where liabilities are positive and receivables are negative.
This change ensures that user balances accurately reflect the amount the castle owes the user (positive) or the amount the user owes the castle (negative). It simplifies the logic by consistently negating the Beancount balance rather than using conditional checks based on account type.
Replaces direct database queries for transactions with calls to the Fava API,
centralizing transaction logic and improving data consistency.
This change removes redundant code and simplifies the API by relying on Fava
for querying transactions based on account patterns and other criteria.
Specifically, the commit introduces new methods in the FavaClient class for
querying transactions, retrieving account transactions, and retrieving user
transactions. The API endpoints are updated to utilize these methods.
Implements an asynchronous HTTP client to interact with the Fava accounting API.
This client provides methods for adding journal entries, retrieving account balances,
and querying user balances, allowing the application to delegate all accounting
logic to Fava and Beancount.
