castle/DAILY_RECONCILIATION.md

# Automated Daily Reconciliation

The Castle extension includes automated daily balance checking to ensure accounting accuracy.

## Overview

The daily reconciliation task:
- Checks all balance assertions
- Identifies discrepancies
- Logs results
- Can send alerts (future enhancement)

## Manual Trigger

You can manually trigger the reconciliation check from the UI or via API:

### Via API
```bash
curl -X POST https://your-lnbits-instance.com/castle/api/v1/tasks/daily-reconciliation \
  -H "X-Api-Key: YOUR_ADMIN_KEY"
```

## Automated Scheduling

### Option 1: Cron Job (Recommended)

Add to your crontab:

```bash
# Run daily at 2 AM
0 2 * * * curl -X POST http://localhost:5000/castle/api/v1/tasks/daily-reconciliation -H "X-Api-Key: YOUR_ADMIN_KEY" >> /var/log/castle-reconciliation.log 2>&1
```

To edit crontab:
```bash
crontab -e
```

### Option 2: Systemd Timer

Create `/etc/systemd/system/castle-reconciliation.service`:
```ini
[Unit]
Description=Castle Daily Reconciliation Check
After=network.target

[Service]
Type=oneshot
User=lnbits
ExecStart=/usr/bin/curl -X POST http://localhost:5000/castle/api/v1/tasks/daily-reconciliation -H "X-Api-Key: YOUR_ADMIN_KEY"
```

Create `/etc/systemd/system/castle-reconciliation.timer`:
```ini
[Unit]
Description=Run Castle reconciliation daily

[Timer]
OnCalendar=daily
OnCalendar=02:00
Persistent=true

[Install]
WantedBy=timers.target
```

Enable and start:
```bash
sudo systemctl enable castle-reconciliation.timer
sudo systemctl start castle-reconciliation.timer
```

### Option 3: Docker/Kubernetes CronJob

For containerized deployments:

```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: castle-reconciliation
spec:
  schedule: "0 2 * * *"  # Daily at 2 AM
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: reconciliation
            image: curlimages/curl:latest
            args:
            - /bin/sh
            - -c
            - curl -X POST http://lnbits:5000/castle/api/v1/tasks/daily-reconciliation -H "X-Api-Key: ${ADMIN_KEY}"
          restartPolicy: OnFailure
```

## Response Format

The endpoint returns:

```json
{
  "task_id": "abc123",
  "timestamp": "2025-10-23T02:00:00",
  "total": 15,
  "checked": 15,
  "passed": 14,
  "failed": 1,
  "errors": 0,
  "failed_assertions": [
    {
      "id": "assertion_id",
      "account_id": "account_id",
      "expected_sats": 100000,
      "actual_sats": 99500,
      "difference_sats": -500
    }
  ]
}
```

## Monitoring

### Check Logs

```bash
# View cron logs
grep CRON /var/log/syslog

# View custom log (if using cron with redirect)
tail -f /var/log/castle-reconciliation.log
```

### Success Criteria

- `failed: 0` - All assertions passed
- `errors: 0` - No errors during checks
- `checked === total` - All assertions were checked

### Failure Scenarios

If `failed > 0`:
1. Check the `failed_assertions` array for details
2. Investigate discrepancies in the Castle UI
3. Review recent transactions
4. Check for data entry errors
5. Verify exchange rate conversions (for fiat)

## Future Enhancements

Planned features:
- [ ] Email notifications on failures
- [ ] Webhook notifications
- [ ] Slack/Discord integration
- [ ] Configurable schedule from UI
- [ ] Historical reconciliation reports
- [ ] Automatic retry on transient errors

## Troubleshooting

### Task Not Running

1. **Check cron service**:
   ```bash
   sudo systemctl status cron
   ```

2. **Verify API key**:
   - Ensure you're using the admin wallet API key
   - Key must belong to the super user

3. **Check network connectivity**:
   ```bash
   curl http://localhost:5000/castle/api/v1/reconciliation/summary -H "X-Api-Key: YOUR_KEY"
   ```

### Permission Denied

- Ensure the user running cron has execute permissions
- Check file permissions on any scripts
- Verify API key is valid and belongs to super user

### High Failure Rate

- Review your balance assertions
- Some assertions may need tolerance adjustments
- Check for recent changes in exchange rates
- Verify all transactions are properly cleared

## Best Practices

1. **Set Reasonable Tolerances**: Use tolerance levels to account for rounding
2. **Regular Review**: Check reconciliation dashboard weekly
3. **Assertion Coverage**: Create assertions for critical accounts
4. **Maintenance Window**: Run reconciliation during low-activity periods
5. **Backup First**: Run manual check before configuring automation
6. **Monitor Logs**: Set up log rotation and monitoring
7. **Alert Integration**: Plan for notification system integration

## Example Setup Script

```bash
#!/bin/bash
# setup-castle-reconciliation.sh

# Configuration
LNBITS_URL="http://localhost:5000"
ADMIN_KEY="your_admin_key_here"
LOG_FILE="/var/log/castle-reconciliation.log"

# Create log file
touch "$LOG_FILE"
chmod 644 "$LOG_FILE"

# Add cron job
(crontab -l 2>/dev/null; echo "0 2 * * * curl -X POST $LNBITS_URL/castle/api/v1/tasks/daily-reconciliation -H 'X-Api-Key: $ADMIN_KEY' >> $LOG_FILE 2>&1") | crontab -

echo "Daily reconciliation scheduled for 2 AM"
echo "Logs will be written to: $LOG_FILE"

# Test the endpoint
echo "Running test reconciliation..."
curl -X POST "$LNBITS_URL/castle/api/v1/tasks/daily-reconciliation" \
  -H "X-Api-Key: $ADMIN_KEY"
```

Make executable and run:
```bash
chmod +x setup-castle-reconciliation.sh
./setup-castle-reconciliation.sh
```