krops-multi-deploy/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

This is a NixOS multi-machine deployment system using krops with Nix 25.05. It manages multiple machines with different configurations, each potentially running services like LNBits (Lightning Network), pict-rs (image hosting), and custom web applications with machine-specific builds.

Key Architecture Concepts

Two-Stage Deployment Model

This project uses a two-stage deployment process:

1. Local Build Stage (build-local.nix): Builds web applications locally with machine-specific assets (.env files, images)
2. Remote Deploy Stage (krops.nix): Deploys NixOS configurations and pre-built artifacts to target machines

Configuration Inheritance Pattern

• config/shared.nix: Base configuration inherited by all machines (takes domain parameter)
• config/machines/{machine-name}/configuration.nix: Machine-specific entry point that:
  • Defines the domain variable
  • Imports shared.nix with the domain
  • Imports machine-specific modules (bootloader, hardware, services)

LNBits Flake Integration

The project uses a sophisticated LNBits deployment via Nix flakes:

• Source deployed to: /var/src/lnbits-src/ via krops symlink
• Flake reference: builtins.getFlake "path:/var/src/lnbits-src" (mutable local source)
• How it works: Uses uv2nix to convert uv.lock into a reproducible Nix venv in /nix/store
• Runtime: Systemd service runs /nix/store/xxx-lnbits-env/bin/lnbits with LNBITS_PATH pointing to source
• Lock file: flake.lock is deployed with the source and automatically used by Nix

See docs/lnbits-flake-explanation.md for detailed explanation.

Ensuring flake.lock is Used

Nix automatically uses flake.lock when present. To verify:

# Check flake.lock exists locally
ls -lh lnbits/flake.lock

# After deployment, verify on target machine
ssh root@machine "ls -lh /var/src/lnbits-src/flake.lock"

# View locked input versions
nix flake metadata path:/var/src/lnbits-src

To update flake inputs (do this locally before deploying):

cd lnbits/
nix flake update
# Or update specific input:
nix flake lock --update-input nixpkgs

Important: The krops .file source type copies all files including flake.lock. Since lnbits/ is a symlink, krops follows it and deploys the entire directory tree.

Krops Source Deployment

Files are deployed to target machines under /var/src/:

• /var/src/config-shared → config/shared.nix
• /var/src/config-machine → config/machines/{machine-name}/
• /var/src/web-app-dist → build/{machine-name}/dist/
• /var/src/lnbits-src → lnbits/ (full source with flake)
• /var/src/lnbits-extensions → lnbits-extensions/

Common Development Commands

Building Web Applications Locally

# Build for a specific machine
nix-build ./build-local.nix -A machine1 && ./result/bin/build-machine1

# Build for all machines
nix-build ./build-local.nix -A all && ./result/bin/build-all

This copies web-app source to ./build/{machine}/, adds machine-specific .env and images, then runs npm run build.

Deploying to Machines

# Deploy to a specific machine
nix-build ./krops.nix -A machine1 && ./result

# Deploy to all machines
nix-build ./krops.nix -A all && ./result

Complete Workflow (Build + Deploy)

# 1. Build web-apps locally
nix-build ./build-local.nix -A all && ./result/bin/build-all

# 2. Deploy to all machines
nix-build ./krops.nix -A all && ./result

Project Structure

.
├── krops.nix                     # Main deployment config (gitignored)
├── example-krops.nix             # Template for krops.nix
├── build-local.nix               # Local web-app build scripts (gitignored)
├── example-build-local.nix       # Template for build-local.nix
├── config/
│   ├── shared.nix                # Shared config (takes domain parameter)
│   ├── nginx.nix                 # Nginx + ACME + fail2ban
│   ├── lnbits.nix                # LNBits flake integration
│   ├── pict-rs.nix               # Image hosting service
│   └── machines/                 # Per-machine configs (gitignored)
│       ├── example-machine/      # Template (committed)
│       │   ├── configuration.nix # Sets domain, imports shared + modules
│       │   ├── boot.nix          # Bootloader config
│       │   └── example-service.nix # WireGuard and service examples
│       ├── machine1/             # Your machines (gitignored)
│       └── machine2/
├── build/                        # Generated web-app builds (gitignored)
├── machine-specific/             # Machine-specific web-app assets (symlink)
│   └── {machine-name}/
│       ├── env/.env              # Environment variables
│       └── images/               # Logos and images
├── web-app/                      # Shared web-app source (symlink)
├── lnbits/                       # LNBits source with flake (symlink)
└── lnbits-extensions/            # Custom LNBits extensions (symlink)

Adding a New Machine

1. Create machine configuration:

   cp -r config/machines/example-machine config/machines/new-machine
   

2. Edit config/machines/new-machine/configuration.nix:

   • Set domain = "yourdomain.com"
   • Add hardware-configuration.nix from nixos-generate-config
   • Customize boot.nix for your bootloader

3. Create machine-specific web-app assets (if needed):

   mkdir -p machine-specific/new-machine/{env,images}
   # Add .env file and images
   

4. Update build-local.nix:

   • Add new-machine = buildForMachine "new-machine"; to outputs
   • Add to all build script

5. Update krops.nix:

   • Add machine deployment block
   • Add to inherit list and all script

6. Build and deploy:

   nix-build ./build-local.nix -A new-machine && ./result/bin/build-new-machine
   nix-build ./krops.nix -A new-machine && ./result
   

Machine-Specific Services

To add services that only run on certain machines (e.g., WireGuard on one machine):

1. Create config/machines/{machine}/custom-service.nix
2. Import it in config/machines/{machine}/configuration.nix
3. Deploy only affects that machine

See config/machines/example-machine/example-service.nix for WireGuard and other examples.

Service Configuration Details

Virtual Hosts Pattern

All services use domain-based virtual hosts defined in shared.nix or service modules:

• Web app: app.${domain}
• LNBits: lnbits.${domain}
• Pict-rs: img.${domain}

All automatically get SSL via Let's Encrypt ACME.

LNBits Extensions

Two deployment options in config/lnbits.nix:

• Option 1 (Symlink): Replace /var/lib/lnbits/extensions entirely (deletes UI-installed extensions)
• Option 2 (Merge): Copy deployed extensions alongside UI-installed ones using rsync

Currently using Option 1 (symlink) - see commented code in config/lnbits.nix:96-122.

Nginx Configuration

• recommendedProxySettings = false (disabled for WebSocket compatibility)
• WebSocket support configured in LNBits vhost with upgrade headers
• ACME email: admin@aiolabs.dev (set in config/nginx.nix:16)

Important Files for Understanding

• DEPLOYMENT-GUIDE.md: Comprehensive deployment instructions
• docs/lnbits-flake-explanation.md: How LNBits flake deployment works (uv2nix, path: references, etc.)
• example-krops.nix and example-build-local.nix: Templates for configuration

Configuration Management

• Gitignored: krops.nix, build-local.nix, config/machines/* (except example-machine), build/, machine-specific/*
• Committed: Example configs, shared modules, service definitions
• Secrets: Managed separately (see secrets/ directory with age encryption)

Notes for AI Assistants

• When adding machines, update BOTH krops.nix and build-local.nix
• Always use the domain parameter pattern - it's passed to all modules
• LNBits requires deploying the FULL source tree (including flake.nix, uv.lock) to /var/src/lnbits-src
• Web-app builds happen locally to support machine-specific configurations
• The example-machine configuration is the canonical template - keep it updated