krops-multi-deploy/DEPLOYMENT-GUIDE.md

# Web-App Deployment Guide

## Overview

This setup builds the web-app **locally** with machine-specific configuration, then deploys the built artifacts to each target machine. Each machine gets its own customized build with:

- Machine-specific `.env` files
- Machine-specific images in the `public` folder

## Initial Setup

When you first clone this repository, you need to set up your local configuration:

### 1. Create your krops.nix

```bash
# Copy the example template
cp example-krops.nix krops.nix
```

### 2. Create your first machine configuration

```bash
# Copy the example machine template
cp -r config/machines/example-machine config/machines/my-machine

# Edit the configuration
# - Change the domain in configuration.nix
# - Add your hardware-configuration.nix (from nixos-generate-config)
```

### 3. Update krops.nix

Edit `krops.nix` and:
- Replace `example-machine` with your machine name
- Update the SSH target (`root@your-host`)
- Add to the `inherit` list and `all` script

### 4. Deploy!

```bash
nix-build ./krops.nix -A my-machine && ./result
```

**Note:** Your `krops.nix` and machine configs in `config/machines/*` are gitignored. You can safely pull updates without overwriting your local configuration.

## Structure

```
.
├── config/                           # NixOS configuration files
│   ├── shared.nix                    # Shared config for all machines
│   ├── nginx.nix                     # Nginx configuration
│   ├── lnbits.nix                    # LNBits configuration
│   ├── pict-rs.nix                   # Pict-rs configuration
│   └── machines/                     # Machine-specific configs (gitignored)
│       ├── example-machine/          # Template (committed to git)
│       │   ├── configuration.nix     # Main config entry point
│       │   ├── boot.nix             # Bootloader settings
│       │   └── example-service.nix  # Service examples
│       ├── machine1/                 # Your machines (gitignored)
│       └── machine2/                 # Your machines (gitignored)
├── web-app/                          # Shared web-app source (symlink)
├── machine-specific/                 # Machine-specific web-app assets (symlink)
├── lnbits/                          # LNBits source (symlink)
├── secrets/                         # Encrypted secrets
│   ├── example-machine/
│   │   └── README.md                # Secrets usage guide
│   ├── machine1/                    # Machine-specific secrets
│   │   └── *.age                    # Encrypted with age
│   └── machine2/
├── build/                           # Generated locally (gitignored)
├── build-local.nix                  # Local build scripts
└── krops.nix                        # Deployment configuration
```

## How It Works

### 1. Build Locally

First, build the web-app for each machine **on your local machine**:

```bash
# Build for a specific machine
nix-build ./build-local.nix -A machine1 && ./result/bin/build-machine1

# Or build for all machines
nix-build ./build-local.nix -A all && ./result/bin/build-all
```

This:

1. Copies web-app source to `./build/{machine}/`
2. Copies machine-specific `.env` file
3. Copies machine-specific images to `public/`
4. Runs `npm run build`
5. Creates `./build/{machine}/dist/` with the built app

### 2. Deploy Built Artifacts

After building, deploy to target machines:

```bash
# Deploy to specific machine
nix-build ./krops.nix -A machine1 && ./result

# Deploy to all machines
nix-build ./krops.nix -A all && ./result
```

The built files from `./build/{machine}/dist/` are copied to `/var/src/web-app-dist/` on each target machine.

## Complete Workflow

```bash
# 1. Build locally for all machines
nix-build ./build-local.nix -A all && ./result/bin/build-all

# 2. Deploy to all machines
nix-build ./krops.nix -A all && ./result
```

## Customization

### Add a new machine

1. **Copy the example template:**
   ```bash
   cp -r config/machines/example-machine config/machines/my-new-machine
   ```

2. **Edit the configuration:**
   - Open `config/machines/my-new-machine/configuration.nix`
   - Change `domain = "example.com"` to your domain
   - Add your `hardware-configuration.nix` (from `nixos-generate-config`)

3. **Create machine-specific web-app assets** (if using web-app):
   ```bash
   mkdir -p machine-specific/my-new-machine/env
   mkdir -p machine-specific/my-new-machine/images
   # Add .env file and images
   ```

4. **Add to krops.nix and build-local.nix:**
   - Add `my-new-machine` configuration to both files

5. **Build and deploy:**
   ```bash
   nix-build ./build-local.nix -A my-new-machine && ./result/bin/build-my-new-machine
   nix-build ./krops.nix -A my-new-machine && ./result
   ```

### Update environment variables

Edit `.env` in `machine-specific/{machine-name}/env/.env`, then rebuild locally

### Update images

Replace files in `machine-specific/{machine-name}/images/`, then rebuild locally

### Update web-app code

Edit files in `web-app/`, then rebuild locally

After any changes: rebuild locally, then redeploy.

## Adding Machine-Specific Services

Sometimes you need services that only run on certain machines (e.g., WireGuard on machine1 but not machine2).

### Using the Example Template

A complete example machine configuration is provided in `config/example-machine/`:

```
config/example-machine/
  ├── configuration.nix      # Template with domain parameter
  ├── boot.nix              # Bootloader configuration examples
  └── example-service.nix   # WireGuard and other service examples
```

**To use the template:**
1. Copy the `example-machine` directory to your new machine name:
   ```bash
   cp -r config/example-machine config/my-new-machine
   ```
2. Edit `configuration.nix` to set your domain
3. Copy your `hardware-configuration.nix` from `nixos-generate-config`
4. Customize `boot.nix` for your bootloader (UEFI or BIOS)
5. Modify or remove `example-service.nix` as needed
6. Add the machine to `build-local.nix` and `krops.nix`

### Example: Machine1 has WireGuard

**Structure:**
```
config/
  ├── shared.nix                    # Shared config for all machines
  ├── machine1/
  │   ├── configuration.nix         # Imports shared.nix + machine-specific modules
  │   ├── wireguard.nix            # Machine1-specific service
  │   ├── hardware-configuration.nix
  │   └── boot.nix
  └── machine2/
      ├── configuration.nix         # Only imports shared.nix
      ├── hardware-configuration.nix
      └── boot.nix
```

### Steps to Add a Machine-Specific Service

1. **Create a service configuration file** in the machine's directory:
   ```bash
   # Example: config/machine1/wireguard.nix
   { config, lib, pkgs, ... }:
   {
     networking.wireguard.interfaces = {
       wg0 = {
         privateKeyFile = "/etc/wireguard/privatekey";
         ips = [ "10.0.0.2/24" ];
         peers = [ ... ];
       };
     };
   }
   ```

2. **Import it in the machine's configuration.nix**:
   ```nix
   # config/machine1/configuration.nix
   { config, pkgs, ... }:
   {
     imports = [
       (import /var/src/config-shared {
         inherit config pkgs;
         domain = "4lpaca.io";
       })
       ./hardware-configuration.nix
       ./boot.nix
       ./wireguard.nix        # ← Add your service here
     ];
   }
   ```

3. **Deploy** - the service will only be deployed to that specific machine:
   ```bash
   nix-build ./krops.nix -A machine1 && ./result
   ```

### Common Machine-Specific Services

- **WireGuard VPN** - Only on machines that need VPN access
- **Backup services** - Different backup targets per machine
- **Development tools** - Extra packages for staging vs production
- **Custom hardware drivers** - GPU drivers, specific hardware support

The key is that each machine's `configuration.nix` can import different modules while still sharing common configuration through `shared.nix`.

## Deploying LNBits Extensions

You can deploy custom LNBits extensions to `/var/lib/lnbits/extensions` on your target machines.

### Setup

**1. Create extensions directory:**
```bash
mkdir -p lnbits-extensions
```

**2. Add your custom extensions:**
```bash
# Example: Clone a custom extension
git clone https://github.com/your-org/custom-extension lnbits-extensions/custom-extension
```

**3. Enable in krops.nix:**
Uncomment the lnbits-extensions line:
```nix
lnbits-extensions.file = toString ./lnbits-extensions;
```

**4. Enable in config/lnbits.nix:**

Choose one of two options:

**Option 1: Replace extensions directory** (use if you manage ALL extensions via deployment)
```nix
systemd.tmpfiles.rules = [
  "L+ /var/lib/lnbits/extensions - - - - /var/src/lnbits-extensions"
];
```
⚠️ **Warning:** This will DELETE any extensions installed via the LNBits UI!

**Option 2: Merge deployed extensions** (safer - keeps UI-installed extensions)
```nix
systemd.services.lnbits-copy-extensions = {
  description = "Copy deployed LNBits extensions";
  before = [ "lnbits.service" ];
  wantedBy = [ "lnbits.service" ];
  serviceConfig = {
    Type = "oneshot";
    ExecStart = "${pkgs.rsync}/bin/rsync -av /var/src/lnbits-extensions/ /var/lib/lnbits/extensions/";
  };
};
```

**5. Deploy:**
```bash
nix-build ./krops.nix -A machine1 && ./result
```

### How It Works

**Option 1 (Symlink):**
- Your `./lnbits-extensions` directory is deployed to `/var/src/lnbits-extensions`
- A symlink replaces `/var/lib/lnbits/extensions` → `/var/src/lnbits-extensions`
- Any existing extensions directory is deleted
- All extensions must be managed via deployment

**Option 2 (Copy/Merge):**
- Your `./lnbits-extensions` directory is deployed to `/var/src/lnbits-extensions`
- Deployed extensions are copied into `/var/lib/lnbits/extensions/`
- Existing UI-installed extensions are preserved
- You can mix deployed extensions with UI-installed ones