compose-farm/docs/architecture.md

---
icon: lucide/layers
---

# Architecture

This document explains how Compose Farm works under the hood.

## Design Philosophy

Compose Farm follows three core principles:

1. **KISS** - Keep it simple. It's a thin wrapper around `docker compose` over SSH.
2. **YAGNI** - No orchestration, no service discovery, no health checks until needed.
3. **Zero changes** - Your existing compose files work unchanged.

## High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────┐
│                        Compose Farm CLI                         │
│                                                                 │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────────────┐ │
│  │  Config  │  │  State   │  │Operations│  │   Executor       │ │
│  │  Parser  │  │ Tracker  │  │  Logic   │  │  (SSH/Local)     │ │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────────┬─────────┘ │
└───────┼─────────────┼─────────────┼─────────────────┼───────────┘
        │             │             │                 │
        ▼             ▼             ▼                 ▼
┌───────────────────────────────────────────────────────────────┐
│                         SSH / Local                            │
└───────────────────────────────────────────────────────────────┘
        │                                             │
        ▼                                             ▼
┌───────────────┐                           ┌───────────────┐
│   Host: nuc   │                           │   Host: hp    │
│               │                           │               │
│ docker compose│                           │ docker compose│
│    up -d      │                           │    up -d      │
└───────────────┘                           └───────────────┘
```

## Core Components

### Configuration (`src/compose_farm/config.py`)

Pydantic models for YAML configuration:

- **Config** - Root configuration with compose_dir, hosts, stacks
- **Host** - Host address, SSH user, and port

Key features:
- Validation with Pydantic
- Multi-host stack expansion (`all` → list of hosts)
- YAML loading with sensible defaults

### State Tracking (`src/compose_farm/state.py`)

Tracks deployment state in `compose-farm-state.yaml` (stored alongside the config file):

```yaml
deployed:
  plex: nuc
  grafana: nuc
```

Used for:
- Detecting migrations (stack moved to different host)
- Identifying orphans (stacks removed from config)
- `cf ps` status display

### Operations (`src/compose_farm/operations.py`)

Business logic for stack operations:

- **up** - Start stack, handle migration if needed
- **down** - Stop stack
- **preflight checks** - Verify mounts, networks exist before operations
- **discover** - Find running stacks on hosts
- **migrate** - Down on old host, up on new host

### Executor (`src/compose_farm/executor.py`)

SSH and local command execution:

- **Hybrid SSH approach**: asyncssh for parallel streaming, native `ssh -t` for raw mode
- **Parallel by default**: Multiple stacks via `asyncio.gather`
- **Streaming output**: Real-time stdout/stderr with `[stack]` prefix
- **Local detection**: Skips SSH when target matches local machine IP

### CLI (`src/compose_farm/cli/`)

Typer-based CLI with subcommand modules:

```
cli/
├── app.py          # Shared Typer app, version callback
├── common.py       # Shared helpers, options, progress utilities
├── config.py       # config subcommand (init, init-env, show, path, validate, edit, symlink)
├── lifecycle.py    # up, down, stop, pull, restart, update, apply, compose
├── management.py   # refresh, check, init-network, traefik-file
├── monitoring.py   # logs, ps, stats
├── ssh.py          # SSH key management (setup, status, keygen)
└── web.py          # Web UI server command
```

## Command Flow

### cf up plex

```
1. Load configuration
   └─► Parse compose-farm.yaml
   └─► Validate stack exists

2. Check state
   └─► Load state.yaml
   └─► Is plex already running?
   └─► Is it on a different host? (migration needed)

3. Pre-flight checks
   └─► SSH to target host
   └─► Check compose file exists
   └─► Check required mounts exist
   └─► Check required networks exist

4. Execute migration (if needed)
   └─► SSH to old host
   └─► Run: docker compose down

5. Start stack
   └─► SSH to target host
   └─► cd /opt/compose/plex
   └─► Run: docker compose up -d

6. Update state
   └─► Write new state to state.yaml

7. Generate Traefik config (if configured)
   └─► Regenerate traefik file-provider
```

### cf apply

```
1. Load configuration and state

2. Compute diff
   ├─► Orphans: in state, not in config
   ├─► Migrations: in both, different host
   └─► Missing: in config, not in state

3. Stop orphans
   └─► For each orphan: cf down

4. Migrate stacks
   └─► For each migration: down old, up new

5. Start missing
   └─► For each missing: cf up

6. Update state
```

## SSH Execution

### Parallel Streaming (asyncssh)

For most operations, Compose Farm uses asyncssh:

```python
async def run_command(host, command):
    async with asyncssh.connect(host) as conn:
        result = await conn.run(command)
        return result.stdout, result.stderr
```

Multiple stacks run concurrently via `asyncio.gather`.

### Raw Mode (native ssh)

For commands needing PTY (progress bars, interactive):

```bash
ssh -t user@host "docker compose pull"
```

### Local Detection

When target host IP matches local machine:

```python
if is_local(host_address):
    # Run locally, no SSH
    subprocess.run(command)
else:
    # SSH to remote
    ssh.run(command)
```

## State Management

### State File

Location: `compose-farm-state.yaml` (stored alongside the config file)

```yaml
deployed:
  plex: nuc
  grafana: nuc
```

Image digests are stored separately in `dockerfarm-log.toml` (also in the config directory).

### State Transitions

```
Config Change          State Change           Action
─────────────────────────────────────────────────────
Add stack            Missing                 cf up
Remove stack         Orphaned                cf down
Change host           Migration               down old, up new
No change             No change               none (or refresh)
```

### cf refresh

Syncs state with reality by querying Docker on each host:

```bash
docker ps --format '{{.Names}}'
```

Updates state.yaml to match what's actually running.

## Compose File Discovery

For each stack, Compose Farm looks for compose files in:

```
{compose_dir}/{stack}/
├── compose.yaml         # preferred
├── compose.yml
├── docker-compose.yml
└── docker-compose.yaml
```

First match wins.

## Traefik Integration

### Label Extraction

Compose Farm parses Traefik labels from compose files:

```yaml
stacks:
  plex:
    labels:
      - traefik.enable=true
      - traefik.http.routers.plex.rule=Host(`plex.example.com`)
      - traefik.http.services.plex.loadbalancer.server.port=32400
```

### File Provider Generation

Converts labels to Traefik file-provider YAML:

```yaml
http:
  routers:
    plex:
      rule: Host(`plex.example.com`)
      service: plex
  services:
    plex:
      loadBalancer:
        servers:
          - url: http://192.168.1.10:32400
```

### Variable Resolution

Supports `${VAR}` and `${VAR:-default}` from:
1. Service's `.env` file
2. Current environment

## Error Handling

### Pre-flight Failures

Before any operation, Compose Farm checks:
- SSH connectivity
- Compose file existence
- Required mounts
- Required networks

If checks fail, operation aborts with clear error.

### Partial Failures

When operating on multiple stacks:
- Each stack is independent
- Failures are logged, but other stacks continue
- Exit code reflects overall success/failure

## Performance Considerations

### Parallel Execution

Services are started/stopped in parallel:

```python
await asyncio.gather(*[
    up_stack(stack) for stack in stacks
])
```

### SSH Multiplexing

For repeated connections to the same host, SSH reuses connections.

### Caching

- Config is parsed once per command
- State is loaded once, written once
- Host discovery results are cached during command

## Web UI Architecture

```
┌─────────────────────────────────────────────────────────────┐
│                         Web UI                               │
│                                                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐  │
│  │   FastAPI   │  │    Jinja    │  │       HTMX          │  │
│  │   Backend   │  │  Templates  │  │   Dynamic Updates   │  │
│  └─────────────┘  └─────────────┘  └─────────────────────┘  │
│                                                             │
│  Pattern: Custom events, not hx-swap-oob                    │
│  Elements trigger on: cf:refresh from:body                  │
└─────────────────────────────────────────────────────────────┘
```

Icons use [Lucide](https://lucide.dev/). Add new icons as macros in `web/templates/partials/icons.html`.

### Host Resource Monitoring (`src/compose_farm/glances.py`)

Integration with [Glances](https://nicolargo.github.io/glances/) for real-time host stats:

- Fetches CPU, memory, and load from Glances REST API on each host
- Used by web UI dashboard to display host resource usage
- Requires `glances_stack` config option pointing to a Glances stack running on all hosts

### Container Registry Client (`src/compose_farm/registry.py`)

OCI Distribution API client for checking image updates:

- Parses image references (registry, namespace, name, tag, digest)
- Fetches available tags from Docker Hub, GHCR, and other registries
- Compares semantic versions to find newer releases