Compare commits

...

51 Commits

Author SHA1 Message Date
Bas Nijholt
009f3b1403 web: Fix theme resetting to first theme in list (#156) 2026-01-07 15:01:34 +01:00
Bas Nijholt
51f74eab42 examples: Add CoreDNS for *.local domain resolution (#155)
* examples: Add CoreDNS for *.local domain resolution

Adds a CoreDNS example that resolves *.local to the Traefik host,
making the .local routes in all examples work out of the box.

Also removes the redundant Multi-Container Stacks section from
README since paperless-ngx already demonstrates this pattern.

* examples: Add coredns .env file
2026-01-07 12:29:53 +01:00
Bas Nijholt
4acf797128 examples: Update paperless-ngx to use PostgreSQL (#153)
Match the real-world setup with Redis + PostgreSQL + App.
Remove NFS + PostgreSQL warning since it works fine in practice.
2026-01-07 03:23:34 -08:00
Andi Powers-Holmes
d167da9d63 Fix external network name parsing (#152)
* fix: external network name parsing

Compose network definitions may have a "name" field defining the actual network name,
which may differ from the key used in the compose file e.g. when overriding the default
compose network, or using a network name containing special characters that are not valid YAML keys.

Fix: check for "name" field on definition and use that if present, else fall back to key.

* tests: Add test for external network name field parsing

Covers the case where a network definition has a "name" field that
differs from the YAML key (e.g., default key with name: compose-net).

---------

Co-authored-by: Bas Nijholt <bas@nijho.lt>
2026-01-07 02:48:35 -08:00
Bas Nijholt
a5eac339db compose: Quote arguments with shlex to preserve spaces (#151) 2026-01-06 15:37:55 +01:00
Bas Nijholt
9f3813eb72 docs: Add missing source files to architecture docs (#150) 2026-01-06 13:07:20 +01:00
Bas Nijholt
b9ae0ad4d5 docs: Add missing options, aliases, and config settings (#149)
- Add --pull and --build options to cf up (from #146)
- Add --no-strays option to cf apply
- Add command aliases section (a, l, r, u, p, s, c, rf, ck, tf)
- Add cf config init-env subcommand documentation
- Add glances_stack config option (from #124)
- Add Host Resource Monitoring section to architecture docs
2026-01-06 11:06:24 +01:00
Bas Nijholt
ca2a4dd6d9 cli: Add short command aliases (#148)
* cli: Add short command aliases

Add single and two-letter aliases for frequently used commands:

- a  → apply
- l  → logs
- r  → restart
- u  → update
- p  → pull
- s  → stats
- c  → compose
- rf → refresh
- ck → check
- tf → traefik-file

Aliases are hidden from --help to keep output clean.

* docs: Document command aliases in README
2026-01-05 18:46:57 +01:00
Bas Nijholt
fafdce5736 docs: Clarify Docker Compose vs Compose Farm commands (#147)
* docs: Clarify Docker Compose vs Compose Farm commands

Split the Usage section into two tables:
- Docker Compose Commands: wrappers with multi-host additions
- Compose Farm Commands: orchestration Docker Compose can't do

Also update the `update` command docstring to clarify it's
a shorthand for `up --pull --build`.

* chore(docs): update TOC

* docs: Add command type distinction to commands.md

Explain that commands are either Docker Compose wrappers with
multi-host superpowers, or Compose Farm originals for orchestration.
Also update `update` description to clarify it's a shorthand.

* Update README.md
2026-01-05 18:37:41 +01:00
Bas Nijholt
6436becff9 up: Add --pull and --build flags for Docker Compose parity (#146)
* up: Add --pull and --build flags for Docker Compose parity

Add `--pull` and `--build` options to `cf up` to match Docker Compose
naming conventions. This allows users to pull images or rebuild before
starting without using the separate `update` command.

- `cf up --pull` adds `--pull always` to the compose command
- `cf up --build` adds `--build` to the compose command
- Both flags work together: `cf up --pull --build`

The `update` command remains unchanged as a convenience wrapper.

* Update README.md

* up: Run stacks in parallel when no migration needed

Refactor up_stacks to categorize stacks and run them appropriately:
- Simple stacks (no migration): run in parallel via asyncio.gather
- Multi-host stacks: run in parallel
- Migration stacks: run sequentially for clear output and rollback

This makes `cf up --all` as fast as `cf update --all` for typical use.

* refactor: DRY up command building with build_up_cmd helper

Consolidate all 'up -d' command construction into a single helper
function. Now used by up, update, and operations module.

Added tests for the helper function.

* update: Delegate to up --pull --build

Simplify update command to just call up with pull=True and build=True.
This removes duplication and ensures consistent behavior.
2026-01-05 15:55:00 +01:00
Bas Nijholt
3460d8a3ea restart: Match Docker Compose semantics (#145)
* restart: Match Docker Compose semantics

Change `cf restart` from doing `down + up` to using `docker compose
restart`, matching the Docker Compose command behavior.

This provides command naming parity with Docker Compose. Users who want
the old behavior can use `cf down mystack && cf up mystack`.

- Update restart implementation to use `docker compose restart`
- Remove traefik regeneration from restart (no longer recreates containers)
- Update all documentation and help text
- Remove restart from self-update SSH handling (no longer involves down)

* web: Clarify Update tooltip uses 'recreate' not 'restart'

Avoid confusion now that 'restart' means something different.

* web: Fix Update All tooltip to use 'recreates'
2026-01-05 14:29:03 +01:00
Bas Nijholt
8dabc27272 update: Only restart containers when images change (#143)
* update: Only restart containers when images change

Use `up -d --pull always --build` instead of separate pull/build/down/up
steps. This avoids unnecessary container restarts when images haven't
changed.

* Update README.md

* docs: Update update command description across all docs

Reflect new behavior: only recreates containers if images changed.

* Update README.md

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-01-05 10:06:45 +01:00
Bas Nijholt
5e08f1d712 web: Exclude web stack from Update All button (#142) 2026-01-04 19:56:41 +01:00
Bas Nijholt
8302f1d97a Unify vendor assets configuration in single JSON file (#141) 2025-12-31 19:02:41 +01:00
Bas Nijholt
eac9338352 Sort web stack last in bulk operations to prevent self-restart interruption (#140) 2025-12-31 10:10:42 +01:00
Bas Nijholt
667931dc80 docs: Add release checklist to ensure on latest main (#139) 2025-12-30 08:01:19 +01:00
Bas Nijholt
5890221528 Mark package as Linux/macOS only in classifiers (#138) 2025-12-30 07:54:56 +01:00
Bas Nijholt
c8fc3c2496 Remove Windows workarounds (#137) 2025-12-30 07:51:51 +01:00
Bas Nijholt
ffb7a32402 Fix Glances connectivity when web UI runs in Docker container (#135) 2025-12-30 05:35:24 +01:00
Bas Nijholt
beb1630fcf Deduplicate compose parsing (#136) 2025-12-28 17:00:32 +01:00
Bas Nijholt
2af48b2642 feat(web): add Glances integration for host resource stats (#124) 2025-12-28 08:37:57 +01:00
Bas Nijholt
f69993eac8 web: Rename command palette entry to "GitHub Repo" (#134)
Makes the entry searchable by typing "github" in the command palette.
2025-12-28 07:06:32 +01:00
Bas Nijholt
9bdcd143cf Prioritize dedicated SSH key over agent (#133) 2025-12-24 22:34:53 -08:00
Bas Nijholt
9230e12eb0 fix: Make SSH agent socket optional in docker-compose.yml (#132) 2025-12-24 12:22:01 -08:00
Bas Nijholt
2a923e6e81 fix: Include field name in config validation error messages (#131)
Previously, Pydantic validation errors like "Extra inputs are not
permitted" didn't show which field caused the error. Now the error
message includes the field location (e.g., "unknown_key: Extra inputs
are not permitted").
2025-12-22 22:35:19 -08:00
Bas Nijholt
5f2e081298 perf: Batch snapshot collection to 1 SSH call per host (#130)
## Summary

Optimize `cf refresh` SSH calls from O(stacks) to O(hosts):
- Discovery: 1 SSH call per host (unchanged)
- Snapshots: 1 SSH call per host (was 1 per stack)

For 50 stacks across 4 hosts: 54 → 8 SSH calls.

## Changes

**Performance:**
- Use `docker ps` + `docker image inspect` instead of `docker compose images` per stack
- Batch snapshot collection by host in `collect_stacks_entries_on_host()`

**Architecture:**
- Add `build_discovery_results()` to `operations.py` (business logic)
- Keep progress bar wrapper in `cli/management.py` (presentation)
- Remove dead code: `discover_all_stacks_on_all_hosts()`, `collect_all_stacks_entries()`
2025-12-22 22:19:32 -08:00
Bas Nijholt
6fbc7430cb perf: Optimize stray detection to use 1 SSH call per host (#129)
* perf: Optimize stray detection to use 1 SSH call per host

Previously, stray detection checked each stack on each host individually,
resulting in (stacks * hosts) SSH calls. For 50 stacks across 4 hosts,
this meant ~200 parallel SSH connections, causing "Connection lost" errors.

Now queries each host once for all running compose projects using:
  docker ps --format '{{.Label "com.docker.compose.project"}}' | sort -u

This reduces SSH calls from ~200 to just 4 (one per host).

Changes:
- Add get_running_stacks_on_host() in executor.py
- Add discover_all_stacks_on_all_hosts() in operations.py
- Update _discover_stacks_full() to use the batch approach

* Remove unused function and add tests

- Remove discover_stack_on_all_hosts() which is no longer used
- Add tests for get_running_stacks_on_host()
- Add tests for discover_all_stacks_on_all_hosts()
  - Verifies it returns correct StackDiscoveryResult
  - Verifies stray detection works
  - Verifies it makes only 1 call per host (not per stack)
2025-12-22 12:09:59 -08:00
Bas Nijholt
6fdb43e1e9 Add self-healing: detect and stop stray containers (#128)
* Add self-healing: detect and stop rogue containers

Adds the ability to detect and stop "rogue" containers - stacks running
on hosts they shouldn't be according to config.

Changes:
- `cf refresh`: Now scans ALL hosts and warns about rogues/duplicates
- `cf apply`: Stops rogue containers before migrations (new phase)
- New `--no-rogues` flag to skip rogue detection

Implementation:
- Add StackDiscoveryResult for full host scanning results
- Add discover_stack_on_all_hosts() to check all hosts in parallel
- Add stop_rogue_stacks() to stop containers on unauthorized hosts
- Update tests to include new no_rogues parameter

* Update README.md

* fix: Update refresh tests for _discover_stacks_full return type

The function now returns a tuple (discovered, rogues, duplicates)
for rogue/duplicate detection. Update test mocks accordingly.

* Rename "rogue" terminology to "stray" for consistency

Terminology update across the codebase:
- rogue_hosts -> stray_hosts
- is_rogue -> is_stray
- stop_rogue_stacks -> stop_stray_stacks
- _discover_rogues -> _discover_strays
- --no-rogues -> --no-strays
- _report_rogue_stacks -> _report_stray_stacks

"Stray" better complements "orphaned" (both evoke lost things)
while clearly indicating the stack is running somewhere it
shouldn't be.

* Update README.md

* Move asyncio import to top level

* Fix remaining rogue -> stray in docstrings and README

* Refactor: Extract shared helpers to reduce duplication

1. Extract _stop_stacks_on_hosts helper in operations.py
   - Shared by stop_orphaned_stacks and stop_stray_stacks
   - Reduces ~50 lines of duplicated code

2. Refactor _discover_strays to reuse _discover_stacks_full
   - Removes duplicate discovery logic from lifecycle.py
   - Calls management._discover_stacks_full and merges duplicates

* Add PR review prompt

* Fix typos in PR review prompt

* Move import to top level (no in-function imports)

* Update README.md

* Remove obvious comments
2025-12-22 10:22:09 -08:00
Bas Nijholt
620e797671 fix: Add entrypoint to create passwd entry for non-root users (#127) 2025-12-22 07:31:59 -08:00
Bas Nijholt
031a2af6f3 fix: Correct SSH key volume mount path in docker-compose.yml (#126) 2025-12-22 06:55:59 -08:00
Bas Nijholt
f69eed7721 docs(readme): position as Dockge for multi-host (#123)
* docs(readme): position as Dockge for multi-host

- Reference Dockge (which we've used) instead of Portainer
- Move Portainer mention to "Your files" bullet as contrast
- Link to Dockge repo

* docs(readme): add agentless bullet, link Dockge

- Add "Agentless" bullet highlighting SSH-only approach
- Link to Dockge as contrast (they require agents for multi-host)
- Update NOTE to focus on agentless, CLI-first positioning
2025-12-21 23:28:26 -08:00
Bas Nijholt
5a1fd4e29f docs(readme): add value propositions and fix image URL (#122)
- Add bullet points highlighting key benefits after NOTE block
- Update NOTE to position as file-based Portainer alternative
- Fix hero image URL from http to https
- Add alt text to hero image for accessibility
2025-12-21 23:17:18 -08:00
Bas Nijholt
26dea691ca feat(docker): make container user configurable via CF_UID/CF_GID (#118)
* feat(docker): make container user configurable via CF_UID/CF_GID

Add support for running compose-farm containers as a non-root user
to preserve file ownership on mounted volumes. This prevents files
like compose-farm-state.yaml and web UI config edits from being
owned by root on NFS mounts.

Set CF_UID, CF_GID, and CF_HOME environment variables to run as
your user. Defaults to root (0:0) for backwards compatibility.

* docs: document non-root user configuration for Docker

- Add CF_UID/CF_GID/CF_HOME documentation to README and getting-started
- Add XDG config volume mount for backup/log persistence across restarts
- Update SSH volume examples to use CF_HOME variable

* fix(docker): allow non-root user access and add USER env for SSH

- Add `chmod 755 /root` to Dockerfile so non-root users can access
  the installed tool at /root/.local/share/uv/tools/compose-farm
- Add USER environment variable to docker-compose.yml for SSH to work
  when running as non-root (UID not in /etc/passwd)
- Update docs to include CF_USER in the setup instructions
- Support building from local source with SETUPTOOLS_SCM_PRETEND_VERSION

* fix(docker): revert local build changes, keep only chmod 755 /root

Remove the local source build logic that was added during testing.
The only required change is `chmod 755 /root` to allow non-root users
to access the installed tool.

* docs: add .envrc.example for direnv users

* docs: mention direnv option in README and getting-started
2025-12-21 22:19:40 -08:00
Bas Nijholt
56d64bfe7a fix(web): exclude orphaned stacks from running count (#119)
The dashboard showed "stopped: -1" when orphaned stacks existed because
running_count included stacks in state but removed from config. Now only
stacks that are both in config AND deployed are counted as running.
2025-12-21 21:59:05 -08:00
Bas Nijholt
5ddbdcdf9e docs(demos): update recordings and fix demo scripts (#115) 2025-12-21 19:17:16 -08:00
Bas Nijholt
dd16becad1 feat(web): add Repo command to command palette (#117)
Adds a new "Repo" command that opens the GitHub repository in a new tab,
similar to the existing "Docs" command.
2025-12-21 15:25:04 -08:00
Bas Nijholt
df683a223f fix(web): wait for terminal expand transition before scrolling (#116)
- Extracts generic `expandCollapse(toggle, scrollTarget)` function for reuse with any DaisyUI collapse
- Fixes scrolling when clicking action buttons (pull, logs, etc.) while terminal is collapsed - now waits for CSS transition before scrolling
- Fixes shell via command palette - expands Container Shell and scrolls to actual terminal (not collapse header)
- Fixes scroll position not resetting when navigating via command palette
2025-12-21 15:17:59 -08:00
Bas Nijholt
fdb00e7655 refactor(web): store backups in XDG config directory (#113)
* refactor(web): store backups in XDG config directory

Move file backups from `.backups/` alongside the file to
`~/.config/compose-farm/backups/` (respecting XDG_CONFIG_HOME).
The original file path is mirrored inside to avoid name collisions.

* docs(web): document automatic backup location

* refactor(paths): extract shared config_dir() function

* fix(web): use path anchor for Windows compatibility
2025-12-21 15:08:15 -08:00
Bas Nijholt
90657a025f docs: fix missing CLI options and improve docs-review prompt (#114)
* docs: fix missing CLI options and improve docs-review prompt

- Add missing --config option docs for cf ssh setup and cf ssh status
- Enhance .prompts/docs-review.md with:
  - Quick reference table mapping docs to source files
  - Runnable bash commands for quick checks
  - Specific code paths instead of vague references
  - Web UI documentation section
  - Common gotchas section
  - Ready-to-apply fix template format
  - Post-fix verification steps

* docs: add self-review step to docs-review prompt

* docs: make docs-review prompt discovery-based and less brittle

- Use discovery commands (git ls-files, grep, find) instead of hardcoded lists
- Add 'What This Prompt Is For' section clarifying manual vs automated checks
- Simplify checklist to 10 sections focused on judgment-based review
- Remove hardcoded file paths in favor of search patterns
- Make commands dynamically discover CLI structure

* docs: simplify docs-review prompt, avoid duplicating automated checks

- Remove checks already handled by CI (README help output, command table)
- Focus on judgment-based review: accuracy, completeness, clarity
- Reduce from 270 lines to 117 lines
- Highlight that docs/commands.md options tables are manually maintained
2025-12-21 15:07:37 -08:00
Bas Nijholt
7ae8ea0229 feat(web): add tooltips to sidebar header icons (#111)
Use daisyUI tooltip component with bottom positioning for the docs,
GitHub, and theme switcher icons in the sidebar header, matching the
tooltip style used elsewhere in the web UI.
2025-12-21 14:16:57 -08:00
Bas Nijholt
612242eea9 feat(web): add Open Website button and command for stacks with Traefik labels (#110)
* feat(web): add Open Website button and command for stacks with Traefik labels

Parse traefik.http.routers.*.rule labels to extract Host() rules and
display "Open Website" button(s) on stack pages. Also adds the command
to the command palette.

- Add extract_website_urls() function to compose.py
- Determine scheme (http/https) from entrypoint (websecure/web)
- Prefer HTTPS when same host has both protocols
- Support environment variable interpolation
- Add external_link icon from Lucide
- Add comprehensive tests for URL extraction

* refactor: move extract_website_urls to traefik.py and reuse existing parsing

Instead of duplicating the Traefik label parsing logic in compose.py,
reuse generate_traefik_config() with check_all=True to get the parsed
router configuration, then extract Host() rules from it.

- Move extract_website_urls from compose.py to traefik.py
- Reuse generate_traefik_config for label parsing
- Move tests from test_compose.py to test_traefik.py
- Update import in pages.py

* test: add comprehensive tests for extract_website_urls

Cover real-world patterns found in stacks:
- Multiple Host() in one rule with || operator
- Host() combined with PathPrefix (e.g., && PathPrefix(`/api`))
- Multiple services in one stack (like arr stack)
- Labels in list format (- key=value)
- No entrypoints (defaults to http)
- Multiple entrypoints including websecure
2025-12-21 14:16:46 -08:00
Bas Nijholt
ea650bff8a fix: Skip buildable images in pull command (#109)
* fix: Skip buildable images in pull command

Add --ignore-buildable flag to pull command, matching the behavior
of the update command. This prevents pull from failing when a stack
contains services with local build directives (no remote image).

* test: Fix flaky command palette close detection

Use state="hidden" instead of :not([open]) selector when waiting
for the command palette to close. The old approach failed because
wait_for_selector defaults to waiting for visibility, but a closed
<dialog> element is hidden by design.
2025-12-21 10:28:10 -08:00
renovate[bot]
140bca4fd6 ⬆️ Update actions/upload-pages-artifact action to v4 (#108)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-21 10:27:58 -08:00
renovate[bot]
6dad6be8da ⬆️ Update actions/checkout action to v6 (#107)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-21 10:27:51 -08:00
Bas Nijholt
d7f931e301 feat(web): improve container row layout for mobile (#106)
- Stack container name/status above buttons on mobile screens
- Use card-like background for visual separation
- Buttons align right on desktop, full width on mobile
2025-12-21 10:27:36 -08:00
Bas Nijholt
471936439e feat(web): add Edit Config command to command palette (#105)
- Added "Edit Config" command to the command palette (Cmd/Ctrl+K)
- Navigates to console page, focuses the Monaco editor, and scrolls to it
- Uses `#editor` URL hash to signal editor focus instead of terminal focus
2025-12-21 01:24:03 -08:00
Bas Nijholt
36e4bef46d feat(web): add shell command to command palette for services (#104)
- Add "Shell: {service}" commands to the command palette when on a stack page
- Allows quick shell access to containers via `Cmd+K` → type "shell" → select service
- Add `get_container_name()` helper in `compose.py` for consistent container name resolution (used by both api.py and pages.py)
2025-12-21 01:23:54 -08:00
Bas Nijholt
2cac0bf263 feat(web): add Pull All and Update All to command palette (#103)
The dashboard buttons for Pull All and Update All are now also
available in the command palette (Cmd/Ctrl+K) for keyboard access.
2025-12-21 01:00:57 -08:00
Bas Nijholt
3d07cbdff0 fix(web): show stderr in console shell sessions (#102)
- Remove `2>/dev/null` from shell command that was suppressing all stderr output
- Command errors like "command not found" are now properly displayed to users
2025-12-21 00:50:58 -08:00
Bas Nijholt
0f67c17281 test: parallel execution and timeout constants (#101)
- Enable `-n auto` for all test commands in justfile (parallel execution)
- Add redis stack to test fixtures (missing stack was causing test failure)
- Replace hardcoded timeouts with constants: `TIMEOUT` (10s) and `SHORT_TIMEOUT` (5s)
- Rename `test-unit` → `test-cli` and `test-browser` → `test-web`
- Skip CLI startup test when running in parallel mode (`-n auto`)
- Update test assertions for 5 stacks (was 4)
2025-12-21 00:48:52 -08:00
Bas Nijholt
bd22a1a55e fix: Reject unknown keys in config with Pydantic strict mode (#100)
Add extra="forbid" to Host and Config models so typos like
`username` instead of `user` raise an error instead of being
silently ignored. Also simplify _parse_hosts to pass dicts
directly to Pydantic instead of manual field extraction.
2025-12-21 00:19:18 -08:00
131 changed files with 6887 additions and 1289 deletions

6
.envrc.example Normal file
View File

@@ -0,0 +1,6 @@
# Run containers as current user (preserves file ownership on NFS mounts)
# Copy this file to .envrc and run: direnv allow
export CF_UID=$(id -u)
export CF_GID=$(id -g)
export CF_HOME=$HOME
export CF_USER=$USER

View File

@@ -12,7 +12,7 @@ jobs:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
os: [ubuntu-latest, macos-latest]
python-version: ["3.11", "3.12", "3.13"]
steps:
@@ -54,7 +54,7 @@ jobs:
run: uv run playwright install chromium --with-deps
- name: Run browser tests
run: uv run pytest -m browser -v --no-cov
run: uv run pytest -m browser -n auto -v
lint:
runs-on: ubuntu-latest

View File

@@ -27,7 +27,7 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
with:
lfs: true
@@ -49,7 +49,7 @@ jobs:
- name: Upload artifact
if: github.event_name != 'pull_request'
uses: actions/upload-pages-artifact@v3
uses: actions/upload-pages-artifact@v4
with:
path: "./site"

2
.gitignore vendored
View File

@@ -37,6 +37,7 @@ ENV/
.coverage
.pytest_cache/
htmlcov/
.code/
# Local config (don't commit real configs)
compose-farm.yaml
@@ -45,3 +46,4 @@ coverage.xml
.env
homepage/
site/
.playwright-mcp/

View File

@@ -21,7 +21,7 @@ repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.9
hooks:
- id: ruff
- id: ruff-check
args: [--fix]
- id: ruff-format

View File

@@ -1,118 +1,119 @@
Review all documentation in this repository for accuracy, completeness, and consistency. Cross-reference documentation against the actual codebase to identify issues.
Review documentation for accuracy, completeness, and consistency. Focus on things that require judgment—automated checks handle the rest.
## Scope
## What's Already Automated
Review all documentation files:
- docs/*.md (primary documentation)
- README.md (repository landing page)
- CLAUDE.md (development guidelines)
- examples/README.md (example configurations)
Don't waste time on these—CI and pre-commit hooks handle them:
## Review Checklist
- **README help output**: `markdown-code-runner` regenerates `cf --help` blocks in CI
- **README command table**: Pre-commit hook verifies commands are listed
- **Linting/formatting**: Handled by pre-commit
### 1. Command Documentation
## What This Review Is For
For each documented command, verify against the CLI source code:
Focus on things that require judgment:
- Command exists in codebase
- All options are documented with correct names, types, and defaults
- Short options (-x) match long options (--xxx)
- Examples would work as written
- Check for undocumented commands or options
1. **Accuracy**: Does the documentation match what the code actually does?
2. **Completeness**: Are there undocumented features, options, or behaviors?
3. **Clarity**: Would a new user understand this? Are examples realistic?
4. **Consistency**: Do different docs contradict each other?
5. **Freshness**: Has the code changed in ways the docs don't reflect?
Run `--help` for each command to verify.
## Review Process
### 2. Configuration Documentation
### 1. Check Recent Changes
Verify against Pydantic models in the config module:
```bash
# What changed recently that might need doc updates?
git log --oneline -20 | grep -iE "feat|fix|add|remove|change|option"
- All config keys are documented
- Types match Pydantic field types
- Required vs optional fields are correct
- Default values are accurate
- Config file search order matches code
- Example YAML is valid and uses current schema
# What code files changed?
git diff --name-only HEAD~20 | grep "\.py$"
```
### 3. Architecture Documentation
Look for new features, changed defaults, renamed options, or removed functionality.
Verify against actual directory structure:
### 2. Verify docs/commands.md Options Tables
- File paths match actual source code location
- All modules listed actually exist
- No modules are missing from the list
- Component descriptions match code functionality
- CLI module list includes all command files
The README auto-updates help output, but `docs/commands.md` has **manually maintained options tables**. These can drift.
### 4. State and Data Files
For each command's options table, compare against `cf <command> --help`:
- Are all options listed?
- Are short flags correct?
- Are defaults accurate?
- Are descriptions accurate?
Verify against state and path modules:
**Pay special attention to subcommands** (`cf config *`, `cf ssh *`)—these have their own options that are easy to miss.
- State file name and location are correct
- State file format matches actual structure
- Log file name and location are correct
- What triggers state/log updates is accurate
### 3. Verify docs/configuration.md
### 5. Installation Documentation
Compare against Pydantic models in the source:
Verify against pyproject.toml:
```bash
# Find the config models
grep -r "class.*BaseModel" src/ --include="*.py" -A 15
```
- Python version requirement matches requires-python
- Package name is correct
- Optional dependencies are documented
- CLI entry points are mentioned
- Installation methods work as documented
Check:
- All config keys documented
- Types and defaults match code
- Config file search order is accurate
- Example YAML would actually work
### 6. Feature Claims
### 4. Verify docs/architecture.md and CLAUDE.md
For each claimed feature, verify it exists and works as described.
```bash
# What source files actually exist?
git ls-files "src/**/*.py"
```
### 7. Cross-Reference Consistency
Check **both** `docs/architecture.md` and `CLAUDE.md` (Architecture section):
- Listed files exist
- No files are missing from the list
- Descriptions match what the code does
Check for conflicts between documentation files:
Both files have architecture listings that can drift independently.
- README vs docs/index.md (should be consistent)
- CLAUDE.md vs actual code structure
- Command tables match across files
- Config examples are consistent
### 5. Check Examples
### 8. Recent Changes Check
For examples in any doc:
- Would the YAML/commands actually work?
- Are service names, paths, and options realistic?
- Do examples use current syntax (not deprecated options)?
Before starting the review:
### 6. Cross-Reference Consistency
- Run `git log --oneline -20` to see recent commits
- Look for commits with `feat:`, `fix:`, or that mention new options/commands
- Cross-reference these against the documentation to catch undocumented features
The same info appears in multiple places. Check for conflicts:
- README.md vs docs/index.md
- docs/commands.md vs CLAUDE.md command tables
- Config examples across different docs
### 9. Auto-Generated Content
### 7. Self-Check This Prompt
For README.md or docs with `<!-- CODE:BASH:START -->` blocks:
This prompt can become outdated too. If you notice:
- New automated checks that should be listed above
- New doc files that need review guidelines
- Patterns that caused issues
- Run `uv run markdown-code-runner <file>` to regenerate outputs
- Check for missing `<!-- OUTPUT:START -->` markers (blocks that never ran)
- Verify help output matches current CLI behavior
### 10. CLI Options Completeness
For each command, run `cf <command> --help` and verify:
- Every option shown in help is documented
- Short flags (-x) are listed alongside long flags (--xxx)
- Default values in help match documented defaults
Include prompt updates in your fixes.
## Output Format
Provide findings in these categories:
Categorize findings:
1. **Critical Issues**: Incorrect information that would cause user problems
2. **Inaccuracies**: Technical errors, wrong defaults, incorrect paths
3. **Missing Documentation**: Features/commands that exist but aren't documented
4. **Outdated Content**: Information that was once true but no longer is
5. **Inconsistencies**: Conflicts between different documentation files
6. **Minor Issues**: Typos, formatting, unclear wording
7. **Verified Accurate**: Sections confirmed to be correct
1. **Critical**: Wrong info that would break user workflows
2. **Inaccuracy**: Technical errors (wrong defaults, paths, types)
3. **Missing**: Undocumented features or options
4. **Outdated**: Was true, no longer is
5. **Inconsistency**: Docs contradict each other
6. **Minor**: Typos, unclear wording
For each issue, include:
- File path and line number (if applicable)
- What the documentation says
- What the code actually does
- Suggested fix
For each issue, provide a ready-to-apply fix:
```
### Issue: [Brief description]
- **File**: docs/commands.md:652
- **Problem**: `cf ssh setup` has `--config` option but it's not documented
- **Fix**: Add `--config, -c PATH` to the options table
- **Verify**: `cf ssh setup --help`
```

View File

@@ -0,0 +1,79 @@
# Duplication audit and generalization prompt
You are a coding agent working inside a repository. Your job is to find duplicated
functionality (not just identical code) and propose a minimal, safe generalization.
Keep it simple and avoid adding features.
## First steps
- Read project-specific instructions (AGENTS.md, CONTRIBUTING.md, or similar) and follow them.
- If instructions mention tooling or style (e.g., preferred search tools), use those.
- Ask a brief clarification if the request is ambiguous (for example: report only vs refactor).
## Objective
Identify and consolidate duplicated functionality across the codebase. Duplication includes:
- Multiple functions that parse or validate the same data in slightly different ways
- Repeated file reads or config parsing
- Similar command building or subprocess execution paths
- Near-identical error handling or logging patterns
- Repeated data transforms that can become a shared helper
The goal is to propose a general, reusable abstraction that reduces duplication while
preserving behavior. Keep changes minimal and easy to review.
## Search strategy
1) Map the hot paths
- Scan entry points (CLI, web handlers, tasks, jobs) to see what they do repeatedly.
- Look for cross-module patterns: same steps, different files.
2) Find duplicate operations
- Use fast search tools (prefer `rg`) to find repeated keywords and patterns.
- Check for repeated YAML/JSON parsing, env interpolation, file IO, command building,
data validation, or response formatting.
3) Validate duplication is real
- Confirm the functional intent matches (not just similar code).
- Note any subtle differences that must be preserved.
4) Propose a minimal generalization
- Suggest a shared helper, utility, or wrapper.
- Avoid over-engineering. If only two call sites exist, keep the helper small.
- Prefer pure functions and centralized IO if that already exists.
## Deliverables
Provide a concise report with:
1) Findings
- List duplicated behaviors with file references and a short description of the
shared functionality.
- Explain why these are functionally the same (or nearly the same).
2) Proposed generalizations
- For each duplication, propose a shared helper and where it should live.
- Outline any behavior differences that need to be parameterized.
3) Impact and risk
- Note any behavior risks, test needs, or migration steps.
If the user asked you to implement changes:
- Make only the minimal edits needed to dedupe behavior.
- Keep the public API stable unless explicitly requested.
- Add small comments only when the logic is non-obvious.
- Summarize what changed and why.
## Output format
- Start with a short summary of the top 1-3 duplications.
- Then provide a list of findings, ordered by impact.
- Include a small proposed refactor plan (step-by-step, no more than 5 steps).
- End with any questions or assumptions.
## Guardrails
- Do not add new features or change behavior beyond deduplication.
- Avoid deep refactors without explicit request.
- Preserve existing style conventions and import rules.
- If a duplication is better left alone (e.g., clarity, single usage), say so.

15
.prompts/pr-review.md Normal file
View File

@@ -0,0 +1,15 @@
Review the pull request for:
- **Code cleanliness**: Is the implementation clean and well-structured?
- **DRY principle**: Does it avoid duplication?
- **Code reuse**: Are there parts that should be reused from other places?
- **Organization**: Is everything in the right place?
- **Consistency**: Is it in the same style as other parts of the codebase?
- **Simplicity**: Is it not over-engineered? Remember KISS and YAGNI. No dead code paths and NO defensive programming.
- **User experience**: Does it provide a good user experience?
- **PR**: Is the PR description and title clear and informative?
- **Tests**: Are there tests, and do they cover the changes adequately? Are they testing something meaningful or are they just trivial?
- **Live tests**: Test the changes in a REAL live environment to ensure they work as expected, use the config in `/opt/stacks/compose-farm.yaml`.
- **Rules**: Does the code follow the project's coding standards and guidelines as laid out in @CLAUDE.md?
Look at `git diff origin/main..HEAD` for the changes made in this pull request.

51
.prompts/update-demos.md Normal file
View File

@@ -0,0 +1,51 @@
Update demo recordings to match the current compose-farm.yaml configuration.
## Key Gotchas
1. **Never `git checkout` without asking** - check for uncommitted changes first
2. **Prefer `nas` stacks** - demos run locally on nas, SSH adds latency
3. **Terminal captures keyboard** - use `blur()` to release focus before command palette
4. **Clicking sidebar navigates away** - clicking h1 scrolls to top
5. **Buttons have icons, not text** - use `[data-tip="..."]` selectors
6. **`record.py` auto-restores config** - no manual cleanup needed after CLI demos
## Stacks Used in Demos
| Stack | CLI Demos | Web Demos | Notes |
|-------|-----------|-----------|-------|
| `audiobookshelf` | quickstart, migration, apply | - | Migrates nas→anton |
| `grocy` | update | navigation, stack, workflow, console | - |
| `immich` | logs, compose | shell | Multiple containers |
| `dozzle` | - | workflow | - |
## CLI Demos
**Files:** `docs/demos/cli/*.tape`
Check:
- `quickstart.tape`: `bat -r` line ranges match current config structure
- `migration.tape`: nvim keystrokes work, stack exists on nas
- `compose.tape`: exec commands produce meaningful output
Run: `python docs/demos/cli/record.py [demo]`
## Web Demos
**Files:** `docs/demos/web/demo_*.py`
Check:
- Stack names in demos still exist in config
- Selectors match current templates (grep for IDs in `templates/`)
- Shell demo uses command palette for ALL navigation
Run: `python docs/demos/web/record.py [demo]`
## Before Recording
```bash
# Check for uncommitted config changes
git -C /opt/stacks diff compose-farm.yaml
# Verify stacks are running
cf ps audiobookshelf grocy immich dozzle
```

View File

@@ -20,15 +20,17 @@ src/compose_farm/
│ ├── monitoring.py # logs, ps, stats commands
│ ├── ssh.py # SSH key management (setup, status, keygen)
│ └── web.py # Web UI server command
├── config.py # Pydantic models, YAML loading
├── compose.py # Compose file parsing (.env, ports, volumes, networks)
├── config.py # Pydantic models, YAML loading
├── console.py # Shared Rich console instances
├── executor.py # SSH/local command execution, streaming output
├── operations.py # Business logic (up, migrate, discover, preflight checks)
├── state.py # Deployment state tracking (which stack on which host)
├── glances.py # Glances API integration for host resource stats
├── logs.py # Image digest snapshots (dockerfarm-log.toml)
├── operations.py # Business logic (up, migrate, discover, preflight checks)
├── paths.py # Path utilities, config file discovery
├── registry.py # Container registry client for update checking
├── ssh_keys.py # SSH key path constants and utilities
├── state.py # Deployment state tracking (which stack on which host)
├── traefik.py # Traefik file-provider config generation from labels
└── web/ # Web UI (FastAPI + HTMX)
```
@@ -66,8 +68,8 @@ Use `just` for common tasks. Run `just` to list available commands:
|---------|-------------|
| `just install` | Install dev dependencies |
| `just test` | Run all tests |
| `just test-unit` | Run unit tests (parallel) |
| `just test-browser` | Run browser tests |
| `just test-cli` | Run CLI tests (parallel) |
| `just test-web` | Run web UI tests (parallel) |
| `just lint` | Lint, format, and type check |
| `just web` | Start web UI (port 9001) |
| `just doc` | Build and serve docs (port 9002) |
@@ -78,17 +80,17 @@ Use `just` for common tasks. Run `just` to list available commands:
Run tests with `just test` or `uv run pytest`. Browser tests require Chromium (system-installed or via `playwright install chromium`):
```bash
# Unit tests only (skip browser tests, can parallelize)
# Unit tests only (parallel)
uv run pytest -m "not browser" -n auto
# Browser tests only (run sequentially, no coverage)
uv run pytest -m browser --no-cov
# Browser tests only (parallel)
uv run pytest -m browser -n auto
# All tests
uv run pytest --no-cov
uv run pytest
```
Browser tests are marked with `@pytest.mark.browser`. They use Playwright to test HTMX behavior, JavaScript functionality (sidebar filter, command palette, terminals), and content stability during navigation. Run sequentially (no `-n`) to avoid resource contention.
Browser tests are marked with `@pytest.mark.browser`. They use Playwright to test HTMX behavior, JavaScript functionality (sidebar filter, command palette, terminals), and content stability during navigation.
## Communication Notes
@@ -110,6 +112,10 @@ Browser tests are marked with `@pytest.mark.browser`. They use Playwright to tes
Use `gh release create` to create releases. The tag is created automatically.
```bash
# IMPORTANT: Ensure you're on latest origin/main before releasing!
git fetch origin
git checkout origin/main
# Check current version
git tag --sort=-v:refname | head -1
@@ -133,8 +139,8 @@ CLI available as `cf` or `compose-farm`.
| `down` | Stop stacks (`docker compose down`). Use `--orphaned` to stop stacks removed from config |
| `stop` | Stop services without removing containers (`docker compose stop`) |
| `pull` | Pull latest images |
| `restart` | `down` + `up -d` |
| `update` | `pull` + `build` + `down` + `up -d` |
| `restart` | Restart running containers (`docker compose restart`) |
| `update` | Pull, build, recreate only if changed (`up -d --pull always --build`) |
| `apply` | Make reality match config: migrate stacks + stop orphans. Use `--dry-run` to preview |
| `compose` | Run any docker compose command on a stack (passthrough) |
| `logs` | Show stack logs |
@@ -144,6 +150,6 @@ CLI available as `cf` or `compose-farm`.
| `check` | Validate config, traefik labels, mounts, networks; show host compatibility |
| `init-network` | Create Docker network on hosts with consistent subnet/gateway |
| `traefik-file` | Generate Traefik file-provider config from compose labels |
| `config` | Manage config files (init, show, path, validate, edit, symlink) |
| `config` | Manage config files (init, init-env, show, path, validate, edit, symlink) |
| `ssh` | Manage SSH keys (setup, status, keygen) |
| `web` | Start web UI server |

View File

@@ -16,5 +16,13 @@ RUN apk add --no-cache openssh-client
COPY --from=builder /root/.local/share/uv/tools/compose-farm /root/.local/share/uv/tools/compose-farm
COPY --from=builder /usr/local/bin/cf /usr/local/bin/compose-farm /usr/local/bin/
ENTRYPOINT ["cf"]
# Allow non-root users to access the installed tool
# (required when running with user: "${CF_UID:-0}:${CF_GID:-0}")
RUN chmod 755 /root
# Allow non-root users to add passwd entries (required for SSH)
RUN chmod 666 /etc/passwd
# Entrypoint creates /etc/passwd entry for non-root UIDs (required for SSH)
ENTRYPOINT ["sh", "-c", "[ $(id -u) != 0 ] && echo ${USER:-u}:x:$(id -u):$(id -g)::${HOME:-/}:/bin/sh >> /etc/passwd; exec cf \"$@\"", "--"]
CMD ["--help"]

216
README.md
View File

@@ -5,12 +5,19 @@
[![License](https://img.shields.io/github/license/basnijholt/compose-farm)](LICENSE)
[![GitHub stars](https://img.shields.io/github/stars/basnijholt/compose-farm)](https://github.com/basnijholt/compose-farm/stargazers)
<img src="http://files.nijho.lt/compose-farm.png" align="right" style="width: 300px;" />
<img src="https://files.nijho.lt/compose-farm.png" alt="Compose Farm logo" align="right" style="width: 300px;" />
A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.
> [!NOTE]
> Run `docker compose` commands across multiple hosts via SSH. One YAML maps stacks to hosts. Run `cf apply` and reality matches your config—stacks start, migrate, or stop as needed. No Kubernetes, no Swarm, no magic.
> Agentless multi-host Docker Compose. CLI-first with a web UI. Your files stay as plain folders—version-controllable, no lock-in. Run `cf apply` and reality matches your config.
**Why Compose Farm?**
- **Your files, your control** — Plain folders + YAML, not locked in Portainer. Version control everything.
- **Agentless** — Just SSH, no agents to deploy (unlike [Dockge](https://github.com/louislam/dockge)).
- **Zero changes required** — Existing compose files work as-is.
- **Grows with you** — Start single-host, scale to multi-host seamlessly.
- **Declarative** — Change config, run `cf apply`, reality matches.
## Quick Demo
@@ -36,17 +43,21 @@ A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.
- [What Compose Farm doesn't do](#what-compose-farm-doesnt-do)
- [Installation](#installation)
- [SSH Authentication](#ssh-authentication)
- [SSH Agent (default)](#ssh-agent-default)
- [Dedicated SSH Key (recommended for Docker/Web UI)](#dedicated-ssh-key-recommended-for-dockerweb-ui)
- [SSH Agent](#ssh-agent)
- [Dedicated SSH Key (default for Docker)](#dedicated-ssh-key-default-for-docker)
- [Configuration](#configuration)
- [Single-host example](#single-host-example)
- [Multi-host example](#multi-host-example)
- [Multi-Host Stacks](#multi-host-stacks)
- [Config Command](#config-command)
- [Usage](#usage)
- [Docker Compose Commands](#docker-compose-commands)
- [Compose Farm Commands](#compose-farm-commands)
- [Aliases](#aliases)
- [CLI `--help` Output](#cli---help-output)
- [Auto-Migration](#auto-migration)
- [Traefik Multihost Ingress (File Provider)](#traefik-multihost-ingress-file-provider)
- [Host Resource Monitoring (Glances)](#host-resource-monitoring-glances)
- [Comparison with Alternatives](#comparison-with-alternatives)
- [License](#license)
@@ -177,15 +188,33 @@ docker run --rm \
ghcr.io/basnijholt/compose-farm up --all
```
**Running as non-root user** (recommended for NFS mounts):
By default, containers run as root. To preserve file ownership on mounted volumes
(e.g., `compose-farm-state.yaml`, config edits), set these environment variables:
```bash
# Add to .env file (one-time setup)
echo "CF_UID=$(id -u)" >> .env
echo "CF_GID=$(id -g)" >> .env
echo "CF_HOME=$HOME" >> .env
echo "CF_USER=$USER" >> .env
```
Or use [direnv](https://direnv.net/) (copies `.envrc.example` to `.envrc`):
```bash
cp .envrc.example .envrc && direnv allow
```
</details>
## SSH Authentication
Compose Farm uses SSH to run commands on remote hosts. There are two authentication methods:
### SSH Agent (default)
### SSH Agent
Works out of the box if you have an SSH agent running with your keys loaded:
Works out of the box when running locally if you have an SSH agent running with your keys loaded:
```bash
# Verify your agent has keys
@@ -195,9 +224,9 @@ ssh-add -l
cf up --all
```
### Dedicated SSH Key (recommended for Docker/Web UI)
### Dedicated SSH Key (default for Docker)
When running compose-farm in Docker, the SSH agent connection can be lost (e.g., after container restart). The `cf ssh` command sets up a dedicated key that persists:
When running in Docker, SSH agent sockets are ephemeral and can be lost after container restarts. The `cf ssh` command sets up a dedicated key that persists:
```bash
# Generate key and copy to all configured hosts
@@ -216,15 +245,22 @@ When running in Docker, mount a volume to persist the SSH keys. Choose ONE optio
**Option 1: Host path (default)** - keys at `~/.ssh/compose-farm/id_ed25519`
```yaml
volumes:
- ~/.ssh/compose-farm:/root/.ssh
- ~/.ssh/compose-farm:${CF_HOME:-/root}/.ssh
```
**Option 2: Named volume** - managed by Docker
```yaml
volumes:
- cf-ssh:/root/.ssh
- cf-ssh:${CF_HOME:-/root}/.ssh
```
**Option 3: SSH agent forwarding** - if you prefer using your host's ssh-agent
```yaml
volumes:
- ${SSH_AUTH_SOCK}:/ssh-agent:ro
```
Note: Requires `SSH_AUTH_SOCK` environment variable to be set. The socket path is ephemeral and changes across sessions.
Run setup once after starting the container (while the SSH agent still works):
```bash
@@ -233,6 +269,8 @@ docker compose exec web cf ssh setup
The keys will persist across restarts.
**Note:** When running as non-root (with `CF_UID`/`CF_GID`), set `CF_HOME` to your home directory so SSH finds the keys at the correct path.
</details>
## Configuration
@@ -328,24 +366,47 @@ Use `cf config init` to get started with a fully documented template.
The CLI is available as both `compose-farm` and the shorter `cf` alias.
### Docker Compose Commands
These wrap `docker compose` with multi-host superpowers:
| Command | Wraps | Compose Farm Additions |
|---------|-------|------------------------|
| `cf up` | `up -d` | `--all`, `--host`, parallel execution, auto-migration |
| `cf down` | `down` | `--all`, `--host`, `--orphaned`, state tracking |
| `cf stop` | `stop` | `--all`, `--service` |
| `cf restart` | `restart` | `--all`, `--service` |
| `cf pull` | `pull` | `--all`, `--service`, parallel execution |
| `cf logs` | `logs` | `--all`, `--host`, multi-stack output |
| `cf ps` | `ps` | `--all`, `--host`, unified cross-host view |
| `cf compose` | any | passthrough for commands not listed above |
### Compose Farm Commands
Multi-host orchestration that Docker Compose can't do:
| Command | Description |
|---------|-------------|
| **`cf apply`** | **Make reality match config (start + migrate + stop orphans)** |
| `cf up <stack>` | Start stack (auto-migrates if host changed) |
| `cf down <stack>` | Stop and remove stack containers |
| `cf stop <stack>` | Stop stack without removing containers |
| `cf restart <stack>` | down + up |
| `cf update <stack>` | pull + build + down + up |
| `cf pull <stack>` | Pull latest images |
| `cf logs -f <stack>` | Follow logs |
| `cf ps` | Show status of all stacks |
| `cf refresh` | Update state from running stacks |
| **`cf apply`** | **Reconcile: start missing, migrate moved, stop orphans** |
| `cf update` | Shorthand for `up --pull --build` |
| `cf refresh` | Sync state from what's actually running |
| `cf check` | Validate config, mounts, networks |
| `cf init-network` | Create Docker network on hosts |
| `cf init-network` | Create Docker network on all hosts |
| `cf traefik-file` | Generate Traefik file-provider config |
| `cf config <cmd>` | Manage config files (init, show, path, validate, edit, symlink) |
| `cf config` | Manage config files (init, show, validate, edit, symlink) |
| `cf ssh` | Manage SSH keys (setup, status, keygen) |
All commands support `--all` to operate on all stacks.
### Aliases
Short aliases for frequently used commands:
| Alias | Command | Alias | Command |
|-------|---------|-------|---------|
| `cf a` | `apply` | `cf s` | `stats` |
| `cf l` | `logs` | `cf c` | `compose` |
| `cf r` | `restart` | `cf rf` | `refresh` |
| `cf u` | `update` | `cf ck` | `check` |
| `cf p` | `pull` | `cf tf` | `traefik-file` |
Each command replaces: look up host → SSH → find compose file → run `ssh host "cd /opt/compose/plex && docker compose up -d"`.
@@ -365,10 +426,10 @@ cf down --orphaned # stop stacks removed from config
# Pull latest images
cf pull --all
# Restart (down + up)
# Restart running containers
cf restart plex
# Update (pull + build + down + up) - the end-to-end update command
# Update (pull + build, only recreates containers if images changed)
cf update --all
# Update state from reality (discovers running stacks + captures digests)
@@ -422,20 +483,6 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
│ copy it or customize the installation. │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Lifecycle ──────────────────────────────────────────────────────────────────╮
│ up Start stacks (docker compose up -d). Auto-migrates if host │
│ changed. │
│ down Stop stacks (docker compose down). │
│ stop Stop services without removing containers (docker compose │
│ stop). │
│ pull Pull latest images (docker compose pull). │
│ restart Restart stacks (down + up). With --service, restarts just │
│ that service. │
│ update Update stacks (pull + build + down + up). With --service, │
│ updates just that service. │
│ apply Make reality match config (start, migrate, stop as needed). │
│ compose Run any docker compose command on a stack. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Configuration ──────────────────────────────────────────────────────────────╮
│ traefik-file Generate a Traefik file-provider fragment from compose │
│ Traefik labels. │
@@ -445,6 +492,20 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
│ config Manage compose-farm configuration files. │
│ ssh Manage SSH keys for passwordless authentication. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Lifecycle ──────────────────────────────────────────────────────────────────╮
│ up Start stacks (docker compose up -d). Auto-migrates if host │
│ changed. │
│ down Stop stacks (docker compose down). │
│ stop Stop services without removing containers (docker compose │
│ stop). │
│ pull Pull latest images (docker compose pull). │
│ restart Restart running containers (docker compose restart). │
│ update Update stacks (pull + build + up). Shorthand for 'up --pull │
│ --build'. │
│ apply Make reality match config (start, migrate, stop │
│ strays/orphans as needed). │
│ compose Run any docker compose command on a stack. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Monitoring ─────────────────────────────────────────────────────────────────╮
│ logs Show stack logs. With --service, shows logs for just that │
│ service. │
@@ -489,6 +550,8 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
│ --all -a Run on all stacks │
│ --host -H TEXT Filter to stacks on this host │
│ --service -s TEXT Target a specific service within the stack │
│ --pull Pull images before starting (--pull always) │
│ --build Build images before starting │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
@@ -623,7 +686,7 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
Usage: cf restart [OPTIONS] [STACKS]...
Restart stacks (down + up). With --service, restarts just that service.
Restart running containers (docker compose restart).
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ stacks [STACKS]... Stacks to operate on │
@@ -658,8 +721,7 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
Usage: cf update [OPTIONS] [STACKS]...
Update stacks (pull + build + down + up). With --service, updates just that
service.
Update stacks (pull + build + up). Shorthand for 'up --pull --build'.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ stacks [STACKS]... Stacks to operate on │
@@ -694,22 +756,25 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
Usage: cf apply [OPTIONS]
Make reality match config (start, migrate, stop as needed).
Make reality match config (start, migrate, stop strays/orphans as needed).
This is the "reconcile" command that ensures running stacks match your
config file. It will:
1. Stop orphaned stacks (in state but removed from config)
2. Migrate stacks on wrong host (host in state ≠ host in config)
3. Start missing stacks (in config but not in state)
2. Stop stray stacks (running on unauthorized hosts)
3. Migrate stacks on wrong host (host in state ≠ host in config)
4. Start missing stacks (in config but not in state)
Use --dry-run to preview changes before applying.
Use --no-orphans to only migrate/start without stopping orphaned stacks.
Use --no-orphans to skip stopping orphaned stacks.
Use --no-strays to skip stopping stray stacks.
Use --full to also run 'up' on all stacks (picks up compose/env changes).
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --dry-run -n Show what would change without executing │
│ --no-orphans Only migrate, don't stop orphaned stacks │
│ --no-strays Don't stop stray stacks (running on wrong host) │
│ --full -f Also run up on all stacks to apply config │
│ changes │
│ --config -c PATH Path to config file │
@@ -964,6 +1029,7 @@ Full `--help` output for each command. See the [Usage](#usage) table above for a
│ validate Validate the config file syntax and schema. │
│ symlink Create a symlink from the default config location to a config │
│ file. │
│ init-env Generate a .env file for Docker deployment. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
@@ -1243,12 +1309,12 @@ published ports.
**Auto-regeneration**
To automatically regenerate the Traefik config after `up`, `down`, `restart`, or `update`,
To automatically regenerate the Traefik config after `up`, `down`, or `update`,
add `traefik_file` to your config:
```yaml
compose_dir: /opt/compose
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml # auto-regenerate on up/down/restart/update
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml # auto-regenerate on up/down/update
traefik_stack: traefik # skip stacks on same host (docker provider handles them)
hosts:
@@ -1287,6 +1353,60 @@ Update your Traefik config to use directory watching instead of a single file:
- --providers.file.watch=true
```
## Host Resource Monitoring (Glances)
The web UI can display real-time CPU, memory, and load stats for all configured hosts. This uses [Glances](https://nicolargo.github.io/glances/), a cross-platform system monitoring tool with a REST API.
**Setup**
1. Deploy a Glances stack that runs on all hosts:
```yaml
# glances/compose.yaml
name: glances
services:
glances:
image: nicolargo/glances:latest
container_name: glances
restart: unless-stopped
pid: host
ports:
- "61208:61208"
environment:
- GLANCES_OPT=-w # Enable web server mode
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
```
2. Add it to your config as a multi-host stack:
```yaml
# compose-farm.yaml
stacks:
glances: all # Runs on every host
glances_stack: glances # Enables resource stats in web UI
```
3. Deploy: `cf up glances`
4. **(Docker web UI only)** If running the web UI in a Docker container, set `CF_LOCAL_HOST` to your local hostname in `.env`:
```bash
echo "CF_LOCAL_HOST=nas" >> .env # Replace 'nas' with your local host name
```
This tells the web UI to reach the local Glances via container name instead of IP (required due to Docker network isolation).
The web UI dashboard will now show a "Host Resources" section with live stats from all hosts. Hosts where Glances is unreachable show an error indicator.
**Live Stats Page**
With Glances configured, a Live Stats page (`/live-stats`) shows all running containers across all hosts:
- **Columns**: Stack, Service, Host, Image, Status, Uptime, CPU, Memory, Net I/O
- **Features**: Sorting, filtering, live updates (no SSH required—uses Glances REST API)
## Comparison with Alternatives
There are many ways to run containers on multiple hosts. Here is where Compose Farm sits:

View File

@@ -3,7 +3,7 @@
compose_dir: /opt/compose
# Optional: Auto-regenerate Traefik file-provider config after up/down/restart/update
# Optional: Auto-regenerate Traefik file-provider config after up/down/update
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
traefik_stack: traefik # Skip stacks on same host (docker provider handles them)

View File

@@ -1,38 +1,58 @@
services:
cf:
image: ghcr.io/basnijholt/compose-farm:latest
# Run as current user to preserve file ownership on mounted volumes
# Set CF_UID=$(id -u) CF_GID=$(id -g) in your environment or .env file
# Defaults to root (0:0) for backwards compatibility
user: "${CF_UID:-0}:${CF_GID:-0}"
volumes:
- ${SSH_AUTH_SOCK}:/ssh-agent:ro
# Compose directory (contains compose files AND compose-farm.yaml config)
- ${CF_COMPOSE_DIR:-/opt/stacks}:${CF_COMPOSE_DIR:-/opt/stacks}
# SSH keys for passwordless auth (generated by `cf ssh setup`)
# Choose ONE option below (use the same option for both cf and web services):
# Option 1: Host path (default) - keys at ~/.ssh/compose-farm/id_ed25519
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:/root/.ssh
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:${CF_HOME:-/root}/.ssh/compose-farm
# Option 2: Named volume - managed by Docker, shared between services
# - cf-ssh:/root/.ssh
# - cf-ssh:${CF_HOME:-/root}/.ssh
# Option 3: SSH agent forwarding (uncomment if using ssh-agent)
# - ${SSH_AUTH_SOCK}:/ssh-agent:ro
environment:
- SSH_AUTH_SOCK=/ssh-agent
# Config file path (state stored alongside it)
- CF_CONFIG=${CF_COMPOSE_DIR:-/opt/stacks}/compose-farm.yaml
# HOME must match the user running the container for SSH to find keys
- HOME=${CF_HOME:-/root}
# USER is required for SSH when running as non-root (UID not in /etc/passwd)
- USER=${CF_USER:-root}
web:
image: ghcr.io/basnijholt/compose-farm:latest
restart: unless-stopped
command: web --host 0.0.0.0 --port 9000
# Run as current user to preserve file ownership on mounted volumes
user: "${CF_UID:-0}:${CF_GID:-0}"
volumes:
- ${SSH_AUTH_SOCK}:/ssh-agent:ro
- ${CF_COMPOSE_DIR:-/opt/stacks}:${CF_COMPOSE_DIR:-/opt/stacks}
# SSH keys - use the SAME option as cf service above
# Option 1: Host path (default)
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:/root/.ssh
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:${CF_HOME:-/root}/.ssh/compose-farm
# Option 2: Named volume
# - cf-ssh:/root/.ssh
# - cf-ssh:${CF_HOME:-/root}/.ssh
# Option 3: SSH agent forwarding (uncomment if using ssh-agent)
# - ${SSH_AUTH_SOCK}:/ssh-agent:ro
# XDG config dir for backups and image digest logs (persists across restarts)
- ${CF_XDG_CONFIG:-~/.config/compose-farm}:${CF_HOME:-/root}/.config/compose-farm
environment:
- SSH_AUTH_SOCK=/ssh-agent
- CF_CONFIG=${CF_COMPOSE_DIR:-/opt/stacks}/compose-farm.yaml
# Used to detect self-updates and run via SSH to survive container restart
- CF_WEB_STACK=compose-farm
# Local host for Glances (use container name instead of IP to avoid Docker network issues)
- CF_LOCAL_HOST=${CF_LOCAL_HOST:-}
# HOME must match the user running the container for SSH to find keys
- HOME=${CF_HOME:-/root}
# USER is required for SSH when running as non-root (UID not in /etc/passwd)
- USER=${CF_USER:-root}
labels:
- traefik.enable=true
- traefik.http.routers.compose-farm.rule=Host(`compose-farm.${DOMAIN}`)

View File

@@ -96,7 +96,7 @@ Typer-based CLI with subcommand modules:
cli/
├── app.py # Shared Typer app, version callback
├── common.py # Shared helpers, options, progress utilities
├── config.py # config subcommand (init, show, path, validate, edit, symlink)
├── config.py # config subcommand (init, init-env, show, path, validate, edit, symlink)
├── lifecycle.py # up, down, stop, pull, restart, update, apply, compose
├── management.py # refresh, check, init-network, traefik-file
├── monitoring.py # logs, ps, stats
@@ -343,3 +343,19 @@ For repeated connections to the same host, SSH reuses connections.
```
Icons use [Lucide](https://lucide.dev/). Add new icons as macros in `web/templates/partials/icons.html`.
### Host Resource Monitoring (`src/compose_farm/glances.py`)
Integration with [Glances](https://nicolargo.github.io/glances/) for real-time host stats:
- Fetches CPU, memory, and load from Glances REST API on each host
- Used by web UI dashboard to display host resource usage
- Requires `glances_stack` config option pointing to a Glances stack running on all hosts
### Container Registry Client (`src/compose_farm/registry.py`)
OCI Distribution API client for checking image updates:
- Parses image references (registry, namespace, name, tag, digest)
- Fetches available tags from Docker Hub, GHCR, and other registries
- Compares semantic versions to find newer releases

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bb1372a59a4ed1ac74d3864d7a84dd5311fce4cb6c6a00bf3a574bc2f98d5595
size 895927
oid sha256:01dabdd8f62773823ba2b8dc74f9931f1a1b88215117e6a080004096025491b0
size 901456

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f339a85f3d930db5a020c9f77e106edc5f44ea7dee6f68557106721493c24ef8
size 205907
oid sha256:134c903a6b3acfb933617b33755b0cdb9bac2a59e5e35b64236e248a141d396d
size 206883

3
docs/assets/compose.gif Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d8b3cdb3486ec79b3ddb2f7571c13d54ac9aed182edfe708eff76a966a90cfc7
size 1132310

3
docs/assets/compose.webm Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:a3c4d4a62f062f717df4e6752efced3caea29004dc90fe97fd7633e7f0ded9db
size 341057

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:388aa49a1269145698f9763452aaf6b9c6232ea9229abe1dae304df558e29695
size 403442
oid sha256:6c1bb48cc2f364681515a4d8bd0c586d133f5a32789b7bb64524ad7d9ed0a8e9
size 543135

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9b8bf4dcb8ee67270d4a88124b4dd4abe0dab518e73812ee73f7c66d77f146e2
size 228025
oid sha256:5f82d96137f039f21964c15c1550aa1b1f0bb2d52c04d012d253dbfbd6fad096
size 268086

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:16b9a28137dfae25488e2094de85766a039457f5dca20c2d84ac72e3967c10b9
size 164237
oid sha256:2a4045b00d90928f42c7764b3c24751576cfb68a34c6e84d12b4e282d2e67378
size 146467

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0fbe697a1f8256ce3b9a6a64c7019d42769134df9b5b964e5abe98a29e918fd
size 68242
oid sha256:f1b94416ed3740853f863e19bf45f26241a203fb0d7d187160a537f79aa544fa
size 60353

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:629b8c80b98eb996b75439745676fd99a83f391ca25f778a71bd59173f814c2f
size 1194931
oid sha256:848d9c48fb7511da7996149277c038589fad1ee406ff2f30c28f777fc441d919
size 1183641

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:33fd46f2d8538cc43be4cb553b3af9d8b412f282ee354b6373e2793fe41c799b
size 405057
oid sha256:e747ee71bb38b19946005d5a4def4d423dadeaaade452dec875c4cb2d24a5b77
size 407373

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ccd96e33faba5f297999917d89834b29d58bd2a8929eea8d62875e3d8830bd5c
size 3198466
oid sha256:d32c9a3eec06e57df085ad347e6bf61e323f8bd8322d0c540f0b9d4834196dfd
size 3589776

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:979a1a21303bbf284b3510981066ef05c41c1035b34392fecc7bee472116e6db
size 967564
oid sha256:6c54eda599389dac74c24c83527f95cd1399e653d7faf2972c2693d90e590597
size 1085344

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2067f4967a93b7ee3a8db7750c435f41b1fccd2919f3443da4b848c20cc54f23
size 124559
oid sha256:62f9b5ec71496197a3f1c3e3bca8967d603838804279ea7dbf00a70d3391ff6c
size 127123

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5471bd94e6d1b9d415547fa44de6021fdad2e1cc5b8b295680e217104aa749d6
size 98149
oid sha256:ac2b93d3630af87b44a135723c5d10e8287529bed17c28301b2802cd9593e9e8
size 98748

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:dac5660cfe6574857ec055fac7822f25b7c5fcb10a836b19c86142515e2fbf75
size 1816075
oid sha256:7b50a7e9836c496c0989363d1440fa0a6ccdaa38ee16aae92b389b3cf3c3732f
size 2385110

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:d4efec8ef5a99f2cb31d55cd71cdbf0bb8dd0cd6281571886b7c1f8b41c3f9da
size 1660764
oid sha256:ccbb3d5366c7734377e12f98cca0b361028f5722124f1bb7efa231f6aeffc116
size 2208044

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:4135888689a10c5ae2904825d98f2a6d215c174a4bd823e25761f619590f04ff
size 3990104

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:87739cd6f6576a81100392d8d1e59d3e776fecc8f0721a31332df89e7fc8593d
size 5814274

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9348dd36e79192344476d61fbbffdb122a96ecc5829fbece1818590cfc521521
size 3373003
oid sha256:269993b52721ce70674d3aab2a4cd8c58aa621d4ba0739afedae661c90965b26
size 3678371

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bebbf8151434ba37bf5e46566a4e8b57812944281926f579d056bdc835ca26aa
size 2729799
oid sha256:0098b55bb6a52fa39f807a01fa352ce112bcb446e2a2acb963fb02d21b28c934
size 3088813

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:3712afff6fcde00eb951264bb24d4301deb085d082b4e95ed4c1893a571938ee
size 1528294
oid sha256:4bf9d8c247d278799d1daea784fc662a22f12b1bd7883f808ef30f35025ebca6
size 4166443

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:0b218d400836a50661c9cdcce2d2b1e285cc5fe592cb42f58aae41f3e7d60684
size 1327413
oid sha256:02d5124217a94849bf2971d6d13d28da18c557195a81b9cca121fb7c07f0501b
size 3523244

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:6a232ddc1b9ddd9bf6b5d99c05153e1094be56f1952f02636ca498eb7484e096
size 3808675
oid sha256:412a0e68f8e52801cafbb9a703ca9577e7c14cc7c0e439160b9185961997f23c
size 4435697

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5a7c9f5f6d47074a6af135190fda6d0a1936cd7a0b04b3aa04ea7d99167a9e05
size 3333014
oid sha256:0e600a1d3216b44497a889f91eac94d62ef7207b4ed0471465dcb72408caa28e
size 3764693

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:66f4547ed2e83b302d795875588d9a085af76071a480f1096f2bb64344b80c42
size 5428670
oid sha256:3c07a283f4f70c4ab205b0f0acb5d6f55e3ced4c12caa7a8d5914ffe3548233a
size 5768166

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:75c8cdeefbbdcab2a240821d3410539f2a2cbe0a015897f4135404c80c3ac32c
size 6578366
oid sha256:562228841de976d70ee80999b930eadf3866a13ff2867d900279993744c44671
size 6667918

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ff2e3ca5a46397efcd5f3a595e7d3c179266cc4f3f5f528b428f5ef2a423028e
size 12649149
oid sha256:845746ac1cb101c3077d420c4f3fda3ca372492582dc123ac8a031a68ae9b6b1
size 12943150

View File

@@ -1,3 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2d739c5f77ddd9d90b609e31df620b35988081b7341fe225eb717d71a87caa88
size 12284953
oid sha256:189259558b5760c02583885168d7b0b47cf476cba81c7c028ec770f9d6033129
size 12415357

View File

@@ -8,14 +8,16 @@ The Compose Farm CLI is available as both `compose-farm` and the shorter alias `
## Command Overview
Commands are either **Docker Compose wrappers** (`up`, `down`, `stop`, `restart`, `pull`, `logs`, `ps`, `compose`) with multi-host superpowers, or **Compose Farm originals** (`apply`, `update`, `refresh`, `check`) for orchestration Docker Compose can't do.
| Category | Command | Description |
|----------|---------|-------------|
| **Lifecycle** | `apply` | Make reality match config |
| | `up` | Start stacks |
| | `down` | Stop stacks |
| | `stop` | Stop services without removing containers |
| | `restart` | Restart stacks (down + up) |
| | `update` | Update stacks (pull + build + down + up) |
| | `restart` | Restart running containers |
| | `update` | Shorthand for `up --pull --build` |
| | `pull` | Pull latest images |
| | `compose` | Run any docker compose command |
| **Monitoring** | `ps` | Show stack status |
@@ -36,6 +38,18 @@ cf --version, -v # Show version
cf --help, -h # Show help
```
## Command Aliases
Short aliases for frequently used commands:
| Alias | Command | Alias | Command |
|-------|---------|-------|---------|
| `cf a` | `apply` | `cf s` | `stats` |
| `cf l` | `logs` | `cf c` | `compose` |
| `cf r` | `restart` | `cf rf` | `refresh` |
| `cf u` | `update` | `cf ck` | `check` |
| `cf p` | `pull` | `cf tf` | `traefik-file` |
---
## Lifecycle Commands
@@ -58,14 +72,16 @@ cf apply [OPTIONS]
|--------|-------------|
| `--dry-run, -n` | Preview changes without executing |
| `--no-orphans` | Skip stopping orphaned stacks |
| `--full, -f` | Also refresh running stacks |
| `--no-strays` | Skip stopping stray stacks (running on wrong host) |
| `--full, -f` | Also run up on all stacks (applies compose/env changes, triggers migrations) |
| `--config, -c PATH` | Path to config file |
**What it does:**
1. Stops orphaned stacks (in state but removed from config)
2. Migrates stacks on wrong host
3. Starts missing stacks (in config but not running)
2. Stops stray stacks (running on unauthorized hosts)
3. Migrates stacks on wrong host
4. Starts missing stacks (in config but not running)
**Examples:**
@@ -79,7 +95,10 @@ cf apply
# Only start/migrate, don't stop orphans
cf apply --no-orphans
# Also refresh all running stacks
# Don't stop stray stacks
cf apply --no-strays
# Also run up on all stacks (applies compose/env changes, triggers migrations)
cf apply --full
```
@@ -100,6 +119,8 @@ cf up [OPTIONS] [STACKS]...
| `--all, -a` | Start all stacks |
| `--host, -H TEXT` | Filter to stacks on this host |
| `--service, -s TEXT` | Target a specific service within the stack |
| `--pull` | Pull images before starting (`--pull always`) |
| `--build` | Build images before starting |
| `--config, -c PATH` | Path to config file |
**Examples:**
@@ -197,7 +218,7 @@ cf stop immich --service database
### cf restart
Restart stacks (down + up). With `--service`, restarts just that service.
Restart running containers (`docker compose restart`). With `--service`, restarts just that service.
```bash
cf restart [OPTIONS] [STACKS]...
@@ -225,7 +246,7 @@ cf restart immich --service database
### cf update
Update stacks (pull + build + down + up). With `--service`, updates just that service.
Update stacks (pull + build + up). Shorthand for `up --pull --build`. With `--service`, updates just that service.
<video autoplay loop muted playsinline>
<source src="/assets/update.webm" type="video/webm">
@@ -290,6 +311,10 @@ cf pull immich --service database
Run any docker compose command on a stack. This is a passthrough to docker compose for commands not wrapped by cf.
<video autoplay loop muted playsinline>
<source src="/assets/compose.webm" type="video/webm">
</video>
```bash
cf compose [OPTIONS] STACK COMMAND [ARGS]...
```
@@ -583,6 +608,7 @@ cf config COMMAND
| Command | Description |
|---------|-------------|
| `init` | Create new config with examples |
| `init-env` | Generate .env file for Docker deployment |
| `show` | Display config with highlighting |
| `path` | Print config file path |
| `validate` | Validate syntax and schema |
@@ -594,6 +620,7 @@ cf config COMMAND
| Subcommand | Options |
|------------|---------|
| `init` | `--path/-p PATH`, `--force/-f` |
| `init-env` | `--path/-p PATH`, `--output/-o PATH`, `--force/-f` |
| `show` | `--path/-p PATH`, `--raw/-r` |
| `edit` | `--path/-p PATH` |
| `path` | `--path/-p PATH` |
@@ -629,6 +656,12 @@ cf config symlink
# Create symlink to specific file
cf config symlink /opt/compose-farm/config.yaml
# Generate .env file for Docker deployment
cf config init-env
# Generate .env in current directory
cf config init-env -o .env
```
---
@@ -649,7 +682,20 @@ cf ssh COMMAND
| `status` | Show SSH key status and host connectivity |
| `keygen` | Generate key without distributing |
**Options for `cf ssh setup` and `cf ssh keygen`:**
**Options for `cf ssh setup`:**
| Option | Description |
|--------|-------------|
| `--config, -c PATH` | Path to config file |
| `--force, -f` | Regenerate key even if it exists |
**Options for `cf ssh status`:**
| Option | Description |
|--------|-------------|
| `--config, -c PATH` | Path to config file |
**Options for `cf ssh keygen`:**
| Option | Description |
|--------|-------------|

View File

@@ -107,7 +107,7 @@ Supported compose file names (checked in order):
### traefik_file
Path to auto-generated Traefik file-provider config. When set, Compose Farm regenerates this file after `up`, `down`, `restart`, and `update` commands.
Path to auto-generated Traefik file-provider config. When set, Compose Farm regenerates this file after `up`, `down`, and `update` commands.
```yaml
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
@@ -121,6 +121,16 @@ Stack name running Traefik. Stacks on the same host are skipped in file-provider
traefik_stack: traefik
```
### glances_stack
Stack name running [Glances](https://nicolargo.github.io/glances/) for host resource monitoring. When set, the web UI displays CPU, memory, and load stats for all hosts.
```yaml
glances_stack: glances
```
The Glances stack should run on all hosts and expose port 61208. See the README for full setup instructions.
## Hosts Configuration
### Basic Host

View File

@@ -10,10 +10,10 @@ VHS-based terminal demo recordings for Compose Farm CLI.
```bash
# Record all demos
./docs/demos/cli/record.sh
python docs/demos/cli/record.py
# Record single demo
cd /opt/stacks && vhs docs/demos/cli/quickstart.tape
# Record specific demos
python docs/demos/cli/record.py quickstart migration
```
## Demos
@@ -23,6 +23,7 @@ cd /opt/stacks && vhs docs/demos/cli/quickstart.tape
| `install.tape` | Installing with `uv tool install` |
| `quickstart.tape` | `cf ps`, `cf up`, `cf logs` |
| `logs.tape` | Viewing logs |
| `compose.tape` | `cf compose` passthrough (--help, images, exec) |
| `update.tape` | `cf update` |
| `migration.tape` | Service migration |
| `apply.tape` | `cf apply` |

View File

@@ -0,0 +1,50 @@
# Compose Demo
# Shows that cf compose passes through ANY docker compose command
Output docs/assets/compose.gif
Output docs/assets/compose.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 550
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Type "# cf compose runs ANY docker compose command on the right host"
Enter
Sleep 500ms
Type "# See ALL available compose commands"
Enter
Sleep 500ms
Type "cf compose immich --help"
Enter
Sleep 4s
Type "# Show images"
Enter
Sleep 500ms
Type "cf compose immich images"
Enter
Wait+Screen /immich/
Sleep 2s
Type "# Open shell in a container"
Enter
Sleep 500ms
Type "cf compose immich exec immich-machine-learning sh"
Enter
Wait+Screen /#/
Sleep 1s
Type "python3 --version"
Enter
Sleep 1s
Type "exit"
Enter
Sleep 500ms

View File

@@ -21,7 +21,7 @@ Type "# First, define your hosts..."
Enter
Sleep 500ms
Type "bat -r 1:11 compose-farm.yaml"
Type "bat -r 1:16 compose-farm.yaml"
Enter
Sleep 3s
Type "q"
@@ -31,7 +31,7 @@ Type "# Then map each stack to a host"
Enter
Sleep 500ms
Type "bat -r 13:30 compose-farm.yaml"
Type "bat -r 17:35 compose-farm.yaml"
Enter
Sleep 3s
Type "q"

134
docs/demos/cli/record.py Executable file
View File

@@ -0,0 +1,134 @@
#!/usr/bin/env python3
"""Record CLI demos using VHS."""
import shutil
import subprocess
import sys
from pathlib import Path
from rich.console import Console
from compose_farm.config import load_config
from compose_farm.state import load_state
console = Console()
SCRIPT_DIR = Path(__file__).parent
STACKS_DIR = Path("/opt/stacks")
CONFIG_FILE = STACKS_DIR / "compose-farm.yaml"
OUTPUT_DIR = SCRIPT_DIR.parent.parent / "assets"
DEMOS = ["install", "quickstart", "logs", "compose", "update", "migration", "apply"]
def _run(cmd: list[str], **kw) -> bool:
return subprocess.run(cmd, check=False, **kw).returncode == 0
def _set_config(host: str) -> None:
"""Set audiobookshelf host in config file."""
_run(["sed", "-i", f"s/audiobookshelf: .*/audiobookshelf: {host}/", str(CONFIG_FILE)])
def _get_hosts() -> tuple[str | None, str | None]:
"""Return (config_host, state_host) for audiobookshelf."""
config = load_config()
state = load_state(config)
return config.stacks.get("audiobookshelf"), state.get("audiobookshelf")
def _setup_state(demo: str) -> bool:
"""Set up required state for demo. Returns False on failure."""
if demo not in ("migration", "apply"):
return True
config_host, state_host = _get_hosts()
if demo == "migration":
# Migration needs audiobookshelf on nas in BOTH config and state
if config_host != "nas":
console.print("[yellow]Setting up: config → nas[/yellow]")
_set_config("nas")
if state_host != "nas":
console.print("[yellow]Setting up: state → nas[/yellow]")
if not _run(["cf", "apply"], cwd=STACKS_DIR):
return False
elif demo == "apply":
# Apply needs config=nas, state=anton (so there's something to apply)
if config_host != "nas":
console.print("[yellow]Setting up: config → nas[/yellow]")
_set_config("nas")
if state_host == "nas":
console.print("[yellow]Setting up: state → anton[/yellow]")
_set_config("anton")
if not _run(["cf", "apply"], cwd=STACKS_DIR):
return False
_set_config("nas")
return True
def _record(name: str, index: int, total: int) -> bool:
"""Record a single demo."""
console.print(f"[cyan][{index}/{total}][/cyan] [green]Recording:[/green] {name}")
if _run(["vhs", str(SCRIPT_DIR / f"{name}.tape")], cwd=STACKS_DIR):
console.print("[green] ✓ Done[/green]")
return True
console.print("[red] ✗ Failed[/red]")
return False
def _reset_after(demo: str, next_demo: str | None) -> None:
"""Reset state after demos that modify audiobookshelf."""
if demo not in ("quickstart", "migration"):
return
_set_config("nas")
if next_demo != "apply": # Let apply demo show the migration
_run(["cf", "apply"], cwd=STACKS_DIR)
def _restore_config(original: str) -> None:
"""Restore original config and sync state."""
console.print("[yellow]Restoring original config...[/yellow]")
CONFIG_FILE.write_text(original)
_run(["cf", "apply"], cwd=STACKS_DIR)
def _main() -> int:
if not shutil.which("vhs"):
console.print("[red]VHS not found. Install: brew install vhs[/red]")
return 1
if not _run(["git", "-C", str(STACKS_DIR), "diff", "--quiet", "compose-farm.yaml"]):
console.print("[red]compose-farm.yaml has uncommitted changes[/red]")
return 1
demos = [d for d in sys.argv[1:] if d in DEMOS] or DEMOS
if sys.argv[1:] and not demos:
console.print(f"[red]Unknown demo. Available: {', '.join(DEMOS)}[/red]")
return 1
# Save original config to restore after recording
original_config = CONFIG_FILE.read_text()
try:
for i, demo in enumerate(demos, 1):
if not _setup_state(demo):
return 1
if not _record(demo, i, len(demos)):
return 1
_reset_after(demo, demos[i] if i < len(demos) else None)
finally:
_restore_config(original_config)
# Move outputs
OUTPUT_DIR.mkdir(exist_ok=True)
for f in (STACKS_DIR / "docs/assets").glob("*.[gw]*"):
shutil.move(str(f), str(OUTPUT_DIR / f.name))
console.print(f"\n[green]Done![/green] Saved to {OUTPUT_DIR}")
return 0
if __name__ == "__main__":
sys.exit(_main())

View File

@@ -1,89 +0,0 @@
#!/usr/bin/env bash
# Record all VHS demos
# Run this on a Docker host with compose-farm configured
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DEMOS_DIR="$(dirname "$SCRIPT_DIR")"
DOCS_DIR="$(dirname "$DEMOS_DIR")"
REPO_DIR="$(dirname "$DOCS_DIR")"
OUTPUT_DIR="$DOCS_DIR/assets"
# Colors
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[0;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
# Check for VHS
if ! command -v vhs &> /dev/null; then
echo "VHS not found. Install with:"
echo " brew install vhs"
echo " # or"
echo " go install github.com/charmbracelet/vhs@latest"
exit 1
fi
# Ensure output directory exists
mkdir -p "$OUTPUT_DIR"
# Temp output dir (VHS runs from /opt/stacks, so relative paths go here)
TEMP_OUTPUT="/opt/stacks/docs/assets"
mkdir -p "$TEMP_OUTPUT"
# Change to /opt/stacks so cf commands use installed version (not editable install)
cd /opt/stacks
# Ensure compose-farm.yaml has no uncommitted changes (safety check)
if ! git diff --quiet compose-farm.yaml; then
echo -e "${RED}Error: compose-farm.yaml has uncommitted changes${NC}"
echo "Commit or stash your changes before recording demos"
exit 1
fi
echo -e "${BLUE}Recording VHS demos...${NC}"
echo "Output directory: $OUTPUT_DIR"
echo ""
# Function to record a tape
record_tape() {
local tape=$1
local name=$(basename "$tape" .tape)
echo -e "${GREEN}Recording:${NC} $name"
if vhs "$tape"; then
echo -e "${GREEN} ✓ Done${NC}"
else
echo -e "${RED} ✗ Failed${NC}"
return 1
fi
}
# Record demos in logical order
echo -e "${YELLOW}=== Phase 1: Basic demos ===${NC}"
record_tape "$SCRIPT_DIR/install.tape"
record_tape "$SCRIPT_DIR/quickstart.tape"
record_tape "$SCRIPT_DIR/logs.tape"
echo -e "${YELLOW}=== Phase 2: Update demo ===${NC}"
record_tape "$SCRIPT_DIR/update.tape"
echo -e "${YELLOW}=== Phase 3: Migration demo ===${NC}"
record_tape "$SCRIPT_DIR/migration.tape"
git -C /opt/stacks checkout compose-farm.yaml # Reset after migration
echo -e "${YELLOW}=== Phase 4: Apply demo ===${NC}"
record_tape "$SCRIPT_DIR/apply.tape"
# Move GIFs and WebMs from temp location to repo
echo ""
echo -e "${BLUE}Moving recordings to repo...${NC}"
mv "$TEMP_OUTPUT"/*.gif "$OUTPUT_DIR/" 2>/dev/null || true
mv "$TEMP_OUTPUT"/*.webm "$OUTPUT_DIR/" 2>/dev/null || true
rmdir "$TEMP_OUTPUT" 2>/dev/null || true
rmdir "$(dirname "$TEMP_OUTPUT")" 2>/dev/null || true
echo ""
echo -e "${GREEN}Done!${NC} Recordings saved to $OUTPUT_DIR/"
ls -la "$OUTPUT_DIR"/*.gif "$OUTPUT_DIR"/*.webm 2>/dev/null || echo "No recordings found (check for errors above)"

View File

@@ -1,5 +1,5 @@
# Update Demo
# Shows updating stacks (pull + build + down + up)
# Shows updating stacks (only recreates containers if images changed)
Output docs/assets/update.gif
Output docs/assets/update.webm

View File

@@ -21,24 +21,37 @@ import uvicorn
from compose_farm.config import Config as CFConfig
from compose_farm.config import load_config
from compose_farm.executor import (
get_container_compose_labels as _original_get_compose_labels,
)
from compose_farm.glances import ContainerStats
from compose_farm.glances import fetch_container_stats as _original_fetch_container_stats
from compose_farm.state import load_state as _original_load_state
from compose_farm.web.app import create_app
from compose_farm.web.cdn import CDN_ASSETS, ensure_vendor_cache
# NOTE: Do NOT import create_app here - it must be imported AFTER patches are applied
# to ensure the patched get_config is used by all route modules
if TYPE_CHECKING:
from collections.abc import Generator
from playwright.sync_api import BrowserContext, Page, Route
# Stacks to exclude from demo recordings (exact match)
DEMO_EXCLUDE_STACKS = {"arr"}
# Substrings to exclude from demo recordings (case-insensitive)
DEMO_EXCLUDE_PATTERNS = {"arr", "vpn", "tash"}
def _should_exclude(name: str) -> bool:
"""Check if a stack/container name should be excluded from demo."""
name_lower = name.lower()
return any(pattern in name_lower for pattern in DEMO_EXCLUDE_PATTERNS)
def _get_filtered_config() -> CFConfig:
"""Load config but filter out excluded stacks."""
config = load_config()
filtered_stacks = {
name: host for name, host in config.stacks.items() if name not in DEMO_EXCLUDE_STACKS
name: host for name, host in config.stacks.items() if not _should_exclude(name)
}
return CFConfig(
compose_dir=config.compose_dir,
@@ -46,6 +59,7 @@ def _get_filtered_config() -> CFConfig:
stacks=filtered_stacks,
traefik_file=config.traefik_file,
traefik_stack=config.traefik_stack,
glances_stack=config.glances_stack,
config_path=config.config_path,
)
@@ -53,7 +67,37 @@ def _get_filtered_config() -> CFConfig:
def _get_filtered_state(config: CFConfig) -> dict[str, str | list[str]]:
"""Load state but filter out excluded stacks."""
state = _original_load_state(config)
return {name: host for name, host in state.items() if name not in DEMO_EXCLUDE_STACKS}
return {name: host for name, host in state.items() if not _should_exclude(name)}
async def _filtered_fetch_container_stats(
host_name: str,
host_address: str,
port: int = 61208,
request_timeout: float = 10.0,
) -> tuple[list[ContainerStats] | None, str | None]:
"""Fetch container stats but filter out excluded containers."""
containers, error = await _original_fetch_container_stats(
host_name, host_address, port, request_timeout
)
if containers:
# Filter by container name (stack is empty at this point)
containers = [c for c in containers if not _should_exclude(c.name)]
return containers, error
async def _filtered_get_compose_labels(
config: CFConfig,
host_name: str,
) -> dict[str, tuple[str, str]]:
"""Get compose labels but filter out excluded stacks."""
labels = await _original_get_compose_labels(config, host_name)
# Filter out containers whose stack (project) name should be excluded
return {
name: (stack, service)
for name, (stack, service) in labels.items()
if not _should_exclude(stack)
}
@pytest.fixture(scope="session")
@@ -84,19 +128,23 @@ def server_url() -> Generator[str, None, None]:
# Patch at source module level so all callers get filtered versions
patches = [
# Patch load_state at source - all functions calling it get filtered state
# Patch load_config at source - get_config() calls this internally
patch("compose_farm.config.load_config", _get_filtered_config),
# Patch load_state at source and where imported
patch("compose_farm.state.load_state", _get_filtered_state),
# Patch get_config where imported
patch("compose_farm.web.routes.pages.get_config", _get_filtered_config),
patch("compose_farm.web.routes.api.get_config", _get_filtered_config),
patch("compose_farm.web.routes.actions.get_config", _get_filtered_config),
patch("compose_farm.web.app.get_config", _get_filtered_config),
patch("compose_farm.web.ws.get_config", _get_filtered_config),
patch("compose_farm.web.routes.pages.load_state", _get_filtered_state),
# Patch container fetch to filter out excluded containers (Live Stats page)
patch("compose_farm.glances.fetch_container_stats", _filtered_fetch_container_stats),
# Patch compose labels to filter out excluded stacks
patch("compose_farm.executor.get_container_compose_labels", _filtered_get_compose_labels),
]
for p in patches:
p.start()
# Import create_app AFTER patches are started so route modules see patched get_config
from compose_farm.web.app import create_app # noqa: PLC0415
with socket.socket() as s:
s.bind(("127.0.0.1", 0))
port = s.getsockname()[1]
@@ -160,6 +208,7 @@ def recording_context(
if url.startswith(url_prefix):
route.fulfill(status=200, content_type=content_type, body=filepath.read_bytes())
return
print(f"UNCACHED CDN request: {url}")
route.abort("failed")
context.route(re.compile(r"https://(cdn\.jsdelivr\.net|unpkg\.com)/.*"), handle_cdn)
@@ -176,6 +225,35 @@ def recording_page(recording_context: BrowserContext) -> Generator[Page, None, N
page.close()
@pytest.fixture
def wide_recording_context(
browser: Any, # pytest-playwright's browser fixture
recording_output_dir: Path,
) -> Generator[BrowserContext, None, None]:
"""Browser context with wider viewport for demos needing more horizontal space.
NOTE: This fixture does NOT use CDN interception (unlike recording_context).
CDN interception was causing inline scripts from containers.html to be
removed from the DOM, likely due to Tailwind's browser plugin behavior.
"""
context = browser.new_context(
viewport={"width": 1920, "height": 1080},
record_video_dir=str(recording_output_dir),
record_video_size={"width": 1920, "height": 1080},
)
yield context
context.close()
@pytest.fixture
def wide_recording_page(wide_recording_context: BrowserContext) -> Generator[Page, None, None]:
"""Page with wider viewport for demos needing more horizontal space."""
page = wide_recording_context.new_page()
yield page
page.close()
# Demo helper functions

View File

@@ -60,10 +60,14 @@ def test_demo_console(recording_page: Page, server_url: str) -> None:
page.keyboard.press("Enter")
pause(page, 2500) # Wait for output
# Scroll down to show the Editor section with Compose Farm config
editor_section = page.locator(".collapse", has_text="Editor").first
editor_section.scroll_into_view_if_needed()
pause(page, 800)
# Smoothly scroll down to show the Editor section with Compose Farm config
page.evaluate("""
const editor = document.getElementById('console-editor');
if (editor) {
editor.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
""")
pause(page, 1200) # Wait for smooth scroll animation
# Wait for Monaco editor to load with config content
page.wait_for_selector("#console-editor .monaco-editor", timeout=10000)

View File

@@ -0,0 +1,85 @@
"""Demo: Live Stats page.
Records a ~20 second demo showing:
- Navigating to Live Stats via command palette
- Container table with real-time stats
- Filtering containers
- Sorting by different columns
- Auto-refresh countdown
Run: pytest docs/demos/web/demo_live_stats.py -v --no-cov
"""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
from conftest import (
open_command_palette,
pause,
slow_type,
wait_for_sidebar,
)
if TYPE_CHECKING:
from playwright.sync_api import Page
@pytest.mark.browser # type: ignore[misc]
def test_demo_live_stats(wide_recording_page: Page, server_url: str) -> None:
"""Record Live Stats page demo."""
page = wide_recording_page
# Start on dashboard
page.goto(server_url)
wait_for_sidebar(page)
pause(page, 1000)
# Navigate to Live Stats via command palette
open_command_palette(page)
pause(page, 400)
slow_type(page, "#cmd-input", "live", delay=100)
pause(page, 500)
page.keyboard.press("Enter")
page.wait_for_url("**/live-stats", timeout=5000)
# Wait for containers to load (may take ~10s on first load due to SSH)
page.wait_for_selector("#container-rows tr:not(:has(.loading))", timeout=30000)
pause(page, 2000) # Let viewer see the full table with timer
# Demonstrate filtering
slow_type(page, "#filter-input", "grocy", delay=100)
pause(page, 1500) # Show filtered results
# Clear filter
page.fill("#filter-input", "")
pause(page, 1000)
# Sort by memory (click header)
page.click("th:has-text('Mem')")
pause(page, 1500)
# Sort by CPU
page.click("th:has-text('CPU')")
pause(page, 1500)
# Sort by host
page.click("th:has-text('Host')")
pause(page, 1500)
# Watch auto-refresh timer count down
pause(page, 3500) # Wait for refresh to happen
# Hover on action menu to show pause behavior
action_btn = page.locator('button[onclick^="openActionMenu"]').first
action_btn.scroll_into_view_if_needed()
action_btn.hover()
pause(page, 2000) # Show paused state (timer shows ⏸) and action menu
# Move away to close menu and resume refresh
page.locator("h2").first.hover() # Move to header
pause(page, 3500) # Watch countdown resume and refresh happen
# Final pause
pause(page, 1000)

View File

@@ -1,9 +1,11 @@
"""Demo: Container shell exec.
"""Demo: Container shell exec via command palette.
Records a ~25 second demo showing:
- Navigating to a stack page
- Clicking Shell button on a container
- Running top command inside the container
Records a ~35 second demo showing:
- Navigating to immich stack (multiple containers)
- Using command palette with fuzzy matching ("sh mach") to open shell
- Running a command
- Using command palette to switch to server container shell
- Running another command
Run: pytest docs/demos/web/demo_shell.py -v --no-cov
"""
@@ -14,6 +16,7 @@ from typing import TYPE_CHECKING
import pytest
from conftest import (
open_command_palette,
pause,
slow_type,
wait_for_sidebar,
@@ -33,39 +36,71 @@ def test_demo_shell(recording_page: Page, server_url: str) -> None:
wait_for_sidebar(page)
pause(page, 800)
# Navigate to a stack with a running container (grocy)
page.locator("#sidebar-stacks a", has_text="grocy").click()
page.wait_for_url("**/stack/grocy", timeout=5000)
# Navigate to immich via command palette (has multiple containers)
open_command_palette(page)
pause(page, 400)
slow_type(page, "#cmd-input", "immich", delay=100)
pause(page, 600)
page.keyboard.press("Enter")
page.wait_for_url("**/stack/immich", timeout=5000)
pause(page, 1500)
# Wait for containers list to load (loaded via HTMX)
# Wait for containers list to load (so shell commands are available)
page.wait_for_selector("#containers-list button", timeout=10000)
pause(page, 800)
# Click Shell button on the first container
shell_btn = page.locator("#containers-list button", has_text="Shell").first
shell_btn.click()
# Use command palette with fuzzy matching: "sh mach" -> "Shell: immich-machine-learning"
open_command_palette(page)
pause(page, 400)
slow_type(page, "#cmd-input", "sh mach", delay=100)
pause(page, 600)
page.keyboard.press("Enter")
pause(page, 1000)
# Wait for exec terminal to appear
page.wait_for_selector("#exec-terminal .xterm", timeout=10000)
# Scroll down to make the terminal visible
page.locator("#exec-terminal").scroll_into_view_if_needed()
pause(page, 2000)
# Smoothly scroll down to make the terminal visible
page.evaluate("""
const terminal = document.getElementById('exec-terminal');
if (terminal) {
terminal.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
""")
pause(page, 1200)
# Run top command
slow_type(page, "#exec-terminal .xterm-helper-textarea", "top", delay=100)
# Run python version command
slow_type(page, "#exec-terminal .xterm-helper-textarea", "python3 --version", delay=60)
pause(page, 300)
page.keyboard.press("Enter")
pause(page, 4000) # Let top run for a bit
pause(page, 1500)
# Press q to quit top
page.keyboard.press("q")
# Blur the terminal to release focus (won't scroll)
page.evaluate("document.activeElement?.blur()")
pause(page, 500)
# Use command palette to switch to server container: "sh serv" -> "Shell: immich-server"
open_command_palette(page)
pause(page, 400)
slow_type(page, "#cmd-input", "sh serv", delay=100)
pause(page, 600)
page.keyboard.press("Enter")
pause(page, 1000)
# Run another command to show it's interactive
slow_type(page, "#exec-terminal .xterm-helper-textarea", "ps aux | head", delay=60)
# Wait for new terminal
page.wait_for_selector("#exec-terminal .xterm", timeout=10000)
# Scroll to terminal
page.evaluate("""
const terminal = document.getElementById('exec-terminal');
if (terminal) {
terminal.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
""")
pause(page, 1200)
# Run ls command
slow_type(page, "#exec-terminal .xterm-helper-textarea", "ls /usr/src/app", delay=60)
pause(page, 300)
page.keyboard.press("Enter")
pause(page, 2000)

View File

@@ -55,9 +55,14 @@ def test_demo_stack(recording_page: Page, server_url: str) -> None:
page.wait_for_selector("#compose-editor .monaco-editor", timeout=10000)
pause(page, 2000) # Let viewer see the compose file
# Scroll down slightly to show more of the editor
page.locator("#compose-editor").scroll_into_view_if_needed()
pause(page, 1500)
# Smoothly scroll down to show more of the editor
page.evaluate("""
const editor = document.getElementById('compose-editor');
if (editor) {
editor.scrollIntoView({ behavior: 'smooth', block: 'center' });
}
""")
pause(page, 1200) # Wait for smooth scroll animation
# Close the compose file section
compose_collapse.locator("input[type=checkbox]").click(force=True)

View File

@@ -5,7 +5,7 @@ Records a comprehensive demo (~60 seconds) combining all major features:
2. Editor showing Compose Farm YAML config
3. Command palette navigation to grocy stack
4. Stack actions: up, logs
5. Switch to mealie stack via command palette, run update
5. Switch to dozzle stack via command palette, run update
6. Dashboard overview
7. Theme cycling via command palette
@@ -126,13 +126,13 @@ def _demo_stack_actions(page: Page) -> None:
page.wait_for_selector("#terminal-output .xterm", timeout=5000)
pause(page, 2500)
# Switch to mealie via command palette
# Switch to dozzle via command palette (on nas for lower latency)
open_command_palette(page)
pause(page, 300)
slow_type(page, "#cmd-input", "mealie", delay=100)
slow_type(page, "#cmd-input", "dozzle", delay=100)
pause(page, 400)
page.keyboard.press("Enter")
page.wait_for_url("**/stack/mealie", timeout=5000)
page.wait_for_url("**/stack/dozzle", timeout=5000)
pause(page, 1000)
# Run update action
@@ -162,32 +162,20 @@ def _demo_dashboard_and_themes(page: Page, server_url: str) -> None:
page.evaluate("window.scrollTo(0, 0)")
pause(page, 600)
# Open theme picker and arrow down to Luxury (shows live preview)
# Theme order: light, dark, cupcake, bumblebee, emerald, corporate, synthwave,
# retro, cyberpunk, valentine, halloween, garden, forest, aqua, lofi, pastel,
# fantasy, wireframe, black, luxury (index 19)
# Open theme picker and arrow down to Dracula (shows live preview)
page.locator("#theme-btn").click()
page.wait_for_selector("#cmd-palette[open]", timeout=2000)
pause(page, 400)
# Arrow down through themes with live preview until we reach Luxury
# Arrow down through themes with live preview until we reach Dracula
for _ in range(19):
page.keyboard.press("ArrowDown")
pause(page, 180)
# Select Luxury theme
# Select Dracula theme and end on it
pause(page, 400)
page.keyboard.press("Enter")
pause(page, 1000)
# Return to dark theme
page.locator("#theme-btn").click()
page.wait_for_selector("#cmd-palette[open]", timeout=2000)
pause(page, 300)
slow_type(page, "#cmd-input", " dark", delay=80)
pause(page, 400)
page.keyboard.press("Enter")
pause(page, 1000)
pause(page, 1500)
@pytest.mark.browser # type: ignore[misc]

View File

@@ -37,6 +37,7 @@ DEMOS = [
"workflow",
"console",
"shell",
"live_stats",
]
# High-quality ffmpeg settings for VP8 encoding
@@ -88,9 +89,9 @@ def patch_playwright_video_quality() -> None:
console.print("[green]Patched Playwright for high-quality video recording[/green]")
def record_demo(name: str) -> Path | None:
def record_demo(name: str, index: int, total: int) -> Path | None:
"""Run a single demo and return the video path."""
console.print(f"[green]Recording:[/green] web-{name}")
console.print(f"[cyan][{index}/{total}][/cyan] [green]Recording:[/green] web-{name}")
demo_file = SCRIPT_DIR / f"demo_{name}.py"
if not demo_file.exists():
@@ -227,9 +228,7 @@ def main() -> int:
try:
for i, demo in enumerate(demos_to_record, 1):
console.print(f"[yellow]=== Demo {i}/{len(demos_to_record)}: {demo} ===[/yellow]")
video_path = record_demo(demo)
video_path = record_demo(demo, i, len(demos_to_record))
if video_path:
webm, gif = move_recording(video_path, demo)
results[demo] = (webm, gif)

116
docs/docker-deployment.md Normal file
View File

@@ -0,0 +1,116 @@
---
icon: lucide/container
---
# Docker Deployment
Run the Compose Farm web UI in Docker.
## Quick Start
**1. Get the compose file:**
```bash
curl -O https://raw.githubusercontent.com/basnijholt/compose-farm/main/docker-compose.yml
```
**2. Generate `.env` file:**
```bash
cf config init-env -o .env
```
This auto-detects settings from your `compose-farm.yaml`:
- `DOMAIN` from existing traefik labels
- `CF_COMPOSE_DIR` from config
- `CF_UID/GID/HOME/USER` from current user
- `CF_LOCAL_HOST` by matching local IPs to config hosts
Review the output and edit if needed.
**3. Set up SSH keys:**
```bash
docker compose run --rm cf ssh setup
```
**4. Start the web UI:**
```bash
docker compose up -d web
```
Open `http://localhost:9000` (or `https://compose-farm.example.com` if using Traefik).
---
## Configuration
The `cf config init-env` command auto-detects most settings. After running it, review the generated `.env` file and edit if needed:
```bash
$EDITOR .env
```
### What init-env detects
| Variable | How it's detected |
|----------|-------------------|
| `DOMAIN` | Extracted from traefik labels in your stacks |
| `CF_COMPOSE_DIR` | From `compose_dir` in your config |
| `CF_UID/GID/HOME/USER` | From current user (for NFS compatibility) |
| `CF_LOCAL_HOST` | By matching local IPs to configured hosts |
If auto-detection fails for any value, edit the `.env` file manually.
### Glances Monitoring
To show host CPU/memory stats in the dashboard, deploy [Glances](https://nicolargo.github.io/glances/) on your hosts. If `CF_LOCAL_HOST` wasn't detected correctly, set it to your local hostname:
```bash
CF_LOCAL_HOST=nas # Replace with your local host name
```
See [Host Resource Monitoring](https://github.com/basnijholt/compose-farm#host-resource-monitoring-glances) in the README.
---
## Troubleshooting
### SSH "Permission denied" or "Host key verification failed"
Regenerate keys:
```bash
docker compose run --rm cf ssh setup
```
### Glances shows error for local host
Add your local hostname to `.env`:
```bash
echo "CF_LOCAL_HOST=nas" >> .env
docker compose restart web
```
### Files created as root
Add the non-root variables above and restart.
---
## All Environment Variables
For advanced users, here's the complete reference:
| Variable | Description | Default |
|----------|-------------|---------|
| `DOMAIN` | Domain for Traefik labels | *(required)* |
| `CF_COMPOSE_DIR` | Compose files directory | `/opt/stacks` |
| `CF_UID` / `CF_GID` | User/group ID | `0` (root) |
| `CF_HOME` | Home directory | `/root` |
| `CF_USER` | Username for SSH | `root` |
| `CF_LOCAL_HOST` | Local hostname for Glances | *(auto-detect)* |
| `CF_SSH_DIR` | SSH keys directory | `~/.ssh/compose-farm` |
| `CF_XDG_CONFIG` | Config/backup directory | `~/.config/compose-farm` |

View File

@@ -54,6 +54,25 @@ docker run --rm \
ghcr.io/basnijholt/compose-farm up --all
```
**Running as non-root user** (recommended for NFS mounts):
By default, containers run as root. To preserve file ownership on mounted volumes, set these environment variables in your `.env` file:
```bash
# Add to .env file (one-time setup)
echo "CF_UID=$(id -u)" >> .env
echo "CF_GID=$(id -g)" >> .env
echo "CF_HOME=$HOME" >> .env
echo "CF_USER=$USER" >> .env
```
Or use [direnv](https://direnv.net/) to auto-set these variables when entering the directory:
```bash
cp .envrc.example .envrc && direnv allow
```
This ensures files like `compose-farm-state.yaml` and web UI edits are owned by your user instead of root. The `CF_USER` variable is required for SSH to work when running as a non-root user.
### Verify Installation
```bash
@@ -310,7 +329,7 @@ cf apply
```bash
cf update --all
# Runs: pull + build + down + up for each stack
# Only recreates containers if images changed
```
## Next Steps

View File

@@ -139,7 +139,6 @@ stacks:
With `traefik_file` set, these commands auto-regenerate the config:
- `cf up`
- `cf down`
- `cf restart`
- `cf update`
- `cf apply`

View File

@@ -51,10 +51,32 @@ Press `Ctrl+K` (or `Cmd+K` on macOS) to open the command palette. Use fuzzy sear
### Dashboard (`/`)
- Stack overview with status indicators
- Host statistics
- Host statistics (CPU, memory, disk, load via Glances)
- Pending operations (migrations, orphaned stacks)
- Quick actions via command palette
### Live Stats (`/live-stats`)
Real-time container monitoring across all hosts, powered by [Glances](https://nicolargo.github.io/glances/).
- **Live metrics**: CPU, memory, network I/O for every container
- **Auto-refresh**: Updates every 3 seconds (pauses when dropdown menus are open)
- **Filtering**: Type to filter containers by name, stack, host, or image
- **Sorting**: Click column headers to sort by any metric
- **Update detection**: Shows when container images have updates available
<video autoplay loop muted playsinline>
<source src="/assets/web-live_stats.webm" type="video/webm">
</video>
#### Requirements
Live Stats requires Glances to be deployed on all hosts:
1. Add `glances_stack: glances` to your `compose-farm.yaml`
2. Deploy a Glances stack that runs on all hosts (see [example](https://github.com/basnijholt/compose-farm/tree/main/examples/glances))
3. Glances must expose its REST API on port 61208
### Stack Detail (`/stack/{name}`)
- Compose file editor (Monaco)
@@ -63,6 +85,8 @@ Press `Ctrl+K` (or `Cmd+K` on macOS) to open the command palette. Use fuzzy sear
- Container shell access (exec into running containers)
- Terminal output for running commands
Files are automatically backed up before saving to `~/.config/compose-farm/backups/`.
### Console (`/console`)
- Full shell access to any host

View File

@@ -7,9 +7,10 @@ Real-world examples demonstrating compose-farm patterns for multi-host Docker de
| Stack | Type | Demonstrates |
|---------|------|--------------|
| [traefik](traefik/) | Infrastructure | Reverse proxy, Let's Encrypt, file-provider |
| [coredns](coredns/) | Infrastructure | Wildcard DNS for `*.local` domains |
| [mealie](mealie/) | Single container | Traefik labels, resource limits, environment vars |
| [uptime-kuma](uptime-kuma/) | Single container | Docker socket, user mapping, custom DNS |
| [paperless-ngx](paperless-ngx/) | Multi-container | Redis + App stack (SQLite) |
| [paperless-ngx](paperless-ngx/) | Multi-container | Redis + PostgreSQL + App stack |
| [autokuma](autokuma/) | Multi-host | Demonstrates `all` keyword (runs on every host) |
## Key Patterns
@@ -53,7 +54,8 @@ labels:
- traefik.http.routers.myapp-local.entrypoints=web
```
> **Note:** `.local` domains require local DNS (e.g., Pi-hole, Technitium) to resolve to your Traefik host.
> **Note:** `.local` domains require local DNS to resolve to your Traefik host.
> The [coredns](coredns/) example provides this - edit `Corefile` to set your Traefik IP.
### Environment Variables
@@ -88,23 +90,6 @@ stacks:
autokuma: all # Runs on every configured host
```
### Multi-Container Stacks
Database-backed apps with multiple services:
```yaml
services:
redis:
image: redis:7
app:
depends_on:
- redis
```
> **NFS + PostgreSQL Warning:** PostgreSQL should NOT run on NFS storage due to
> fsync and file locking issues. Use SQLite (safe for single-writer on NFS) or
> keep PostgreSQL data on local volumes (non-migratable).
### AutoKuma Labels (Optional)
The autokuma example demonstrates compose-farm's **multi-host feature** - running the same stack on all hosts using the `all` keyword. AutoKuma itself is not part of compose-farm; it's just a good example because it needs to run on every host to monitor local Docker containers.
@@ -125,8 +110,8 @@ cd examples
# 1. Create the shared network on all hosts
compose-farm init-network
# 2. Start Traefik first (the reverse proxy)
compose-farm up traefik
# 2. Start infrastructure (reverse proxy + DNS)
compose-farm up traefik coredns
# 3. Start other stacks
compose-farm up mealie uptime-kuma
@@ -168,4 +153,4 @@ traefik_file: /opt/stacks/traefik/dynamic.d/compose-farm.yml
traefik_stack: traefik
```
With `traefik_file` configured, compose-farm automatically regenerates the config after `up`, `down`, `restart`, and `update` commands.
With `traefik_file` configured, compose-farm automatically regenerates the config after `up`, `down`, and `update` commands.

View File

@@ -3,6 +3,7 @@ deployed:
- primary
- secondary
- local
coredns: primary
mealie: secondary
paperless-ngx: primary
traefik: primary

View File

@@ -5,7 +5,7 @@
compose_dir: /opt/stacks/compose-farm/examples
# Auto-regenerate Traefik file-provider config after up/down/restart/update
# Auto-regenerate Traefik file-provider config after up/down/update
traefik_file: /opt/stacks/compose-farm/examples/traefik/dynamic.d/compose-farm.yml
traefik_stack: traefik # Skip Traefik's host in file-provider (docker provider handles it)
@@ -27,6 +27,7 @@ hosts:
stacks:
# Infrastructure (runs on primary where Traefik is)
traefik: primary
coredns: primary # DNS for *.local resolution
# Multi-host stacks (runs on ALL hosts)
# AutoKuma monitors Docker containers on each host

2
examples/coredns/.env Normal file
View File

@@ -0,0 +1,2 @@
# CoreDNS doesn't need environment variables
# The Traefik IP is configured in the Corefile

22
examples/coredns/Corefile Normal file
View File

@@ -0,0 +1,22 @@
# CoreDNS configuration for .local domain resolution
#
# Resolves *.local to the Traefik host IP (where your reverse proxy runs).
# All other queries are forwarded to upstream DNS.
# Handle .local domains - resolve everything to Traefik's host
local {
template IN A {
answer "{{ .Name }} 60 IN A 192.168.1.10"
}
template IN AAAA {
# Return empty for AAAA to avoid delays on IPv4-only networks
rcode NOERROR
}
}
# Forward everything else to upstream DNS
. {
forward . 1.1.1.1 8.8.8.8
cache 300
errors
}

View File

@@ -0,0 +1,27 @@
# CoreDNS - DNS server for .local domain resolution
#
# Demonstrates:
# - Wildcard DNS for *.local domains
# - Config file mounting from stack directory
# - UDP/TCP port exposure
#
# This enables all the .local routes in the examples to work.
# Point your devices/router DNS to this server's IP.
name: coredns
services:
coredns:
image: coredns/coredns:latest
container_name: coredns
restart: unless-stopped
networks:
- mynetwork
ports:
- "53:53/udp"
- "53:53/tcp"
volumes:
- ./Corefile:/root/Corefile:ro
command: -conf /root/Corefile
networks:
mynetwork:
external: true

View File

@@ -1,3 +1,4 @@
# Copy to .env and fill in your values
DOMAIN=example.com
PAPERLESS_SECRET_KEY=change-me-to-a-random-string
POSTGRES_PASSWORD=change-me-to-a-secure-password
PAPERLESS_SECRET_KEY=change-me-to-a-long-random-string

View File

@@ -1,44 +1,57 @@
# Paperless-ngx - Document management system
#
# Demonstrates:
# - HTTPS route: paperless.${DOMAIN} (e.g., paperless.example.com) with Let's Encrypt
# - HTTP route: paperless.local for LAN access without TLS
# - Multi-container stack (Redis + App with SQLite)
#
# NOTE: This example uses SQLite (the default) instead of PostgreSQL.
# PostgreSQL should NOT be used with NFS storage due to fsync/locking issues.
# If you need PostgreSQL, use local volumes for the database.
# - HTTPS route: paperless.${DOMAIN} with Let's Encrypt
# - HTTP route: paperless.local for LAN access
# - Multi-container stack (Redis + PostgreSQL + App)
# - Separate env_file for app-specific settings
name: paperless-ngx
services:
redis:
image: redis:8
broker:
image: redis:7
container_name: paperless-redis
restart: unless-stopped
networks:
- mynetwork
volumes:
- /mnt/data/paperless/redis:/data
- /mnt/data/paperless/redisdata:/data
db:
image: postgres:16
container_name: paperless-db
restart: unless-stopped
networks:
- mynetwork
volumes:
- /mnt/data/paperless/pgdata:/var/lib/postgresql/data
environment:
POSTGRES_DB: paperless
POSTGRES_USER: paperless
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
paperless:
image: ghcr.io/paperless-ngx/paperless-ngx:latest
container_name: paperless
restart: unless-stopped
depends_on:
- redis
- db
- broker
networks:
- mynetwork
ports:
- "8000:8000"
volumes:
# SQLite database stored here (safe on NFS for single-writer)
- /mnt/data/paperless/data:/usr/src/paperless/data
- /mnt/data/paperless/media:/usr/src/paperless/media
- /mnt/data/paperless/export:/usr/src/paperless/export
- /mnt/data/paperless/consume:/usr/src/paperless/consume
environment:
PAPERLESS_REDIS: redis://redis:6379
PAPERLESS_REDIS: redis://broker:6379
PAPERLESS_DBHOST: db
PAPERLESS_URL: https://paperless.${DOMAIN}
PAPERLESS_SECRET_KEY: ${PAPERLESS_SECRET_KEY}
PAPERLESS_TIME_ZONE: America/Los_Angeles
PAPERLESS_OCR_LANGUAGE: eng
USERMAP_UID: 1000
USERMAP_GID: 1000
labels:

View File

@@ -1,7 +1,7 @@
"""Hatch build hook to vendor CDN assets for offline use.
During wheel builds, this hook:
1. Parses base.html to find elements with data-vendor attributes
1. Reads vendor-assets.json to find assets marked for vendoring
2. Downloads each CDN asset to a temporary vendor directory
3. Rewrites base.html to use local /static/vendor/ paths
4. Fetches and bundles license information
@@ -13,6 +13,7 @@ distributed wheel has vendored assets.
from __future__ import annotations
import json
import re
import shutil
import subprocess
@@ -23,22 +24,6 @@ from urllib.request import Request, urlopen
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
# Matches elements with data-vendor attribute: extracts URL and target filename
# Example: <script src="https://..." data-vendor="htmx.js">
# Captures: (1) src/href, (2) URL, (3) attributes between, (4) vendor filename
VENDOR_PATTERN = re.compile(r'(src|href)="(https://[^"]+)"([^>]*?)data-vendor="([^"]+)"')
# License URLs for each package (GitHub raw URLs)
LICENSE_URLS: dict[str, tuple[str, str]] = {
"htmx": ("MIT", "https://raw.githubusercontent.com/bigskysoftware/htmx/master/LICENSE"),
"xterm": ("MIT", "https://raw.githubusercontent.com/xtermjs/xterm.js/master/LICENSE"),
"daisyui": ("MIT", "https://raw.githubusercontent.com/saadeghi/daisyui/master/LICENSE"),
"tailwindcss": (
"MIT",
"https://raw.githubusercontent.com/tailwindlabs/tailwindcss/master/LICENSE",
),
}
def _download(url: str) -> bytes:
"""Download a URL, trying urllib first then curl as fallback."""
@@ -61,7 +46,14 @@ def _download(url: str) -> bytes:
return bytes(result.stdout)
def _generate_licenses_file(temp_dir: Path) -> None:
def _load_vendor_assets(root: Path) -> dict[str, Any]:
"""Load vendor-assets.json from the web module."""
json_path = root / "src" / "compose_farm" / "web" / "vendor-assets.json"
with json_path.open() as f:
return json.load(f)
def _generate_licenses_file(temp_dir: Path, licenses: dict[str, dict[str, str]]) -> None:
"""Download and combine license files into LICENSES.txt."""
lines = [
"# Vendored Dependencies - License Information",
@@ -73,7 +65,9 @@ def _generate_licenses_file(temp_dir: Path) -> None:
"",
]
for pkg_name, (license_type, license_url) in LICENSE_URLS.items():
for pkg_name, license_info in licenses.items():
license_type = license_info["type"]
license_url = license_info["url"]
lines.append(f"## {pkg_name} ({license_type})")
lines.append(f"Source: {license_url}")
lines.append("")
@@ -107,44 +101,57 @@ class VendorAssetsHook(BuildHookInterface): # type: ignore[misc]
if not base_html_path.exists():
return
# Load vendor assets configuration
vendor_config = _load_vendor_assets(Path(self.root))
assets_to_vendor = vendor_config["assets"]
if not assets_to_vendor:
return
# Create temp directory for vendored assets
temp_dir = Path(tempfile.mkdtemp(prefix="compose_farm_vendor_"))
vendor_dir = temp_dir / "vendor"
vendor_dir.mkdir()
# Read and parse base.html
# Read base.html
html_content = base_html_path.read_text()
# Build URL to filename mapping and download assets
url_to_filename: dict[str, str] = {}
# Find all elements with data-vendor attribute and download them
for match in VENDOR_PATTERN.finditer(html_content):
url = match.group(2)
filename = match.group(4)
if url in url_to_filename:
continue
for asset in assets_to_vendor:
url = asset["url"]
filename = asset["filename"]
url_to_filename[url] = filename
filepath = vendor_dir / filename
filepath.parent.mkdir(parents=True, exist_ok=True)
content = _download(url)
(vendor_dir / filename).write_bytes(content)
filepath.write_bytes(content)
if not url_to_filename:
return
# Generate LICENSES.txt from the JSON config
_generate_licenses_file(vendor_dir, vendor_config["licenses"])
# Generate LICENSES.txt
_generate_licenses_file(vendor_dir)
# Rewrite HTML: replace CDN URLs with local paths and remove data-vendor attributes
# Pattern matches: src="URL" ... data-vendor="filename" or href="URL" ... data-vendor="filename"
vendor_pattern = re.compile(r'(src|href)="(https://[^"]+)"([^>]*?)data-vendor="([^"]+)"')
# Rewrite HTML to use local paths (remove data-vendor, update URL)
def replace_vendor_tag(match: re.Match[str]) -> str:
attr = match.group(1) # src or href
url = match.group(2)
between = match.group(3) # attributes between URL and data-vendor
filename = match.group(4)
if url in url_to_filename:
filename = url_to_filename[url]
return f'{attr}="/static/vendor/{filename}"{between}'
return match.group(0)
modified_html = VENDOR_PATTERN.sub(replace_vendor_tag, html_content)
modified_html = vendor_pattern.sub(replace_vendor_tag, html_content)
# Inject vendored mode flag for JavaScript to detect
# Insert right after <head> tag so it's available early
modified_html = modified_html.replace(
"<head>",
"<head>\n <script>window.CF_VENDORED=true;</script>",
1, # Only replace first occurrence
)
# Write modified base.html to temp
templates_dir = temp_dir / "templates"

View File

@@ -9,17 +9,17 @@ default:
install:
uv sync --all-extras --dev
# Run all tests (no coverage for speed)
# Run all tests (parallel)
test:
uv run pytest --no-cov
uv run pytest -n auto
# Run unit tests only (parallel, with coverage)
test-unit:
# Run CLI tests only (parallel, with coverage)
test-cli:
uv run pytest -m "not browser" -n auto
# Run browser tests only (sequential, no coverage)
test-browser:
uv run pytest -m browser --no-cov
# Run web UI tests only (parallel)
test-web:
uv run pytest -m browser -n auto
# Lint, format, and type check
lint:
@@ -45,6 +45,14 @@ doc:
kill-doc:
lsof -ti :9002 | xargs kill -9 2>/dev/null || true
# Record CLI demos (all or specific: just record-cli quickstart)
record-cli *demos:
python docs/demos/cli/record.py {{demos}}
# Record web UI demos (all or specific: just record-web navigation)
record-web *demos:
python docs/demos/web/record.py {{demos}}
# Clean up build artifacts and caches
clean:
rm -rf .pytest_cache .mypy_cache .ruff_cache .coverage htmlcov dist build

View File

@@ -30,7 +30,8 @@ classifiers = [
"Intended Audience :: Developers",
"Intended Audience :: System Administrators",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Operating System :: MacOS",
"Operating System :: POSIX :: Linux",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
@@ -46,6 +47,7 @@ dependencies = [
"asyncssh>=2.14.0",
"pyyaml>=6.0",
"rich>=13.0.0",
"python-dotenv>=1.0.0",
]
[project.optional-dependencies]
@@ -53,6 +55,7 @@ web = [
"fastapi[standard]>=0.109.0",
"jinja2>=3.1.0",
"websockets>=12.0",
"humanize>=4.0.0",
]
[project.urls]

View File

@@ -142,6 +142,9 @@ def load_config_or_exit(config_path: Path | None) -> Config:
except FileNotFoundError as e:
print_error(str(e))
raise typer.Exit(1) from e
except Exception as e:
print_error(f"Invalid config: {e}")
raise typer.Exit(1) from e
def get_stacks(

View File

@@ -3,13 +3,12 @@
from __future__ import annotations
import os
import platform
import shlex
import shutil
import subprocess
from importlib import resources
from pathlib import Path
from typing import Annotated
from typing import TYPE_CHECKING, Annotated
import typer
@@ -17,6 +16,9 @@ from compose_farm.cli.app import app
from compose_farm.console import MSG_CONFIG_NOT_FOUND, console, print_error, print_success
from compose_farm.paths import config_search_paths, default_config_path, find_config_path
if TYPE_CHECKING:
from compose_farm.config import Config
config_app = typer.Typer(
name="config",
help="Manage compose-farm configuration files.",
@@ -43,8 +45,6 @@ def _get_editor() -> str:
"""Get the user's preferred editor ($EDITOR > $VISUAL > platform default)."""
if editor := os.environ.get("EDITOR") or os.environ.get("VISUAL"):
return editor
if platform.system() == "Windows":
return "notepad"
return next((e for e in ("nano", "vim", "vi") if shutil.which(e)), "vi")
@@ -68,6 +68,22 @@ def _get_config_file(path: Path | None) -> Path | None:
return config_path.resolve() if config_path else None
def _load_config_with_path(path: Path | None) -> tuple[Path, Config]:
"""Load config and return both the resolved path and Config object.
Exits with error if config not found or invalid.
"""
from compose_farm.cli.common import load_config_or_exit # noqa: PLC0415
config_file = _get_config_file(path)
if config_file is None:
print_error(MSG_CONFIG_NOT_FOUND)
raise typer.Exit(1)
cfg = load_config_or_exit(config_file)
return config_file, cfg
def _report_missing_config(explicit_path: Path | None = None) -> None:
"""Report that a config file was not found."""
console.print("[yellow]Config file not found.[/yellow]")
@@ -135,7 +151,7 @@ def config_edit(
console.print(f"[dim]Opening {config_file} with {editor}...[/dim]")
try:
editor_cmd = shlex.split(editor, posix=os.name != "nt")
editor_cmd = shlex.split(editor)
except ValueError as e:
print_error("Invalid editor command. Check [bold]$EDITOR[/]/[bold]$VISUAL[/]")
raise typer.Exit(1) from e
@@ -207,23 +223,7 @@ def config_validate(
path: _PathOption = None,
) -> None:
"""Validate the config file syntax and schema."""
config_file = _get_config_file(path)
if config_file is None:
print_error(MSG_CONFIG_NOT_FOUND)
raise typer.Exit(1)
# Lazy import: pydantic adds ~50ms to startup, only load when actually needed
from compose_farm.config import load_config # noqa: PLC0415
try:
cfg = load_config(config_file)
except FileNotFoundError as e:
print_error(str(e))
raise typer.Exit(1) from e
except Exception as e:
print_error(f"Invalid config: {e}")
raise typer.Exit(1) from e
config_file, cfg = _load_config_with_path(path)
print_success(f"Valid config: {config_file}")
console.print(f" Hosts: {len(cfg.hosts)}")
@@ -293,5 +293,129 @@ def config_symlink(
console.print(f" -> {target_path}")
def _detect_domain(cfg: Config) -> str | None:
"""Try to detect DOMAIN from traefik Host() rules in existing stacks.
Uses extract_website_urls from traefik module to get interpolated
URLs, then extracts the domain from the first valid URL.
Skips local domains (.local, localhost, etc.).
"""
from urllib.parse import urlparse # noqa: PLC0415
from compose_farm.traefik import extract_website_urls # noqa: PLC0415
max_stacks_to_check = 10
min_domain_parts = 2
subdomain_parts = 4
skip_tlds = {"local", "localhost", "internal", "lan", "home"}
for stack_name in list(cfg.stacks.keys())[:max_stacks_to_check]:
urls = extract_website_urls(cfg, stack_name)
for url in urls:
host = urlparse(url).netloc
parts = host.split(".")
# Skip local/internal domains
if parts[-1].lower() in skip_tlds:
continue
if len(parts) >= subdomain_parts:
# e.g., "app.lab.nijho.lt" -> "lab.nijho.lt"
return ".".join(parts[-3:])
if len(parts) >= min_domain_parts:
# e.g., "app.example.com" -> "example.com"
return ".".join(parts[-2:])
return None
def _detect_local_host(cfg: Config) -> str | None:
"""Find which config host matches local machine's IPs."""
from compose_farm.executor import is_local # noqa: PLC0415
for name, host in cfg.hosts.items():
if is_local(host):
return name
return None
@config_app.command("init-env")
def config_init_env(
path: _PathOption = None,
output: Annotated[
Path | None,
typer.Option(
"--output", "-o", help="Output .env file path. Defaults to .env in config directory."
),
] = None,
force: _ForceOption = False,
) -> None:
"""Generate a .env file for Docker deployment.
Reads the compose-farm.yaml config and auto-detects settings:
- CF_COMPOSE_DIR from compose_dir
- CF_LOCAL_HOST by detecting which config host matches local IPs
- CF_UID/GID/HOME/USER from current user
- DOMAIN from traefik labels in stacks (if found)
Example::
cf config init-env # Create .env next to config
cf config init-env -o .env # Create .env in current directory
"""
config_file, cfg = _load_config_with_path(path)
# Determine output path
env_path = output.expanduser().resolve() if output else config_file.parent / ".env"
if env_path.exists() and not force:
console.print(f"[yellow].env file already exists:[/] {env_path}")
if not typer.confirm("Overwrite?"):
console.print("[dim]Aborted.[/dim]")
raise typer.Exit(0)
# Auto-detect values
uid = os.getuid()
gid = os.getgid()
home = os.environ.get("HOME", "/root")
user = os.environ.get("USER", "root")
compose_dir = str(cfg.compose_dir)
local_host = _detect_local_host(cfg)
domain = _detect_domain(cfg)
# Generate .env content
lines = [
"# Generated by: cf config init-env",
f"# From config: {config_file}",
"",
"# Domain for Traefik labels",
f"DOMAIN={domain or 'example.com'}",
"",
"# Compose files location",
f"CF_COMPOSE_DIR={compose_dir}",
"",
"# Run as current user (recommended for NFS)",
f"CF_UID={uid}",
f"CF_GID={gid}",
f"CF_HOME={home}",
f"CF_USER={user}",
"",
"# Local hostname for Glances integration",
f"CF_LOCAL_HOST={local_host or '# auto-detect failed - set manually'}",
"",
]
env_path.write_text("\n".join(lines), encoding="utf-8")
print_success(f"Created .env file: {env_path}")
console.print()
console.print("[dim]Detected settings:[/dim]")
console.print(f" DOMAIN: {domain or '[yellow]example.com[/] (edit this)'}")
console.print(f" CF_COMPOSE_DIR: {compose_dir}")
console.print(f" CF_UID/GID: {uid}:{gid}")
console.print(f" CF_LOCAL_HOST: {local_host or '[yellow]not detected[/] (set manually)'}")
console.print()
console.print("[dim]Review and edit as needed:[/dim]")
console.print(f" [cyan]$EDITOR {env_path}[/cyan]")
# Register config subcommand on the shared app
app.add_typer(config_app, name="config", rich_help_panel="Configuration")

View File

@@ -2,11 +2,15 @@
from __future__ import annotations
import shlex
from pathlib import Path
from typing import Annotated
from typing import TYPE_CHECKING, Annotated
import typer
if TYPE_CHECKING:
from compose_farm.config import Config
from compose_farm.cli.app import app
from compose_farm.cli.common import (
AllOption,
@@ -23,9 +27,15 @@ from compose_farm.cli.common import (
validate_host_for_stack,
validate_stacks,
)
from compose_farm.cli.management import _discover_stacks_full
from compose_farm.console import MSG_DRY_RUN, console, print_error, print_success
from compose_farm.executor import run_compose_on_host, run_on_stacks, run_sequential_on_stacks
from compose_farm.operations import stop_orphaned_stacks, up_stacks
from compose_farm.executor import run_compose_on_host, run_on_stacks
from compose_farm.operations import (
build_up_cmd,
stop_orphaned_stacks,
stop_stray_stacks,
up_stacks,
)
from compose_farm.state import (
get_orphaned_stacks,
get_stack_host,
@@ -41,6 +51,14 @@ def up(
all_stacks: AllOption = False,
host: HostOption = None,
service: ServiceOption = None,
pull: Annotated[
bool,
typer.Option("--pull", help="Pull images before starting (--pull always)"),
] = False,
build: Annotated[
bool,
typer.Option("--build", help="Build images before starting"),
] = False,
config: ConfigOption = None,
) -> None:
"""Start stacks (docker compose up -d). Auto-migrates if host changed."""
@@ -50,9 +68,13 @@ def up(
print_error("--service requires exactly one stack")
raise typer.Exit(1)
# For service-level up, use run_on_stacks directly (no migration logic)
results = run_async(run_on_stacks(cfg, stack_list, f"up -d {service}", raw=True))
results = run_async(
run_on_stacks(
cfg, stack_list, build_up_cmd(pull=pull, build=build, service=service), raw=True
)
)
else:
results = run_async(up_stacks(cfg, stack_list, raw=True))
results = run_async(up_stacks(cfg, stack_list, raw=True, pull=pull, build=build))
maybe_regenerate_traefik(cfg, results)
report_results(results)
@@ -140,7 +162,7 @@ def pull(
if service and len(stack_list) != 1:
print_error("--service requires exactly one stack")
raise typer.Exit(1)
cmd = f"pull {service}" if service else "pull"
cmd = f"pull --ignore-buildable {service}" if service else "pull --ignore-buildable"
raw = len(stack_list) == 1
results = run_async(run_on_stacks(cfg, stack_list, cmd, raw=raw))
report_results(results)
@@ -153,19 +175,17 @@ def restart(
service: ServiceOption = None,
config: ConfigOption = None,
) -> None:
"""Restart stacks (down + up). With --service, restarts just that service."""
"""Restart running containers (docker compose restart)."""
stack_list, cfg = get_stacks(stacks or [], all_stacks, config)
if service:
if len(stack_list) != 1:
print_error("--service requires exactly one stack")
raise typer.Exit(1)
# For service-level restart, use docker compose restart (more efficient)
raw = True
results = run_async(run_on_stacks(cfg, stack_list, f"restart {service}", raw=raw))
cmd = f"restart {service}"
else:
raw = len(stack_list) == 1
results = run_async(run_sequential_on_stacks(cfg, stack_list, ["down", "up -d"], raw=raw))
maybe_regenerate_traefik(cfg, results)
cmd = "restart"
raw = len(stack_list) == 1
results = run_async(run_on_stacks(cfg, stack_list, cmd, raw=raw))
report_results(results)
@@ -176,40 +196,27 @@ def update(
service: ServiceOption = None,
config: ConfigOption = None,
) -> None:
"""Update stacks (pull + build + down + up). With --service, updates just that service."""
stack_list, cfg = get_stacks(stacks or [], all_stacks, config)
if service:
if len(stack_list) != 1:
print_error("--service requires exactly one stack")
raise typer.Exit(1)
# For service-level update: pull + build + stop + up (stop instead of down)
raw = True
results = run_async(
run_sequential_on_stacks(
cfg,
stack_list,
[
f"pull --ignore-buildable {service}",
f"build {service}",
f"stop {service}",
f"up -d {service}",
],
raw=raw,
)
)
else:
raw = len(stack_list) == 1
results = run_async(
run_sequential_on_stacks(
cfg, stack_list, ["pull --ignore-buildable", "build", "down", "up -d"], raw=raw
)
)
maybe_regenerate_traefik(cfg, results)
report_results(results)
"""Update stacks (pull + build + up). Shorthand for 'up --pull --build'."""
up(stacks=stacks, all_stacks=all_stacks, service=service, pull=True, build=True, config=config)
def _discover_strays(cfg: Config) -> dict[str, list[str]]:
"""Discover stacks running on unauthorized hosts by scanning all hosts."""
_, strays, duplicates = _discover_stacks_full(cfg)
# Merge duplicates into strays (for single-host stacks on multiple hosts,
# keep correct host and stop others)
for stack, running_hosts in duplicates.items():
configured = cfg.get_hosts(stack)[0]
stray_hosts = [h for h in running_hosts if h != configured]
if stray_hosts:
strays[stack] = stray_hosts
return strays
@app.command(rich_help_panel="Lifecycle")
def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
def apply( # noqa: C901, PLR0912, PLR0915 (multi-phase reconciliation needs these branches)
dry_run: Annotated[
bool,
typer.Option("--dry-run", "-n", help="Show what would change without executing"),
@@ -218,23 +225,29 @@ def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
bool,
typer.Option("--no-orphans", help="Only migrate, don't stop orphaned stacks"),
] = False,
no_strays: Annotated[
bool,
typer.Option("--no-strays", help="Don't stop stray stacks (running on wrong host)"),
] = False,
full: Annotated[
bool,
typer.Option("--full", "-f", help="Also run up on all stacks to apply config changes"),
] = False,
config: ConfigOption = None,
) -> None:
"""Make reality match config (start, migrate, stop as needed).
"""Make reality match config (start, migrate, stop strays/orphans as needed).
This is the "reconcile" command that ensures running stacks match your
config file. It will:
1. Stop orphaned stacks (in state but removed from config)
2. Migrate stacks on wrong host (host in state ≠ host in config)
3. Start missing stacks (in config but not in state)
2. Stop stray stacks (running on unauthorized hosts)
3. Migrate stacks on wrong host (host in state ≠ host in config)
4. Start missing stacks (in config but not in state)
Use --dry-run to preview changes before applying.
Use --no-orphans to only migrate/start without stopping orphaned stacks.
Use --no-orphans to skip stopping orphaned stacks.
Use --no-strays to skip stopping stray stacks.
Use --full to also run 'up' on all stacks (picks up compose/env changes).
"""
cfg = load_config_or_exit(config)
@@ -242,16 +255,28 @@ def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
migrations = get_stacks_needing_migration(cfg)
missing = get_stacks_not_in_state(cfg)
strays: dict[str, list[str]] = {}
if not no_strays:
console.print("[dim]Scanning hosts for stray containers...[/]")
strays = _discover_strays(cfg)
# For --full: refresh all stacks not already being started/migrated
handled = set(migrations) | set(missing)
to_refresh = [stack for stack in cfg.stacks if stack not in handled] if full else []
has_orphans = bool(orphaned) and not no_orphans
has_strays = bool(strays)
has_migrations = bool(migrations)
has_missing = bool(missing)
has_refresh = bool(to_refresh)
if not has_orphans and not has_migrations and not has_missing and not has_refresh:
if (
not has_orphans
and not has_strays
and not has_migrations
and not has_missing
and not has_refresh
):
print_success("Nothing to apply - reality matches config")
return
@@ -260,6 +285,14 @@ def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
console.print(f"[yellow]Orphaned stacks to stop ({len(orphaned)}):[/]")
for svc, hosts in orphaned.items():
console.print(f" [cyan]{svc}[/] on [magenta]{format_host(hosts)}[/]")
if has_strays:
console.print(f"[red]Stray stacks to stop ({len(strays)}):[/]")
for stack, hosts in strays.items():
configured = cfg.get_hosts(stack)
console.print(
f" [cyan]{stack}[/] on [magenta]{', '.join(hosts)}[/] "
f"[dim](should be on {', '.join(configured)})[/]"
)
if has_migrations:
console.print(f"[cyan]Stacks to migrate ({len(migrations)}):[/]")
for stack in migrations:
@@ -288,21 +321,26 @@ def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
console.print("[yellow]Stopping orphaned stacks...[/]")
all_results.extend(run_async(stop_orphaned_stacks(cfg)))
# 2. Migrate stacks on wrong host
# 2. Stop stray stacks (running on unauthorized hosts)
if has_strays:
console.print("[red]Stopping stray stacks...[/]")
all_results.extend(run_async(stop_stray_stacks(cfg, strays)))
# 3. Migrate stacks on wrong host
if has_migrations:
console.print("[cyan]Migrating stacks...[/]")
migrate_results = run_async(up_stacks(cfg, migrations, raw=True))
all_results.extend(migrate_results)
maybe_regenerate_traefik(cfg, migrate_results)
# 3. Start missing stacks (reuse up_stacks which handles state updates)
# 4. Start missing stacks (reuse up_stacks which handles state updates)
if has_missing:
console.print("[green]Starting missing stacks...[/]")
start_results = run_async(up_stacks(cfg, missing, raw=True))
all_results.extend(start_results)
maybe_regenerate_traefik(cfg, start_results)
# 4. Refresh remaining stacks (--full: run up to apply config changes)
# 5. Refresh remaining stacks (--full: run up to apply config changes)
if has_refresh:
console.print("[blue]Refreshing stacks...[/]")
refresh_results = run_async(up_stacks(cfg, to_refresh, raw=True))
@@ -356,10 +394,10 @@ def compose(
else:
target_host = hosts[0]
# Build the full compose command
# Build the full compose command (quote args to preserve spaces)
full_cmd = command
if args:
full_cmd += " " + " ".join(args)
full_cmd += " " + " ".join(shlex.quote(arg) for arg in args)
# Run with raw=True for proper TTY handling (progress bars, interactive)
result = run_async(run_compose_on_host(cfg, resolved_stack, target_host, full_cmd, raw=True))
@@ -369,5 +407,9 @@ def compose(
raise typer.Exit(result.exit_code)
# Alias: cf a = cf apply
app.command("a", hidden=True)(apply)
# Aliases (hidden from help, shown in --help as "Aliases: ...")
app.command("a", hidden=True)(apply) # cf a = cf apply
app.command("r", hidden=True)(restart) # cf r = cf restart
app.command("u", hidden=True)(update) # cf u = cf update
app.command("p", hidden=True)(pull) # cf p = cf pull
app.command("c", hidden=True)(compose) # cf c = cf compose

View File

@@ -37,61 +37,62 @@ from compose_farm.console import (
)
from compose_farm.executor import (
CommandResult,
get_running_stacks_on_host,
is_local,
run_command,
)
from compose_farm.logs import (
DEFAULT_LOG_PATH,
SnapshotEntry,
collect_stack_entries,
collect_stacks_entries_on_host,
isoformat,
load_existing_entries,
merge_entries,
write_toml,
)
from compose_farm.operations import (
build_discovery_results,
check_host_compatibility,
check_stack_requirements,
discover_stack_host,
)
from compose_farm.state import get_orphaned_stacks, load_state, save_state
from compose_farm.traefik import generate_traefik_config, render_traefik_config
# --- Sync helpers ---
def _discover_stacks(cfg: Config, stacks: list[str] | None = None) -> dict[str, str | list[str]]:
"""Discover running stacks with a progress bar."""
stack_list = stacks if stacks is not None else list(cfg.stacks)
results = run_parallel_with_progress(
"Discovering",
stack_list,
lambda s: discover_stack_host(cfg, s),
)
return {svc: host for svc, host in results if host is not None}
def _snapshot_stacks(
cfg: Config,
stacks: list[str],
discovered: dict[str, str | list[str]],
log_path: Path | None,
) -> Path:
"""Capture image digests with a progress bar."""
"""Capture image digests using batched SSH calls (1 per host).
Args:
cfg: Configuration
discovered: Dict mapping stack -> host(s) where it's running
log_path: Optional path to write the log file
Returns:
Path to the written log file.
"""
effective_log_path = log_path or DEFAULT_LOG_PATH
now_dt = datetime.now(UTC)
now_iso = isoformat(now_dt)
async def collect_stack(stack: str) -> tuple[str, list[SnapshotEntry]]:
try:
return stack, await collect_stack_entries(cfg, stack, now=now_dt)
except RuntimeError:
return stack, []
# Group stacks by host for batched SSH calls
stacks_by_host: dict[str, set[str]] = {}
for stack, hosts in discovered.items():
# Use first host for multi-host stacks (they use the same images)
host = hosts[0] if isinstance(hosts, list) else hosts
stacks_by_host.setdefault(host, set()).add(stack)
results = run_parallel_with_progress(
"Capturing",
stacks,
collect_stack,
)
# Collect entries with 1 SSH call per host (with progress bar)
async def collect_on_host(host: str) -> tuple[str, list[SnapshotEntry]]:
entries = await collect_stacks_entries_on_host(cfg, host, stacks_by_host[host], now=now_dt)
return host, entries
results = run_parallel_with_progress("Capturing", list(stacks_by_host.keys()), collect_on_host)
snapshot_entries = [entry for _, entries in results for entry in entries]
if not snapshot_entries:
@@ -147,6 +148,61 @@ def _report_sync_changes(
console.print(f" [red]-[/] [cyan]{stack}[/] (was on [magenta]{host_str}[/])")
def _discover_stacks_full(
cfg: Config,
stacks: list[str] | None = None,
) -> tuple[dict[str, str | list[str]], dict[str, list[str]], dict[str, list[str]]]:
"""Discover running stacks with full host scanning for stray detection.
Queries each host once for all running stacks (with progress bar),
then delegates to build_discovery_results for categorization.
"""
all_hosts = list(cfg.hosts.keys())
# Query each host for running stacks (with progress bar)
async def get_stacks_on_host(host: str) -> tuple[str, set[str]]:
running = await get_running_stacks_on_host(cfg, host)
return host, running
host_results = run_parallel_with_progress("Discovering", all_hosts, get_stacks_on_host)
running_on_host: dict[str, set[str]] = dict(host_results)
return build_discovery_results(cfg, running_on_host, stacks)
def _report_stray_stacks(
strays: dict[str, list[str]],
cfg: Config,
) -> None:
"""Report stacks running on unauthorized hosts."""
if strays:
console.print(f"\n[red]Stray stacks[/] (running on wrong host, {len(strays)}):")
console.print("[dim]Run [bold]cf apply[/bold] to stop them.[/]")
for stack in sorted(strays):
stray_hosts = strays[stack]
configured = cfg.get_hosts(stack)
console.print(
f" [red]![/] [cyan]{stack}[/] on [magenta]{', '.join(stray_hosts)}[/] "
f"[dim](should be on {', '.join(configured)})[/]"
)
def _report_duplicate_stacks(duplicates: dict[str, list[str]], cfg: Config) -> None:
"""Report single-host stacks running on multiple hosts."""
if duplicates:
console.print(
f"\n[yellow]Duplicate stacks[/] (running on multiple hosts, {len(duplicates)}):"
)
console.print("[dim]Run [bold]cf apply[/bold] to stop extras.[/]")
for stack in sorted(duplicates):
hosts = duplicates[stack]
configured = cfg.get_hosts(stack)[0]
console.print(
f" [yellow]![/] [cyan]{stack}[/] on [magenta]{', '.join(hosts)}[/] "
f"[dim](should only be on {configured})[/]"
)
# --- Check helpers ---
@@ -271,6 +327,8 @@ def _report_orphaned_stacks(cfg: Config) -> bool:
def _report_traefik_status(cfg: Config, stacks: list[str]) -> None:
"""Check and report traefik label status."""
from compose_farm.traefik import generate_traefik_config # noqa: PLC0415
try:
_, warnings = generate_traefik_config(cfg, stacks, check_all=True)
except (FileNotFoundError, ValueError):
@@ -390,6 +448,11 @@ def traefik_file(
config: ConfigOption = None,
) -> None:
"""Generate a Traefik file-provider fragment from compose Traefik labels."""
from compose_farm.traefik import ( # noqa: PLC0415
generate_traefik_config,
render_traefik_config,
)
stack_list, cfg = get_stacks(stacks or [], all_stacks, config)
try:
dynamic, warnings = generate_traefik_config(cfg, stack_list)
@@ -440,7 +503,7 @@ def refresh(
current_state = load_state(cfg)
discovered = _discover_stacks(cfg, stack_list)
discovered, strays, duplicates = _discover_stacks_full(cfg, stack_list)
# Calculate changes (only for the stacks we're refreshing)
added = [s for s in discovered if s not in current_state]
@@ -463,6 +526,9 @@ def refresh(
else:
print_success("State is already in sync.")
_report_stray_stacks(strays, cfg)
_report_duplicate_stacks(duplicates, cfg)
if dry_run:
console.print(f"\n{MSG_DRY_RUN}")
return
@@ -475,10 +541,10 @@ def refresh(
save_state(cfg, new_state)
print_success(f"State updated: {len(new_state)} stacks tracked.")
# Capture image digests for running stacks
# Capture image digests for running stacks (1 SSH call per host)
if discovered:
try:
path = _snapshot_stacks(cfg, list(discovered.keys()), log_path)
path = _snapshot_stacks(cfg, discovered, log_path)
print_success(f"Digests written to {path}")
except RuntimeError as exc:
print_warning(str(exc))
@@ -593,3 +659,9 @@ def init_network(
failed = [r for r in results if not r.success]
if failed:
raise typer.Exit(1)
# Aliases (hidden from help)
app.command("rf", hidden=True)(refresh) # cf rf = cf refresh
app.command("ck", hidden=True)(check) # cf ck = cf check
app.command("tf", hidden=True)(traefik_file) # cf tf = cf traefik-file

View File

@@ -201,3 +201,8 @@ def stats(
console.print()
console.print(_build_summary_table(cfg, state, pending))
# Aliases (hidden from help)
app.command("l", hidden=True)(logs) # cf l = cf logs
app.command("s", hidden=True)(stats) # cf s = cf stats

View File

@@ -13,6 +13,7 @@ from pathlib import Path
from typing import TYPE_CHECKING, Any
import yaml
from dotenv import dotenv_values
if TYPE_CHECKING:
from .config import Config
@@ -40,25 +41,37 @@ def _load_env(compose_path: Path) -> dict[str, str]:
Reads from .env file in the same directory as compose file,
then overlays current environment variables.
"""
env: dict[str, str] = {}
env_path = compose_path.parent / ".env"
if env_path.exists():
for line in env_path.read_text().splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#") or "=" not in stripped:
continue
key, value = stripped.split("=", 1)
key = key.strip()
value = value.strip()
if (value.startswith('"') and value.endswith('"')) or (
value.startswith("'") and value.endswith("'")
):
value = value[1:-1]
env[key] = value
env: dict[str, str] = {k: v for k, v in dotenv_values(env_path).items() if v is not None}
env.update({k: v for k, v in os.environ.items() if isinstance(v, str)})
return env
def parse_compose_data(content: str) -> dict[str, Any]:
"""Parse compose YAML content into a dict."""
compose_data = yaml.safe_load(content) or {}
return compose_data if isinstance(compose_data, dict) else {}
def load_compose_data(compose_path: Path) -> dict[str, Any]:
"""Load compose YAML from a file path."""
return parse_compose_data(compose_path.read_text())
def load_compose_data_for_stack(config: Config, stack: str) -> tuple[Path, dict[str, Any]]:
"""Load compose YAML for a stack, returning (path, data)."""
compose_path = config.get_compose_path(stack)
if not compose_path.exists():
return compose_path, {}
return compose_path, load_compose_data(compose_path)
def extract_services(compose_data: dict[str, Any]) -> dict[str, Any]:
"""Extract services mapping from compose data."""
raw_services = compose_data.get("services", {})
return raw_services if isinstance(raw_services, dict) else {}
def _interpolate(value: str, env: dict[str, str]) -> str:
"""Perform ${VAR} and ${VAR:-default} interpolation."""
@@ -185,16 +198,15 @@ def parse_host_volumes(config: Config, stack: str) -> list[str]:
Returns a list of absolute host paths used as volume mounts.
Skips named volumes and resolves relative paths.
"""
compose_path = config.get_compose_path(stack)
compose_path, compose_data = load_compose_data_for_stack(config, stack)
if not compose_path.exists():
return []
env = _load_env(compose_path)
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
raw_services = extract_services(compose_data)
if not raw_services:
return []
env = _load_env(compose_path)
paths: list[str] = []
compose_dir = compose_path.parent
@@ -221,16 +233,15 @@ def parse_devices(config: Config, stack: str) -> list[str]:
Returns a list of host device paths (e.g., /dev/dri, /dev/dri/renderD128).
"""
compose_path = config.get_compose_path(stack)
compose_path, compose_data = load_compose_data_for_stack(config, stack)
if not compose_path.exists():
return []
env = _load_env(compose_path)
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
raw_services = extract_services(compose_data)
if not raw_services:
return []
env = _load_env(compose_path)
devices: list[str] = []
for definition in raw_services.values():
if not isinstance(definition, dict):
@@ -260,18 +271,20 @@ def parse_external_networks(config: Config, stack: str) -> list[str]:
Returns a list of network names marked as external: true.
"""
compose_path = config.get_compose_path(stack)
compose_path, compose_data = load_compose_data_for_stack(config, stack)
if not compose_path.exists():
return []
compose_data = yaml.safe_load(compose_path.read_text()) or {}
networks = compose_data.get("networks", {})
if not isinstance(networks, dict):
return []
external_networks: list[str] = []
for name, definition in networks.items():
for key, definition in networks.items():
if isinstance(definition, dict) and definition.get("external") is True:
# Networks may have a "name" field, which may differ from the key.
# Use it if present, else fall back to the key.
name = str(definition.get("name", key))
external_networks.append(name)
return external_networks
@@ -285,15 +298,14 @@ def load_compose_services(
Returns (services_dict, env_dict, host_address).
"""
compose_path = config.get_compose_path(stack)
compose_path, compose_data = load_compose_data_for_stack(config, stack)
if not compose_path.exists():
message = f"[{stack}] Compose file not found: {compose_path}"
raise FileNotFoundError(message)
env = _load_env(compose_path)
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
raw_services = extract_services(compose_data)
if not raw_services:
return {}, env, config.get_host(stack).address
return raw_services, env, config.get_host(stack).address
@@ -336,3 +348,18 @@ def get_ports_for_service(
if isinstance(ref_def, dict):
return _parse_ports(ref_def.get("ports"), env)
return _parse_ports(definition.get("ports"), env)
def get_container_name(
service_name: str,
service_def: dict[str, Any] | None,
project_name: str,
) -> str:
"""Get the container name for a service.
Uses container_name from compose if set, otherwise defaults to {project}-{service}-1.
This matches Docker Compose's default naming convention.
"""
if isinstance(service_def, dict) and service_def.get("container_name"):
return str(service_def["container_name"])
return f"{project_name}-{service_name}-1"

View File

@@ -4,6 +4,7 @@ from __future__ import annotations
import getpass
from pathlib import Path
from typing import Any
import yaml
from pydantic import BaseModel, Field, model_validator
@@ -14,7 +15,7 @@ from .paths import config_search_paths, find_config_path
COMPOSE_FILENAMES = ("compose.yaml", "compose.yml", "docker-compose.yml", "docker-compose.yaml")
class Host(BaseModel):
class Host(BaseModel, extra="forbid"):
"""SSH host configuration."""
address: str
@@ -22,7 +23,7 @@ class Host(BaseModel):
port: int = 22
class Config(BaseModel):
class Config(BaseModel, extra="forbid"):
"""Main configuration."""
compose_dir: Path = Path("/opt/compose")
@@ -30,6 +31,9 @@ class Config(BaseModel):
stacks: dict[str, str | list[str]] # stack_name -> host_name or list of hosts
traefik_file: Path | None = None # Auto-regenerate traefik config after up/down
traefik_stack: str | None = None # Stack name for Traefik (skip its host in file-provider)
glances_stack: str | None = (
None # Stack name for Glances (enables host resource stats in web UI)
)
config_path: Path = Path() # Set by load_config()
def get_state_path(self) -> Path:
@@ -113,7 +117,7 @@ class Config(BaseModel):
return found
def _parse_hosts(raw_hosts: dict[str, str | dict[str, str | int]]) -> dict[str, Host]:
def _parse_hosts(raw_hosts: dict[str, Any]) -> dict[str, Host]:
"""Parse hosts from config, handling both simple and full forms."""
hosts = {}
for name, value in raw_hosts.items():
@@ -122,11 +126,7 @@ def _parse_hosts(raw_hosts: dict[str, str | dict[str, str | int]]) -> dict[str,
hosts[name] = Host(address=value)
else:
# Full form: hostname: {address: ..., user: ..., port: ...}
hosts[name] = Host(
address=str(value.get("address", "")),
user=str(value["user"]) if "user" in value else getpass.getuser(),
port=int(value["port"]) if "port" in value else 22,
)
hosts[name] = Host(**value)
return hosts

View File

@@ -76,7 +76,7 @@ stacks:
# traefik_file: (optional) Auto-generate Traefik file-provider config
# ------------------------------------------------------------------------------
# When set, compose-farm automatically regenerates this file after
# up/down/restart/update commands. Traefik watches this file for changes.
# up/down/update commands. Traefik watches this file for changes.
#
# traefik_file: /opt/compose/traefik/dynamic.d/compose-farm.yml

View File

@@ -5,6 +5,7 @@ from __future__ import annotations
import asyncio
import socket
import subprocess
import time
from dataclasses import dataclass
from functools import lru_cache
from typing import TYPE_CHECKING, Any
@@ -23,6 +24,38 @@ LOCAL_ADDRESSES = frozenset({"local", "localhost", "127.0.0.1", "::1"})
_DEFAULT_SSH_PORT = 22
class TTLCache:
"""Simple TTL cache for async function results."""
def __init__(self, ttl_seconds: float = 30.0) -> None:
"""Initialize cache with default TTL in seconds."""
# Cache stores: key -> (timestamp, value, item_ttl)
self._cache: dict[str, tuple[float, Any, float]] = {}
self._default_ttl = ttl_seconds
def get(self, key: str) -> Any | None:
"""Get value if exists and not expired."""
if key in self._cache:
timestamp, value, item_ttl = self._cache[key]
if time.monotonic() - timestamp < item_ttl:
return value
del self._cache[key]
return None
def set(self, key: str, value: Any, ttl_seconds: float | None = None) -> None:
"""Set value with current timestamp and optional custom TTL."""
ttl = ttl_seconds if ttl_seconds is not None else self._default_ttl
self._cache[key] = (time.monotonic(), value, ttl)
def clear(self) -> None:
"""Clear all cached values."""
self._cache.clear()
# Cache compose labels per host for 30 seconds
_compose_labels_cache = TTLCache(ttl_seconds=30.0)
def _print_compose_command(
host_name: str,
compose_dir: str,
@@ -158,15 +191,20 @@ def ssh_connect_kwargs(host: Host) -> dict[str, Any]:
"port": host.port,
"username": host.user,
"known_hosts": None,
"gss_auth": False, # Disable GSSAPI - causes multi-second delays
}
# Add SSH agent path (auto-detect forwarded agent if needed)
agent_path = get_ssh_auth_sock()
if agent_path:
kwargs["agent_path"] = agent_path
# Add key file fallback for when SSH agent is unavailable
# Add key file fallback (prioritized over agent if present)
key_path = get_key_path()
agent_path = get_ssh_auth_sock()
if key_path:
# If dedicated key exists, force use of it and ignore agent
# This avoids issues with stale/broken forwarded agents in Docker
kwargs["client_keys"] = [str(key_path)]
elif agent_path:
# Fallback to agent if no dedicated key
kwargs["agent_path"] = agent_path
return kwargs
@@ -497,6 +535,72 @@ async def check_stack_running(
return result.success and bool(result.stdout.strip())
async def get_running_stacks_on_host(
config: Config,
host_name: str,
) -> set[str]:
"""Get all running compose stacks on a host in a single SSH call.
Uses docker ps with the compose.project label to identify running stacks.
Much more efficient than checking each stack individually.
"""
host = config.hosts[host_name]
# Get unique project names from running containers
command = "docker ps --format '{{.Label \"com.docker.compose.project\"}}' | sort -u"
result = await run_command(host, command, stack=host_name, stream=False, prefix="")
if not result.success:
return set()
# Filter out empty lines and return as set
return {line.strip() for line in result.stdout.splitlines() if line.strip()}
async def get_container_compose_labels(
config: Config,
host_name: str,
) -> dict[str, tuple[str, str]]:
"""Get compose labels for all containers on a host.
Returns dict of container_name -> (project, service).
Includes all containers (-a flag) since Glances shows stopped containers too.
Falls back to empty dict on timeout/error (5s timeout).
Results are cached for 30 seconds to reduce SSH overhead.
"""
# Check cache first
cached: dict[str, tuple[str, str]] | None = _compose_labels_cache.get(host_name)
if cached is not None:
return cached
host = config.hosts[host_name]
cmd = (
"docker ps -a --format "
'\'{{.Names}}\t{{.Label "com.docker.compose.project"}}\t'
'{{.Label "com.docker.compose.service"}}\''
)
try:
async with asyncio.timeout(5.0):
result = await run_command(host, cmd, stack=host_name, stream=False, prefix="")
except TimeoutError:
return {}
except Exception:
return {}
labels: dict[str, tuple[str, str]] = {}
if result.success:
for line in result.stdout.splitlines():
parts = line.strip().split("\t")
if len(parts) >= 3: # noqa: PLR2004
name, project, service = parts[0], parts[1], parts[2]
labels[name] = (project or "", service or "")
# Cache the result
_compose_labels_cache.set(host_name, labels)
return labels
async def _batch_check_existence(
config: Config,
host_name: str,

277
src/compose_farm/glances.py Normal file
View File

@@ -0,0 +1,277 @@
"""Glances API client for host resource monitoring."""
from __future__ import annotations
import asyncio
import os
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
from .executor import is_local
if TYPE_CHECKING:
from .config import Config, Host
# Default Glances REST API port
DEFAULT_GLANCES_PORT = 61208
def _get_glances_address(
host_name: str,
host: Host,
glances_container: str | None,
) -> str:
"""Get the address to use for Glances API requests.
When running in a Docker container (CF_WEB_STACK set), the local host's Glances
may not be reachable via its LAN IP due to Docker network isolation. In this case,
we use the Glances container name for the local host.
Set CF_LOCAL_HOST=<hostname> to explicitly specify which host is local.
"""
# Only use container name when running inside a Docker container
in_container = os.environ.get("CF_WEB_STACK") is not None
if not in_container or not glances_container:
return host.address
# CF_LOCAL_HOST explicitly tells us which host to reach via container name
explicit_local = os.environ.get("CF_LOCAL_HOST")
if explicit_local and host_name == explicit_local:
return glances_container
# Fall back to is_local detection (may not work in container)
if is_local(host):
return glances_container
return host.address
@dataclass
class HostStats:
"""Resource statistics for a host."""
host: str
cpu_percent: float
mem_percent: float
swap_percent: float
load: float
disk_percent: float
net_rx_rate: float = 0.0 # bytes/sec
net_tx_rate: float = 0.0 # bytes/sec
error: str | None = None
@classmethod
def from_error(cls, host: str, error: str) -> HostStats:
"""Create a HostStats with an error."""
return cls(
host=host,
cpu_percent=0,
mem_percent=0,
swap_percent=0,
load=0,
disk_percent=0,
net_rx_rate=0,
net_tx_rate=0,
error=error,
)
async def fetch_host_stats(
host_name: str,
host_address: str,
port: int = DEFAULT_GLANCES_PORT,
request_timeout: float = 10.0,
) -> HostStats:
"""Fetch stats from a single host's Glances API."""
import httpx # noqa: PLC0415
base_url = f"http://{host_address}:{port}/api/4"
try:
async with httpx.AsyncClient(timeout=request_timeout) as client:
# Fetch quicklook stats (CPU, mem, load)
response = await client.get(f"{base_url}/quicklook")
if not response.is_success:
return HostStats.from_error(host_name, f"HTTP {response.status_code}")
data = response.json()
# Fetch filesystem stats for disk usage (root fs or max across all)
disk_percent = 0.0
try:
fs_response = await client.get(f"{base_url}/fs")
if fs_response.is_success:
fs_data = fs_response.json()
root = next((fs for fs in fs_data if fs.get("mnt_point") == "/"), None)
disk_percent = (
root.get("percent", 0)
if root
else max((fs.get("percent", 0) for fs in fs_data), default=0)
)
except httpx.HTTPError:
pass # Disk stats are optional
# Fetch network stats for rate (sum across non-loopback interfaces)
net_rx_rate, net_tx_rate = 0.0, 0.0
try:
net_response = await client.get(f"{base_url}/network")
if net_response.is_success:
for iface in net_response.json():
if not iface.get("interface_name", "").startswith("lo"):
net_rx_rate += iface.get("bytes_recv_rate_per_sec") or 0
net_tx_rate += iface.get("bytes_sent_rate_per_sec") or 0
except httpx.HTTPError:
pass # Network stats are optional
return HostStats(
host=host_name,
cpu_percent=data.get("cpu", 0),
mem_percent=data.get("mem", 0),
swap_percent=data.get("swap", 0),
load=data.get("load", 0),
disk_percent=disk_percent,
net_rx_rate=net_rx_rate,
net_tx_rate=net_tx_rate,
)
except httpx.TimeoutException:
return HostStats.from_error(host_name, "timeout")
except httpx.HTTPError as e:
return HostStats.from_error(host_name, str(e))
except Exception as e:
return HostStats.from_error(host_name, str(e))
async def fetch_all_host_stats(
config: Config,
port: int = DEFAULT_GLANCES_PORT,
) -> dict[str, HostStats]:
"""Fetch stats from all hosts in parallel."""
glances_container = config.glances_stack
tasks = [
fetch_host_stats(name, _get_glances_address(name, host, glances_container), port)
for name, host in config.hosts.items()
]
results = await asyncio.gather(*tasks)
return {stats.host: stats for stats in results}
@dataclass
class ContainerStats:
"""Container statistics from Glances."""
name: str
host: str
status: str
image: str
cpu_percent: float
memory_usage: int # bytes
memory_limit: int # bytes
memory_percent: float
network_rx: int # cumulative bytes received
network_tx: int # cumulative bytes sent
uptime: str
ports: str
engine: str # docker, podman, etc.
stack: str = "" # compose project name (from docker labels)
service: str = "" # compose service name (from docker labels)
def _parse_container(data: dict[str, Any], host_name: str) -> ContainerStats:
"""Parse container data from Glances API response."""
# Image can be a list or string
image = data.get("image", ["unknown"])
if isinstance(image, list):
image = image[0] if image else "unknown"
# Calculate memory percent
mem_usage = data.get("memory_usage", 0) or 0
mem_limit = data.get("memory_limit", 1) or 1 # Avoid division by zero
mem_percent = (mem_usage / mem_limit) * 100 if mem_limit > 0 else 0
# Network stats
network = data.get("network", {}) or {}
network_rx = network.get("cumulative_rx", 0) or 0
network_tx = network.get("cumulative_tx", 0) or 0
return ContainerStats(
name=data.get("name", "unknown"),
host=host_name,
status=data.get("status", "unknown"),
image=image,
cpu_percent=data.get("cpu_percent", 0) or 0,
memory_usage=mem_usage,
memory_limit=mem_limit,
memory_percent=mem_percent,
network_rx=network_rx,
network_tx=network_tx,
uptime=data.get("uptime", ""),
ports=data.get("ports", "") or "",
engine=data.get("engine", "docker"),
)
async def fetch_container_stats(
host_name: str,
host_address: str,
port: int = DEFAULT_GLANCES_PORT,
request_timeout: float = 10.0,
) -> tuple[list[ContainerStats] | None, str | None]:
"""Fetch container stats from a single host's Glances API.
Returns:
(containers, error_message)
- Success: ([...], None)
- Failure: (None, "error message")
"""
import httpx # noqa: PLC0415
url = f"http://{host_address}:{port}/api/4/containers"
try:
async with httpx.AsyncClient(timeout=request_timeout) as client:
response = await client.get(url)
if not response.is_success:
return None, f"HTTP {response.status_code}: {response.reason_phrase}"
data = response.json()
return [_parse_container(c, host_name) for c in data], None
except httpx.ConnectError:
return None, "Connection refused (Glances offline?)"
except httpx.TimeoutException:
return None, "Connection timed out"
except Exception as e:
return None, str(e)
async def fetch_all_container_stats(
config: Config,
port: int = DEFAULT_GLANCES_PORT,
) -> list[ContainerStats]:
"""Fetch container stats from all hosts in parallel, enriched with compose labels."""
from .executor import get_container_compose_labels # noqa: PLC0415
glances_container = config.glances_stack
async def fetch_host_data(
host_name: str,
host_address: str,
) -> list[ContainerStats]:
# Fetch Glances stats and compose labels in parallel
stats_task = fetch_container_stats(host_name, host_address, port)
labels_task = get_container_compose_labels(config, host_name)
(containers, _), labels = await asyncio.gather(stats_task, labels_task)
if containers is None:
# Skip failed hosts in aggregate view
return []
# Enrich containers with compose labels (mutate in place)
for c in containers:
c.stack, c.service = labels.get(c.name, ("", ""))
return containers
tasks = [
fetch_host_data(name, _get_glances_address(name, host, glances_container))
for name, host in config.hosts.items()
]
results = await asyncio.gather(*tasks)
# Flatten list of lists
return [container for host_containers in results for container in host_containers]

View File

@@ -6,21 +6,22 @@ import json
import tomllib
from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING
from .executor import run_compose
from .executor import run_command
from .paths import xdg_config_home
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Iterable
from collections.abc import Iterable
from pathlib import Path
from .config import Config
from .executor import CommandResult
# Separator used to split output sections
_SECTION_SEPARATOR = "---CF-SEP---"
DEFAULT_LOG_PATH = xdg_config_home() / "compose-farm" / "dockerfarm-log.toml"
_DIGEST_HEX_LENGTH = 64
@dataclass(frozen=True)
@@ -56,87 +57,97 @@ def _escape(value: str) -> str:
return value.replace("\\", "\\\\").replace('"', '\\"')
def _parse_images_output(raw: str) -> list[dict[str, Any]]:
"""Parse `docker compose images --format json` output.
Handles both a JSON array and newline-separated JSON objects for robustness.
"""
raw = raw.strip()
if not raw:
return []
def _parse_image_digests(image_json: str) -> dict[str, str]:
"""Parse docker image inspect JSON to build image tag -> digest map."""
if not image_json:
return {}
try:
parsed = json.loads(raw)
image_data = json.loads(image_json)
except json.JSONDecodeError:
objects = []
for line in raw.splitlines():
if not line.strip():
continue
objects.append(json.loads(line))
return objects
return {}
if isinstance(parsed, list):
return parsed
if isinstance(parsed, dict):
return [parsed]
return []
image_digests: dict[str, str] = {}
for img in image_data:
tags = img.get("RepoTags") or []
digests = img.get("RepoDigests") or []
digest = digests[0].split("@")[-1] if digests else img.get("Id", "")
for tag in tags:
image_digests[tag] = digest
if img.get("Id"):
image_digests[img["Id"]] = digest
return image_digests
def _extract_image_fields(record: dict[str, Any]) -> tuple[str, str]:
"""Extract image name and digest with fallbacks."""
image = record.get("Image") or record.get("Repository") or record.get("Name") or ""
tag = record.get("Tag") or record.get("Version")
if tag and ":" not in image.rsplit("/", 1)[-1]:
image = f"{image}:{tag}"
digest = (
record.get("Digest")
or record.get("Image ID")
or record.get("ImageID")
or record.get("ID")
or ""
)
if digest and not digest.startswith("sha256:") and len(digest) == _DIGEST_HEX_LENGTH:
digest = f"sha256:{digest}"
return image, digest
async def collect_stack_entries(
async def collect_stacks_entries_on_host(
config: Config,
stack: str,
host_name: str,
stacks: set[str],
*,
now: datetime,
run_compose_fn: Callable[..., Awaitable[CommandResult]] = run_compose,
) -> list[SnapshotEntry]:
"""Run `docker compose images` for a stack and normalize results."""
result = await run_compose_fn(config, stack, "images --format json", stream=False)
"""Collect image entries for stacks on one host using 2 docker commands.
Uses `docker ps` to get running containers + their compose project labels,
then `docker image inspect` to get digests for all unique images.
Much faster than running N `docker compose images` commands.
"""
if not stacks:
return []
host = config.hosts[host_name]
# Single SSH call with 2 docker commands:
# 1. Get project|image pairs from running containers
# 2. Get image info (including digests) for all unique images
command = (
f"docker ps --format '{{{{.Label \"com.docker.compose.project\"}}}}|{{{{.Image}}}}' && "
f"echo '{_SECTION_SEPARATOR}' && "
"docker image inspect $(docker ps --format '{{.Image}}' | sort -u) 2>/dev/null || true"
)
result = await run_command(host, command, host_name, stream=False, prefix="")
if not result.success:
msg = result.stderr or f"compose images exited with {result.exit_code}"
error = f"[{stack}] Unable to read images: {msg}"
raise RuntimeError(error)
return []
records = _parse_images_output(result.stdout)
# Use first host for snapshots (multi-host stacks use same images on all hosts)
host_name = config.get_hosts(stack)[0]
compose_path = config.get_compose_path(stack)
# Split output into two sections
parts = result.stdout.split(_SECTION_SEPARATOR)
if len(parts) != 2: # noqa: PLR2004
return []
entries: list[SnapshotEntry] = []
for record in records:
image, digest = _extract_image_fields(record)
if not digest:
container_lines, image_json = parts[0].strip(), parts[1].strip()
# Parse project|image pairs, filtering to only stacks we care about
stack_images: dict[str, set[str]] = {}
for line in container_lines.splitlines():
if "|" not in line:
continue
entries.append(
SnapshotEntry(
stack=stack,
host=host_name,
compose_file=compose_path,
image=image,
digest=digest,
captured_at=now,
)
)
project, image = line.split("|", 1)
if project in stacks:
stack_images.setdefault(project, set()).add(image)
if not stack_images:
return []
# Parse image inspect JSON to build image -> digest map
image_digests = _parse_image_digests(image_json)
# Build entries
entries: list[SnapshotEntry] = []
for stack, images in stack_images.items():
for image in images:
digest = image_digests.get(image, "")
if digest:
entries.append(
SnapshotEntry(
stack=stack,
host=host_name,
compose_file=config.get_compose_path(stack),
image=image,
digest=digest,
captured_at=now,
)
)
return entries

View File

@@ -76,29 +76,37 @@ def get_stack_paths(cfg: Config, stack: str) -> list[str]:
return paths
async def discover_stack_host(cfg: Config, stack: str) -> tuple[str, str | list[str] | None]:
"""Discover where a stack is running.
class StackDiscoveryResult(NamedTuple):
"""Result of discovering where a stack is running across all hosts."""
For multi-host stacks, checks all assigned hosts in parallel.
For single-host, checks assigned host first, then others.
stack: str
configured_hosts: list[str] # From config (where it SHOULD run)
running_hosts: list[str] # From reality (where it IS running)
Returns (stack_name, host_or_hosts_or_none).
"""
assigned_hosts = cfg.get_hosts(stack)
@property
def is_multi_host(self) -> bool:
"""Check if this is a multi-host stack."""
return len(self.configured_hosts) > 1
if cfg.is_multi_host(stack):
# Check all assigned hosts in parallel
checks = await asyncio.gather(*[check_stack_running(cfg, stack, h) for h in assigned_hosts])
running = [h for h, is_running in zip(assigned_hosts, checks, strict=True) if is_running]
return stack, running if running else None
@property
def stray_hosts(self) -> list[str]:
"""Hosts where stack is running but shouldn't be."""
return [h for h in self.running_hosts if h not in self.configured_hosts]
# Single-host: check assigned host first, then others
if await check_stack_running(cfg, stack, assigned_hosts[0]):
return stack, assigned_hosts[0]
for host in cfg.hosts:
if host != assigned_hosts[0] and await check_stack_running(cfg, stack, host):
return stack, host
return stack, None
@property
def missing_hosts(self) -> list[str]:
"""Hosts where stack should be running but isn't."""
return [h for h in self.configured_hosts if h not in self.running_hosts]
@property
def is_stray(self) -> bool:
"""Stack is running on unauthorized host(s)."""
return len(self.stray_hosts) > 0
@property
def is_duplicate(self) -> bool:
"""Single-host stack running on multiple hosts."""
return not self.is_multi_host and len(self.running_hosts) > 1
async def check_stack_requirements(
@@ -177,18 +185,37 @@ def _report_preflight_failures(
print_error(f" missing device: {dev}")
def build_up_cmd(
*,
pull: bool = False,
build: bool = False,
service: str | None = None,
) -> str:
"""Build compose 'up' subcommand with optional flags."""
parts = ["up", "-d"]
if pull:
parts.append("--pull always")
if build:
parts.append("--build")
if service:
parts.append(service)
return " ".join(parts)
async def _up_multi_host_stack(
cfg: Config,
stack: str,
prefix: str,
*,
raw: bool = False,
pull: bool = False,
build: bool = False,
) -> list[CommandResult]:
"""Start a multi-host stack on all configured hosts."""
host_names = cfg.get_hosts(stack)
results: list[CommandResult] = []
compose_path = cfg.get_compose_path(stack)
command = f"docker compose -f {compose_path} up -d"
command = f"docker compose -f {compose_path} {build_up_cmd(pull=pull, build=build)}"
# Pre-flight checks on all hosts
for host_name in host_names:
@@ -261,6 +288,8 @@ async def _up_single_stack(
prefix: str,
*,
raw: bool,
pull: bool = False,
build: bool = False,
) -> CommandResult:
"""Start a single-host stack with migration support."""
target_host = cfg.get_hosts(stack)[0]
@@ -289,7 +318,7 @@ async def _up_single_stack(
# Start on target host
console.print(f"{prefix} Starting on [magenta]{target_host}[/]...")
up_result = await _run_compose_step(cfg, stack, "up -d", raw=raw)
up_result = await _run_compose_step(cfg, stack, build_up_cmd(pull=pull, build=build), raw=raw)
# Update state on success, or rollback on failure
if up_result.success:
@@ -308,24 +337,101 @@ async def _up_single_stack(
return up_result
async def _up_stack_simple(
cfg: Config,
stack: str,
*,
raw: bool = False,
pull: bool = False,
build: bool = False,
) -> CommandResult:
"""Start a single-host stack without migration (parallel-safe)."""
target_host = cfg.get_hosts(stack)[0]
# Pre-flight check
preflight = await check_stack_requirements(cfg, stack, target_host)
if not preflight.ok:
_report_preflight_failures(stack, target_host, preflight)
return CommandResult(stack=stack, exit_code=1, success=False)
# Run with streaming for parallel output
result = await run_compose(cfg, stack, build_up_cmd(pull=pull, build=build), raw=raw)
if raw:
print()
if result.interrupted:
raise OperationInterruptedError
# Update state on success
if result.success:
set_stack_host(cfg, stack, target_host)
return result
async def up_stacks(
cfg: Config,
stacks: list[str],
*,
raw: bool = False,
pull: bool = False,
build: bool = False,
) -> list[CommandResult]:
"""Start stacks with automatic migration if host changed."""
"""Start stacks with automatic migration if host changed.
Stacks without migration run in parallel. Migration stacks run sequentially.
"""
# Categorize stacks
multi_host: list[str] = []
needs_migration: list[str] = []
simple: list[str] = []
for stack in stacks:
if cfg.is_multi_host(stack):
multi_host.append(stack)
else:
target = cfg.get_hosts(stack)[0]
current = get_stack_host(cfg, stack)
if current and current != target:
needs_migration.append(stack)
else:
simple.append(stack)
results: list[CommandResult] = []
total = len(stacks)
try:
for idx, stack in enumerate(stacks, 1):
prefix = f"[dim][{idx}/{total}][/] [cyan]\\[{stack}][/]"
# Simple stacks: run in parallel (no migration needed)
if simple:
use_raw = raw and len(simple) == 1
simple_results = await asyncio.gather(
*[
_up_stack_simple(cfg, stack, raw=use_raw, pull=pull, build=build)
for stack in simple
]
)
results.extend(simple_results)
# Multi-host stacks: run in parallel
if multi_host:
multi_results = await asyncio.gather(
*[
_up_multi_host_stack(
cfg, stack, f"[cyan]\\[{stack}][/]", raw=raw, pull=pull, build=build
)
for stack in multi_host
]
)
for result_list in multi_results:
results.extend(result_list)
# Migration stacks: run sequentially for clear output and rollback
if needs_migration:
total = len(needs_migration)
for idx, stack in enumerate(needs_migration, 1):
prefix = f"[dim][{idx}/{total}][/] [cyan]\\[{stack}][/]"
results.append(
await _up_single_stack(cfg, stack, prefix, raw=raw, pull=pull, build=build)
)
if cfg.is_multi_host(stack):
results.extend(await _up_multi_host_stack(cfg, stack, prefix, raw=raw))
else:
results.append(await _up_single_stack(cfg, stack, prefix, raw=raw))
except OperationInterruptedError:
raise KeyboardInterrupt from None
@@ -359,26 +465,33 @@ async def check_host_compatibility(
return results
async def stop_orphaned_stacks(cfg: Config) -> list[CommandResult]:
"""Stop orphaned stacks (in state but not in config).
async def _stop_stacks_on_hosts(
cfg: Config,
stacks_to_hosts: dict[str, list[str]],
label: str = "",
) -> list[CommandResult]:
"""Stop stacks on specific hosts.
Runs docker compose down on each stack on its tracked host(s).
Only removes from state on successful stop.
Shared helper for stop_orphaned_stacks and stop_stray_stacks.
Args:
cfg: Config object.
stacks_to_hosts: Dict mapping stack name to list of hosts to stop on.
label: Optional label for success message (e.g., "stray", "orphaned").
Returns:
List of CommandResults for each stack@host.
Returns list of CommandResults for each stack@host.
"""
orphaned = get_orphaned_stacks(cfg)
if not orphaned:
if not stacks_to_hosts:
return []
results: list[CommandResult] = []
tasks: list[tuple[str, str, asyncio.Task[CommandResult]]] = []
suffix = f" ({label})" if label else ""
# Build list of (stack, host, task) for all orphaned stacks
for stack, hosts in orphaned.items():
host_list = hosts if isinstance(hosts, list) else [hosts]
for host in host_list:
# Skip hosts no longer in config
for stack, hosts in stacks_to_hosts.items():
for host in hosts:
if host not in cfg.hosts:
print_warning(f"{stack}@{host}: host no longer in config, skipping")
results.append(
@@ -393,30 +506,48 @@ async def stop_orphaned_stacks(cfg: Config) -> list[CommandResult]:
coro = run_compose_on_host(cfg, stack, host, "down")
tasks.append((stack, host, asyncio.create_task(coro)))
# Run all down commands in parallel
if tasks:
for stack, host, task in tasks:
try:
result = await task
results.append(result)
if result.success:
print_success(f"{stack}@{host}: stopped")
else:
print_error(f"{stack}@{host}: {result.stderr or 'failed'}")
except Exception as e:
print_error(f"{stack}@{host}: {e}")
results.append(
CommandResult(
stack=f"{stack}@{host}",
exit_code=1,
success=False,
stderr=str(e),
)
for stack, host, task in tasks:
try:
result = await task
results.append(result)
if result.success:
print_success(f"{stack}@{host}: stopped{suffix}")
else:
print_error(f"{stack}@{host}: {result.stderr or 'failed'}")
except Exception as e:
print_error(f"{stack}@{host}: {e}")
results.append(
CommandResult(
stack=f"{stack}@{host}",
exit_code=1,
success=False,
stderr=str(e),
)
)
return results
async def stop_orphaned_stacks(cfg: Config) -> list[CommandResult]:
"""Stop orphaned stacks (in state but not in config).
Runs docker compose down on each stack on its tracked host(s).
Only removes from state on successful stop.
Returns list of CommandResults for each stack@host.
"""
orphaned = get_orphaned_stacks(cfg)
if not orphaned:
return []
normalized: dict[str, list[str]] = {
stack: (hosts if isinstance(hosts, list) else [hosts]) for stack, hosts in orphaned.items()
}
results = await _stop_stacks_on_hosts(cfg, normalized)
# Remove from state only for stacks where ALL hosts succeeded
for stack, hosts in orphaned.items():
host_list = hosts if isinstance(hosts, list) else [hosts]
for stack in normalized:
all_succeeded = all(
r.success for r in results if r.stack.startswith(f"{stack}@") or r.stack == stack
)
@@ -424,3 +555,77 @@ async def stop_orphaned_stacks(cfg: Config) -> list[CommandResult]:
remove_stack(cfg, stack)
return results
async def stop_stray_stacks(
cfg: Config,
strays: dict[str, list[str]],
) -> list[CommandResult]:
"""Stop stacks running on unauthorized hosts.
Args:
cfg: Config object.
strays: Dict mapping stack name to list of stray hosts.
Returns:
List of CommandResults for each stack@host stopped.
"""
return await _stop_stacks_on_hosts(cfg, strays, label="stray")
def build_discovery_results(
cfg: Config,
running_on_host: dict[str, set[str]],
stacks: list[str] | None = None,
) -> tuple[dict[str, str | list[str]], dict[str, list[str]], dict[str, list[str]]]:
"""Build discovery results from per-host running stacks.
Takes the raw data of which stacks are running on which hosts and
categorizes them into discovered (running correctly), strays (wrong host),
and duplicates (single-host stack on multiple hosts).
Args:
cfg: Config object.
running_on_host: Dict mapping host -> set of running stack names.
stacks: Optional list of stacks to check. Defaults to all configured stacks.
Returns:
Tuple of (discovered, strays, duplicates):
- discovered: stack -> host(s) where running correctly
- strays: stack -> list of unauthorized hosts
- duplicates: stack -> list of all hosts (for single-host stacks on multiple)
"""
stack_list = stacks if stacks is not None else list(cfg.stacks)
all_hosts = list(running_on_host.keys())
# Build StackDiscoveryResult for each stack
results: list[StackDiscoveryResult] = [
StackDiscoveryResult(
stack=stack,
configured_hosts=cfg.get_hosts(stack),
running_hosts=[h for h in all_hosts if stack in running_on_host[h]],
)
for stack in stack_list
]
discovered: dict[str, str | list[str]] = {}
strays: dict[str, list[str]] = {}
duplicates: dict[str, list[str]] = {}
for result in results:
correct_hosts = [h for h in result.running_hosts if h in result.configured_hosts]
if correct_hosts:
if result.is_multi_host:
discovered[result.stack] = correct_hosts
else:
discovered[result.stack] = correct_hosts[0]
if result.is_stray:
strays[result.stack] = result.stray_hosts
if result.is_duplicate:
duplicates[result.stack] = result.running_hosts
return discovered, strays, duplicates

View File

@@ -11,9 +11,19 @@ def xdg_config_home() -> Path:
return Path(os.environ.get("XDG_CONFIG_HOME", Path.home() / ".config"))
def config_dir() -> Path:
"""Get the compose-farm config directory."""
return xdg_config_home() / "compose-farm"
def default_config_path() -> Path:
"""Get the default user config path."""
return xdg_config_home() / "compose-farm" / "compose-farm.yaml"
return config_dir() / "compose-farm.yaml"
def backup_dir() -> Path:
"""Get the backup directory for file edits."""
return config_dir() / "backups"
def config_search_paths() -> list[Path]:

View File

@@ -0,0 +1,220 @@
"""Container registry API client for tag discovery."""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import httpx
# Image reference pattern: [registry/][namespace/]name[:tag][@digest]
IMAGE_PATTERN = re.compile(
r"^(?:(?P<registry>[^/]+\.[^/]+)/)?(?:(?P<namespace>[^/:@]+)/)?(?P<name>[^/:@]+)(?::(?P<tag>[^@]+))?(?:@(?P<digest>.+))?$"
)
# Docker Hub aliases
DOCKER_HUB_ALIASES = frozenset(
{"docker.io", "index.docker.io", "registry.hub.docker.com", "registry-1.docker.io"}
)
# Token endpoints per registry: (url, extra_params)
TOKEN_ENDPOINTS: dict[str, tuple[str, dict[str, str]]] = {
"docker.io": ("https://auth.docker.io/token", {"service": "registry.docker.io"}),
"ghcr.io": ("https://ghcr.io/token", {}),
}
# Registry URL overrides (Docker Hub uses a different host for API)
REGISTRY_URLS: dict[str, str] = {
"docker.io": "https://registry-1.docker.io",
}
HTTP_OK = 200
MANIFEST_ACCEPT = (
"application/vnd.docker.distribution.manifest.v2+json, "
"application/vnd.oci.image.manifest.v1+json, "
"application/vnd.oci.image.index.v1+json"
)
@dataclass(frozen=True)
class ImageRef:
"""Parsed container image reference."""
registry: str
namespace: str
name: str
tag: str
digest: str | None = None
@property
def full_name(self) -> str:
"""Full image name with namespace."""
return f"{self.namespace}/{self.name}" if self.namespace else self.name
@property
def display_name(self) -> str:
"""Display name (omits docker.io/library for official images)."""
if self.registry in DOCKER_HUB_ALIASES:
if self.namespace == "library":
return self.name
return self.full_name
return f"{self.registry}/{self.full_name}"
@classmethod
def parse(cls, image: str) -> ImageRef:
"""Parse image string into components."""
match = IMAGE_PATTERN.match(image)
if not match:
return cls("docker.io", "library", image.split(":")[0].split("@")[0], "latest")
groups = match.groupdict()
registry = groups.get("registry") or "docker.io"
namespace = groups.get("namespace") or ""
name = groups.get("name") or image
tag = groups.get("tag") or "latest"
digest = groups.get("digest")
# Docker Hub official images have implicit "library" namespace
if registry in DOCKER_HUB_ALIASES and not namespace:
namespace = "library"
return cls(registry, namespace, name, tag, digest)
@dataclass
class TagCheckResult:
"""Result of checking tags for an image."""
image: ImageRef
current_digest: str
available_updates: list[str] = field(default_factory=list)
error: str | None = None
class RegistryClient:
"""Unified OCI Distribution API client."""
def __init__(self, registry: str) -> None:
"""Initialize for a specific registry."""
self.registry = registry.lower()
# Normalize Docker Hub aliases
if self.registry in DOCKER_HUB_ALIASES:
self.registry = "docker.io"
self.registry_url = REGISTRY_URLS.get(self.registry, f"https://{self.registry}")
self._token_cache: dict[str, str] = {}
async def _get_token(self, image: ImageRef, client: httpx.AsyncClient) -> str | None:
"""Get auth token for the registry (cached per image)."""
cache_key = image.full_name
if cache_key in self._token_cache:
return self._token_cache[cache_key]
endpoint = TOKEN_ENDPOINTS.get(self.registry)
if not endpoint:
return None # No auth needed or unknown registry
url, extra_params = endpoint
params = {"scope": f"repository:{image.full_name}:pull", **extra_params}
resp = await client.get(url, params=params)
if resp.status_code == HTTP_OK:
token: str | None = resp.json().get("token")
if token:
self._token_cache[cache_key] = token
return token
return None
async def get_tags(self, image: ImageRef, client: httpx.AsyncClient) -> list[str]:
"""Fetch available tags for an image."""
headers = {}
token = await self._get_token(image, client)
if token:
headers["Authorization"] = f"Bearer {token}"
url = f"{self.registry_url}/v2/{image.full_name}/tags/list"
resp = await client.get(url, headers=headers)
if resp.status_code != HTTP_OK:
return []
tags: list[str] = resp.json().get("tags", [])
return tags
async def get_digest(self, image: ImageRef, tag: str, client: httpx.AsyncClient) -> str | None:
"""Get digest for a specific tag."""
headers = {"Accept": MANIFEST_ACCEPT}
token = await self._get_token(image, client)
if token:
headers["Authorization"] = f"Bearer {token}"
url = f"{self.registry_url}/v2/{image.full_name}/manifests/{tag}"
resp = await client.head(url, headers=headers)
if resp.status_code == HTTP_OK:
digest: str | None = resp.headers.get("docker-content-digest")
return digest
return None
def _parse_version(tag: str) -> tuple[int, ...] | None:
"""Parse version string into comparable tuple."""
tag = tag.lstrip("vV")
parts = tag.split(".")
try:
return tuple(int(p) for p in parts)
except ValueError:
return None
def _find_updates(current_tag: str, tags: list[str]) -> list[str]:
"""Find tags newer than current based on version comparison."""
current_version = _parse_version(current_tag)
if current_version is None:
return []
updates = []
for tag in tags:
tag_version = _parse_version(tag)
if tag_version and tag_version > current_version:
updates.append(tag)
updates.sort(key=lambda t: _parse_version(t) or (), reverse=True)
return updates
async def check_image_updates(
image_str: str,
client: httpx.AsyncClient,
) -> TagCheckResult:
"""Check if newer versions are available for an image.
Args:
image_str: Image string like "nginx:1.25" or "ghcr.io/user/repo:tag"
client: httpx async client
Returns:
TagCheckResult with available updates
"""
image = ImageRef.parse(image_str)
registry_client = RegistryClient(image.registry)
try:
tags = await registry_client.get_tags(image, client)
updates = _find_updates(image.tag, tags)
current_digest = await registry_client.get_digest(image, image.tag, client) or ""
return TagCheckResult(
image=image,
current_digest=current_digest,
available_updates=updates,
)
except Exception as e:
return TagCheckResult(
image=image,
current_digest="",
error=str(e),
)

View File

@@ -8,6 +8,7 @@ use host-published ports for cross-host reachability.
from __future__ import annotations
import re
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any
@@ -383,3 +384,53 @@ def render_traefik_config(dynamic: dict[str, Any]) -> str:
"""Render Traefik dynamic config as YAML with a header comment."""
body = yaml.safe_dump(dynamic, sort_keys=False)
return _TRAEFIK_CONFIG_HEADER + body
_HOST_RULE_PATTERN = re.compile(r"Host\(`([^`]+)`\)")
def extract_website_urls(config: Config, stack: str) -> list[str]:
"""Extract website URLs from Traefik labels in a stack's compose file.
Reuses generate_traefik_config to parse labels, then extracts Host() rules
from router configurations.
Returns a list of unique URLs, preferring HTTPS over HTTP.
"""
try:
dynamic, _ = generate_traefik_config(config, [stack], check_all=True)
except FileNotFoundError:
return []
routers = dynamic.get("http", {}).get("routers", {})
if not routers:
return []
# Track URLs with their scheme preference (https > http)
urls: dict[str, str] = {} # host -> scheme
for router_info in routers.values():
if not isinstance(router_info, dict):
continue
rule = router_info.get("rule", "")
entrypoints = router_info.get("entrypoints", [])
# entrypoints can be a list or string
if isinstance(entrypoints, list):
entrypoints_str = ",".join(entrypoints)
else:
entrypoints_str = str(entrypoints)
# Determine scheme from entrypoint
scheme = "https" if "websecure" in entrypoints_str else "http"
# Extract host(s) from rule
for match in _HOST_RULE_PATTERN.finditer(str(rule)):
host = match.group(1)
# Prefer https over http
if host not in urls or scheme == "https":
urls[host] = scheme
# Build URL list, sorted for consistency
return sorted(f"{scheme}://{host}" for host, scheme in urls.items())

View File

@@ -4,18 +4,19 @@ from __future__ import annotations
import asyncio
import logging
import sys
from contextlib import asynccontextmanager, suppress
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Any, cast
from fastapi import FastAPI
from fastapi.middleware.gzip import GZipMiddleware
from fastapi.staticfiles import StaticFiles
from pydantic import ValidationError
from rich.logging import RichHandler
from compose_farm.web.deps import STATIC_DIR, get_config
from compose_farm.web.routes import actions, api, pages
from compose_farm.web.routes import actions, api, containers, pages
from compose_farm.web.streaming import TASK_TTL_SECONDS, cleanup_stale_tasks
from compose_farm.web.ws import router as ws_router
# Configure logging with Rich handler for compose_farm.web modules
logging.basicConfig(
@@ -64,17 +65,17 @@ def create_app() -> FastAPI:
lifespan=lifespan,
)
# Enable Gzip compression for faster transfers over slow networks
app.add_middleware(cast("Any", GZipMiddleware), minimum_size=1000)
# Mount static files
app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
app.include_router(pages.router)
app.include_router(containers.router)
app.include_router(api.router, prefix="/api")
app.include_router(actions.router, prefix="/api")
# WebSocket routes use Unix-only modules (fcntl, pty)
if sys.platform != "win32":
from compose_farm.web.ws import router as ws_router # noqa: PLC0415
app.include_router(ws_router)
app.include_router(ws_router)
return app

View File

@@ -1,70 +1,39 @@
"""CDN asset definitions and caching for tests and demo recordings.
This module provides a single source of truth for CDN asset URLs used in
browser tests and demo recordings. Assets are intercepted and served from
a local cache to eliminate network variability.
This module provides CDN asset URLs used in browser tests and demo recordings.
Assets are intercepted and served from a local cache to eliminate network
variability.
Note: The canonical list of CDN assets for production is in base.html
(with data-vendor attributes). This module includes those plus dynamically
loaded assets (like Monaco editor modules loaded by app.js).
The canonical list of CDN assets is in vendor-assets.json. This module loads
that file and provides the CDN_ASSETS dict for test caching.
"""
from __future__ import annotations
import json
import subprocess
from typing import TYPE_CHECKING
from pathlib import Path
def _load_cdn_assets() -> dict[str, tuple[str, str]]:
"""Load CDN assets from vendor-assets.json.
Returns:
Dict mapping URL to (filename, content_type) tuple.
"""
json_path = Path(__file__).parent / "vendor-assets.json"
with json_path.open() as f:
config = json.load(f)
return {asset["url"]: (asset["filename"], asset["content_type"]) for asset in config["assets"]}
if TYPE_CHECKING:
from pathlib import Path
# CDN assets to cache locally for tests/demos
# Format: URL -> (local_filename, content_type)
#
# If tests fail with "Uncached CDN request", add the URL here.
CDN_ASSETS: dict[str, tuple[str, str]] = {
# From base.html (data-vendor attributes)
"https://cdn.jsdelivr.net/npm/daisyui@5/themes.css": ("daisyui-themes.css", "text/css"),
"https://cdn.jsdelivr.net/npm/daisyui@5": ("daisyui.css", "text/css"),
"https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4": (
"tailwind.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/css/xterm.css": ("xterm.css", "text/css"),
"https://unpkg.com/htmx.org@2.0.4": ("htmx.js", "application/javascript"),
"https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/lib/xterm.js": (
"xterm.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/@xterm/addon-fit@0.10.0/lib/addon-fit.js": (
"xterm-fit.js",
"application/javascript",
),
# Monaco editor - dynamically loaded by app.js
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/loader.js": (
"monaco-loader.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/editor/editor.main.js": (
"monaco-editor-main.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/editor/editor.main.css": (
"monaco-editor-main.css",
"text/css",
),
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/base/worker/workerMain.js": (
"monaco-workerMain.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/basic-languages/yaml/yaml.js": (
"monaco-yaml.js",
"application/javascript",
),
"https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/base/browser/ui/codicons/codicon/codicon.ttf": (
"monaco-codicon.ttf",
"font/ttf",
),
}
# If tests fail with "Uncached CDN request", add the URL to vendor-assets.json.
CDN_ASSETS: dict[str, tuple[str, str]] = _load_cdn_assets()
def download_url(url: str) -> bytes | None:
@@ -99,6 +68,7 @@ def ensure_vendor_cache(cache_dir: Path) -> Path:
filepath = cache_dir / filename
if filepath.exists():
continue
filepath.parent.mkdir(parents=True, exist_ok=True)
content = download_url(url)
if not content:
msg = f"Failed to download {url} - check network/curl"

View File

@@ -38,7 +38,17 @@ def get_templates() -> Jinja2Templates:
def extract_config_error(exc: Exception) -> str:
"""Extract a user-friendly error message from a config exception."""
if isinstance(exc, ValidationError):
return "; ".join(err.get("msg", str(err)) for err in exc.errors())
parts = []
for err in exc.errors():
msg = err.get("msg", str(err))
loc = err.get("loc", ())
if loc:
# Format location as dot-separated path (e.g., "hosts.nas.port")
loc_str = ".".join(str(part) for part in loc)
parts.append(f"{loc_str}: {msg}")
else:
parts.append(msg)
return "; ".join(parts)
return str(exc)

View File

@@ -1,5 +1,5 @@
"""Web routes."""
from compose_farm.web.routes import actions, api, pages
from compose_farm.web.routes import actions, api, containers, pages
__all__ = ["actions", "api", "pages"]
__all__ = ["actions", "api", "containers", "pages"]

View File

@@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
import os
import uuid
from typing import TYPE_CHECKING, Any
@@ -14,6 +15,9 @@ if TYPE_CHECKING:
from compose_farm.web.deps import get_config
from compose_farm.web.streaming import run_cli_streaming, run_compose_streaming, tasks
# Environment variable to identify the web stack (for exclusion from bulk updates)
CF_WEB_STACK = os.environ.get("CF_WEB_STACK", "")
router = APIRouter(tags=["actions"])
# Store task references to prevent garbage collection
@@ -96,7 +100,15 @@ async def pull_all() -> dict[str, Any]:
@router.post("/update-all")
async def update_all() -> dict[str, Any]:
"""Update all stacks (pull + build + down + up)."""
"""Update all stacks, excluding the web stack. Only recreates if images changed.
The web stack is excluded to prevent the UI from shutting down mid-operation.
Use 'cf update <web-stack>' manually to update the web UI.
"""
config = get_config()
task_id = _start_task(lambda tid: run_cli_streaming(config, ["update", "--all"], tid))
return {"task_id": task_id, "command": "update --all"}
# Get all stacks except the web stack to avoid self-shutdown
stacks = [s for s in config.stacks if s != CF_WEB_STACK]
if not stacks:
return {"task_id": "", "command": "update (no stacks)", "skipped": True}
task_id = _start_task(lambda tid: run_cli_streaming(config, ["update", *stacks], tid))
return {"task_id": task_id, "command": f"update {' '.join(stacks)}"}

View File

@@ -19,8 +19,10 @@ import yaml
from fastapi import APIRouter, Body, HTTPException, Query
from fastapi.responses import HTMLResponse
from compose_farm.compose import extract_services, get_container_name, load_compose_data_for_stack
from compose_farm.executor import is_local, run_compose_on_host, ssh_connect_kwargs
from compose_farm.paths import find_config_path
from compose_farm.glances import fetch_all_host_stats
from compose_farm.paths import backup_dir, find_config_path
from compose_farm.state import load_state
from compose_farm.web.deps import get_config, get_templates
@@ -40,26 +42,29 @@ def _validate_yaml(content: str) -> None:
def _backup_file(file_path: Path) -> Path | None:
"""Create a timestamped backup of a file if it exists and content differs.
Backups are stored in a .backups directory alongside the file.
Backups are stored in XDG config dir under compose-farm/backups/.
The original file's absolute path is mirrored in the backup directory.
Returns the backup path if created, None if no backup was needed.
"""
if not file_path.exists():
return None
# Create backup directory
backup_dir = file_path.parent / ".backups"
backup_dir.mkdir(exist_ok=True)
# Create backup directory mirroring original path structure
# e.g., /opt/stacks/plex/compose.yaml -> ~/.config/compose-farm/backups/opt/stacks/plex/
resolved = file_path.resolve()
file_backup_dir = backup_dir() / resolved.parent.relative_to(resolved.anchor)
file_backup_dir.mkdir(parents=True, exist_ok=True)
# Generate timestamped backup filename
timestamp = datetime.now(tz=UTC).strftime("%Y%m%d_%H%M%S")
backup_name = f"{file_path.name}.{timestamp}"
backup_path = backup_dir / backup_name
backup_path = file_backup_dir / backup_name
# Copy current content to backup
backup_path.write_text(file_path.read_text())
# Clean up old backups (keep last 200)
backups = sorted(backup_dir.glob(f"{file_path.name}.*"), reverse=True)
backups = sorted(file_backup_dir.glob(f"{file_path.name}.*"), reverse=True)
for old_backup in backups[200:]:
old_backup.unlink()
@@ -101,13 +106,11 @@ def _get_compose_services(config: Any, stack: str, hosts: list[str]) -> list[dic
Returns one entry per container per host for multi-host stacks.
"""
compose_path = config.get_compose_path(stack)
if not compose_path or not compose_path.exists():
compose_path, compose_data = load_compose_data_for_stack(config, stack)
if not compose_path.exists():
return []
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
raw_services = extract_services(compose_data)
if not raw_services:
return []
# Project name is the directory name (docker compose default)
@@ -116,14 +119,9 @@ def _get_compose_services(config: Any, stack: str, hosts: list[str]) -> list[dic
containers = []
for host in hosts:
for svc_name, svc_def in raw_services.items():
# Use container_name if set, otherwise default to {project}-{service}-1
if isinstance(svc_def, dict) and svc_def.get("container_name"):
container_name = svc_def["container_name"]
else:
container_name = f"{project_name}-{svc_name}-1"
containers.append(
{
"Name": container_name,
"Name": get_container_name(svc_name, svc_def, project_name),
"Service": svc_name,
"Host": host,
"State": "unknown", # Status requires Docker query
@@ -385,3 +383,19 @@ async def write_console_file(
except Exception as e:
logger.exception("Failed to write file %s to host %s", path, host)
raise HTTPException(status_code=500, detail=str(e)) from e
@router.get("/glances", response_class=HTMLResponse)
async def get_glances_stats() -> HTMLResponse:
"""Get resource stats from Glances for all hosts."""
config = get_config()
if not config.glances_stack:
return HTMLResponse("") # Glances not configured
stats = await fetch_all_host_stats(config)
templates = get_templates()
template = templates.env.get_template("partials/glances.html")
html = template.render(stats=stats)
return HTMLResponse(html)

View File

@@ -0,0 +1,370 @@
"""Container dashboard routes using Glances API."""
from __future__ import annotations
import html
import re
from typing import TYPE_CHECKING
from urllib.parse import quote
import humanize
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse, JSONResponse
from compose_farm.executor import TTLCache
from compose_farm.glances import ContainerStats, fetch_all_container_stats
from compose_farm.registry import DOCKER_HUB_ALIASES, ImageRef
from compose_farm.web.deps import get_config, get_templates
router = APIRouter(tags=["containers"])
if TYPE_CHECKING:
from compose_farm.registry import TagCheckResult
# Cache registry update checks for 5 minutes (300 seconds)
# Registry calls are slow and often rate-limited
_update_check_cache = TTLCache(ttl_seconds=300.0)
# Minimum parts needed to infer stack/service from container name
MIN_NAME_PARTS = 2
# HTML for "no update info" dash
_DASH_HTML = '<span class="text-xs opacity-50">-</span>'
def _format_bytes(bytes_val: int) -> str:
"""Format bytes to human readable string."""
return humanize.naturalsize(bytes_val, binary=True, format="%.1f")
def _parse_image(image: str) -> tuple[str, str]:
"""Parse image string into (name, tag)."""
# Handle registry prefix (e.g., ghcr.io/user/repo:tag)
if ":" in image:
# Find last colon that's not part of port
parts = image.rsplit(":", 1)
if "/" in parts[-1]:
# The "tag" contains a slash, so it's probably a port
return image, "latest"
return parts[0], parts[1]
return image, "latest"
def _infer_stack_service(name: str) -> tuple[str, str]:
"""Fallback: infer stack and service from container name.
Used when compose labels are not available.
Docker Compose naming conventions:
- Default: {project}_{service}_{instance} or {project}-{service}-{instance}
- Custom: {container_name} from compose file
"""
# Try underscore separator first (older compose)
if "_" in name:
parts = name.split("_")
if len(parts) >= MIN_NAME_PARTS:
return parts[0], parts[1]
# Try hyphen separator (newer compose)
if "-" in name:
parts = name.split("-")
if len(parts) >= MIN_NAME_PARTS:
return parts[0], "-".join(parts[1:-1]) if len(parts) > MIN_NAME_PARTS else parts[1]
# Fallback: use name as both stack and service
return name, name
@router.get("/live-stats", response_class=HTMLResponse)
async def containers_page(request: Request) -> HTMLResponse:
"""Container dashboard page."""
config = get_config()
templates = get_templates()
# Check if Glances is configured
glances_enabled = config.glances_stack is not None
return templates.TemplateResponse(
"containers.html",
{
"request": request,
"glances_enabled": glances_enabled,
"hosts": sorted(config.hosts.keys()) if glances_enabled else [],
},
)
_STATUS_CLASSES = {
"running": "badge badge-success badge-sm",
"exited": "badge badge-error badge-sm",
"paused": "badge badge-warning badge-sm",
}
def _status_class(status: str) -> str:
"""Get CSS class for status badge."""
return _STATUS_CLASSES.get(status.lower(), "badge badge-ghost badge-sm")
def _progress_class(percent: float) -> str:
"""Get CSS class for progress bar color."""
if percent > 80: # noqa: PLR2004
return "bg-error"
if percent > 50: # noqa: PLR2004
return "bg-warning"
return "bg-success"
def _render_update_cell(image: str, tag: str) -> str:
"""Render update check cell with client-side batch updates."""
encoded_image = quote(image, safe="")
encoded_tag = quote(tag, safe="")
cached_html = _update_check_cache.get(f"{image}:{tag}")
inner = cached_html if cached_html is not None else _DASH_HTML
return (
f"""<td class="update-cell" data-image="{encoded_image}" data-tag="{encoded_tag}">"""
f"{inner}</td>"
)
def _image_web_url(image: str) -> str | None:
"""Return a human-friendly registry URL for an image (without tag)."""
ref = ImageRef.parse(image)
if ref.registry in DOCKER_HUB_ALIASES:
if ref.namespace == "library":
return f"https://hub.docker.com/_/{ref.name}"
return f"https://hub.docker.com/r/{ref.namespace}/{ref.name}"
return f"https://{ref.registry}/{ref.full_name}"
def _render_row(c: ContainerStats, idx: int | str) -> str:
"""Render a single container as an HTML table row."""
image_name, tag = _parse_image(c.image)
stack = c.stack if c.stack else _infer_stack_service(c.name)[0]
service = c.service if c.service else _infer_stack_service(c.name)[1]
cpu = c.cpu_percent
mem = c.memory_percent
cpu_class = _progress_class(cpu)
mem_class = _progress_class(mem)
# Highlight rows with high resource usage
high_cpu = cpu > 80 # noqa: PLR2004
high_mem = mem > 90 # noqa: PLR2004
row_class = "high-usage" if (high_cpu or high_mem) else ""
uptime_sec = _parse_uptime_seconds(c.uptime)
actions = _render_actions(stack)
update_cell = _render_update_cell(image_name, tag)
image_label = f"{image_name}:{tag}"
image_url = _image_web_url(image_name)
if image_url:
image_html = (
f'<a href="{image_url}" target="_blank" rel="noopener noreferrer" '
f'class="link link-hover">'
f'<code class="text-xs bg-base-200 px-1 rounded">{image_label}</code></a>'
)
else:
image_html = f'<code class="text-xs bg-base-200 px-1 rounded">{image_label}</code>'
# Render as single line to avoid whitespace nodes in DOM
row_id = f"c-{c.host}-{c.name}"
class_attr = f' class="{row_class}"' if row_class else ""
return (
f'<tr id="{row_id}" data-host="{c.host}"{class_attr}><td class="text-xs opacity-50">{idx}</td>'
f'<td data-sort="{stack.lower()}"><a href="/stack/{stack}" class="link link-hover link-primary" hx-boost="true">{stack}</a></td>'
f'<td data-sort="{service.lower()}" class="text-xs opacity-70">{service}</td>'
f"<td>{actions}</td>"
f'<td data-sort="{c.host.lower()}"><span class="badge badge-outline badge-xs">{c.host}</span></td>'
f'<td data-sort="{c.image.lower()}">{image_html}</td>'
f"{update_cell}"
f'<td data-sort="{c.status.lower()}"><span class="{_status_class(c.status)}">{c.status}</span></td>'
f'<td data-sort="{uptime_sec}" class="text-xs text-right font-mono">{c.uptime or "-"}</td>'
f'<td data-sort="{cpu}" class="text-right font-mono"><div class="flex flex-col items-end gap-0.5"><div class="w-12 h-2 bg-base-300 rounded-full overflow-hidden"><div class="h-full {cpu_class}" style="width: {min(cpu, 100)}%"></div></div><span class="text-xs">{cpu:.0f}%</span></div></td>'
f'<td data-sort="{c.memory_usage}" class="text-right font-mono"><div class="flex flex-col items-end gap-0.5"><div class="w-12 h-2 bg-base-300 rounded-full overflow-hidden"><div class="h-full {mem_class}" style="width: {min(mem, 100)}%"></div></div><span class="text-xs">{_format_bytes(c.memory_usage)}</span></div></td>'
f'<td data-sort="{c.network_rx + c.network_tx}" class="text-xs text-right font-mono">↓{_format_bytes(c.network_rx)}{_format_bytes(c.network_tx)}</td>'
"</tr>"
)
def _render_actions(stack: str) -> str:
"""Render actions dropdown for a container row."""
return f"""<button class="btn btn-circle btn-ghost btn-xs" onclick="openActionMenu(event, '{stack}')" aria-label="Actions for {stack}">
<svg class="h-4 w-4"><use href="#icon-menu" /></svg>
</button>"""
def _parse_uptime_seconds(uptime: str) -> int:
"""Parse uptime string to seconds for sorting."""
if not uptime:
return 0
uptime = uptime.lower().strip()
# Handle "a/an" as 1
uptime = uptime.replace("an ", "1 ").replace("a ", "1 ")
total = 0
multipliers = {
"second": 1,
"minute": 60,
"hour": 3600,
"day": 86400,
"week": 604800,
"month": 2592000,
"year": 31536000,
}
for match in re.finditer(r"(\d+)\s*(\w+)", uptime):
num = int(match.group(1))
unit = match.group(2).rstrip("s") # Remove plural 's'
total += num * multipliers.get(unit, 0)
return total
@router.get("/api/containers/rows", response_class=HTMLResponse)
async def get_containers_rows() -> HTMLResponse:
"""Get container table rows as HTML for HTMX.
Each cell has data-sort attribute for instant client-side sorting.
"""
config = get_config()
if not config.glances_stack:
return HTMLResponse(
'<tr><td colspan="12" class="text-center text-error">Glances not configured</td></tr>'
)
containers = await fetch_all_container_stats(config)
if not containers:
return HTMLResponse(
'<tr><td colspan="12" class="text-center py-4 opacity-60">No containers found</td></tr>'
)
rows = "\n".join(_render_row(c, i + 1) for i, c in enumerate(containers))
return HTMLResponse(rows)
@router.get("/api/containers/rows/{host_name}", response_class=HTMLResponse)
async def get_containers_rows_by_host(host_name: str) -> HTMLResponse:
"""Get container rows for a specific host.
Returns immediately with Glances data. Stack/service are inferred from
container names for instant display (no SSH wait).
"""
import logging # noqa: PLC0415
import time # noqa: PLC0415
from compose_farm.executor import get_container_compose_labels # noqa: PLC0415
from compose_farm.glances import fetch_container_stats # noqa: PLC0415
logger = logging.getLogger(__name__)
config = get_config()
if host_name not in config.hosts:
return HTMLResponse("")
host = config.hosts[host_name]
t0 = time.monotonic()
containers, error = await fetch_container_stats(host_name, host.address)
t1 = time.monotonic()
fetch_ms = (t1 - t0) * 1000
if containers is None:
logger.error(
"Failed to fetch stats for %s in %.1fms: %s",
host_name,
fetch_ms,
error,
)
return HTMLResponse(
f'<tr class="text-error"><td colspan="12" class="text-center py-2">Error: {error}</td></tr>'
)
if not containers:
return HTMLResponse("") # No rows for this host
labels = await get_container_compose_labels(config, host_name)
for c in containers:
stack, service = labels.get(c.name, ("", ""))
if not stack or not service:
stack, service = _infer_stack_service(c.name)
c.stack, c.service = stack, service
# Only show containers from stacks in config (filters out orphaned/unknown stacks)
containers = [c for c in containers if not c.stack or c.stack in config.stacks]
# Use placeholder index (will be renumbered by JS after all hosts load)
rows = "\n".join(_render_row(c, "-") for c in containers)
t2 = time.monotonic()
render_ms = (t2 - t1) * 1000
logger.info(
"Loaded %d rows for %s in %.1fms (fetch) + %.1fms (render)",
len(containers),
host_name,
fetch_ms,
render_ms,
)
return HTMLResponse(rows)
def _render_update_badge(result: TagCheckResult) -> str:
if result.error:
return _DASH_HTML
if result.available_updates:
updates = result.available_updates
count = len(updates)
title = f"Newer: {', '.join(updates[:3])}" + ("..." if count > 3 else "") # noqa: PLR2004
tip = html.escape(title, quote=True)
return (
f'<span class="tooltip" data-tip="{tip}">'
f'<span class="badge badge-warning badge-xs cursor-help">{count} new</span>'
"</span>"
)
return '<span class="tooltip" data-tip="Up to date"><span class="text-success text-xs">✓</span></span>'
@router.post("/api/containers/check-updates", response_class=JSONResponse)
async def check_container_updates_batch(request: Request) -> JSONResponse:
"""Batch update checks for a list of images.
Payload: {"items": [{"image": "...", "tag": "..."}, ...]}
Returns: {"results": [{"image": "...", "tag": "...", "html": "..."}, ...]}
"""
import httpx # noqa: PLC0415
payload = await request.json()
items = payload.get("items", []) if isinstance(payload, dict) else []
if not items:
return JSONResponse({"results": []})
results = []
from compose_farm.registry import check_image_updates # noqa: PLC0415
async with httpx.AsyncClient(timeout=10.0) as client:
for item in items:
image = item.get("image", "")
tag = item.get("tag", "")
full_image = f"{image}:{tag}"
if not image or not tag:
results.append({"image": image, "tag": tag, "html": _DASH_HTML})
continue
# NOTE: Tag-based checks cannot detect digest changes for moving tags
# like "latest". A future improvement could compare remote vs local
# digests using dockerfarm-log.toml (from `cf refresh`) or a per-host
# digest lookup.
cached_html: str | None = _update_check_cache.get(full_image)
if cached_html is not None:
results.append({"image": image, "tag": tag, "html": cached_html})
continue
try:
result = await check_image_updates(full_image, client)
html = _render_update_badge(result)
_update_check_cache.set(full_image, html)
except Exception:
_update_check_cache.set(full_image, _DASH_HTML, ttl_seconds=60.0)
html = _DASH_HTML
results.append({"image": image, "tag": tag, "html": html})
return JSONResponse({"results": results})

View File

@@ -7,6 +7,7 @@ from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse
from pydantic import ValidationError
from compose_farm.compose import extract_services, get_container_name, parse_compose_data
from compose_farm.paths import find_config_path
from compose_farm.state import (
get_orphaned_stacks,
@@ -16,6 +17,7 @@ from compose_farm.state import (
group_running_stacks_by_host,
load_state,
)
from compose_farm.traefik import extract_website_urls
from compose_farm.web.deps import (
extract_config_error,
get_config,
@@ -89,8 +91,8 @@ async def index(request: Request) -> HTMLResponse:
# Get state
deployed = load_state(config)
# Stats
running_count = len(deployed)
# Stats (only count stacks that are both in config AND deployed)
running_count = sum(1 for stack in deployed if stack in config.stacks)
stopped_count = len(config.stacks) - running_count
# Pending operations
@@ -159,13 +161,28 @@ async def stack_detail(request: Request, name: str) -> HTMLResponse:
# Get state
current_host = get_stack_host(config, name)
# Get service names from compose file
# Get service names and container info from compose file
services: list[str] = []
containers: dict[str, dict[str, str]] = {}
shell_host = current_host[0] if isinstance(current_host, list) else current_host
if compose_content:
compose_data = yaml.safe_load(compose_content) or {}
raw_services = compose_data.get("services", {})
if isinstance(raw_services, dict):
compose_data = parse_compose_data(compose_content)
raw_services = extract_services(compose_data)
if raw_services:
services = list(raw_services.keys())
# Build container info for shell access (only if stack is running)
if shell_host:
project_name = compose_path.parent.name if compose_path else name
containers = {
svc: {
"container": get_container_name(svc, svc_def, project_name),
"host": shell_host,
}
for svc, svc_def in raw_services.items()
}
# Extract website URLs from Traefik labels
website_urls = extract_website_urls(config, name)
return templates.TemplateResponse(
"stack.html",
@@ -179,6 +196,8 @@ async def stack_detail(request: Request, name: str) -> HTMLResponse:
"env_content": env_content,
"env_path": str(env_path) if env_path else None,
"services": services,
"containers": containers,
"website_urls": website_urls,
},
)
@@ -231,7 +250,8 @@ async def stats_partial(request: Request) -> HTMLResponse:
templates = get_templates()
deployed = load_state(config)
running_count = len(deployed)
# Only count stacks that are both in config AND deployed
running_count = sum(1 for stack in deployed if stack in config.stacks)
stopped_count = len(config.stacks) - running_count
return templates.TemplateResponse(

View File

@@ -9,7 +9,6 @@
// ANSI escape codes for terminal output
const ANSI = {
RED: '\x1b[31m',
GREEN: '\x1b[32m',
DIM: '\x1b[2m',
RESET: '\x1b[0m',
CRLF: '\r\n'
@@ -122,7 +121,6 @@ function whenXtermReady(callback, maxAttempts = 20) {
};
tryInit(maxAttempts);
}
window.whenXtermReady = whenXtermReady;
// ============================================================================
// TERMINAL
@@ -209,8 +207,6 @@ function initTerminal(elementId, taskId) {
return { term, ws };
}
window.initTerminal = initTerminal;
/**
* Initialize an interactive exec terminal
*/
@@ -223,7 +219,9 @@ function initExecTerminal(stack, container, host) {
return;
}
// Unhide the terminal container first, then expand/scroll
containerEl.classList.remove('hidden');
expandCollapse(document.getElementById('exec-collapse'), containerEl);
// Clean up existing (use wrapper's dispose to clean up ResizeObserver)
if (execWs) { execWs.close(); execWs = null; }
@@ -259,17 +257,42 @@ function initExecTerminal(stack, container, host) {
window.initExecTerminal = initExecTerminal;
/**
* Expand a collapse component and scroll to a target element
* @param {HTMLInputElement} toggle - The checkbox input that controls the collapse
* @param {HTMLElement} [scrollTarget] - Element to scroll to (defaults to collapse container)
*/
function expandCollapse(toggle, scrollTarget = null) {
if (!toggle) return;
// Find the parent collapse container
const collapse = toggle.closest('.collapse');
if (!collapse) return;
const target = scrollTarget || collapse;
const scrollToTarget = () => {
target.scrollIntoView({ behavior: 'smooth', block: 'start' });
};
if (!toggle.checked) {
// Collapsed - expand first, then scroll after transition
const onTransitionEnd = () => {
collapse.removeEventListener('transitionend', onTransitionEnd);
scrollToTarget();
};
collapse.addEventListener('transitionend', onTransitionEnd);
toggle.checked = true;
} else {
// Already expanded - just scroll
scrollToTarget();
}
}
/**
* Expand terminal collapse and scroll to it
*/
function expandTerminal() {
const toggle = document.getElementById('terminal-toggle');
if (toggle) toggle.checked = true;
const collapse = document.getElementById('terminal-collapse');
if (collapse) {
collapse.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
expandCollapse(document.getElementById('terminal-toggle'));
}
/**
@@ -309,10 +332,14 @@ function loadMonaco(callback) {
monacoLoading = true;
// Load the Monaco loader script
// Use local paths when running from vendored wheel, CDN otherwise
const monacoBase = window.CF_VENDORED
? '/static/vendor/monaco'
: 'https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs';
const script = document.createElement('script');
script.src = 'https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/loader.js';
script.src = monacoBase + '/loader.js';
script.onload = function() {
require.config({ paths: { vs: 'https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs' }});
require.config({ paths: { vs: monacoBase }});
require(['vs/editor/editor.main'], function() {
monacoLoaded = true;
monacoLoading = false;
@@ -405,7 +432,7 @@ function initMonacoEditors() {
* Save all editors
*/
async function saveAllEditors() {
const saveBtn = document.getElementById('save-btn') || document.getElementById('save-config-btn');
const saveBtn = getSaveButton();
const results = [];
for (const [id, editor] of Object.entries(editors)) {
@@ -441,12 +468,16 @@ async function saveAllEditors() {
* Initialize save button handler
*/
function initSaveButton() {
const saveBtn = document.getElementById('save-btn') || document.getElementById('save-config-btn');
const saveBtn = getSaveButton();
if (!saveBtn) return;
saveBtn.onclick = saveAllEditors;
}
function getSaveButton() {
return document.getElementById('save-btn') || document.getElementById('save-config-btn');
}
// ============================================================================
// UI HELPERS
// ============================================================================
@@ -520,12 +551,18 @@ function playFabIntro() {
let commands = [];
let filtered = [];
let selected = 0;
let originalTheme = null; // Store theme when palette opens for preview/restore
const post = (url) => () => htmx.ajax('POST', url, {swap: 'none'});
const nav = (url) => () => {
const nav = (url, afterNav) => () => {
// Set hash before HTMX swap so inline scripts can read it
const hashIndex = url.indexOf('#');
if (hashIndex !== -1) {
window.location.hash = url.substring(hashIndex);
}
htmx.ajax('GET', url, {target: '#main-content', select: '#main-content', swap: 'outerHTML'}).then(() => {
history.pushState({}, '', url);
window.scrollTo(0, 0);
afterNav?.();
});
};
// Navigate to dashboard (if needed) and trigger action
@@ -533,23 +570,25 @@ function playFabIntro() {
if (window.location.pathname !== '/') {
await htmx.ajax('GET', '/', {target: '#main-content', select: '#main-content', swap: 'outerHTML'});
history.pushState({}, '', '/');
window.scrollTo(0, 0);
}
htmx.ajax('POST', `/api/${endpoint}`, {swap: 'none'});
};
// Get saved theme from localStorage (source of truth)
const getSavedTheme = () => localStorage.getItem(THEME_KEY) || 'dark';
// Apply theme and save to localStorage
const setTheme = (theme) => () => {
document.documentElement.setAttribute('data-theme', theme);
localStorage.setItem(THEME_KEY, theme);
};
// Preview theme without saving (for hover)
// Preview theme without saving (for hover). Guards against undefined/invalid themes.
const previewTheme = (theme) => {
document.documentElement.setAttribute('data-theme', theme);
if (theme) document.documentElement.setAttribute('data-theme', theme);
};
// Restore original theme (when closing without selection)
// Restore theme from localStorage (source of truth)
const restoreTheme = () => {
if (originalTheme) {
document.documentElement.setAttribute('data-theme', originalTheme);
}
document.documentElement.setAttribute('data-theme', getSavedTheme());
};
// Generate color swatch HTML for a theme
const themeSwatch = (theme) => `<span class="flex gap-0.5" data-theme="${theme}"><span class="w-2 h-4 rounded-l bg-primary"></span><span class="w-2 h-4 bg-secondary"></span><span class="w-2 h-4 bg-accent"></span><span class="w-2 h-4 rounded-r bg-neutral"></span></span>`;
@@ -568,10 +607,15 @@ function playFabIntro() {
const actions = [
cmd('action', 'Apply', 'Make reality match config', dashboardAction('apply'), icons.check),
cmd('action', 'Refresh', 'Update state from reality', dashboardAction('refresh'), icons.refresh_cw),
cmd('action', 'Pull All', 'Pull latest images for all stacks', dashboardAction('pull-all'), icons.cloud_download),
cmd('action', 'Update All', 'Update all stacks except web', dashboardAction('update-all'), icons.refresh_cw),
cmd('app', 'Theme', 'Change color theme', openThemePicker, icons.palette),
cmd('app', 'Dashboard', 'Go to dashboard', nav('/'), icons.home),
cmd('app', 'Live Stats', 'View all containers across hosts', nav('/live-stats'), icons.box),
cmd('app', 'Console', 'Go to console', nav('/console'), icons.terminal),
cmd('app', 'Edit Config', 'Edit compose-farm.yaml', nav('/console#editor'), icons.file_code),
cmd('app', 'Docs', 'Open documentation', openExternal('https://compose-farm.nijho.lt/'), icons.book_open),
cmd('app', 'GitHub Repo', 'Open GitHub repository', openExternal('https://github.com/basnijholt/compose-farm'), icons.external_link),
];
// Add stack-specific actions if on a stack page
@@ -584,15 +628,30 @@ function playFabIntro() {
stackCmd('Down', 'Stop', 'down', icons.square),
stackCmd('Restart', 'Restart', 'restart', icons.rotate_cw),
stackCmd('Pull', 'Pull', 'pull', icons.cloud_download),
stackCmd('Update', 'Pull + restart', 'update', icons.refresh_cw),
stackCmd('Update', 'Pull + recreate', 'update', icons.refresh_cw),
stackCmd('Logs', 'View logs for', 'logs', icons.file_text),
);
// Add service-specific commands from data-services attribute
// Add Open Website commands if website URLs are available
const websiteUrlsAttr = document.querySelector('[data-website-urls]')?.getAttribute('data-website-urls');
if (websiteUrlsAttr) {
const websiteUrls = JSON.parse(websiteUrlsAttr);
for (const url of websiteUrls) {
const displayUrl = url.replace(/^https?:\/\//, '');
const label = websiteUrls.length > 1 ? `Open: ${displayUrl}` : 'Open Website';
actions.unshift(cmd('stack', label, `Open ${displayUrl} in browser`, openExternal(url), icons.external_link));
}
}
// Add service-specific commands from data-services and data-containers attributes
// Grouped by action (all Logs together, all Pull together, etc.) with services sorted alphabetically
const servicesAttr = document.querySelector('[data-services]')?.getAttribute('data-services');
const containersAttr = document.querySelector('[data-containers]')?.getAttribute('data-containers');
if (servicesAttr) {
const services = servicesAttr.split(',').filter(s => s).sort();
// Parse container info for shell access: {service: {container, host}}
const containers = containersAttr ? JSON.parse(containersAttr) : {};
const svcCmd = (action, service, desc, endpoint, icon) =>
cmd('service', `${action}: ${service}`, desc, post(`/api/stack/${stack}/service/${service}/${endpoint}`), icon);
const svcActions = [
@@ -607,6 +666,14 @@ function playFabIntro() {
actions.push(svcCmd(action, service, desc, endpoint, icon));
}
}
// Add Shell commands if container info is available
for (const service of services) {
const info = containers[service];
if (info?.container && info?.host) {
actions.push(cmd('service', `Shell: ${service}`, 'Open interactive shell',
() => initExecTerminal(stack, info.container, info.host), icons.terminal));
}
}
}
}
@@ -654,26 +721,24 @@ function playFabIntro() {
// Scroll selected item into view
const sel = list.querySelector(`[data-idx="${selected}"]`);
if (sel) sel.scrollIntoView({ block: 'nearest' });
// Preview theme if selected item is a theme command
// Preview theme if selected item is a theme command, otherwise restore saved
const selectedCmd = filtered[selected];
if (selectedCmd?.themeId) {
previewTheme(selectedCmd.themeId);
} else if (originalTheme) {
// Restore original when navigating away from theme commands
previewTheme(originalTheme);
} else {
restoreTheme();
}
}
function open(initialFilter = '') {
// Store original theme for preview/restore
originalTheme = document.documentElement.getAttribute('data-theme') || 'dark';
buildCommands();
selected = 0;
input.value = initialFilter;
filter();
// If opening theme picker, select current theme
if (initialFilter.startsWith('theme:')) {
const currentIdx = filtered.findIndex(c => c.themeId === originalTheme);
const savedTheme = getSavedTheme();
const currentIdx = filtered.findIndex(c => c.themeId === savedTheme);
if (currentIdx >= 0) selected = currentIdx;
}
render();
@@ -681,18 +746,9 @@ function playFabIntro() {
input.focus();
}
function close() {
dialog.close();
restoreTheme();
}
function exec() {
const cmd = filtered[selected];
if (cmd) {
if (cmd.themeId) {
// Theme command commits the previewed choice.
originalTheme = null;
}
dialog.close();
cmd.action();
}
@@ -732,19 +788,14 @@ function playFabIntro() {
if (a) previewTheme(a.dataset.themeId);
});
// Mouse leaving list restores to selected item's theme (or original)
// Mouse leaving list restores to selected item's theme (or saved)
list.addEventListener('mouseleave', () => {
const cmd = filtered[selected];
previewTheme(cmd?.themeId || originalTheme);
previewTheme(cmd?.themeId || getSavedTheme());
});
// Restore theme when dialog closes without selection (Escape, backdrop click)
dialog.addEventListener('close', () => {
if (originalTheme) {
restoreTheme();
originalTheme = null;
}
});
// Restore theme from localStorage when dialog closes
dialog.addEventListener('close', restoreTheme);
// FAB click to open
if (fab) fab.addEventListener('click', () => open());
@@ -807,6 +858,119 @@ function initPage() {
initMonacoEditors();
initSaveButton();
updateShortcutKeys();
initLiveStats();
initSharedActionMenu();
maybeRunStackAction();
}
function navigateToStack(stack, action = null) {
const url = action ? `/stack/${stack}?action=${action}` : `/stack/${stack}`;
window.location.href = url;
}
/**
* Initialize shared action menu for container rows
*/
function initSharedActionMenu() {
const menuEl = document.getElementById('shared-action-menu');
if (!menuEl) return;
if (menuEl.dataset.bound === '1') return;
menuEl.dataset.bound = '1';
let hoverTimeout = null;
function showMenuForButton(btn, stack) {
menuEl.dataset.stack = stack;
// Position menu relative to button
const rect = btn.getBoundingClientRect();
menuEl.classList.remove('hidden');
menuEl.style.visibility = 'hidden';
const menuRect = menuEl.getBoundingClientRect();
const left = rect.right - menuRect.width + window.scrollX;
const top = rect.bottom + window.scrollY;
menuEl.style.top = `${top}px`;
menuEl.style.left = `${left}px`;
menuEl.style.visibility = '';
if (typeof liveStats !== 'undefined') liveStats.dropdownOpen = true;
}
function closeMenu() {
menuEl.classList.add('hidden');
if (typeof liveStats !== 'undefined') liveStats.dropdownOpen = false;
menuEl.dataset.stack = '';
}
function scheduleClose() {
if (hoverTimeout) clearTimeout(hoverTimeout);
hoverTimeout = setTimeout(closeMenu, 100);
}
function cancelClose() {
if (hoverTimeout) {
clearTimeout(hoverTimeout);
hoverTimeout = null;
}
}
// Button hover: show menu (event delegation on tbody)
const tbody = document.getElementById('container-rows');
if (tbody) {
tbody.addEventListener('mouseenter', (e) => {
const btn = e.target.closest('button[onclick^="openActionMenu"]');
if (!btn) return;
// Extract stack from onclick attribute
const match = btn.getAttribute('onclick')?.match(/openActionMenu\(event,\s*'([^']+)'\)/);
if (!match) return;
cancelClose();
showMenuForButton(btn, match[1]);
}, true);
tbody.addEventListener('mouseleave', (e) => {
const btn = e.target.closest('button[onclick^="openActionMenu"]');
if (btn) scheduleClose();
}, true);
}
// Keep menu open while hovering over it
menuEl.addEventListener('mouseenter', cancelClose);
menuEl.addEventListener('mouseleave', scheduleClose);
// Click action in menu
menuEl.addEventListener('click', (e) => {
const link = e.target.closest('a[data-action]');
const stack = menuEl.dataset.stack;
if (!link || !stack) return;
e.preventDefault();
navigateToStack(stack, link.dataset.action);
closeMenu();
});
// Also support click on button (for touch/accessibility)
window.openActionMenu = function(event, stack) {
event.stopPropagation();
showMenuForButton(event.currentTarget, stack);
};
// Close on outside click
document.body.addEventListener('click', (e) => {
if (!menuEl.classList.contains('hidden') &&
!menuEl.contains(e.target) &&
!e.target.closest('button[onclick^="openActionMenu"]')) {
closeMenu();
}
});
// Close on Escape
document.body.addEventListener('keydown', (e) => {
if (e.key === 'Escape') closeMenu();
});
}
/**
@@ -827,6 +991,30 @@ function tryReconnectToTask(path) {
});
}
function maybeRunStackAction() {
const params = new URLSearchParams(window.location.search);
const stackEl = document.querySelector('[data-stack-name]');
const stackName = stackEl?.dataset?.stackName;
if (!stackName) return;
const action = params.get('action');
if (!action) return;
const button = document.querySelector(`button[hx-post="/api/stack/${stackName}/${action}"]`);
if (!button) return;
params.delete('action');
const newQuery = params.toString();
const newUrl = newQuery ? `${window.location.pathname}?${newQuery}` : window.location.pathname;
history.replaceState({}, '', newUrl);
if (window.htmx) {
htmx.trigger(button, 'click');
} else {
button.click();
}
}
// Initialize on page load
document.addEventListener('DOMContentLoaded', function() {
initPage();
@@ -868,3 +1056,443 @@ document.body.addEventListener('htmx:afterRequest', function(evt) {
// Not valid JSON, ignore
}
});
// ============================================================================
// LIVE STATS PAGE
// ============================================================================
// State persists across SPA navigation (intervals must be cleared on re-init)
let liveStats = {
sortCol: 9,
sortAsc: false,
lastUpdate: 0,
dropdownOpen: false,
scrolling: false,
scrollTimer: null,
loadingHosts: new Set(),
eventsBound: false,
intervals: [],
updateCheckTimes: new Map(),
autoRefresh: true
};
const REFRESH_INTERVAL = 5000;
const UPDATE_CHECK_TTL = 120000;
const NUMERIC_COLS = new Set([8, 9, 10, 11]); // uptime, cpu, mem, net
function filterTable() {
const textFilter = document.getElementById('filter-input')?.value.toLowerCase() || '';
const hostFilter = document.getElementById('host-filter')?.value || '';
const rows = document.querySelectorAll('#container-rows tr');
let visible = 0;
let total = 0;
rows.forEach(row => {
// Skip loading/empty/error rows (they have colspan)
if (row.cells[0]?.colSpan > 1) return;
total++;
const matchesText = !textFilter || row.textContent.toLowerCase().includes(textFilter);
const matchesHost = !hostFilter || row.dataset.host === hostFilter;
const show = matchesText && matchesHost;
row.style.display = show ? '' : 'none';
if (show) visible++;
});
const countEl = document.getElementById('container-count');
if (countEl) {
const isFiltering = textFilter || hostFilter;
countEl.textContent = total > 0
? (isFiltering ? `${visible} of ${total} containers` : `${total} containers`)
: '';
}
}
window.filterTable = filterTable;
function sortTable(col) {
if (liveStats.sortCol === col) {
liveStats.sortAsc = !liveStats.sortAsc;
} else {
liveStats.sortCol = col;
liveStats.sortAsc = false;
}
updateSortIndicators();
doSort();
}
window.sortTable = sortTable;
function updateSortIndicators() {
document.querySelectorAll('thead th').forEach((th, i) => {
const span = th.querySelector('.sort-indicator');
if (span) {
span.textContent = (i === liveStats.sortCol) ? (liveStats.sortAsc ? '↑' : '↓') : '';
span.style.opacity = (i === liveStats.sortCol) ? '1' : '0.3';
}
});
}
function doSort() {
const tbody = document.getElementById('container-rows');
if (!tbody) return;
const rows = Array.from(tbody.querySelectorAll('tr'));
if (rows.length === 0) return;
if (rows.length === 1 && rows[0].cells[0]?.colSpan > 1) return; // Empty state row
const isNumeric = NUMERIC_COLS.has(liveStats.sortCol);
rows.sort((a, b) => {
// Pin placeholders/empty rows to the bottom
const aLoading = a.classList.contains('loading-row') || a.classList.contains('host-empty') || a.cells[0]?.colSpan > 1;
const bLoading = b.classList.contains('loading-row') || b.classList.contains('host-empty') || b.cells[0]?.colSpan > 1;
if (aLoading && !bLoading) return 1;
if (!aLoading && bLoading) return -1;
if (aLoading && bLoading) return 0;
const aVal = a.cells[liveStats.sortCol]?.dataset?.sort ?? '';
const bVal = b.cells[liveStats.sortCol]?.dataset?.sort ?? '';
const cmp = isNumeric ? aVal - bVal : aVal.localeCompare(bVal);
return liveStats.sortAsc ? cmp : -cmp;
});
let index = 1;
const fragment = document.createDocumentFragment();
rows.forEach((row) => {
if (row.cells.length > 1) {
row.cells[0].textContent = index++;
}
fragment.appendChild(row);
});
tbody.appendChild(fragment);
}
function isLoading() {
return liveStats.loadingHosts.size > 0;
}
function getLiveStatsHosts() {
const tbody = document.getElementById('container-rows');
if (!tbody) return [];
const dataHosts = tbody.dataset.hosts || '';
return dataHosts.split(',').map(h => h.trim()).filter(Boolean);
}
function buildHostRow(host, message, className) {
return (
`<tr class="${className}" data-host="${host}">` +
`<td colspan="12" class="text-center py-2">` +
`<span class="text-sm opacity-60">${message}</span>` +
`</td></tr>`
);
}
async function checkUpdatesForHost(host) {
// Update checks always run - they only update small cells, not disruptive
const last = liveStats.updateCheckTimes.get(host) || 0;
if (Date.now() - last < UPDATE_CHECK_TTL) return;
const cells = Array.from(
document.querySelectorAll(`tr[data-host="${host}"] td.update-cell[data-image][data-tag]`)
);
if (cells.length === 0) return;
const items = [];
const seen = new Set();
cells.forEach(cell => {
const image = decodeURIComponent(cell.dataset.image || '');
const tag = decodeURIComponent(cell.dataset.tag || '');
const key = `${image}:${tag}`;
if (!image || seen.has(key)) return;
seen.add(key);
items.push({ image, tag });
});
if (items.length === 0) return;
try {
const response = await fetch('/api/containers/check-updates', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ items })
});
if (!response.ok) return;
const data = await response.json();
const results = Array.isArray(data?.results) ? data.results : [];
const htmlMap = new Map();
results.forEach(result => {
const key = `${result.image}:${result.tag}`;
htmlMap.set(key, result.html);
});
cells.forEach(cell => {
const image = decodeURIComponent(cell.dataset.image || '');
const tag = decodeURIComponent(cell.dataset.tag || '');
const key = `${image}:${tag}`;
const html = htmlMap.get(key);
if (html && cell.innerHTML !== html) {
cell.innerHTML = html;
}
});
liveStats.updateCheckTimes.set(host, Date.now());
} catch (e) {
console.error('Update check failed:', e);
}
}
function replaceHostRows(host, html) {
const tbody = document.getElementById('container-rows');
if (!tbody) return;
// Remove loading indicator for this host if present
const loadingRow = tbody.querySelector(`tr.loading-row[data-host="${host}"]`);
if (loadingRow) loadingRow.remove();
const template = document.createElement('template');
template.innerHTML = html.trim();
let newRows = Array.from(template.content.children).filter(el => el.tagName === 'TR');
if (newRows.length === 0) {
// Only show empty message if we don't have any rows for this host
const existing = tbody.querySelector(`tr[data-host="${host}"]:not(.loading-row)`);
if (!existing) {
template.innerHTML = buildHostRow(host, `No containers on ${host}`, 'host-empty');
newRows = Array.from(template.content.children);
}
}
// Track which IDs we've seen in this update
const newIds = new Set();
newRows.forEach(newRow => {
const id = newRow.id;
if (id) newIds.add(id);
if (id) {
const existing = document.getElementById(id);
if (existing) {
// Morph in place if Idiomorph is available, otherwise replace
if (typeof Idiomorph !== 'undefined') {
Idiomorph.morph(existing, newRow);
} else {
existing.replaceWith(newRow);
}
// Re-process HTMX if needed (though inner content usually carries attributes)
const morphedRow = document.getElementById(id);
if (window.htmx) htmx.process(morphedRow);
// Trigger refresh animation
if (morphedRow) {
morphedRow.classList.add('row-updated');
setTimeout(() => morphedRow.classList.remove('row-updated'), 500);
}
} else {
// New row - append (will be sorted later)
tbody.appendChild(newRow);
if (window.htmx) htmx.process(newRow);
// Animate new rows too
newRow.classList.add('row-updated');
setTimeout(() => newRow.classList.remove('row-updated'), 500);
}
} else {
// Fallback for rows without ID (like error/empty messages)
// Just append them, cleaning up previous generic rows handled below
tbody.appendChild(newRow);
}
});
// Remove orphaned rows for this host (rows that exist in DOM but not in new response)
// Be careful not to remove rows that were just added (if they lack IDs)
const currentHostRows = Array.from(tbody.querySelectorAll(`tr[data-host="${host}"]`));
currentHostRows.forEach(row => {
// Skip if it's one of the new rows we just appended (check presence in newRows?)
// Actually, if we just appended it, it is in DOM.
// We rely on ID matching.
// Error/Empty rows usually don't have ID, but we handle them by clearing old ones?
// Let's assume data rows have IDs.
if (row.id && !newIds.has(row.id)) {
row.remove();
}
// Also remove old empty/error messages if we now have data
if (!row.id && newRows.length > 0 && newRows[0].id) {
row.remove();
}
});
liveStats.loadingHosts.delete(host);
checkUpdatesForHost(host);
scheduleRowUpdate();
}
async function loadHostRows(host) {
liveStats.loadingHosts.add(host);
try {
const response = await fetch(`/api/containers/rows/${encodeURIComponent(host)}`);
const html = response.ok ? await response.text() : '';
replaceHostRows(host, html);
} catch (e) {
console.error(`Failed to load ${host}:`, e);
const msg = e.message || String(e);
// Fallback to simpler error display if replaceHostRows fails (e.g. Idiomorph missing)
try {
replaceHostRows(host, buildHostRow(host, `Error: ${msg}`, 'text-error'));
} catch (err2) {
// Last resort: find row and force innerHTML
const tbody = document.getElementById('container-rows');
const row = tbody?.querySelector(`tr[data-host="${host}"]`);
if (row) row.innerHTML = `<td colspan="12" class="text-center text-error">Error: ${msg}</td>`;
}
} finally {
liveStats.loadingHosts.delete(host);
}
}
function refreshLiveStats() {
if (liveStats.dropdownOpen || liveStats.scrolling) return;
const hosts = getLiveStatsHosts();
if (hosts.length === 0) return;
liveStats.lastUpdate = Date.now();
hosts.forEach(loadHostRows);
}
window.refreshLiveStats = refreshLiveStats;
function toggleAutoRefresh() {
liveStats.autoRefresh = !liveStats.autoRefresh;
const timer = document.getElementById('refresh-timer');
if (timer) {
timer.classList.toggle('btn-error', !liveStats.autoRefresh);
timer.classList.toggle('btn-outline', liveStats.autoRefresh);
}
if (liveStats.autoRefresh) {
// Re-enabling: trigger immediate refresh
refreshLiveStats();
} else {
// Disabling: ensure update checks run for current data
const hosts = getLiveStatsHosts();
hosts.forEach(host => checkUpdatesForHost(host));
}
}
window.toggleAutoRefresh = toggleAutoRefresh;
function initLiveStats() {
if (!document.getElementById('refresh-timer')) return;
// Clear previous intervals (important for SPA navigation)
liveStats.intervals.forEach(clearInterval);
liveStats.intervals = [];
liveStats.lastUpdate = Date.now();
liveStats.dropdownOpen = false;
liveStats.scrolling = false;
if (liveStats.scrollTimer) clearTimeout(liveStats.scrollTimer);
liveStats.scrollTimer = null;
liveStats.loadingHosts.clear();
liveStats.updateCheckTimes = new Map();
liveStats.autoRefresh = true;
if (!liveStats.eventsBound) {
liveStats.eventsBound = true;
// Dropdown pauses refresh
document.body.addEventListener('click', e => {
liveStats.dropdownOpen = !!e.target.closest('.dropdown');
});
document.body.addEventListener('focusin', e => {
if (e.target.closest('.dropdown')) liveStats.dropdownOpen = true;
});
document.body.addEventListener('focusout', () => {
setTimeout(() => {
liveStats.dropdownOpen = !!document.activeElement?.closest('.dropdown');
}, 150);
});
document.body.addEventListener('keydown', e => {
if (e.key === 'Escape') liveStats.dropdownOpen = false;
});
// Pause refresh while scrolling (helps on slow mobile browsers)
window.addEventListener('scroll', () => {
liveStats.scrolling = true;
if (liveStats.scrollTimer) clearTimeout(liveStats.scrollTimer);
liveStats.scrollTimer = setTimeout(() => {
liveStats.scrolling = false;
}, 200);
}, { passive: true });
}
// Auto-refresh every 5 seconds (skip if disabled, loading, or dropdown open)
liveStats.intervals.push(setInterval(() => {
if (!liveStats.autoRefresh) return;
if (liveStats.dropdownOpen || liveStats.scrolling || isLoading()) return;
refreshLiveStats();
}, REFRESH_INTERVAL));
// Timer display (updates every 100ms)
liveStats.intervals.push(setInterval(() => {
const timer = document.getElementById('refresh-timer');
if (!timer) {
liveStats.intervals.forEach(clearInterval);
return;
}
const loading = isLoading();
const paused = liveStats.dropdownOpen || liveStats.scrolling;
const elapsed = Date.now() - liveStats.lastUpdate;
window.refreshPaused = paused || loading || !liveStats.autoRefresh;
// Update refresh timer button
let text;
if (!liveStats.autoRefresh) {
text = 'OFF';
} else if (paused) {
text = '❚❚';
} else {
const remaining = Math.max(0, REFRESH_INTERVAL - elapsed);
text = loading ? '↻ …' : `${Math.ceil(remaining / 1000)}s`;
}
if (timer.textContent !== text) {
timer.textContent = text;
}
// Update "last updated" display
const lastUpdatedEl = document.getElementById('last-updated');
if (lastUpdatedEl) {
const secs = Math.floor(elapsed / 1000);
const updatedText = secs < 5 ? 'Updated just now' : `Updated ${secs}s ago`;
if (lastUpdatedEl.textContent !== updatedText) {
lastUpdatedEl.textContent = updatedText;
}
}
}, 100));
updateSortIndicators();
refreshLiveStats();
}
function scheduleRowUpdate() {
// Sort and filter immediately to prevent flicker
doSort();
filterTable();
}
// ============================================================================
// STACKS BY HOST FILTER
// ============================================================================
function sbhFilter() {
const query = (document.getElementById('sbh-filter')?.value || '').toLowerCase();
const hostFilter = document.getElementById('sbh-host-select')?.value || '';
document.querySelectorAll('.sbh-group').forEach(group => {
if (hostFilter && group.dataset.h !== hostFilter) {
group.hidden = true;
return;
}
let visibleCount = 0;
group.querySelectorAll('li[data-s]').forEach(li => {
const show = !query || li.dataset.s.includes(query);
li.hidden = !show;
if (show) visibleCount++;
});
group.hidden = visibleCount === 0;
});
}
window.sbhFilter = sbhFilter;

View File

@@ -103,8 +103,8 @@ def _is_self_update(stack: str, command: str) -> bool:
"""
if not CF_WEB_STACK or stack != CF_WEB_STACK:
return False
# Commands that involve 'down' need SSH: update, restart, down
return command in ("update", "restart", "down")
# Commands that involve 'down' need SSH: update, down
return command in ("update", "down")
async def _run_cli_via_ssh(

View File

@@ -26,6 +26,23 @@
</script>
</head>
<body class="min-h-screen bg-base-200">
<svg style="display: none">
<symbol id="icon-menu" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="5" r="1" /><circle cx="12" cy="12" r="1" /><circle cx="12" cy="19" r="1" />
</symbol>
<symbol id="icon-restart" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M4 4v5h.582m15.356 2A8.001 8.001 0 004.582 9m0 0H9m11 11v-5h-.581m0 0a8.003 8.003 0 01-15.357-2m15.357 2H15" />
</symbol>
<symbol id="icon-pull" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-4l-4 4m0 0l-4-4m4 4V4" />
</symbol>
<symbol id="icon-update" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M4 16v1a3 3 0 003 3h10a3 3 0 003-3v-1m-4-8l-4-4m0 0L8 8m4-4v12" />
</symbol>
<symbol id="icon-logs" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5a2 2 0 012-2h5.586a1 1 0 01.707.293l5.414 5.414a1 1 0 01.293.707V19a2 2 0 01-2 2z" />
</symbol>
</svg>
<div class="drawer lg:drawer-open">
<input id="drawer-toggle" type="checkbox" class="drawer-toggle" />
@@ -51,15 +68,21 @@
<header class="p-4 border-b border-base-300">
<h2 class="text-lg font-semibold flex items-center gap-2">
<span class="rainbow-hover">Compose Farm</span>
<a href="https://compose-farm.nijho.lt/" target="_blank" title="Docs" class="opacity-50 hover:opacity-100 transition-opacity">
{{ book_open() }}
</a>
<a href="https://github.com/basnijholt/compose-farm" target="_blank" title="GitHub" class="opacity-50 hover:opacity-100 transition-opacity">
{{ github() }}
</a>
<button type="button" id="theme-btn" class="opacity-50 hover:opacity-100 transition-opacity cursor-pointer" title="Change theme (opens command palette)">
{{ palette() }}
</button>
<div class="tooltip tooltip-bottom" data-tip="Docs">
<a href="https://compose-farm.nijho.lt/" target="_blank" class="opacity-50 hover:opacity-100 transition-opacity">
{{ book_open() }}
</a>
</div>
<div class="tooltip tooltip-bottom" data-tip="GitHub">
<a href="https://github.com/basnijholt/compose-farm" target="_blank" class="opacity-50 hover:opacity-100 transition-opacity">
{{ github() }}
</a>
</div>
<div class="tooltip tooltip-bottom" data-tip="Change theme">
<button type="button" id="theme-btn" class="opacity-50 hover:opacity-100 transition-opacity cursor-pointer">
{{ palette() }}
</button>
</div>
</h2>
</header>
<nav class="flex-1 overflow-y-auto p-2" hx-get="/partials/sidebar" hx-trigger="load, cf:refresh from:body" hx-swap="innerHTML">
@@ -74,6 +97,8 @@
<!-- Scripts - HTMX first -->
<script src="https://unpkg.com/htmx.org@2.0.4" data-vendor="htmx.js"></script>
<script src="https://unpkg.com/idiomorph/dist/idiomorph.min.js" data-vendor="idiomorph.js"></script>
<script src="https://unpkg.com/idiomorph/dist/idiomorph-ext.min.js" data-vendor="idiomorph-ext.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/lib/xterm.js" data-vendor="xterm.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@xterm/addon-fit@0.10.0/lib/addon-fit.js" data-vendor="xterm-fit.js"></script>
<script src="/static/app.js"></script>

View File

@@ -97,7 +97,10 @@ function connectConsole() {
consoleWs.onopen = () => {
statusEl.textContent = `Connected to ${host}`;
sendSize(term.cols, term.rows);
term.focus();
// Focus terminal unless #editor hash is present (command palette Edit Config)
if (window.location.hash !== '#editor') {
term.focus();
}
// Auto-load the default file once editor is ready
const pathInput = document.getElementById('console-file-path');
if (pathInput && pathInput.value) {
@@ -133,6 +136,14 @@ function initConsoleEditor() {
loadMonaco(() => {
consoleEditor = createEditor(editorEl, '', 'plaintext', { onSave: saveFile });
// Focus editor if #editor hash is present (command palette Edit Config)
if (window.location.hash === '#editor') {
// Small delay for Monaco to fully initialize before focusing
setTimeout(() => {
consoleEditor.focus();
editorEl.scrollIntoView({ behavior: 'smooth', block: 'center' });
}, 100);
}
});
}

Some files were not shown because too many files have changed in this diff Show More