Compare commits

...

148 Commits

Author SHA1 Message Date
Bas Nijholt
81e1a482f4 fix(docs): use Nerd Font icon for emoji in quickstart demo (#68) 2025-12-20 12:36:29 -08:00
Bas Nijholt
435b014251 docs: move demo up and add Dockge comparison (#67) 2025-12-20 10:28:59 -08:00
Bas Nijholt
58585ac73c docs: fix inaccuracies and add missing documentation (#66) 2025-12-20 10:27:15 -08:00
Bas Nijholt
5a848ec416 fix(docs): fix video display on GitHub Pages (#65) 2025-12-20 10:14:51 -08:00
Bas Nijholt
b4595cb117 docs: add comprehensive Zensical-based documentation (#62) 2025-12-20 09:57:59 -08:00
Bas Nijholt
5f1c31b780 feat: show docker compose command before execution (#64) 2025-12-20 00:35:35 -08:00
Bas Nijholt
9974f87976 feat: add bootstrap script for one-liner installation (#63)
Adds a curl-able install script that installs uv (if needed) and
compose-farm as a uv tool. Updated README with the one-liner.
2025-12-19 23:54:00 -08:00
Bas Nijholt
8b16484ce2 feat(web): add theme switcher with 35 DaisyUI themes (#61) 2025-12-19 22:33:10 -08:00
Bas Nijholt
d75f9cca64 refactor(web): organize app.js into logical sections (#60)
Reorganize JavaScript into 8 clear sections for better maintainability:
- Constants (ANSI, theme, language map)
- State (all globals in one place)
- Utilities (createWebSocket, whenXtermReady, etc.)
- Terminal (all xterm.js functions together)
- Editor (all Monaco functions together)
- UI Helpers (dashboard refresh, sidebar filter)
- Command Palette (self-contained IIFE)
- Initialization (entry points and event handlers)

No functional changes - only reordering and section headers added.
2025-12-19 20:23:39 -08:00
Bas Nijholt
7ccb0734a2 refactor(web): consolidate JS patterns and use icon macros (#58) 2025-12-19 14:55:31 -08:00
Bas Nijholt
61a845fad8 test: add comprehensive browser tests for HTMX/JS functionality (#59) 2025-12-19 14:27:00 -08:00
Bas Nijholt
e7efae0153 refactor: remove dead code and reduce duplication (#57)
- Delete unused add_service_to_host/remove_service_from_host from state.py
  (42 lines of dead code never called anywhere)

- Extract _stream_output_lines helper in executor.py to deduplicate
  identical read_stream functions in _run_local_command and _run_ssh_command

- Simplify unique-list logic in compose.py using dict.fromkeys()
  instead of manual seen/unique set/list pattern

Total: -67 lines
2025-12-18 23:56:49 -08:00
Bas Nijholt
b4ebe15dd1 refactor: simplify codebase with reduced abstractions (#56)
- Remove dead code: `run_host_operation` in cli/common.py (never called)
- Inline `_report_*` helpers in lifecycle.py (each called once)
- Merge `validate_host` into `validate_hosts` with flexible str|list param
- Merge `_report_no_config_found` and `_report_config_path_not_exists`
  into single `_report_missing_config` function
- Simplify `_get_editor` from 18 lines to 6 using walrus operator
- Extract `COMPOSE_FILENAMES` constant to avoid duplication in config.py
- Extract `_stream_subprocess` helper to reduce duplication in streaming.py

Net reduction: ~130 lines of code with no functionality changes.
2025-12-18 23:45:34 -08:00
Bas Nijholt
9f55dcdd6e refactor(web): Modernize JavaScript with cleaner patterns (#55) 2025-12-18 23:02:07 -08:00
Bas Nijholt
0694bbe56d feat(web): Show (local) label in sidebar host selector (#52) 2025-12-18 21:59:41 -08:00
Bas Nijholt
3045948d0a feat(web): Show (local) label in sidebar host selector (#50)
* feat(web): Show (local) label in sidebar host selector

Add local host detection to sidebar partial, matching the console page
behavior where the current machine is labeled with "(local)" in the
host dropdown.

* refactor: Extract get_local_host() helper to deps.py

DRY up the local host detection logic that was duplicated between
console and sidebar_partial routes.

* revert
2025-12-18 20:12:29 -08:00
Bas Nijholt
1fa17b4e07 feat(web): Auto-refresh dashboard and clean up HTMX inheritance (#49) 2025-12-18 20:07:31 -08:00
Bas Nijholt
cd25a1914c fix(web): Show exit code for stopped containers instead of loading spinner (#51)
One-shot containers (like CLI tools) were showing a perpetual loading
spinner because they weren't in `docker compose ps` output. Now we:
- Use `ps -a` to include stopped/exited containers
- Display exit code: neutral badge for clean exit (0), error badge for failures
- Show "created" state for containers that were never started
2025-12-18 20:03:12 -08:00
Bas Nijholt
a71200b199 feat(test): Add Playwright browser tests for web UI (#48) 2025-12-18 18:26:23 -08:00
Bas Nijholt
967d68b14a revert: Remove mobile rainbow glow adjustments
Reverts #46 and #47. The reduced background-size caused a green
tint at rest. The improvement in animation visibility wasn't
worth the trade-off.
2025-12-18 16:16:31 -08:00
Bas Nijholt
b7614aeab7 fix(web): Adjust mobile rainbow glow to avoid green edge (#47)
500% background-size showed too much of the gradient at rest,
revealing green (#bfff80) at the button edge. 650% shows ~15%
of the gradient, landing safely on white while still improving
color visibility during animation.
2025-12-18 16:11:58 -08:00
Bas Nijholt
d931784935 fix(web): Make rainbow glow animation more visible on mobile (#46)
The 900% background-size meant only ~11% of the gradient was visible
at any time. On smaller screens, the rainbow colors would flash by
too quickly during the intro animation, appearing mostly white.

Use a CSS variable for background-size and reduce it to 500% on
mobile (<768px), showing ~20% of the gradient for a more visible
rainbow effect.
2025-12-18 15:53:03 -08:00
Bas Nijholt
4755065229 feat(web): Add collapsible blocks to console terminal and editor (#44) 2025-12-18 15:52:36 -08:00
Bas Nijholt
e86bbf7681 fix(web): Make task-not-found message more general (#45) 2025-12-18 15:37:08 -08:00
Bas Nijholt
be136eb916 fix(web): Show friendlier message when task not found after restart
After a self-update, the browser tries to reconnect to the old task_id
but the in-memory task registry is empty (new container). Show a
helpful message instead of a scary "Error" message.
2025-12-18 15:34:07 -08:00
Bas Nijholt
78a223878f fix(web): Use nohup for self-updates to survive container death (#41) 2025-12-18 15:29:37 -08:00
Bas Nijholt
f5be23d626 fix(web): Ensure URL updates after HTMX navigation in command palette (#43)
* fix(web): Ensure URL updates after HTMX navigation in command palette

Use history.pushState() after HTMX swap completes to ensure
window.location.pathname is correct when rebuilding commands.

* docs: Add rule about unchecked checklists in PR descriptions
2025-12-18 15:22:10 -08:00
Bas Nijholt
3bdc483c2a feat(web): Add rainbow glow effect to command palette button (#42) 2025-12-18 15:13:49 -08:00
Bas Nijholt
3a3591a0f7 feat(web): Allow reconnection to running tasks after navigation (#38) 2025-12-18 14:27:06 -08:00
Bas Nijholt
7f8ea49d7f fix(web): Enable TTY for self-update SSH to show progress bars (#40)
* fix(web): Add PATH for self-update SSH command

Non-interactive SSH sessions don't source profile files, so `cf` isn't
found when installed in ~/.local/bin. Prepend common install locations
to PATH before running the remote command.

* fix(web): Enable TTY for self-update SSH to show progress bars
2025-12-18 14:19:21 -08:00
Bas Nijholt
1e67bde96c fix(web): Add PATH for self-update SSH command (#39)
Non-interactive SSH sessions don't source profile files, so `cf` isn't
found when installed in ~/.local/bin. Prepend common install locations
to PATH before running the remote command.
2025-12-18 14:17:03 -08:00
Bas Nijholt
d8353dbb7e fix: Skip socket paths in preflight volume checks (#37)
Socket paths like SSH_AUTH_SOCK are machine-local and shouldn't be
validated on remote hosts during preflight checks.
2025-12-18 13:59:06 -08:00
Bas Nijholt
2e6146a94b feat(ps): Add service filtering to ps command (#33) 2025-12-18 13:31:18 -08:00
Bas Nijholt
87849a8161 fix(web): Run self-updates via SSH to survive container restart (#35) 2025-12-18 13:10:30 -08:00
Bas Nijholt
c8bf792a9a refactor: Store SSH keys in subdirectory for cleaner volume mounting (#36)
* refactor: Store SSH keys in subdirectory for cleaner volume mounting

Change SSH key location from ~/.ssh/compose-farm (file) to
~/.ssh/compose-farm/id_ed25519 (file in directory).

This allows docker-compose to mount just the compose-farm directory
to /root/.ssh without exposing all host SSH keys to the container.

Also make host path the default option in docker-compose.yml with
clearer comments about the two options.

* docs: Update README for new SSH key directory structure

* docs: Clarify cf ssh setup must run inside container
2025-12-18 13:07:41 -08:00
Bas Nijholt
d37295fbee feat(web): Add distinct color for Dashboard/Console in command palette (#34)
Give Dashboard and Console a purple accent to visually distinguish
them from service navigation items in the Command K palette.
2025-12-18 12:38:28 -08:00
Bas Nijholt
266f541d35 fix(web): Auto-scroll Command K palette when navigating with arrow keys (#32)
When using arrow keys to navigate through the command palette list,
items outside the visible area now scroll into view automatically.
2025-12-18 12:30:29 -08:00
Bas Nijholt
aabdd550ba feat(cli): Add progress bar to ssh status host connectivity check (#31)
Use run_parallel_with_progress for visual feedback during host checks.
Results are now sorted alphabetically for consistent output.

Also adds code style rule to CLAUDE.md about keeping imports at top level.
2025-12-18 12:21:47 -08:00
Bas Nijholt
8ff60a1e3e refactor(ssh): Unify ssh_status to use run_command like check command (#29) 2025-12-18 12:17:47 -08:00
Bas Nijholt
2497bd727a feat(web): Navigate to dashboard for Apply/Refresh from command palette (#28)
When triggering Apply or Refresh from the command palette on a non-dashboard
page, navigate to the dashboard first and then execute the action, opening
the terminal output.
2025-12-18 12:12:50 -08:00
Bas Nijholt
e37d9d87ba feat(web): Add icons to Command K palette items (#27) 2025-12-18 12:08:55 -08:00
Bas Nijholt
80a1906d90 fix(web): Fix console page not initializing on HTMX navigation (#26)
* fix(web): Fix console page not initializing on HTMX navigation

Move inline script from {% block scripts %} to inside {% block content %}
so it's included in HTMX swaps. The script block was outside #main-content,
so hx-select="#main-content" was discarding it during navigation.

Also wrap script in IIFE to prevent let re-declaration errors when
navigating back to the console page.

* refactor(web): Simplify console script using var instead of IIFE
2025-12-18 12:05:30 -08:00
Bas Nijholt
282de12336 feat(cli): Add ssh subcommand for SSH key management (#22) 2025-12-18 11:58:33 -08:00
Bas Nijholt
2c5308aea3 fix(web): Add Console navigation to Command K palette (#25)
The Command K menu was missing an option to navigate to the Console page,
even though it's available in the sidebar.
2025-12-18 11:55:30 -08:00
Bas Nijholt
5057202938 refactor: DRY cleanup and message consistency (#24) 2025-12-18 11:45:32 -08:00
Bas Nijholt
5e1b9987dd fix(web): Set PTY as controlling terminal for local shell sessions (#23)
Local shell sessions weren't receiving SIGINT (Ctrl+C) because the PTY
wasn't set as the controlling terminal. Add preexec_fn that calls
setsid() and TIOCSCTTY to properly set up the terminal.
2025-12-18 11:12:37 -08:00
Bas Nijholt
d9c26f7f2c Merge pull request #21 from basnijholt/refactor/dry-cleanup
refactor: DRY cleanup - consolidate duplicate code patterns
2025-12-18 11:12:24 -08:00
Bas Nijholt
adfcd4bb31 style: Capitalize "Hint:" consistently 2025-12-18 11:05:53 -08:00
Bas Nijholt
95f7d9c3cf style(cli): Unify "not found" message format with color highlighting
- Services use [cyan] highlighting consistently
- Hosts use [magenta] highlighting consistently
- All use the same "X not found in config" pattern
2025-12-18 11:05:05 -08:00
Bas Nijholt
4c1674cfd8 style(cli): Unify error message format with ✗ prefix
All CLI error messages now consistently use the [red]✗[/] prefix
pattern instead of wrapping the entire message in [red]...[/red].
2025-12-18 11:04:28 -08:00
Bas Nijholt
f65ca8420e fix(web): Filter empty hosts from services_by_host
Preserve original behavior where only hosts with running services are
shown in the dashboard, rather than all configured hosts.
2025-12-18 11:00:01 -08:00
Bas Nijholt
85aff2c271 refactor(state): Move group_services_by_host to state.py
Consolidate duplicate service grouping logic from monitoring.py and
pages.py into a shared function in state.py.
2025-12-18 10:55:53 -08:00
Bas Nijholt
61ca24bb8e refactor(cli): Remove unused get_description parameter
All callers used the same pattern (r[0]), so hardcode it in the helper
and remove the parameter entirely.
2025-12-18 10:54:12 -08:00
Bas Nijholt
ed36588358 refactor(cli): Add validate_host and validate_hosts helpers
Extract common host validation patterns into reusable helpers.
Also simplifies validate_host_for_service to use the new validate_host
helper internally.
2025-12-18 10:49:57 -08:00
Bas Nijholt
80c8079a8c refactor(executor): Add ssh_connect_kwargs helper
Extract common asyncssh.connect parameters into a reusable
ssh_connect_kwargs() function. Used by executor.py, api.py, and ws.py.

Lines: 2608 → 2601 (-7)
2025-12-18 10:48:29 -08:00
Bas Nijholt
763bedf9f6 refactor(cli): Extract config not found helpers
Consolidate repeated "config not found" and "path doesn't exist"
messages into _report_no_config_found() and _report_config_path_not_exists()
helper functions. Also unifies the UX to always show status of searched
paths.
2025-12-18 10:46:58 -08:00
Bas Nijholt
641f7e91a8 refactor(cli): Consolidate _report_*_errors() functions
Merge _report_mount_errors, _report_network_errors, and _report_device_errors
into a single _report_requirement_errors function that takes a category
parameter.

Lines: 2634 → 2608 (-26)
2025-12-18 10:43:49 -08:00
Bas Nijholt
4e8e925d59 refactor(cli): Add run_parallel_with_progress helper
Extract common async progress bar pattern into a reusable helper in
common.py. Updates _discover_services, _check_ssh_connectivity,
_check_service_requirements, _get_container_counts, and _snapshot_services
to use the new helper.

Lines: 2642 → 2634 (-8)
2025-12-18 10:42:45 -08:00
Bas Nijholt
d84858dcfb fix(docker): Add restart policy to web service (#19)
* fix(docker): Add restart policy to containers

* fix: Only add restart policy to web service
2025-12-18 10:39:09 -08:00
Bas Nijholt
3121ee04eb feat(web): Show ⌘K shortcut on command palette button (#20) 2025-12-18 10:38:57 -08:00
Bas Nijholt
a795132a04 refactor(cli): Move format_host to common.py
Consolidate duplicate _format_host() function from lifecycle.py and
management.py into a single format_host() function in common.py.

Lines: 2647 → 2642 (-5)
2025-12-18 10:38:52 -08:00
Bas Nijholt
a6e491575a feat(web): Add Console page with terminal and editor (#17) 2025-12-18 10:29:15 -08:00
Bas Nijholt
78bf90afd9 docs: Improve Releases section in CLAUDE.md 2025-12-18 10:04:56 -08:00
Bas Nijholt
76b60bdd96 feat(web): Add Console page with terminal and editor
Add a new Console page accessible from the sidebar that provides:
- Interactive terminal with full shell access to any configured host
- SSH agent forwarding for authentication to remote hosts
- Monaco editor for viewing/editing files on hosts
- Host selector dropdown with local host listed first
- Auto-loads compose-farm config file on page load

Changes:
- Add /console route and console.html template
- Add /ws/shell/{host} WebSocket endpoint for shell sessions
- Add /api/console/file GET/PUT endpoints for remote file operations
- Update sidebar to include Console navigation link
2025-12-18 10:02:54 -08:00
Bas Nijholt
98bfb1bf6d fix(executor): Disable SSH host key checking in raw mode (#18)
Add SSH options to match asyncssh behavior:
- StrictHostKeyChecking=no
- UserKnownHostsFile=/dev/null
- LogLevel=ERROR (suppress warnings)
- Use -tt to force TTY allocation without stdin TTY

Fixes "Host key verification failed" errors when running from web UI.
2025-12-18 09:59:22 -08:00
Bas Nijholt
3c1cc79684 refactor(docker): Use multi-stage build to reduce image size
Reduces image size from 880MB to 139MB (84% smaller) by:
- Building with uv in a separate stage
- Using python:3.14-alpine as runtime base (no uv overhead)
- Pre-compiling bytecode with --compile-bytecode
- Copying only the tool virtualenv and bin symlinks to runtime
2025-12-18 00:58:06 -08:00
Bas Nijholt
12bbcee374 feat(web): Handle invalid config gracefully with error banner (#16) 2025-12-18 00:40:19 -08:00
Bas Nijholt
6e73ae0157 feat(web): Add command palette with Cmd+K (#15) 2025-12-18 00:12:38 -08:00
Bas Nijholt
d90b951a8c feat(web): Vendor CDN assets at build time for offline use
Add a Hatch build hook that downloads JS/CSS dependencies during wheel
builds and rewrites base.html to use local paths. This allows the web UI
to work in environments without internet access.

- Add data-vendor attributes to base.html for declarative asset mapping
- Download htmx, tailwind, daisyui, and xterm.js during build
- Bundle LICENSES.txt with attribution for vendored dependencies
- Source HTML keeps CDN links for development; wheel has local paths
2025-12-17 23:45:06 -08:00
Bas Nijholt
14558131ed feat(web): Add search and host filter to sidebar and services list
- Add search input and host dropdown to sidebar for filtering services
- Add search input and host dropdown to Services by Host section
- Show dynamic service count in sidebar that updates with filter
- Multi-host services appear when any host is selected
2025-12-17 23:35:18 -08:00
Bas Nijholt
a422363337 Update uv.lock 2025-12-17 23:21:37 -08:00
Bas Nijholt
1278d0b3af fix(web): Remove config caching so changes are detected immediately
Config was cached with @lru_cache, causing the web UI to show stale
sync status after external config file edits.
2025-12-17 23:17:25 -08:00
Bas Nijholt
c8ab6271a8 feat(web): Add rainbow hover effect to Compose Farm headers
Animated gradient appears on hover for both sidebar and mobile navbar headers.
2025-12-17 23:09:57 -08:00
Bas Nijholt
957e828a5b feat(web): Add Lucide icons to web UI (#14) 2025-12-17 23:04:53 -08:00
Bas Nijholt
5afda8cbb2 Add web UI with FastAPI + HTMX + xterm.js (#13) 2025-12-17 22:52:40 -08:00
Bas Nijholt
1bbf324f1e Validate services exist in config with friendly error
Show a clean error message instead of a traceback when a service
is not found in config. Includes a hint about adding to config.
2025-12-17 11:04:48 -08:00
Bas Nijholt
1be5b987a2 Support "." as shorthand for current directory service name
Running `cf up .` now resolves to the current directory name,
allowing quick operations when inside a service directory.
2025-12-17 10:57:48 -08:00
Bas Nijholt
6b684b19f2 Run startup time test 6 times 2025-12-17 09:08:45 -08:00
Bas Nijholt
4a37982e30 Cleanup 2025-12-17 09:07:52 -08:00
Bas Nijholt
55cb44e0e7 Drop service discovery mention 2025-12-17 09:07:30 -08:00
Bas Nijholt
5c242d08bf Add cf apply to post 2025-12-17 09:06:59 -08:00
Bas Nijholt
5bf65d3849 Raise Linux CLI startup threshold to 0.25s for CI headroom 2025-12-17 09:05:53 -08:00
Bas Nijholt
21d5dfa175 Fix check-readme-commands hook to use uv run for CI compatibility 2025-12-17 08:58:45 -08:00
Bas Nijholt
e49ad29999 Use OS-specific thresholds for CLI startup test (Linux: 0.2s, macOS: 0.35s, Windows: 2s) 2025-12-17 08:57:50 -08:00
Bas Nijholt
cdbe74ed89 Return early from CLI startup test when under threshold 2025-12-17 08:56:35 -08:00
Bas Nijholt
129970379c Increase CLI startup threshold to 0.35s for macOS/Windows CI 2025-12-17 08:55:40 -08:00
Bas Nijholt
c5c47d14dd Add CLI startup time test to catch slow imports
Runs `cf --help` and fails if startup exceeds 200ms. Shows timing
info in CI logs on both pass and failure.
2025-12-17 08:53:32 -08:00
Bas Nijholt
95f19e7333 Add pre-commit hook to verify all CLI commands are documented in README
Extracts commands from the Typer app and checks each has a corresponding
--help section in the README. Runs when README.md or CLI files change.
2025-12-17 08:45:16 -08:00
Bas Nijholt
9c6edd3f18 refactor(docs): move reddit-post.md into docs folder 2025-12-17 08:45:16 -08:00
github-actions[bot]
bda9210354 Update README.md 2025-12-17 16:35:24 +00:00
Bas Nijholt
f57951e8dc Fix cf up -h output in README.md 2025-12-17 08:34:52 -08:00
basnijholt
ba8c04caf8 chore(docs): update TOC 2025-12-17 16:31:40 +00:00
Bas Nijholt
ff0658117d Add all --help outputs 2025-12-17 08:31:14 -08:00
Bas Nijholt
920b593d5f Fix mypy error: add type annotation for proc variable 2025-12-17 00:17:20 -08:00
Bas Nijholt
27d9b08ce2 Add -f shorthand for --full in apply command 2025-12-17 00:10:16 -08:00
Bas Nijholt
700cdacb4d Add 'a' alias for apply command (cf a = cf apply) 2025-12-17 00:09:45 -08:00
Bas Nijholt
3c7a532704 Add comments explaining lazy imports for startup performance 2025-12-17 00:08:28 -08:00
Bas Nijholt
6048f37ad5 Lazy import pydantic for faster CLI startup
- Create paths.py module with lightweight path utilities (no pydantic)
- Move Config imports to TYPE_CHECKING blocks in CLI modules
- Lazy import load_config only when needed

Combined with asyncssh lazy loading, cf --help now starts in ~150ms
instead of ~500ms (70% faster).
2025-12-17 00:07:15 -08:00
Bas Nijholt
f18952633f Lazy import asyncssh for faster CLI startup
Move asyncssh import inside _run_ssh_command() so it's only loaded
when actually executing SSH commands. This cuts CLI import time
from 414ms to 200ms (52% faster).

cf --help now starts in ~260ms instead of ~500ms.
2025-12-16 23:59:37 -08:00
Bas Nijholt
437257e631 Add cf config symlink command
Creates a symlink from ~/.config/compose-farm/compose-farm.yaml to a
local config file using absolute paths (avoids broken relative symlinks).

Usage:
  cf config symlink                     # Link to ./compose-farm.yaml
  cf config symlink /path/to/config.yaml  # Link to specific file

State files are automatically stored next to the actual config (not
the symlink) since config_path.resolve() follows symlinks.
2025-12-16 23:56:58 -08:00
Bas Nijholt
c720170f26 Add --full flag to apply command
When --full is passed, apply also runs 'docker compose up' on all
services (not just missing/migrating ones) to pick up any config
changes (compose file, .env, etc).

- cf apply          # Fast: state reconciliation only
- cf apply --full   # Thorough: also refresh all running services
2025-12-16 23:54:19 -08:00
Bas Nijholt
d9c03d6509 Feature apply as the hero command in README
- Update intro and NOTE block to lead with cf apply
- Rewrite "How It Works" to show declarative workflow first
- Move apply to top of command table (bolded)
- Reorder examples to show apply as "the main command"
- Update automation bullet to highlight one-command reconciliation
2025-12-16 23:49:49 -08:00
Bas Nijholt
3b7066711f Merge pull request #12 from basnijholt/feature/orphaned-services
Add apply command and refactor CLI for clearer UX
2025-12-16 23:34:34 -08:00
Bas Nijholt
6a630c40a1 Update apply command description to include starting missing services 2025-12-16 23:32:27 -08:00
Bas Nijholt
9f9c042b66 Remove up --migrate flag in favor of apply
Simplifies CLI by having one clear reconciliation command:
- cf up <service>  = start specific services (auto-migrates if needed)
- cf apply         = full reconcile (stop orphans + migrate + start missing)

The --migrate flag was redundant with 'apply --no-orphans'.
2025-12-16 23:27:19 -08:00
github-actions[bot]
2a6d7d0b85 Update README.md 2025-12-17 07:21:38 +00:00
Bas Nijholt
6d813ccd84 Merge af9c760fb8 into affed2edcf 2025-12-17 07:21:23 +00:00
Bas Nijholt
af9c760fb8 Add missing service detection to apply command
Previously, apply only handled:
1. Stopping orphans (in state, not in config)
2. Migrating services (in state, wrong host)

Now it also handles:
3. Starting missing services (in config, not in state)

This fixes the case where a service was stopped as an orphan, then
re-added to config - apply would say "nothing to do" instead of
starting it.

Added get_services_not_in_state() to state.py and updated tests.
2025-12-16 23:21:09 -08:00
Bas Nijholt
90656b05e3 Add tests for apply command and down --orphaned flag
Tests cover:
- apply: nothing to do, dry-run preview, migrations, orphan cleanup, --no-orphans
- down --orphaned: no orphans, stops services, error on invalid combinations

Lifecycle.py coverage improved from 20% to 61%.
2025-12-16 23:15:46 -08:00
github-actions[bot]
d7a3d4e8c7 Update README.md 2025-12-17 07:10:52 +00:00
Bas Nijholt
35f0b8bf99 Merge be6b391121 into affed2edcf 2025-12-16 23:10:36 -08:00
Bas Nijholt
be6b391121 Refactor CLI commands for clearer UX
Separate "read state from reality" from "write config to reality":
- Rename `sync` to `refresh` (updates local state from running services)
- Add `apply` command (makes reality match config: migrate + stop orphans)
- Add `down --orphaned` flag (stops services removed from config)
- Modify `up --migrate` to only handle migrations (not orphans)

The new mental model:
- `refresh` = Reality → State (discover what's running)
- `apply` = Config → Reality (reconcile: migrate services + stop orphans)

Also extract private helper functions for reporting to match codebase style.
2025-12-16 23:06:42 -08:00
Bas Nijholt
7f56ba6a41 Add orphaned service detection and cleanup
When services are removed from config but still tracked in state,
`cf up --migrate` now stops them automatically. This makes the
config truly declarative - comment out a service, run migrate,
and it stops.

Changes:
- Add get_orphaned_services() to state.py for detecting orphans
- Add stop_orphaned_services() to operations.py for cleanup
- Update lifecycle.py to call stop_orphaned_services on --migrate
- Refactor _report_orphaned_services to use shared function
- Rename "missing_from_config" to "unmanaged" for clarity
- Add tests for get_orphaned_services
- Only remove from state on successful down (not on failure)
2025-12-16 22:53:26 -08:00
Bas Nijholt
4b3d7a861e Fix migration and update for services with buildable images
Use `pull --ignore-buildable` to skip images that have `build:` defined
in the compose file, preventing pull failures for locally-built images
like gitea-runner-custom. The build step then handles these images.
2025-12-16 19:42:24 -08:00
Bas Nijholt
affed2edcf Refactor operations.py into smaller helpers
- Add PreflightResult NamedTuple with .ok property
- Extract _run_compose_step to handle raw output and interrupts
- Extract _up_single_service for single-host migration logic
- Deduplicate pull/build handling in _migrate_service
- Add was_running check to only rollback if service was running
2025-12-16 17:08:25 -08:00
Bas Nijholt
34642e8b8e Rollback to old host when migration up fails
When up fails on the target host after migration has already stopped
the service on the old host, attempt to restart on the old host as a
rollback. This prevents services from being left down after a failed
migration attempt.
2025-12-16 16:59:09 -08:00
Bas Nijholt
4c8b6c5209 Add init-network hint when network is missing 2025-12-16 16:22:06 -08:00
Bas Nijholt
2b38ed28c0 Skip traefik regeneration when all services fail
Don't update traefik config if all services in the operation failed.
This prevents adding routes for services that aren't actually running.
2025-12-16 16:21:09 -08:00
Bas Nijholt
26b57895ce Clean up orphaned containers when migration up fails
If `up` fails after migration (when we've already run `down` on the
old host), run `down` on the target host to clean up any containers
that were created but couldn't start (e.g., due to missing devices).

This prevents orphaned containers from lingering on the failed host.
2025-12-16 16:16:09 -08:00
Bas Nijholt
367da13fae Fix path existence check for permission denied
Use stat instead of test -e to distinguish "no such file" from
"permission denied". If stat fails with permission denied, the
path exists (just not accessible), so report it as existing.

Fixes false "missing path" errors for directories with restricted
permissions like /mnt/data/immich/library.
2025-12-16 15:08:38 -08:00
Bas Nijholt
d6ecd42559 Consolidate service requirement checks into shared function
Move check_service_requirements() to operations.py as a public function
that verifies paths, networks, and devices exist on a target host. Both
CLI check command and pre-flight migration checks now use this shared
function, eliminating duplication. Also adds device checking to the
check command output.
2025-12-16 14:53:59 -08:00
Bas Nijholt
233c33fa52 Add device checking to pre-flight migration checks
Services with devices: mappings (e.g., /dev/dri for GPU acceleration)
now have those devices verified on the target host before migration.
This prevents the scenario where a service is stopped on the old host
but fails to start on the new host due to missing devices.

Adds parse_devices() to extract host device paths from compose files.
2025-12-16 14:35:52 -08:00
Bas Nijholt
43974c5743 Abort on first Ctrl+C during migrations
Detect when SSH subprocess is killed by signal (exit code < 0 or 255)
and treat it as an interrupt. This allows single Ctrl+C to abort the
entire operation instead of requiring two presses.
2025-12-16 14:31:33 -08:00
Bas Nijholt
cf94a62f37 docs: Clarify pull/build comments in migration 2025-12-16 14:26:48 -08:00
Bas Nijholt
81b4074827 Pre-build Dockerfile services during migration
After pulling images, also run build for services with Dockerfiles.
This ensures build-based services have their images ready before
stopping the old service, minimizing downtime.

If build fails, abort the migration and leave the service running
on the old host.

Extract _migrate_service helper to reduce function complexity.
2025-12-16 14:17:19 -08:00
Bas Nijholt
455657c8df Abort migration if pre-pull fails
If pulling images on the target host fails (e.g., rate limit),
abort the migration and leave the service running on the old host.
This prevents downtime when Docker Hub rate limits are hit.
2025-12-16 14:14:35 -08:00
Bas Nijholt
ee5a92788a Pre-pull images during migration to reduce downtime
When migrating a service to a new host, pull images on the target
host before stopping the service on the old host. This minimizes
downtime since images are cached when the up command runs.

Migration flow:
1. Pull images on new host (service still running on old)
2. Down on old host
3. Up on new host (fast, images already pulled)
2025-12-16 14:12:53 -08:00
Bas Nijholt
2ba396a419 docs: Move Compose Farm to first column in comparison table 2025-12-16 13:48:40 -08:00
Bas Nijholt
7144d58160 build: Include LICENSE file in package distribution 2025-12-16 13:37:15 -08:00
Bas Nijholt
279fa2e5ef Create LICENSE 2025-12-16 13:36:35 -08:00
Bas Nijholt
dbe0b8b597 docs: Add app.py to CLAUDE.md architecture diagram 2025-12-16 13:14:51 -08:00
Bas Nijholt
b7315d255a refactor: Split CLI into modular subpackage (#11) 2025-12-16 13:08:08 -08:00
renovate[bot]
f003d2931f ⬆️ Update actions/checkout action to v6 (#5)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-16 12:19:45 -08:00
renovate[bot]
6f7c557065 ⬆️ Update actions/setup-python action to v6 (#6)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-16 12:18:34 -08:00
renovate[bot]
ecb6ee46b1 ⬆️ Update astral-sh/setup-uv action to v7 (#8)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-16 12:18:28 -08:00
renovate[bot]
354967010f ⬆️ Update redis Docker tag to v8 (#9)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2025-12-16 12:18:22 -08:00
github-actions[bot]
57122f31a3 Update README.md 2025-12-16 20:01:03 +00:00
Bas Nijholt
cbbcec0d14 Add config subcommand for managing configuration files (#10) 2025-12-16 12:00:44 -08:00
Bas Nijholt
de38c35b8a docs: Add one-liner showing manual equivalent 2025-12-16 11:19:56 -08:00
github-actions[bot]
def996ddf4 Update README.md 2025-12-16 19:14:07 +00:00
Bas Nijholt
790e32e96b Fix test_load_config_not_found for CF_CONFIG env var 2025-12-16 11:13:44 -08:00
Bas Nijholt
fd75c4d87f Add CLI --help output to README 2025-12-16 11:12:43 -08:00
Bas Nijholt
411a99cbc4 Wait for PyPI propagation before Docker build
Also add Python 3.14 to classifiers.
2025-12-16 11:04:35 -08:00
Bas Nijholt
d2c6ab72b2 Add CF_CONFIG env var for simpler Docker workflow
Config search order is now:
1. --config CLI option
2. CF_CONFIG environment variable
3. ./compose-farm.yaml
4. ~/.config/compose-farm/compose-farm.yaml

Docker workflow simplified: mount compose_dir once, set CF_CONFIG
to config file within it. No more symlink issues or multiple mounts.
2025-12-16 10:12:55 -08:00
Bas Nijholt
3656584eda Friendly error when config path is a directory
Docker creates empty directories for missing file mounts,
causing confusing IsADirectoryError tracebacks. Now shows
a clear message explaining the likely cause.
2025-12-16 09:49:40 -08:00
Bas Nijholt
8be370098d Use env vars for docker-compose.yml mounts
- CF_CONFIG_DIR: config directory (default: ~/.config/compose-farm)
- CF_COMPOSE_DIR: compose directory (default: /opt/compose)

Mounts preserve paths so compose_dir in config works correctly.
2025-12-16 09:49:34 -08:00
Bas Nijholt
45057cb6df feat: Add docker-compose.yml for easier Docker usage
Example compose file that mounts SSH agent and config.
Users uncomment the compose_dir mount for their setup.
2025-12-16 09:40:18 -08:00
Bas Nijholt
3f24484d60 fix: Fix VERSION expansion in Dockerfile 2025-12-16 09:24:46 -08:00
111 changed files with 15432 additions and 1644 deletions

2
.gitattributes vendored Normal file
View File

@@ -0,0 +1,2 @@
*.gif filter=lfs diff=lfs merge=lfs -text
*.webm filter=lfs diff=lfs merge=lfs -text

88
.github/check_readme_commands.py vendored Executable file
View File

@@ -0,0 +1,88 @@
#!/usr/bin/env python3
"""Check that all CLI commands are documented in the README."""
from __future__ import annotations
import re
import sys
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
import typer
from compose_farm.cli import app
def get_all_commands(typer_app: typer.Typer, prefix: str = "cf") -> set[str]:
"""Extract all command names from a Typer app, including nested subcommands."""
commands = set()
# Get registered commands (skip hidden ones like aliases)
for command in typer_app.registered_commands:
if command.hidden:
continue
name = command.name
if not name and command.callback:
name = command.callback.__name__
if name:
commands.add(f"{prefix} {name}")
# Get registered sub-apps (like 'config')
for group in typer_app.registered_groups:
sub_app = group.typer_instance
sub_name = group.name
if sub_app and sub_name:
commands.add(f"{prefix} {sub_name}")
# Don't recurse into subcommands - we only document the top-level subcommand
return commands
def get_documented_commands(readme_path: Path) -> set[str]:
"""Extract commands documented in README from help output sections."""
content = readme_path.read_text()
# Match patterns like: <code>cf command --help</code>
pattern = r"<code>(cf\s+[\w-]+)\s+--help</code>"
matches = re.findall(pattern, content)
return set(matches)
def main() -> int:
"""Check that all CLI commands are documented in the README."""
readme_path = Path(__file__).parent.parent / "README.md"
if not readme_path.exists():
print(f"ERROR: README.md not found at {readme_path}")
return 1
cli_commands = get_all_commands(app)
documented_commands = get_documented_commands(readme_path)
# Also check for the main 'cf' help
if "<code>cf --help</code>" in readme_path.read_text():
documented_commands.add("cf")
cli_commands.add("cf")
missing = cli_commands - documented_commands
extra = documented_commands - cli_commands
if missing or extra:
if missing:
print("ERROR: Commands missing from README --help documentation:")
for cmd in sorted(missing):
print(f" - {cmd}")
if extra:
print("WARNING: Commands documented but not in CLI:")
for cmd in sorted(extra):
print(f" - {cmd}")
return 1
print(f"✓ All {len(cli_commands)} commands documented in README")
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -16,10 +16,10 @@ jobs:
python-version: ["3.11", "3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
- name: Set up Python ${{ matrix.python-version }}
run: uv python install ${{ matrix.python-version }}
@@ -27,8 +27,8 @@ jobs:
- name: Install dependencies
run: uv sync --all-extras --dev
- name: Run tests
run: uv run pytest
- name: Run tests (excluding browser tests)
run: uv run pytest -m "not browser"
- name: Upload coverage reports to Codecov
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.13'
@@ -36,13 +36,33 @@ jobs:
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
browser-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Set up Python
run: uv python install 3.13
- name: Install dependencies
run: uv sync --all-extras --dev
- name: Install Playwright browsers
run: uv run playwright install chromium --with-deps
- name: Run browser tests
run: uv run pytest -m browser -v --no-cov
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
- name: Set up Python
run: uv python install 3.12

View File

@@ -25,7 +25,7 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v6
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
@@ -52,6 +52,22 @@ jobs:
fi
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Wait for PyPI
if: steps.version.outputs.version != ''
run: |
VERSION="${{ steps.version.outputs.version }}"
echo "Waiting for compose-farm==$VERSION on PyPI..."
for i in {1..30}; do
if curl -sf "https://pypi.org/pypi/compose-farm/$VERSION/json" > /dev/null; then
echo "✓ Version $VERSION available on PyPI"
exit 0
fi
echo "Attempt $i: not yet available, waiting 10s..."
sleep 10
done
echo "✗ Timeout waiting for PyPI"
exit 1
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5

58
.github/workflows/docs.yml vendored Normal file
View File

@@ -0,0 +1,58 @@
name: Docs
on:
push:
branches: [main]
paths:
- "docs/**"
- "zensical.toml"
- ".github/workflows/docs.yml"
workflow_dispatch:
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: "pages"
cancel-in-progress: false
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
lfs: true
- name: Install uv
uses: astral-sh/setup-uv@v4
- name: Set up Python
run: uv python install 3.12
- name: Install Zensical
run: uv tool install zensical
- name: Build docs
run: zensical build
- name: Setup Pages
uses: actions/configure-pages@v5
- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: "./site"
deploy:
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
runs-on: ubuntu-latest
needs: build
steps:
- name: Deploy to GitHub Pages
id: deployment
uses: actions/deploy-pages@v4

View File

@@ -13,9 +13,9 @@ jobs:
permissions:
id-token: write
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
- name: Build
run: uv build
- name: Publish package distributions to PyPI

View File

@@ -11,16 +11,16 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
uses: actions/checkout@v6
with:
persist-credentials: false
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
- name: Run markdown-code-runner
env:

2
.gitignore vendored
View File

@@ -42,3 +42,5 @@ htmlcov/
compose-farm.yaml
!examples/compose-farm.yaml
coverage.xml
.env
homepage/

View File

@@ -1,4 +1,13 @@
repos:
- repo: local
hooks:
- id: check-readme-commands
name: Check README documents all CLI commands
entry: uv run python .github/check_readme_commands.py
language: system
files: ^(README\.md|src/compose_farm/cli/.*)$
pass_filenames: false
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:

94
.prompts/docs-review.md Normal file
View File

@@ -0,0 +1,94 @@
Review all documentation in this repository for accuracy, completeness, and consistency. Cross-reference documentation against the actual codebase to identify issues.
## Scope
Review all documentation files:
- docs/*.md (primary documentation)
- README.md (repository landing page)
- CLAUDE.md (development guidelines)
- examples/README.md (example configurations)
## Review Checklist
### 1. Command Documentation
For each documented command, verify against the CLI source code:
- Command exists in codebase
- All options are documented with correct names, types, and defaults
- Short options (-x) match long options (--xxx)
- Examples would work as written
- Check for undocumented commands or options
Run `--help` for each command to verify.
### 2. Configuration Documentation
Verify against Pydantic models in the config module:
- All config keys are documented
- Types match Pydantic field types
- Required vs optional fields are correct
- Default values are accurate
- Config file search order matches code
- Example YAML is valid and uses current schema
### 3. Architecture Documentation
Verify against actual directory structure:
- File paths match actual source code location
- All modules listed actually exist
- No modules are missing from the list
- Component descriptions match code functionality
- CLI module list includes all command files
### 4. State and Data Files
Verify against state and path modules:
- State file name and location are correct
- State file format matches actual structure
- Log file name and location are correct
- What triggers state/log updates is accurate
### 5. Installation Documentation
Verify against pyproject.toml:
- Python version requirement matches requires-python
- Package name is correct
- Optional dependencies are documented
- CLI entry points are mentioned
- Installation methods work as documented
### 6. Feature Claims
For each claimed feature, verify it exists and works as described.
### 7. Cross-Reference Consistency
Check for conflicts between documentation files:
- README vs docs/index.md (should be consistent)
- CLAUDE.md vs actual code structure
- Command tables match across files
- Config examples are consistent
## Output Format
Provide findings in these categories:
1. **Critical Issues**: Incorrect information that would cause user problems
2. **Inaccuracies**: Technical errors, wrong defaults, incorrect paths
3. **Missing Documentation**: Features/commands that exist but aren't documented
4. **Outdated Content**: Information that was once true but no longer is
5. **Inconsistencies**: Conflicts between different documentation files
6. **Minor Issues**: Typos, formatting, unclear wording
7. **Verified Accurate**: Sections confirmed to be correct
For each issue, include:
- File path and line number (if applicable)
- What the documentation says
- What the code actually does
- Suggested fix

View File

@@ -9,18 +9,40 @@
## Architecture
```
compose_farm/
├── cli.py # Typer commands (thin layer, delegates to operations)
├── config.py # Pydantic models, YAML loading
├── compose.py # Compose file parsing (.env, ports, volumes, networks)
├── console.py # Shared Rich console instances
├── executor.py # SSH/local command execution, streaming output
├── operations.py # Business logic (up, migrate, discover, preflight checks)
├── state.py # Deployment state tracking (which service on which host)
├── logs.py # Image digest snapshots (dockerfarm-log.toml)
└── traefik.py # Traefik file-provider config generation from labels
src/compose_farm/
├── cli/ # CLI subpackage
│ ├── __init__.py # Imports modules to trigger command registration
│ ├── app.py # Shared Typer app instance, version callback
├── common.py # Shared helpers, options, progress bar utilities
│ ├── config.py # Config subcommand (init, show, path, validate, edit, symlink)
│ ├── lifecycle.py # up, down, pull, restart, update, apply commands
│ ├── management.py # refresh, check, init-network, traefik-file commands
│ ├── monitoring.py # logs, ps, stats commands
│ ├── ssh.py # SSH key management (setup, status, keygen)
│ └── web.py # Web UI server command
├── config.py # Pydantic models, YAML loading
├── compose.py # Compose file parsing (.env, ports, volumes, networks)
├── console.py # Shared Rich console instances
├── executor.py # SSH/local command execution, streaming output
├── operations.py # Business logic (up, migrate, discover, preflight checks)
├── state.py # Deployment state tracking (which service on which host)
├── logs.py # Image digest snapshots (dockerfarm-log.toml)
├── paths.py # Path utilities, config file discovery
├── ssh_keys.py # SSH key path constants and utilities
├── traefik.py # Traefik file-provider config generation from labels
└── web/ # Web UI (FastAPI + HTMX)
```
## Web UI Icons
Icons use [Lucide](https://lucide.dev/). Add new icons as macros in `web/templates/partials/icons.html` by copying SVG paths from their site. The `action_btn`, `stat_card`, and `collapse` macros in `components.html` accept an optional `icon` parameter.
## HTMX Patterns
- **Multi-element refresh**: Use custom events, not `hx-swap-oob`. Elements have `hx-trigger="cf:refresh from:body"` and JS calls `document.body.dispatchEvent(new CustomEvent('cf:refresh'))`. Simpler to debug/test.
- **SPA navigation**: Sidebar uses `hx-boost="true"` to AJAX-ify links.
- **Attribute inheritance**: Set `hx-target`/`hx-swap` on parent elements.
## Key Design Decisions
1. **Hybrid SSH approach**: asyncssh for parallel streaming with prefixes; native `ssh -t` for raw mode (progress bars)
@@ -32,6 +54,27 @@ compose_farm/
7. **State tracking**: Tracks where services are deployed for auto-migration
8. **Pre-flight checks**: Verifies NFS mounts and Docker networks exist before starting/migrating
## Code Style
- **Imports at top level**: Never add imports inside functions unless they are explicitly marked with `# noqa: PLC0415` and a comment explaining it speeds up CLI startup. Heavy modules like `pydantic`, `yaml`, and `rich.table` are lazily imported to keep `cf --help` fast.
## Testing
Run tests with `uv run pytest`. Browser tests require Chromium (system-installed or via `playwright install chromium`):
```bash
# Unit tests only (skip browser tests, can parallelize)
uv run pytest -m "not browser" -n auto
# Browser tests only (run sequentially, no coverage)
uv run pytest -m browser --no-cov
# All tests
uv run pytest --no-cov
```
Browser tests are marked with `@pytest.mark.browser`. They use Playwright to test HTMX behavior, JavaScript functionality (sidebar filter, command palette, terminals), and content stability during navigation. Run sequentially (no `-n`) to avoid resource contention.
## Communication Notes
- Clarify ambiguous wording (e.g., homophones like "right"/"write", "their"/"there").
@@ -42,21 +85,48 @@ compose_farm/
- **NEVER merge anything into main.** Always commit directly or use fast-forward/rebase.
- Never force push.
## Pull Requests
- Never include unchecked checklists (e.g., `- [ ] ...`) in PR descriptions. Either omit the checklist or use checked items.
- **NEVER run `gh pr merge`**. PRs are merged via the GitHub UI, not the CLI.
## Releases
Use `gh release create` to create releases. The tag is created automatically.
```bash
# Check current version
git tag --sort=-v:refname | head -1
# Create release (minor version bump: v0.21.1 -> v0.22.0)
gh release create v0.22.0 --title "v0.22.0" --notes "release notes here"
```
Versioning:
- **Patch** (v0.21.0 → v0.21.1): Bug fixes
- **Minor** (v0.21.1 → v0.22.0): New features, non-breaking changes
Write release notes manually describing what changed. Group by features and bug fixes.
## Commands Quick Reference
CLI available as `cf` or `compose-farm`.
| Command | Description |
|---------|-------------|
| `up` | Start services (`docker compose up -d`), auto-migrates if host changed. Use `--migrate` for auto-detection |
| `down` | Stop services (`docker compose down`) |
| `up` | Start services (`docker compose up -d`), auto-migrates if host changed |
| `down` | Stop services (`docker compose down`). Use `--orphaned` to stop services removed from config |
| `pull` | Pull latest images |
| `restart` | `down` + `up -d` |
| `update` | `pull` + `down` + `up -d` |
| `update` | `pull` + `build` + `down` + `up -d` |
| `apply` | Make reality match config: migrate services + stop orphans. Use `--dry-run` to preview |
| `logs` | Show service logs |
| `ps` | Show status of all services |
| `stats` | Show overview (hosts, services, pending migrations; `--live` for container counts) |
| `sync` | Discover running services, update state, capture image digests |
| `refresh` | Update state from reality: discover running services, capture image digests |
| `check` | Validate config, traefik labels, mounts, networks; show host compatibility |
| `init-network` | Create Docker network on hosts with consistent subnet/gateway |
| `traefik-file` | Generate Traefik file-provider config from compose labels |
| `config` | Manage config files (init, show, path, validate, edit, symlink) |
| `ssh` | Manage SSH keys (setup, status, keygen) |
| `web` | Start web UI server |

View File

@@ -1,16 +1,20 @@
# syntax=docker/dockerfile:1
FROM ghcr.io/astral-sh/uv:python3.14-alpine
# Install SSH client (required for remote host connections)
# Build stage - install with uv
FROM ghcr.io/astral-sh/uv:python3.14-alpine AS builder
ARG VERSION
RUN uv tool install --compile-bytecode "compose-farm[web]${VERSION:+==$VERSION}"
# Runtime stage - minimal image without uv
FROM python:3.14-alpine
# Install only runtime requirements
RUN apk add --no-cache openssh-client
# Install compose-farm from PyPI
ARG VERSION
RUN uv tool install ${VERSION:+compose-farm==$VERSION} ${VERSION:-compose-farm}
# Copy installed tool virtualenv and bin symlinks from builder
COPY --from=builder /root/.local/share/uv/tools/compose-farm /root/.local/share/uv/tools/compose-farm
COPY --from=builder /usr/local/bin/cf /usr/local/bin/compose-farm /usr/local/bin/
# Add uv tool bin to PATH
ENV PATH="/root/.local/bin:$PATH"
# Default entrypoint
ENTRYPOINT ["cf"]
CMD ["--help"]

21
LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Bas Nijholt
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

794
README.md
View File

@@ -10,7 +10,7 @@
A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.
> [!NOTE]
> Run `docker compose` commands across multiple hosts via SSH. One YAML maps services to hosts. Change the mapping, run `up`, and it auto-migrates. No Kubernetes, no Swarm, no magic.
> Run `docker compose` commands across multiple hosts via SSH. One YAML maps services to hosts. Run `cf apply` and reality matches your config—services start, migrate, or stop as needed. No Kubernetes, no Swarm, no magic.
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
@@ -23,9 +23,14 @@ A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.
- [Best practices](#best-practices)
- [What Compose Farm doesn't do](#what-compose-farm-doesnt-do)
- [Installation](#installation)
- [SSH Authentication](#ssh-authentication)
- [SSH Agent (default)](#ssh-agent-default)
- [Dedicated SSH Key (recommended for Docker/Web UI)](#dedicated-ssh-key-recommended-for-dockerweb-ui)
- [Configuration](#configuration)
- [Multi-Host Services](#multi-host-services)
- [Config Command](#config-command)
- [Usage](#usage)
- [CLI `--help` Output](#cli---help-output)
- [Auto-Migration](#auto-migration)
- [Traefik Multihost Ingress (File Provider)](#traefik-multihost-ingress-file-provider)
- [Comparison with Alternatives](#comparison-with-alternatives)
@@ -42,7 +47,7 @@ I used to run 100+ Docker Compose stacks on a single machine that kept running o
Both require changes to your compose files. **Compose Farm requires zero changes**—your existing `docker-compose.yml` files work as-is.
I also wanted a declarative setup—one config file that defines where everything runs. Change the config, run `up`, and services migrate automatically. See [Comparison with Alternatives](#comparison-with-alternatives) for how this compares to other approaches.
I also wanted a declarative setup—one config file that defines where everything runs. Change the config, run `cf apply`, and everything reconciles—services start, migrate, or stop as needed. See [Comparison with Alternatives](#comparison-with-alternatives) for how this compares to other approaches.
<p align="center">
<a href="https://xkcd.com/927/">
@@ -55,18 +60,26 @@ Before you say it—no, this is not a new standard. I changed nothing about my e
Compose Farm just automates what you'd do by hand:
- Runs `docker compose` commands over SSH
- Tracks which service runs on which host
- Auto-migrates services when you change the host assignment
- **One command (`cf apply`) to reconcile everything**—start missing services, migrate moved ones, stop removed ones
- Generates Traefik file-provider config for cross-host routing
**It's a convenience wrapper, not a new paradigm.**
## How It Works
1. You run `cf up plex`
2. Compose Farm looks up which host runs `plex` (e.g., `server-1`)
3. It SSHs to `server-1` (or runs locally if `localhost`)
4. It executes `docker compose -f /opt/compose/plex/docker-compose.yml up -d`
5. Output is streamed back with `[plex]` prefix
**The declarative way** — run `cf apply` and reality matches your config:
1. Compose Farm compares your config to what's actually running
2. Services in config but not running? **Starts them**
3. Services on the wrong host? **Migrates them** (stops on old host, starts on new)
4. Services running but removed from config? **Stops them**
**Under the hood** — each service operation is just SSH + docker compose:
1. Look up which host runs the service (e.g., `plex``server-1`)
2. SSH to `server-1` (or run locally if `localhost`)
3. Execute `docker compose -f /opt/compose/plex/docker-compose.yml up -d`
4. Stream output back with `[plex]` prefix
That's it. No orchestration, no service discovery, no magic.
@@ -127,13 +140,22 @@ If you need containers on different hosts to communicate seamlessly, you need Do
## Installation
```bash
# One-liner (installs uv if needed)
curl -fsSL https://raw.githubusercontent.com/basnijholt/compose-farm/main/bootstrap.sh | sh
# Or if you already have uv/pip
uv tool install compose-farm
# or
pip install compose-farm
```
<details><summary>🐳 Docker</summary>
Using the provided `docker-compose.yml`:
```bash
docker compose run --rm cf up --all
```
Or directly:
```bash
docker run --rm \
-v $SSH_AUTH_SOCK:/ssh-agent -e SSH_AUTH_SOCK=/ssh-agent \
@@ -141,11 +163,62 @@ docker run --rm \
ghcr.io/basnijholt/compose-farm up --all
```
Or create an alias:
</details>
## SSH Authentication
Compose Farm uses SSH to run commands on remote hosts. There are two authentication methods:
### SSH Agent (default)
Works out of the box if you have an SSH agent running with your keys loaded:
```bash
alias cf='docker run --rm -v $SSH_AUTH_SOCK:/ssh-agent -e SSH_AUTH_SOCK=/ssh-agent -v ./compose-farm.yaml:/root/.config/compose-farm/compose-farm.yaml:ro ghcr.io/basnijholt/compose-farm'
# Verify your agent has keys
ssh-add -l
# Run compose-farm commands
cf up --all
```
### Dedicated SSH Key (recommended for Docker/Web UI)
When running compose-farm in Docker, the SSH agent connection can be lost (e.g., after container restart). The `cf ssh` command sets up a dedicated key that persists:
```bash
# Generate key and copy to all configured hosts
cf ssh setup
# Check status
cf ssh status
```
This creates `~/.ssh/compose-farm/id_ed25519` (ED25519, no passphrase) and copies the public key to each host's `authorized_keys`. Compose Farm tries the SSH agent first, then falls back to this key.
<details><summary>🐳 Docker volume options for SSH keys</summary>
When running in Docker, mount a volume to persist the SSH keys. Choose ONE option and use it for both `cf` and `web` services:
**Option 1: Host path (default)** - keys at `~/.ssh/compose-farm/id_ed25519`
```yaml
volumes:
- ~/.ssh/compose-farm:/root/.ssh
```
**Option 2: Named volume** - managed by Docker
```yaml
volumes:
- cf-ssh:/root/.ssh
```
Run setup once after starting the container (while the SSH agent still works):
```bash
docker compose exec web cf ssh setup
```
The keys will persist across restarts.
</details>
## Configuration
@@ -206,12 +279,27 @@ When you run `cf up autokuma`, it starts the service on all hosts in parallel. M
- Show output with `[service@host]` prefix for each host
- Track all running hosts in state
### Config Command
Compose Farm includes a `config` subcommand to help manage configuration files:
```bash
cf config init # Create a new config file with documented example
cf config show # Display current config with syntax highlighting
cf config path # Print the config file path (useful for scripting)
cf config validate # Validate config syntax and schema
cf config edit # Open config in $EDITOR
```
Use `cf config init` to get started with a fully documented template.
## Usage
The CLI is available as both `compose-farm` and the shorter `cf` alias.
| Command | Description |
|---------|-------------|
| **`cf apply`** | **Make reality match config (start + migrate + stop orphans)** |
| `cf up <svc>` | Start service (auto-migrates if host changed) |
| `cf down <svc>` | Stop service |
| `cf restart <svc>` | down + up |
@@ -219,21 +307,28 @@ The CLI is available as both `compose-farm` and the shorter `cf` alias.
| `cf pull <svc>` | Pull latest images |
| `cf logs -f <svc>` | Follow logs |
| `cf ps` | Show status of all services |
| `cf sync` | Discover running services + capture image digests |
| `cf refresh` | Update state from running services |
| `cf check` | Validate config, mounts, networks |
| `cf init-network` | Create Docker network on hosts |
| `cf traefik-file` | Generate Traefik file-provider config |
| `cf config <cmd>` | Manage config files (init, show, path, validate, edit, symlink) |
All commands support `--all` to operate on all services.
```bash
# Start services (auto-migrates if host changed in config)
cf up plex jellyfin
cf up --all
cf up --migrate # only services needing migration (state ≠ config)
Each command replaces: look up host → SSH → find compose file → run `ssh host "cd /opt/compose/plex && docker compose up -d"`.
# Stop services
cf down plex
```bash
# The main command: make reality match your config
cf apply # start missing + migrate + stop orphans
cf apply --dry-run # preview what would change
cf apply --no-orphans # skip stopping orphaned services
cf apply --full # also refresh all services (picks up config changes)
# Or operate on individual services
cf up plex jellyfin # start services (auto-migrates if host changed)
cf up --all
cf down plex # stop services
cf down --orphaned # stop services removed from config
# Pull latest images
cf pull --all
@@ -244,9 +339,9 @@ cf restart plex
# Update (pull + down + up) - the end-to-end update command
cf update --all
# Sync state with reality (discovers running services + captures image digests)
cf sync # updates state.yaml and dockerfarm-log.toml
cf sync --dry-run # preview without writing
# Update state from reality (discovers running services + captures digests)
cf refresh # updates state.yaml and dockerfarm-log.toml
cf refresh --dry-run # preview without writing
# Validate config, traefik labels, mounts, and networks
cf check # full validation (includes SSH checks)
@@ -265,6 +360,633 @@ cf logs -f plex # follow
cf ps
```
### CLI `--help` Output
Full `--help` output for each command. See the [Usage](#usage) table above for a quick overview.
<details>
<summary>See the output of <code>cf --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf [OPTIONS] COMMAND [ARGS]...
Compose Farm - run docker compose commands across multiple hosts
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --version -v Show version and exit │
│ --install-completion Install completion for the current shell. │
│ --show-completion Show completion for the current shell, to │
│ copy it or customize the installation. │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Lifecycle ──────────────────────────────────────────────────────────────────╮
│ up Start services (docker compose up -d). Auto-migrates if host │
│ changed. │
│ down Stop services (docker compose down). │
│ pull Pull latest images (docker compose pull). │
│ restart Restart services (down + up). │
│ update Update services (pull + build + down + up). │
│ apply Make reality match config (start, migrate, stop as needed). │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Configuration ──────────────────────────────────────────────────────────────╮
│ traefik-file Generate a Traefik file-provider fragment from compose │
│ Traefik labels. │
│ refresh Update local state from running services. │
│ check Validate configuration, traefik labels, mounts, and networks. │
│ init-network Create Docker network on hosts with consistent settings. │
│ config Manage compose-farm configuration files. │
│ ssh Manage SSH keys for passwordless authentication. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Monitoring ─────────────────────────────────────────────────────────────────╮
│ logs Show service logs. │
│ ps Show status of services. │
│ stats Show overview statistics for hosts and services. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Server ─────────────────────────────────────────────────────────────────────╮
│ web Start the web UI server. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
**Lifecycle**
<details>
<summary>See the output of <code>cf up --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf up --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf up [OPTIONS] [SERVICES]...
Start services (docker compose up -d). Auto-migrates if host changed.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --host -H TEXT Filter to services on this host │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf down --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf down --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf down [OPTIONS] [SERVICES]...
Stop services (docker compose down).
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --orphaned Stop orphaned services (in state but removed from │
│ config) │
│ --host -H TEXT Filter to services on this host │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf pull --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf pull --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf pull [OPTIONS] [SERVICES]...
Pull latest images (docker compose pull).
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf restart --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf restart --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf restart [OPTIONS] [SERVICES]...
Restart services (down + up).
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf update --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf update --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf update [OPTIONS] [SERVICES]...
Update services (pull + build + down + up).
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf apply --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf apply --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf apply [OPTIONS]
Make reality match config (start, migrate, stop as needed).
This is the "reconcile" command that ensures running services match your
config file. It will:
1. Stop orphaned services (in state but removed from config)
2. Migrate services on wrong host (host in state ≠ host in config)
3. Start missing services (in config but not in state)
Use --dry-run to preview changes before applying.
Use --no-orphans to only migrate/start without stopping orphaned services.
Use --full to also run 'up' on all services (picks up compose/env changes).
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --dry-run -n Show what would change without executing │
│ --no-orphans Only migrate, don't stop orphaned services │
│ --full -f Also run up on all services to apply config │
│ changes │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
**Configuration**
<details>
<summary>See the output of <code>cf traefik-file --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf traefik-file --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf traefik-file [OPTIONS] [SERVICES]...
Generate a Traefik file-provider fragment from compose Traefik labels.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --output -o PATH Write Traefik file-provider YAML to this path │
│ (stdout if omitted) │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf refresh --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf refresh --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf refresh [OPTIONS]
Update local state from running services.
Discovers which services are running on which hosts, updates the state
file, and captures image digests. This is a read operation - it updates
your local state to match reality, not the other way around.
Use 'cf apply' to make reality match your config (stop orphans, migrate).
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --config -c PATH Path to config file │
│ --log-path -l PATH Path to Dockerfarm TOML log │
│ --dry-run -n Show what would change without writing │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf check --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf check --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf check [OPTIONS] [SERVICES]...
Validate configuration, traefik labels, mounts, and networks.
Without arguments: validates all services against configured hosts.
With service arguments: validates specific services and shows host
compatibility.
Use --local to skip SSH-based checks for faster validation.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --local Skip SSH-based checks (faster) │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf init-network --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf init-network --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf init-network [OPTIONS] [HOSTS]...
Create Docker network on hosts with consistent settings.
Creates an external Docker network that services can use for cross-host
communication. Uses the same subnet/gateway on all hosts to ensure
consistent networking.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ hosts [HOSTS]... Hosts to create network on (default: all) │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --network -n TEXT Network name [default: mynetwork] │
│ --subnet -s TEXT Network subnet [default: 172.20.0.0/16] │
│ --gateway -g TEXT Network gateway [default: 172.20.0.1] │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf config --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf config --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf config [OPTIONS] COMMAND [ARGS]...
Manage compose-farm configuration files.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ init Create a new config file with documented example. │
│ edit Open the config file in your default editor. │
│ show Display the config file location and contents. │
│ path Print the config file path (useful for scripting). │
│ validate Validate the config file syntax and schema. │
│ symlink Create a symlink from the default config location to a config │
│ file. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf ssh --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf ssh --help -->
<!-- echo '```' -->
<!-- CODE:END -->
</details>
**Monitoring**
<details>
<summary>See the output of <code>cf logs --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf logs --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf logs [OPTIONS] [SERVICES]...
Show service logs.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --host -H TEXT Filter to services on this host │
│ --follow -f Follow logs │
│ --tail -n INTEGER Number of lines (default: 20 for --all, 100 │
│ otherwise) │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf ps --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf ps --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf ps [OPTIONS] [SERVICES]...
Show status of services.
Without arguments: shows all services (same as --all).
With service names: shows only those services.
With --host: shows services on that host.
╭─ Arguments ──────────────────────────────────────────────────────────────────╮
│ services [SERVICES]... Services to operate on │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --all -a Run on all services │
│ --host -H TEXT Filter to services on this host │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
<details>
<summary>See the output of <code>cf stats --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf stats --help -->
<!-- echo '```' -->
<!-- CODE:END -->
<!-- OUTPUT:START -->
<!-- ⚠️ This content is auto-generated by `markdown-code-runner`. -->
```yaml
Usage: cf stats [OPTIONS]
Show overview statistics for hosts and services.
Without --live: Shows config/state info (hosts, services, pending migrations).
With --live: Also queries Docker on each host for container counts.
╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --live -l Query Docker for live container stats │
│ --config -c PATH Path to config file │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────╯
```
<!-- OUTPUT:END -->
</details>
**Server**
<details>
<summary>See the output of <code>cf web --help</code></summary>
<!-- CODE:BASH:START -->
<!-- echo '```yaml' -->
<!-- export NO_COLOR=1 -->
<!-- export TERM=dumb -->
<!-- export TERMINAL_WIDTH=90 -->
<!-- cf web --help -->
<!-- echo '```' -->
<!-- CODE:END -->
</details>
### Auto-Migration
When you change a service's host assignment in config and run `up`, Compose Farm automatically:
@@ -273,7 +995,7 @@ When you change a service's host assignment in config and run `up`, Compose Farm
3. Runs `up -d` on the new host
4. Updates state tracking
Use `cf up --migrate` (or `-m`) to automatically find and migrate all services where the current state differs from config—no need to list them manually.
Use `cf apply` to automatically reconcile all services—it finds and migrates services on wrong hosts, stops orphaned services, and starts missing services.
```yaml
# Before: plex runs on server-1
@@ -285,6 +1007,14 @@ services:
plex: server-2 # Compose Farm will migrate automatically
```
**Orphaned services**: When you remove (or comment out) a service from config, it becomes "orphaned"—tracked in state but no longer in config. Use these commands to handle orphans:
- `cf apply` — Migrate services AND stop orphans (the full reconcile)
- `cf down --orphaned` — Only stop orphaned services
- `cf apply --dry-run` — Preview what would change before applying
This makes the config truly declarative: comment out a service, run `cf apply`, and it stops.
## Traefik Multihost Ingress (File Provider)
If you run a single Traefik instance on one "frontdoor" host and want it to route to
@@ -389,16 +1119,16 @@ Update your Traefik config to use directory watching instead of a single file:
There are many ways to run containers on multiple hosts. Here is where Compose Farm sits:
| | Docker Contexts | K8s / Swarm | Ansible / Terraform | Portainer / Coolify | Compose Farm |
| | Compose Farm | Docker Contexts | K8s / Swarm | Ansible / Terraform | Portainer / Coolify |
|---|:---:|:---:|:---:|:---:|:---:|
| No compose rewrites | ✅ | | | ✅ | ✅ |
| Version controlled | ✅ | ✅ | ✅ | | |
| State tracking | | | ✅ | ✅ | ✅ |
| Auto-migration | ❌ | ✅ | ❌ | ❌ | ✅ |
| Interactive CLI | | ❌ | ❌ | ❌ | |
| Parallel execution | | | ✅ | ✅ | ✅ |
| Agentless | ✅ | ❌ | ✅ | ❌ | ✅ |
| High availability | ❌ | | | ❌ | ❌ |
| No compose rewrites | ✅ | | | ✅ | ✅ |
| Version controlled | ✅ | ✅ | ✅ | | |
| State tracking | | | ✅ | ✅ | ✅ |
| Auto-migration | ✅ | ❌ | ✅ | ❌ | ❌ |
| Interactive CLI | | ❌ | ❌ | ❌ | |
| Parallel execution | | | ✅ | ✅ | ✅ |
| Agentless | ✅ | ✅ | ❌ | ✅ | ❌ |
| High availability | ❌ | | | ❌ | ❌ |
**Docker Contexts** — You can use `docker context create remote ssh://...` and `docker compose --context remote up`. But it's manual: you must remember which host runs which service, there's no global view, no parallel execution, and no auto-migration.

29
bootstrap.sh Executable file
View File

@@ -0,0 +1,29 @@
#!/bin/sh
# Compose Farm bootstrap script
# Usage: curl -fsSL https://raw.githubusercontent.com/basnijholt/compose-farm/main/bootstrap.sh | sh
#
# This script installs uv (if needed) and then installs compose-farm as a uv tool.
set -e
if ! command -v uv >/dev/null 2>&1; then
echo "uv is not installed. Installing..."
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "uv installation complete!"
echo ""
if [ -x ~/.local/bin/uv ]; then
~/.local/bin/uv tool install compose-farm
else
echo "Please restart your shell and run this script again"
echo ""
exit 0
fi
else
uv tool install compose-farm
fi
echo ""
echo "compose-farm is installed!"
echo "Run 'cf --help' to get started."
echo "If 'cf' is not found, restart your shell or run: source ~/.bashrc"

52
docker-compose.yml Normal file
View File

@@ -0,0 +1,52 @@
services:
cf:
image: ghcr.io/basnijholt/compose-farm:latest
volumes:
- ${SSH_AUTH_SOCK}:/ssh-agent:ro
# Compose directory (contains compose files AND compose-farm.yaml config)
- ${CF_COMPOSE_DIR:-/opt/stacks}:${CF_COMPOSE_DIR:-/opt/stacks}
# SSH keys for passwordless auth (generated by `cf ssh setup`)
# Choose ONE option below (use the same option for both cf and web services):
# Option 1: Host path (default) - keys at ~/.ssh/compose-farm/id_ed25519
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:/root/.ssh
# Option 2: Named volume - managed by Docker, shared between services
# - cf-ssh:/root/.ssh
environment:
- SSH_AUTH_SOCK=/ssh-agent
# Config file path (state stored alongside it)
- CF_CONFIG=${CF_COMPOSE_DIR:-/opt/stacks}/compose-farm.yaml
web:
image: ghcr.io/basnijholt/compose-farm:latest
restart: unless-stopped
command: web --host 0.0.0.0 --port 9000
volumes:
- ${SSH_AUTH_SOCK}:/ssh-agent:ro
- ${CF_COMPOSE_DIR:-/opt/stacks}:${CF_COMPOSE_DIR:-/opt/stacks}
# SSH keys - use the SAME option as cf service above
# Option 1: Host path (default)
- ${CF_SSH_DIR:-~/.ssh/compose-farm}:/root/.ssh
# Option 2: Named volume
# - cf-ssh:/root/.ssh
environment:
- SSH_AUTH_SOCK=/ssh-agent
- CF_CONFIG=${CF_COMPOSE_DIR:-/opt/stacks}/compose-farm.yaml
# Used to detect self-updates and run via SSH to survive container restart
- CF_WEB_SERVICE=compose-farm
labels:
- traefik.enable=true
- traefik.http.routers.compose-farm.rule=Host(`compose-farm.${DOMAIN}`)
- traefik.http.routers.compose-farm.entrypoints=websecure
- traefik.http.routers.compose-farm-local.rule=Host(`compose-farm.local`)
- traefik.http.routers.compose-farm-local.entrypoints=web
- traefik.http.services.compose-farm.loadbalancer.server.port=9000
networks:
- mynetwork
networks:
mynetwork:
external: true
volumes:
cf-ssh:
# Only used if Option 2 is selected above

1
docs/CNAME Normal file
View File

@@ -0,0 +1 @@
compose-farm.nijho.lt

346
docs/architecture.md Normal file
View File

@@ -0,0 +1,346 @@
---
icon: lucide/layers
---
# Architecture
This document explains how Compose Farm works under the hood.
## Design Philosophy
Compose Farm follows three core principles:
1. **KISS** - Keep it simple. It's a thin wrapper around `docker compose` over SSH.
2. **YAGNI** - No orchestration, no service discovery, no health checks until needed.
3. **Zero changes** - Your existing compose files work unchanged.
## High-Level Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Compose Farm CLI │
│ │
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │
│ │ Config │ │ State │ │Operations│ │ Executor │ │
│ │ Parser │ │ Tracker │ │ Logic │ │ (SSH/Local) │ │
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────────┬─────────┘ │
└───────┼─────────────┼─────────────┼─────────────────┼───────────┘
│ │ │ │
▼ ▼ ▼ ▼
┌───────────────────────────────────────────────────────────────┐
│ SSH / Local │
└───────────────────────────────────────────────────────────────┘
│ │
▼ ▼
┌───────────────┐ ┌───────────────┐
│ Host: nuc │ │ Host: hp │
│ │ │ │
│ docker compose│ │ docker compose│
│ up -d │ │ up -d │
└───────────────┘ └───────────────┘
```
## Core Components
### Configuration (`src/compose_farm/config.py`)
Pydantic models for YAML configuration:
- **Config** - Root configuration with compose_dir, hosts, services
- **HostConfig** - Host address and SSH user
- **ServiceConfig** - Service-to-host mappings
Key features:
- Validation with Pydantic
- Multi-host service expansion (`all` → list of hosts)
- YAML loading with sensible defaults
### State Tracking (`src/compose_farm/state.py`)
Tracks deployment state in `compose-farm-state.yaml` (stored alongside the config file):
```yaml
deployed:
plex: nuc
sonarr: nuc
```
Used for:
- Detecting migrations (service moved to different host)
- Identifying orphans (services removed from config)
- `cf ps` status display
### Operations (`src/compose_farm/operations.py`)
Business logic for service operations:
- **up** - Start service, handle migration if needed
- **down** - Stop service
- **preflight checks** - Verify mounts, networks exist before operations
- **discover** - Find running services on hosts
- **migrate** - Down on old host, up on new host
### Executor (`src/compose_farm/executor.py`)
SSH and local command execution:
- **Hybrid SSH approach**: asyncssh for parallel streaming, native `ssh -t` for raw mode
- **Parallel by default**: Multiple services via `asyncio.gather`
- **Streaming output**: Real-time stdout/stderr with `[service]` prefix
- **Local detection**: Skips SSH when target matches local machine IP
### CLI (`src/compose_farm/cli/`)
Typer-based CLI with subcommand modules:
```
cli/
├── app.py # Shared Typer app, version callback
├── common.py # Shared helpers, options, progress utilities
├── config.py # config subcommand (init, show, path, validate, edit, symlink)
├── lifecycle.py # up, down, pull, restart, update, apply
├── management.py # refresh, check, init-network, traefik-file
├── monitoring.py # logs, ps, stats
├── ssh.py # SSH key management (setup, status, keygen)
└── web.py # Web UI server command
```
## Command Flow
### cf up plex
```
1. Load configuration
└─► Parse compose-farm.yaml
└─► Validate service exists
2. Check state
└─► Load state.yaml
└─► Is plex already running?
└─► Is it on a different host? (migration needed)
3. Pre-flight checks
└─► SSH to target host
└─► Check compose file exists
└─► Check required mounts exist
└─► Check required networks exist
4. Execute migration (if needed)
└─► SSH to old host
└─► Run: docker compose down
5. Start service
└─► SSH to target host
└─► cd /opt/compose/plex
└─► Run: docker compose up -d
6. Update state
└─► Write new state to state.yaml
7. Generate Traefik config (if configured)
└─► Regenerate traefik file-provider
```
### cf apply
```
1. Load configuration and state
2. Compute diff
├─► Orphans: in state, not in config
├─► Migrations: in both, different host
└─► Missing: in config, not in state
3. Stop orphans
└─► For each orphan: cf down
4. Migrate services
└─► For each migration: down old, up new
5. Start missing
└─► For each missing: cf up
6. Update state
```
## SSH Execution
### Parallel Streaming (asyncssh)
For most operations, Compose Farm uses asyncssh:
```python
async def run_command(host, command):
async with asyncssh.connect(host) as conn:
result = await conn.run(command)
return result.stdout, result.stderr
```
Multiple services run concurrently via `asyncio.gather`.
### Raw Mode (native ssh)
For commands needing PTY (progress bars, interactive):
```bash
ssh -t user@host "docker compose pull"
```
### Local Detection
When target host IP matches local machine:
```python
if is_local(host_address):
# Run locally, no SSH
subprocess.run(command)
else:
# SSH to remote
ssh.run(command)
```
## State Management
### State File
Location: `compose-farm-state.yaml` (stored alongside the config file)
```yaml
deployed:
plex: nuc
sonarr: nuc
```
Image digests are stored separately in `dockerfarm-log.toml` (also in the config directory).
### State Transitions
```
Config Change State Change Action
─────────────────────────────────────────────────────
Add service Missing cf up
Remove service Orphaned cf down
Change host Migration down old, up new
No change No change none (or refresh)
```
### cf refresh
Syncs state with reality by querying Docker on each host:
```bash
docker ps --format '{{.Names}}'
```
Updates state.yaml to match what's actually running.
## Compose File Discovery
For each service, Compose Farm looks for compose files in:
```
{compose_dir}/{service}/
├── compose.yaml # preferred
├── compose.yml
├── docker-compose.yml
└── docker-compose.yaml
```
First match wins.
## Traefik Integration
### Label Extraction
Compose Farm parses Traefik labels from compose files:
```yaml
services:
plex:
labels:
- traefik.enable=true
- traefik.http.routers.plex.rule=Host(`plex.example.com`)
- traefik.http.services.plex.loadbalancer.server.port=32400
```
### File Provider Generation
Converts labels to Traefik file-provider YAML:
```yaml
http:
routers:
plex:
rule: Host(`plex.example.com`)
service: plex
services:
plex:
loadBalancer:
servers:
- url: http://192.168.1.10:32400
```
### Variable Resolution
Supports `${VAR}` and `${VAR:-default}` from:
1. Service's `.env` file
2. Current environment
## Error Handling
### Pre-flight Failures
Before any operation, Compose Farm checks:
- SSH connectivity
- Compose file existence
- Required mounts
- Required networks
If checks fail, operation aborts with clear error.
### Partial Failures
When operating on multiple services:
- Each service is independent
- Failures are logged, but other services continue
- Exit code reflects overall success/failure
## Performance Considerations
### Parallel Execution
Services are started/stopped in parallel:
```python
await asyncio.gather(*[
up_service(service) for service in services
])
```
### SSH Multiplexing
For repeated connections to the same host, SSH reuses connections.
### Caching
- Config is parsed once per command
- State is loaded once, written once
- Host discovery results are cached during command
## Web UI Architecture
```
┌─────────────────────────────────────────────────────────────┐
│ Web UI │
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ FastAPI │ │ Jinja │ │ HTMX │ │
│ │ Backend │ │ Templates │ │ Dynamic Updates │ │
│ └─────────────┘ └─────────────┘ └─────────────────────┘ │
│ │
│ Pattern: Custom events, not hx-swap-oob │
│ Elements trigger on: cf:refresh from:body │
└─────────────────────────────────────────────────────────────┘
```
Icons use [Lucide](https://lucide.dev/). Add new icons as macros in `web/templates/partials/icons.html`.

3
docs/assets/apply.gif Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:bb1372a59a4ed1ac74d3864d7a84dd5311fce4cb6c6a00bf3a574bc2f98d5595
size 895927

3
docs/assets/apply.webm Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:f339a85f3d930db5a020c9f77e106edc5f44ea7dee6f68557106721493c24ef8
size 205907

3
docs/assets/install.gif Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:388aa49a1269145698f9763452aaf6b9c6232ea9229abe1dae304df558e29695
size 403442

3
docs/assets/install.webm Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:9b8bf4dcb8ee67270d4a88124b4dd4abe0dab518e73812ee73f7c66d77f146e2
size 228025

3
docs/assets/logs.gif Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:16b9a28137dfae25488e2094de85766a039457f5dca20c2d84ac72e3967c10b9
size 164237

3
docs/assets/logs.webm Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:e0fbe697a1f8256ce3b9a6a64c7019d42769134df9b5b964e5abe98a29e918fd
size 68242

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:629b8c80b98eb996b75439745676fd99a83f391ca25f778a71bd59173f814c2f
size 1194931

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:33fd46f2d8538cc43be4cb553b3af9d8b412f282ee354b6373e2793fe41c799b
size 405057

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:ccd96e33faba5f297999917d89834b29d58bd2a8929eea8d62875e3d8830bd5c
size 3198466

View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:979a1a21303bbf284b3510981066ef05c41c1035b34392fecc7bee472116e6db
size 967564

3
docs/assets/update.gif Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:2067f4967a93b7ee3a8db7750c435f41b1fccd2919f3443da4b848c20cc54f23
size 124559

3
docs/assets/update.webm Normal file
View File

@@ -0,0 +1,3 @@
version https://git-lfs.github.com/spec/v1
oid sha256:5471bd94e6d1b9d415547fa44de6021fdad2e1cc5b8b295680e217104aa749d6
size 98149

381
docs/best-practices.md Normal file
View File

@@ -0,0 +1,381 @@
---
icon: lucide/lightbulb
---
# Best Practices
Tips, limitations, and recommendations for using Compose Farm effectively.
## Limitations
### No Cross-Host Networking
Compose Farm moves containers between hosts but **does not provide cross-host networking**. Docker's internal DNS and networks don't span hosts.
**What breaks when you move a service:**
| Feature | Works? | Why |
|---------|--------|-----|
| `http://redis:6379` | No | Docker DNS doesn't cross hosts |
| Docker network names | No | Networks are per-host |
| `DATABASE_URL=postgres://db:5432` | No | Container name won't resolve |
| Host IP addresses | Yes | Use `192.168.1.10:5432` |
### What Compose Farm Doesn't Do
- No overlay networking (use Swarm/Kubernetes)
- No service discovery across hosts
- No automatic dependency tracking between compose files
- No health checks or restart policies beyond Docker's
- No secrets management beyond Docker's
## Service Organization
### Keep Dependencies Together
If services talk to each other, keep them in the same compose file on the same host:
```yaml
# /opt/compose/myapp/docker-compose.yml
services:
app:
image: myapp
depends_on:
- db
- redis
db:
image: postgres
redis:
image: redis
```
```yaml
# compose-farm.yaml
services:
myapp: nuc # All three containers stay together
```
### Separate Standalone Services
Services that don't talk to other containers can be anywhere:
```yaml
services:
# These can run on any host
plex: nuc
jellyfin: hp
homeassistant: nas
# These should stay together
myapp: nuc # includes app + db + redis
```
### Cross-Host Communication
If services MUST communicate across hosts, publish ports:
```yaml
# Instead of
DATABASE_URL=postgres://db:5432
# Use
DATABASE_URL=postgres://192.168.1.10:5432
```
```yaml
# And publish the port
services:
db:
ports:
- "5432:5432"
```
## Multi-Host Services
### When to Use `all`
Use `all` for services that need local access to each host:
```yaml
services:
# Need Docker socket
dozzle: all # Log viewer
portainer-agent: all # Portainer agents
autokuma: all # Auto-creates monitors
# Need host metrics
node-exporter: all # Prometheus metrics
promtail: all # Log shipping
```
### Host-Specific Lists
For services on specific hosts only:
```yaml
services:
# Only on compute nodes
gitlab-runner: [nuc, hp]
# Only on storage nodes
minio: [nas-1, nas-2]
```
## Migration Safety
### Pre-flight Checks
Before migrating, Compose Farm verifies:
- Compose file is accessible on new host
- Required mounts exist on new host
- Required networks exist on new host
### Data Considerations
**Compose Farm doesn't move data.** Ensure:
1. **Shared storage**: Data volumes on NFS/shared storage
2. **External databases**: Data in external DB, not container
3. **Backup first**: Always backup before migration
### Safe Migration Pattern
```bash
# 1. Preview changes
cf apply --dry-run
# 2. Verify target host can run the service
cf check myservice
# 3. Apply changes
cf apply
```
## State Management
### When to Refresh
Run `cf refresh` after:
- Manual `docker compose` commands
- Container restarts
- Host reboots
- Any changes outside Compose Farm
```bash
cf refresh --dry-run # Preview
cf refresh # Sync
```
### State Conflicts
If state doesn't match reality:
```bash
# See what's actually running
cf refresh --dry-run
# Sync state
cf refresh
# Then apply config
cf apply
```
## Shared Storage
### NFS Best Practices
```bash
# Mount options for Docker compatibility
nas:/compose /opt/compose nfs rw,hard,intr,rsize=8192,wsize=8192 0 0
```
### Directory Ownership
Ensure consistent UID/GID across hosts:
```yaml
services:
myapp:
environment:
- PUID=1000
- PGID=1000
```
### Config vs Data
Keep config and data separate:
```
/opt/compose/ # Shared: compose files + config
├── plex/
│ ├── docker-compose.yml
│ └── config/ # Small config files OK
/mnt/data/ # Shared: large media files
├── movies/
├── tv/
└── music/
/opt/appdata/ # Local: per-host app data
├── plex/
└── sonarr/
```
## Performance
### Parallel Operations
Compose Farm runs operations in parallel. For large deployments:
```bash
# Good: parallel by default
cf up --all
# Avoid: sequential updates when possible
for svc in plex sonarr radarr; do
cf update $svc
done
```
### SSH Connection Reuse
SSH connections are reused within a command. For many operations:
```bash
# One command, one connection per host
cf update --all
# Multiple commands, multiple connections (slower)
cf update plex && cf update sonarr && cf update radarr
```
## Traefik Setup
### Service Placement
Put Traefik on a reliable host:
```yaml
services:
traefik: nuc # Primary host with good uptime
```
### Same-Host Services
Services on the same host as Traefik use Docker provider:
```yaml
traefik_service: traefik
services:
traefik: nuc
portainer: nuc # Docker provider handles this
plex: hp # File provider handles this
```
### Middleware in Separate File
Define middlewares outside Compose Farm's generated file:
```yaml
# /opt/traefik/dynamic.d/middlewares.yml
http:
middlewares:
redirect-https:
redirectScheme:
scheme: https
```
## Backup Strategy
### What to Backup
| Item | Location | Method |
|------|----------|--------|
| Compose Farm config | `~/.config/compose-farm/` | Git or copy |
| Compose files | `/opt/compose/` | Git |
| State file | `~/.config/compose-farm/state.yaml` | Optional (can refresh) |
| App data | `/opt/appdata/` | Backup solution |
### Disaster Recovery
```bash
# Restore config
cp backup/compose-farm.yaml ~/.config/compose-farm/
# Refresh state from running containers
cf refresh
# Or start fresh
cf apply
```
## Troubleshooting
### Common Issues
**Service won't start:**
```bash
cf check myservice # Verify mounts/networks
cf logs myservice # Check container logs
```
**Migration fails:**
```bash
cf check myservice # Verify new host is ready
cf init-network newhost # Create network if missing
```
**State out of sync:**
```bash
cf refresh --dry-run # See differences
cf refresh # Sync state
```
**SSH issues:**
```bash
cf ssh status # Check key status
cf ssh setup # Re-setup keys
```
### Debug Mode
For more verbose output:
```bash
# See exact commands being run
cf --verbose up myservice
```
## Security Considerations
### SSH Keys
- Use dedicated SSH key for Compose Farm
- Limit key to specific hosts if possible
- Don't store keys in Docker images
### Network Exposure
- Published ports are accessible from network
- Use firewalls for sensitive services
- Consider VPN for cross-host communication
### Secrets
- Don't commit `.env` files with secrets
- Use Docker secrets or external secret management
- Avoid secrets in compose file labels
## Comparison: When to Use Alternatives
| Scenario | Solution |
|----------|----------|
| 2-10 hosts, static services | **Compose Farm** |
| Cross-host container networking | Docker Swarm |
| Auto-scaling, self-healing | Kubernetes |
| Infrastructure as code | Ansible + Compose Farm |
| High availability requirements | Kubernetes or Swarm |

650
docs/commands.md Normal file
View File

@@ -0,0 +1,650 @@
---
icon: lucide/terminal
---
# Commands Reference
The Compose Farm CLI is available as both `compose-farm` and the shorter alias `cf`.
## Command Overview
| Category | Command | Description |
|----------|---------|-------------|
| **Lifecycle** | `apply` | Make reality match config |
| | `up` | Start services |
| | `down` | Stop services |
| | `restart` | Restart services (down + up) |
| | `update` | Update services (pull + down + up) |
| | `pull` | Pull latest images |
| **Monitoring** | `ps` | Show service status |
| | `logs` | Show service logs |
| | `stats` | Show overview statistics |
| **Configuration** | `check` | Validate config and mounts |
| | `refresh` | Sync state from reality |
| | `init-network` | Create Docker network |
| | `traefik-file` | Generate Traefik config |
| | `config` | Manage config files |
| | `ssh` | Manage SSH keys |
| **Server** | `web` | Start web UI |
## Global Options
```bash
cf --version, -v # Show version
cf --help, -h # Show help
```
---
## Lifecycle Commands
### cf apply
Make reality match your configuration. The primary reconciliation command.
<video autoplay loop muted playsinline>
<source src="assets/apply.webm" type="video/webm">
</video>
```bash
cf apply [OPTIONS]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--dry-run, -n` | Preview changes without executing |
| `--no-orphans` | Skip stopping orphaned services |
| `--full, -f` | Also refresh running services |
| `--config, -c PATH` | Path to config file |
**What it does:**
1. Stops orphaned services (in state but removed from config)
2. Migrates services on wrong host
3. Starts missing services (in config but not running)
**Examples:**
```bash
# Preview what would change
cf apply --dry-run
# Apply all changes
cf apply
# Only start/migrate, don't stop orphans
cf apply --no-orphans
# Also refresh all running services
cf apply --full
```
---
### cf up
Start services. Auto-migrates if host assignment changed.
```bash
cf up [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Start all services |
| `--host, -H TEXT` | Filter to services on this host |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Start specific services
cf up plex sonarr
# Start all services
cf up --all
# Start all services on a specific host
cf up --all --host nuc
```
**Auto-migration:**
If you change a service's host in config and run `cf up`:
1. Verifies mounts/networks exist on new host
2. Runs `down` on old host
3. Runs `up -d` on new host
4. Updates state
---
### cf down
Stop services.
```bash
cf down [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Stop all services |
| `--orphaned` | Stop orphaned services only |
| `--host, -H TEXT` | Filter to services on this host |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Stop specific services
cf down plex
# Stop all services
cf down --all
# Stop services removed from config
cf down --orphaned
# Stop all services on a host
cf down --all --host nuc
```
---
### cf restart
Restart services (down + up).
```bash
cf restart [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Restart all services |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
cf restart plex
cf restart --all
```
---
### cf update
Update services (pull + build + down + up).
<video autoplay loop muted playsinline>
<source src="assets/update.webm" type="video/webm">
</video>
```bash
cf update [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Update all services |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Update specific service
cf update plex
# Update all services
cf update --all
```
---
### cf pull
Pull latest images.
```bash
cf pull [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Pull for all services |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
cf pull plex
cf pull --all
```
---
## Monitoring Commands
### cf ps
Show status of services.
```bash
cf ps [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Show all services (default) |
| `--host, -H TEXT` | Filter to services on this host |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Show all services
cf ps
# Show specific services
cf ps plex sonarr
# Filter by host
cf ps --host nuc
```
---
### cf logs
Show service logs.
<video autoplay loop muted playsinline>
<source src="assets/logs.webm" type="video/webm">
</video>
```bash
cf logs [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Show logs for all services |
| `--host, -H TEXT` | Filter to services on this host |
| `--follow, -f` | Follow logs (live stream) |
| `--tail, -n INTEGER` | Number of lines (default: 20 for --all, 100 otherwise) |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Show last 100 lines
cf logs plex
# Follow logs
cf logs -f plex
# Show last 50 lines of multiple services
cf logs -n 50 plex sonarr
# Show last 20 lines of all services
cf logs --all
```
---
### cf stats
Show overview statistics.
```bash
cf stats [OPTIONS]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--live, -l` | Query Docker for live container counts |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Config/state overview
cf stats
# Include live container counts
cf stats --live
```
---
## Configuration Commands
### cf check
Validate configuration, mounts, and networks.
```bash
cf check [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--local` | Skip SSH-based checks (faster) |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Full validation with SSH
cf check
# Fast local-only validation
cf check --local
# Check specific service and show host compatibility
cf check jellyfin
```
---
### cf refresh
Update local state from running services.
```bash
cf refresh [OPTIONS]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--dry-run, -n` | Show what would change |
| `--log-path, -l PATH` | Path to Dockerfarm TOML log |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Sync state with reality
cf refresh
# Preview changes
cf refresh --dry-run
```
---
### cf init-network
Create Docker network on hosts with consistent settings.
```bash
cf init-network [OPTIONS] [HOSTS]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--network, -n TEXT` | Network name (default: mynetwork) |
| `--subnet, -s TEXT` | Network subnet (default: 172.20.0.0/16) |
| `--gateway, -g TEXT` | Network gateway (default: 172.20.0.1) |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Create on all hosts
cf init-network
# Create on specific hosts
cf init-network nuc hp
# Custom network settings
cf init-network -n production -s 10.0.0.0/16 -g 10.0.0.1
```
---
### cf traefik-file
Generate Traefik file-provider config from compose labels.
```bash
cf traefik-file [OPTIONS] [SERVICES]...
```
**Options:**
| Option | Description |
|--------|-------------|
| `--all, -a` | Generate for all services |
| `--output, -o PATH` | Output file (stdout if omitted) |
| `--config, -c PATH` | Path to config file |
**Examples:**
```bash
# Preview to stdout
cf traefik-file --all
# Write to file
cf traefik-file --all -o /opt/traefik/dynamic.d/cf.yml
# Specific services
cf traefik-file plex jellyfin -o /opt/traefik/cf.yml
```
---
### cf config
Manage configuration files.
```bash
cf config COMMAND
```
**Subcommands:**
| Command | Description |
|---------|-------------|
| `init` | Create new config with examples |
| `show` | Display config with highlighting |
| `path` | Print config file path |
| `validate` | Validate syntax and schema |
| `edit` | Open in $EDITOR |
| `symlink` | Create symlink from default location |
**Options by subcommand:**
| Subcommand | Options |
|------------|---------|
| `init` | `--path/-p PATH`, `--force/-f` |
| `show` | `--path/-p PATH`, `--raw/-r` |
| `edit` | `--path/-p PATH` |
| `path` | `--path/-p PATH` |
| `validate` | `--path/-p PATH` |
| `symlink` | `--force/-f` |
**Examples:**
```bash
# Create config at default location
cf config init
# Create config at custom path
cf config init --path /opt/compose-farm/config.yaml
# Show config with syntax highlighting
cf config show
# Show raw config (for copy-paste)
cf config show --raw
# Validate config
cf config validate
# Edit config in $EDITOR
cf config edit
# Print config path
cf config path
# Create symlink to local config
cf config symlink
# Create symlink to specific file
cf config symlink /opt/compose-farm/config.yaml
```
---
### cf ssh
Manage SSH keys for passwordless authentication.
```bash
cf ssh COMMAND
```
**Subcommands:**
| Command | Description |
|---------|-------------|
| `setup` | Generate key and copy to all hosts |
| `status` | Show SSH key status and host connectivity |
| `keygen` | Generate key without distributing |
**Options for `cf ssh setup` and `cf ssh keygen`:**
| Option | Description |
|--------|-------------|
| `--force, -f` | Regenerate key even if it exists |
**Examples:**
```bash
# Set up SSH keys (generates and distributes)
cf ssh setup
# Check status and connectivity
cf ssh status
# Generate key only (don't distribute)
cf ssh keygen
```
---
## Server Commands
### cf web
Start the web UI server.
```bash
cf web [OPTIONS]
```
**Options:**
| Option | Description |
|--------|-------------|
| `--host, -H TEXT` | Host to bind to (default: 0.0.0.0) |
| `--port, -p INTEGER` | Port to listen on (default: 8000) |
| `--reload, -r` | Enable auto-reload for development |
**Note:** Requires web dependencies: `pip install compose-farm[web]`
**Examples:**
```bash
# Start on default port
cf web
# Start on custom port
cf web --port 3000
# Development mode with auto-reload
cf web --reload
```
---
## Common Patterns
### Daily Operations
```bash
# Morning: check status
cf ps
cf stats --live
# Update a specific service
cf update plex
# View logs
cf logs -f plex
```
### Maintenance
```bash
# Update all services
cf update --all
# Refresh state after manual changes
cf refresh
```
### Migration
```bash
# Preview what would change
cf apply --dry-run
# Move a service: edit config, then
cf up plex # auto-migrates
# Or reconcile everything
cf apply
```
### Troubleshooting
```bash
# Validate config
cf check --local
cf check
# Check specific service
cf check jellyfin
# Sync state
cf refresh --dry-run
cf refresh
```

402
docs/configuration.md Normal file
View File

@@ -0,0 +1,402 @@
---
icon: lucide/settings
---
# Configuration Reference
Compose Farm uses a YAML configuration file to define hosts and service assignments.
## Config File Location
Compose Farm looks for configuration in this order:
1. `-c` / `--config` flag (if provided)
2. `CF_CONFIG` environment variable
3. `./compose-farm.yaml` (current directory)
4. `$XDG_CONFIG_HOME/compose-farm/compose-farm.yaml` (defaults to `~/.config`)
Use `-c` / `--config` to specify a custom path:
```bash
cf ps -c /path/to/config.yaml
```
Or set the environment variable:
```bash
export CF_CONFIG=/path/to/config.yaml
```
## Full Example
```yaml
# Required: directory containing compose files
compose_dir: /opt/compose
# Optional: auto-regenerate Traefik config
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
traefik_service: traefik
# Define Docker hosts
hosts:
nuc:
address: 192.168.1.10
user: docker
hp:
address: 192.168.1.11
user: admin
local: localhost
# Map services to hosts
services:
# Single-host services
plex: nuc
sonarr: nuc
radarr: hp
jellyfin: local
# Multi-host services
dozzle: all # Run on ALL hosts
node-exporter: [nuc, hp] # Run on specific hosts
```
## Settings Reference
### compose_dir (required)
Directory containing your compose service folders. Must be the same path on all hosts.
```yaml
compose_dir: /opt/compose
```
**Directory structure:**
```
/opt/compose/
├── plex/
│ ├── docker-compose.yml # or compose.yaml
│ └── .env # optional environment file
├── sonarr/
│ └── docker-compose.yml
└── ...
```
Supported compose file names (checked in order):
- `compose.yaml`
- `compose.yml`
- `docker-compose.yml`
- `docker-compose.yaml`
### traefik_file
Path to auto-generated Traefik file-provider config. When set, Compose Farm regenerates this file after `up`, `down`, `restart`, and `update` commands.
```yaml
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
```
### traefik_service
Service name running Traefik. Services on the same host are skipped in file-provider config (Traefik's docker provider handles them).
```yaml
traefik_service: traefik
```
## Hosts Configuration
### Basic Host
```yaml
hosts:
myserver:
address: 192.168.1.10
```
### With SSH User
```yaml
hosts:
myserver:
address: 192.168.1.10
user: docker
```
If `user` is omitted, the current user is used.
### With Custom SSH Port
```yaml
hosts:
myserver:
address: 192.168.1.10
user: docker
port: 2222 # SSH port (default: 22)
```
### Localhost
For services running on the same machine where you invoke Compose Farm:
```yaml
hosts:
local: localhost
```
No SSH is used for localhost services.
### Multiple Hosts
```yaml
hosts:
nuc:
address: 192.168.1.10
user: docker
hp:
address: 192.168.1.11
user: admin
truenas:
address: 192.168.1.100
local: localhost
```
## Services Configuration
### Single-Host Service
```yaml
services:
plex: nuc
sonarr: nuc
radarr: hp
```
### Multi-Host Service
For services that need to run on every host (e.g., log shippers, monitoring agents):
```yaml
services:
# Run on ALL configured hosts
dozzle: all
promtail: all
# Run on specific hosts
node-exporter: [nuc, hp, truenas]
```
**Common multi-host services:**
- **Dozzle** - Docker log viewer (needs local socket)
- **Promtail/Alloy** - Log shipping (needs local socket)
- **node-exporter** - Host metrics (needs /proc, /sys)
- **AutoKuma** - Uptime Kuma monitors (needs local socket)
### Service Names
Service names must match directory names in `compose_dir`:
```yaml
compose_dir: /opt/compose
services:
plex: nuc # expects /opt/compose/plex/docker-compose.yml
my-app: hp # expects /opt/compose/my-app/docker-compose.yml
```
## State File
Compose Farm tracks deployment state in `compose-farm-state.yaml`, stored alongside the config file.
For example, if your config is at `~/.config/compose-farm/compose-farm.yaml`, the state file will be at `~/.config/compose-farm/compose-farm-state.yaml`.
```yaml
deployed:
plex: nuc
sonarr: nuc
```
This file records which services are deployed and on which host.
**Don't edit manually.** Use `cf refresh` to sync state with reality.
## Environment Variables
### In Compose Files
Your compose files can use `.env` files as usual:
```
/opt/compose/plex/
├── docker-compose.yml
└── .env
```
Compose Farm runs `docker compose` which handles `.env` automatically.
### In Traefik Labels
When generating Traefik config, Compose Farm resolves `${VAR}` and `${VAR:-default}` from:
1. The service's `.env` file
2. Current environment
## Config Commands
### Initialize Config
```bash
cf config init
```
Creates a new config file with documented examples.
### Validate Config
```bash
cf config validate
```
Checks syntax and schema.
### Show Config
```bash
cf config show
```
Displays current config with syntax highlighting.
### Edit Config
```bash
cf config edit
```
Opens config in `$EDITOR`.
### Show Config Path
```bash
cf config path
```
Prints the config file location (useful for scripting).
### Create Symlink
```bash
cf config symlink # Link to ./compose-farm.yaml
cf config symlink /path/to/my-config.yaml # Link to specific file
```
Creates a symlink from the default location (`~/.config/compose-farm/compose-farm.yaml`) to your config file. Use `--force` to overwrite an existing symlink.
## Validation
### Local Validation
Fast validation without SSH:
```bash
cf check --local
```
Checks:
- Config syntax
- Service-to-host mappings
- Compose file existence
### Full Validation
```bash
cf check
```
Additional SSH-based checks:
- Host connectivity
- Mount point existence
- Docker network existence
- Traefik label validation
### Service-Specific Check
```bash
cf check jellyfin
```
Shows which hosts can run the service (have required mounts/networks).
## Example Configurations
### Minimal
```yaml
compose_dir: /opt/compose
hosts:
server: 192.168.1.10
services:
myapp: server
```
### Home Lab
```yaml
compose_dir: /opt/compose
hosts:
nuc:
address: 192.168.1.10
user: docker
nas:
address: 192.168.1.100
user: admin
services:
# Media
plex: nuc
sonarr: nuc
radarr: nuc
# Infrastructure
traefik: nuc
portainer: nuc
# Monitoring (on all hosts)
dozzle: all
```
### Production
```yaml
compose_dir: /opt/compose
network: production
traefik_file: /opt/traefik/dynamic.d/cf.yml
traefik_service: traefik
hosts:
web-1:
address: 10.0.1.10
user: deploy
web-2:
address: 10.0.1.11
user: deploy
db:
address: 10.0.1.20
user: deploy
services:
# Load balanced
api: [web-1, web-2]
# Single instance
postgres: db
redis: db
# Infrastructure
traefik: web-1
# Monitoring
promtail: all
```

26
docs/demos/README.md Normal file
View File

@@ -0,0 +1,26 @@
# Terminal Demos
[VHS](https://github.com/charmbracelet/vhs) tape files for recording terminal demos.
## Demos
| File | Shows |
|------|-------|
| `install.tape` | Installing with `uv tool install` |
| `quickstart.tape` | `cf ps`, `cf up`, `cf logs` |
| `logs.tape` | Viewing logs |
| `update.tape` | `cf update` |
| `migration.tape` | Service migration |
| `apply.tape` | `cf apply` |
## Recording
```bash
# Record all demos (outputs to docs/assets/)
./docs/demos/record.sh
# Single demo
cd /opt/stacks && vhs /path/to/docs/demos/quickstart.tape
```
Output files (GIF + WebM) are tracked with Git LFS.

39
docs/demos/apply.tape Normal file
View File

@@ -0,0 +1,39 @@
# Apply Demo
# Shows cf apply previewing and reconciling state
Output docs/assets/apply.gif
Output docs/assets/apply.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 600
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Type "# Preview what would change"
Enter
Sleep 500ms
Type "cf apply --dry-run"
Enter
Wait
Type "# Check current status"
Enter
Sleep 500ms
Type "cf stats"
Enter
Wait+Screen /Summary/
Sleep 2s
Type "# Apply the changes"
Enter
Sleep 500ms
Type "cf apply"
Enter
# Wait for shell prompt (command complete)
Wait
Sleep 4s

42
docs/demos/install.tape Normal file
View File

@@ -0,0 +1,42 @@
# Installation Demo
# Shows installing compose-farm with uv
Output docs/assets/install.gif
Output docs/assets/install.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 600
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Env FORCE_COLOR "1"
Hide
Type "export PATH=$HOME/.local/bin:$PATH && uv tool uninstall compose-farm 2>/dev/null; clear"
Enter
Show
Type "# Install with uv (recommended)"
Enter
Sleep 500ms
Type "uv tool install compose-farm"
Enter
Wait+Screen /Installed|already installed/
Type "# Verify installation"
Enter
Sleep 500ms
Type "cf --version"
Enter
Wait+Screen /compose-farm/
Sleep 1s
Type "cf --help | less"
Enter
Sleep 2s
PageDown
Sleep 2s
Type "q"
Sleep 2s

21
docs/demos/logs.tape Normal file
View File

@@ -0,0 +1,21 @@
# Logs Demo
# Shows viewing service logs
Output docs/assets/logs.gif
Output docs/assets/logs.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 550
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Type "# View recent logs"
Enter
Sleep 500ms
Type "cf logs immich --tail 20"
Enter
Wait+Screen /immich/
Sleep 2s

71
docs/demos/migration.tape Normal file
View File

@@ -0,0 +1,71 @@
# Migration Demo
# Shows automatic service migration when host changes
Output docs/assets/migration.gif
Output docs/assets/migration.webm
Set Shell "bash"
Set FontSize 14
Set Width 1000
Set Height 600
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Type "# Current status: audiobookshelf on 'nas'"
Enter
Sleep 500ms
Type "cf ps audiobookshelf"
Enter
Wait+Screen /PORTS/
Type "# Edit config to move it to 'anton'"
Enter
Sleep 1s
Type "nvim /opt/stacks/compose-farm.yaml"
Enter
Wait+Screen /services:/
# Search for audiobookshelf
Type "/audiobookshelf"
Enter
Sleep 1s
# Move to the host value (nas) and change it
Type "f:"
Sleep 500ms
Type "w"
Sleep 500ms
Type "ciw"
Sleep 500ms
Type "anton"
Escape
Sleep 1s
# Save and quit
Type ":wq"
Enter
Sleep 1s
Type "# Run up - automatically migrates!"
Enter
Sleep 500ms
Type "cf up audiobookshelf"
Enter
# Wait for migration phases: first the stop on old host
Wait+Screen /Migrating|down/
# Then wait for start on new host
Wait+Screen /Starting|up/
# Finally wait for completion
Wait
Type "# Verify: audiobookshelf now on 'anton'"
Enter
Sleep 500ms
Type "cf ps audiobookshelf"
Enter
Wait+Screen /PORTS/
Sleep 3s

View File

@@ -0,0 +1,91 @@
# Quick Start Demo
# Shows basic cf commands
Output docs/assets/quickstart.gif
Output docs/assets/quickstart.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 600
Set Theme "Catppuccin Mocha"
Set FontFamily "FiraCode Nerd Font"
Set TypingSpeed 50ms
Env BAT_PAGING "always"
Type "# Config is just: service  host"
Enter
Sleep 500ms
Type "# First, define your hosts..."
Enter
Sleep 500ms
Type "bat -r 1:11 compose-farm.yaml"
Enter
Sleep 3s
Type "q"
Sleep 500ms
Type "# Then map each service to a host"
Enter
Sleep 500ms
Type "bat -r 13:30 compose-farm.yaml"
Enter
Sleep 3s
Type "q"
Sleep 500ms
Type "# Check service status"
Enter
Sleep 500ms
Type "cf ps immich"
Enter
Wait+Screen /PORTS/
Type "# Start a service"
Enter
Sleep 500ms
Type "cf up immich"
Enter
Wait
Type "# View logs"
Enter
Sleep 500ms
Type "cf logs immich --tail 5"
Enter
Wait+Screen /immich/
Sleep 2s
Type "#  The magic: move between hosts (nas  anton)"
Enter
Sleep 500ms
Type "# Change host in config (using sed)"
Enter
Sleep 500ms
Type "sed -i 's/audiobookshelf: nas/audiobookshelf: anton/' compose-farm.yaml"
Enter
Sleep 500ms
Type "# Apply changes - auto-migrates!"
Enter
Sleep 500ms
Type "cf apply"
Enter
Sleep 15s
Type "# Verify: now on anton"
Enter
Sleep 500ms
Type "cf ps audiobookshelf"
Enter
Sleep 5s

88
docs/demos/record.sh Executable file
View File

@@ -0,0 +1,88 @@
#!/usr/bin/env bash
# Record all VHS demos
# Run this on a Docker host with compose-farm configured
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
DOCS_DIR="$(dirname "$SCRIPT_DIR")"
REPO_DIR="$(dirname "$DOCS_DIR")"
OUTPUT_DIR="$DOCS_DIR/assets"
# Colors
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[0;33m'
RED='\033[0;31m'
NC='\033[0m' # No Color
# Check for VHS
if ! command -v vhs &> /dev/null; then
echo "VHS not found. Install with:"
echo " brew install vhs"
echo " # or"
echo " go install github.com/charmbracelet/vhs@latest"
exit 1
fi
# Ensure output directory exists
mkdir -p "$OUTPUT_DIR"
# Temp output dir (VHS runs from /opt/stacks, so relative paths go here)
TEMP_OUTPUT="/opt/stacks/docs/assets"
mkdir -p "$TEMP_OUTPUT"
# Change to /opt/stacks so cf commands use installed version (not editable install)
cd /opt/stacks
# Ensure compose-farm.yaml has no uncommitted changes (safety check)
if ! git diff --quiet compose-farm.yaml; then
echo -e "${RED}Error: compose-farm.yaml has uncommitted changes${NC}"
echo "Commit or stash your changes before recording demos"
exit 1
fi
echo -e "${BLUE}Recording VHS demos...${NC}"
echo "Output directory: $OUTPUT_DIR"
echo ""
# Function to record a tape
record_tape() {
local tape=$1
local name=$(basename "$tape" .tape)
echo -e "${GREEN}Recording:${NC} $name"
if vhs "$tape"; then
echo -e "${GREEN} ✓ Done${NC}"
else
echo -e "${RED} ✗ Failed${NC}"
return 1
fi
}
# Record demos in logical order
echo -e "${YELLOW}=== Phase 1: Basic demos ===${NC}"
record_tape "$SCRIPT_DIR/install.tape"
record_tape "$SCRIPT_DIR/quickstart.tape"
record_tape "$SCRIPT_DIR/logs.tape"
echo -e "${YELLOW}=== Phase 2: Update demo ===${NC}"
record_tape "$SCRIPT_DIR/update.tape"
echo -e "${YELLOW}=== Phase 3: Migration demo ===${NC}"
record_tape "$SCRIPT_DIR/migration.tape"
git -C /opt/stacks checkout compose-farm.yaml # Reset after migration
echo -e "${YELLOW}=== Phase 4: Apply demo ===${NC}"
record_tape "$SCRIPT_DIR/apply.tape"
# Move GIFs and WebMs from temp location to repo
echo ""
echo -e "${BLUE}Moving recordings to repo...${NC}"
mv "$TEMP_OUTPUT"/*.gif "$OUTPUT_DIR/" 2>/dev/null || true
mv "$TEMP_OUTPUT"/*.webm "$OUTPUT_DIR/" 2>/dev/null || true
rmdir "$TEMP_OUTPUT" 2>/dev/null || true
rmdir "$(dirname "$TEMP_OUTPUT")" 2>/dev/null || true
echo ""
echo -e "${GREEN}Done!${NC} Recordings saved to $OUTPUT_DIR/"
ls -la "$OUTPUT_DIR"/*.gif "$OUTPUT_DIR"/*.webm 2>/dev/null || echo "No recordings found (check for errors above)"

32
docs/demos/update.tape Normal file
View File

@@ -0,0 +1,32 @@
# Update Demo
# Shows updating services (pull + down + up)
Output docs/assets/update.gif
Output docs/assets/update.webm
Set Shell "bash"
Set FontSize 14
Set Width 900
Set Height 500
Set Theme "Catppuccin Mocha"
Set TypingSpeed 50ms
Type "# Update a single service"
Enter
Sleep 500ms
Type "cf update grocy"
Enter
# Wait for command to complete (chain waits for longer timeout)
Wait+Screen /pull/
Wait+Screen /grocy/
Wait@60s
Type "# Check current status"
Enter
Sleep 500ms
Type "cf ps grocy"
Enter
Wait+Screen /PORTS/
Sleep 1s

287
docs/getting-started.md Normal file
View File

@@ -0,0 +1,287 @@
---
icon: lucide/rocket
---
# Getting Started
This guide walks you through installing Compose Farm and setting up your first multi-host deployment.
## Prerequisites
Before you begin, ensure you have:
- **[uv](https://docs.astral.sh/uv/)** (recommended) or Python 3.11+
- **SSH key-based authentication** to your Docker hosts
- **Docker and Docker Compose** installed on all target hosts
- **Shared storage** for compose files (NFS, Syncthing, etc.)
## Installation
<video autoplay loop muted playsinline>
<source src="assets/install.webm" type="video/webm">
</video>
### One-liner (recommended)
```bash
curl -fsSL https://raw.githubusercontent.com/basnijholt/compose-farm/main/bootstrap.sh | sh
```
This installs [uv](https://docs.astral.sh/uv/) if needed, then installs compose-farm.
### Using uv
If you already have [uv](https://docs.astral.sh/uv/) installed:
```bash
uv tool install compose-farm
```
### Using pip
If you already have Python 3.11+ installed:
```bash
pip install compose-farm
```
### Using Docker
```bash
docker run --rm \
-v $SSH_AUTH_SOCK:/ssh-agent -e SSH_AUTH_SOCK=/ssh-agent \
-v ./compose-farm.yaml:/root/.config/compose-farm/compose-farm.yaml:ro \
ghcr.io/basnijholt/compose-farm up --all
```
### Verify Installation
```bash
cf --version
cf --help
```
## SSH Setup
Compose Farm uses SSH to run commands on remote hosts. You need passwordless SSH access.
### Option 1: SSH Agent (default)
If you already have SSH keys loaded in your agent:
```bash
# Verify keys are loaded
ssh-add -l
# Test connection
ssh user@192.168.1.10 "docker --version"
```
### Option 2: Dedicated Key (recommended for Docker)
For persistent access when running in Docker:
```bash
# Generate and distribute key to all hosts
cf ssh setup
# Check status
cf ssh status
```
This creates `~/.ssh/compose-farm/id_ed25519` and copies the public key to each host.
## Shared Storage Setup
Compose files must be accessible at the **same path** on all hosts. Common approaches:
### NFS Mount
```bash
# On each Docker host
sudo mount nas:/volume1/compose /opt/compose
# Or add to /etc/fstab
nas:/volume1/compose /opt/compose nfs defaults 0 0
```
### Directory Structure
```
/opt/compose/ # compose_dir in config
├── plex/
│ └── docker-compose.yml
├── sonarr/
│ └── docker-compose.yml
├── radarr/
│ └── docker-compose.yml
└── jellyfin/
└── docker-compose.yml
```
## Configuration
### Create Config File
Create `~/.config/compose-farm/compose-farm.yaml`:
```yaml
# Where compose files are located (same path on all hosts)
compose_dir: /opt/compose
# Define your Docker hosts
hosts:
nuc:
address: 192.168.1.10
user: docker # SSH user
hp:
address: 192.168.1.11
# user defaults to current user
local: localhost # Run locally without SSH
# Map services to hosts
services:
plex: nuc
sonarr: nuc
radarr: hp
jellyfin: local
```
### Validate Configuration
```bash
cf check --local
```
This validates syntax without SSH connections. For full validation:
```bash
cf check
```
## First Commands
### Check Status
```bash
cf ps
```
Shows all configured services and their status.
### Start All Services
```bash
cf up --all
```
Starts all services on their assigned hosts.
### Start Specific Services
```bash
cf up plex sonarr
```
### Apply Configuration
The most powerful command - reconciles reality with your config:
```bash
cf apply --dry-run # Preview changes
cf apply # Execute changes
```
This will:
1. Start services in config but not running
2. Migrate services on wrong host
3. Stop services removed from config
## Docker Network Setup
If your services use an external Docker network:
```bash
# Create network on all hosts
cf init-network
# Or specific hosts
cf init-network nuc hp
```
Default network: `mynetwork` with subnet `172.20.0.0/16`
## Example Workflow
### 1. Add a New Service
Create the compose file:
```bash
# On any host (shared storage)
mkdir -p /opt/compose/prowlarr
cat > /opt/compose/prowlarr/docker-compose.yml << 'EOF'
services:
prowlarr:
image: lscr.io/linuxserver/prowlarr:latest
container_name: prowlarr
environment:
- PUID=1000
- PGID=1000
volumes:
- /opt/config/prowlarr:/config
ports:
- "9696:9696"
restart: unless-stopped
EOF
```
Add to config:
```yaml
services:
# ... existing services
prowlarr: nuc
```
Start the service:
```bash
cf up prowlarr
```
### 2. Move a Service to Another Host
Edit `compose-farm.yaml`:
```yaml
services:
plex: hp # Changed from nuc
```
Apply the change:
```bash
cf up plex
# Automatically: down on nuc, up on hp
```
Or use apply to reconcile everything:
```bash
cf apply
```
### 3. Update All Services
```bash
cf update --all
# Runs: pull + down + up for each service
```
## Next Steps
- [Configuration Reference](configuration.md) - All config options
- [Commands Reference](commands.md) - Full CLI documentation
- [Traefik Integration](traefik.md) - Multi-host routing
- [Best Practices](best-practices.md) - Tips and limitations

129
docs/index.md Normal file
View File

@@ -0,0 +1,129 @@
---
icon: lucide/server
---
# Compose Farm
A minimal CLI tool to run Docker Compose commands across multiple hosts via SSH.
## What is Compose Farm?
Compose Farm lets you manage Docker Compose services across multiple machines from a single command line. Think [Dockge](https://dockge.kuma.pet/) but with a CLI and web interface, designed for multi-host deployments.
Define which services run where in one YAML file, then use `cf apply` to make reality match your configuration.
## Quick Demo
<video autoplay loop muted playsinline>
<source src="assets/quickstart.webm" type="video/webm">
</video>
## Why Compose Farm?
| Problem | Compose Farm Solution |
|---------|----------------------|
| 100+ containers on one machine | Distribute across multiple hosts |
| Kubernetes too complex | Just SSH + docker compose |
| Swarm in maintenance mode | Zero infrastructure changes |
| Manual SSH for each host | Single command for all |
**It's a convenience wrapper, not a new paradigm.** Your existing `docker-compose.yml` files work unchanged.
## Quick Start
```yaml
# compose-farm.yaml
compose_dir: /opt/compose
hosts:
server-1:
address: 192.168.1.10
server-2:
address: 192.168.1.11
services:
plex: server-1
jellyfin: server-2
sonarr: server-1
```
```bash
cf apply # Services start, migrate, or stop as needed
```
### Installation
```bash
uv tool install compose-farm
# or
pip install compose-farm
```
### Configuration
Create `~/.config/compose-farm/compose-farm.yaml`:
```yaml
compose_dir: /opt/compose
hosts:
nuc:
address: 192.168.1.10
user: docker
hp:
address: 192.168.1.11
services:
plex: nuc
sonarr: nuc
radarr: hp
```
### Usage
```bash
# Make reality match config
cf apply
# Start specific services
cf up plex sonarr
# Check status
cf ps
# View logs
cf logs -f plex
```
## Key Features
- **Declarative configuration**: One YAML defines where everything runs
- **Auto-migration**: Change a host assignment, run `cf up`, service moves automatically
<video autoplay loop muted playsinline>
<source src="assets/migration.webm" type="video/webm">
</video>
- **Parallel execution**: Multiple services start/stop concurrently
- **State tracking**: Knows which services are running where
- **Traefik integration**: Generate file-provider config for cross-host routing
- **Zero changes**: Your compose files work as-is
## Requirements
- [uv](https://docs.astral.sh/uv/) (recommended) or Python 3.11+
- SSH key-based authentication to your Docker hosts
- Docker and Docker Compose on all target hosts
- Shared storage (compose files at same path on all hosts)
## Documentation
- [Getting Started](getting-started.md) - Installation and first steps
- [Configuration](configuration.md) - All configuration options
- [Commands](commands.md) - CLI reference
- [Architecture](architecture.md) - How it works under the hood
- [Traefik Integration](traefik.md) - Multi-host routing setup
- [Best Practices](best-practices.md) - Tips and limitations
## License
MIT

79
docs/reddit-post.md Normal file
View File

@@ -0,0 +1,79 @@
# Title options
- Multi-host Docker Compose without Kubernetes or file changes
- I built a CLI to run Docker Compose across hosts. Zero changes to your files.
- I made a CLI to run Docker Compose across multiple hosts without Kubernetes or Swarm
---
I've been running 100+ Docker Compose stacks on a single machine, and it kept running out of memory. I needed to spread services across multiple hosts, but:
- **Kubernetes** felt like overkill. I don't need pods, ingress controllers, or 10x more YAML.
- **Docker Swarm** is basically in maintenance mode.
- Both require rewriting my compose files.
So I built **Compose Farm**, a simple CLI that runs `docker compose` commands over SSH. No agents, no cluster setup, no changes to your existing compose files.
## How it works
One YAML file maps services to hosts:
```yaml
compose_dir: /opt/stacks
hosts:
nuc: 192.168.1.10
hp: 192.168.1.11
services:
plex: nuc
jellyfin: hp
sonarr: nuc
radarr: nuc
```
Then just:
```bash
cf up plex # runs on nuc via SSH
cf apply # makes config state match desired state on all hosts (like Terraform apply)
cf up --all # starts everything on their assigned hosts
cf logs -f plex # streams logs
cf ps # shows status across all hosts
```
## Auto-migration
Change a service's host in the config and run `cf up`. It stops the service on the old host and starts it on the new one. No manual SSH needed.
```yaml
# Before
plex: nuc
# After (just change this)
plex: hp
```
```bash
cf up plex # migrates automatically
```
## Requirements
- SSH key auth to your hosts
- Same paths on all hosts (I use NFS from my NAS)
- That's it. No agents, no daemons.
## What it doesn't do
- No high availability (if a host goes down, services don't auto-migrate)
- No overlay networking (containers on different hosts can't talk via Docker DNS)
- No health checks or automatic restarts
It's a convenience wrapper around `docker compose` + SSH. If you need failover or cross-host container networking, you probably do need Swarm or Kubernetes.
## Links
- GitHub: https://github.com/basnijholt/compose-farm
- Install: `uv tool install compose-farm` or `pip install compose-farm`
Happy to answer questions or take feedback!

385
docs/traefik.md Normal file
View File

@@ -0,0 +1,385 @@
---
icon: lucide/globe
---
# Traefik Integration
Compose Farm can generate Traefik file-provider configuration for routing traffic across multiple hosts.
## The Problem
When you run Traefik on one host but services on others, Traefik's docker provider can't see remote containers. The file provider bridges this gap.
```
Internet
┌─────────────────────────────────────────────────────────────┐
│ Host: nuc │
│ │
│ ┌─────────┐ │
│ │ Traefik │◄─── Docker provider sees local containers │
│ │ │ │
│ │ │◄─── File provider sees remote services │
│ └────┬────┘ (from compose-farm.yml) │
│ │ │
└───────┼─────────────────────────────────────────────────────┘
├────────────────────┐
│ │
▼ ▼
┌───────────────┐ ┌───────────────┐
│ Host: hp │ │ Host: nas │
│ │ │ │
│ plex:32400 │ │ jellyfin:8096 │
└───────────────┘ └───────────────┘
```
## How It Works
1. Your compose files have standard Traefik labels
2. Compose Farm reads labels and generates file-provider config
3. Traefik watches the generated file
4. Traffic routes to remote services via host IP + published port
## Setup
### Step 1: Configure Traefik File Provider
Add directory watching to your Traefik config:
```yaml
# traefik.yml or docker-compose.yml command
providers:
file:
directory: /opt/traefik/dynamic.d
watch: true
```
Or via command line:
```yaml
services:
traefik:
command:
- --providers.file.directory=/dynamic.d
- --providers.file.watch=true
volumes:
- /opt/traefik/dynamic.d:/dynamic.d:ro
```
### Step 2: Add Traefik Labels to Services
Your compose files use standard Traefik labels:
```yaml
# /opt/compose/plex/docker-compose.yml
services:
plex:
image: lscr.io/linuxserver/plex
ports:
- "32400:32400" # IMPORTANT: Must publish port!
labels:
- traefik.enable=true
- traefik.http.routers.plex.rule=Host(`plex.example.com`)
- traefik.http.routers.plex.entrypoints=websecure
- traefik.http.routers.plex.tls.certresolver=letsencrypt
- traefik.http.services.plex.loadbalancer.server.port=32400
```
**Important:** Services must publish ports for cross-host routing. Traefik connects via `host_ip:published_port`.
### Step 3: Generate File Provider Config
```bash
cf traefik-file --all -o /opt/traefik/dynamic.d/compose-farm.yml
```
This generates:
```yaml
# /opt/traefik/dynamic.d/compose-farm.yml
http:
routers:
plex:
rule: Host(`plex.example.com`)
entryPoints:
- websecure
tls:
certResolver: letsencrypt
service: plex
services:
plex:
loadBalancer:
servers:
- url: http://192.168.1.11:32400
```
## Auto-Regeneration
Configure automatic regeneration in `compose-farm.yaml`:
```yaml
compose_dir: /opt/compose
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
traefik_service: traefik
hosts:
nuc:
address: 192.168.1.10
hp:
address: 192.168.1.11
services:
traefik: nuc # Traefik runs here
plex: hp # Routed via file-provider
sonarr: hp
```
With `traefik_file` set, these commands auto-regenerate the config:
- `cf up`
- `cf down`
- `cf restart`
- `cf update`
- `cf apply`
### traefik_service Option
When set, services on the **same host as Traefik** are skipped in file-provider output. Traefik's docker provider handles them directly.
```yaml
traefik_service: traefik # traefik runs on nuc
services:
traefik: nuc # NOT in file-provider (docker provider)
portainer: nuc # NOT in file-provider (docker provider)
plex: hp # IN file-provider (cross-host)
```
## Label Syntax
### Routers
```yaml
labels:
# Basic router
- traefik.http.routers.myapp.rule=Host(`app.example.com`)
- traefik.http.routers.myapp.entrypoints=websecure
# With TLS
- traefik.http.routers.myapp.tls=true
- traefik.http.routers.myapp.tls.certresolver=letsencrypt
# With middleware
- traefik.http.routers.myapp.middlewares=auth@file
```
### Services
```yaml
labels:
# Load balancer port
- traefik.http.services.myapp.loadbalancer.server.port=8080
# Health check
- traefik.http.services.myapp.loadbalancer.healthcheck.path=/health
```
### Middlewares
Middlewares should be defined in a separate file (not generated by Compose Farm):
```yaml
# /opt/traefik/dynamic.d/middlewares.yml
http:
middlewares:
auth:
basicAuth:
users:
- "user:$apr1$..."
```
Reference in labels:
```yaml
labels:
- traefik.http.routers.myapp.middlewares=auth@file
```
## Variable Substitution
Labels can use environment variables:
```yaml
labels:
- traefik.http.routers.myapp.rule=Host(`${DOMAIN}`)
```
Compose Farm resolves variables from:
1. Service's `.env` file
2. Current environment
```bash
# /opt/compose/myapp/.env
DOMAIN=app.example.com
```
## Port Resolution
Compose Farm determines the target URL from published ports:
```yaml
ports:
- "8080:80" # Uses 8080
- "192.168.1.11:8080:80" # Uses 8080 on specific IP
```
If no suitable port is found, a warning is shown.
## Complete Example
### compose-farm.yaml
```yaml
compose_dir: /opt/compose
traefik_file: /opt/traefik/dynamic.d/compose-farm.yml
traefik_service: traefik
hosts:
nuc:
address: 192.168.1.10
hp:
address: 192.168.1.11
nas:
address: 192.168.1.100
services:
traefik: nuc
plex: hp
jellyfin: nas
sonarr: nuc
radarr: nuc
```
### /opt/compose/plex/docker-compose.yml
```yaml
services:
plex:
image: lscr.io/linuxserver/plex
container_name: plex
ports:
- "32400:32400"
labels:
- traefik.enable=true
- traefik.http.routers.plex.rule=Host(`plex.example.com`)
- traefik.http.routers.plex.entrypoints=websecure
- traefik.http.routers.plex.tls.certresolver=letsencrypt
- traefik.http.services.plex.loadbalancer.server.port=32400
# ... other config
```
### Generated compose-farm.yml
```yaml
http:
routers:
plex:
rule: Host(`plex.example.com`)
entryPoints:
- websecure
tls:
certResolver: letsencrypt
service: plex
jellyfin:
rule: Host(`jellyfin.example.com`)
entryPoints:
- websecure
tls:
certResolver: letsencrypt
service: jellyfin
services:
plex:
loadBalancer:
servers:
- url: http://192.168.1.11:32400
jellyfin:
loadBalancer:
servers:
- url: http://192.168.1.100:8096
```
Note: `sonarr` and `radarr` are NOT in the file because they're on the same host as Traefik (`nuc`).
## Combining with Existing Config
If you have existing Traefik dynamic config:
```bash
# Move existing config to directory
mkdir -p /opt/traefik/dynamic.d
mv /opt/traefik/dynamic.yml /opt/traefik/dynamic.d/manual.yml
# Generate Compose Farm config
cf traefik-file --all -o /opt/traefik/dynamic.d/compose-farm.yml
# Update Traefik to watch directory
# --providers.file.directory=/dynamic.d
```
Traefik merges all YAML files in the directory.
## Troubleshooting
### Service Not Accessible
1. **Check port is published:**
```yaml
ports:
- "8080:80" # Must be published, not just exposed
```
2. **Check label syntax:**
```bash
cf check myservice
```
3. **Verify generated config:**
```bash
cf traefik-file myservice
```
4. **Check Traefik logs:**
```bash
docker logs traefik
```
### Config Not Regenerating
1. **Verify traefik_file is set:**
```bash
cf config show | grep traefik
```
2. **Check file permissions:**
```bash
ls -la /opt/traefik/dynamic.d/
```
3. **Manually regenerate:**
```bash
cf traefik-file --all -o /opt/traefik/dynamic.d/compose-farm.yml
```
### Variable Not Resolved
1. **Check .env file exists:**
```bash
cat /opt/compose/myservice/.env
```
2. **Test variable resolution:**
```bash
cd /opt/compose/myservice
docker compose config
```

View File

@@ -11,7 +11,7 @@
name: paperless-ngx
services:
redis:
image: redis:7
image: redis:8
container_name: paperless-redis
restart: unless-stopped
networks:

170
hatch_build.py Normal file
View File

@@ -0,0 +1,170 @@
"""Hatch build hook to vendor CDN assets for offline use.
During wheel builds, this hook:
1. Parses base.html to find elements with data-vendor attributes
2. Downloads each CDN asset to a temporary vendor directory
3. Rewrites base.html to use local /static/vendor/ paths
4. Fetches and bundles license information
5. Includes everything in the wheel via force_include
The source base.html keeps CDN links for development; only the
distributed wheel has vendored assets.
"""
from __future__ import annotations
import re
import shutil
import subprocess
import tempfile
from pathlib import Path
from typing import Any
from urllib.request import Request, urlopen
from hatchling.builders.hooks.plugin.interface import BuildHookInterface
# Matches elements with data-vendor attribute: extracts URL and target filename
# Example: <script src="https://..." data-vendor="htmx.js">
# Captures: (1) src/href, (2) URL, (3) attributes between, (4) vendor filename
VENDOR_PATTERN = re.compile(r'(src|href)="(https://[^"]+)"([^>]*?)data-vendor="([^"]+)"')
# License URLs for each package (GitHub raw URLs)
LICENSE_URLS: dict[str, tuple[str, str]] = {
"htmx": ("MIT", "https://raw.githubusercontent.com/bigskysoftware/htmx/master/LICENSE"),
"xterm": ("MIT", "https://raw.githubusercontent.com/xtermjs/xterm.js/master/LICENSE"),
"daisyui": ("MIT", "https://raw.githubusercontent.com/saadeghi/daisyui/master/LICENSE"),
"tailwindcss": (
"MIT",
"https://raw.githubusercontent.com/tailwindlabs/tailwindcss/master/LICENSE",
),
}
def _download(url: str) -> bytes:
"""Download a URL, trying urllib first then curl as fallback."""
# Try urllib first
try:
req = Request( # noqa: S310
url, headers={"User-Agent": "Mozilla/5.0 (compatible; compose-farm build)"}
)
with urlopen(req, timeout=30) as resp: # noqa: S310
return resp.read() # type: ignore[no-any-return]
except Exception: # noqa: S110
pass # Fall through to curl
# Fallback to curl (handles SSL proxies better)
result = subprocess.run(
["curl", "-fsSL", "--max-time", "30", url], # noqa: S607
capture_output=True,
check=True,
)
return bytes(result.stdout)
def _generate_licenses_file(temp_dir: Path) -> None:
"""Download and combine license files into LICENSES.txt."""
lines = [
"# Vendored Dependencies - License Information",
"",
"This file contains license information for JavaScript/CSS libraries",
"bundled with compose-farm for offline use.",
"",
"=" * 70,
"",
]
for pkg_name, (license_type, license_url) in LICENSE_URLS.items():
lines.append(f"## {pkg_name} ({license_type})")
lines.append(f"Source: {license_url}")
lines.append("")
lines.append(_download(license_url).decode("utf-8"))
lines.append("")
lines.append("=" * 70)
lines.append("")
(temp_dir / "LICENSES.txt").write_text("\n".join(lines))
class VendorAssetsHook(BuildHookInterface): # type: ignore[misc]
"""Hatch build hook that vendors CDN assets into the wheel."""
PLUGIN_NAME = "vendor-assets"
def initialize(
self,
_version: str,
build_data: dict[str, Any],
) -> None:
"""Download CDN assets and prepare them for inclusion in the wheel."""
# Only run for wheel builds
if self.target_name != "wheel":
return
# Paths
src_dir = Path(self.root) / "src" / "compose_farm"
base_html_path = src_dir / "web" / "templates" / "base.html"
if not base_html_path.exists():
return
# Create temp directory for vendored assets
temp_dir = Path(tempfile.mkdtemp(prefix="compose_farm_vendor_"))
vendor_dir = temp_dir / "vendor"
vendor_dir.mkdir()
# Read and parse base.html
html_content = base_html_path.read_text()
url_to_filename: dict[str, str] = {}
# Find all elements with data-vendor attribute and download them
for match in VENDOR_PATTERN.finditer(html_content):
url = match.group(2)
filename = match.group(4)
if url in url_to_filename:
continue
url_to_filename[url] = filename
content = _download(url)
(vendor_dir / filename).write_bytes(content)
if not url_to_filename:
return
# Generate LICENSES.txt
_generate_licenses_file(vendor_dir)
# Rewrite HTML to use local paths (remove data-vendor, update URL)
def replace_vendor_tag(match: re.Match[str]) -> str:
attr = match.group(1) # src or href
url = match.group(2)
between = match.group(3) # attributes between URL and data-vendor
filename = match.group(4)
if url in url_to_filename:
return f'{attr}="/static/vendor/{filename}"{between}'
return match.group(0)
modified_html = VENDOR_PATTERN.sub(replace_vendor_tag, html_content)
# Write modified base.html to temp
templates_dir = temp_dir / "templates"
templates_dir.mkdir()
(templates_dir / "base.html").write_text(modified_html)
# Add to force_include to override files in the wheel
force_include = build_data.setdefault("force_include", {})
force_include[str(vendor_dir)] = "compose_farm/web/static/vendor"
force_include[str(templates_dir / "base.html")] = "compose_farm/web/templates/base.html"
# Store temp_dir path for cleanup
self._temp_dir = temp_dir
def finalize(
self,
_version: str,
_build_data: dict[str, Any],
_artifact_path: str,
) -> None:
"""Clean up temporary directory after build."""
if hasattr(self, "_temp_dir") and self._temp_dir.exists():
shutil.rmtree(self._temp_dir, ignore_errors=True)

View File

@@ -4,6 +4,7 @@ dynamic = ["version"]
description = "Compose Farm - run docker compose commands across multiple hosts"
readme = "README.md"
license = "MIT"
license-files = ["LICENSE"]
authors = [
{ name = "Bas Nijholt", email = "bas@nijho.lt" }
]
@@ -34,6 +35,7 @@ classifiers = [
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python :: 3.14",
"Topic :: System :: Systems Administration",
"Topic :: Utilities",
"Typing :: Typed",
@@ -46,6 +48,13 @@ dependencies = [
"rich>=13.0.0",
]
[project.optional-dependencies]
web = [
"fastapi[standard]>=0.109.0",
"jinja2>=3.1.0",
"websockets>=12.0",
]
[project.urls]
Homepage = "https://github.com/basnijholt/compose-farm"
Repository = "https://github.com/basnijholt/compose-farm"
@@ -70,6 +79,9 @@ version-file = "src/compose_farm/_version.py"
[tool.hatch.build.targets.wheel]
packages = ["src/compose_farm"]
[tool.hatch.build.hooks.custom]
# Vendors CDN assets (JS/CSS) into the wheel for offline use
[tool.ruff]
target-version = "py311"
line-length = 100
@@ -99,7 +111,7 @@ ignore = [
]
[tool.ruff.lint.per-file-ignores]
"tests/*" = ["S101", "PLR2004", "S108", "D102", "D103"] # relaxed docstrings + asserts in tests
"tests/*" = ["S101", "PLR2004", "S108", "D102", "D103", "PLC0415", "ARG001", "ARG002", "TC003"] # relaxed for tests
[tool.ruff.lint.mccabe]
max-complexity = 18
@@ -117,6 +129,10 @@ ignore_missing_imports = true
module = "tests.*"
disallow_untyped_decorators = false
[[tool.mypy.overrides]]
module = "compose_farm.web.*"
disallow_untyped_decorators = false
[tool.pytest.ini_options]
asyncio_mode = "auto"
testpaths = ["tests"]
@@ -129,6 +145,9 @@ addopts = [
"--no-cov-on-fail",
"-v",
]
markers = [
"browser: marks tests as browser tests (deselect with '-m \"not browser\"')",
]
[tool.coverage.run]
omit = []
@@ -151,4 +170,15 @@ dev = [
"ruff>=0.14.8",
"types-pyyaml>=6.0.12.20250915",
"markdown-code-runner>=0.7.0",
# Web deps for type checking (these ship with inline types)
"fastapi>=0.109.0",
"uvicorn[standard]>=0.27.0",
"jinja2>=3.1.0",
"websockets>=12.0",
# For FastAPI TestClient
"httpx>=0.28.0",
# For browser tests (use system chromium via nix-shell -p chromium)
"pytest-playwright>=0.7.0",
# For parallel test execution
"pytest-xdist>=3.0.0",
]

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,21 @@
"""CLI interface using Typer."""
from __future__ import annotations
# Import command modules to trigger registration via @app.command() decorators
from compose_farm.cli import (
config, # noqa: F401
lifecycle, # noqa: F401
management, # noqa: F401
monitoring, # noqa: F401
ssh, # noqa: F401
web, # noqa: F401
)
# Import the shared app instance
from compose_farm.cli.app import app
__all__ = ["app"]
if __name__ == "__main__":
app()

View File

@@ -0,0 +1,42 @@
"""Shared Typer app instance."""
from __future__ import annotations
from typing import Annotated
import typer
from compose_farm import __version__
__all__ = ["app"]
def _version_callback(value: bool) -> None:
"""Print version and exit."""
if value:
typer.echo(f"compose-farm {__version__}")
raise typer.Exit
app = typer.Typer(
name="compose-farm",
help="Compose Farm - run docker compose commands across multiple hosts",
no_args_is_help=True,
context_settings={"help_option_names": ["-h", "--help"]},
)
@app.callback()
def main(
version: Annotated[
bool,
typer.Option(
"--version",
"-v",
help="Show version and exit",
callback=_version_callback,
is_eager=True,
),
] = False,
) -> None:
"""Compose Farm - run docker compose commands across multiple hosts."""

View File

@@ -0,0 +1,324 @@
"""Shared CLI helpers, options, and utilities."""
from __future__ import annotations
import asyncio
import contextlib
from pathlib import Path
from typing import TYPE_CHECKING, Annotated, TypeVar
import typer
from rich.progress import (
BarColumn,
MofNCompleteColumn,
Progress,
SpinnerColumn,
TaskID,
TextColumn,
TimeElapsedColumn,
)
from compose_farm.console import (
MSG_HOST_NOT_FOUND,
MSG_SERVICE_NOT_FOUND,
console,
print_error,
print_hint,
print_success,
print_warning,
)
if TYPE_CHECKING:
from collections.abc import Callable, Coroutine, Generator
from compose_farm.config import Config
from compose_farm.executor import CommandResult
_T = TypeVar("_T")
_R = TypeVar("_R")
# --- Shared CLI Options ---
ServicesArg = Annotated[
list[str] | None,
typer.Argument(help="Services to operate on"),
]
AllOption = Annotated[
bool,
typer.Option("--all", "-a", help="Run on all services"),
]
ConfigOption = Annotated[
Path | None,
typer.Option("--config", "-c", help="Path to config file"),
]
LogPathOption = Annotated[
Path | None,
typer.Option("--log-path", "-l", help="Path to Dockerfarm TOML log"),
]
HostOption = Annotated[
str | None,
typer.Option("--host", "-H", help="Filter to services on this host"),
]
# --- Constants (internal) ---
_MISSING_PATH_PREVIEW_LIMIT = 2
_STATS_PREVIEW_LIMIT = 3 # Max number of pending migrations to show by name
def format_host(host: str | list[str]) -> str:
"""Format a host value for display."""
if isinstance(host, list):
return ", ".join(host)
return host
@contextlib.contextmanager
def progress_bar(
label: str, total: int, *, initial_description: str = "[dim]connecting...[/]"
) -> Generator[tuple[Progress, TaskID], None, None]:
"""Create a standardized progress bar with consistent styling.
Yields (progress, task_id). Use progress.update(task_id, advance=1, description=...)
to advance.
"""
with Progress(
SpinnerColumn(),
TextColumn(f"[bold blue]{label}[/]"),
BarColumn(),
MofNCompleteColumn(),
TextColumn(""),
TimeElapsedColumn(),
TextColumn(""),
TextColumn("[progress.description]{task.description}"),
console=console,
transient=True,
) as progress:
task_id = progress.add_task(initial_description, total=total)
yield progress, task_id
def run_parallel_with_progress(
label: str,
items: list[_T],
async_fn: Callable[[_T], Coroutine[None, None, _R]],
) -> list[_R]:
"""Run async tasks in parallel with a progress bar.
Args:
label: Progress bar label (e.g., "Discovering", "Querying hosts")
items: List of items to process
async_fn: Async function to call for each item, returns tuple where
first element is used for progress description
Returns:
List of results from async_fn in completion order.
"""
async def gather() -> list[_R]:
with progress_bar(label, len(items)) as (progress, task_id):
tasks = [asyncio.create_task(async_fn(item)) for item in items]
results: list[_R] = []
for coro in asyncio.as_completed(tasks):
result = await coro
results.append(result)
progress.update(task_id, advance=1, description=f"[cyan]{result[0]}[/]") # type: ignore[index]
return results
return asyncio.run(gather())
def load_config_or_exit(config_path: Path | None) -> Config:
"""Load config or exit with a friendly error message."""
# Lazy import: pydantic adds ~50ms to startup, only load when actually needed
from compose_farm.config import load_config # noqa: PLC0415
try:
return load_config(config_path)
except FileNotFoundError as e:
print_error(str(e))
raise typer.Exit(1) from e
def get_services(
services: list[str],
all_services: bool,
config_path: Path | None,
*,
host: str | None = None,
default_all: bool = False,
) -> tuple[list[str], Config]:
"""Resolve service list and load config.
Handles three mutually exclusive selection methods:
- Explicit service names
- --all flag
- --host filter
Args:
services: Explicit service names
all_services: Whether --all was specified
config_path: Path to config file
host: Filter to services on this host
default_all: If True, default to all services when nothing specified (for ps)
Supports "." as shorthand for the current directory name.
"""
validate_service_selection(services, all_services, host)
config = load_config_or_exit(config_path)
if host is not None:
validate_hosts(config, host)
svc_list = [s for s in config.services if host in config.get_hosts(s)]
if not svc_list:
print_warning(f"No services configured for host [magenta]{host}[/]")
raise typer.Exit(0)
return svc_list, config
if all_services:
return list(config.services.keys()), config
if not services:
if default_all:
return list(config.services.keys()), config
print_error("Specify services or use [bold]--all[/] / [bold]--host[/]")
raise typer.Exit(1)
# Resolve "." to current directory name
resolved = [Path.cwd().name if svc == "." else svc for svc in services]
# Validate all services exist in config
validate_services(
config, resolved, hint="Add the service to compose-farm.yaml or use [bold]--all[/]"
)
return resolved, config
def run_async(coro: Coroutine[None, None, _T]) -> _T:
"""Run async coroutine."""
try:
return asyncio.run(coro)
except KeyboardInterrupt:
console.print("\n[yellow]Interrupted[/]")
raise typer.Exit(130) from None # Standard exit code for SIGINT
def report_results(results: list[CommandResult]) -> None:
"""Report command results and exit with appropriate code."""
succeeded = [r for r in results if r.success]
failed = [r for r in results if not r.success]
# Always print summary when there are multiple results
if len(results) > 1:
console.print() # Blank line before summary
if failed:
for r in failed:
print_error(f"[cyan]{r.service}[/] failed with exit code {r.exit_code}")
console.print()
console.print(
f"[green]✓[/] {len(succeeded)}/{len(results)} services succeeded, "
f"[red]✗[/] {len(failed)} failed"
)
else:
print_success(f"All {len(results)} services succeeded")
elif failed:
# Single service failed
r = failed[0]
print_error(f"[cyan]{r.service}[/] failed with exit code {r.exit_code}")
if failed:
raise typer.Exit(1)
def maybe_regenerate_traefik(
cfg: Config,
results: list[CommandResult] | None = None,
) -> None:
"""Regenerate traefik config if traefik_file is configured.
If results are provided, skips regeneration if all services failed.
"""
if cfg.traefik_file is None:
return
# Skip if all services failed
if results and not any(r.success for r in results):
return
# Lazy import: traefik/yaml adds startup time, only load when traefik_file is configured
from compose_farm.traefik import ( # noqa: PLC0415
generate_traefik_config,
render_traefik_config,
)
try:
dynamic, warnings = generate_traefik_config(cfg, list(cfg.services.keys()))
new_content = render_traefik_config(dynamic)
# Check if content changed
old_content = ""
if cfg.traefik_file.exists():
old_content = cfg.traefik_file.read_text()
if new_content != old_content:
cfg.traefik_file.parent.mkdir(parents=True, exist_ok=True)
cfg.traefik_file.write_text(new_content)
console.print() # Ensure we're on a new line after streaming output
print_success(f"Traefik config updated: {cfg.traefik_file}")
for warning in warnings:
print_warning(warning)
except (FileNotFoundError, ValueError) as exc:
print_warning(f"Failed to update traefik config: {exc}")
def validate_services(cfg: Config, services: list[str], *, hint: str | None = None) -> None:
"""Validate that all services exist in config. Exits with error if any not found."""
invalid = [s for s in services if s not in cfg.services]
if invalid:
for svc in invalid:
print_error(MSG_SERVICE_NOT_FOUND.format(name=svc))
if hint:
print_hint(hint)
raise typer.Exit(1)
def validate_hosts(cfg: Config, hosts: str | list[str]) -> None:
"""Validate that host(s) exist in config. Exits with error if any not found."""
host_list = [hosts] if isinstance(hosts, str) else hosts
invalid = [h for h in host_list if h not in cfg.hosts]
if invalid:
for h in invalid:
print_error(MSG_HOST_NOT_FOUND.format(name=h))
raise typer.Exit(1)
def validate_host_for_service(cfg: Config, service: str, host: str) -> None:
"""Validate that a host is valid for a service."""
validate_hosts(cfg, host)
allowed_hosts = cfg.get_hosts(service)
if host not in allowed_hosts:
print_error(
f"Service [cyan]{service}[/] is not configured for host [magenta]{host}[/] "
f"(configured: {', '.join(allowed_hosts)})"
)
raise typer.Exit(1)
def validate_service_selection(
services: list[str] | None,
all_services: bool,
host: str | None,
) -> None:
"""Validate that only one service selection method is used.
The three selection methods (explicit services, --all, --host) are mutually
exclusive. This ensures consistent behavior across all commands.
"""
methods = sum([bool(services), all_services, host is not None])
if methods > 1:
print_error("Use only one of: service names, [bold]--all[/], or [bold]--host[/]")
raise typer.Exit(1)

View File

@@ -0,0 +1,297 @@
"""Configuration management commands for compose-farm."""
from __future__ import annotations
import os
import platform
import shlex
import shutil
import subprocess
from importlib import resources
from pathlib import Path
from typing import Annotated
import typer
from compose_farm.cli.app import app
from compose_farm.console import MSG_CONFIG_NOT_FOUND, console, print_error, print_success
from compose_farm.paths import config_search_paths, default_config_path, find_config_path
config_app = typer.Typer(
name="config",
help="Manage compose-farm configuration files.",
no_args_is_help=True,
)
# --- CLI Options (same pattern as cli.py) ---
_PathOption = Annotated[
Path | None,
typer.Option("--path", "-p", help="Path to config file. Uses auto-detection if not specified."),
]
_ForceOption = Annotated[
bool,
typer.Option("--force", "-f", help="Overwrite existing config without confirmation."),
]
_RawOption = Annotated[
bool,
typer.Option("--raw", "-r", help="Output raw file contents (for copy-paste)."),
]
def _get_editor() -> str:
"""Get the user's preferred editor ($EDITOR > $VISUAL > platform default)."""
if editor := os.environ.get("EDITOR") or os.environ.get("VISUAL"):
return editor
if platform.system() == "Windows":
return "notepad"
return next((e for e in ("nano", "vim", "vi") if shutil.which(e)), "vi")
def _generate_template() -> str:
"""Generate a config template with documented schema."""
try:
template_file = resources.files("compose_farm") / "example-config.yaml"
return template_file.read_text(encoding="utf-8")
except FileNotFoundError as e:
print_error("Example config template is missing from the package")
console.print("Reinstall compose-farm or report this issue.")
raise typer.Exit(1) from e
def _get_config_file(path: Path | None) -> Path | None:
"""Resolve config path, or auto-detect from standard locations."""
if path:
return path.expanduser().resolve()
config_path = find_config_path()
return config_path.resolve() if config_path else None
def _report_missing_config(explicit_path: Path | None = None) -> None:
"""Report that a config file was not found."""
console.print("[yellow]Config file not found.[/yellow]")
if explicit_path:
console.print(f"\nProvided path does not exist: [cyan]{explicit_path}[/cyan]")
else:
console.print("\nSearched locations:")
for p in config_search_paths():
status = "[green]exists[/green]" if p.exists() else "[dim]not found[/dim]"
console.print(f" - {p} ({status})")
console.print("\nRun [bold cyan]cf config init[/bold cyan] to create one.")
@config_app.command("init")
def config_init(
path: _PathOption = None,
force: _ForceOption = False,
) -> None:
"""Create a new config file with documented example.
The generated config file serves as a template showing all available
options with explanatory comments.
"""
target_path = (path.expanduser().resolve() if path else None) or default_config_path()
if target_path.exists() and not force:
console.print(
f"[bold yellow]Config file already exists at:[/bold yellow] [cyan]{target_path}[/cyan]",
)
if not typer.confirm("Overwrite existing config file?"):
console.print("[dim]Aborted.[/dim]")
raise typer.Exit(0)
# Create parent directories
target_path.parent.mkdir(parents=True, exist_ok=True)
# Generate and write template
template_content = _generate_template()
target_path.write_text(template_content, encoding="utf-8")
print_success(f"Config file created at: {target_path}")
console.print("\n[dim]Edit the file to customize your settings:[/dim]")
console.print(" [cyan]cf config edit[/cyan]")
@config_app.command("edit")
def config_edit(
path: _PathOption = None,
) -> None:
"""Open the config file in your default editor.
The editor is determined by: $EDITOR > $VISUAL > platform default.
"""
config_file = _get_config_file(path)
if config_file is None:
_report_missing_config()
raise typer.Exit(1)
if not config_file.exists():
_report_missing_config(config_file)
raise typer.Exit(1)
editor = _get_editor()
console.print(f"[dim]Opening {config_file} with {editor}...[/dim]")
try:
editor_cmd = shlex.split(editor, posix=os.name != "nt")
except ValueError as e:
print_error("Invalid editor command. Check [bold]$EDITOR[/]/[bold]$VISUAL[/]")
raise typer.Exit(1) from e
if not editor_cmd:
print_error("Editor command is empty")
raise typer.Exit(1)
try:
subprocess.run([*editor_cmd, str(config_file)], check=True)
except FileNotFoundError:
print_error(f"Editor [cyan]{editor_cmd[0]}[/] not found")
console.print("Set [bold]$EDITOR[/] environment variable to your preferred editor.")
raise typer.Exit(1) from None
except subprocess.CalledProcessError as e:
print_error(f"Editor exited with error code {e.returncode}")
raise typer.Exit(e.returncode) from None
@config_app.command("show")
def config_show(
path: _PathOption = None,
raw: _RawOption = False,
) -> None:
"""Display the config file location and contents."""
config_file = _get_config_file(path)
if config_file is None:
_report_missing_config()
raise typer.Exit(0)
if not config_file.exists():
_report_missing_config(config_file)
raise typer.Exit(1)
content = config_file.read_text(encoding="utf-8")
if raw:
print(content, end="")
return
from rich.syntax import Syntax # noqa: PLC0415
console.print(f"[bold green]Config file:[/bold green] [cyan]{config_file}[/cyan]")
console.print()
syntax = Syntax(content, "yaml", theme="monokai", line_numbers=True, word_wrap=True)
console.print(syntax)
console.print()
console.print("[dim]Tip: Use -r for copy-paste friendly output[/dim]")
@config_app.command("path")
def config_path(
path: _PathOption = None,
) -> None:
"""Print the config file path (useful for scripting)."""
config_file = _get_config_file(path)
if config_file is None:
_report_missing_config()
raise typer.Exit(1)
# Just print the path for easy piping
print(config_file)
@config_app.command("validate")
def config_validate(
path: _PathOption = None,
) -> None:
"""Validate the config file syntax and schema."""
config_file = _get_config_file(path)
if config_file is None:
print_error(MSG_CONFIG_NOT_FOUND)
raise typer.Exit(1)
# Lazy import: pydantic adds ~50ms to startup, only load when actually needed
from compose_farm.config import load_config # noqa: PLC0415
try:
cfg = load_config(config_file)
except FileNotFoundError as e:
print_error(str(e))
raise typer.Exit(1) from e
except Exception as e:
print_error(f"Invalid config: {e}")
raise typer.Exit(1) from e
print_success(f"Valid config: {config_file}")
console.print(f" Hosts: {len(cfg.hosts)}")
console.print(f" Services: {len(cfg.services)}")
@config_app.command("symlink")
def config_symlink(
target: Annotated[
Path | None,
typer.Argument(help="Config file to link to. Defaults to ./compose-farm.yaml"),
] = None,
force: _ForceOption = False,
) -> None:
"""Create a symlink from the default config location to a config file.
This makes a local config file discoverable globally without copying.
Always uses absolute paths to avoid broken symlinks.
Examples:
cf config symlink # Link to ./compose-farm.yaml
cf config symlink /opt/compose/config.yaml # Link to specific file
"""
# Default to compose-farm.yaml in current directory
target_path = (target or Path("compose-farm.yaml")).expanduser().resolve()
if not target_path.exists():
print_error(f"Target config file not found: {target_path}")
raise typer.Exit(1)
if not target_path.is_file():
print_error(f"Target is not a file: {target_path}")
raise typer.Exit(1)
symlink_path = default_config_path()
# Check if symlink location already exists
if symlink_path.exists() or symlink_path.is_symlink():
if symlink_path.is_symlink():
current_target = symlink_path.resolve() if symlink_path.exists() else None
if current_target == target_path:
print_success(f"Symlink already points to: {target_path}")
return
# Update existing symlink
if not force:
existing = symlink_path.readlink()
console.print(f"[yellow]Symlink exists:[/] {symlink_path} -> {existing}")
if not typer.confirm(f"Update to point to {target_path}?"):
console.print("[dim]Aborted.[/dim]")
raise typer.Exit(0)
symlink_path.unlink()
else:
# Regular file exists
print_error(f"A regular file exists at: {symlink_path}")
console.print(" Back it up or remove it first, then retry.")
raise typer.Exit(1)
# Create parent directories
symlink_path.parent.mkdir(parents=True, exist_ok=True)
# Create symlink with absolute path
symlink_path.symlink_to(target_path)
print_success("Created symlink:")
console.print(f" {symlink_path}")
console.print(f" -> {target_path}")
# Register config subcommand on the shared app
app.add_typer(config_app, name="config", rich_help_panel="Configuration")

View File

@@ -0,0 +1,253 @@
"""Lifecycle commands: up, down, pull, restart, update, apply."""
from __future__ import annotations
from typing import Annotated
import typer
from compose_farm.cli.app import app
from compose_farm.cli.common import (
AllOption,
ConfigOption,
HostOption,
ServicesArg,
format_host,
get_services,
load_config_or_exit,
maybe_regenerate_traefik,
report_results,
run_async,
)
from compose_farm.console import MSG_DRY_RUN, console, print_error, print_success
from compose_farm.executor import run_on_services, run_sequential_on_services
from compose_farm.operations import stop_orphaned_services, up_services
from compose_farm.state import (
get_orphaned_services,
get_service_host,
get_services_needing_migration,
get_services_not_in_state,
remove_service,
)
@app.command(rich_help_panel="Lifecycle")
def up(
services: ServicesArg = None,
all_services: AllOption = False,
host: HostOption = None,
config: ConfigOption = None,
) -> None:
"""Start services (docker compose up -d). Auto-migrates if host changed."""
svc_list, cfg = get_services(services or [], all_services, config, host=host)
results = run_async(up_services(cfg, svc_list, raw=True))
maybe_regenerate_traefik(cfg, results)
report_results(results)
@app.command(rich_help_panel="Lifecycle")
def down(
services: ServicesArg = None,
all_services: AllOption = False,
orphaned: Annotated[
bool,
typer.Option(
"--orphaned", help="Stop orphaned services (in state but removed from config)"
),
] = False,
host: HostOption = None,
config: ConfigOption = None,
) -> None:
"""Stop services (docker compose down)."""
# Handle --orphaned flag (mutually exclusive with other selection methods)
if orphaned:
if services or all_services or host:
print_error(
"Cannot combine [bold]--orphaned[/] with services, [bold]--all[/], or [bold]--host[/]"
)
raise typer.Exit(1)
cfg = load_config_or_exit(config)
orphaned_services = get_orphaned_services(cfg)
if not orphaned_services:
print_success("No orphaned services to stop")
return
console.print(
f"[yellow]Stopping {len(orphaned_services)} orphaned service(s):[/] "
f"{', '.join(orphaned_services.keys())}"
)
results = run_async(stop_orphaned_services(cfg))
report_results(results)
return
svc_list, cfg = get_services(services or [], all_services, config, host=host)
raw = len(svc_list) == 1
results = run_async(run_on_services(cfg, svc_list, "down", raw=raw))
# Remove from state on success
# For multi-host services, result.service is "svc@host", extract base name
removed_services: set[str] = set()
for result in results:
if result.success:
base_service = result.service.split("@")[0]
if base_service not in removed_services:
remove_service(cfg, base_service)
removed_services.add(base_service)
maybe_regenerate_traefik(cfg, results)
report_results(results)
@app.command(rich_help_panel="Lifecycle")
def pull(
services: ServicesArg = None,
all_services: AllOption = False,
config: ConfigOption = None,
) -> None:
"""Pull latest images (docker compose pull)."""
svc_list, cfg = get_services(services or [], all_services, config)
raw = len(svc_list) == 1
results = run_async(run_on_services(cfg, svc_list, "pull", raw=raw))
report_results(results)
@app.command(rich_help_panel="Lifecycle")
def restart(
services: ServicesArg = None,
all_services: AllOption = False,
config: ConfigOption = None,
) -> None:
"""Restart services (down + up)."""
svc_list, cfg = get_services(services or [], all_services, config)
raw = len(svc_list) == 1
results = run_async(run_sequential_on_services(cfg, svc_list, ["down", "up -d"], raw=raw))
maybe_regenerate_traefik(cfg, results)
report_results(results)
@app.command(rich_help_panel="Lifecycle")
def update(
services: ServicesArg = None,
all_services: AllOption = False,
config: ConfigOption = None,
) -> None:
"""Update services (pull + build + down + up)."""
svc_list, cfg = get_services(services or [], all_services, config)
raw = len(svc_list) == 1
results = run_async(
run_sequential_on_services(
cfg, svc_list, ["pull --ignore-buildable", "build", "down", "up -d"], raw=raw
)
)
maybe_regenerate_traefik(cfg, results)
report_results(results)
@app.command(rich_help_panel="Lifecycle")
def apply( # noqa: PLR0912 (multi-phase reconciliation needs these branches)
dry_run: Annotated[
bool,
typer.Option("--dry-run", "-n", help="Show what would change without executing"),
] = False,
no_orphans: Annotated[
bool,
typer.Option("--no-orphans", help="Only migrate, don't stop orphaned services"),
] = False,
full: Annotated[
bool,
typer.Option("--full", "-f", help="Also run up on all services to apply config changes"),
] = False,
config: ConfigOption = None,
) -> None:
"""Make reality match config (start, migrate, stop as needed).
This is the "reconcile" command that ensures running services match your
config file. It will:
1. Stop orphaned services (in state but removed from config)
2. Migrate services on wrong host (host in state ≠ host in config)
3. Start missing services (in config but not in state)
Use --dry-run to preview changes before applying.
Use --no-orphans to only migrate/start without stopping orphaned services.
Use --full to also run 'up' on all services (picks up compose/env changes).
"""
cfg = load_config_or_exit(config)
orphaned = get_orphaned_services(cfg)
migrations = get_services_needing_migration(cfg)
missing = get_services_not_in_state(cfg)
# For --full: refresh all services not already being started/migrated
handled = set(migrations) | set(missing)
to_refresh = [svc for svc in cfg.services if svc not in handled] if full else []
has_orphans = bool(orphaned) and not no_orphans
has_migrations = bool(migrations)
has_missing = bool(missing)
has_refresh = bool(to_refresh)
if not has_orphans and not has_migrations and not has_missing and not has_refresh:
print_success("Nothing to apply - reality matches config")
return
# Report what will be done
if has_orphans:
console.print(f"[yellow]Orphaned services to stop ({len(orphaned)}):[/]")
for svc, hosts in orphaned.items():
console.print(f" [cyan]{svc}[/] on [magenta]{format_host(hosts)}[/]")
if has_migrations:
console.print(f"[cyan]Services to migrate ({len(migrations)}):[/]")
for svc in migrations:
current = get_service_host(cfg, svc)
target = cfg.get_hosts(svc)[0]
console.print(f" [cyan]{svc}[/]: [magenta]{current}[/] → [magenta]{target}[/]")
if has_missing:
console.print(f"[green]Services to start ({len(missing)}):[/]")
for svc in missing:
console.print(f" [cyan]{svc}[/] on [magenta]{format_host(cfg.get_hosts(svc))}[/]")
if has_refresh:
console.print(f"[blue]Services to refresh ({len(to_refresh)}):[/]")
for svc in to_refresh:
console.print(f" [cyan]{svc}[/] on [magenta]{format_host(cfg.get_hosts(svc))}[/]")
if dry_run:
console.print(f"\n{MSG_DRY_RUN}")
return
# Execute changes
console.print()
all_results = []
# 1. Stop orphaned services first
if has_orphans:
console.print("[yellow]Stopping orphaned services...[/]")
all_results.extend(run_async(stop_orphaned_services(cfg)))
# 2. Migrate services on wrong host
if has_migrations:
console.print("[cyan]Migrating services...[/]")
migrate_results = run_async(up_services(cfg, migrations, raw=True))
all_results.extend(migrate_results)
maybe_regenerate_traefik(cfg, migrate_results)
# 3. Start missing services (reuse up_services which handles state updates)
if has_missing:
console.print("[green]Starting missing services...[/]")
start_results = run_async(up_services(cfg, missing, raw=True))
all_results.extend(start_results)
maybe_regenerate_traefik(cfg, start_results)
# 4. Refresh remaining services (--full: run up to apply config changes)
if has_refresh:
console.print("[blue]Refreshing services...[/]")
refresh_results = run_async(up_services(cfg, to_refresh, raw=True))
all_results.extend(refresh_results)
maybe_regenerate_traefik(cfg, refresh_results)
report_results(all_results)
# Alias: cf a = cf apply
app.command("a", hidden=True)(apply)

View File

@@ -0,0 +1,564 @@
"""Management commands: sync, check, init-network, traefik-file."""
from __future__ import annotations
import asyncio
from datetime import UTC, datetime
from pathlib import Path # noqa: TC003
from typing import TYPE_CHECKING, Annotated
import typer
from compose_farm.cli.app import app
from compose_farm.cli.common import (
_MISSING_PATH_PREVIEW_LIMIT,
AllOption,
ConfigOption,
LogPathOption,
ServicesArg,
format_host,
get_services,
load_config_or_exit,
run_async,
run_parallel_with_progress,
validate_hosts,
validate_services,
)
if TYPE_CHECKING:
from compose_farm.config import Config
from compose_farm.console import (
MSG_DRY_RUN,
console,
print_error,
print_success,
print_warning,
)
from compose_farm.executor import (
CommandResult,
is_local,
run_command,
)
from compose_farm.logs import (
DEFAULT_LOG_PATH,
SnapshotEntry,
collect_service_entries,
isoformat,
load_existing_entries,
merge_entries,
write_toml,
)
from compose_farm.operations import (
check_host_compatibility,
check_service_requirements,
discover_service_host,
)
from compose_farm.state import get_orphaned_services, load_state, save_state
from compose_farm.traefik import generate_traefik_config, render_traefik_config
# --- Sync helpers ---
def _discover_services(cfg: Config) -> dict[str, str | list[str]]:
"""Discover running services with a progress bar."""
results = run_parallel_with_progress(
"Discovering",
list(cfg.services),
lambda s: discover_service_host(cfg, s),
)
return {svc: host for svc, host in results if host is not None}
def _snapshot_services(
cfg: Config,
services: list[str],
log_path: Path | None,
) -> Path:
"""Capture image digests with a progress bar."""
effective_log_path = log_path or DEFAULT_LOG_PATH
now_dt = datetime.now(UTC)
now_iso = isoformat(now_dt)
async def collect_service(service: str) -> tuple[str, list[SnapshotEntry]]:
try:
return service, await collect_service_entries(cfg, service, now=now_dt)
except RuntimeError:
return service, []
results = run_parallel_with_progress(
"Capturing",
services,
collect_service,
)
snapshot_entries = [entry for _, entries in results for entry in entries]
if not snapshot_entries:
msg = "No image digests were captured"
raise RuntimeError(msg)
existing_entries = load_existing_entries(effective_log_path)
merged_entries = merge_entries(existing_entries, snapshot_entries, now_iso=now_iso)
meta = {"generated_at": now_iso, "compose_dir": str(cfg.compose_dir)}
write_toml(effective_log_path, meta=meta, entries=merged_entries)
return effective_log_path
def _report_sync_changes(
added: list[str],
removed: list[str],
changed: list[tuple[str, str | list[str], str | list[str]]],
discovered: dict[str, str | list[str]],
current_state: dict[str, str | list[str]],
) -> None:
"""Report sync changes to the user."""
if added:
console.print(f"\nNew services found ({len(added)}):")
for service in sorted(added):
host_str = format_host(discovered[service])
console.print(f" [green]+[/] [cyan]{service}[/] on [magenta]{host_str}[/]")
if changed:
console.print(f"\nServices on different hosts ({len(changed)}):")
for service, old_host, new_host in sorted(changed):
old_str = format_host(old_host)
new_str = format_host(new_host)
console.print(
f" [yellow]~[/] [cyan]{service}[/]: [magenta]{old_str}[/] → [magenta]{new_str}[/]"
)
if removed:
console.print(f"\nServices no longer running ({len(removed)}):")
for service in sorted(removed):
host_str = format_host(current_state[service])
console.print(f" [red]-[/] [cyan]{service}[/] (was on [magenta]{host_str}[/])")
# --- Check helpers ---
def _check_ssh_connectivity(cfg: Config) -> list[str]:
"""Check SSH connectivity to all hosts. Returns list of unreachable hosts."""
# Filter out local hosts - no SSH needed
remote_hosts = [h for h in cfg.hosts if not is_local(cfg.hosts[h])]
if not remote_hosts:
return []
console.print() # Spacing before progress bar
async def check_host(host_name: str) -> tuple[str, bool]:
host = cfg.hosts[host_name]
try:
result = await asyncio.wait_for(
run_command(host, "echo ok", host_name, stream=False),
timeout=5.0,
)
return host_name, result.success
except TimeoutError:
return host_name, False
results = run_parallel_with_progress(
"Checking SSH connectivity",
remote_hosts,
check_host,
)
return [host for host, success in results if not success]
def _check_service_requirements(
cfg: Config,
services: list[str],
) -> tuple[list[tuple[str, str, str]], list[tuple[str, str, str]], list[tuple[str, str, str]]]:
"""Check mounts, networks, and devices for all services with a progress bar.
Returns (mount_errors, network_errors, device_errors) where each is a list of
(service, host, missing_item) tuples.
"""
async def check_service(
service: str,
) -> tuple[
str,
list[tuple[str, str, str]],
list[tuple[str, str, str]],
list[tuple[str, str, str]],
]:
"""Check requirements for a single service on all its hosts."""
host_names = cfg.get_hosts(service)
mount_errors: list[tuple[str, str, str]] = []
network_errors: list[tuple[str, str, str]] = []
device_errors: list[tuple[str, str, str]] = []
for host_name in host_names:
missing_paths, missing_nets, missing_devs = await check_service_requirements(
cfg, service, host_name
)
mount_errors.extend((service, host_name, p) for p in missing_paths)
network_errors.extend((service, host_name, n) for n in missing_nets)
device_errors.extend((service, host_name, d) for d in missing_devs)
return service, mount_errors, network_errors, device_errors
results = run_parallel_with_progress(
"Checking requirements",
services,
check_service,
)
all_mount_errors: list[tuple[str, str, str]] = []
all_network_errors: list[tuple[str, str, str]] = []
all_device_errors: list[tuple[str, str, str]] = []
for _, mount_errs, net_errs, dev_errs in results:
all_mount_errors.extend(mount_errs)
all_network_errors.extend(net_errs)
all_device_errors.extend(dev_errs)
return all_mount_errors, all_network_errors, all_device_errors
def _report_config_status(cfg: Config) -> bool:
"""Check and report config vs disk status. Returns True if errors found."""
configured = set(cfg.services.keys())
on_disk = cfg.discover_compose_dirs()
unmanaged = sorted(on_disk - configured)
missing_from_disk = sorted(configured - on_disk)
if unmanaged:
console.print(f"\n[yellow]Unmanaged[/] (on disk but not in config, {len(unmanaged)}):")
for name in unmanaged:
console.print(f" [yellow]+[/] [cyan]{name}[/]")
if missing_from_disk:
console.print(f"\n[red]In config but no compose file[/] ({len(missing_from_disk)}):")
for name in missing_from_disk:
console.print(f" [red]-[/] [cyan]{name}[/]")
if not unmanaged and not missing_from_disk:
print_success("Config matches disk")
return bool(missing_from_disk)
def _report_orphaned_services(cfg: Config) -> bool:
"""Check for services in state but not in config. Returns True if orphans found."""
orphaned = get_orphaned_services(cfg)
if orphaned:
console.print("\n[yellow]Orphaned services[/] (in state but not in config):")
console.print(
"[dim]Run [bold]cf apply[/bold] to stop them, or [bold]cf down --orphaned[/bold] for just orphans.[/]"
)
for name, hosts in sorted(orphaned.items()):
console.print(f" [yellow]![/] [cyan]{name}[/] on [magenta]{format_host(hosts)}[/]")
return True
return False
def _report_traefik_status(cfg: Config, services: list[str]) -> None:
"""Check and report traefik label status."""
try:
_, warnings = generate_traefik_config(cfg, services, check_all=True)
except (FileNotFoundError, ValueError):
return
if warnings:
console.print(f"\n[yellow]Traefik issues[/] ({len(warnings)}):")
for warning in warnings:
print_warning(warning)
else:
print_success("Traefik labels valid")
def _report_requirement_errors(errors: list[tuple[str, str, str]], category: str) -> None:
"""Report requirement errors (mounts, networks, devices) grouped by service."""
by_service: dict[str, list[tuple[str, str]]] = {}
for svc, host, item in errors:
by_service.setdefault(svc, []).append((host, item))
console.print(f"[red]Missing {category}[/] ({len(errors)}):")
for svc, items in sorted(by_service.items()):
host = items[0][0]
missing = [i for _, i in items]
console.print(f" [cyan]{svc}[/] on [magenta]{host}[/]:")
for item in missing:
console.print(f" [red]✗[/] {item}")
def _report_ssh_status(unreachable_hosts: list[str]) -> bool:
"""Report SSH connectivity status. Returns True if there are errors."""
if unreachable_hosts:
console.print(f"[red]Unreachable hosts[/] ({len(unreachable_hosts)}):")
for host in sorted(unreachable_hosts):
print_error(f"[magenta]{host}[/]")
return True
print_success("All hosts reachable")
return False
def _report_host_compatibility(
compat: dict[str, tuple[int, int, list[str]]],
assigned_hosts: list[str],
) -> None:
"""Report host compatibility for a service."""
for host_name, (found, total, missing) in sorted(compat.items()):
is_assigned = host_name in assigned_hosts
marker = " [dim](assigned)[/]" if is_assigned else ""
if found == total:
console.print(f" [green]✓[/] [magenta]{host_name}[/] {found}/{total}{marker}")
else:
preview = ", ".join(missing[:_MISSING_PATH_PREVIEW_LIMIT])
if len(missing) > _MISSING_PATH_PREVIEW_LIMIT:
preview += f", +{len(missing) - _MISSING_PATH_PREVIEW_LIMIT} more"
console.print(
f" [red]✗[/] [magenta]{host_name}[/] {found}/{total} "
f"[dim](missing: {preview})[/]{marker}"
)
def _run_remote_checks(cfg: Config, svc_list: list[str], *, show_host_compat: bool) -> bool:
"""Run SSH-based checks for mounts, networks, and host compatibility.
Returns True if any errors were found.
"""
has_errors = False
# Check SSH connectivity first
if _report_ssh_status(_check_ssh_connectivity(cfg)):
has_errors = True
console.print() # Spacing before mounts/networks check
# Check mounts, networks, and devices
mount_errors, network_errors, device_errors = _check_service_requirements(cfg, svc_list)
if mount_errors:
_report_requirement_errors(mount_errors, "mounts")
has_errors = True
if network_errors:
_report_requirement_errors(network_errors, "networks")
has_errors = True
if device_errors:
_report_requirement_errors(device_errors, "devices")
has_errors = True
if not mount_errors and not network_errors and not device_errors:
print_success("All mounts, networks, and devices exist")
if show_host_compat:
for service in svc_list:
console.print(f"\n[bold]Host compatibility for[/] [cyan]{service}[/]:")
compat = run_async(check_host_compatibility(cfg, service))
assigned_hosts = cfg.get_hosts(service)
_report_host_compatibility(compat, assigned_hosts)
return has_errors
# Default network settings for cross-host Docker networking
_DEFAULT_NETWORK_NAME = "mynetwork"
_DEFAULT_NETWORK_SUBNET = "172.20.0.0/16"
_DEFAULT_NETWORK_GATEWAY = "172.20.0.1"
@app.command("traefik-file", rich_help_panel="Configuration")
def traefik_file(
services: ServicesArg = None,
all_services: AllOption = False,
output: Annotated[
Path | None,
typer.Option(
"--output",
"-o",
help="Write Traefik file-provider YAML to this path (stdout if omitted)",
),
] = None,
config: ConfigOption = None,
) -> None:
"""Generate a Traefik file-provider fragment from compose Traefik labels."""
svc_list, cfg = get_services(services or [], all_services, config)
try:
dynamic, warnings = generate_traefik_config(cfg, svc_list)
except (FileNotFoundError, ValueError) as exc:
print_error(str(exc))
raise typer.Exit(1) from exc
rendered = render_traefik_config(dynamic)
if output:
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(rendered)
print_success(f"Traefik config written to {output}")
else:
console.print(rendered)
for warning in warnings:
print_warning(warning)
@app.command(rich_help_panel="Configuration")
def refresh(
config: ConfigOption = None,
log_path: LogPathOption = None,
dry_run: Annotated[
bool,
typer.Option("--dry-run", "-n", help="Show what would change without writing"),
] = False,
) -> None:
"""Update local state from running services.
Discovers which services are running on which hosts, updates the state
file, and captures image digests. This is a read operation - it updates
your local state to match reality, not the other way around.
Use 'cf apply' to make reality match your config (stop orphans, migrate).
"""
cfg = load_config_or_exit(config)
current_state = load_state(cfg)
discovered = _discover_services(cfg)
# Calculate changes
added = [s for s in discovered if s not in current_state]
removed = [s for s in current_state if s not in discovered]
changed = [
(s, current_state[s], discovered[s])
for s in discovered
if s in current_state and current_state[s] != discovered[s]
]
# Report state changes
state_changed = bool(added or removed or changed)
if state_changed:
_report_sync_changes(added, removed, changed, discovered, current_state)
else:
print_success("State is already in sync.")
if dry_run:
console.print(f"\n{MSG_DRY_RUN}")
return
# Update state file
if state_changed:
save_state(cfg, discovered)
print_success(f"State updated: {len(discovered)} services tracked.")
# Capture image digests for running services
if discovered:
try:
path = _snapshot_services(cfg, list(discovered.keys()), log_path)
print_success(f"Digests written to {path}")
except RuntimeError as exc:
print_warning(str(exc))
@app.command(rich_help_panel="Configuration")
def check(
services: ServicesArg = None,
local: Annotated[
bool,
typer.Option("--local", help="Skip SSH-based checks (faster)"),
] = False,
config: ConfigOption = None,
) -> None:
"""Validate configuration, traefik labels, mounts, and networks.
Without arguments: validates all services against configured hosts.
With service arguments: validates specific services and shows host compatibility.
Use --local to skip SSH-based checks for faster validation.
"""
cfg = load_config_or_exit(config)
# Determine which services to check and whether to show host compatibility
if services:
svc_list = list(services)
validate_services(cfg, svc_list)
show_host_compat = True
else:
svc_list = list(cfg.services.keys())
show_host_compat = False
# Run checks
has_errors = _report_config_status(cfg)
_report_traefik_status(cfg, svc_list)
if not local and _run_remote_checks(cfg, svc_list, show_host_compat=show_host_compat):
has_errors = True
# Check for orphaned services (in state but removed from config)
if _report_orphaned_services(cfg):
has_errors = True
if has_errors:
raise typer.Exit(1)
@app.command("init-network", rich_help_panel="Configuration")
def init_network(
hosts: Annotated[
list[str] | None,
typer.Argument(help="Hosts to create network on (default: all)"),
] = None,
network: Annotated[
str,
typer.Option("--network", "-n", help="Network name"),
] = _DEFAULT_NETWORK_NAME,
subnet: Annotated[
str,
typer.Option("--subnet", "-s", help="Network subnet"),
] = _DEFAULT_NETWORK_SUBNET,
gateway: Annotated[
str,
typer.Option("--gateway", "-g", help="Network gateway"),
] = _DEFAULT_NETWORK_GATEWAY,
config: ConfigOption = None,
) -> None:
"""Create Docker network on hosts with consistent settings.
Creates an external Docker network that services can use for cross-host
communication. Uses the same subnet/gateway on all hosts to ensure
consistent networking.
"""
cfg = load_config_or_exit(config)
target_hosts = list(hosts) if hosts else list(cfg.hosts.keys())
validate_hosts(cfg, target_hosts)
async def create_network_on_host(host_name: str) -> CommandResult:
host = cfg.hosts[host_name]
# Check if network already exists
check_cmd = f"docker network inspect '{network}' >/dev/null 2>&1"
check_result = await run_command(host, check_cmd, host_name, stream=False)
if check_result.success:
console.print(f"[cyan]\\[{host_name}][/] Network '{network}' already exists")
return CommandResult(service=host_name, exit_code=0, success=True)
# Create the network
create_cmd = (
f"docker network create "
f"--driver bridge "
f"--subnet '{subnet}' "
f"--gateway '{gateway}' "
f"'{network}'"
)
result = await run_command(host, create_cmd, host_name, stream=False)
if result.success:
console.print(f"[cyan]\\[{host_name}][/] [green]✓[/] Created network '{network}'")
else:
print_error(
f"[cyan]\\[{host_name}][/] Failed to create network: {result.stderr.strip()}"
)
return result
async def run_all() -> list[CommandResult]:
return await asyncio.gather(*[create_network_on_host(h) for h in target_hosts])
results = run_async(run_all())
failed = [r for r in results if not r.success]
if failed:
raise typer.Exit(1)

View File

@@ -0,0 +1,190 @@
"""Monitoring commands: logs, ps, stats."""
from __future__ import annotations
import contextlib
from typing import TYPE_CHECKING, Annotated
import typer
from rich.table import Table
from compose_farm.cli.app import app
from compose_farm.cli.common import (
_STATS_PREVIEW_LIMIT,
AllOption,
ConfigOption,
HostOption,
ServicesArg,
get_services,
load_config_or_exit,
report_results,
run_async,
run_parallel_with_progress,
)
from compose_farm.console import console
from compose_farm.executor import run_command, run_on_services
from compose_farm.state import get_services_needing_migration, group_services_by_host, load_state
if TYPE_CHECKING:
from compose_farm.config import Config
def _get_container_counts(cfg: Config) -> dict[str, int]:
"""Get container counts from all hosts with a progress bar."""
async def get_count(host_name: str) -> tuple[str, int]:
host = cfg.hosts[host_name]
result = await run_command(host, "docker ps -q | wc -l", host_name, stream=False)
count = 0
if result.success:
with contextlib.suppress(ValueError):
count = int(result.stdout.strip())
return host_name, count
results = run_parallel_with_progress(
"Querying hosts",
list(cfg.hosts.keys()),
get_count,
)
return dict(results)
def _build_host_table(
cfg: Config,
services_by_host: dict[str, list[str]],
running_by_host: dict[str, list[str]],
container_counts: dict[str, int],
*,
show_containers: bool,
) -> Table:
"""Build the hosts table."""
table = Table(title="Hosts", show_header=True, header_style="bold cyan")
table.add_column("Host", style="magenta")
table.add_column("Address")
table.add_column("Configured", justify="right")
table.add_column("Running", justify="right")
if show_containers:
table.add_column("Containers", justify="right")
for host_name in sorted(cfg.hosts.keys()):
host = cfg.hosts[host_name]
configured = len(services_by_host[host_name])
running = len(running_by_host[host_name])
row = [
host_name,
host.address,
str(configured),
str(running) if running > 0 else "[dim]0[/]",
]
if show_containers:
count = container_counts.get(host_name, 0)
row.append(str(count) if count > 0 else "[dim]0[/]")
table.add_row(*row)
return table
def _build_summary_table(
cfg: Config, state: dict[str, str | list[str]], pending: list[str]
) -> Table:
"""Build the summary table."""
on_disk = cfg.discover_compose_dirs()
table = Table(title="Summary", show_header=False)
table.add_column("Label", style="dim")
table.add_column("Value", style="bold")
table.add_row("Total hosts", str(len(cfg.hosts)))
table.add_row("Services (configured)", str(len(cfg.services)))
table.add_row("Services (tracked)", str(len(state)))
table.add_row("Compose files on disk", str(len(on_disk)))
if pending:
preview = ", ".join(pending[:_STATS_PREVIEW_LIMIT])
suffix = "..." if len(pending) > _STATS_PREVIEW_LIMIT else ""
table.add_row("Pending migrations", f"[yellow]{len(pending)}[/] ({preview}{suffix})")
else:
table.add_row("Pending migrations", "[green]0[/]")
return table
# --- Command functions ---
@app.command(rich_help_panel="Monitoring")
def logs(
services: ServicesArg = None,
all_services: AllOption = False,
host: HostOption = None,
follow: Annotated[bool, typer.Option("--follow", "-f", help="Follow logs")] = False,
tail: Annotated[
int | None,
typer.Option("--tail", "-n", help="Number of lines (default: 20 for --all, 100 otherwise)"),
] = None,
config: ConfigOption = None,
) -> None:
"""Show service logs."""
svc_list, cfg = get_services(services or [], all_services, config, host=host)
# Default to fewer lines when showing multiple services
many_services = all_services or host is not None or len(svc_list) > 1
effective_tail = tail if tail is not None else (20 if many_services else 100)
cmd = f"logs --tail {effective_tail}"
if follow:
cmd += " -f"
results = run_async(run_on_services(cfg, svc_list, cmd))
report_results(results)
@app.command(rich_help_panel="Monitoring")
def ps(
services: ServicesArg = None,
all_services: AllOption = False,
host: HostOption = None,
config: ConfigOption = None,
) -> None:
"""Show status of services.
Without arguments: shows all services (same as --all).
With service names: shows only those services.
With --host: shows services on that host.
"""
svc_list, cfg = get_services(services or [], all_services, config, host=host, default_all=True)
results = run_async(run_on_services(cfg, svc_list, "ps"))
report_results(results)
@app.command(rich_help_panel="Monitoring")
def stats(
live: Annotated[
bool,
typer.Option("--live", "-l", help="Query Docker for live container stats"),
] = False,
config: ConfigOption = None,
) -> None:
"""Show overview statistics for hosts and services.
Without --live: Shows config/state info (hosts, services, pending migrations).
With --live: Also queries Docker on each host for container counts.
"""
cfg = load_config_or_exit(config)
state = load_state(cfg)
pending = get_services_needing_migration(cfg)
all_hosts = list(cfg.hosts.keys())
services_by_host = group_services_by_host(cfg.services, cfg.hosts, all_hosts)
running_by_host = group_services_by_host(state, cfg.hosts, all_hosts)
container_counts: dict[str, int] = {}
if live:
container_counts = _get_container_counts(cfg)
host_table = _build_host_table(
cfg, services_by_host, running_by_host, container_counts, show_containers=live
)
console.print(host_table)
console.print()
console.print(_build_summary_table(cfg, state, pending))

282
src/compose_farm/cli/ssh.py Normal file
View File

@@ -0,0 +1,282 @@
"""SSH key management commands for compose-farm."""
from __future__ import annotations
import asyncio
import subprocess
from typing import TYPE_CHECKING, Annotated
import typer
from compose_farm.cli.app import app
from compose_farm.cli.common import ConfigOption, load_config_or_exit, run_parallel_with_progress
from compose_farm.console import console, err_console
from compose_farm.executor import run_command
if TYPE_CHECKING:
from compose_farm.config import Host
from compose_farm.ssh_keys import (
SSH_KEY_PATH,
SSH_PUBKEY_PATH,
get_pubkey_content,
get_ssh_env,
key_exists,
)
_DEFAULT_SSH_PORT = 22
_PUBKEY_DISPLAY_THRESHOLD = 60
ssh_app = typer.Typer(
name="ssh",
help="Manage SSH keys for passwordless authentication.",
no_args_is_help=True,
)
_ForceOption = Annotated[
bool,
typer.Option("--force", "-f", help="Regenerate key even if it exists."),
]
def _generate_key(*, force: bool = False) -> bool:
"""Generate an ED25519 SSH key with no passphrase.
Returns True if key was generated, False if skipped.
"""
if key_exists() and not force:
console.print(f"[yellow]![/] SSH key already exists: {SSH_KEY_PATH}")
console.print("[dim]Use --force to regenerate[/]")
return False
# Create .ssh directory if it doesn't exist
SSH_KEY_PATH.parent.mkdir(parents=True, exist_ok=True, mode=0o700)
# Remove existing key if forcing regeneration
if force:
SSH_KEY_PATH.unlink(missing_ok=True)
SSH_PUBKEY_PATH.unlink(missing_ok=True)
console.print(f"[dim]Generating SSH key at {SSH_KEY_PATH}...[/]")
try:
subprocess.run(
[ # noqa: S607
"ssh-keygen",
"-t",
"ed25519",
"-N",
"", # No passphrase
"-f",
str(SSH_KEY_PATH),
"-C",
"compose-farm",
],
check=True,
capture_output=True,
)
except subprocess.CalledProcessError as e:
err_console.print(f"[red]Failed to generate SSH key:[/] {e.stderr.decode()}")
return False
except FileNotFoundError:
err_console.print("[red]ssh-keygen not found. Is OpenSSH installed?[/]")
return False
# Set correct permissions
SSH_KEY_PATH.chmod(0o600)
SSH_PUBKEY_PATH.chmod(0o644)
console.print(f"[green]Generated SSH key:[/] {SSH_KEY_PATH}")
return True
def _copy_key_to_host(host_name: str, address: str, user: str, port: int) -> bool:
"""Copy public key to a host's authorized_keys.
Uses ssh-copy-id which handles agent vs password fallback automatically.
Returns True on success, False on failure.
"""
target = f"{user}@{address}"
console.print(f"[dim]Copying key to {host_name} ({target})...[/]")
cmd = ["ssh-copy-id"]
# Disable strict host key checking (consistent with executor.py)
cmd.extend(["-o", "StrictHostKeyChecking=no"])
cmd.extend(["-o", "UserKnownHostsFile=/dev/null"])
if port != _DEFAULT_SSH_PORT:
cmd.extend(["-p", str(port)])
cmd.extend(["-i", str(SSH_PUBKEY_PATH), target])
try:
# Don't capture output so user can see password prompt
result = subprocess.run(cmd, check=False, env=get_ssh_env())
if result.returncode == 0:
console.print(f"[green]Key copied to {host_name}[/]")
return True
err_console.print(f"[red]Failed to copy key to {host_name}[/]")
return False
except FileNotFoundError:
err_console.print("[red]ssh-copy-id not found. Is OpenSSH installed?[/]")
return False
@ssh_app.command("keygen")
def ssh_keygen(
force: _ForceOption = False,
) -> None:
"""Generate SSH key (does not distribute to hosts).
Creates an ED25519 key at ~/.ssh/compose-farm/id_ed25519 with no passphrase.
Use 'cf ssh setup' to also distribute the key to all configured hosts.
"""
success = _generate_key(force=force)
if not success and not key_exists():
raise typer.Exit(1)
@ssh_app.command("setup")
def ssh_setup(
config: ConfigOption = None,
force: _ForceOption = False,
) -> None:
"""Generate SSH key and distribute to all configured hosts.
Creates an ED25519 key at ~/.ssh/compose-farm/id_ed25519 (no passphrase)
and copies the public key to authorized_keys on each host.
For each host, tries SSH agent first. If agent is unavailable,
prompts for password.
"""
cfg = load_config_or_exit(config)
# Skip localhost hosts
remote_hosts = {
name: host
for name, host in cfg.hosts.items()
if host.address.lower() not in ("localhost", "127.0.0.1")
}
if not remote_hosts:
console.print("[yellow]No remote hosts configured.[/]")
raise typer.Exit(0)
# Generate key if needed
if not key_exists() or force:
if not _generate_key(force=force):
raise typer.Exit(1)
else:
console.print(f"[dim]Using existing key: {SSH_KEY_PATH}[/]")
console.print()
console.print(f"[bold]Distributing key to {len(remote_hosts)} host(s)...[/]")
console.print()
# Copy key to each host
succeeded = 0
failed = 0
for host_name, host in remote_hosts.items():
if _copy_key_to_host(host_name, host.address, host.user, host.port):
succeeded += 1
else:
failed += 1
console.print()
if failed == 0:
console.print(
f"[green]Setup complete.[/] {succeeded}/{len(remote_hosts)} hosts configured."
)
else:
console.print(
f"[yellow]Setup partially complete.[/] {succeeded}/{len(remote_hosts)} hosts configured, "
f"[red]{failed} failed[/]."
)
raise typer.Exit(1)
@ssh_app.command("status")
def ssh_status(
config: ConfigOption = None,
) -> None:
"""Show SSH key status and host connectivity."""
from rich.table import Table # noqa: PLC0415
cfg = load_config_or_exit(config)
# Key status
console.print("[bold]SSH Key Status[/]")
console.print()
if key_exists():
console.print(f" [green]Key exists:[/] {SSH_KEY_PATH}")
pubkey = get_pubkey_content()
if pubkey:
# Show truncated public key
if len(pubkey) > _PUBKEY_DISPLAY_THRESHOLD:
console.print(f" [dim]Public key:[/] {pubkey[:30]}...{pubkey[-20:]}")
else:
console.print(f" [dim]Public key:[/] {pubkey}")
else:
console.print(f" [yellow]No key found:[/] {SSH_KEY_PATH}")
console.print(" [dim]Run 'cf ssh setup' to generate and distribute a key[/]")
console.print()
console.print("[bold]Host Connectivity[/]")
console.print()
# Skip localhost hosts
remote_hosts = {
name: host
for name, host in cfg.hosts.items()
if host.address.lower() not in ("localhost", "127.0.0.1")
}
if not remote_hosts:
console.print(" [dim]No remote hosts configured[/]")
return
async def check_host(item: tuple[str, Host]) -> tuple[str, str, str]:
"""Check connectivity to a single host."""
host_name, host = item
target = f"{host.user}@{host.address}"
if host.port != _DEFAULT_SSH_PORT:
target += f":{host.port}"
try:
result = await asyncio.wait_for(
run_command(host, "echo ok", host_name, stream=False),
timeout=5.0,
)
status = "[green]OK[/]" if result.success else "[red]Auth failed[/]"
except TimeoutError:
status = "[red]Timeout (5s)[/]"
except Exception as e:
status = f"[red]Error: {e}[/]"
return host_name, target, status
# Check connectivity in parallel with progress bar
results = run_parallel_with_progress(
"Checking hosts",
list(remote_hosts.items()),
check_host,
)
# Build table from results
table = Table(show_header=True, header_style="bold")
table.add_column("Host")
table.add_column("Address")
table.add_column("Status")
# Sort by host name for consistent order
for host_name, target, status in sorted(results, key=lambda r: r[0]):
table.add_row(host_name, target, status)
console.print(table)
# Register ssh subcommand on the shared app
app.add_typer(ssh_app, name="ssh", rich_help_panel="Configuration")

View File

@@ -0,0 +1,48 @@
"""Web server command."""
from __future__ import annotations
from typing import Annotated
import typer
from compose_farm.cli.app import app
from compose_farm.console import console
@app.command(rich_help_panel="Server")
def web(
host: Annotated[
str,
typer.Option("--host", "-H", help="Host to bind to"),
] = "0.0.0.0", # noqa: S104
port: Annotated[
int,
typer.Option("--port", "-p", help="Port to listen on"),
] = 8000,
reload: Annotated[
bool,
typer.Option("--reload", "-r", help="Enable auto-reload for development"),
] = False,
) -> None:
"""Start the web UI server."""
try:
import uvicorn # noqa: PLC0415
except ImportError:
console.print(
"[red]Error:[/] Web dependencies not installed. "
"Install with: [cyan]pip install compose-farm[web][/]"
)
raise typer.Exit(1) from None
console.print(f"[green]Starting Compose Farm Web UI[/] at http://{host}:{port}")
console.print("[dim]Press Ctrl+C to stop[/]")
uvicorn.run(
"compose_farm.web:create_app",
factory=True,
host=host,
port=port,
reload=reload,
log_level="info",
)

View File

@@ -7,14 +7,14 @@ from __future__ import annotations
import os
import re
import stat
from dataclasses import dataclass
from pathlib import Path
from typing import TYPE_CHECKING, Any
import yaml
if TYPE_CHECKING:
from pathlib import Path
from .config import Config
# Port parsing constants
@@ -141,23 +141,42 @@ def _resolve_host_path(host_path: str, compose_dir: Path) -> str | None:
return None # Named volume
def _is_socket(path: str) -> bool:
"""Check if a path is a socket (e.g., SSH agent socket)."""
try:
return stat.S_ISSOCK(Path(path).stat().st_mode)
except (FileNotFoundError, PermissionError, OSError):
return False
def _parse_volume_item(
item: str | dict[str, Any],
env: dict[str, str],
compose_dir: Path,
) -> str | None:
"""Parse a single volume item and return host path if it's a bind mount."""
"""Parse a single volume item and return host path if it's a bind mount.
Skips socket paths (e.g., SSH_AUTH_SOCK) since they're machine-local
and shouldn't be validated on remote hosts.
"""
host_path: str | None = None
if isinstance(item, str):
interpolated = _interpolate(item, env)
parts = interpolated.split(":")
if len(parts) >= _MIN_VOLUME_PARTS:
return _resolve_host_path(parts[0], compose_dir)
host_path = _resolve_host_path(parts[0], compose_dir)
elif isinstance(item, dict) and item.get("type") == "bind":
source = item.get("source")
if source:
interpolated = _interpolate(str(source), env)
return _resolve_host_path(interpolated, compose_dir)
return None
host_path = _resolve_host_path(interpolated, compose_dir)
# Skip sockets - they're machine-local (e.g., SSH agent)
if host_path and _is_socket(host_path):
return None
return host_path
def parse_host_volumes(config: Config, service: str) -> list[str]:
@@ -194,13 +213,46 @@ def parse_host_volumes(config: Config, service: str) -> list[str]:
paths.append(host_path)
# Return unique paths, preserving order
seen: set[str] = set()
unique: list[str] = []
for p in paths:
if p not in seen:
seen.add(p)
unique.append(p)
return unique
return list(dict.fromkeys(paths))
def parse_devices(config: Config, service: str) -> list[str]:
"""Extract host device paths from a service's compose file.
Returns a list of host device paths (e.g., /dev/dri, /dev/dri/renderD128).
"""
compose_path = config.get_compose_path(service)
if not compose_path.exists():
return []
env = _load_env(compose_path)
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
return []
devices: list[str] = []
for definition in raw_services.values():
if not isinstance(definition, dict):
continue
device_list = definition.get("devices")
if not device_list or not isinstance(device_list, list):
continue
for item in device_list:
if not isinstance(item, str):
continue
interpolated = _interpolate(item, env)
# Format: host_path:container_path[:options]
parts = interpolated.split(":")
if parts:
host_path = parts[0]
if host_path.startswith("/dev/"):
devices.append(host_path)
# Return unique devices, preserving order
return list(dict.fromkeys(devices))
def parse_external_networks(config: Config, service: str) -> list[str]:

View File

@@ -3,16 +3,15 @@
from __future__ import annotations
import getpass
import os
from pathlib import Path
import yaml
from pydantic import BaseModel, Field, model_validator
from .paths import config_search_paths, find_config_path
def xdg_config_home() -> Path:
"""Get XDG config directory, respecting XDG_CONFIG_HOME env var."""
return Path(os.environ.get("XDG_CONFIG_HOME", Path.home() / ".config"))
# Supported compose filenames, in priority order
COMPOSE_FILENAMES = ("compose.yaml", "compose.yml", "docker-compose.yml", "docker-compose.yaml")
class Host(BaseModel):
@@ -94,17 +93,9 @@ class Config(BaseModel):
return self.hosts[host_names[0]]
def get_compose_path(self, service: str) -> Path:
"""Get compose file path for a service.
Tries compose.yaml first, then docker-compose.yml.
"""
"""Get compose file path for a service (tries compose.yaml first)."""
service_dir = self.compose_dir / service
for filename in (
"compose.yaml",
"compose.yml",
"docker-compose.yml",
"docker-compose.yaml",
):
for filename in COMPOSE_FILENAMES:
candidate = service_dir / filename
if candidate.exists():
return candidate
@@ -113,21 +104,12 @@ class Config(BaseModel):
def discover_compose_dirs(self) -> set[str]:
"""Find all directories in compose_dir that contain a compose file."""
compose_filenames = {
"compose.yaml",
"compose.yml",
"docker-compose.yml",
"docker-compose.yaml",
}
found: set[str] = set()
if not self.compose_dir.exists():
return found
for subdir in self.compose_dir.iterdir():
if subdir.is_dir():
for filename in compose_filenames:
if (subdir / filename).exists():
found.add(subdir.name)
break
if subdir.is_dir() and any((subdir / f).exists() for f in COMPOSE_FILENAMES):
found.add(subdir.name)
return found
@@ -148,26 +130,22 @@ def load_config(path: Path | None = None) -> Config:
"""Load configuration from YAML file.
Search order:
1. Explicit path if provided
2. ./compose-farm.yaml
3. $XDG_CONFIG_HOME/compose-farm/compose-farm.yaml (defaults to ~/.config)
1. Explicit path if provided via --config
2. CF_CONFIG environment variable
3. ./compose-farm.yaml
4. $XDG_CONFIG_HOME/compose-farm/compose-farm.yaml (defaults to ~/.config)
"""
search_paths = [
Path("compose-farm.yaml"),
xdg_config_home() / "compose-farm" / "compose-farm.yaml",
]
if path:
config_path = path
else:
config_path = None
for p in search_paths:
if p.exists():
config_path = p
break
config_path = path or find_config_path()
if config_path is None or not config_path.exists():
msg = f"Config file not found. Searched: {', '.join(str(p) for p in search_paths)}"
msg = f"Config file not found. Searched: {', '.join(str(p) for p in config_search_paths())}"
raise FileNotFoundError(msg)
if config_path.is_dir():
msg = (
f"Config path is a directory, not a file: {config_path}\n"
"This often happens when Docker creates an empty directory for a missing mount."
)
raise FileNotFoundError(msg)
with config_path.open() as f:

View File

@@ -4,3 +4,35 @@ from rich.console import Console
console = Console(highlight=False)
err_console = Console(stderr=True, highlight=False)
# --- Message Constants ---
# Standardized message templates for consistent user-facing output
MSG_SERVICE_NOT_FOUND = "Service [cyan]{name}[/] not found in config"
MSG_HOST_NOT_FOUND = "Host [magenta]{name}[/] not found in config"
MSG_CONFIG_NOT_FOUND = "Config file not found"
MSG_DRY_RUN = "[dim](dry-run: no changes made)[/]"
# --- Message Helper Functions ---
def print_error(msg: str) -> None:
"""Print error message with ✗ prefix to stderr."""
err_console.print(f"[red]✗[/] {msg}")
def print_success(msg: str) -> None:
"""Print success message with ✓ prefix to stdout."""
console.print(f"[green]✓[/] {msg}")
def print_warning(msg: str) -> None:
"""Print warning message with ! prefix to stderr."""
err_console.print(f"[yellow]![/] {msg}")
def print_hint(msg: str) -> None:
"""Print hint message in dim style to stdout."""
console.print(f"[dim]Hint: {msg}[/]")

View File

@@ -0,0 +1,89 @@
# Compose Farm configuration
# Documentation: https://github.com/basnijholt/compose-farm
#
# This file configures compose-farm to manage Docker Compose services
# across multiple hosts via SSH.
#
# Place this file at:
# - ./compose-farm.yaml (current directory)
# - ~/.config/compose-farm/compose-farm.yaml
# - Or specify with: cf --config /path/to/config.yaml
# - Or set CF_CONFIG environment variable
# ------------------------------------------------------------------------------
# compose_dir: Directory containing service subdirectories with compose files
# ------------------------------------------------------------------------------
# Each subdirectory should contain a compose.yaml (or docker-compose.yml).
# This path must be the same on all hosts (NFS mount recommended).
#
compose_dir: /opt/compose
# ------------------------------------------------------------------------------
# hosts: SSH connection details for each host
# ------------------------------------------------------------------------------
# Simple form:
# hostname: ip-or-fqdn
#
# Full form:
# hostname:
# address: ip-or-fqdn
# user: ssh-username # default: current user
# port: 22 # default: 22
#
# Note: "all" is a reserved keyword and cannot be used as a host name.
#
hosts:
# Example: simple form (uses current user, port 22)
server1: 192.168.1.10
# Example: full form with explicit user
server2:
address: 192.168.1.20
user: admin
# Example: full form with custom port
server3:
address: 192.168.1.30
user: root
port: 2222
# ------------------------------------------------------------------------------
# services: Map service names to their target host(s)
# ------------------------------------------------------------------------------
# Each service name must match a subdirectory in compose_dir.
#
# Single host:
# service-name: hostname
#
# Multiple hosts (explicit list):
# service-name: [host1, host2]
#
# All hosts:
# service-name: all
#
services:
# Example: service runs on a single host
nginx: server1
postgres: server2
# Example: service runs on multiple specific hosts
# prometheus: [server1, server2]
# Example: service runs on ALL hosts (e.g., monitoring agents)
# node-exporter: all
# ------------------------------------------------------------------------------
# traefik_file: (optional) Auto-generate Traefik file-provider config
# ------------------------------------------------------------------------------
# When set, compose-farm automatically regenerates this file after
# up/down/restart/update commands. Traefik watches this file for changes.
#
# traefik_file: /opt/compose/traefik/dynamic.d/compose-farm.yml
# ------------------------------------------------------------------------------
# traefik_service: (optional) Service name running Traefik
# ------------------------------------------------------------------------------
# When generating traefik_file, services on the same host as Traefik are
# skipped (they're handled by Traefik's Docker provider directly).
#
# traefik_service: traefik

View File

@@ -9,10 +9,10 @@ from dataclasses import dataclass
from functools import lru_cache
from typing import TYPE_CHECKING, Any
import asyncssh
from rich.markup import escape
from .console import console, err_console
from .ssh_keys import get_key_path, get_ssh_auth_sock, get_ssh_env
if TYPE_CHECKING:
from collections.abc import Callable
@@ -23,6 +23,85 @@ LOCAL_ADDRESSES = frozenset({"local", "localhost", "127.0.0.1", "::1"})
_DEFAULT_SSH_PORT = 22
def _print_compose_command(
host_name: str,
compose_dir: str,
compose_path: str,
compose_cmd: str,
) -> None:
"""Print the docker compose command being executed.
Shows the host and a simplified command with relative path from compose_dir.
"""
# Show relative path from compose_dir for cleaner output
if compose_path.startswith(compose_dir):
rel_path = compose_path[len(compose_dir) :].lstrip("/")
else:
rel_path = compose_path
console.print(
f"[dim][magenta]{host_name}[/magenta]: docker compose -f {rel_path} {compose_cmd}[/dim]"
)
async def _stream_output_lines(
reader: Any,
prefix: str,
*,
is_stderr: bool = False,
) -> None:
"""Stream lines from a reader to console with a service prefix.
Works with both asyncio.StreamReader (bytes) and asyncssh readers (str).
If prefix is empty, output is printed without a prefix.
"""
out = err_console if is_stderr else console
async for line in reader:
text = line.decode() if isinstance(line, bytes) else line
if text.strip():
if prefix:
out.print(f"[cyan]\\[{prefix}][/] {escape(text)}", end="")
else:
out.print(escape(text), end="")
def build_ssh_command(host: Host, command: str, *, tty: bool = False) -> list[str]:
"""Build SSH command args for executing a command on a remote host.
Args:
host: Host configuration with address, port, user
command: Command to run on the remote host
tty: Whether to allocate a TTY (for interactive/progress bar commands)
Returns:
List of command args suitable for subprocess
"""
ssh_args = [
"ssh",
"-o",
"StrictHostKeyChecking=no",
"-o",
"UserKnownHostsFile=/dev/null",
"-o",
"LogLevel=ERROR",
]
if tty:
ssh_args.insert(1, "-tt") # Force TTY allocation
key_path = get_key_path()
if key_path:
ssh_args.extend(["-i", str(key_path)])
if host.port != _DEFAULT_SSH_PORT:
ssh_args.extend(["-p", str(host.port)])
ssh_args.append(f"{host.user}@{host.address}")
ssh_args.append(command)
return ssh_args
@lru_cache(maxsize=1)
def _get_local_ips() -> frozenset[str]:
"""Get all IP addresses of the current machine."""
@@ -53,6 +132,15 @@ class CommandResult:
stdout: str = ""
stderr: str = ""
# SSH returns 255 when connection is closed unexpectedly (e.g., Ctrl+C)
_SSH_CONNECTION_CLOSED = 255
@property
def interrupted(self) -> bool:
"""Check if command was killed by SIGINT (Ctrl+C)."""
# Negative exit codes indicate signal termination; -2 = SIGINT
return self.exit_code < 0 or self.exit_code == self._SSH_CONNECTION_CLOSED
def is_local(host: Host) -> bool:
"""Check if host should run locally (no SSH)."""
@@ -63,12 +151,32 @@ def is_local(host: Host) -> bool:
return addr in _get_local_ips()
def ssh_connect_kwargs(host: Host) -> dict[str, Any]:
"""Get kwargs for asyncssh.connect() from a Host config."""
kwargs: dict[str, Any] = {
"host": host.address,
"port": host.port,
"username": host.user,
"known_hosts": None,
}
# Add SSH agent path (auto-detect forwarded agent if needed)
agent_path = get_ssh_auth_sock()
if agent_path:
kwargs["agent_path"] = agent_path
# Add key file fallback for when SSH agent is unavailable
key_path = get_key_path()
if key_path:
kwargs["client_keys"] = [str(key_path)]
return kwargs
async def _run_local_command(
command: str,
service: str,
*,
stream: bool = True,
raw: bool = False,
prefix: str = "",
) -> CommandResult:
"""Run a command locally with streaming output."""
try:
@@ -93,25 +201,9 @@ async def _run_local_command(
)
if stream and proc.stdout and proc.stderr:
async def read_stream(
reader: asyncio.StreamReader,
prefix: str,
*,
is_stderr: bool = False,
) -> None:
out = err_console if is_stderr else console
while True:
line = await reader.readline()
if not line:
break
text = line.decode()
if text.strip(): # Skip empty lines
out.print(f"[cyan]\\[{prefix}][/] {escape(text)}", end="")
await asyncio.gather(
read_stream(proc.stdout, service),
read_stream(proc.stderr, service, is_stderr=True),
_stream_output_lines(proc.stdout, prefix),
_stream_output_lines(proc.stderr, prefix, is_stderr=True),
)
stdout_data = b""
@@ -140,47 +232,31 @@ async def _run_ssh_command(
*,
stream: bool = True,
raw: bool = False,
prefix: str = "",
) -> CommandResult:
"""Run a command on a remote host via SSH with streaming output."""
if raw:
# Use native ssh with TTY for proper progress bar rendering
ssh_args = ["ssh", "-t"]
if host.port != _DEFAULT_SSH_PORT:
ssh_args.extend(["-p", str(host.port)])
ssh_args.extend([f"{host.user}@{host.address}", command])
ssh_args = build_ssh_command(host, command, tty=True)
# Run in thread to avoid blocking the event loop
result = await asyncio.to_thread(subprocess.run, ssh_args, check=False)
# Use get_ssh_env() to auto-detect SSH agent socket
result = await asyncio.to_thread(subprocess.run, ssh_args, check=False, env=get_ssh_env())
return CommandResult(
service=service,
exit_code=result.returncode,
success=result.returncode == 0,
)
import asyncssh # noqa: PLC0415 - lazy import for faster CLI startup
proc: asyncssh.SSHClientProcess[Any]
try:
async with asyncssh.connect( # noqa: SIM117 - conn needed before create_process
host.address,
port=host.port,
username=host.user,
known_hosts=None,
) as conn:
async with asyncssh.connect(**ssh_connect_kwargs(host)) as conn: # noqa: SIM117
async with conn.create_process(command) as proc:
if stream:
async def read_stream(
reader: Any,
prefix: str,
*,
is_stderr: bool = False,
) -> None:
out = err_console if is_stderr else console
async for line in reader:
if line.strip(): # Skip empty lines
out.print(f"[cyan]\\[{prefix}][/] {escape(line)}", end="")
await asyncio.gather(
read_stream(proc.stdout, service),
read_stream(proc.stderr, service, is_stderr=True),
_stream_output_lines(proc.stdout, prefix),
_stream_output_lines(proc.stderr, prefix, is_stderr=True),
)
stdout_data = ""
@@ -209,11 +285,27 @@ async def run_command(
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> CommandResult:
"""Run a command on a host (locally or via SSH)."""
"""Run a command on a host (locally or via SSH).
Args:
host: Host configuration
command: Command to run
service: Service name (stored in result)
stream: Whether to stream output (default True)
raw: Whether to use raw mode with TTY (default False)
prefix: Output prefix. None=use service name, ""=no prefix.
"""
output_prefix = service if prefix is None else prefix
if is_local(host):
return await _run_local_command(command, service, stream=stream, raw=raw)
return await _run_ssh_command(host, command, service, stream=stream, raw=raw)
return await _run_local_command(
command, service, stream=stream, raw=raw, prefix=output_prefix
)
return await _run_ssh_command(
host, command, service, stream=stream, raw=raw, prefix=output_prefix
)
async def run_compose(
@@ -223,13 +315,17 @@ async def run_compose(
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> CommandResult:
"""Run a docker compose command for a service."""
host = config.get_host(service)
host_name = config.get_hosts(service)[0]
host = config.hosts[host_name]
compose_path = config.get_compose_path(service)
_print_compose_command(host_name, str(config.compose_dir), str(compose_path), compose_cmd)
command = f"docker compose -f {compose_path} {compose_cmd}"
return await run_command(host, command, service, stream=stream, raw=raw)
return await run_command(host, command, service, stream=stream, raw=raw, prefix=prefix)
async def run_compose_on_host(
@@ -240,6 +336,7 @@ async def run_compose_on_host(
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> CommandResult:
"""Run a docker compose command for a service on a specific host.
@@ -248,8 +345,10 @@ async def run_compose_on_host(
host = config.hosts[host_name]
compose_path = config.get_compose_path(service)
_print_compose_command(host_name, str(config.compose_dir), str(compose_path), compose_cmd)
command = f"docker compose -f {compose_path} {compose_cmd}"
return await run_command(host, command, service, stream=stream, raw=raw)
return await run_command(host, command, service, stream=stream, raw=raw, prefix=prefix)
async def run_on_services(
@@ -275,10 +374,11 @@ async def _run_sequential_commands(
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> CommandResult:
"""Run multiple compose commands sequentially for a service."""
for cmd in commands:
result = await run_compose(config, service, cmd, stream=stream, raw=raw)
result = await run_compose(config, service, cmd, stream=stream, raw=raw, prefix=prefix)
if not result.success:
return result
return CommandResult(service=service, exit_code=0, success=True)
@@ -291,10 +391,12 @@ async def _run_sequential_commands_multi_host(
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> list[CommandResult]:
"""Run multiple compose commands sequentially for a multi-host service.
Commands are run sequentially, but each command runs on all hosts in parallel.
For multi-host services, prefix defaults to service@host format.
"""
host_names = config.get_hosts(service)
compose_path = config.get_compose_path(service)
@@ -304,9 +406,16 @@ async def _run_sequential_commands_multi_host(
command = f"docker compose -f {compose_path} {cmd}"
tasks = []
for host_name in host_names:
_print_compose_command(host_name, str(config.compose_dir), str(compose_path), cmd)
host = config.hosts[host_name]
# For multi-host services, always use service@host prefix to distinguish output
label = f"{service}@{host_name}" if len(host_names) > 1 else service
tasks.append(run_command(host, command, label, stream=stream, raw=raw))
# Multi-host services always need prefixes to distinguish output from different hosts
# (ignore empty prefix from single-service batches - we still need to distinguish hosts)
effective_prefix = label if len(host_names) > 1 else prefix
tasks.append(
run_command(host, command, label, stream=stream, raw=raw, prefix=effective_prefix)
)
results = await asyncio.gather(*tasks)
final_results = list(results)
@@ -331,6 +440,9 @@ async def run_sequential_on_services(
For multi-host services, runs on all configured hosts.
Note: raw=True only makes sense for single-service operations.
"""
# Skip prefix for single-service operations (command line already shows context)
prefix: str | None = "" if len(services) == 1 else None
# Separate multi-host and single-host services for type-safe gathering
multi_host_tasks = []
single_host_tasks = []
@@ -339,12 +451,14 @@ async def run_sequential_on_services(
if config.is_multi_host(service):
multi_host_tasks.append(
_run_sequential_commands_multi_host(
config, service, commands, stream=stream, raw=raw
config, service, commands, stream=stream, raw=raw, prefix=prefix
)
)
else:
single_host_tasks.append(
_run_sequential_commands(config, service, commands, stream=stream, raw=raw)
_run_sequential_commands(
config, service, commands, stream=stream, raw=raw, prefix=prefix
)
)
# Gather results separately to maintain type safety
@@ -418,12 +532,15 @@ async def check_paths_exist(
"""Check if multiple paths exist on a specific host.
Returns a dict mapping path -> exists.
Handles permission denied as "exists" (path is there, just not accessible).
"""
# Only report missing if stat says "No such file", otherwise assume exists
# (handles permission denied correctly - path exists, just not accessible)
return await _batch_check_existence(
config,
host_name,
paths,
lambda esc: f"test -e '{esc}' && echo 'Y:{esc}' || echo 'N:{esc}'",
lambda esc: f"stat '{esc}' 2>&1 | grep -q 'No such file' && echo 'N:{esc}' || echo 'Y:{esc}'",
"mount-check",
)

View File

@@ -8,8 +8,8 @@ from dataclasses import dataclass
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
from .config import xdg_config_home
from .executor import run_compose
from .paths import xdg_config_home
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable, Iterable

View File

@@ -6,24 +6,69 @@ CLI commands are thin wrappers around these functions.
from __future__ import annotations
from typing import TYPE_CHECKING
import asyncio
from typing import TYPE_CHECKING, NamedTuple
from .compose import parse_external_networks, parse_host_volumes
from .console import console, err_console
from .compose import parse_devices, parse_external_networks, parse_host_volumes
from .console import console, err_console, print_error, print_success, print_warning
from .executor import (
CommandResult,
check_networks_exist,
check_paths_exist,
check_service_running,
run_command,
run_compose,
run_compose_on_host,
)
from .state import get_service_host, set_multi_host_service, set_service_host
from .state import (
get_orphaned_services,
get_service_host,
remove_service,
set_multi_host_service,
set_service_host,
)
if TYPE_CHECKING:
from .config import Config
class OperationInterruptedError(Exception):
"""Raised when a command is interrupted by Ctrl+C."""
class PreflightResult(NamedTuple):
"""Result of pre-flight checks for a service on a host."""
missing_paths: list[str]
missing_networks: list[str]
missing_devices: list[str]
@property
def ok(self) -> bool:
"""Return True if all checks passed."""
return not (self.missing_paths or self.missing_networks or self.missing_devices)
async def _run_compose_step(
cfg: Config,
service: str,
command: str,
*,
raw: bool,
host: str | None = None,
) -> CommandResult:
"""Run a compose command, handle raw output newline, and check for interrupts."""
if host:
result = await run_compose_on_host(cfg, service, host, command, raw=raw)
else:
result = await run_compose(cfg, service, command, raw=raw)
if raw:
print() # Ensure newline after raw output
if result.interrupted:
raise OperationInterruptedError
return result
def get_service_paths(cfg: Config, service: str) -> list[str]:
"""Get all required paths for a service (compose_dir + volumes)."""
paths = [str(cfg.compose_dir)]
@@ -31,58 +76,107 @@ def get_service_paths(cfg: Config, service: str) -> list[str]:
return paths
async def _check_mounts_for_migration(
cfg: Config,
service: str,
target_host: str,
) -> list[str]:
"""Check if mount paths exist on target host. Returns list of missing paths."""
paths = get_service_paths(cfg, service)
exists = await check_paths_exist(cfg, target_host, paths)
return [p for p, found in exists.items() if not found]
async def discover_service_host(cfg: Config, service: str) -> tuple[str, str | list[str] | None]:
"""Discover where a service is running.
For multi-host services, checks all assigned hosts in parallel.
For single-host, checks assigned host first, then others.
async def _check_networks_for_migration(
cfg: Config,
service: str,
target_host: str,
) -> list[str]:
"""Check if Docker networks exist on target host. Returns list of missing networks."""
networks = parse_external_networks(cfg, service)
if not networks:
return []
exists = await check_networks_exist(cfg, target_host, networks)
return [n for n, found in exists.items() if not found]
async def _preflight_check(
cfg: Config,
service: str,
target_host: str,
) -> tuple[list[str], list[str]]:
"""Run pre-flight checks for a service on target host.
Returns (missing_paths, missing_networks).
Returns (service_name, host_or_hosts_or_none).
"""
missing_paths = await _check_mounts_for_migration(cfg, service, target_host)
missing_networks = await _check_networks_for_migration(cfg, service, target_host)
return missing_paths, missing_networks
assigned_hosts = cfg.get_hosts(service)
if cfg.is_multi_host(service):
# Check all assigned hosts in parallel
checks = await asyncio.gather(
*[check_service_running(cfg, service, h) for h in assigned_hosts]
)
running = [h for h, is_running in zip(assigned_hosts, checks, strict=True) if is_running]
return service, running if running else None
# Single-host: check assigned host first, then others
if await check_service_running(cfg, service, assigned_hosts[0]):
return service, assigned_hosts[0]
for host in cfg.hosts:
if host != assigned_hosts[0] and await check_service_running(cfg, service, host):
return service, host
return service, None
async def check_service_requirements(
cfg: Config,
service: str,
host_name: str,
) -> PreflightResult:
"""Check if a service can run on a specific host.
Verifies that all required paths (volumes), networks, and devices exist.
"""
# Check mount paths
paths = get_service_paths(cfg, service)
path_exists = await check_paths_exist(cfg, host_name, paths)
missing_paths = [p for p, found in path_exists.items() if not found]
# Check external networks
networks = parse_external_networks(cfg, service)
missing_networks: list[str] = []
if networks:
net_exists = await check_networks_exist(cfg, host_name, networks)
missing_networks = [n for n, found in net_exists.items() if not found]
# Check devices
devices = parse_devices(cfg, service)
missing_devices: list[str] = []
if devices:
dev_exists = await check_paths_exist(cfg, host_name, devices)
missing_devices = [d for d, found in dev_exists.items() if not found]
return PreflightResult(missing_paths, missing_networks, missing_devices)
async def _cleanup_and_rollback(
cfg: Config,
service: str,
target_host: str,
current_host: str,
prefix: str,
*,
was_running: bool,
raw: bool = False,
) -> None:
"""Clean up failed start and attempt rollback to old host if it was running."""
print_warning(f"{prefix} Cleaning up failed start on [magenta]{target_host}[/]")
await run_compose(cfg, service, "down", raw=raw)
if not was_running:
err_console.print(
f"{prefix} [dim]Service was not running on [magenta]{current_host}[/], skipping rollback[/]"
)
return
print_warning(f"{prefix} Rolling back to [magenta]{current_host}[/]...")
rollback_result = await run_compose_on_host(cfg, service, current_host, "up -d", raw=raw)
if rollback_result.success:
print_success(f"{prefix} Rollback succeeded on [magenta]{current_host}[/]")
else:
print_error(f"{prefix} Rollback failed - service is down")
def _report_preflight_failures(
service: str,
target_host: str,
missing_paths: list[str],
missing_networks: list[str],
preflight: PreflightResult,
) -> None:
"""Report pre-flight check failures."""
err_console.print(
f"[cyan]\\[{service}][/] [red]✗[/] Cannot start on [magenta]{target_host}[/]:"
)
for path in missing_paths:
err_console.print(f" [red]✗[/] missing path: {path}")
for net in missing_networks:
err_console.print(f" [red]✗[/] missing network: {net}")
print_error(f"[cyan]\\[{service}][/] Cannot start on [magenta]{target_host}[/]:")
for path in preflight.missing_paths:
print_error(f" missing path: {path}")
for net in preflight.missing_networks:
print_error(f" missing network: {net}")
if preflight.missing_networks:
err_console.print(f" [dim]Hint: cf init-network {target_host}[/]")
for dev in preflight.missing_devices:
print_error(f" missing device: {dev}")
async def _up_multi_host_service(
@@ -100,9 +194,9 @@ async def _up_multi_host_service(
# Pre-flight checks on all hosts
for host_name in host_names:
missing_paths, missing_networks = await _preflight_check(cfg, service, host_name)
if missing_paths or missing_networks:
_report_preflight_failures(service, host_name, missing_paths, missing_networks)
preflight = await check_service_requirements(cfg, service, host_name)
if not preflight.ok:
_report_preflight_failures(service, host_name, preflight)
results.append(
CommandResult(service=f"{service}@{host_name}", exit_code=1, success=False)
)
@@ -130,6 +224,96 @@ async def _up_multi_host_service(
return results
async def _migrate_service(
cfg: Config,
service: str,
current_host: str,
target_host: str,
prefix: str,
*,
raw: bool = False,
) -> CommandResult | None:
"""Migrate a service from current_host to target_host.
Pre-pulls/builds images on target, then stops service on current host.
Returns failure result if migration prep fails, None on success.
"""
console.print(
f"{prefix} Migrating from [magenta]{current_host}[/] → [magenta]{target_host}[/]..."
)
# Prepare images on target host before stopping old service to minimize downtime.
# Pull handles image-based services; build handles Dockerfile-based services.
# --ignore-buildable makes pull skip images that have build: defined.
for cmd, label in [("pull --ignore-buildable", "Pull"), ("build", "Build")]:
result = await _run_compose_step(cfg, service, cmd, raw=raw)
if not result.success:
print_error(
f"{prefix} {label} failed on [magenta]{target_host}[/], "
"leaving service on current host"
)
return result
# Stop on current host
down_result = await _run_compose_step(cfg, service, "down", raw=raw, host=current_host)
return down_result if not down_result.success else None
async def _up_single_service(
cfg: Config,
service: str,
prefix: str,
*,
raw: bool,
) -> CommandResult:
"""Start a single-host service with migration support."""
target_host = cfg.get_hosts(service)[0]
current_host = get_service_host(cfg, service)
# Pre-flight check: verify paths, networks, and devices exist on target
preflight = await check_service_requirements(cfg, service, target_host)
if not preflight.ok:
_report_preflight_failures(service, target_host, preflight)
return CommandResult(service=service, exit_code=1, success=False)
# If service is deployed elsewhere, migrate it
did_migration = False
was_running = False
if current_host and current_host != target_host:
if current_host in cfg.hosts:
was_running = await check_service_running(cfg, service, current_host)
failure = await _migrate_service(
cfg, service, current_host, target_host, prefix, raw=raw
)
if failure:
return failure
did_migration = True
else:
print_warning(
f"{prefix} was on [magenta]{current_host}[/] (not in config), skipping down"
)
# Start on target host
console.print(f"{prefix} Starting on [magenta]{target_host}[/]...")
up_result = await _run_compose_step(cfg, service, "up -d", raw=raw)
# Update state on success, or rollback on failure
if up_result.success:
set_service_host(cfg, service, target_host)
elif did_migration and current_host:
await _cleanup_and_rollback(
cfg,
service,
target_host,
current_host,
prefix,
was_running=was_running,
raw=raw,
)
return up_result
async def up_services(
cfg: Config,
services: list[str],
@@ -140,54 +324,16 @@ async def up_services(
results: list[CommandResult] = []
total = len(services)
for idx, service in enumerate(services, 1):
prefix = f"[dim][{idx}/{total}][/] [cyan]\\[{service}][/]"
try:
for idx, service in enumerate(services, 1):
prefix = f"[dim][{idx}/{total}][/] [cyan]\\[{service}][/]"
# Handle multi-host services separately (no migration)
if cfg.is_multi_host(service):
multi_results = await _up_multi_host_service(cfg, service, prefix, raw=raw)
results.extend(multi_results)
continue
target_host = cfg.get_hosts(service)[0]
current_host = get_service_host(cfg, service)
# Pre-flight check: verify paths and networks exist on target
missing_paths, missing_networks = await _preflight_check(cfg, service, target_host)
if missing_paths or missing_networks:
_report_preflight_failures(service, target_host, missing_paths, missing_networks)
results.append(CommandResult(service=service, exit_code=1, success=False))
continue
# If service is deployed elsewhere, migrate it
if current_host and current_host != target_host:
if current_host in cfg.hosts:
console.print(
f"{prefix} Migrating from "
f"[magenta]{current_host}[/] → [magenta]{target_host}[/]..."
)
down_result = await run_compose_on_host(cfg, service, current_host, "down", raw=raw)
if raw:
print() # Ensure newline after raw output
if not down_result.success:
results.append(down_result)
continue
if cfg.is_multi_host(service):
results.extend(await _up_multi_host_service(cfg, service, prefix, raw=raw))
else:
err_console.print(
f"{prefix} [yellow]![/] was on "
f"[magenta]{current_host}[/] (not in config), skipping down"
)
# Start on target host
console.print(f"{prefix} Starting on [magenta]{target_host}[/]...")
up_result = await run_compose(cfg, service, "up -d", raw=raw)
if raw:
print() # Ensure newline after raw output (progress bars end with \r)
results.append(up_result)
# Update state on success
if up_result.success:
set_service_host(cfg, service, target_host)
results.append(await _up_single_service(cfg, service, prefix, raw=raw))
except OperationInterruptedError:
raise KeyboardInterrupt from None
return results
@@ -196,17 +342,93 @@ async def check_host_compatibility(
cfg: Config,
service: str,
) -> dict[str, tuple[int, int, list[str]]]:
"""Check which hosts can run a service based on mount paths.
"""Check which hosts can run a service based on paths, networks, and devices.
Returns dict of host_name -> (found_count, total_count, missing_paths).
Returns dict of host_name -> (found_count, total_count, missing_items).
"""
# Get total requirements count
paths = get_service_paths(cfg, service)
networks = parse_external_networks(cfg, service)
devices = parse_devices(cfg, service)
total = len(paths) + len(networks) + len(devices)
results: dict[str, tuple[int, int, list[str]]] = {}
for host_name in cfg.hosts:
exists = await check_paths_exist(cfg, host_name, paths)
found = sum(1 for v in exists.values() if v)
missing = [p for p, v in exists.items() if not v]
results[host_name] = (found, len(paths), missing)
preflight = await check_service_requirements(cfg, service, host_name)
all_missing = (
preflight.missing_paths + preflight.missing_networks + preflight.missing_devices
)
found = total - len(all_missing)
results[host_name] = (found, total, all_missing)
return results
async def stop_orphaned_services(cfg: Config) -> list[CommandResult]:
"""Stop orphaned services (in state but not in config).
Runs docker compose down on each service on its tracked host(s).
Only removes from state on successful stop.
Returns list of CommandResults for each service@host.
"""
orphaned = get_orphaned_services(cfg)
if not orphaned:
return []
results: list[CommandResult] = []
tasks: list[tuple[str, str, asyncio.Task[CommandResult]]] = []
# Build list of (service, host, task) for all orphaned services
for service, hosts in orphaned.items():
host_list = hosts if isinstance(hosts, list) else [hosts]
for host in host_list:
# Skip hosts no longer in config
if host not in cfg.hosts:
print_warning(f"{service}@{host}: host no longer in config, skipping")
results.append(
CommandResult(
service=f"{service}@{host}",
exit_code=1,
success=False,
stderr="host no longer in config",
)
)
continue
coro = run_compose_on_host(cfg, service, host, "down")
tasks.append((service, host, asyncio.create_task(coro)))
# Run all down commands in parallel
if tasks:
for service, host, task in tasks:
try:
result = await task
results.append(result)
if result.success:
print_success(f"{service}@{host}: stopped")
else:
print_error(f"{service}@{host}: {result.stderr or 'failed'}")
except Exception as e:
print_error(f"{service}@{host}: {e}")
results.append(
CommandResult(
service=f"{service}@{host}",
exit_code=1,
success=False,
stderr=str(e),
)
)
# Remove from state only for services where ALL hosts succeeded
for service, hosts in orphaned.items():
host_list = hosts if isinstance(hosts, list) else [hosts]
all_succeeded = all(
r.success
for r in results
if r.service.startswith(f"{service}@") or r.service == service
)
if all_succeeded:
remove_service(cfg, service)
return results

33
src/compose_farm/paths.py Normal file
View File

@@ -0,0 +1,33 @@
"""Path utilities - lightweight module with no heavy dependencies."""
from __future__ import annotations
import os
from pathlib import Path
def xdg_config_home() -> Path:
"""Get XDG config directory, respecting XDG_CONFIG_HOME env var."""
return Path(os.environ.get("XDG_CONFIG_HOME", Path.home() / ".config"))
def default_config_path() -> Path:
"""Get the default user config path."""
return xdg_config_home() / "compose-farm" / "compose-farm.yaml"
def config_search_paths() -> list[Path]:
"""Get search paths for config files."""
return [Path("compose-farm.yaml"), default_config_path()]
def find_config_path() -> Path | None:
"""Find the config file path, checking CF_CONFIG env var and search paths."""
if env_path := os.environ.get("CF_CONFIG"):
p = Path(env_path)
if p.exists() and p.is_file():
return p
for p in config_search_paths():
if p.exists() and p.is_file():
return p
return None

View File

@@ -0,0 +1,67 @@
"""SSH key utilities for compose-farm."""
from __future__ import annotations
import os
from pathlib import Path
# Default key paths for compose-farm SSH key
# Keys are stored in a subdirectory for cleaner docker volume mounting
SSH_KEY_DIR = Path.home() / ".ssh" / "compose-farm"
SSH_KEY_PATH = SSH_KEY_DIR / "id_ed25519"
SSH_PUBKEY_PATH = SSH_KEY_PATH.with_suffix(".pub")
def get_ssh_auth_sock() -> str | None:
"""Get SSH_AUTH_SOCK, auto-detecting forwarded agent if needed.
Checks in order:
1. SSH_AUTH_SOCK environment variable (if socket exists)
2. Forwarded agent sockets in ~/.ssh/agent/ (most recent first)
Returns the socket path or None if no valid socket found.
"""
sock = os.environ.get("SSH_AUTH_SOCK")
if sock and Path(sock).is_socket():
return sock
# Try to find a forwarded SSH agent socket
agent_dir = Path.home() / ".ssh" / "agent"
if agent_dir.is_dir():
sockets = sorted(
agent_dir.glob("s.*.sshd.*"), key=lambda p: p.stat().st_mtime, reverse=True
)
for s in sockets:
if s.is_socket():
return str(s)
return None
def get_ssh_env() -> dict[str, str]:
"""Get environment dict for SSH subprocess with auto-detected agent.
Returns a copy of the current environment with SSH_AUTH_SOCK set
to the auto-detected agent socket (if found).
"""
env = os.environ.copy()
sock = get_ssh_auth_sock()
if sock:
env["SSH_AUTH_SOCK"] = sock
return env
def key_exists() -> bool:
"""Check if the compose-farm SSH key pair exists."""
return SSH_KEY_PATH.exists() and SSH_PUBKEY_PATH.exists()
def get_key_path() -> Path | None:
"""Get the SSH key path if it exists, None otherwise."""
return SSH_KEY_PATH if key_exists() else None
def get_pubkey_content() -> str | None:
"""Get the public key content if it exists, None otherwise."""
if not SSH_PUBKEY_PATH.exists():
return None
return SSH_PUBKEY_PATH.read_text().strip()

View File

@@ -8,11 +8,44 @@ from typing import TYPE_CHECKING, Any
import yaml
if TYPE_CHECKING:
from collections.abc import Generator
from collections.abc import Generator, Mapping
from .config import Config
def group_services_by_host(
services: dict[str, str | list[str]],
hosts: Mapping[str, object],
all_hosts: list[str] | None = None,
) -> dict[str, list[str]]:
"""Group services by their assigned host(s).
For multi-host services (list or "all"), the service appears in multiple host lists.
"""
by_host: dict[str, list[str]] = {h: [] for h in hosts}
for service, host_value in services.items():
if isinstance(host_value, list):
for host_name in host_value:
if host_name in by_host:
by_host[host_name].append(service)
elif host_value == "all" and all_hosts:
for host_name in all_hosts:
if host_name in by_host:
by_host[host_name].append(service)
elif host_value in by_host:
by_host[host_value].append(service)
return by_host
def group_running_services_by_host(
state: dict[str, str | list[str]],
hosts: Mapping[str, object],
) -> dict[str, list[str]]:
"""Group running services by host, filtering out hosts with no services."""
by_host = group_services_by_host(state, hosts)
return {h: svcs for h, svcs in by_host.items() if svcs}
def load_state(config: Config) -> dict[str, str | list[str]]:
"""Load the current deployment state.
@@ -82,50 +115,6 @@ def remove_service(config: Config, service: str) -> None:
state.pop(service, None)
def add_service_to_host(config: Config, service: str, host: str) -> None:
"""Add a specific host to a service's state.
For multi-host services, adds the host to the list if not present.
For single-host services, sets the host.
"""
with _modify_state(config) as state:
current = state.get(service)
if config.is_multi_host(service):
# Multi-host: add to list if not present
if isinstance(current, list):
if host not in current:
state[service] = [*current, host]
else:
state[service] = [host]
else:
# Single-host: just set it
state[service] = host
def remove_service_from_host(config: Config, service: str, host: str) -> None:
"""Remove a specific host from a service's state.
For multi-host services, removes just that host from the list.
For single-host services, removes the service entirely if host matches.
"""
with _modify_state(config) as state:
current = state.get(service)
if current is None:
return
if isinstance(current, list):
# Multi-host: remove this host from list
remaining = [h for h in current if h != host]
if remaining:
state[service] = remaining
else:
state.pop(service, None)
elif current == host:
# Single-host: remove if matches
state.pop(service, None)
def get_services_needing_migration(config: Config) -> list[str]:
"""Get services where current host differs from configured host.
@@ -142,3 +131,25 @@ def get_services_needing_migration(config: Config) -> list[str]:
if current_host and current_host != configured_host:
needs_migration.append(service)
return needs_migration
def get_orphaned_services(config: Config) -> dict[str, str | list[str]]:
"""Get services that are in state but not in config.
These are services that were previously deployed but have been
removed from the config file (e.g., commented out).
Returns a dict mapping service name to host(s) where it's deployed.
"""
state = load_state(config)
return {service: hosts for service, hosts in state.items() if service not in config.services}
def get_services_not_in_state(config: Config) -> list[str]:
"""Get services that are in config but not in state.
These are services that should be running but aren't tracked
(e.g., newly added to config, or previously stopped as orphans).
"""
state = load_state(config)
return [service for service in config.services if service not in state]

View File

@@ -0,0 +1,7 @@
"""Compose Farm Web UI."""
from __future__ import annotations
from compose_farm.web.app import create_app
__all__ = ["create_app"]

View File

@@ -0,0 +1,68 @@
"""FastAPI application setup."""
from __future__ import annotations
import asyncio
import sys
from contextlib import asynccontextmanager, suppress
from typing import TYPE_CHECKING
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from pydantic import ValidationError
from compose_farm.web.deps import STATIC_DIR, get_config
from compose_farm.web.routes import actions, api, pages
from compose_farm.web.streaming import TASK_TTL_SECONDS, cleanup_stale_tasks
if TYPE_CHECKING:
from collections.abc import AsyncGenerator
async def _task_cleanup_loop() -> None:
"""Periodically clean up stale completed tasks."""
while True:
await asyncio.sleep(TASK_TTL_SECONDS // 2) # Run every 5 minutes
cleanup_stale_tasks()
@asynccontextmanager
async def lifespan(_app: FastAPI) -> AsyncGenerator[None, None]:
"""Application lifespan handler."""
# Startup: pre-load config (ignore errors - handled per-request)
with suppress(ValidationError, FileNotFoundError):
get_config()
# Start background cleanup task
cleanup_task = asyncio.create_task(_task_cleanup_loop())
yield
# Shutdown: cancel cleanup task
cleanup_task.cancel()
with suppress(asyncio.CancelledError):
await cleanup_task
def create_app() -> FastAPI:
"""Create and configure the FastAPI application."""
app = FastAPI(
title="Compose Farm",
description="Web UI for managing Docker Compose services across multiple hosts",
lifespan=lifespan,
)
# Mount static files
app.mount("/static", StaticFiles(directory=str(STATIC_DIR)), name="static")
app.include_router(pages.router)
app.include_router(api.router, prefix="/api")
app.include_router(actions.router, prefix="/api")
# WebSocket routes use Unix-only modules (fcntl, pty)
if sys.platform != "win32":
from compose_farm.web.ws import router as ws_router # noqa: PLC0415
app.include_router(ws_router)
return app

View File

@@ -0,0 +1,50 @@
"""Shared dependencies for web modules.
This module contains shared config and template accessors to avoid circular imports
between app.py and route modules.
"""
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
from fastapi.templating import Jinja2Templates
from pydantic import ValidationError
from compose_farm.executor import is_local
if TYPE_CHECKING:
from compose_farm.config import Config
# Paths
WEB_DIR = Path(__file__).parent
TEMPLATES_DIR = WEB_DIR / "templates"
STATIC_DIR = WEB_DIR / "static"
def get_config() -> Config:
"""Load config from disk (always fresh)."""
from compose_farm.config import load_config # noqa: PLC0415
return load_config()
def get_templates() -> Jinja2Templates:
"""Get Jinja2 templates instance."""
return Jinja2Templates(directory=str(TEMPLATES_DIR))
def extract_config_error(exc: Exception) -> str:
"""Extract a user-friendly error message from a config exception."""
if isinstance(exc, ValidationError):
return "; ".join(err.get("msg", str(err)) for err in exc.errors())
return str(exc)
def get_local_host(config: Config) -> str | None:
"""Find the local host name from config, if any."""
for name, host in config.hosts.items():
if is_local(host):
return name
return None

View File

@@ -0,0 +1,5 @@
"""Web routes."""
from compose_farm.web.routes import actions, api, pages
__all__ = ["actions", "api", "pages"]

View File

@@ -0,0 +1,66 @@
"""Action routes for service operations."""
from __future__ import annotations
import asyncio
import uuid
from typing import TYPE_CHECKING, Any
from fastapi import APIRouter, HTTPException
if TYPE_CHECKING:
from collections.abc import Callable, Coroutine
from compose_farm.web.deps import get_config
from compose_farm.web.streaming import run_cli_streaming, run_compose_streaming, tasks
router = APIRouter(tags=["actions"])
# Store task references to prevent garbage collection
_background_tasks: set[asyncio.Task[None]] = set()
def _start_task(coro_factory: Callable[[str], Coroutine[Any, Any, None]]) -> str:
"""Create a task, register it, and return the task_id."""
task_id = str(uuid.uuid4())
tasks[task_id] = {"status": "running", "output": []}
task: asyncio.Task[None] = asyncio.create_task(coro_factory(task_id))
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
return task_id
# Allowed service commands
ALLOWED_COMMANDS = {"up", "down", "restart", "pull", "update", "logs"}
@router.post("/service/{name}/{command}")
async def service_action(name: str, command: str) -> dict[str, Any]:
"""Run a compose command for a service (up, down, restart, pull, update, logs)."""
if command not in ALLOWED_COMMANDS:
raise HTTPException(status_code=404, detail=f"Unknown command '{command}'")
config = get_config()
if name not in config.services:
raise HTTPException(status_code=404, detail=f"Service '{name}' not found")
task_id = _start_task(lambda tid: run_compose_streaming(config, name, command, tid))
return {"task_id": task_id, "service": name, "command": command}
@router.post("/apply")
async def apply_all() -> dict[str, Any]:
"""Run cf apply to reconcile all services."""
config = get_config()
task_id = _start_task(lambda tid: run_cli_streaming(config, ["apply"], tid))
return {"task_id": task_id, "command": "apply"}
@router.post("/refresh")
async def refresh_state() -> dict[str, Any]:
"""Refresh state from running services."""
config = get_config()
task_id = _start_task(lambda tid: run_cli_streaming(config, ["refresh"], tid))
return {"task_id": task_id, "command": "refresh"}

View File

@@ -0,0 +1,372 @@
"""JSON API routes."""
from __future__ import annotations
import asyncio
import contextlib
import json
import shlex
from datetime import UTC, datetime
from pathlib import Path
from typing import Annotated, Any
import asyncssh
import yaml
from fastapi import APIRouter, Body, HTTPException, Query
from fastapi.responses import HTMLResponse
from compose_farm.executor import is_local, run_compose_on_host, ssh_connect_kwargs
from compose_farm.paths import find_config_path
from compose_farm.state import load_state
from compose_farm.web.deps import get_config, get_templates
router = APIRouter(tags=["api"])
def _validate_yaml(content: str) -> None:
"""Validate YAML content, raise HTTPException on error."""
try:
yaml.safe_load(content)
except yaml.YAMLError as e:
raise HTTPException(status_code=400, detail=f"Invalid YAML: {e}") from e
def _backup_file(file_path: Path) -> Path | None:
"""Create a timestamped backup of a file if it exists and content differs.
Backups are stored in a .backups directory alongside the file.
Returns the backup path if created, None if no backup was needed.
"""
if not file_path.exists():
return None
# Create backup directory
backup_dir = file_path.parent / ".backups"
backup_dir.mkdir(exist_ok=True)
# Generate timestamped backup filename
timestamp = datetime.now(tz=UTC).strftime("%Y%m%d_%H%M%S")
backup_name = f"{file_path.name}.{timestamp}"
backup_path = backup_dir / backup_name
# Copy current content to backup
backup_path.write_text(file_path.read_text())
# Clean up old backups (keep last 200)
backups = sorted(backup_dir.glob(f"{file_path.name}.*"), reverse=True)
for old_backup in backups[200:]:
old_backup.unlink()
return backup_path
def _save_with_backup(file_path: Path, content: str) -> bool:
"""Save content to file, creating a backup first if content changed.
Returns True if file was saved, False if content was unchanged.
"""
# Check if content actually changed
if file_path.exists():
current_content = file_path.read_text()
if current_content == content:
return False # No change, skip save
_backup_file(file_path)
file_path.write_text(content)
return True
def _get_service_compose_path(name: str) -> Path:
"""Get compose path for service, raising HTTPException if not found."""
config = get_config()
if name not in config.services:
raise HTTPException(status_code=404, detail=f"Service '{name}' not found")
compose_path = config.get_compose_path(name)
if not compose_path:
raise HTTPException(status_code=404, detail="Compose file not found")
return compose_path
def _get_compose_services(config: Any, service: str, hosts: list[str]) -> list[dict[str, Any]]:
"""Get container info from compose file (fast, local read).
Returns one entry per container per host for multi-host services.
"""
compose_path = config.get_compose_path(service)
if not compose_path or not compose_path.exists():
return []
compose_data = yaml.safe_load(compose_path.read_text()) or {}
raw_services = compose_data.get("services", {})
if not isinstance(raw_services, dict):
return []
# Project name is the directory name (docker compose default)
project_name = compose_path.parent.name
containers = []
for host in hosts:
for svc_name, svc_def in raw_services.items():
# Use container_name if set, otherwise default to {project}-{service}-1
if isinstance(svc_def, dict) and svc_def.get("container_name"):
container_name = svc_def["container_name"]
else:
container_name = f"{project_name}-{svc_name}-1"
containers.append(
{
"Name": container_name,
"Service": svc_name,
"Host": host,
"State": "unknown", # Status requires Docker query
}
)
return containers
async def _get_container_states(
config: Any, service: str, containers: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Query Docker for actual container states on a single host."""
if not containers:
return containers
# All containers should be on the same host
host_name = containers[0]["Host"]
# Use -a to include stopped/exited containers
result = await run_compose_on_host(
config, service, host_name, "ps -a --format json", stream=False
)
if not result.success:
return containers
# Build state map: name -> (state, exit_code)
state_map: dict[str, tuple[str, int]] = {}
for line in result.stdout.strip().split("\n"):
if line.strip():
with contextlib.suppress(json.JSONDecodeError):
data = json.loads(line)
name = data.get("Name", "")
state = data.get("State", "unknown")
exit_code = data.get("ExitCode", 0)
state_map[name] = (state, exit_code)
# Update container states
for c in containers:
if c["Name"] in state_map:
state, exit_code = state_map[c["Name"]]
c["State"] = state
c["ExitCode"] = exit_code
else:
# Container not in ps output means it was never started
c["State"] = "created"
c["ExitCode"] = None
return containers
def _render_containers(
service: str, host: str, containers: list[dict[str, Any]], *, show_header: bool = False
) -> str:
"""Render containers HTML using Jinja template."""
templates = get_templates()
template = templates.env.get_template("partials/containers.html")
module = template.make_module()
result: str = module.host_containers(service, host, containers, show_header=show_header)
return result
@router.get("/service/{name}/containers", response_class=HTMLResponse)
async def get_containers(name: str, host: str | None = None) -> HTMLResponse:
"""Get containers for a service as HTML buttons.
If host is specified, queries Docker for that host's status.
Otherwise returns all hosts with loading spinners that auto-fetch.
"""
config = get_config()
if name not in config.services:
raise HTTPException(status_code=404, detail=f"Service '{name}' not found")
# Get hosts where service is running from state
state = load_state(config)
current_hosts = state.get(name)
if not current_hosts:
return HTMLResponse('<span class="text-base-content/60">Service not running</span>')
all_hosts = current_hosts if isinstance(current_hosts, list) else [current_hosts]
# If host specified, return just that host's containers with status
if host:
if host not in all_hosts:
return HTMLResponse(f'<span class="text-error">Host {host} not found</span>')
containers = _get_compose_services(config, name, [host])
containers = await _get_container_states(config, name, containers)
return HTMLResponse(_render_containers(name, host, containers))
# Initial load: return all hosts with loading spinners, each fetches its own status
html_parts = []
is_multi_host = len(all_hosts) > 1
for h in all_hosts:
host_id = f"containers-{name}-{h}".replace(".", "-")
containers = _get_compose_services(config, name, [h])
if is_multi_host:
html_parts.append(f'<div class="font-semibold text-sm mt-3 mb-1">{h}</div>')
# Container for this host that auto-fetches its own status
html_parts.append(f"""
<div id="{host_id}"
hx-get="/api/service/{name}/containers?host={h}"
hx-trigger="load"
hx-target="this"
hx-select="unset"
hx-swap="innerHTML">
{_render_containers(name, h, containers)}
</div>
""")
return HTMLResponse("".join(html_parts))
@router.put("/service/{name}/compose")
async def save_compose(
name: str, content: Annotated[str, Body(media_type="text/plain")]
) -> dict[str, Any]:
"""Save compose file content."""
compose_path = _get_service_compose_path(name)
_validate_yaml(content)
saved = _save_with_backup(compose_path, content)
msg = "Compose file saved" if saved else "No changes to save"
return {"success": True, "message": msg}
@router.put("/service/{name}/env")
async def save_env(
name: str, content: Annotated[str, Body(media_type="text/plain")]
) -> dict[str, Any]:
"""Save .env file content."""
env_path = _get_service_compose_path(name).parent / ".env"
saved = _save_with_backup(env_path, content)
msg = ".env file saved" if saved else "No changes to save"
return {"success": True, "message": msg}
@router.put("/config")
async def save_config(
content: Annotated[str, Body(media_type="text/plain")],
) -> dict[str, Any]:
"""Save compose-farm.yaml config file."""
config_path = find_config_path()
if not config_path:
raise HTTPException(status_code=404, detail="Config file not found")
_validate_yaml(content)
saved = _save_with_backup(config_path, content)
msg = "Config saved" if saved else "No changes to save"
return {"success": True, "message": msg}
async def _read_file_local(path: str) -> str:
"""Read a file from the local filesystem."""
expanded = Path(path).expanduser()
return await asyncio.to_thread(expanded.read_text, encoding="utf-8")
async def _write_file_local(path: str, content: str) -> bool:
"""Write content to a file on the local filesystem with backup.
Returns True if file was saved, False if content was unchanged.
"""
expanded = Path(path).expanduser()
return await asyncio.to_thread(_save_with_backup, expanded, content)
async def _read_file_remote(host: Any, path: str) -> str:
"""Read a file from a remote host via SSH."""
# Expand ~ on remote by using shell
cmd = f"cat {shlex.quote(path)}"
if path.startswith("~/"):
cmd = f"cat ~/{shlex.quote(path[2:])}"
async with asyncssh.connect(**ssh_connect_kwargs(host)) as conn:
result = await conn.run(cmd, check=True)
stdout = result.stdout or ""
return stdout.decode() if isinstance(stdout, bytes) else stdout
async def _write_file_remote(host: Any, path: str, content: str) -> None:
"""Write content to a file on a remote host via SSH."""
# Expand ~ on remote: keep ~ unquoted for shell expansion, quote the rest
target = f"~/{shlex.quote(path[2:])}" if path.startswith("~/") else shlex.quote(path)
cmd = f"cat > {target}"
async with asyncssh.connect(**ssh_connect_kwargs(host)) as conn:
result = await conn.run(cmd, input=content, check=True)
if result.returncode != 0:
stderr = result.stderr.decode() if isinstance(result.stderr, bytes) else result.stderr
msg = f"Failed to write file: {stderr}"
raise RuntimeError(msg)
def _get_console_host(host: str, path: str) -> Any:
"""Validate and return host config for console file operations."""
config = get_config()
host_config = config.hosts.get(host)
if not host_config:
raise HTTPException(status_code=404, detail=f"Host '{host}' not found")
if not path:
raise HTTPException(status_code=400, detail="Path is required")
return host_config
@router.get("/console/file")
async def read_console_file(
host: Annotated[str, Query(description="Host name")],
path: Annotated[str, Query(description="File path")],
) -> dict[str, Any]:
"""Read a file from a host for the console editor."""
host_config = _get_console_host(host, path)
try:
if is_local(host_config):
content = await _read_file_local(path)
else:
content = await _read_file_remote(host_config, path)
return {"success": True, "content": content}
except FileNotFoundError:
raise HTTPException(status_code=404, detail=f"File not found: {path}") from None
except PermissionError:
raise HTTPException(status_code=403, detail=f"Permission denied: {path}") from None
except Exception as e:
raise HTTPException(status_code=500, detail=str(e)) from e
@router.put("/console/file")
async def write_console_file(
host: Annotated[str, Query(description="Host name")],
path: Annotated[str, Query(description="File path")],
content: Annotated[str, Body(media_type="text/plain")],
) -> dict[str, Any]:
"""Write a file to a host from the console editor."""
host_config = _get_console_host(host, path)
try:
if is_local(host_config):
saved = await _write_file_local(path, content)
msg = f"Saved: {path}" if saved else "No changes to save"
else:
await _write_file_remote(host_config, path, content)
msg = f"Saved: {path}" # Remote doesn't track changes
return {"success": True, "message": msg}
except PermissionError:
raise HTTPException(status_code=403, detail=f"Permission denied: {path}") from None
except Exception as e:
raise HTTPException(status_code=500, detail=str(e)) from e

View File

@@ -0,0 +1,279 @@
"""HTML page routes."""
from __future__ import annotations
import yaml
from fastapi import APIRouter, Request
from fastapi.responses import HTMLResponse
from pydantic import ValidationError
from compose_farm.paths import find_config_path
from compose_farm.state import (
get_orphaned_services,
get_service_host,
get_services_needing_migration,
get_services_not_in_state,
group_running_services_by_host,
load_state,
)
from compose_farm.web.deps import (
extract_config_error,
get_config,
get_local_host,
get_templates,
)
router = APIRouter()
@router.get("/console", response_class=HTMLResponse)
async def console(request: Request) -> HTMLResponse:
"""Console page with terminal and editor."""
config = get_config()
templates = get_templates()
# Sort hosts with local first
local_host = get_local_host(config)
hosts = sorted(config.hosts.keys())
if local_host:
hosts = [local_host] + [h for h in hosts if h != local_host]
# Get config path for default editor file
config_path = str(config.config_path) if config.config_path else ""
return templates.TemplateResponse(
"console.html",
{
"request": request,
"hosts": hosts,
"local_host": local_host,
"config_path": config_path,
},
)
@router.get("/", response_class=HTMLResponse)
async def index(request: Request) -> HTMLResponse:
"""Dashboard page - combined view of all cluster info."""
templates = get_templates()
# Try to load config, handle errors gracefully
config_error = None
try:
config = get_config()
except (ValidationError, FileNotFoundError) as e:
config_error = extract_config_error(e)
# Read raw config content for the editor
config_path = find_config_path()
config_content = config_path.read_text() if config_path else ""
return templates.TemplateResponse(
"index.html",
{
"request": request,
"config_error": config_error,
"hosts": {},
"services": {},
"config_content": config_content,
"state_content": "",
"running_count": 0,
"stopped_count": 0,
"orphaned": [],
"migrations": [],
"not_started": [],
"services_by_host": {},
},
)
# Get state
deployed = load_state(config)
# Stats
running_count = len(deployed)
stopped_count = len(config.services) - running_count
# Pending operations
orphaned = get_orphaned_services(config)
migrations = get_services_needing_migration(config)
not_started = get_services_not_in_state(config)
# Group services by host (filter out hosts with no running services)
services_by_host = group_running_services_by_host(deployed, config.hosts)
# Config file content
config_content = ""
if config.config_path and config.config_path.exists():
config_content = config.config_path.read_text()
# State file content
state_content = yaml.dump({"deployed": deployed}, default_flow_style=False, sort_keys=False)
return templates.TemplateResponse(
"index.html",
{
"request": request,
"config_error": None,
# Config data
"hosts": config.hosts,
"services": config.services,
"config_content": config_content,
# State data
"state_content": state_content,
# Stats
"running_count": running_count,
"stopped_count": stopped_count,
# Pending operations
"orphaned": orphaned,
"migrations": migrations,
"not_started": not_started,
# Services by host
"services_by_host": services_by_host,
},
)
@router.get("/service/{name}", response_class=HTMLResponse)
async def service_detail(request: Request, name: str) -> HTMLResponse:
"""Service detail page."""
config = get_config()
templates = get_templates()
# Get compose file content
compose_path = config.get_compose_path(name)
compose_content = ""
if compose_path and compose_path.exists():
compose_content = compose_path.read_text()
# Get .env file content
env_content = ""
env_path = None
if compose_path:
env_path = compose_path.parent / ".env"
if env_path.exists():
env_content = env_path.read_text()
# Get host info
hosts = config.get_hosts(name)
# Get state
current_host = get_service_host(config, name)
return templates.TemplateResponse(
"service.html",
{
"request": request,
"name": name,
"hosts": hosts,
"current_host": current_host,
"compose_content": compose_content,
"compose_path": str(compose_path) if compose_path else None,
"env_content": env_content,
"env_path": str(env_path) if env_path else None,
},
)
@router.get("/partials/sidebar", response_class=HTMLResponse)
async def sidebar_partial(request: Request) -> HTMLResponse:
"""Sidebar service list partial."""
config = get_config()
templates = get_templates()
state = load_state(config)
# Build service -> host mapping (empty string for multi-host services)
service_hosts = {
svc: "" if host_val == "all" or isinstance(host_val, list) else host_val
for svc, host_val in config.services.items()
}
return templates.TemplateResponse(
"partials/sidebar.html",
{
"request": request,
"services": sorted(config.services.keys()),
"service_hosts": service_hosts,
"hosts": sorted(config.hosts.keys()),
"local_host": get_local_host(config),
"state": state,
},
)
@router.get("/partials/config-error", response_class=HTMLResponse)
async def config_error_partial(request: Request) -> HTMLResponse:
"""Config error banner partial."""
templates = get_templates()
try:
get_config()
return HTMLResponse("") # No error
except (ValidationError, FileNotFoundError) as e:
error = extract_config_error(e)
return templates.TemplateResponse(
"partials/config_error.html", {"request": request, "config_error": error}
)
@router.get("/partials/stats", response_class=HTMLResponse)
async def stats_partial(request: Request) -> HTMLResponse:
"""Stats cards partial."""
config = get_config()
templates = get_templates()
deployed = load_state(config)
running_count = len(deployed)
stopped_count = len(config.services) - running_count
return templates.TemplateResponse(
"partials/stats.html",
{
"request": request,
"hosts": config.hosts,
"services": config.services,
"running_count": running_count,
"stopped_count": stopped_count,
},
)
@router.get("/partials/pending", response_class=HTMLResponse)
async def pending_partial(request: Request, expanded: bool = True) -> HTMLResponse:
"""Pending operations partial."""
config = get_config()
templates = get_templates()
orphaned = get_orphaned_services(config)
migrations = get_services_needing_migration(config)
not_started = get_services_not_in_state(config)
return templates.TemplateResponse(
"partials/pending.html",
{
"request": request,
"orphaned": orphaned,
"migrations": migrations,
"not_started": not_started,
"expanded": expanded,
},
)
@router.get("/partials/services-by-host", response_class=HTMLResponse)
async def services_by_host_partial(request: Request, expanded: bool = True) -> HTMLResponse:
"""Services by host partial."""
config = get_config()
templates = get_templates()
deployed = load_state(config)
services_by_host = group_running_services_by_host(deployed, config.hosts)
return templates.TemplateResponse(
"partials/services_by_host.html",
{
"request": request,
"hosts": config.hosts,
"services_by_host": services_by_host,
"expanded": expanded,
},
)

View File

@@ -0,0 +1,125 @@
/* Sidebar inputs - remove focus outline (DaisyUI 5 uses outline + outline-offset) */
#sidebar .input:focus,
#sidebar .input:focus-within,
#sidebar .select:focus {
outline: none;
outline-offset: 0;
}
/* Editors (Monaco) - wrapper makes it resizable */
.editor-wrapper {
resize: vertical;
overflow: hidden;
min-height: 150px;
}
.editor-wrapper .yaml-editor,
.editor-wrapper .env-editor,
.editor-wrapper .yaml-viewer {
height: 100%;
border: 1px solid oklch(var(--bc) / 0.2);
border-radius: 0.5rem;
}
.editor-wrapper.yaml-wrapper { height: 400px; }
.editor-wrapper.env-wrapper { height: 250px; }
.editor-wrapper.viewer-wrapper { height: 300px; }
/* Terminal - no custom CSS needed, using h-full class in HTML */
/* Prevent save button resize when text changes */
#save-btn, #save-config-btn {
min-width: 5rem;
}
/* Rainbow hover effect for headers */
.rainbow-hover {
transition: color 0.3s;
}
.rainbow-hover:hover {
background: linear-gradient(
90deg,
#e07070,
#e0a070,
#d0d070,
#70c080,
#7090d0,
#9080b0,
#b080a0,
#e07070
);
background-size: 16em 100%;
background-clip: text;
-webkit-background-clip: text;
color: transparent;
animation: rainbow 4s linear infinite;
}
@keyframes rainbow {
to {
background-position: 16em center;
}
}
/* Command palette FAB - rainbow glow effect */
@property --cmd-pos { syntax: "<number>"; inherits: true; initial-value: 100; }
@property --cmd-blur { syntax: "<number>"; inherits: true; initial-value: 10; }
@property --cmd-scale { syntax: "<number>"; inherits: true; initial-value: 1; }
@property --cmd-opacity { syntax: "<number>"; inherits: true; initial-value: 0.3; }
#cmd-fab {
--g: linear-gradient(to right, #fff, #fff, #0ff, #00f, #8000ff, #e066a3, #f00, #ff0, #bfff80, #fff, #fff);
all: unset;
position: fixed;
bottom: 1.5rem;
right: 1.5rem;
z-index: 50;
cursor: pointer;
transform: scale(var(--cmd-scale));
transition: --cmd-pos 3s, --cmd-blur 0.3s, --cmd-opacity 0.3s, --cmd-scale 0.2s cubic-bezier(.76,-.25,.51,1.13);
}
.cmd-fab-inner {
display: block;
padding: 0.6em 1em;
background: #1d232a;
border-radius: 8px;
font-size: 14px;
position: relative;
}
.cmd-fab-inner > span {
background: var(--g) no-repeat calc(var(--cmd-pos) * 1%) 0 / 900%;
-webkit-background-clip: text;
background-clip: text;
-webkit-text-fill-color: transparent;
letter-spacing: 0.15ch;
font-weight: 600;
}
.cmd-fab-inner::before, .cmd-fab-inner::after {
content: "";
position: absolute;
border-radius: 8px;
}
.cmd-fab-inner::before {
inset: -1.5px;
background: var(--g) no-repeat calc(var(--cmd-pos) * 1%) 0 / 900%;
border-radius: 9px;
z-index: -1;
opacity: var(--cmd-opacity);
}
.cmd-fab-inner::after {
inset: 0;
background: #000;
transform: translateY(10px);
z-index: -2;
filter: blur(calc(var(--cmd-blur) * 1px));
}
#cmd-fab:hover { --cmd-scale: 1.05; --cmd-pos: 0; --cmd-blur: 30; --cmd-opacity: 1; }
#cmd-fab:hover .cmd-fab-inner::after { background: var(--g); opacity: 0.3; }
#cmd-fab:active { --cmd-scale: 0.98; --cmd-blur: 15; }

View File

@@ -0,0 +1,806 @@
/**
* Compose Farm Web UI JavaScript
*/
// ============================================================================
// CONSTANTS
// ============================================================================
// ANSI escape codes for terminal output
const ANSI = {
RED: '\x1b[31m',
GREEN: '\x1b[32m',
DIM: '\x1b[2m',
RESET: '\x1b[0m',
CRLF: '\r\n'
};
// Terminal color theme (dark mode matching PicoCSS)
const TERMINAL_THEME = {
background: '#1a1a2e',
foreground: '#e4e4e7',
cursor: '#e4e4e7',
cursorAccent: '#1a1a2e',
black: '#18181b',
red: '#ef4444',
green: '#22c55e',
yellow: '#eab308',
blue: '#3b82f6',
magenta: '#a855f7',
cyan: '#06b6d4',
white: '#e4e4e7',
brightBlack: '#52525b',
brightRed: '#f87171',
brightGreen: '#4ade80',
brightYellow: '#facc15',
brightBlue: '#60a5fa',
brightMagenta: '#c084fc',
brightCyan: '#22d3ee',
brightWhite: '#fafafa'
};
// Language detection from file path
const LANGUAGE_MAP = {
'yaml': 'yaml', 'yml': 'yaml',
'json': 'json',
'js': 'javascript', 'mjs': 'javascript',
'ts': 'typescript', 'tsx': 'typescript',
'py': 'python',
'sh': 'shell', 'bash': 'shell',
'md': 'markdown',
'html': 'html', 'htm': 'html',
'css': 'css',
'sql': 'sql',
'toml': 'toml',
'ini': 'ini', 'conf': 'ini',
'dockerfile': 'dockerfile',
'env': 'plaintext'
};
// ============================================================================
// STATE
// ============================================================================
// Store active terminals and editors
const terminals = {};
const editors = {};
let monacoLoaded = false;
let monacoLoading = false;
// LocalStorage key prefix for active tasks (scoped by page)
const TASK_KEY_PREFIX = 'cf_task:';
const getTaskKey = () => TASK_KEY_PREFIX + window.location.pathname;
// Exec terminal state
let execTerminalWrapper = null; // {term, dispose}
let execWs = null;
// ============================================================================
// UTILITIES
// ============================================================================
/**
* Get Monaco language from file path
* @param {string} path - File path
* @returns {string} Monaco language identifier
*/
function getLanguageFromPath(path) {
const ext = path.split('.').pop().toLowerCase();
return LANGUAGE_MAP[ext] || 'plaintext';
}
window.getLanguageFromPath = getLanguageFromPath;
/**
* Create WebSocket connection with standard handlers
* @param {string} path - WebSocket path
* @returns {WebSocket}
*/
function createWebSocket(path) {
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
return new WebSocket(`${protocol}//${window.location.host}${path}`);
}
window.createWebSocket = createWebSocket;
/**
* Wait for xterm.js to load, then execute callback
* @param {function} callback - Function to call when xterm is ready
* @param {number} maxAttempts - Max attempts (default 20 = 2 seconds)
*/
function whenXtermReady(callback, maxAttempts = 20) {
const tryInit = (attempts) => {
if (typeof Terminal !== 'undefined' && typeof FitAddon !== 'undefined') {
callback();
} else if (attempts > 0) {
setTimeout(() => tryInit(attempts - 1), 100);
} else {
console.error('xterm.js failed to load');
}
};
tryInit(maxAttempts);
}
window.whenXtermReady = whenXtermReady;
// ============================================================================
// TERMINAL
// ============================================================================
/**
* Create a terminal with fit addon and resize observer
* @param {HTMLElement} container - Container element
* @param {object} extraOptions - Additional terminal options
* @param {function} onResize - Optional callback called with (cols, rows) after resize
* @returns {{term: Terminal, fitAddon: FitAddon, dispose: function}}
*/
function createTerminal(container, extraOptions = {}, onResize = null) {
container.innerHTML = '';
const term = new Terminal({
convertEol: true,
theme: TERMINAL_THEME,
fontSize: 13,
fontFamily: 'Monaco, Menlo, "Ubuntu Mono", monospace',
scrollback: 5000,
...extraOptions
});
const fitAddon = new FitAddon.FitAddon();
term.loadAddon(fitAddon);
term.open(container);
const handleResize = () => {
fitAddon.fit();
onResize?.(term.cols, term.rows);
};
// Use ResizeObserver only (handles both container and window resize)
const resizeObserver = new ResizeObserver(handleResize);
resizeObserver.observe(container);
handleResize(); // Initial fit
return {
term,
fitAddon,
dispose() {
resizeObserver.disconnect();
term.dispose();
}
};
}
/**
* Initialize a terminal and connect to WebSocket for streaming
*/
function initTerminal(elementId, taskId) {
const container = document.getElementById(elementId);
if (!container) {
console.error('Terminal container not found:', elementId);
return;
}
const wrapper = createTerminal(container);
const { term } = wrapper;
const ws = createWebSocket(`/ws/terminal/${taskId}`);
const taskKey = getTaskKey();
ws.onopen = () => {
term.write(`${ANSI.DIM}[Connected]${ANSI.RESET}${ANSI.CRLF}`);
setTerminalLoading(true);
localStorage.setItem(taskKey, taskId);
};
ws.onmessage = (event) => {
term.write(event.data);
if (event.data.includes('[Done]') || event.data.includes('[Failed]')) {
localStorage.removeItem(taskKey);
}
};
ws.onclose = () => setTerminalLoading(false);
ws.onerror = (error) => {
term.write(`${ANSI.RED}[WebSocket Error]${ANSI.RESET}${ANSI.CRLF}`);
console.error('WebSocket error:', error);
setTerminalLoading(false);
};
terminals[taskId] = { ...wrapper, ws };
return { term, ws };
}
window.initTerminal = initTerminal;
/**
* Initialize an interactive exec terminal
*/
function initExecTerminal(service, container, host) {
const containerEl = document.getElementById('exec-terminal-container');
const terminalEl = document.getElementById('exec-terminal');
if (!containerEl || !terminalEl) {
console.error('Exec terminal elements not found');
return;
}
containerEl.classList.remove('hidden');
// Clean up existing (use wrapper's dispose to clean up ResizeObserver)
if (execWs) { execWs.close(); execWs = null; }
if (execTerminalWrapper) { execTerminalWrapper.dispose(); execTerminalWrapper = null; }
// Create WebSocket first so resize callback can use it
execWs = createWebSocket(`/ws/exec/${service}/${container}/${host}`);
// Resize callback sends size to WebSocket
const sendSize = (cols, rows) => {
if (execWs && execWs.readyState === WebSocket.OPEN) {
execWs.send(JSON.stringify({ type: 'resize', cols, rows }));
}
};
execTerminalWrapper = createTerminal(terminalEl, { cursorBlink: true }, sendSize);
const term = execTerminalWrapper.term;
execWs.onopen = () => { sendSize(term.cols, term.rows); term.focus(); };
execWs.onmessage = (event) => term.write(event.data);
execWs.onclose = () => term.write(`${ANSI.CRLF}${ANSI.DIM}[Connection closed]${ANSI.RESET}${ANSI.CRLF}`);
execWs.onerror = (error) => {
term.write(`${ANSI.RED}[WebSocket Error]${ANSI.RESET}${ANSI.CRLF}`);
console.error('Exec WebSocket error:', error);
};
term.onData((data) => {
if (execWs && execWs.readyState === WebSocket.OPEN) {
execWs.send(data);
}
});
}
window.initExecTerminal = initExecTerminal;
/**
* Expand terminal collapse and scroll to it
*/
function expandTerminal() {
const toggle = document.getElementById('terminal-toggle');
if (toggle) toggle.checked = true;
const collapse = document.getElementById('terminal-collapse');
if (collapse) {
collapse.scrollIntoView({ behavior: 'smooth', block: 'start' });
}
}
/**
* Show/hide terminal loading spinner
*/
function setTerminalLoading(loading) {
const spinner = document.getElementById('terminal-spinner');
if (spinner) {
spinner.classList.toggle('hidden', !loading);
}
}
// ============================================================================
// EDITOR (Monaco)
// ============================================================================
/**
* Load Monaco editor dynamically (only once)
*/
function loadMonaco(callback) {
if (monacoLoaded) {
callback();
return;
}
if (monacoLoading) {
// Wait for it to load
const checkInterval = setInterval(() => {
if (monacoLoaded) {
clearInterval(checkInterval);
callback();
}
}, 100);
return;
}
monacoLoading = true;
// Load the Monaco loader script
const script = document.createElement('script');
script.src = 'https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs/loader.js';
script.onload = function() {
require.config({ paths: { vs: 'https://cdn.jsdelivr.net/npm/monaco-editor@0.52.2/min/vs' }});
require(['vs/editor/editor.main'], function() {
monacoLoaded = true;
monacoLoading = false;
callback();
});
};
document.head.appendChild(script);
}
/**
* Create a Monaco editor instance
* @param {HTMLElement} container - Container element
* @param {string} content - Initial content
* @param {string} language - Editor language (yaml, plaintext, etc.)
* @param {object} opts - Options: { readonly, onSave }
* @returns {object} Monaco editor instance
*/
function createEditor(container, content, language, opts = {}) {
const { readonly = false, onSave = null } = opts;
const options = {
value: content,
language,
theme: 'vs-dark',
minimap: { enabled: false },
automaticLayout: true,
scrollBeyondLastLine: false,
fontSize: 14,
lineNumbers: 'on',
wordWrap: 'on'
};
if (readonly) {
options.readOnly = true;
options.domReadOnly = true;
}
const editor = monaco.editor.create(container, options);
// Add Command+S / Ctrl+S handler for editable editors
if (!readonly) {
editor.addCommand(monaco.KeyMod.CtrlCmd | monaco.KeyCode.KeyS, () => {
if (onSave) {
onSave(editor);
} else {
saveAllEditors();
}
});
}
return editor;
}
window.createEditor = createEditor;
/**
* Initialize all Monaco editors on the page
*/
function initMonacoEditors() {
// Dispose existing editors
Object.values(editors).forEach(ed => ed?.dispose?.());
for (const key in editors) delete editors[key];
const editorConfigs = [
{ id: 'compose-editor', language: 'yaml', readonly: false },
{ id: 'env-editor', language: 'plaintext', readonly: false },
{ id: 'config-editor', language: 'yaml', readonly: false },
{ id: 'state-viewer', language: 'yaml', readonly: true }
];
// Check if any editor elements exist
const hasEditors = editorConfigs.some(({ id }) => document.getElementById(id));
if (!hasEditors) return;
// Load Monaco and create editors
loadMonaco(() => {
editorConfigs.forEach(({ id, language, readonly }) => {
const el = document.getElementById(id);
if (!el) return;
const content = el.dataset.content || '';
editors[id] = createEditor(el, content, language, { readonly });
if (!readonly) {
editors[id].saveUrl = el.dataset.saveUrl;
}
});
});
}
/**
* Save all editors
*/
async function saveAllEditors() {
const saveBtn = document.getElementById('save-btn') || document.getElementById('save-config-btn');
const results = [];
for (const [id, editor] of Object.entries(editors)) {
if (!editor || !editor.saveUrl) continue;
const content = editor.getValue();
try {
const response = await fetch(editor.saveUrl, {
method: 'PUT',
headers: { 'Content-Type': 'text/plain' },
body: content
});
const data = await response.json();
if (!response.ok || !data.success) {
results.push({ id, success: false, error: data.detail || 'Unknown error' });
} else {
results.push({ id, success: true });
}
} catch (e) {
results.push({ id, success: false, error: e.message });
}
}
// Show result
if (saveBtn && results.length > 0) {
saveBtn.textContent = 'Saved!';
setTimeout(() => saveBtn.textContent = saveBtn.id === 'save-config-btn' ? 'Save Config' : 'Save All', 2000);
refreshDashboard();
}
}
/**
* Initialize save button handler
*/
function initSaveButton() {
const saveBtn = document.getElementById('save-btn') || document.getElementById('save-config-btn');
if (!saveBtn) return;
saveBtn.onclick = saveAllEditors;
}
// ============================================================================
// UI HELPERS
// ============================================================================
/**
* Refresh dashboard partials by dispatching a custom event.
* Elements with hx-trigger="cf:refresh from:body" will automatically refresh.
*/
function refreshDashboard() {
document.body.dispatchEvent(new CustomEvent('cf:refresh'));
}
/**
* Filter sidebar services by name and host
*/
function sidebarFilter() {
const q = (document.getElementById('sidebar-filter')?.value || '').toLowerCase();
const h = document.getElementById('sidebar-host-select')?.value || '';
let n = 0;
document.querySelectorAll('#sidebar-services li').forEach(li => {
const show = (!q || li.dataset.svc.includes(q)) && (!h || !li.dataset.h || li.dataset.h === h);
li.hidden = !show;
if (show) n++;
});
document.getElementById('sidebar-count').textContent = '(' + n + ')';
}
window.sidebarFilter = sidebarFilter;
// Play intro animation on command palette button
function playFabIntro() {
const fab = document.getElementById('cmd-fab');
if (!fab) return;
setTimeout(() => {
fab.style.setProperty('--cmd-pos', '0');
fab.style.setProperty('--cmd-opacity', '1');
fab.style.setProperty('--cmd-blur', '30');
setTimeout(() => {
fab.style.removeProperty('--cmd-pos');
fab.style.removeProperty('--cmd-opacity');
fab.style.removeProperty('--cmd-blur');
}, 3000);
}, 500);
}
// ============================================================================
// COMMAND PALETTE
// ============================================================================
(function() {
const dialog = document.getElementById('cmd-palette');
const input = document.getElementById('cmd-input');
const list = document.getElementById('cmd-list');
const fab = document.getElementById('cmd-fab');
const themeBtn = document.getElementById('theme-btn');
if (!dialog || !input || !list) return;
// Load icons from template (rendered server-side from icons.html)
const iconTemplate = document.getElementById('cmd-icons');
const icons = {};
if (iconTemplate) {
iconTemplate.content.querySelectorAll('[data-icon]').forEach(el => {
icons[el.dataset.icon] = el.innerHTML;
});
}
// All available DaisyUI themes
const THEMES = ['light', 'dark', 'cupcake', 'bumblebee', 'emerald', 'corporate', 'synthwave', 'retro', 'cyberpunk', 'valentine', 'halloween', 'garden', 'forest', 'aqua', 'lofi', 'pastel', 'fantasy', 'wireframe', 'black', 'luxury', 'dracula', 'cmyk', 'autumn', 'business', 'acid', 'lemonade', 'night', 'coffee', 'winter', 'dim', 'nord', 'sunset', 'caramellatte', 'abyss', 'silk'];
const THEME_KEY = 'cf_theme';
const colors = { service: '#22c55e', action: '#eab308', nav: '#3b82f6', app: '#a855f7', theme: '#ec4899' };
let commands = [];
let filtered = [];
let selected = 0;
let originalTheme = null; // Store theme when palette opens for preview/restore
const post = (url) => () => htmx.ajax('POST', url, {swap: 'none'});
const nav = (url) => () => {
htmx.ajax('GET', url, {target: '#main-content', select: '#main-content', swap: 'outerHTML'}).then(() => {
history.pushState({}, '', url);
});
};
// Navigate to dashboard (if needed) and trigger action
const dashboardAction = (endpoint) => async () => {
if (window.location.pathname !== '/') {
await htmx.ajax('GET', '/', {target: '#main-content', select: '#main-content', swap: 'outerHTML'});
history.pushState({}, '', '/');
}
htmx.ajax('POST', `/api/${endpoint}`, {swap: 'none'});
};
// Apply theme and save to localStorage
const setTheme = (theme) => () => {
document.documentElement.setAttribute('data-theme', theme);
localStorage.setItem(THEME_KEY, theme);
};
// Preview theme without saving (for hover)
const previewTheme = (theme) => {
document.documentElement.setAttribute('data-theme', theme);
};
// Restore original theme (when closing without selection)
const restoreTheme = () => {
if (originalTheme) {
document.documentElement.setAttribute('data-theme', originalTheme);
}
};
// Generate color swatch HTML for a theme
const themeSwatch = (theme) => `<span class="flex gap-0.5" data-theme="${theme}"><span class="w-2 h-4 rounded-l bg-primary"></span><span class="w-2 h-4 bg-secondary"></span><span class="w-2 h-4 bg-accent"></span><span class="w-2 h-4 rounded-r bg-neutral"></span></span>`;
const cmd = (type, name, desc, action, icon = null, themeId = null) => ({ type, name, desc, action, icon, themeId });
// Reopen palette with theme filter
const openThemePicker = () => {
// Small delay to let dialog close before reopening
setTimeout(() => open('theme:'), 50);
};
function buildCommands() {
const actions = [
cmd('action', 'Apply', 'Make reality match config', dashboardAction('apply'), icons.check),
cmd('action', 'Refresh', 'Update state from reality', dashboardAction('refresh'), icons.refresh_cw),
cmd('app', 'Theme', 'Change color theme', openThemePicker, icons.palette),
cmd('app', 'Dashboard', 'Go to dashboard', nav('/'), icons.home),
cmd('app', 'Console', 'Go to console', nav('/console'), icons.terminal),
];
// Add service-specific actions if on a service page
const match = window.location.pathname.match(/^\/service\/(.+)$/);
if (match) {
const svc = decodeURIComponent(match[1]);
const svcCmd = (name, desc, endpoint, icon) => cmd('service', name, `${desc} ${svc}`, post(`/api/service/${svc}/${endpoint}`), icon);
actions.unshift(
svcCmd('Up', 'Start', 'up', icons.play),
svcCmd('Down', 'Stop', 'down', icons.square),
svcCmd('Restart', 'Restart', 'restart', icons.rotate_cw),
svcCmd('Pull', 'Pull', 'pull', icons.cloud_download),
svcCmd('Update', 'Pull + restart', 'update', icons.refresh_cw),
svcCmd('Logs', 'View logs for', 'logs', icons.file_text),
);
}
// Add nav commands for all services from sidebar
const services = [...document.querySelectorAll('#sidebar-services li[data-svc] a[href]')].map(a => {
const name = a.getAttribute('href').replace('/service/', '');
return cmd('nav', name, 'Go to service', nav(`/service/${name}`), icons.box);
});
// Add theme commands with color swatches
const currentTheme = document.documentElement.getAttribute('data-theme') || 'dark';
const themeCommands = THEMES.map(theme =>
cmd('theme', `theme: ${theme}`, theme === currentTheme ? '(current)' : 'Switch theme', setTheme(theme), themeSwatch(theme), theme)
);
commands = [...actions, ...services, ...themeCommands];
}
function filter() {
const q = input.value.toLowerCase();
filtered = commands.filter(c => c.name.toLowerCase().includes(q));
selected = Math.max(0, Math.min(selected, filtered.length - 1));
}
function render() {
list.innerHTML = filtered.map((c, i) => `
<a class="flex justify-between items-center px-3 py-2 rounded-r cursor-pointer hover:bg-base-200 border-l-4 ${i === selected ? 'bg-base-300' : ''}" style="border-left-color: ${colors[c.type] || '#666'}" data-idx="${i}"${c.themeId ? ` data-theme-id="${c.themeId}"` : ''}>
<span class="flex items-center gap-2">${c.icon || ''}<span>${c.name}</span></span>
<span class="opacity-40 text-xs">${c.desc}</span>
</a>
`).join('') || '<div class="opacity-50 p-2">No matches</div>';
// Scroll selected item into view
const sel = list.querySelector(`[data-idx="${selected}"]`);
if (sel) sel.scrollIntoView({ block: 'nearest' });
// Preview theme if selected item is a theme command
const selectedCmd = filtered[selected];
if (selectedCmd?.themeId) {
previewTheme(selectedCmd.themeId);
} else if (originalTheme) {
// Restore original when navigating away from theme commands
previewTheme(originalTheme);
}
}
function open(initialFilter = '') {
// Store original theme for preview/restore
originalTheme = document.documentElement.getAttribute('data-theme') || 'dark';
buildCommands();
selected = 0;
input.value = initialFilter;
filter();
// If opening theme picker, select current theme
if (initialFilter === 'theme:') {
const currentIdx = filtered.findIndex(c => c.themeId === originalTheme);
if (currentIdx >= 0) selected = currentIdx;
}
render();
dialog.showModal();
input.focus();
}
function close() {
dialog.close();
restoreTheme();
}
function exec() {
const cmd = filtered[selected];
if (cmd) {
if (cmd.themeId) {
// Theme command commits the previewed choice.
originalTheme = null;
}
dialog.close();
cmd.action();
}
}
// Keyboard: Cmd+K to open
document.addEventListener('keydown', e => {
if ((e.metaKey || e.ctrlKey) && e.key === 'k') {
e.preventDefault();
open();
}
});
// Input filtering
input.addEventListener('input', () => { filter(); render(); });
// Keyboard nav inside palette
dialog.addEventListener('keydown', e => {
if (!dialog.open) return;
if (e.key === 'ArrowDown') { e.preventDefault(); selected = Math.min(selected + 1, filtered.length - 1); render(); }
else if (e.key === 'ArrowUp') { e.preventDefault(); selected = Math.max(selected - 1, 0); render(); }
else if (e.key === 'Enter') { e.preventDefault(); exec(); }
});
// Click to execute
list.addEventListener('click', e => {
const a = e.target.closest('a[data-idx]');
if (a) {
selected = parseInt(a.dataset.idx, 10);
exec();
}
});
// Hover previews theme without changing selection
list.addEventListener('mouseover', e => {
const a = e.target.closest('a[data-theme-id]');
if (a) previewTheme(a.dataset.themeId);
});
// Mouse leaving list restores to selected item's theme (or original)
list.addEventListener('mouseleave', () => {
const cmd = filtered[selected];
previewTheme(cmd?.themeId || originalTheme);
});
// Restore theme when dialog closes without selection (Escape, backdrop click)
dialog.addEventListener('close', () => {
if (originalTheme) {
restoreTheme();
originalTheme = null;
}
});
// FAB click to open
if (fab) fab.addEventListener('click', () => open());
// Theme button opens palette with "theme:" filter
if (themeBtn) themeBtn.addEventListener('click', () => open('theme:'));
})();
// ============================================================================
// THEME PERSISTENCE
// ============================================================================
// Restore saved theme on load (also handled in inline script to prevent flash)
(function() {
const saved = localStorage.getItem('cf_theme');
if (saved) document.documentElement.setAttribute('data-theme', saved);
})();
// ============================================================================
// INITIALIZATION
// ============================================================================
/**
* Global keyboard shortcut handler
*/
function initKeyboardShortcuts() {
document.addEventListener('keydown', function(e) {
// Command+S (Mac) or Ctrl+S (Windows/Linux)
if ((e.metaKey || e.ctrlKey) && e.key === 's') {
// Only handle if we have editors and no Monaco editor is focused
if (Object.keys(editors).length > 0) {
// Check if any Monaco editor is focused
const focusedEditor = Object.values(editors).find(ed => ed?.hasTextFocus?.());
if (!focusedEditor) {
e.preventDefault();
saveAllEditors();
}
}
}
});
}
/**
* Initialize page components
*/
function initPage() {
initMonacoEditors();
initSaveButton();
}
/**
* Attempt to reconnect to an active task from localStorage
*/
function tryReconnectToTask() {
const taskId = localStorage.getItem(getTaskKey());
if (!taskId) return;
whenXtermReady(() => {
expandTerminal();
initTerminal('terminal-output', taskId);
});
}
// Initialize on page load
document.addEventListener('DOMContentLoaded', function() {
initPage();
initKeyboardShortcuts();
playFabIntro();
// Try to reconnect to any active task
tryReconnectToTask();
});
// Re-initialize after HTMX swaps main content
document.body.addEventListener('htmx:afterSwap', function(evt) {
if (evt.detail.target.id === 'main-content') {
initPage();
// Try to reconnect when navigating back to dashboard
tryReconnectToTask();
}
});
// Handle action responses (terminal streaming)
document.body.addEventListener('htmx:afterRequest', function(evt) {
if (!evt.detail.successful || !evt.detail.xhr) return;
const text = evt.detail.xhr.responseText;
// Only try to parse if it looks like JSON (starts with {)
if (!text || !text.trim().startsWith('{')) return;
try {
const response = JSON.parse(text);
if (response.task_id) {
expandTerminal();
whenXtermReady(() => initTerminal('terminal-output', response.task_id));
}
} catch (e) {
// Not valid JSON, ignore
}
});

View File

@@ -0,0 +1,176 @@
"""Streaming executor adapter for web UI."""
from __future__ import annotations
import asyncio
import os
import time
from typing import TYPE_CHECKING, Any
from compose_farm.executor import build_ssh_command
from compose_farm.ssh_keys import get_ssh_auth_sock
if TYPE_CHECKING:
from compose_farm.config import Config
# Environment variable to identify the web service (for self-update detection)
CF_WEB_SERVICE = os.environ.get("CF_WEB_SERVICE", "")
# ANSI escape codes for terminal output
RED = "\x1b[31m"
GREEN = "\x1b[32m"
DIM = "\x1b[2m"
RESET = "\x1b[0m"
CRLF = "\r\n"
# In-memory task registry
tasks: dict[str, dict[str, Any]] = {}
# How long to keep completed tasks (10 minutes)
TASK_TTL_SECONDS = 600
def cleanup_stale_tasks() -> int:
"""Remove tasks that completed more than TASK_TTL_SECONDS ago.
Returns the number of tasks removed.
"""
cutoff = time.time() - TASK_TTL_SECONDS
stale = [
tid
for tid, task in tasks.items()
if task.get("completed_at") and task["completed_at"] < cutoff
]
for tid in stale:
tasks.pop(tid, None)
return len(stale)
async def stream_to_task(task_id: str, message: str) -> None:
"""Send a message to a task's output buffer."""
if task_id in tasks:
tasks[task_id]["output"].append(message)
async def _stream_subprocess(task_id: str, args: list[str], env: dict[str, str]) -> int:
"""Run subprocess and stream output to task buffer. Returns exit code."""
process = await asyncio.create_subprocess_exec(
*args,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.STDOUT,
env=env,
)
if process.stdout:
async for line in process.stdout:
text = line.decode("utf-8", errors="replace")
# Convert \n to \r\n for xterm.js
if text.endswith("\n") and not text.endswith("\r\n"):
text = text[:-1] + "\r\n"
await stream_to_task(task_id, text)
return await process.wait()
async def run_cli_streaming(
config: Config,
args: list[str],
task_id: str,
) -> None:
"""Run a cf CLI command as subprocess and stream output to task buffer."""
try:
cmd = ["cf", *args, f"--config={config.config_path}"]
await stream_to_task(task_id, f"{DIM}$ {' '.join(['cf', *args])}{RESET}{CRLF}")
# Build environment with color support and SSH agent
env = {**os.environ, "FORCE_COLOR": "1", "TERM": "xterm-256color", "COLUMNS": "120"}
if ssh_sock := get_ssh_auth_sock():
env["SSH_AUTH_SOCK"] = ssh_sock
exit_code = await _stream_subprocess(task_id, cmd, env)
tasks[task_id]["status"] = "completed" if exit_code == 0 else "failed"
tasks[task_id]["completed_at"] = time.time()
except Exception as e:
await stream_to_task(task_id, f"{RED}Error: {e}{RESET}{CRLF}")
tasks[task_id]["status"] = "failed"
tasks[task_id]["completed_at"] = time.time()
def _is_self_update(service: str, command: str) -> bool:
"""Check if this is a self-update (updating the web service itself).
Self-updates need special handling because running 'down' on the container
we're running in would kill the process before 'up' can execute.
"""
if not CF_WEB_SERVICE or service != CF_WEB_SERVICE:
return False
# Commands that involve 'down' need SSH: update, restart, down
return command in ("update", "restart", "down")
async def _run_cli_via_ssh(
config: Config,
args: list[str],
task_id: str,
) -> None:
"""Run a cf CLI command via SSH for self-updates (survives container restart)."""
try:
host = config.get_host(CF_WEB_SERVICE)
cf_cmd = f"cf {' '.join(args)} --config={config.config_path}"
# Include task_id to prevent collision with concurrent updates
log_file = f"/tmp/cf-self-update-{task_id}.log" # noqa: S108
# setsid detaches command; tail streams output until SSH dies
remote_cmd = (
f"rm -f {log_file} && "
f"PATH=$HOME/.local/bin:/usr/local/bin:$PATH "
f"setsid sh -c '{cf_cmd} > {log_file} 2>&1' & "
f"sleep 0.3 && tail -f {log_file} 2>/dev/null"
)
await stream_to_task(task_id, f"{DIM}$ {cf_cmd}{RESET}{CRLF}")
await stream_to_task(task_id, f"{GREEN}Running via SSH (detached with setsid){RESET}{CRLF}")
ssh_args = build_ssh_command(host, remote_cmd, tty=False)
env = {**os.environ}
if ssh_sock := get_ssh_auth_sock():
env["SSH_AUTH_SOCK"] = ssh_sock
exit_code = await _stream_subprocess(task_id, ssh_args, env)
# Exit code 255 = SSH closed (container died during down) - expected for self-updates
if exit_code == 255: # noqa: PLR2004
await stream_to_task(
task_id,
f"{CRLF}{GREEN}Container restarting... refresh the page in a few seconds.{RESET}{CRLF}",
)
tasks[task_id]["status"] = "completed"
else:
tasks[task_id]["status"] = "completed" if exit_code == 0 else "failed"
tasks[task_id]["completed_at"] = time.time()
except Exception as e:
await stream_to_task(task_id, f"{RED}Error: {e}{RESET}{CRLF}")
tasks[task_id]["status"] = "failed"
tasks[task_id]["completed_at"] = time.time()
async def run_compose_streaming(
config: Config,
service: str,
command: str,
task_id: str,
) -> None:
"""Run a compose command (up/down/pull/restart) via CLI subprocess."""
# Split command into args (e.g., "up -d" -> ["up", "-d"])
args = command.split()
cli_cmd = args[0] # up, down, pull, restart
extra_args = args[1:] # -d, etc.
# Build CLI args
cli_args = [cli_cmd, service, *extra_args]
# Use SSH for self-updates to survive container restart
if _is_self_update(service, cli_cmd):
await _run_cli_via_ssh(config, cli_args, task_id)
else:
await run_cli_streaming(config, cli_args, task_id)

View File

@@ -0,0 +1,79 @@
{% from "partials/icons.html" import github, hamburger, palette %}
<!DOCTYPE html>
<html lang="en" data-theme="dark">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{% block title %}Compose Farm{% endblock %}</title>
<!-- daisyUI + Tailwind -->
<link href="https://cdn.jsdelivr.net/npm/daisyui@5" data-vendor="daisyui.css" rel="stylesheet" type="text/css" />
<link href="https://cdn.jsdelivr.net/npm/daisyui@5/themes.css" data-vendor="daisyui-themes.css" rel="stylesheet" type="text/css" />
<script src="https://cdn.jsdelivr.net/npm/@tailwindcss/browser@4" data-vendor="tailwind.js"></script>
<!-- xterm.js -->
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/css/xterm.css" data-vendor="xterm.css">
<!-- Custom styles -->
<link rel="stylesheet" href="/static/app.css">
<!-- Apply saved theme before render to prevent flash -->
<script>
(function() {
const t = localStorage.getItem('cf_theme');
if (t) document.documentElement.setAttribute('data-theme', t);
})();
</script>
</head>
<body class="min-h-screen bg-base-200">
<div class="drawer lg:drawer-open">
<input id="drawer-toggle" type="checkbox" class="drawer-toggle" />
<!-- Main content -->
<div class="drawer-content flex flex-col">
<!-- Mobile navbar with hamburger -->
<header class="navbar bg-base-100 border-b border-base-300 lg:hidden">
<label for="drawer-toggle" class="btn btn-ghost btn-square">
{{ hamburger() }}
</label>
<span class="font-semibold rainbow-hover">Compose Farm</span>
</header>
<main id="main-content" class="flex-1 p-6 overflow-y-auto">
{% block content %}{% endblock %}
</main>
</div>
<!-- Sidebar -->
<div class="drawer-side">
<label for="drawer-toggle" class="drawer-overlay" aria-label="close sidebar"></label>
<aside id="sidebar" class="w-64 bg-base-100 border-r border-base-300 flex flex-col min-h-screen">
<header class="p-4 border-b border-base-300">
<h2 class="text-lg font-semibold flex items-center gap-2">
<span class="rainbow-hover">Compose Farm</span>
<a href="https://github.com/basnijholt/compose-farm" target="_blank" title="GitHub" class="opacity-50 hover:opacity-100 transition-opacity">
{{ github() }}
</a>
<button type="button" id="theme-btn" class="opacity-50 hover:opacity-100 transition-opacity cursor-pointer" title="Change theme (opens command palette)">
{{ palette() }}
</button>
</h2>
</header>
<nav class="flex-1 overflow-y-auto p-2" hx-get="/partials/sidebar" hx-trigger="load, cf:refresh from:body" hx-swap="innerHTML">
<span class="loading loading-spinner loading-sm"></span> Loading...
</nav>
</aside>
</div>
</div>
<!-- Command Palette -->
{% include "partials/command_palette.html" %}
<!-- Scripts - HTMX first -->
<script src="https://unpkg.com/htmx.org@2.0.4" data-vendor="htmx.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@xterm/xterm@5.5.0/lib/xterm.js" data-vendor="xterm.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@xterm/addon-fit@0.10.0/lib/addon-fit.js" data-vendor="xterm-fit.js"></script>
<script src="/static/app.js"></script>
{% block scripts %}{% endblock %}
</body>
</html>

View File

@@ -0,0 +1,237 @@
{% extends "base.html" %}
{% from "partials/components.html" import page_header, collapse %}
{% from "partials/icons.html" import terminal, file_code, save %}
{% block title %}Console - Compose Farm{% endblock %}
{% block content %}
<div class="max-w-6xl">
{{ page_header("Console", "Terminal and editor access") }}
<!-- Host Selector -->
<div class="flex items-center gap-4 mb-4">
<label class="font-semibold">Host:</label>
<select id="console-host-select" class="select select-sm select-bordered">
{% for name in hosts %}
<option value="{{ name }}">{{ name }}{% if name == local_host %} (local){% endif %}</option>
{% endfor %}
</select>
<button id="console-connect-btn" class="btn btn-sm btn-primary" onclick="connectConsole()">Connect</button>
<span id="console-status" class="text-sm opacity-60"></span>
</div>
<!-- Terminal -->
{% call collapse("Terminal", checked=True, icon=terminal(), subtitle="Full shell access to selected host") %}
<div id="console-terminal" class="w-full bg-base-300 rounded-lg overflow-hidden resize-y" style="height: 384px; min-height: 200px;"></div>
{% endcall %}
<!-- Editor -->
{% call collapse("Editor", checked=True, icon=file_code()) %}
<div class="flex items-center justify-between mb-2">
<div class="flex items-center gap-4">
<input type="text" id="console-file-path" class="input input-sm input-bordered w-96" placeholder="Enter file path (e.g., ~/docker-compose.yaml)" value="{{ config_path }}">
<button class="btn btn-sm btn-outline" onclick="loadFile()">Open</button>
</div>
<div class="flex items-center gap-2">
<span id="editor-status" class="text-sm opacity-60"></span>
<button id="console-save-btn" class="btn btn-sm btn-primary" onclick="saveFile()">{{ save() }} Save</button>
</div>
</div>
<div id="console-editor" class="resize-y overflow-hidden rounded-lg" style="height: 512px; min-height: 200px;"></div>
{% endcall %}
</div>
<script>
// Use var to allow re-declaration on HTMX navigation
var consoleTerminalWrapper = null; // {term, dispose}
var consoleWs = null;
var consoleEditor = null;
var currentFilePath = null;
var currentHost = null;
// Helper to show status with monospace path
function setEditorStatus(prefix, path) {
const statusEl = document.getElementById('editor-status');
const escaped = path.replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
statusEl.innerHTML = `${prefix} <code class="font-mono">${escaped}</code>`;
}
function connectConsole() {
const hostSelect = document.getElementById('console-host-select');
const host = hostSelect.value;
const statusEl = document.getElementById('console-status');
const terminalEl = document.getElementById('console-terminal');
if (!host) {
statusEl.textContent = 'Please select a host';
return;
}
currentHost = host;
// Clean up existing connection (use wrapper's dispose to clean up ResizeObserver)
if (consoleWs) {
consoleWs.close();
consoleWs = null;
}
if (consoleTerminalWrapper) {
consoleTerminalWrapper.dispose();
consoleTerminalWrapper = null;
}
statusEl.textContent = 'Connecting...';
// Create WebSocket
consoleWs = createWebSocket(`/ws/shell/${host}`);
// Resize callback - createTerminal's ResizeObserver calls this on container resize
const sendSize = (cols, rows) => {
if (consoleWs && consoleWs.readyState === WebSocket.OPEN) {
consoleWs.send(JSON.stringify({ type: 'resize', cols, rows }));
}
};
// Create terminal with resize callback
consoleTerminalWrapper = createTerminal(terminalEl, { cursorBlink: true }, sendSize);
const term = consoleTerminalWrapper.term;
consoleWs.onopen = () => {
statusEl.textContent = `Connected to ${host}`;
sendSize(term.cols, term.rows);
term.focus();
// Auto-load the default file once editor is ready
const pathInput = document.getElementById('console-file-path');
if (pathInput && pathInput.value) {
const tryLoad = () => consoleEditor ? loadFile() : setTimeout(tryLoad, 100);
tryLoad();
}
};
consoleWs.onmessage = (event) => term.write(event.data);
consoleWs.onclose = () => {
statusEl.textContent = 'Disconnected';
term.write(`${ANSI.CRLF}${ANSI.DIM}[Connection closed]${ANSI.RESET}${ANSI.CRLF}`);
};
consoleWs.onerror = (error) => {
statusEl.textContent = 'Connection error';
term.write(`${ANSI.RED}[WebSocket Error]${ANSI.RESET}${ANSI.CRLF}`);
console.error('Console WebSocket error:', error);
};
// Send input to WebSocket
term.onData((data) => {
if (consoleWs && consoleWs.readyState === WebSocket.OPEN) {
consoleWs.send(data);
}
});
}
function initConsoleEditor() {
const editorEl = document.getElementById('console-editor');
if (!editorEl || consoleEditor) return;
loadMonaco(() => {
consoleEditor = createEditor(editorEl, '', 'plaintext', { onSave: saveFile });
});
}
async function loadFile() {
const pathInput = document.getElementById('console-file-path');
const path = pathInput.value.trim();
const statusEl = document.getElementById('editor-status');
if (!path) {
statusEl.textContent = 'Enter a file path';
return;
}
if (!currentHost) {
statusEl.textContent = 'Connect to a host first';
return;
}
setEditorStatus('Loading', path + '...');
try {
const response = await fetch(`/api/console/file?host=${encodeURIComponent(currentHost)}&path=${encodeURIComponent(path)}`);
const data = await response.json();
if (!response.ok || !data.success) {
statusEl.textContent = data.detail || 'Failed to load file';
return;
}
const language = getLanguageFromPath(path);
if (consoleEditor) {
consoleEditor.setValue(data.content);
monaco.editor.setModelLanguage(consoleEditor.getModel(), language);
currentFilePath = path; // Only set after content is loaded
setEditorStatus('Loaded:', path);
} else {
statusEl.textContent = 'Editor not ready';
}
} catch (e) {
statusEl.textContent = `Error: ${e.message}`;
}
}
async function saveFile() {
const statusEl = document.getElementById('editor-status');
if (!currentFilePath) {
statusEl.textContent = 'No file loaded';
return;
}
if (!currentHost) {
statusEl.textContent = 'Not connected to a host';
return;
}
if (!consoleEditor) {
statusEl.textContent = 'Editor not ready';
return;
}
setEditorStatus('Saving', currentFilePath + '...');
try {
const content = consoleEditor.getValue();
const response = await fetch(`/api/console/file?host=${encodeURIComponent(currentHost)}&path=${encodeURIComponent(currentFilePath)}`, {
method: 'PUT',
headers: { 'Content-Type': 'text/plain' },
body: content
});
const data = await response.json();
if (!response.ok || !data.success) {
statusEl.textContent = data.detail || 'Failed to save file';
return;
}
setEditorStatus('Saved:', currentFilePath);
} catch (e) {
statusEl.textContent = `Error: ${e.message}`;
}
}
// Initialize editor and auto-connect to first host
function init() {
initConsoleEditor();
const hostSelect = document.getElementById('console-host-select');
if (hostSelect && hostSelect.options.length > 0) {
connectConsole();
}
}
// On HTMX navigation, dependencies (app.js) are already loaded.
// On hard refresh, this script runs before app.js, so wait for DOMContentLoaded.
if (typeof createTerminal === 'function') {
init();
} else {
document.addEventListener('DOMContentLoaded', init);
}
</script>
{% endblock content %}

View File

@@ -0,0 +1,85 @@
{% extends "base.html" %}
{% from "partials/components.html" import page_header, collapse, stat_card, table, action_btn %}
{% from "partials/icons.html" import check, refresh_cw, save, settings, server, database %}
{% block title %}Dashboard - Compose Farm{% endblock %}
{% block content %}
<div class="max-w-5xl">
{{ page_header("Compose Farm", "Cluster overview and management") }}
<!-- Stats Cards -->
<div id="stats-cards" class="grid grid-cols-2 md:grid-cols-4 gap-4 mb-6"
hx-get="/partials/stats" hx-trigger="cf:refresh from:body" hx-swap="innerHTML">
{% include "partials/stats.html" %}
</div>
<!-- Global Actions -->
<div class="flex flex-wrap gap-2 mb-6">
{{ action_btn("Apply", "/api/apply", "primary", "Make reality match config", check()) }}
{{ action_btn("Refresh", "/api/refresh", "outline", "Update state from reality", refresh_cw()) }}
<button id="save-config-btn" class="btn btn-outline">{{ save() }} Save Config</button>
</div>
{% include "partials/terminal.html" %}
<!-- Config Error Banner -->
<div id="config-error"
hx-get="/partials/config-error"
hx-trigger="cf:refresh from:body"
hx-swap="innerHTML">
{% if config_error %}
{% include "partials/config_error.html" %}
{% endif %}
</div>
<!-- Config Editor -->
{% call collapse("Edit Config", badge="compose-farm.yaml", icon=settings(), checked=config_error) %}
<div class="editor-wrapper yaml-wrapper">
<div id="config-editor" class="yaml-editor" data-content="{{ config_content | e }}" data-save-url="/api/config"></div>
</div>
{% endcall %}
<!-- Pending Operations -->
<div id="pending-operations"
hx-get="/partials/pending" hx-trigger="cf:refresh from:body" hx-swap="innerHTML">
{% include "partials/pending.html" %}
</div>
<!-- Services by Host -->
<div id="services-by-host"
hx-get="/partials/services-by-host" hx-trigger="cf:refresh from:body" hx-swap="innerHTML">
{% include "partials/services_by_host.html" %}
</div>
<!-- Hosts Configuration -->
{% call collapse("Hosts (" ~ (hosts | length) ~ ")", icon=server()) %}
{% call table() %}
<thead>
<tr>
<th>Name</th>
<th>Address</th>
<th>User</th>
<th>Port</th>
</tr>
</thead>
<tbody>
{% for name, host in hosts.items() %}
<tr class="hover:bg-base-300">
<td class="font-semibold">{{ name }}</td>
<td><code class="text-sm">{{ host.address }}</code></td>
<td><code class="text-sm">{{ host.user }}</code></td>
<td><code class="text-sm">{{ host.port }}</code></td>
</tr>
{% endfor %}
</tbody>
{% endcall %}
{% endcall %}
<!-- State Viewer -->
{% call collapse("Raw State", badge="compose-farm-state.yaml", icon=database()) %}
<div class="editor-wrapper viewer-wrapper">
<div id="state-viewer" class="yaml-viewer" data-content="{{ state_content | e }}"></div>
</div>
{% endcall %}
</div>
{% endblock %}

View File

@@ -0,0 +1,36 @@
{% from "partials/icons.html" import search, play, square, rotate_cw, cloud_download, refresh_cw, file_text, check, home, terminal, box, palette %}
<!-- Icons for command palette (referenced by JS) -->
<template id="cmd-icons">
<span data-icon="play">{{ play() }}</span>
<span data-icon="square">{{ square() }}</span>
<span data-icon="rotate_cw">{{ rotate_cw() }}</span>
<span data-icon="cloud_download">{{ cloud_download() }}</span>
<span data-icon="refresh_cw">{{ refresh_cw() }}</span>
<span data-icon="file_text">{{ file_text() }}</span>
<span data-icon="check">{{ check() }}</span>
<span data-icon="home">{{ home() }}</span>
<span data-icon="terminal">{{ terminal() }}</span>
<span data-icon="box">{{ box() }}</span>
<span data-icon="palette">{{ palette() }}</span>
</template>
<dialog id="cmd-palette" class="modal">
<div class="modal-box max-w-lg p-0">
<label class="input input-lg bg-base-100 border-0 border-b border-base-300 w-full rounded-none rounded-t-box sticky top-0 z-10 focus-within:outline-none">
{{ search(20) }}
<input type="text" id="cmd-input" class="grow" placeholder="Type a command..." autocomplete="off" />
<kbd class="kbd kbd-sm opacity-50">esc</kbd>
</label>
<div id="cmd-list" class="flex flex-col p-2 max-h-80 overflow-y-auto">
<!-- Populated by JS -->
</div>
</div>
<form method="dialog" class="modal-backdrop"><button>close</button></form>
</dialog>
<!-- Floating button to open command palette -->
<button id="cmd-fab" class="fixed bottom-6 right-6 z-50" title="Command Palette (⌘K)">
<div class="cmd-fab-inner">
<span>⌘ + K</span>
</div>
</button>

View File

@@ -0,0 +1,53 @@
{# Page header with title and optional subtitle (supports HTML) #}
{% macro page_header(title, subtitle=None) %}
<div class="mb-6">
<h1 class="text-3xl font-bold rainbow-hover">{{ title }}</h1>
{% if subtitle %}
<p class="text-base-content/60 text-lg">{{ subtitle | safe }}</p>
{% endif %}
</div>
{% endmacro %}
{# Collapsible section #}
{% macro collapse(title, id=None, checked=False, badge=None, icon=None, subtitle=None) %}
<div class="collapse collapse-arrow bg-base-100 shadow mb-4">
<input type="checkbox" {% if id %}id="{{ id }}"{% endif %} {% if checked %}checked{% endif %} />
<div class="collapse-title font-semibold flex items-center gap-2">
{% if icon %}{{ icon }}{% endif %}{{ title }}
{% if badge %}<code class="text-xs ml-2 opacity-60">{{ badge }}</code>{% endif %}
{% if subtitle %}<span class="text-xs opacity-50 font-normal">{{ subtitle }}</span>{% endif %}
</div>
<div class="collapse-content">
{{ caller() }}
</div>
</div>
{% endmacro %}
{# Action button with htmx #}
{% macro action_btn(label, url, style="outline", title=None, icon=None) %}
<button hx-post="{{ url }}"
hx-swap="none"
class="btn btn-{{ style }}"
{% if title %}title="{{ title }}"{% endif %}>
{% if icon %}{{ icon }}{% endif %}{{ label }}
</button>
{% endmacro %}
{# Stat card for dashboard #}
{% macro stat_card(label, value, color=None, icon=None) %}
<div class="card bg-base-100 shadow">
<div class="card-body items-center text-center">
<h2 class="card-title text-base-content/60 text-sm gap-1">{% if icon %}{{ icon }}{% endif %}{{ label }}</h2>
<p class="text-4xl font-bold {% if color %}text-{{ color }}{% endif %}">{{ value }}</p>
</div>
</div>
{% endmacro %}
{# Data table wrapper #}
{% macro table() %}
<div class="overflow-x-auto">
<table class="table table-zebra">
{{ caller() }}
</table>
</div>
{% endmacro %}

View File

@@ -0,0 +1,8 @@
{% from "partials/icons.html" import alert_triangle %}
<div class="alert alert-error mb-4">
{{ alert_triangle(size=20) }}
<div>
<h3 class="font-bold">Configuration Error</h3>
<div class="text-sm">{{ config_error }}</div>
</div>
</div>

View File

@@ -0,0 +1,35 @@
{# Container list for a service on a single host #}
{% from "partials/icons.html" import terminal %}
{% macro container_row(service, container, host) %}
<div class="flex items-center gap-2 mb-2">
{% if container.State == "running" %}
<span class="badge badge-success">running</span>
{% elif container.State == "unknown" %}
<span class="badge badge-ghost"><span class="loading loading-spinner loading-xs"></span></span>
{% elif container.State == "exited" %}
{% if container.ExitCode == 0 %}
<span class="badge badge-neutral">exited (0)</span>
{% else %}
<span class="badge badge-error">exited ({{ container.ExitCode }})</span>
{% endif %}
{% elif container.State == "created" %}
<span class="badge badge-neutral">created</span>
{% else %}
<span class="badge badge-warning">{{ container.State }}</span>
{% endif %}
<code class="text-sm flex-1">{{ container.Name }}</code>
<button class="btn btn-sm btn-outline"
onclick="initExecTerminal('{{ service }}', '{{ container.Name }}', '{{ host }}')">
{{ terminal() }} Shell
</button>
</div>
{% endmacro %}
{% macro host_containers(service, host, containers, show_header=False) %}
{% if show_header %}
<div class="font-semibold text-sm mt-3 mb-1">{{ host }}</div>
{% endif %}
{% for container in containers %}
{{ container_row(service, container, host) }}
{% endfor %}
{% endmacro %}

View File

@@ -0,0 +1,154 @@
{# Lucide-style icons (https://lucide.dev) - 24x24 viewBox, 2px stroke, round caps #}
{# Brand icons #}
{% macro github(size=16) %}
<svg height="{{ size }}" width="{{ size }}" viewBox="0 0 16 16" fill="currentColor"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg>
{% endmacro %}
{# UI icons #}
{% macro hamburger(size=20) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<line x1="4" x2="20" y1="6" y2="6"/><line x1="4" x2="20" y1="12" y2="12"/><line x1="4" x2="20" y1="18" y2="18"/>
</svg>
{% endmacro %}
{% macro command(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M15 6v12a3 3 0 1 0 3-3H6a3 3 0 1 0 3 3V6a3 3 0 1 0-3 3h12a3 3 0 1 0-3-3"/>
</svg>
{% endmacro %}
{# Action icons #}
{% macro play(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polygon points="6 3 20 12 6 21 6 3"/>
</svg>
{% endmacro %}
{% macro square(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<rect width="14" height="14" x="5" y="5" rx="2"/>
</svg>
{% endmacro %}
{% macro rotate_cw(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21 12a9 9 0 1 1-9-9c2.52 0 4.93 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/>
</svg>
{% endmacro %}
{% macro download(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21 15v4a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2v-4"/><polyline points="7 10 12 15 17 10"/><line x1="12" x2="12" y1="15" y2="3"/>
</svg>
{% endmacro %}
{% macro cloud_download(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M12 13v8l-4-4"/><path d="m12 21 4-4"/><path d="M4.393 15.269A7 7 0 1 1 15.71 8h1.79a4.5 4.5 0 0 1 2.436 8.284"/>
</svg>
{% endmacro %}
{% macro file_text(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M10 9H8"/><path d="M16 13H8"/><path d="M16 17H8"/>
</svg>
{% endmacro %}
{% macro save(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M15.2 3a2 2 0 0 1 1.4.6l3.8 3.8a2 2 0 0 1 .6 1.4V19a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2V5a2 2 0 0 1 2-2z"/><path d="M17 21v-7a1 1 0 0 0-1-1H8a1 1 0 0 0-1 1v7"/><path d="M7 3v4a1 1 0 0 0 1 1h7"/>
</svg>
{% endmacro %}
{% macro check(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M20 6 9 17l-5-5"/>
</svg>
{% endmacro %}
{% macro refresh_cw(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M3 12a9 9 0 0 1 9-9 9.75 9.75 0 0 1 6.74 2.74L21 8"/><path d="M21 3v5h-5"/><path d="M21 12a9 9 0 0 1-9 9 9.75 9.75 0 0 1-6.74-2.74L3 16"/><path d="M8 16H3v5"/>
</svg>
{% endmacro %}
{% macro terminal(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<polyline points="4 17 10 11 4 5"/><line x1="12" x2="20" y1="19" y2="19"/>
</svg>
{% endmacro %}
{# Stats/navigation icons #}
{% macro server(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<rect width="20" height="8" x="2" y="2" rx="2" ry="2"/><rect width="20" height="8" x="2" y="14" rx="2" ry="2"/><line x1="6" x2="6.01" y1="6" y2="6"/><line x1="6" x2="6.01" y1="18" y2="18"/>
</svg>
{% endmacro %}
{% macro layers(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="m12.83 2.18a2 2 0 0 0-1.66 0L2.6 6.08a1 1 0 0 0 0 1.83l8.58 3.91a2 2 0 0 0 1.66 0l8.58-3.9a1 1 0 0 0 0-1.83Z"/><path d="m22 17.65-9.17 4.16a2 2 0 0 1-1.66 0L2 17.65"/><path d="m22 12.65-9.17 4.16a2 2 0 0 1-1.66 0L2 12.65"/>
</svg>
{% endmacro %}
{% macro circle_check(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="10"/><path d="m9 12 2 2 4-4"/>
</svg>
{% endmacro %}
{% macro circle_x(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="12" cy="12" r="10"/><path d="m15 9-6 6"/><path d="m9 9 6 6"/>
</svg>
{% endmacro %}
{% macro home(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M15 21v-8a1 1 0 0 0-1-1h-4a1 1 0 0 0-1 1v8"/><path d="M3 10a2 2 0 0 1 .709-1.528l7-5.999a2 2 0 0 1 2.582 0l7 5.999A2 2 0 0 1 21 10v9a2 2 0 0 1-2 2H5a2 2 0 0 1-2-2z"/>
</svg>
{% endmacro %}
{% macro box(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M21 8a2 2 0 0 0-1-1.73l-7-4a2 2 0 0 0-2 0l-7 4A2 2 0 0 0 3 8v8a2 2 0 0 0 1 1.73l7 4a2 2 0 0 0 2 0l7-4A2 2 0 0 0 21 16Z"/><path d="m3.3 7 8.7 5 8.7-5"/><path d="M12 22V12"/>
</svg>
{% endmacro %}
{# Section icons #}
{% macro settings(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M12.22 2h-.44a2 2 0 0 0-2 2v.18a2 2 0 0 1-1 1.73l-.43.25a2 2 0 0 1-2 0l-.15-.08a2 2 0 0 0-2.73.73l-.22.38a2 2 0 0 0 .73 2.73l.15.1a2 2 0 0 1 1 1.72v.51a2 2 0 0 1-1 1.74l-.15.09a2 2 0 0 0-.73 2.73l.22.38a2 2 0 0 0 2.73.73l.15-.08a2 2 0 0 1 2 0l.43.25a2 2 0 0 1 1 1.73V20a2 2 0 0 0 2 2h.44a2 2 0 0 0 2-2v-.18a2 2 0 0 1 1-1.73l.43-.25a2 2 0 0 1 2 0l.15.08a2 2 0 0 0 2.73-.73l.22-.39a2 2 0 0 0-.73-2.73l-.15-.08a2 2 0 0 1-1-1.74v-.5a2 2 0 0 1 1-1.74l.15-.09a2 2 0 0 0 .73-2.73l-.22-.38a2 2 0 0 0-2.73-.73l-.15.08a2 2 0 0 1-2 0l-.43-.25a2 2 0 0 1-1-1.73V4a2 2 0 0 0-2-2z"/><circle cx="12" cy="12" r="3"/>
</svg>
{% endmacro %}
{% macro file_code(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="M10 12.5 8 15l2 2.5"/><path d="m14 12.5 2 2.5-2 2.5"/><path d="M14 2v4a2 2 0 0 0 2 2h4"/><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7z"/>
</svg>
{% endmacro %}
{% macro database(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<ellipse cx="12" cy="5" rx="9" ry="3"/><path d="M3 5V19A9 3 0 0 0 21 19V5"/><path d="M3 12A9 3 0 0 0 21 12"/>
</svg>
{% endmacro %}
{% macro search(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="11" cy="11" r="8"/><path d="m21 21-4.3-4.3"/>
</svg>
{% endmacro %}
{% macro alert_triangle(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<path d="m21.73 18-8-14a2 2 0 0 0-3.48 0l-8 14A2 2 0 0 0 4 21h16a2 2 0 0 0 1.73-3"/><path d="M12 9v4"/><path d="M12 17h.01"/>
</svg>
{% endmacro %}
{% macro palette(size=16) %}
<svg xmlns="http://www.w3.org/2000/svg" width="{{ size }}" height="{{ size }}" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
<circle cx="13.5" cy="6.5" r="0.5" fill="currentColor"/><circle cx="17.5" cy="10.5" r="0.5" fill="currentColor"/><circle cx="8.5" cy="7.5" r="0.5" fill="currentColor"/><circle cx="6.5" cy="12.5" r="0.5" fill="currentColor"/><path d="M12 2C6.5 2 2 6.5 2 12s4.5 10 10 10c.926 0 1.648-.746 1.648-1.688 0-.437-.18-.835-.437-1.125-.29-.289-.438-.652-.438-1.125a1.64 1.64 0 0 1 1.668-1.668h1.996c3.051 0 5.555-2.503 5.555-5.555C21.965 6.012 17.461 2 12 2z"/>
</svg>
{% endmacro %}

View File

@@ -0,0 +1,37 @@
{% from "partials/components.html" import collapse %}
{% from "partials/icons.html" import circle_check %}
{% if orphaned or migrations or not_started %}
{% call collapse("Pending Operations", id="pending-collapse", checked=expanded|default(true)) %}
{% if orphaned %}
<h4 class="font-semibold mt-2 mb-1">Orphaned Services (will be stopped)</h4>
<ul class="list-disc list-inside mb-4">
{% for svc, host in orphaned.items() %}
<li><a href="/service/{{ svc }}" class="badge badge-warning hover:badge-primary">{{ svc }}</a> on {{ host }}</li>
{% endfor %}
</ul>
{% endif %}
{% if migrations %}
<h4 class="font-semibold mt-2 mb-1">Services Needing Migration</h4>
<ul class="list-disc list-inside mb-4">
{% for svc in migrations %}
<li><a href="/service/{{ svc }}" class="badge badge-info hover:badge-primary">{{ svc }}</a></li>
{% endfor %}
</ul>
{% endif %}
{% if not_started %}
<h4 class="font-semibold mt-2 mb-1">Services Not Started</h4>
<ul class="menu menu-horizontal bg-base-200 rounded-box mb-2">
{% for svc in not_started | sort %}
<li><a href="/service/{{ svc }}">{{ svc }}</a></li>
{% endfor %}
</ul>
{% endif %}
{% endcall %}
{% else %}
<div role="alert" class="alert alert-success mb-4">
<span class="shrink-0">{{ circle_check(24) }}</span>
<span>All services are in sync with configuration.</span>
</div>
{% endif %}

View File

@@ -0,0 +1,39 @@
{% from "partials/components.html" import collapse %}
{% from "partials/icons.html" import layers, search %}
{% call collapse("Services by Host", id="services-by-host-collapse", checked=expanded|default(true), icon=layers()) %}
<div class="flex flex-wrap gap-2 mb-4 items-center">
<label class="input input-sm input-bordered flex items-center gap-2 bg-base-200">
{{ search() }}<input type="text" id="sbh-filter" class="w-32" placeholder="Filter..." onkeyup="sbhFilter()" />
</label>
<select id="sbh-host-select" class="select select-sm select-bordered bg-base-200" onchange="sbhFilter()">
<option value="">All hosts</option>
{% for h in services_by_host.keys() | sort %}<option value="{{ h }}">{{ h }}</option>{% endfor %}
</select>
</div>
{% for host_name, host_services in services_by_host.items() | sort %}
<div class="sbh-group" data-h="{{ host_name }}">
<h4 class="font-semibold mt-3 mb-1">{{ host_name }}{% if host_name in hosts %}<code class="text-xs ml-2 opacity-60">{{ hosts[host_name].address }}</code>{% endif %}</h4>
<ul class="menu menu-horizontal bg-base-200 rounded-box mb-2 flex-wrap">
{% for svc in host_services | sort %}<li data-s="{{ svc | lower }}"><a href="/service/{{ svc }}">{{ svc }}</a></li>{% endfor %}
</ul>
</div>
{% else %}
<p class="text-base-content/60 italic">No services currently running.</p>
{% endfor %}
<script>
function sbhFilter() {
const q = (document.getElementById('sbh-filter')?.value || '').toLowerCase();
const h = document.getElementById('sbh-host-select')?.value || '';
document.querySelectorAll('.sbh-group').forEach(g => {
if (h && g.dataset.h !== h) { g.hidden = true; return; }
let n = 0;
g.querySelectorAll('li[data-s]').forEach(li => {
const show = !q || li.dataset.s.includes(q);
li.hidden = !show;
if (show) n++;
});
g.hidden = !n;
});
}
</script>
{% endcall %}

View File

@@ -0,0 +1,33 @@
{% from "partials/icons.html" import home, search, terminal %}
<!-- Navigation Links -->
<div class="mb-4">
<ul class="menu" hx-boost="true" hx-target="#main-content" hx-select="#main-content" hx-swap="outerHTML">
<li><a href="/" class="font-semibold">{{ home() }} Dashboard</a></li>
<li><a href="/console" class="font-semibold">{{ terminal() }} Console</a></li>
</ul>
</div>
<!-- Services Section -->
<div class="mb-4">
<h4 class="text-xs uppercase tracking-wide text-base-content/60 px-3 py-1">Services <span class="opacity-50" id="sidebar-count">({{ services | length }})</span></h4>
<div class="px-2 mb-2 flex flex-col gap-1">
<label class="input input-xs flex items-center gap-2 bg-base-200">
{{ search(14) }}<input type="text" id="sidebar-filter" placeholder="Filter..." onkeyup="sidebarFilter()" />
</label>
<select id="sidebar-host-select" class="select select-xs bg-base-200 w-full" onchange="sidebarFilter()">
<option value="">All hosts</option>
{% for h in hosts %}<option value="{{ h }}">{{ h }}{% if h == local_host %} (local){% endif %}</option>{% endfor %}
</select>
</div>
<ul class="menu menu-sm" id="sidebar-services" hx-boost="true" hx-target="#main-content" hx-select="#main-content" hx-swap="outerHTML">
{% for service in services %}
<li data-svc="{{ service | lower }}" data-h="{{ service_hosts.get(service, '') }}">
<a href="/service/{{ service }}" class="flex items-center gap-2">
{% if service in state %}<span class="status status-success" title="In state file"></span>
{% else %}<span class="status status-neutral" title="Not in state file"></span>{% endif %}
{{ service }}
</a>
</li>
{% endfor %}
</ul>
</div>

View File

@@ -0,0 +1,6 @@
{% from "partials/components.html" import stat_card %}
{% from "partials/icons.html" import server, layers, circle_check, circle_x %}
{{ stat_card("Hosts", hosts | length, icon=server()) }}
{{ stat_card("Services", services | length, icon=layers()) }}
{{ stat_card("Running", running_count, "success", circle_check()) }}
{{ stat_card("Stopped", stopped_count, icon=circle_x()) }}

View File

@@ -0,0 +1,14 @@
{% from "partials/icons.html" import terminal %}
<!-- Shared Terminal Component -->
<div class="collapse collapse-arrow bg-base-100 shadow mb-4" id="terminal-collapse">
<input type="checkbox" id="terminal-toggle" />
<div class="collapse-title font-medium flex items-center gap-2">
{{ terminal() }} Terminal Output
<span id="terminal-spinner" class="loading loading-spinner loading-sm hidden"></span>
</div>
<div class="collapse-content">
<div id="terminal-container" class="bg-[#1a1a2e] rounded-lg h-[300px] border border-white/10 resize-y overflow-hidden">
<div id="terminal-output" class="h-full"></div>
</div>
</div>
</div>

View File

@@ -0,0 +1,67 @@
{% extends "base.html" %}
{% from "partials/components.html" import collapse, action_btn %}
{% from "partials/icons.html" import play, square, rotate_cw, download, cloud_download, file_text, save, file_code, terminal, settings %}
{% block title %}{{ name }} - Compose Farm{% endblock %}
{% block content %}
<div class="max-w-5xl">
<div class="mb-6">
<h1 class="text-3xl font-bold rainbow-hover">{{ name }}</h1>
<div class="flex flex-wrap items-center gap-2 mt-2">
{% if current_host %}
<span class="badge badge-success">Running on {{ current_host }}</span>
{% else %}
<span class="badge badge-neutral">Not running</span>
{% endif %}
<span class="badge badge-outline">{{ hosts | join(', ') }}</span>
</div>
</div>
<!-- Action Buttons -->
<div class="flex flex-wrap gap-2 mb-6">
<!-- Lifecycle -->
{{ action_btn("Up", "/api/service/" ~ name ~ "/up", "primary", "Start service (docker compose up -d)", play()) }}
{{ action_btn("Down", "/api/service/" ~ name ~ "/down", "outline", "Stop service (docker compose down)", square()) }}
{{ action_btn("Restart", "/api/service/" ~ name ~ "/restart", "secondary", "Restart service (down + up)", rotate_cw()) }}
{{ action_btn("Update", "/api/service/" ~ name ~ "/update", "accent", "Update to latest (pull + build + down + up)", download()) }}
<div class="divider divider-horizontal mx-0"></div>
<!-- Other -->
{{ action_btn("Pull", "/api/service/" ~ name ~ "/pull", "outline", "Pull latest images (no restart)", cloud_download()) }}
{{ action_btn("Logs", "/api/service/" ~ name ~ "/logs", "outline", "Show recent logs", file_text()) }}
<button id="save-btn" class="btn btn-outline">{{ save() }} Save All</button>
</div>
{% call collapse("Compose File", badge=compose_path, icon=file_code()) %}
<div class="editor-wrapper yaml-wrapper">
<div id="compose-editor" class="yaml-editor" data-content="{{ compose_content | e }}" data-save-url="/api/service/{{ name }}/compose"></div>
</div>
{% endcall %}
{% call collapse(".env File", badge=env_path, icon=settings()) %}
<div class="editor-wrapper env-wrapper">
<div id="env-editor" class="env-editor" data-content="{{ env_content | e }}" data-save-url="/api/service/{{ name }}/env"></div>
</div>
{% endcall %}
{% include "partials/terminal.html" %}
<!-- Exec Terminal -->
{% if current_host %}
{% call collapse("Container Shell", id="exec-collapse", checked=True, icon=terminal()) %}
<div id="containers-list" class="mb-4"
hx-get="/api/service/{{ name }}/containers"
hx-trigger="load"
hx-target="this"
hx-select="unset"
hx-swap="innerHTML">
<span class="loading loading-spinner loading-sm"></span> Loading containers...
</div>
<div id="exec-terminal-container" class="bg-[#1a1a2e] rounded-lg h-[400px] border border-white/10 hidden">
<div id="exec-terminal" class="h-full"></div>
</div>
{% endcall %}
{% endif %}
</div>
{% endblock %}

300
src/compose_farm/web/ws.py Normal file
View File

@@ -0,0 +1,300 @@
"""WebSocket handler for terminal streaming."""
from __future__ import annotations
import asyncio
import contextlib
import fcntl
import json
import os
import pty
import shlex
import signal
import struct
import termios
from typing import TYPE_CHECKING, Any
import asyncssh
from fastapi import APIRouter, WebSocket, WebSocketDisconnect
from compose_farm.executor import is_local, ssh_connect_kwargs
from compose_farm.web.deps import get_config
from compose_farm.web.streaming import CRLF, DIM, GREEN, RED, RESET, tasks
# Shell command to prefer bash over sh
SHELL_FALLBACK = "command -v bash >/dev/null && exec bash || exec sh"
if TYPE_CHECKING:
from compose_farm.config import Host
router = APIRouter()
def _parse_resize(msg: str) -> tuple[int, int] | None:
"""Parse a resize message, return (cols, rows) or None if not a resize."""
try:
data = json.loads(msg)
if data.get("type") == "resize":
return int(data["cols"]), int(data["rows"])
except (json.JSONDecodeError, KeyError, TypeError, ValueError):
pass
return None
def _resize_pty(
fd: int, cols: int, rows: int, proc: asyncio.subprocess.Process | None = None
) -> None:
"""Resize a local PTY and send SIGWINCH to the process."""
winsize = struct.pack("HHHH", rows, cols, 0, 0)
fcntl.ioctl(fd, termios.TIOCSWINSZ, winsize)
# Explicitly send SIGWINCH so docker exec forwards it to the container
if proc and proc.pid:
os.kill(proc.pid, signal.SIGWINCH)
async def _bridge_websocket_to_fd(
websocket: WebSocket,
master_fd: int,
proc: asyncio.subprocess.Process,
) -> None:
"""Bridge WebSocket to a local PTY file descriptor."""
loop = asyncio.get_event_loop()
async def read_output() -> None:
while proc.returncode is None:
try:
data = await loop.run_in_executor(None, lambda: os.read(master_fd, 4096))
except BlockingIOError:
await asyncio.sleep(0.01)
continue
except OSError:
break
if not data:
break
await websocket.send_text(data.decode("utf-8", errors="replace"))
read_task = asyncio.create_task(read_output())
try:
while proc.returncode is None:
try:
msg = await asyncio.wait_for(websocket.receive_text(), timeout=0.1)
except TimeoutError:
continue
if size := _parse_resize(msg):
_resize_pty(master_fd, *size, proc)
else:
os.write(master_fd, msg.encode())
finally:
read_task.cancel()
os.close(master_fd)
if proc.returncode is None:
proc.terminate()
async def _bridge_websocket_to_ssh(
websocket: WebSocket,
proc: Any, # asyncssh.SSHClientProcess
) -> None:
"""Bridge WebSocket to an SSH process with PTY."""
assert proc.stdout is not None
assert proc.stdin is not None
async def read_stdout() -> None:
while proc.returncode is None:
data = await proc.stdout.read(4096)
if not data:
break
text = data if isinstance(data, str) else data.decode()
await websocket.send_text(text)
read_task = asyncio.create_task(read_stdout())
try:
while proc.returncode is None:
try:
msg = await asyncio.wait_for(websocket.receive_text(), timeout=0.1)
except TimeoutError:
continue
if size := _parse_resize(msg):
proc.change_terminal_size(*size)
else:
proc.stdin.write(msg)
finally:
read_task.cancel()
proc.terminate()
def _make_controlling_tty(slave_fd: int) -> None:
"""Set up the slave PTY as the controlling terminal for the child process."""
# Create a new session
os.setsid()
# Make the slave fd the controlling terminal
fcntl.ioctl(slave_fd, termios.TIOCSCTTY, 0)
async def _run_local_exec(websocket: WebSocket, argv: list[str]) -> None:
"""Run command locally with PTY using argv list (no shell interpretation)."""
master_fd, slave_fd = pty.openpty()
proc = await asyncio.create_subprocess_exec(
*argv,
stdin=slave_fd,
stdout=slave_fd,
stderr=slave_fd,
close_fds=True,
preexec_fn=lambda: _make_controlling_tty(slave_fd),
start_new_session=False, # We handle setsid in preexec_fn
)
os.close(slave_fd)
# Set non-blocking
flags = fcntl.fcntl(master_fd, fcntl.F_GETFL)
fcntl.fcntl(master_fd, fcntl.F_SETFL, flags | os.O_NONBLOCK)
await _bridge_websocket_to_fd(websocket, master_fd, proc)
async def _run_remote_exec(
websocket: WebSocket, host: Host, exec_cmd: str, *, agent_forwarding: bool = False
) -> None:
"""Run docker exec on remote host via SSH with PTY."""
# ssh_connect_kwargs includes agent_path and client_keys fallback
async with asyncssh.connect(
**ssh_connect_kwargs(host),
agent_forwarding=agent_forwarding,
) as conn:
proc: asyncssh.SSHClientProcess[Any] = await conn.create_process(
exec_cmd,
term_type="xterm-256color",
term_size=(80, 24),
)
async with proc:
await _bridge_websocket_to_ssh(websocket, proc)
async def _run_exec_session(
websocket: WebSocket,
container: str,
host_name: str,
) -> None:
"""Run an interactive docker exec session over WebSocket."""
config = get_config()
host = config.hosts.get(host_name)
if not host:
await websocket.send_text(f"{RED}Host '{host_name}' not found{RESET}{CRLF}")
return
if is_local(host):
# Local: use argv list (no shell interpretation)
argv = ["docker", "exec", "-it", container, "/bin/sh", "-c", SHELL_FALLBACK]
await _run_local_exec(websocket, argv)
else:
# Remote: quote container name to prevent injection
exec_cmd = (
f"docker exec -it {shlex.quote(container)} /bin/sh -c {shlex.quote(SHELL_FALLBACK)}"
)
await _run_remote_exec(websocket, host, exec_cmd)
@router.websocket("/ws/exec/{service}/{container}/{host}")
async def exec_websocket(
websocket: WebSocket,
service: str, # noqa: ARG001
container: str,
host: str,
) -> None:
"""WebSocket endpoint for interactive container exec."""
await websocket.accept()
try:
await websocket.send_text(f"{DIM}[Connecting to {container} on {host}...]{RESET}{CRLF}")
await _run_exec_session(websocket, container, host)
await websocket.send_text(f"{CRLF}{DIM}[Disconnected]{RESET}{CRLF}")
except WebSocketDisconnect:
pass
except Exception as e:
with contextlib.suppress(Exception):
await websocket.send_text(f"{RED}Error: {e}{RESET}{CRLF}")
finally:
with contextlib.suppress(Exception):
await websocket.close()
async def _run_shell_session(
websocket: WebSocket,
host_name: str,
) -> None:
"""Run an interactive shell session on a host over WebSocket."""
config = get_config()
host = config.hosts.get(host_name)
if not host:
await websocket.send_text(f"{RED}Host '{host_name}' not found{RESET}{CRLF}")
return
# Start interactive shell in home directory (avoid login shell to prevent job control warnings)
shell_cmd = "cd ~ && exec bash -i 2>/dev/null || exec sh -i"
if is_local(host):
# Local: use argv list with shell -c to interpret the command
argv = ["/bin/sh", "-c", shell_cmd]
await _run_local_exec(websocket, argv)
else:
await _run_remote_exec(websocket, host, shell_cmd, agent_forwarding=True)
@router.websocket("/ws/shell/{host}")
async def shell_websocket(
websocket: WebSocket,
host: str,
) -> None:
"""WebSocket endpoint for interactive host shell access."""
await websocket.accept()
try:
await websocket.send_text(f"{DIM}[Connecting to {host}...]{RESET}{CRLF}")
await _run_shell_session(websocket, host)
await websocket.send_text(f"{CRLF}{DIM}[Disconnected]{RESET}{CRLF}")
except WebSocketDisconnect:
pass
except Exception as e:
with contextlib.suppress(Exception):
await websocket.send_text(f"{RED}Error: {e}{RESET}{CRLF}")
finally:
with contextlib.suppress(Exception):
await websocket.close()
@router.websocket("/ws/terminal/{task_id}")
async def terminal_websocket(websocket: WebSocket, task_id: str) -> None:
"""WebSocket endpoint for terminal streaming."""
await websocket.accept()
if task_id not in tasks:
await websocket.send_text(
f"{DIM}Task not found (expired or container restarted).{RESET}{CRLF}"
)
await websocket.close(code=4004)
return
task = tasks[task_id]
sent_count = 0
try:
while True:
# Send any new output
while sent_count < len(task["output"]):
await websocket.send_text(task["output"][sent_count])
sent_count += 1
if task["status"] in ("completed", "failed"):
status = "[Done]" if task["status"] == "completed" else "[Failed]"
color = GREEN if task["status"] == "completed" else RED
await websocket.send_text(f"{CRLF}{color}{status}{RESET}{CRLF}")
await websocket.close()
break
await asyncio.sleep(0.05)
except WebSocketDisconnect:
pass
# Task stays in memory for reconnection; cleanup_stale_tasks() handles expiry

426
tests/test_cli_lifecycle.py Normal file
View File

@@ -0,0 +1,426 @@
"""Tests for CLI lifecycle commands (apply, down --orphaned)."""
from pathlib import Path
from unittest.mock import patch
import pytest
import typer
from compose_farm.cli.lifecycle import apply, down
from compose_farm.config import Config, Host
from compose_farm.executor import CommandResult
def _make_config(tmp_path: Path, services: dict[str, str] | None = None) -> Config:
"""Create a minimal config for testing."""
compose_dir = tmp_path / "compose"
compose_dir.mkdir()
svc_dict = services or {"svc1": "host1", "svc2": "host2"}
for svc in svc_dict:
svc_dir = compose_dir / svc
svc_dir.mkdir()
(svc_dir / "docker-compose.yml").write_text("services: {}\n")
config_path = tmp_path / "compose-farm.yaml"
config_path.write_text("")
return Config(
compose_dir=compose_dir,
hosts={"host1": Host(address="localhost"), "host2": Host(address="localhost")},
services=svc_dict,
config_path=config_path,
)
def _make_result(service: str, success: bool = True) -> CommandResult:
"""Create a command result."""
return CommandResult(
service=service,
exit_code=0 if success else 1,
success=success,
stdout="",
stderr="",
)
class TestApplyCommand:
"""Tests for the apply command."""
def test_apply_nothing_to_do(self, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None:
"""When no migrations, orphans, or missing services, prints success message."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
):
apply(dry_run=False, no_orphans=False, full=False, config=None)
captured = capsys.readouterr()
assert "Nothing to apply" in captured.out
def test_apply_dry_run_shows_preview(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Dry run shows what would be done without executing."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch(
"compose_farm.cli.lifecycle.get_orphaned_services",
return_value={"old-svc": "host1"},
),
patch(
"compose_farm.cli.lifecycle.get_services_needing_migration",
return_value=["svc1"],
),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
patch("compose_farm.cli.lifecycle.get_service_host", return_value="host1"),
patch("compose_farm.cli.lifecycle.stop_orphaned_services") as mock_stop,
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
):
apply(dry_run=True, no_orphans=False, full=False, config=None)
captured = capsys.readouterr()
assert "Services to migrate" in captured.out
assert "svc1" in captured.out
assert "Orphaned services to stop" in captured.out
assert "old-svc" in captured.out
assert "dry-run" in captured.out
# Should not have called the actual operations
mock_stop.assert_not_called()
mock_up.assert_not_called()
def test_apply_executes_migrations(self, tmp_path: Path) -> None:
"""Apply runs migrations when services need migration."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("svc1")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch(
"compose_farm.cli.lifecycle.get_services_needing_migration",
return_value=["svc1"],
),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
patch("compose_farm.cli.lifecycle.get_service_host", return_value="host1"),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=False, full=False, config=None)
mock_up.assert_called_once()
call_args = mock_up.call_args
assert call_args[0][1] == ["svc1"] # services list
def test_apply_executes_orphan_cleanup(self, tmp_path: Path) -> None:
"""Apply stops orphaned services."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("old-svc@host1")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch(
"compose_farm.cli.lifecycle.get_orphaned_services",
return_value={"old-svc": "host1"},
),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.stop_orphaned_services") as mock_stop,
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=False, full=False, config=None)
mock_stop.assert_called_once_with(cfg)
def test_apply_no_orphans_skips_orphan_cleanup(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""--no-orphans flag skips orphan cleanup."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("svc1")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch(
"compose_farm.cli.lifecycle.get_orphaned_services",
return_value={"old-svc": "host1"},
),
patch(
"compose_farm.cli.lifecycle.get_services_needing_migration",
return_value=["svc1"],
),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
patch("compose_farm.cli.lifecycle.get_service_host", return_value="host1"),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
patch("compose_farm.cli.lifecycle.stop_orphaned_services") as mock_stop,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=True, full=False, config=None)
# Should run migrations but not orphan cleanup
mock_up.assert_called_once()
mock_stop.assert_not_called()
# Orphans should not appear in output
captured = capsys.readouterr()
assert "old-svc" not in captured.out
def test_apply_no_orphans_nothing_to_do(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""--no-orphans with only orphans means nothing to do."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch(
"compose_farm.cli.lifecycle.get_orphaned_services",
return_value={"old-svc": "host1"},
),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
):
apply(dry_run=False, no_orphans=True, full=False, config=None)
captured = capsys.readouterr()
assert "Nothing to apply" in captured.out
def test_apply_starts_missing_services(self, tmp_path: Path) -> None:
"""Apply starts services that are in config but not in state."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("svc1")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch(
"compose_farm.cli.lifecycle.get_services_not_in_state",
return_value=["svc1"],
),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=False, full=False, config=None)
mock_up.assert_called_once()
call_args = mock_up.call_args
assert call_args[0][1] == ["svc1"]
def test_apply_dry_run_shows_missing_services(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""Dry run shows services that would be started."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch(
"compose_farm.cli.lifecycle.get_services_not_in_state",
return_value=["svc1"],
),
):
apply(dry_run=True, no_orphans=False, full=False, config=None)
captured = capsys.readouterr()
assert "Services to start" in captured.out
assert "svc1" in captured.out
assert "dry-run" in captured.out
def test_apply_full_refreshes_all_services(self, tmp_path: Path) -> None:
"""--full runs up on all services to pick up config changes."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("svc1"), _make_result("svc2")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=False, full=True, config=None)
mock_up.assert_called_once()
call_args = mock_up.call_args
# Should refresh all services in config
assert set(call_args[0][1]) == {"svc1", "svc2"}
def test_apply_full_dry_run_shows_refresh(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""--full --dry-run shows services that would be refreshed."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch("compose_farm.cli.lifecycle.get_services_needing_migration", return_value=[]),
patch("compose_farm.cli.lifecycle.get_services_not_in_state", return_value=[]),
):
apply(dry_run=True, no_orphans=False, full=True, config=None)
captured = capsys.readouterr()
assert "Services to refresh" in captured.out
assert "svc1" in captured.out
assert "svc2" in captured.out
assert "dry-run" in captured.out
def test_apply_full_excludes_already_handled_services(self, tmp_path: Path) -> None:
"""--full doesn't double-process services that are migrating or starting."""
cfg = _make_config(tmp_path, {"svc1": "host1", "svc2": "host2", "svc3": "host1"})
mock_results = [_make_result("svc1"), _make_result("svc3")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
patch(
"compose_farm.cli.lifecycle.get_services_needing_migration",
return_value=["svc1"],
),
patch(
"compose_farm.cli.lifecycle.get_services_not_in_state",
return_value=["svc2"],
),
patch("compose_farm.cli.lifecycle.get_service_host", return_value="host2"),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.up_services") as mock_up,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
apply(dry_run=False, no_orphans=False, full=True, config=None)
# up_services should be called 3 times: migrate, start, refresh
assert mock_up.call_count == 3
# Get the third call (refresh) and check it only has svc3
refresh_call = mock_up.call_args_list[2]
assert refresh_call[0][1] == ["svc3"]
class TestDownOrphaned:
"""Tests for down --orphaned flag."""
def test_down_orphaned_no_orphans(
self, tmp_path: Path, capsys: pytest.CaptureFixture[str]
) -> None:
"""When no orphans exist, prints success message."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_orphaned_services", return_value={}),
):
down(
services=None,
all_services=False,
orphaned=True,
host=None,
config=None,
)
captured = capsys.readouterr()
assert "No orphaned services to stop" in captured.out
def test_down_orphaned_stops_services(self, tmp_path: Path) -> None:
"""--orphaned stops orphaned services."""
cfg = _make_config(tmp_path)
mock_results = [_make_result("old-svc@host1")]
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch(
"compose_farm.cli.lifecycle.get_orphaned_services",
return_value={"old-svc": "host1"},
),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=mock_results,
),
patch("compose_farm.cli.lifecycle.stop_orphaned_services") as mock_stop,
patch("compose_farm.cli.lifecycle.report_results"),
):
down(
services=None,
all_services=False,
orphaned=True,
host=None,
config=None,
)
mock_stop.assert_called_once_with(cfg)
def test_down_orphaned_with_services_errors(self) -> None:
"""--orphaned cannot be combined with service arguments."""
with pytest.raises(typer.Exit) as exc_info:
down(
services=["svc1"],
all_services=False,
orphaned=True,
host=None,
config=None,
)
assert exc_info.value.exit_code == 1
def test_down_orphaned_with_all_errors(self) -> None:
"""--orphaned cannot be combined with --all."""
with pytest.raises(typer.Exit) as exc_info:
down(
services=None,
all_services=True,
orphaned=True,
host=None,
config=None,
)
assert exc_info.value.exit_code == 1
def test_down_orphaned_with_host_errors(self) -> None:
"""--orphaned cannot be combined with --host."""
with pytest.raises(typer.Exit) as exc_info:
down(
services=None,
all_services=False,
orphaned=True,
host="host1",
config=None,
)
assert exc_info.value.exit_code == 1

View File

@@ -8,7 +8,7 @@ from unittest.mock import patch
import pytest
import typer
from compose_farm.cli import logs
from compose_farm.cli.monitoring import logs
from compose_farm.config import Config, Host
from compose_farm.executor import CommandResult
@@ -37,7 +37,7 @@ def _make_result(service: str) -> CommandResult:
def _mock_run_async_factory(
services: list[str],
) -> tuple[Any, list[CommandResult]]:
"""Create a mock _run_async that returns results for given services."""
"""Create a mock run_async that returns results for given services."""
results = [_make_result(s) for s in services]
def mock_run_async(_coro: Coroutine[Any, Any, Any]) -> list[CommandResult]:
@@ -55,9 +55,10 @@ class TestLogsContextualDefault:
mock_run_async, _ = _mock_run_async_factory(["svc1", "svc2", "svc3"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.monitoring.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
mock_run.return_value = None
@@ -73,9 +74,10 @@ class TestLogsContextualDefault:
mock_run_async, _ = _mock_run_async_factory(["svc1"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.monitoring.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
logs(
services=["svc1"],
@@ -96,9 +98,10 @@ class TestLogsContextualDefault:
mock_run_async, _ = _mock_run_async_factory(["svc1", "svc2", "svc3"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.monitoring.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
logs(
services=None,
@@ -119,9 +122,10 @@ class TestLogsContextualDefault:
mock_run_async, _ = _mock_run_async_factory(["svc1"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.monitoring.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
logs(
services=["svc1"],
@@ -146,9 +150,9 @@ class TestLogsHostFilter:
mock_run_async, _ = _mock_run_async_factory(["svc1", "svc2"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
logs(
services=None,
@@ -170,9 +174,9 @@ class TestLogsHostFilter:
mock_run_async, _ = _mock_run_async_factory(["svc1", "svc2"])
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
patch("compose_farm.cli._run_async", side_effect=mock_run_async),
patch("compose_farm.cli.run_on_services") as mock_run,
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_services") as mock_run,
):
logs(
services=None,
@@ -187,14 +191,10 @@ class TestLogsHostFilter:
call_args = mock_run.call_args
assert call_args[0][2] == "logs --tail 20"
def test_logs_all_and_host_mutually_exclusive(self, tmp_path: Path) -> None:
def test_logs_all_and_host_mutually_exclusive(self) -> None:
"""Using --all and --host together should error."""
cfg = _make_config(tmp_path)
with (
patch("compose_farm.cli._load_config_or_exit", return_value=cfg),
pytest.raises(typer.Exit) as exc_info,
):
# No config mock needed - error is raised before config is loaded
with pytest.raises(typer.Exit) as exc_info:
logs(
services=None,
all_services=True,

114
tests/test_cli_ssh.py Normal file
View File

@@ -0,0 +1,114 @@
"""Tests for CLI ssh commands."""
from pathlib import Path
from unittest.mock import patch
from typer.testing import CliRunner
from compose_farm.cli.app import app
runner = CliRunner()
class TestSshKeygen:
"""Tests for cf ssh keygen command."""
def test_keygen_generates_key(self, tmp_path: Path) -> None:
"""Generate SSH key when none exists."""
key_path = tmp_path / "compose-farm"
pubkey_path = tmp_path / "compose-farm.pub"
with (
patch("compose_farm.cli.ssh.SSH_KEY_PATH", key_path),
patch("compose_farm.cli.ssh.SSH_PUBKEY_PATH", pubkey_path),
patch("compose_farm.cli.ssh.key_exists", return_value=False),
):
result = runner.invoke(app, ["ssh", "keygen"])
# Command runs (may fail if ssh-keygen not available in test env)
assert result.exit_code in (0, 1)
def test_keygen_skips_if_exists(self, tmp_path: Path) -> None:
"""Skip key generation if key already exists."""
key_path = tmp_path / "compose-farm"
pubkey_path = tmp_path / "compose-farm.pub"
with (
patch("compose_farm.cli.ssh.SSH_KEY_PATH", key_path),
patch("compose_farm.cli.ssh.SSH_PUBKEY_PATH", pubkey_path),
patch("compose_farm.cli.ssh.key_exists", return_value=True),
):
result = runner.invoke(app, ["ssh", "keygen"])
assert "already exists" in result.output
class TestSshStatus:
"""Tests for cf ssh status command."""
def test_status_shows_no_key(self, tmp_path: Path) -> None:
"""Show message when no key exists."""
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text("""
hosts:
local:
address: localhost
services:
test: local
""")
with patch("compose_farm.cli.ssh.key_exists", return_value=False):
result = runner.invoke(app, ["ssh", "status", f"--config={config_file}"])
assert "No key found" in result.output
def test_status_shows_key_exists(self, tmp_path: Path) -> None:
"""Show key info when key exists."""
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text("""
hosts:
local:
address: localhost
services:
test: local
""")
with (
patch("compose_farm.cli.ssh.key_exists", return_value=True),
patch("compose_farm.cli.ssh.get_pubkey_content", return_value="ssh-ed25519 AAAA..."),
):
result = runner.invoke(app, ["ssh", "status", f"--config={config_file}"])
assert "Key exists" in result.output
class TestSshSetup:
"""Tests for cf ssh setup command."""
def test_setup_no_remote_hosts(self, tmp_path: Path) -> None:
"""Show message when no remote hosts configured."""
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text("""
hosts:
local:
address: localhost
services:
test: local
""")
result = runner.invoke(app, ["ssh", "setup", f"--config={config_file}"])
assert "No remote hosts" in result.output
class TestSshHelp:
"""Tests for cf ssh help."""
def test_ssh_help(self) -> None:
"""Show help for ssh command."""
result = runner.invoke(app, ["ssh", "--help"])
assert result.exit_code == 0
assert "setup" in result.output
assert "status" in result.output
assert "keygen" in result.output

58
tests/test_cli_startup.py Normal file
View File

@@ -0,0 +1,58 @@
"""Test CLI startup performance."""
from __future__ import annotations
import shutil
import subprocess
import sys
import time
# Thresholds in seconds, per OS
if sys.platform == "win32":
CLI_STARTUP_THRESHOLD = 2.0
elif sys.platform == "darwin":
CLI_STARTUP_THRESHOLD = 0.35
else: # Linux
CLI_STARTUP_THRESHOLD = 0.25
def test_cli_startup_time() -> None:
"""Verify CLI startup time stays within acceptable bounds.
This test ensures we don't accidentally introduce slow imports
that degrade the user experience.
"""
cf_path = shutil.which("cf")
assert cf_path is not None, "cf command not found in PATH"
# Run up to 6 times, return early if we hit the threshold
times: list[float] = []
for _ in range(6):
start = time.perf_counter()
result = subprocess.run(
[cf_path, "--help"],
check=False,
capture_output=True,
text=True,
)
elapsed = time.perf_counter() - start
times.append(elapsed)
# Verify the command succeeded
assert result.returncode == 0, f"CLI failed: {result.stderr}"
# Pass early if under threshold
if elapsed < CLI_STARTUP_THRESHOLD:
print(f"\nCLI startup: {elapsed:.3f}s (threshold: {CLI_STARTUP_THRESHOLD}s)")
return
# All attempts exceeded threshold
best_time = min(times)
msg = (
f"\nCLI startup times: {[f'{t:.3f}s' for t in times]}\n"
f"Best: {best_time:.3f}s, Threshold: {CLI_STARTUP_THRESHOLD}s"
)
print(msg)
err_msg = f"CLI startup too slow!\n{msg}\nCheck for slow imports."
raise AssertionError(err_msg)

View File

@@ -128,6 +128,8 @@ class TestLoadConfig:
def test_load_config_not_found(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "empty_config"))
with pytest.raises(FileNotFoundError, match="Config file not found"):
load_config()

230
tests/test_config_cmd.py Normal file
View File

@@ -0,0 +1,230 @@
"""Tests for config command module."""
from pathlib import Path
from typing import Any
import pytest
import yaml
from typer.testing import CliRunner
from compose_farm.cli import app
from compose_farm.cli.config import (
_generate_template,
_get_config_file,
_get_editor,
)
@pytest.fixture
def runner() -> CliRunner:
return CliRunner()
@pytest.fixture
def valid_config_data() -> dict[str, Any]:
return {
"compose_dir": "/opt/compose",
"hosts": {"server1": "192.168.1.10"},
"services": {"nginx": "server1"},
}
class TestGetEditor:
"""Tests for _get_editor function."""
def test_uses_editor_env(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("EDITOR", "code")
monkeypatch.delenv("VISUAL", raising=False)
assert _get_editor() == "code"
def test_uses_visual_env(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.delenv("EDITOR", raising=False)
monkeypatch.setenv("VISUAL", "subl")
assert _get_editor() == "subl"
def test_editor_takes_precedence(self, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("EDITOR", "vim")
monkeypatch.setenv("VISUAL", "code")
assert _get_editor() == "vim"
class TestGetConfigFile:
"""Tests for _get_config_file function."""
def test_explicit_path(self, tmp_path: Path) -> None:
config_file = tmp_path / "my-config.yaml"
config_file.touch()
result = _get_config_file(config_file)
assert result == config_file.resolve()
def test_cf_config_env(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
config_file = tmp_path / "env-config.yaml"
config_file.touch()
monkeypatch.setenv("CF_CONFIG", str(config_file))
result = _get_config_file(None)
assert result == config_file.resolve()
def test_returns_none_when_not_found(
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
# Set XDG_CONFIG_HOME to a nonexistent path - config_search_paths() will
# now return paths that don't exist
monkeypatch.setenv("XDG_CONFIG_HOME", str(tmp_path / "nonexistent"))
result = _get_config_file(None)
assert result is None
class TestGenerateTemplate:
"""Tests for _generate_template function."""
def test_generates_valid_yaml(self) -> None:
template = _generate_template()
# Should be valid YAML
data = yaml.safe_load(template)
assert "compose_dir" in data
assert "hosts" in data
assert "services" in data
def test_has_documentation_comments(self) -> None:
template = _generate_template()
assert "# Compose Farm configuration" in template
assert "hosts:" in template
assert "services:" in template
class TestConfigInit:
"""Tests for cf config init command."""
def test_init_creates_file(
self, runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "new-config.yaml"
result = runner.invoke(app, ["config", "init", "-p", str(config_file)])
assert result.exit_code == 0
assert config_file.exists()
assert "Config file created" in result.stdout
def test_init_force_overwrites(
self, runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "existing.yaml"
config_file.write_text("old content")
result = runner.invoke(app, ["config", "init", "-p", str(config_file), "-f"])
assert result.exit_code == 0
content = config_file.read_text()
assert "old content" not in content
assert "compose_dir" in content
def test_init_prompts_on_existing(
self, runner: CliRunner, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "existing.yaml"
config_file.write_text("old content")
result = runner.invoke(app, ["config", "init", "-p", str(config_file)], input="n\n")
assert result.exit_code == 0
assert "Aborted" in result.stdout
assert config_file.read_text() == "old content"
class TestConfigPath:
"""Tests for cf config path command."""
def test_path_shows_config(
self,
runner: CliRunner,
tmp_path: Path,
valid_config_data: dict[str, Any],
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text(yaml.dump(valid_config_data))
result = runner.invoke(app, ["config", "path"])
assert result.exit_code == 0
assert str(config_file) in result.stdout
def test_path_with_explicit_path(self, runner: CliRunner, tmp_path: Path) -> None:
# When explicitly provided, path is returned even if file doesn't exist
nonexistent = tmp_path / "nonexistent.yaml"
result = runner.invoke(app, ["config", "path", "-p", str(nonexistent)])
assert result.exit_code == 0
assert str(nonexistent) in result.stdout
class TestConfigShow:
"""Tests for cf config show command."""
def test_show_displays_content(
self,
runner: CliRunner,
tmp_path: Path,
valid_config_data: dict[str, Any],
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text(yaml.dump(valid_config_data))
result = runner.invoke(app, ["config", "show"])
assert result.exit_code == 0
assert "Config file:" in result.stdout
def test_show_raw_output(
self,
runner: CliRunner,
tmp_path: Path,
valid_config_data: dict[str, Any],
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "compose-farm.yaml"
content = yaml.dump(valid_config_data)
config_file.write_text(content)
result = runner.invoke(app, ["config", "show", "-r"])
assert result.exit_code == 0
assert content in result.stdout
class TestConfigValidate:
"""Tests for cf config validate command."""
def test_validate_valid_config(
self,
runner: CliRunner,
tmp_path: Path,
valid_config_data: dict[str, Any],
monkeypatch: pytest.MonkeyPatch,
) -> None:
monkeypatch.chdir(tmp_path)
monkeypatch.delenv("CF_CONFIG", raising=False)
config_file = tmp_path / "compose-farm.yaml"
config_file.write_text(yaml.dump(valid_config_data))
result = runner.invoke(app, ["config", "validate"])
assert result.exit_code == 0
assert "Valid config" in result.stdout
assert "Hosts: 1" in result.stdout
assert "Services: 1" in result.stdout
def test_validate_invalid_config(self, runner: CliRunner, tmp_path: Path) -> None:
config_file = tmp_path / "invalid.yaml"
config_file.write_text("invalid: [yaml: content")
result = runner.invoke(app, ["config", "validate", "-p", str(config_file)])
assert result.exit_code == 1
# Error goes to stderr (captured in output when using CliRunner)
output = result.stdout + (result.stderr or "")
assert "Invalid config" in output or "" in output
def test_validate_missing_config(self, runner: CliRunner, tmp_path: Path) -> None:
nonexistent = tmp_path / "nonexistent.yaml"
result = runner.invoke(app, ["config", "validate", "-p", str(nonexistent)])
assert result.exit_code == 1
# Error goes to stderr
output = result.stdout + (result.stderr or "")
assert "Config file not found" in output or "not found" in output.lower()

111
tests/test_operations.py Normal file
View File

@@ -0,0 +1,111 @@
"""Tests for operations module."""
from __future__ import annotations
import inspect
from pathlib import Path
from unittest.mock import patch
import pytest
from compose_farm.cli import lifecycle
from compose_farm.config import Config, Host
from compose_farm.executor import CommandResult
from compose_farm.operations import _migrate_service
@pytest.fixture
def basic_config(tmp_path: Path) -> Config:
"""Create a basic test config."""
compose_dir = tmp_path / "compose"
service_dir = compose_dir / "test-service"
service_dir.mkdir(parents=True)
(service_dir / "docker-compose.yml").write_text("services: {}")
return Config(
compose_dir=compose_dir,
hosts={
"host1": Host(address="localhost"),
"host2": Host(address="localhost"),
},
services={"test-service": "host2"},
)
class TestMigrationCommands:
"""Tests for migration command sequence."""
@pytest.fixture
def config(self, tmp_path: Path) -> Config:
"""Create a test config."""
compose_dir = tmp_path / "compose"
service_dir = compose_dir / "test-service"
service_dir.mkdir(parents=True)
(service_dir / "docker-compose.yml").write_text("services: {}")
return Config(
compose_dir=compose_dir,
hosts={
"host1": Host(address="localhost"),
"host2": Host(address="localhost"),
},
services={"test-service": "host2"},
)
async def test_migration_uses_pull_ignore_buildable(self, config: Config) -> None:
"""Migration should use 'pull --ignore-buildable' to skip buildable images."""
commands_called: list[str] = []
async def mock_run_compose_step(
cfg: Config,
service: str,
command: str,
*,
raw: bool,
host: str | None = None,
) -> CommandResult:
commands_called.append(command)
return CommandResult(
service=service,
exit_code=0,
success=True,
)
with patch(
"compose_farm.operations._run_compose_step",
side_effect=mock_run_compose_step,
):
await _migrate_service(
config,
"test-service",
current_host="host1",
target_host="host2",
prefix="[test]",
raw=False,
)
# Migration should call pull with --ignore-buildable, then build, then down
assert "pull --ignore-buildable" in commands_called
assert "build" in commands_called
assert "down" in commands_called
# pull should come before build
pull_idx = commands_called.index("pull --ignore-buildable")
build_idx = commands_called.index("build")
assert pull_idx < build_idx
class TestUpdateCommandSequence:
"""Tests for update command sequence."""
def test_update_command_sequence_includes_build(self) -> None:
"""Update command should use pull --ignore-buildable and build."""
# This is a static check of the command sequence in lifecycle.py
# The actual command sequence is defined in the update function
source = inspect.getsource(lifecycle.update)
# Verify the command sequence includes pull --ignore-buildable
assert "pull --ignore-buildable" in source
# Verify build is included
assert '"build"' in source or "'build'" in source
# Verify the sequence is pull, build, down, up
assert "down" in source
assert "up -d" in source

Some files were not shown because too many files have changed in this diff Show More