fix: --host filter now limits multi-host stack operations to single host (#175)
Some checks failed
Update README.md / update_readme (push) Has been cancelled
CI / test (macos-latest, 3.11) (push) Has been cancelled
CI / test (macos-latest, 3.12) (push) Has been cancelled
CI / test (macos-latest, 3.13) (push) Has been cancelled
CI / test (ubuntu-latest, 3.11) (push) Has been cancelled
CI / test (ubuntu-latest, 3.12) (push) Has been cancelled
CI / test (ubuntu-latest, 3.13) (push) Has been cancelled
CI / browser-tests (push) Has been cancelled
CI / lint (push) Has been cancelled
Release Drafter / update_release_draft (push) Has been cancelled
TOC Generator / TOC Generator (push) Has been cancelled

* fix: --host filter now limits multi-host stack operations to single host

Previously, when using `-H host` with a multi-host stack like `glances: all`,
the command would find the stack (correct) but then operate on ALL hosts for
that stack (incorrect). For example, `cf down -H nas` with `glances` would
stop glances on all 5 hosts instead of just nas.

Now, when `--host` is specified:
- `cf down -H nas` only stops stacks on nas, including only the nas instance
  of multi-host stacks
- `cf up -H nas` only starts stacks on nas (skips migration logic since
  host is explicitly specified)

Added tests for the new filter_host behavior in both executor and CLI.

* fix: Apply filter_host to logs and ps commands as well

Same bug as up/down: when using `-H host` with multi-host stacks,
logs and ps would show results from all hosts instead of just the
filtered host.

* fix: Don't remove multi-host stacks from state when host-filtered

When using `-H host` with a multi-host stack, we only stop one instance.
The stack is still running on other hosts, so we shouldn't remove it
from state entirely.

This prevents issues where:
- `cf apply` would try to re-start the stack
- `cf ps` would show incorrect running status
- Orphan detection would be confused

Added tests to verify state is preserved for host-filtered multi-host
operations and removed for full stack operations.

* refactor: Introduce StackSelection dataclass for cleaner context passing

Instead of passing filter_host separately through multiple layers,
bundle the selection context into a StackSelection dataclass:

- stacks: list of selected stack names
- config: the loaded Config
- host_filter: optional host filter from -H flag

This provides:
1. Cleaner APIs - context travels together instead of being scattered
2. is_instance_level() method - encapsulates the check for whether
   this is an instance-level operation (host-filtered multi-host stack)
3. Future extensibility - can add more context (dry_run, verbose, etc.)

Updated all callers of get_stacks() to use the new return type.

* Revert "refactor: Introduce StackSelection dataclass for cleaner context passing"

This reverts commit e6e9eed93e.

* feat: Proper per-host state tracking for multi-host stacks

- Add `remove_stack_host()` to remove a single host from a multi-host stack's state
- Add `add_stack_host()` to add a single host to a stack's state
- Update `down` command to use `remove_stack_host` for host-filtered multi-host stacks
- Update `up` command to use `add_stack_host` for host-filtered operations

This ensures the state file accurately reflects which hosts each stack is running on,
rather than just tracking if it's running at all.

* fix: Use set comparisons for host list tests

Host lists may be reordered during YAML save/load, so test for
set equality rather than list equality.

* refactor: Merge remove_stack_host into remove_stack as optional parameter

Instead of a separate function, `remove_stack` now takes an optional
`host` parameter. When specified, it removes only that host from
multi-host stacks. This reduces API surface and follows the existing
pattern.

* fix: Restore deterministic host list sorting and add filter_host test

- Restore sorting of list values in _sorted_dict for consistent YAML output
- Add test for logs --host passing filter_host to run_on_stacks
This commit is contained in:
Bas Nijholt
2026-02-01 13:43:17 -08:00
committed by GitHub
parent 4d65702868
commit fd1b04297e
8 changed files with 440 additions and 17 deletions

View File

@@ -37,6 +37,7 @@ from compose_farm.operations import (
up_stacks,
)
from compose_farm.state import (
add_stack_host,
get_orphaned_stacks,
get_stack_host,
get_stacks_needing_migration,
@@ -73,6 +74,23 @@ def up(
cfg, stack_list, build_up_cmd(pull=pull, build=build, service=service), raw=True
)
)
elif host:
# For host-filtered up, use run_on_stacks to only affect that host
# (skips migration logic, which is intended when explicitly specifying a host)
results = run_async(
run_on_stacks(
cfg,
stack_list,
build_up_cmd(pull=pull, build=build),
raw=True,
filter_host=host,
)
)
# Update state for successful host-filtered operations
for result in results:
if result.success:
base_stack = result.stack.split("@")[0]
add_stack_host(cfg, base_stack, host)
else:
results = run_async(up_stacks(cfg, stack_list, raw=True, pull=pull, build=build))
maybe_regenerate_traefik(cfg, results)
@@ -116,17 +134,20 @@ def down(
stack_list, cfg = get_stacks(stacks or [], all_stacks, config, host=host)
raw = len(stack_list) == 1
results = run_async(run_on_stacks(cfg, stack_list, "down", raw=raw))
results = run_async(run_on_stacks(cfg, stack_list, "down", raw=raw, filter_host=host))
# Remove from state on success
# Update state on success
# For multi-host stacks, result.stack is "stack@host", extract base name
removed_stacks: set[str] = set()
updated_stacks: set[str] = set()
for result in results:
if result.success:
base_stack = result.stack.split("@")[0]
if base_stack not in removed_stacks:
remove_stack(cfg, base_stack)
removed_stacks.add(base_stack)
if base_stack not in updated_stacks:
# When host is specified for multi-host stack, removes just that host
# Otherwise removes entire stack from state
filter_host = host if host and cfg.is_multi_host(base_stack) else None
remove_stack(cfg, base_stack, filter_host)
updated_stacks.add(base_stack)
maybe_regenerate_traefik(cfg, results)
report_results(results)

View File

@@ -248,7 +248,7 @@ def logs(
cmd += " -f"
if service:
cmd += f" {service}"
results = run_async(run_on_stacks(cfg, stack_list, cmd))
results = run_async(run_on_stacks(cfg, stack_list, cmd, filter_host=host))
report_results(results)
@@ -272,7 +272,7 @@ def ps(
print_error("--service requires exactly one stack")
raise typer.Exit(1)
cmd = f"ps {service}" if service else "ps"
results = run_async(run_on_stacks(cfg, stack_list, cmd))
results = run_async(run_on_stacks(cfg, stack_list, cmd, filter_host=host))
report_results(results)

View File

@@ -392,13 +392,17 @@ async def run_on_stacks(
*,
stream: bool = True,
raw: bool = False,
filter_host: str | None = None,
) -> list[CommandResult]:
"""Run a docker compose command on multiple stacks in parallel.
For multi-host stacks, runs on all configured hosts.
Note: raw=True only makes sense for single-stack operations.
For multi-host stacks, runs on all configured hosts unless filter_host is set,
in which case only the filtered host is affected. raw=True only makes sense
for single-stack operations.
"""
return await run_sequential_on_stacks(config, stacks, [compose_cmd], stream=stream, raw=raw)
return await run_sequential_on_stacks(
config, stacks, [compose_cmd], stream=stream, raw=raw, filter_host=filter_host
)
async def _run_sequential_stack_commands(
@@ -418,6 +422,33 @@ async def _run_sequential_stack_commands(
return CommandResult(stack=stack, exit_code=0, success=True)
async def _run_sequential_stack_commands_on_host(
config: Config,
stack: str,
host_name: str,
commands: list[str],
*,
stream: bool = True,
raw: bool = False,
prefix: str | None = None,
) -> CommandResult:
"""Run multiple compose commands sequentially for a stack on a specific host.
Used when --host filter is applied to a multi-host stack.
"""
stack_dir = config.get_stack_dir(stack)
host = config.hosts[host_name]
label = f"{stack}@{host_name}"
for cmd in commands:
_print_compose_command(host_name, stack, cmd)
command = f'cd "{stack_dir}" && docker compose {cmd}'
result = await run_command(host, command, label, stream=stream, raw=raw, prefix=prefix)
if not result.success:
return result
return CommandResult(stack=label, exit_code=0, success=True)
async def _run_sequential_stack_commands_multi_host(
config: Config,
stack: str,
@@ -469,11 +500,13 @@ async def run_sequential_on_stacks(
*,
stream: bool = True,
raw: bool = False,
filter_host: str | None = None,
) -> list[CommandResult]:
"""Run sequential commands on multiple stacks in parallel.
For multi-host stacks, runs on all configured hosts.
Note: raw=True only makes sense for single-stack operations.
For multi-host stacks, runs on all configured hosts unless filter_host is set,
in which case only the filtered host is affected. raw=True only makes sense
for single-stack operations.
"""
# Skip prefix for single-stack operations (command line already shows context)
prefix: str | None = "" if len(stacks) == 1 else None
@@ -483,12 +516,20 @@ async def run_sequential_on_stacks(
single_host_tasks = []
for stack in stacks:
if config.is_multi_host(stack):
if config.is_multi_host(stack) and filter_host is None:
# Multi-host stack without filter: run on all hosts
multi_host_tasks.append(
_run_sequential_stack_commands_multi_host(
config, stack, commands, stream=stream, raw=raw, prefix=prefix
)
)
elif config.is_multi_host(stack) and filter_host is not None:
# Multi-host stack with filter: run only on filtered host
single_host_tasks.append(
_run_sequential_stack_commands_on_host(
config, stack, filter_host, commands, stream=stream, raw=raw, prefix=prefix
)
)
else:
single_host_tasks.append(
_run_sequential_stack_commands(

View File

@@ -112,10 +112,46 @@ def set_multi_host_stack(config: Config, stack: str, hosts: list[str]) -> None:
state[stack] = hosts
def remove_stack(config: Config, stack: str) -> None:
"""Remove a stack from the state (after down)."""
def remove_stack(config: Config, stack: str, host: str | None = None) -> None:
"""Remove a stack from the state (after down).
If host is provided, only removes that host from a multi-host stack.
If the list becomes empty, removes the stack entirely.
For single-host stacks with host specified, removes only if host matches.
"""
with _modify_state(config) as state:
if stack not in state:
return
if host is None:
state.pop(stack, None)
return
current = state[stack]
if isinstance(current, list):
new_hosts = [h for h in current if h != host]
if new_hosts:
state[stack] = new_hosts
else:
del state[stack]
elif current == host:
del state[stack]
def add_stack_host(config: Config, stack: str, host: str) -> None:
"""Add a single host to a stack's state.
For multi-host stacks, adds the host to the list if not present.
For single-host stacks or new entries, sets the host directly.
"""
with _modify_state(config) as state:
current = state.get(stack)
if current is None:
state[stack] = host
elif isinstance(current, list):
if host not in current:
state[stack] = [*current, host]
elif current != host:
# Convert single host to list
state[stack] = [current, host]
def get_stacks_needing_migration(config: Config) -> list[str]:

View File

@@ -437,3 +437,132 @@ class TestDownOrphaned:
)
assert exc_info.value.exit_code == 1
class TestHostFilterMultiHost:
"""Tests for --host filter with multi-host stacks."""
def _make_multi_host_config(self, tmp_path: Path) -> Config:
"""Create a config with a multi-host stack."""
compose_dir = tmp_path / "compose"
compose_dir.mkdir()
# Create stack directories
for stack in ["single-host", "multi-host"]:
stack_dir = compose_dir / stack
stack_dir.mkdir()
(stack_dir / "docker-compose.yml").write_text("services: {}\n")
config_path = tmp_path / "compose-farm.yaml"
config_path.write_text("")
return Config(
compose_dir=compose_dir,
hosts={
"host1": Host(address="192.168.1.1"),
"host2": Host(address="192.168.1.2"),
"host3": Host(address="192.168.1.3"),
},
stacks={
"single-host": "host1",
"multi-host": ["host1", "host2", "host3"], # runs on all 3 hosts
},
config_path=config_path,
)
def test_down_host_filter_limits_multi_host_stack(self, tmp_path: Path) -> None:
"""--host filter should only run down on that host for multi-host stacks."""
cfg = self._make_multi_host_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_stacks") as mock_get_stacks,
patch("compose_farm.cli.lifecycle.run_on_stacks") as mock_run,
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=[_make_result("multi-host@host1")],
),
patch("compose_farm.cli.lifecycle.remove_stack"),
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
mock_get_stacks.return_value = (["multi-host"], cfg)
down(
stacks=None,
all_stacks=False,
orphaned=False,
host="host1",
config=None,
)
# Verify run_on_stacks was called with filter_host="host1"
mock_run.assert_called_once()
call_kwargs = mock_run.call_args.kwargs
assert call_kwargs.get("filter_host") == "host1"
def test_down_host_filter_removes_host_from_state(self, tmp_path: Path) -> None:
"""--host filter should remove just that host from multi-host stack's state.
When stopping only one instance of a multi-host stack, we should update
state to remove just that host, not the entire stack.
"""
cfg = self._make_multi_host_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_stacks") as mock_get_stacks,
patch("compose_farm.cli.lifecycle.run_on_stacks"),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=[_make_result("multi-host@host1")],
),
patch("compose_farm.cli.lifecycle.remove_stack") as mock_remove,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
mock_get_stacks.return_value = (["multi-host"], cfg)
down(
stacks=None,
all_stacks=False,
orphaned=False,
host="host1",
config=None,
)
# remove_stack should be called with the host parameter
mock_remove.assert_called_once_with(cfg, "multi-host", "host1")
def test_down_without_host_filter_removes_from_state(self, tmp_path: Path) -> None:
"""Without --host filter, multi-host stacks SHOULD be removed from state."""
cfg = self._make_multi_host_config(tmp_path)
with (
patch("compose_farm.cli.lifecycle.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.lifecycle.get_stacks") as mock_get_stacks,
patch("compose_farm.cli.lifecycle.run_on_stacks"),
patch(
"compose_farm.cli.lifecycle.run_async",
return_value=[
_make_result("multi-host@host1"),
_make_result("multi-host@host2"),
_make_result("multi-host@host3"),
],
),
patch("compose_farm.cli.lifecycle.remove_stack") as mock_remove,
patch("compose_farm.cli.lifecycle.maybe_regenerate_traefik"),
patch("compose_farm.cli.lifecycle.report_results"),
):
mock_get_stacks.return_value = (["multi-host"], cfg)
down(
stacks=None,
all_stacks=False,
orphaned=False,
host=None, # No host filter
config=None,
)
# remove_stack SHOULD be called with host=None when stopping all instances
mock_remove.assert_called_once_with(cfg, "multi-host", None)

View File

@@ -29,6 +29,28 @@ def _make_config(tmp_path: Path) -> Config:
)
def _make_multi_host_config(tmp_path: Path) -> Config:
"""Create a config with a multi-host stack for testing."""
compose_dir = tmp_path / "compose"
compose_dir.mkdir()
for svc in ("single-host", "multi-host"):
svc_dir = compose_dir / svc
svc_dir.mkdir()
(svc_dir / "docker-compose.yml").write_text("services: {}\n")
return Config(
compose_dir=compose_dir,
hosts={
"host1": Host(address="192.168.1.1"),
"host2": Host(address="192.168.1.2"),
},
stacks={
"single-host": "host1",
"multi-host": ["host1", "host2"],
},
)
def _make_result(stack: str) -> CommandResult:
"""Create a successful command result."""
return CommandResult(stack=stack, exit_code=0, success=True, stdout="", stderr="")
@@ -205,3 +227,26 @@ class TestLogsHostFilter:
)
assert exc_info.value.exit_code == 1
def test_logs_host_filter_passes_filter_host_to_run_on_stacks(self, tmp_path: Path) -> None:
"""--host should pass filter_host to run_on_stacks for multi-host stacks."""
cfg = _make_multi_host_config(tmp_path)
mock_run_async, _ = _mock_run_async_factory(["multi-host@host1"])
with (
patch("compose_farm.cli.common.load_config_or_exit", return_value=cfg),
patch("compose_farm.cli.monitoring.run_async", side_effect=mock_run_async),
patch("compose_farm.cli.monitoring.run_on_stacks") as mock_run,
):
logs(
stacks=None,
all_stacks=False,
host="host1",
follow=False,
tail=None,
config=None,
)
mock_run.assert_called_once()
call_kwargs = mock_run.call_args.kwargs
assert call_kwargs.get("filter_host") == "host1"

View File

@@ -229,6 +229,52 @@ class TestRunOnStacks:
assert results[0].stack == "svc1"
assert results[1].stack == "svc2"
async def test_run_on_stacks_filter_host_limits_multi_host(self) -> None:
"""filter_host should only run on that host for multi-host stacks."""
config = Config(
compose_dir=Path("/tmp"),
hosts={
"host1": Host(address="192.168.1.1"),
"host2": Host(address="192.168.1.2"),
"host3": Host(address="192.168.1.3"),
},
stacks={"multi-svc": ["host1", "host2", "host3"]}, # multi-host stack
)
mock_result = CommandResult(stack="multi-svc@host1", exit_code=0, success=True)
with patch("compose_farm.executor.run_command", new_callable=AsyncMock) as mock_run:
mock_run.return_value = mock_result
results = await run_on_stacks(
config, ["multi-svc"], "down", stream=False, filter_host="host1"
)
# Should only call run_command once (for host1), not 3 times
assert mock_run.call_count == 1
# Result should be for the filtered host
assert len(results) == 1
assert results[0].stack == "multi-svc@host1"
async def test_run_on_stacks_no_filter_runs_all_hosts(self) -> None:
"""Without filter_host, multi-host stacks run on all configured hosts."""
config = Config(
compose_dir=Path("/tmp"),
hosts={
"host1": Host(address="192.168.1.1"),
"host2": Host(address="192.168.1.2"),
},
stacks={"multi-svc": ["host1", "host2"]}, # multi-host stack
)
mock_result = CommandResult(stack="multi-svc", exit_code=0, success=True)
with patch("compose_farm.executor.run_command", new_callable=AsyncMock) as mock_run:
mock_run.return_value = mock_result
results = await run_on_stacks(config, ["multi-svc"], "down", stream=False)
# Should call run_command twice (once per host)
assert mock_run.call_count == 2
# Results should be for both hosts
assert len(results) == 2
@linux_only
class TestCheckPathsExist:

View File

@@ -6,6 +6,7 @@ import pytest
from compose_farm.config import Config, Host
from compose_farm.state import (
add_stack_host,
get_orphaned_stacks,
get_stack_host,
get_stacks_not_in_state,
@@ -143,6 +144,110 @@ class TestRemoveStack:
result = load_state(config)
assert result["plex"] == "nas01"
def test_remove_host_from_list(self, config: Config) -> None:
"""Removes one host from a multi-host stack's list."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n glances:\n - nas\n - nuc\n - hp\n")
remove_stack(config, "glances", "nas")
result = load_state(config)
assert set(result["glances"]) == {"nuc", "hp"}
def test_remove_last_host_removes_stack(self, config: Config) -> None:
"""Removing the last host removes the stack entirely."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n glances:\n - nas\n")
remove_stack(config, "glances", "nas")
result = load_state(config)
assert "glances" not in result
def test_remove_host_from_single_host_stack(self, config: Config) -> None:
"""Removing host from single-host stack removes it if host matches."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n plex: nas\n")
remove_stack(config, "plex", "nas")
result = load_state(config)
assert "plex" not in result
def test_remove_wrong_host_from_single_host_stack(self, config: Config) -> None:
"""Removing wrong host from single-host stack does nothing."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n plex: nas\n")
remove_stack(config, "plex", "nuc")
result = load_state(config)
assert result["plex"] == "nas"
def test_remove_host_from_nonexistent_stack(self, config: Config) -> None:
"""Removing host from nonexistent stack doesn't error."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n plex: nas\n")
remove_stack(config, "unknown", "nas") # Should not raise
result = load_state(config)
assert result["plex"] == "nas"
class TestAddStackHost:
"""Tests for add_stack_host function."""
def test_add_host_to_new_stack(self, config: Config) -> None:
"""Adding host to new stack creates single-host entry."""
state_file = config.get_state_path()
state_file.write_text("deployed: {}\n")
add_stack_host(config, "plex", "nas")
result = load_state(config)
assert result["plex"] == "nas"
def test_add_host_to_list(self, config: Config) -> None:
"""Adding host to existing list appends it."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n glances:\n - nas\n - nuc\n")
add_stack_host(config, "glances", "hp")
result = load_state(config)
assert set(result["glances"]) == {"nas", "nuc", "hp"}
def test_add_duplicate_host_to_list(self, config: Config) -> None:
"""Adding duplicate host to list does nothing."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n glances:\n - nas\n - nuc\n")
add_stack_host(config, "glances", "nas")
result = load_state(config)
assert set(result["glances"]) == {"nas", "nuc"}
def test_add_second_host_converts_to_list(self, config: Config) -> None:
"""Adding second host to single-host stack converts to list."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n plex: nas\n")
add_stack_host(config, "plex", "nuc")
result = load_state(config)
assert set(result["plex"]) == {"nas", "nuc"}
def test_add_same_host_to_single_host_stack(self, config: Config) -> None:
"""Adding same host to single-host stack does nothing."""
state_file = config.get_state_path()
state_file.write_text("deployed:\n plex: nas\n")
add_stack_host(config, "plex", "nas")
result = load_state(config)
assert result["plex"] == "nas"
class TestGetOrphanedStacks:
"""Tests for get_orphaned_stacks function."""