webmin/t/README.md

Webmin test suite

Two kinds of tests live under this tree:

• Repo-root t/ for core code (miniserv.pl, web-lib-funcs.pl, top-level scripts, bin/).
• Per-module t/ for module-internal coverage (see nftables/t/ for the established pattern: perlcritic.t plus a run-tests.t smoke harness).

Running tests

prove -lr                                # everything, including modules t/
prove -lr t                              # everything under repo-root t/
prove t/compile.t                        # one test file
WEBMIN_COMPILE_T_FILTER='^\./acl/' prove t/compile.t   # one module

prove and Test::More are core, though on RPM-based distros, you need perl-Test-Harness.

Socket compatibility tests on a VM

web-lib-funcs-open-socket.t tests address selection without network access.

Run the additional compatibility tests as root on a disposable Linux VM with Webmin, Python 3, OpenSSL, and IPv6 loopback support:

WEBMIN_OPEN_SOCKET_VM_TEST=1 timeout 90 prove -v t/web-lib-funcs-open-socket-vm.t

The VM test uses temporary loopback servers for HTTP, HTTPS, CONNECT proxies, passive FTP, and service greetings. It checks IPv4 and IPv6 fallback, source binding, and error reporting without changing service configuration. It skips unless explicitly enabled.

By default, it tests open_socket from this checkout with the installed Webmin helpers. Set WEBMIN_OPEN_SOCKET_SOURCE to another web-lib-funcs.pl to compare versions.

API-only account tests on a VM

api-only-accounts.t, api-only-lifecycle.t, api-only-sql.t and miniserv-api-only.t cover the api_only global permission without a server.

The live fixtures under fixtures/ run as root on a disposable Linux guest with Webmin installed, never on the workstation. They read the Webmin root from /etc/webmin/miniserv.conf, so Debian and RPM layouts both work. Put the changed source files in candidate.tar inside a task directory under /tmp, then:

python3 fixtures/api-only-sandbox.py /tmp/task      # isolated copy on 127.0.0.1:18443
python3 fixtures/api-only-create.py /tmp/task       # new accounts through Webmin Users
python3 fixtures/api-only-groups.py /tmp/task       # inherited controls and group name reuse
python3 /tmp/task/api-only-live.py /tmp/task        # edits, resets, copies, clones, renames
python3 fixtures/api-only-boundary.py /tmp/task     # CGI modes, static, DAV, WebSocket, sessions
python3 fixtures/api-only-userdb.py /tmp/task mysql setup    # or postgresql; rerun the account tests
python3 fixtures/api-only-userdb.py /tmp/task mysql cleanup

api-only-live.py with fixtures/api-only-account.pl beside it also runs against the guest's real Webmin on port 10000. Stop the sandbox PID in miniserv.pid and remove the task directory afterwards; it holds copies of the guest configuration.

Coverage reports

HARNESS_PERL_SWITCHES=-MDevel::Cover prove -lr
cover

You need Devel::Cover installed.

What's here

File	What it checks
compile.t	Every .pl, .cgi, and shebang-perl script in bin/ parses cleanly (perl -c). Catches breakage from bulk refactors without browsing every page. ~12s for the full tree.
miniserv.t	Contract test for miniserv-lib.pl helpers used by miniserv.pl — status codes, headers, body rendering, log behaviour.

Testing helpers from executable code

Many Webmin scripts mix helper definitions with a main body that opens sockets, reads /etc/webmin/*, or spawns CGIs. Prefer moving testable helpers into a small *-lib.pl file, keeping the executable script focused on startup and request handling, and requiring the library from tests. miniserv.pl follows that pattern: daemon startup remains in miniserv.pl, while helper coverage loads miniserv-lib.pl directly.

A second useful idiom still applies to simple CLI tools whose sub definitions precede the main call.

One-liner (sub definitions precede the main call)

Used by most bin/ CLI tools, which already define sub main and dispatch to it at the bottom:

#!/usr/bin/env perl
use strict; use warnings;

sub main {
    ...
    return 0;
}
exit main(\@ARGV) if !caller(0);

sub helper { ... }

!caller(0) is true at script invocation (depth-0 frame absent) and false under require.

Sub-stubbing in tests

miniserv.t is the canonical example. The pattern:

1. require the helper library.
2. Replace side-effecting subs (socket I/O, logging, disk reads) with capture-buffer overrides under no warnings 'redefine'.
3. Populate package globals (%miniserv::config, @miniserv::roots, etc.) yourself instead of running the config loader.
4. Call the sub under test. Assert on contract (status code, presence of required headers, structural balance of emitted markup) — not on cosmetics like exact wording or class names.

Tying tests to the contract rather than the rendering lets the UI evolve without breaking the test, while still catching real regressions.

Tiered coverage policy

Not all 268 modules deserve the same investment.

• Tier 1 — security-critical, network-facing. miniserv.pl, web-lib-funcs.pl, acl/, auth, file upload paths. Comprehensive contract tests; mock filesystem and backquote_command for parser coverage of external-binary output.
• Tier 2 — active refactor surface. Currently nftables/, firewalld/, and whichever module is being reviewed. Mandatory tests for new code; perlcritic.t at severity 5 as a gate.
• Tier 3 — stable OS-config wrappers. Covered by compile.t plus optional per-module perlcritic.t. Don't chase line coverage on parsers for config files that haven't changed in a decade.

The goal is not coverage-as-a-number. It's:

• Every parser round-trips its serializer.
• Every privilege boundary has a test.
• Every external-binary call has a mock-driven test for its output parser.

Perlcritic conventions

A few rewrites come up repeatedly when bringing a module under perlcritic.t severity 5. Document them here so reviewers and future refactors land on the same shape.

Stringy eval "use Module" → block eval + require

perlcritic (BuiltinFunctions::ProhibitStringyEval) flags eval "use Module::Name;";. The fix is a block eval that does the work use would have done at compile time, deferred to runtime:

# before
eval "use DBI;";
if ($@) { return "DBI not installed"; }

# after
eval { require DBI; DBI->import; 1 }
        or return "DBI not installed";

The trailing 1 makes the block return true on success so the caller's or ... only fires on failure. Pass import args the same way a use would: Module->import(qw(foo bar)). If the module exports nothing you need, drop the ->import call but keep the 1.

acl/acl-lib.pl has worked examples for the common shapes — single-module probes, fallback chains (SDBM_File → NDBM_File, MD5 → Digest::MD5), and driver lookups (DBD::mysql, DBD::Pg, Net::LDAP).

Adding a per-module test directory

yourmodule/
  t/
    perlcritic.t   # see nftables/t/perlcritic.t for the template
    run-tests.t    # see nftables/t/run-tests.t for the WEBMIN_CONFIG / tmpdir setup

A module's tests are reachable from prove -lr at the repo root (no path arg, so the recursive walk starts at the cwd). prove -lr t only walks within t/ and will miss <module>/t/.

Caveats

• WEBMIN_COMPILE_T_STRICT=1 turns missing-CPAN-module skips into failures. Use this in CI on a fully-provisioned image; leave it off on dev boxes where optional deps (Pod::Simple::Wiki, Encode::Detect::Detector) may not be installed.
• .pl is also the Polish translation suffix in Webmin. compile.t skips <file>.pl when a sibling <file> (no extension) exists; this catches config.info.pl and module.info.pl data files without a hardcoded list.