Testing and CI

Overview

Use this skill for caddy-security test selection, fixture maintenance, and CI reproduction. Prefer the narrowest go test command while editing, then use Makefile targets when the user asks for the repository workflow, report artifacts, or CI-like validation.

The Go module is rooted at the repository top level. Most tests live in the root security package and cover Caddyfile parsing, Caddyfile adaptation, and runtime authcrunch config resolution.

Command Selection

Use direct Go tests for quick feedback:

go test ./...
go test -run TestParseCaddyfileAuthorization ./...
go test -run TestCaddyfileAdaptAuthenticationToJSON ./...
go test -run TestResolveRuntimeAppConfig ./...

Use make test for the full local workflow. It creates .coverage, installs test report tools if missing, runs go test -json -v ./... with a coverage profile, writes .coverage/test_output.jsonl, generates HTML reports, and fails if any JSON test action failed.

Use make qtest only when the Makefile's current QUICK_TEST_PATTERN is the intended scope. Prefer direct go test -run ... ./... for a different focused test instead of editing the Makefile just to run one command.

Use make coverage after .coverage/coverage.out exists, usually after make test. It writes .coverage/coverage.html, refreshes .coverage/coverage.out, and prints non-100% function coverage.

Use make build when validation needs the bin/authcrunch command binary or when Caddyfile fixture formatting may matter. Use make fmtcfg after a build to format Caddyfile fixtures under testdata/caddyfile_adapt and assets/config.

make dep, make install-test-tools, go mod tidy, go mod verify, go mod download, and go install may require network access.

Test Surfaces

Parser tests use inline Caddyfile snippets and caddyfile.NewTestDispenser. They call parser functions, unpack generated JSON into maps, and compare with cmp.Diff. Whitespace in inline want JSON is not semantically important. Add or update these tests when directive parsing behavior changes:

• caddyfile_authn_test.go: authentication portal parsing.
• caddyfile_authn_misc_test.go: authentication misc/cookie/crypto/UI paths.
• caddyfile_authz_test.go: authorization policy parsing.
• caddyfile_identity*_test.go: identity stores and providers.
• caddyfile_credentials_test.go: credential directives.
• caddyfile_messaging_test.go: messaging directives.
• caddyfile_sso_provider_test.go: SSO provider directives.
• caddyfile_test.go: app-level parse coverage.

Adapt tests live in TestCaddyfileAdaptAuthenticationToJSON in caddyfile_adapt_test.go. Each case uses testdata/caddyfile_adapt/<prefix>.Caddyfile as input and compares against <prefix>.json. Optional <prefix>.env files provide environment variables; blank lines and comments are ignored, and variables are cleaned up by the test. Use this path when a user-visible Caddyfile-to-JSON output changes.

Runtime resolution tests live in TestResolveRuntimeAppConfig in caddyfile_resolve_test.go. Each case reads <prefix>.json, extracts the security.config object, runs ResolveRuntimeAppConfig, and compares against <prefix>_resolved.json. The test also fails if unresolved {env...} tokens remain. It writes temporary *_tmp_input.json and *_tmp_output.json files, removing them on success and leaving them on failure for debugging.

Expected-error tests set shouldErr: true and compare the exact error string with cmp.Diff. Keep expected errors specific. The static secrets manager fixture currently expects a module-not-registered error because the external secrets plugin is not registered in this test binary.

Adding Coverage

When adding or changing a Caddyfile directive, add focused parser coverage in the nearest caddyfile_*_test.go file. Include both the successful config shape and a malformed input when the parser has a meaningful error path.

When the change affects Caddy's adapted JSON, add or update a fixture triplet in testdata/caddyfile_adapt: <prefix>.Caddyfile, <prefix>.json, and optionally <prefix>.env. If runtime defaults, replacements, credentials, UI, OAuth, registration, or cookie behavior changes after adaptation, also add or update <prefix>_resolved.json and include the prefix in TestResolveRuntimeAppConfig.

After fixture edits, run the focused test first, then a broader command:

go test -run TestCaddyfileAdaptAuthenticationToJSON ./...
go test -run TestResolveRuntimeAppConfig ./...
go test ./...

If *_tmp_input.json or *_tmp_output.json files remain after a failing runtime resolution test, inspect them, fix the fixture or implementation, and remove the generated temp files before finishing unless the user explicitly wants debug artifacts kept.

CI Workflow

.github/workflows/build.yml runs on pushes and pull requests to main using Ubuntu and Go 1.25.x. It installs make and libnss3-tools, sets GOBIN to the runner-local bin directory, runs make dep, go mod tidy, go mod verify, go mod download, runs make test || true followed by make test, then runs make coverage and uploads .coverage/coverage.html.

For local CI reproduction, use:

make dep
go mod tidy
go mod verify
go mod download
make test
make coverage

The release workflow runs GoReleaser on v* tags or manual dispatch with Go ~1.25. Treat release workflows, tags, pushes, and chained release Makefile targets as human-operator actions unless the user explicitly requests them.

The CLA workflow may update assets/cla/signatures.json through GitHub automation. Do not edit CLA signatures or consent files unless the user asks.

Generated Artifacts

Treat these as generated outputs unless the user explicitly asks to preserve or commit them:

bin/authcrunch
.coverage/coverage.html
.coverage/coverage.out
.coverage/test_output.jsonl
.coverage/test_output.html
testdata/caddyfile_adapt/*_tmp_input.json
testdata/caddyfile_adapt/*_tmp_output.json

Formatted Caddyfiles and JSON fixtures can be intentional source changes. Review the diff after running format, build, test, or coverage commands.