test-management-sync

Overview

Teams that manage test cases in a test management tool need automation result sync - without it, automated runs don't update the tool and the test-management view drifts from reality. The sync job has the same five-step shape across every vendor; only the auth model, the endpoints, and the case-ID scheme differ.

This skill wires that sync: the vendor-independent workflow below, TestRail as the worked example, and per-vendor API detail in references/.

Vendor routing

Routing rules:

• Test management is standalone TestRail → references/testrail.md.
• Test management is a Jira app → Xray (references/xray.md) or Zephyr Scale (references/zephyr.md) - check which app the Jira instance has installed; Zephyr further disambiguates into Scale / Squad / Enterprise (the zephyr reference's variant table).
• The team wants cross-run flakiness analytics, not case sync → currents-integration (different job entirely).

When to use

• The team runs both manual and automated tests and the automated results need to land in the test management tool.
• A CI pipeline must auto-create a run / execution / cycle per build and populate results so release management has the full picture.
• The team manages test cases in the tool and wants per-case mapping back to the automated test (via case-ID labels in test names or annotations).

The push-results workflow (vendor-independent)

Every vendor sync follows the same five steps:

1. Authenticate from CI secrets - never URL params, never logged (TestRail: Basic email+key; Xray Cloud: JWT exchange per run; Zephyr Scale: Bearer token).
2. Map tests to case IDs - embed the vendor's case ID / issue key in the test name (test('adds to cart [C1234]')) or use an annotation (@XrayTest(key=...), @TestRailCase(id=...), @TestCaseKey(...)). Name-embedding is lowest-friction; either way the sync script must recover the ID from the result.
3. Open a container per build - a TestRail Test Run, an Xray Test Execution, or a Zephyr Test Cycle - scoped to exactly the cases the automated suite covers, named <branch> · <sha-short>.
4. Batch results back - one batched POST (or bounded-concurrency loop) rather than one call per test; every vendor has a rate limit that per-test posting trips on real suites.
5. Close / finalize on main only - closed containers are read-only; PR runs stay open so flake-fix reruns can update them.

Run the sync as an if: always() CI step after the test step so failed runs still update the tool. The input is the runner's JUnit XML - parse it with junit-xml-analysis.

Worked example - TestRail

A Jest suite syncing one build to TestRail project 42, suite 7.

1. Tests carry the case ID in the name (Step 2, name-embedding): test('can add to cart [C1234]', ...).
2. Export credentials and run the suite to JUnit XML:

export TESTRAIL_HOST=https://acme.testrail.io
export TESTRAIL_USER=ci@acme.com
export TESTRAIL_API_KEY=...            # My Settings > API Keys
npm test -- --reporters=jest-junit

3. The sync script opens a run scoped to the covered cases (include_all: False + explicit case_ids - without this a 5,000-case project produces a 5,000-row run of empty cells), then batches the results in one POST:

run_id = open_run(42, 7, "main · a1b2c3d", [1234])   # add_run → Run ID
add_results(run_id, [                                 # add_results_for_cases (batch)
    {'case_id': 1234, 'status_id': 1, 'comment': 'green on CI', 'elapsed': '12s'},
])

status_id: 1 is Passed per the stock convention (1 Passed / 2 Blocked / 3 Untested / 4 Retest / 5 Failed) - but read get_statuses at script init and verify every ID in your map exists, because TestRail accepts unknown IDs and writes to the wrong status silently. Full script, per-result fields, and CI wiring: references/testrail.md.

4. Open the run in TestRail to see C1234 marked Passed with the 12s elapsed time. Close the run only when this is the main build (Step 5).

For the same build against Xray the container is a Test Execution fed by POST /api/v2/import/execution/junit?projectKey=... (references/xray.md); against Zephyr Scale it is a Test Cycle fed by POST /testexecutions (references/zephyr.md).

Anti-patterns (all vendors)

Limitations

• Cases must already exist in the tool. The sync maps results to existing case IDs; it doesn't author cases (that's qa-test-management's tcm-case-management).
• The mapping lives in test names / annotations. Renames that drop the ID silently unmap the test - surface unmapped tests as warnings, never drop them (per-vendor references show how).
• Vendor APIs differ in batch semantics. TestRail batches natively, Xray imports whole runner files, Zephyr Scale is per-execution with bounded concurrency - don't port one vendor's script shape blindly to another.

References

• references/testrail.md - TestRail auth, case-ID mapping, add_run / add_results_for_cases / close_run, per-result fields, CI wiring, untested-case handling.
• references/xray.md - Xray Cloud JWT auth, per-format import endpoints, @XrayTest / @Requirement mapping, testExecKey lifecycle, Playwright reporter.
• references/zephyr.md - Zephyr Scale variant disambiguation, Bearer auth, Test Cycles, /testexecutions, bounded-concurrency batching, bulk JUnit import.
• junit-xml-analysis - upstream parser for the input every sync script consumes.
• currents-integration - different role: test analytics over time, not test management.