CtrlK
BlogDocsLog inGet started
Tessl Logo

testland/test-management-sync

Syncs automated test results into test management tools - TestRail (standalone, `add_run` + batched `add_results_for_cases`), Xray for Jira (JWT auth + `/api/v2/import/execution/*`), and Zephyr Scale (Bearer token + `/testexecutions`) - from CI. The body carries the vendor-independent push-results workflow (map tests to case IDs, open a run / execution / cycle per build, batch results back, close on main only, run as an `if: always()` step) with TestRail as the worked example; full vendor specifics live in references/ (testrail.md, xray.md, zephyr.md). Use when automated suites must keep the team's test management view in sync without a human copy-paste step; for hosted cross-run flakiness analytics rather than TCM sync use currents-integration, and for authoring / migrating test CASES rather than pushing results see qa-test-management's tcm-case-management.

80

Quality

100%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Low

Low-risk findings worth noting

Overview
Quality
Evals
Security
Files

xray.mdreferences/

Xray for Jira - vendor specifics

Reference detail for test-management-sync. Xray exposes test cases / executions as Jira issue types (Test, Test Set, Test Plan, Test Execution, Pre-Condition) - automated test sync writes results into Test Execution issues.

Xray comes in two flavors:

FlavorAuthImport endpoints
Xray Cloudclient_id + client_secret → JWThttps://xray.cloud.getxray.app/api/v2/import/...
Xray Server / DCJira PAT or Basic authhttps://<jira>/rest/raven/2.0/import/...

This page covers the Cloud flow as the primary path; Server notes are inline.

The official documentation is at docs.getxray.app. At the time of authoring (2026-05-05), the Cloud import-results page was 403 to automated WebFetch; the URL is the canonical reference for real-browser navigation. The well-known endpoints + payload shapes below are documented in the official xray-junit-extensions GitHub repo (xray-junit-ext) and the xray-postman-collections public collections, both first-party Xray-App tools.

How to use

  1. Determine the flavor - Xray Cloud vs Server / DC - since it sets the auth model and import host (table above).
  2. Authenticate: Cloud exchanges client_id + client_secret for a 24h JWT; Server / DC uses a Jira PAT or Basic auth (Step 1).
  3. Pick the import endpoint that matches your runner's output format (/junit, /cucumber, /testng, /nunit, /robot, or generic JSON) (Step 2).
  4. Pin each test method to an existing Test issue with @XrayTest(key=...) (and @Requirement for coverage) so renames don't orphan issues (Step 3).
  5. Emit Xray-extended output - JVM via xray-junit-extensions (Step 5), JavaScript via the official Playwright reporter (below).
  6. POST the results with projectKey=..., adding testExecKey=... to update an existing Test Execution instead of creating a new one (Step 7).
  7. Wire it as an if: always() CI step (End-to-end CI shape below).

Step 1 - Authenticate (Cloud)

Per the xray-junit-ext reference, Cloud auth is a two-step flow:

# 1. Exchange credentials for a JWT
JWT=$(curl -X POST 'https://xray.cloud.getxray.app/api/v2/authenticate' \
  -H 'Content-Type: application/json' \
  -d '{"client_id": "'"$XRAY_CLIENT_ID"'", "client_secret": "'"$XRAY_CLIENT_SECRET"'"}' \
  | tr -d '"')

# 2. Use the JWT in subsequent requests
curl -X POST 'https://xray.cloud.getxray.app/api/v2/import/execution/junit' \
  -H "Authorization: Bearer $JWT" \
  -H 'Content-Type: application/xml' \
  --data-binary @junit.xml

The JWT has a 24-hour validity (well-documented across Xray API clients); refresh per CI run.

For Xray Server / DC, use a Jira PAT or Basic auth instead; the JWT step is skipped.

Step 2 - Pick the import endpoint per format

Test runner outputCloud endpoint
JUnit XML (Maven, Gradle, Jest, pytest, etc.)/api/v2/import/execution/junit
Cucumber JSON/api/v2/import/execution/cucumber
TestNG XML/api/v2/import/execution/testng
NUnit XML (.NET)/api/v2/import/execution/nunit
xUnit XML (.NET)/api/v2/import/execution/xunit
Robot Framework XML/api/v2/import/execution/robot
Generic JSON (Xray-shape)/api/v2/import/execution

Each endpoint accepts the format's native output and parses it server-side; no per-test pre-mapping is needed.

For the generic JSON endpoint, payload shape:

{
  "info": {
    "summary": "CI run for ABC-123",
    "description": "Automated regression run",
    "user": "ci-runner",
    "version": "1.4.5",
    "revision": "abc1234",
    "testPlanKey": "PROJ-100",
    "testEnvironments": ["staging", "chrome"]
  },
  "tests": [
    {
      "testKey": "PROJ-1234",
      "start": "2026-05-05T14:00:00Z",
      "finish": "2026-05-05T14:00:12Z",
      "comment": "Test passed cleanly",
      "status": "PASSED"
    }
  ]
}

Step 3 - Map test methods to Xray Test issues

Per xray-junit-ext, the JUnit 5/6 extension provides two annotations:

@XrayTest

"enforce mapping of result to specific, existing Test identified by issue key, using the key attribute" (xray-junit-ext)

@Test
@XrayTest(key = "CALC-1000")
public void canAddNumbers() { /* ... */ }

Without @XrayTest, the extension auto-creates a Test issue per JUnit method on first run (auto-provisioning). Pinning with key prevents drift across renames.

@Requirement

"identify the covered requirement(s) ... it's possible to identify one covered issue or more" (xray-junit-ext)

@Test
@Requirement("CALC-1234")
public void canAddNumbers() { /* ... */ }

This populates the Jira-side coverage link from the test back to the requirement issue.

Step 4 - XrayTestReporter for evidence

Per xray-junit-ext, the XrayTestReporterParameterResolver extension injects an XrayTestReporter into test methods:

"Add comments to Test Runs / Define Test Run custom field values / Attach evidence files" (xray-junit-ext)

@Test
@ExtendWith(XrayTestReporterParameterResolver.class)
@XrayTest(key = "CALC-1000")
public void canAddNumbers(XrayTestReporter reporter) {
    // ... test logic ...
    reporter.addComment("Calculator returned correct sum");
    reporter.addEvidence("screenshot.png");
}

Evidence files are attached to the Test Run inside the Test Execution issue - useful for failure debugging from Jira.

Step 5 - Configure the extension

Per xray-junit-ext, the extension reads xray-junit-extensions.properties for output config:

# xray-junit-extensions.properties (place on the test classpath)
report_filename=TEST-results
report_directory=target/xray-reports
add_timestamp_to_report_filename=false

The output is JUnit XML augmented with Xray-specific metadata; pass this enriched XML to the import endpoint (Step 2).

Step 6 - End-to-end CI shape

# .github/workflows/xray-sync.yml
- name: Run tests with Xray-aware JUnit reporter
  run: ./mvnw -B verify
  # Produces target/xray-reports/TEST-results.xml

- name: Get Xray JWT
  id: xray_auth
  env:
    XRAY_CLIENT_ID: ${{ secrets.XRAY_CLIENT_ID }}
    XRAY_CLIENT_SECRET: ${{ secrets.XRAY_CLIENT_SECRET }}
  run: |
    JWT=$(curl -s -X POST 'https://xray.cloud.getxray.app/api/v2/authenticate' \
      -H 'Content-Type: application/json' \
      -d '{"client_id":"'"$XRAY_CLIENT_ID"'","client_secret":"'"$XRAY_CLIENT_SECRET"'"}' \
      | tr -d '"')
    echo "::add-mask::$JWT"
    echo "jwt=$JWT" >> "$GITHUB_OUTPUT"

- name: Import to Xray
  if: always()
  run: |
    curl -X POST 'https://xray.cloud.getxray.app/api/v2/import/execution/junit?projectKey=CALC' \
      -H "Authorization: Bearer ${{ steps.xray_auth.outputs.jwt }}" \
      -H 'Content-Type: application/xml' \
      --data-binary @target/xray-reports/TEST-results.xml

projectKey (the Jira project key) is the critical query param; without it the import fails or lands in the wrong project.

Verify after the POST: assert a 2xx whose body carries the Test Execution issue key the import created or updated. A 4xx (bad projectKey, expired JWT, or malformed XML) or a body with no key means nothing landed - fix that cause and re-run; retry once on a 5xx (transient), and never treat a non-2xx as success.

Non-JVM teams: Playwright reporter

Per the Xray-App GitHub org, the playwright-junit-reporter project ships a Playwright reporter that emits Xray-compatible JUnit XML:

// playwright.config.ts
reporter: [
  ['list'],
  ['@xray-app/playwright-junit-reporter', {
    outputFile: 'target/xray-reports/results.xml',
  }],
],

Then the same import endpoint consumes the output.

Step 7 - Test Execution issue lifecycle

By default, each import creates a new Test Execution issue. For "update an existing execution per build" (e.g. one execution per release branch), pass testExecKey=PROJ-XYZ in the query string:

POST /api/v2/import/execution/junit?projectKey=CALC&testExecKey=CALC-9999

Pattern:

  • PR runs: new Test Execution per push (lots of issues; auto-archive via Jira workflow after PR merge).
  • Release runs: one Test Execution per release branch; updated on every push (uses testExecKey).

Xray-specific anti-patterns

Anti-patternWhy it failsFix
Auto-provisioning Test issues without @XrayTest(key=...)Renaming a test method creates a new Test issue; old one orphans.Pin every test method to an existing issue with @XrayTest(key="...").
Storing JWT secret in repo / logCloud secret leak; immediate quota abuse.Mask in CI; fetch fresh per run; never log (::add-mask::).
Importing without projectKeyImport lands in default project; other teams see your tests.Always pass projectKey.
New Test Execution per PR push100+ Jira issues per active PR; project clutter.Reuse testExecKey per PR; create new only on push to main.
Using regular JUnit XML reporter (not the Xray-extended one)Loses @XrayTest / @Requirement annotations; mapping fails.Use xray-junit-extensions for JVM or the official Playwright reporter.
Long-lived JWT cache (>24h)Auth fails; CI runs broken silently.Fetch JWT per run; respect the 24h validity.
Importing 5,000 results in one requestServer times out; partial state.Split per suite or per Test Execution; Xray Cloud's import endpoints are sized for typical CI batches.

Limitations

  • Cloud and Server have different endpoints + auth. Don't share scripts; maintain per-flavor variants.
  • Auto-provisioning creates issues in the project. Heavy auto-provisioning can balloon Jira project size; pre-create Test issues via the Xray UI or xray-postman-collections for a pre-seeded suite.
  • No first-party Cypress reporter in the Xray-App org; the community-maintained cypress-xray-junit-reporter is the de-facto choice but isn't officially supported.
  • Documentation domain is JS-rendered + auth-gated. Per the source-fetch failure documented here (2026-05-05), the primary canonical references for the import API require real browser navigation; the GitHub repos under https://github.com/Xray-App/ are the most reliable programmatically-fetchable sources.

References

  • xray-junit-ext - official xray-junit-extensions repo: @XrayTest(key=...), @Requirement(...), XrayTestReporter injection, xray-junit-extensions.properties config.
  • https://github.com/Xray-App/xray-postman-collections - official Postman collections for every Xray Cloud public API endpoint (including /api/v2/authenticate and /api/v2/import/execution/*).
  • https://github.com/Xray-App/xray-maven-plugin - Maven-side integration with the same import + auth shape.
  • https://github.com/Xray-App/playwright-junit-reporter - official Playwright reporter for Xray-compatible JUnit XML.
  • https://docs.getxray.app/ - canonical doc portal (auth/region-gated; consult in a real browser).

SKILL.md

tile.json