testland/jaeger-trace-tests

Author integration tests that query a tracing backend for cross-service trace verification - Jaeger, Zipkin, or Grafana Tempo, same run-query-assert workflow. Jaeger all-in-one Docker for CI (OTLP gRPC :4317 + HTTP :4318 ingest, query API on :16686), `/api/traces?service=X&operation=Y` query patterns, span set + parent-child + duration assertions; Zipkin (:9411 REST API, B3 single/multi-header propagation tests, dependency graph) in references/zipkin.md; Tempo (TraceQL span selectors + structural operators, /api/search, single-binary Docker) in references/tempo.md. Use when verifying that a request produces the expected spans across service boundaries in a running Jaeger, Zipkin, or Tempo backend.

Quality

93%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Grafana Tempo backend - TraceQL queries, same workflow

Name: testland/jaeger-trace-tests
Rating: 74.4 (1 reviews)
Author: testland

Grafana Tempo ("an open source and high-scale distributed tracing backend", per Tempo getting started) follows the same run-query-assert workflow, with two differences: queries use TraceQL, and structural operators give parent-child / descendant assertions Jaeger's flat span-list API does not expose natively.

Run single-binary in CI

Ports: 3200 (HTTP API + UI), 4317/4318 (OTLP ingest). Tempo needs a minimal tempo.yaml (per the configuration reference, monolithic target: all):

# tempo.yaml
server:
  http_listen_port: 3200
distributor:
  receivers:
    otlp:
      protocols:
        grpc: { endpoint: 0.0.0.0:4317 }
        http: { endpoint: 0.0.0.0:4318 }
storage:
  trace:
    backend: local
    local: { path: /var/tempo/traces }

docker run --rm --name tempo -p 3200:3200 -p 4317:4317 -p 4318:4318 \
  -v "$PWD/tempo.yaml:/etc/tempo.yaml" \
  grafana/tempo:latest -target=all -config.file=/etc/tempo.yaml

until curl -sf http://localhost:3200/ready; do sleep 1; done   # readiness gate

The SDK exporter config is identical to Jaeger's (OTLP to :4317).

TraceQL essentials

Per Construct a TraceQL query, span selectors use { }:

Prefix	Meaning	Example
`span.`	Span attribute	`span.http.status_code`
`resource.`	Resource attribute	`resource.service.name`
`span:`	Intrinsic span field	`span:status`, `span:duration`, `span:name`, `span:kind`
`trace:`	Trace intrinsic	`trace:duration`, `trace:rootService`, `trace:rootName`

Operators: =, !=, >, >=, <, <=, =~ (regex, fully anchored - wrap with .* for partial match), !~; connectives &&, ||.

Structural operators assert span relationships: >> descendant, > direct child, << ancestor, ~ sibling:

{ span.http.url = "/checkout" } >> { span.db.system = "postgresql" }

Pipelines aggregate: { span:status = error } | count() > 1, { resource.service.name = "api" } | avg(span:duration) > 500ms.

Query via /api/search

Per the Tempo API docs, /api/search takes q (URL-encoded TraceQL), limit (default 20), start/end (epoch seconds), spss (spans per span-set, default 3), minDuration/maxDuration.

def test_checkout_span_reaches_tempo():
    with tracer.start_as_current_span("POST /order"):
        place_order(items=["widget"])
    trace.get_tracer_provider().force_flush(timeout_millis=5000)
    time.sleep(0.5)

    query = '{ resource.service.name = "checkout" && span.http.url = "/order" }'
    traces = requests.get("http://localhost:3200/api/search",
                          params={"q": query, "limit": 1}).json()["traces"]
    assert len(traces) == 1
    assert traces[0]["rootServiceName"] == "checkout"

Full trace by ID - parent-child assertions

GET /api/traces/{traceID} returns OpenTelemetry JSON; walk resourceSpans[].scopeSpans[].spans[] and assert db_span["parentSpanId"] == root_span["spanId"]. Attributes are { "key": ..., "value": { "<type>Value": ... } } objects (OTel proto format).

Tempo-specific anti-patterns

Anti-pattern	Why it fails	Fix
Omit `start`/`end` on long CI runs	Searches all backend blocks; slow	Epoch bounds scoped to the test window
Span counts from `/api/search` with default `spss=3`	`spss` caps spans per span-set	Fetch the full trace via `/api/traces/{id}`
Partial-match `=~` without wrapping	TraceQL regex is fully anchored	`=~ ".substring."`
Grafana UI as the assertion surface	HTML scraping is fragile	`/api/search` + `/api/traces/{id}`

Limitations

backend: local suits CI, not production (object storage recommended per the configuration reference).
TraceQL requires the Parquet block format (Tempo's default); legacy TSDB blocks don't support it.