Sematext OTel onboarding

Use this skill to wire a service that emits OpenTelemetry data into Sematext Cloud. The skill is parameter-driven: ask the user the questions in the Triage section, then assemble the env-var block from the matrices below and point them at the matching reference example in this repo.

Triage

Ask the user, in order:

1. Which Sematext region? US or EU.
2. Which App types are you wiring up? Tracing, Logs, Monitoring — any combination. Each App has its own token; the user must have created the App(s) already in Sematext Cloud.
3. Which flow?
   • Managed OTLP endpoint — service ships directly to otlp-receiver.sematext.com (or EU). Simpler. Default for new users.
   • Sematext Agent — service ships to a local Sematext Agent which forwards. Required if the agent is already deployed for infra monitoring and you want one collector for everything.
4. HTTP or gRPC? Default HTTP (http/protobuf). gRPC only if the user has a specific reason.
5. Language and deployment env? Pick from the supported matrix below. Determines which reference example to point at and which instrumentation style (auto vs manual) to recommend.
6. Auto or manual instrumentation? Auto = traces + metrics, zero code changes. Manual = traces + metrics + logs, requires SDK init code. OTel logs only ship via manual instrumentation. If the user wants Logs App data and is reaching for auto, flag this tradeoff.

Sematext fundamentals

One token per App. Each Tracing / Logs / Monitoring App you create in Sematext Cloud has its own apiKey-style token. You wire them as separate signal-specific headers — the OTel exporter sends each signal to whichever App's token is set, and skips signals with no header.

Custom auth header. Sematext uses X-API-TOKEN=<token>, not the standard Authorization: Bearer …. Some hand-coded OTLP exporters assume Bearer; those need overriding. The env-var path below works uniformly across language SDKs.

Region matters. Different OTLP endpoint hostnames for US vs EU. The token also belongs to one region; using a US token against the EU endpoint will silently drop data.

Flow A — Managed OTLP endpoint

Endpoint matrix

Env-var block

Set the headers only for the signals the user is wiring up. Each <token> is the token of the corresponding Sematext App.

# Endpoint + protocol — pick one row from the matrix above
export OTEL_EXPORTER_OTLP_ENDPOINT=https://otlp-receiver.sematext.com
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

# Per-signal token. Omit a line if the user doesn't have that App type.
export OTEL_EXPORTER_OTLP_TRACES_HEADERS=X-API-TOKEN=<tracing-app-token>
export OTEL_EXPORTER_OTLP_LOGS_HEADERS=X-API-TOKEN=<logs-app-token>
export OTEL_EXPORTER_OTLP_METRICS_HEADERS=X-API-TOKEN=<monitoring-app-token>

# Resource attributes — service.name is what shows up in the UI
export OTEL_SERVICE_NAME=my-service
export OTEL_SERVICE_VERSION=1.0.0

If the user is on auto-instrumentation, this env block plus the SDK's auto-instrumentation hook is all they need. If manual, they additionally need the SDK init code from the reference example.

Flow B — Sematext Agent

The service ships to the locally-running Sematext Agent, which forwards to Sematext Cloud. No token in the service config — the agent already has one.

Default ports

These are signal-specific, so the umbrella OTEL_EXPORTER_OTLP_ENDPOINT is not used here — the per-signal OTEL_EXPORTER_OTLP_*_ENDPOINT env vars are.

Env-var block

export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4338
export OTEL_EXPORTER_OTLP_METRICS_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://localhost:4328   # manual only

export OTEL_SERVICE_NAME=my-service
export OTEL_SERVICE_VERSION=1.0.0

Agent enabling commands (run once per signal type the user wants):

sudo /opt/spm/spm-monitor/bin/st-agent otel enable --type traces
sudo /opt/spm/spm-monitor/bin/st-agent otel enable --type metrics
sudo /opt/spm/spm-monitor/bin/st-agent otel enable --type logs

See Sematext Agent OpenTelemetry docs for install and enable details.

Reference examples in this repo

Once the user has picked language + env + instrumentation, send them to the corresponding directory. The READMEs there have language-specific build and run commands.

Single-service (one App at a time)

Each language directory has the same structure:

{lang}/
├── README.md
├── baremetal/
│   ├── auto-instrumentation/{framework}/
│   └── manual-instrumentation/{framework}/
├── docker/
│   ├── auto-instrumentation/{framework}/
│   └── manual-instrumentation/{framework}/
└── kubernetes/
    ├── auto-instrumentation/{framework}/
    └── manual-instrumentation/{framework}/

The per-language examples target the Sematext Agent flow. If the user picked the managed OTLP flow instead, the SDK init code is identical — only the endpoint + auth headers differ (follow the env-var block in Flow A above).

End-to-end (multi-service trace with W3C context propagation)

The e2e example is the natural reference for any setup that includes browser-side OpenTelemetry — browsers can't ship OTLP directly to a remote receiver (CORS), so the frontend POSTs spans to a same-origin endpoint on its own backend, which forwards to Sematext.

Verify the data is landing

Within 60 seconds of starting the instrumented service:

If nothing arrives, see Troubleshooting below.

Troubleshooting

Next steps after the user has data flowing

• Tracing App → set up a few starter alert rules (the platform now ships defaults: high response time, error count, HTTP 5xx, slow DB ops, volume anomaly, error rate anomaly).
• Monitoring App → if the user also runs the Sematext Agent for infrastructure, the OTel metrics will correlate with infra metrics automatically.
• Logs App → if the user wants logs correlated with traces, ensure traceId / spanId are emitted with each log record (manual instrumentation gives full control over this).

Resources

• Sematext Docs
• Sematext Agent — OpenTelemetry
• OpenTelemetry SDK documentation
• This repo's per-language READMEs (linked above)