secure-log2test

Turn a Kibana API log export into an executable pytest suite. Auth headers and secret-looking body fields redacted before they reach the output.

Status: v1.1.0 on PyPI. Stable per semver. Active roadmap, see open issues.

📖 Read the design write-up on Dev.to: privacy constraint, three-layer redaction, the v1.0.0 to v1.0.1 user-feedback story.

Why

You have Kibana logs from staging or production. Each entry is a real request: method, URL, status, duration, headers, body. That's a regression suite waiting to happen. Most teams either ignore it, screenshot interesting failures into Jira, or hand-write pytest cases from log entries one at a time.

I needed a faster path. secure-log2test reads a Kibana JSON export and writes a pytest module you can run and commit. Auth values get replaced with ***REDACTED*** before they ever touch the output, so a generated suite is safe to push to a public repo.

The tool exists because at Лента I kept doing the same five steps by hand for every production incident: open Kibana, scroll, copy the failing request, paste into a new test, repeat. Five minutes per request times ten requests means an hour gone before any actual debugging starts.

Quickstart

pip install secure-log2test

secure-log2test data/sample_kibana_export.json --output tests_generated.py
pytest tests_generated.py -v

A sample export ships with the repo (data/sample_kibana_export.json), so you can see real output without setting up a Kibana instance first. Grab it from the GitHub repo if you installed from PyPI.

For local development:

git clone https://github.com/golikovichev/secure-log2test
cd secure-log2test
python -m venv .venv && source .venv/bin/activate  # or .venv\Scripts\activate on Windows
pip install -e ".[dev]"
pytest tests/ -v

How it works

Two stages, kept separate.

Parse (core/parser.py). Reads the Kibana JSON and validates each entry through Pydantic v2. Two layers of redaction run before any further processing:

• A static list of well-known headers (authorization, proxy-authorization, proxy-authenticate, cookie, set-cookie, x-api-key, x-auth-token, x-csrf-token, x-access-token, refresh-token, id-token, x-amz-security-token, authentication, dpop, x-hub-signature, x-hub-signature-256). The last three carry credential material (a DPoP proof JWT, webhook HMAC signatures) whose names the regex below would otherwise miss.
• A regex pattern (auth|token|secret|key|session|cookie|credential|bearer|password|passwd) that catches custom header names and body field names project teams invent.

The same logic walks request bodies recursively, so {"password": "..."}, {"client_secret": "..."}, OAuth {"refresh_token": "..."} all get scrubbed at parse time. It also runs over URL query strings, so ?access_token=... or ?api_key=... are redacted while the path and non-sensitive parameters stay intact. Name matching is case-insensitive. Values get replaced with ***REDACTED***. The original input dict is not mutated.

Generate (core/generator.py). Takes the cleaned entries and renders a Jinja2 template (templates/test_module.py.j2) into a pytest module. Each log entry becomes one test_* function. The slug filter turns /api/v1/users/42 into a stable function name.

Options

Pick a marker your downstream pipeline expects:

secure-log2test data/sample_kibana_export.json --redact-marker "[SCRUBBED]"

The split lets you reuse the parser for other formats. If you want to generate Locust scripts, k6 scenarios, or an OpenAPI spec from the same logs, the parser stays. Only the template changes.

Sample output

Given this Kibana log entry:

{
  "method": "POST",
  "url": "/api/v1/users",
  "status": 201,
  "headers": {"Authorization": "Bearer abc.xyz", "Content-Type": "application/json"},
  "body": {"name": "Test", "email": "test@example.com"}
}

The generator emits something like:

def test_post_api_v1_users():
    response = requests.post(
        f"{BASE_URL}/api/v1/users",
        headers={"Authorization": "***REDACTED***", "Content-Type": "application/json"},
        json={"name": "Test", "email": "test@example.com"},
    )
    assert response.status_code == 201, (
        f"Expected 201, got {response.status_code}: {response.text[:200]}"
    )

The Authorization value never leaves the parser intact. You set the real token in your environment at run time.

Limitations

What v1.0.1 does not handle yet. Calling them out so the tool stays trustworthy.

• Kibana / Elasticsearch JSON export shape only. Grafana Loki Explore exports are tracked in #4.
• Single-file input. Multi-file batch mode is on the roadmap.
• Output format: pytest, JSON, or CSV.
• Response body assertions. Status code only for now, full body match is on the v1.1 list (#1).
• Custom redaction rules via config file are on the v1.2 list (#2).
• OAuth replay. Only static Authorization headers, redacted to a placeholder.
• Multipart bodies and file uploads.
• Streaming responses or chunked transfer.

If something on this list blocks you, open an issue.

Roadmap

Open the issue tracker for the live picture; two good first issue slots are currently open if you want to jump in.

Tests

pytest tests/ -v

86 tests, covering:

• Parser unit tests for valid input, malformed input, header redaction, body redaction walker, empty bodies.
• Edge cases for 5xx responses, missing fields, custom auth header patterns, OAuth refresh tokens in request bodies.
• CI smoke test that runs the CLI end-to-end on the sample export and parses the generated Python with ast.parse.

CI runs on Python 3.10, 3.11, 3.12, and 3.13 via GitHub Actions.

Security note

The redaction layer catches the well-known auth headers plus anything whose name contains auth, token, secret, key, session, cookie, credential, bearer, password, or passwd. This works for both header names and JSON body field names. If your team uses something the pattern misses (truly opaque internal name), add it to SENSITIVE_HEADERS in core/parser.py before generating output. PRs welcome.

Never commit a generated suite that includes real production tokens. The redaction layer is a safety net, not a substitute for review.

Related projects and patterns

Once secure-log2test has produced your replay suite, the next layer of work is usually fixture organisation, shared auth, and the parametrize patterns that scale across hundreds of generated cases. The tessl-labs/pytest-api-testing skill on the Tessl Registry collects those follow-on conventions: httpx AsyncClient setup, conftest.py fixture shape, database isolation, parametrize for edge cases, and auth-flow handling. Useful reference when the generated suite starts growing its own test infrastructure.

Sister projects in the same workspace:

• postman2pytest: same idea but the input is Postman Collection JSON instead of Kibana log exports.
• pytest-conversational: pytest plugin for multi-turn dialogue testing.

Contributing

Issue templates and PR guidance live in CONTRIBUTING.md. Bug reports with a redacted sample log are the most useful kind.

Licence

MIT. See LICENSE.