testland/statsig-test

Wraps Statsig SDK testing patterns - server-side statsig.initialize with an API key, gate / experiment / dynamic-config evaluation (checkGate, getExperiment, getConfig), local-evaluation offline mode, overrideGate / overrideConfig to force a user into an arm, assignment-integrity tests. Use when writing tests for Statsig-instrumented application code. For another experimentation SDK use the matching harness - optimizely-test, vwo-test, amplitude-experiment-test, or split-io-test; for experiment DESIGN gates not SDK code use ab-test-validity-checklist.

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

name:: statsig-test
description:: Wraps Statsig SDK testing patterns - server-side statsig.initialize with an API key, gate / experiment / dynamic-config evaluation (checkGate, getExperiment, getConfig), local-evaluation offline mode, overrideGate / overrideConfig to force a user into an arm, assignment-integrity tests. Use when writing tests for Statsig-instrumented application code. For another experimentation SDK use the matching harness - optimizely-test, vwo-test, amplitude-experiment-test, or split-io-test; for experiment DESIGN gates not SDK code use ab-test-validity-checklist.

statsig-test

Name: testland/statsig-test
Rating: 75.2 (1 reviews)
Author: testland

Overview

Per docs.statsig.com, the Statsig SDK is available for Node.js, Java, Python, Go, Ruby, .NET, PHP, Rust, and C++ - all with the same conceptual surface: gates, experiments, dynamic configs.

When to use

Tests for code that reads a Statsig gate / experiment.
Verifying assignment integrity per ab-test-validity-checklist Step 3.
Local-evaluation tests when network access is restricted.

Authoring

Install

npm install --save-dev statsig-node       # Node
pip install statsig                        # Python

Initialize for testing

import statsig from 'statsig-node';

beforeAll(async () => {
  await statsig.initialize(process.env.STATSIG_SERVER_KEY!, {
    localMode: true,    // No network; all gates / configs return defaults
  });
});

afterAll(async () => {
  await statsig.shutdown();
});

localMode: true is the test-mode flag - gates fall back to the default value, configs return empty, no network calls.

Override gates / experiments per user

test('user in treatment sees new UI', async () => {
  statsig.overrideGate('new_ui_gate', true, 'user-1');

  const enabled = await statsig.checkGate({ userID: 'user-1' }, 'new_ui_gate');
  expect(enabled).toBe(true);

  const disabledForOthers = await statsig.checkGate({ userID: 'user-2' }, 'new_ui_gate');
  expect(disabledForOthers).toBe(false);
});

statsig.overrideGate(gateName, value, userID) - pin a user to a value for the lifetime of the test.

Experiment evaluation

test('user in arm B sees increased font size', async () => {
  statsig.overrideConfig('font_size_experiment', { font_size: 20 }, 'user-1');

  const exp = await statsig.getExperiment({ userID: 'user-1' }, 'font_size_experiment');
  expect(exp.value).toEqual({ font_size: 20 });
});

Assignment integrity tests

Per ab-test-validity-checklist Step 3:

test('same user always gets same arm (determinism)', async () => {
  const arm1 = await statsig.getExperiment({ userID: 'user-1' }, 'exp-x');
  const arm2 = await statsig.getExperiment({ userID: 'user-1' }, 'exp-x');
  expect(arm1.value).toEqual(arm2.value);
});

test('different users may get different arms', async () => {
  const arms = await Promise.all(
    Array.from({ length: 100 }, (_, i) =>
      statsig.getExperiment({ userID: `user-${i}` }, 'exp-x').then(e => e.value)
    )
  );
  const uniqueArms = new Set(arms.map(a => JSON.stringify(a)));
  expect(uniqueArms.size).toBeGreaterThan(1);
});

Exposure event firing

Statsig fires an exposure event per evaluation by default; verify in tests:

test('exposure logged on evaluation', async () => {
  const events: any[] = [];
  // Statsig SDK exposes a hook for testing event logging
  statsig.flush();  // Force flush any pending events
  // Inspect via test mock of the event-uploader
});

Running

npm test

For CI, use STATSIG_SERVER_KEY set to a test-tier key OR rely on localMode: true for fully-offline tests.

CI integration

jobs:
  statsig-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npm test
        env:
          STATSIG_SERVER_KEY: ${{ secrets.STATSIG_TEST_KEY }}

Anti-patterns

Anti-pattern	Why it fails	Fix
Tests using production Statsig API key	Production traffic polluted	Per-env keys; or `localMode: true`
Skipping `statsig.shutdown()`	Pending event upload leaks	Always shutdown
Asserting on exact internal config IDs	Statsig config IDs change	Assert on returned values
Tests rely on real evaluation (no override)	Flaky if Statsig service changes	Override per test
Forgetting `userID` in evaluation	Returns default; not the test you wrote	Always pass full user object
Sharing one Statsig instance across test files	Override leaks	Per-test cleanup

Limitations

localMode is most-but-not-fully-offline. Some SDK initialization paths still ping Statsig.
No deterministic arm assignment without override. Test randomness via override; for hash-stability use the network-mode SDK.
Overrides are SDK-instance scoped. Don't help if the app spawns multiple processes.
Doesn't validate Statsig's server-side analysis. That's the platform's responsibility; tests verify your code's interaction with the SDK.

References

Statsig docs: docs.statsig.com/.
Statsig SDK reference (per language): docs.statsig.com/server-core/.
Companion catalogs: guardrail-metrics-reference, peeking-problem-reference, ab-test-validity-checklist.
Sibling SDKs: optimizely-test, vwo-test, amplitude-experiment-test.

.tessl-plugin

SKILL.md

tile.json