testland/feature-flag-test-harness

Builds a test harness that runs the same suite under every relevant flag combination - picks the minimum cover (single flags + pairwise interactions where the team marks them, not the full 2^N cartesian product), wires an OpenFeature in-memory provider so the suite never hits the production flag service, runs each combination as its own labeled CI matrix shard, and emits a per-combination result matrix. Use when a feature behind a flag must be verified on AND off (release toggles + experiment toggles per Hodgson) and the team wants those runs deterministic and parallel.

Quality

99%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Provider wiring - OpenFeature in-memory provider

Name: testland/feature-flag-test-harness
Rating: 79.2 (1 reviews)
Author: testland

Per openfeature-providers, "Providers are responsible for performing flag evaluations" - the in-memory test provider returns the flag values the test wants.

Node / TypeScript

// tests/harness/flag-harness.ts
import { OpenFeature, InMemoryProvider } from '@openfeature/server-sdk';

export function withFlags(flags: Record<string, unknown>) {
  const provider = new InMemoryProvider(
    Object.fromEntries(
      Object.entries(flags).map(([k, v]) => [k, {
        defaultVariant: 'configured',
        variants: { configured: v },
        disabled: false,
      }]),
    ),
  );
  return OpenFeature.setProviderAndWait(provider);
}

Then in the test setup:

import { withFlags } from './harness/flag-harness';

beforeAll(async () => {
  await withFlags(JSON.parse(process.env.FLAGS_JSON || '{}'));
});

Python

# tests/harness/flag_harness.py
from openfeature.api import set_provider
from openfeature.provider.in_memory_provider import InMemoryProvider, InMemoryFlag

def with_flags(flags: dict):
    set_provider(InMemoryProvider({
        k: InMemoryFlag(default_variant='configured',
                        variants={'configured': v})
        for k, v in flags.items()
    }))

Java

import dev.openfeature.sdk.OpenFeatureAPI;
import dev.openfeature.contrib.providers.memory.InMemoryProvider;

@BeforeAll
static void wireFlags() {
    var flags = parseEnv(System.getenv("FLAGS_JSON"));  // your JSON parser
    OpenFeatureAPI.getInstance().setProvider(new InMemoryProvider(flags));
}

Evaluation API

The application code calls the standard OpenFeature evaluation API (openfeature-eval):

const client = OpenFeature.getClient();
const enabled = await client.getBooleanValue('new_checkout', false);

Per openfeature-eval: "the default value must also be specified ... In the case of any error during flag evaluation, the default value will be returned, so give consideration to your default values!" The harness picks the value the in-memory provider returns; the application's hard-coded default is what runs in prod-flag-failure scenarios.