Field Set Expander

A utility for expanding field set references with order-preserving deduplication for data query systems.

Capabilities

Basic field set expansion

• Given a field list with a set reference ["basic_fields*"] and a table with a set named basic_fields containing ["field1", "field2"], the function returns ["field1", "field2"]. @test
• Given a field list with regular fields ["field1", "field2"] and no set references, the function returns the same list unchanged. @test
• Given a field list mixing set references and regular fields ["basic_set*", "field3", "field4"] where basic_set contains ["field1", "field2"], the function returns ["field1", "field2", "field3", "field4"]. @test

Field exclusion

• Given a field list ["all_fields*", "-field2"] where all_fields contains ["field1", "field2", "field3"], the function returns ["field1", "field3"] with field2 excluded. @test
• Given a field list with multiple exclusions ["all_fields*", "-field2", "-field4"] where all_fields contains ["field1", "field2", "field3", "field4"], the function returns ["field1", "field3"]. @test

Order-preserving deduplication

• Given a field list ["set1*", "set2*"] where set1 contains ["field1", "field2"] and set2 contains ["field2", "field3"], the function returns ["field1", "field2", "field3"] with field2 appearing only once. @test
• Given a field list ["field1", "set1*"] where set1 contains ["field1", "field2"], the function returns ["field1", "field2"] with field1 appearing only once. @test

Nested set expansion

• Given a field list ["outer_set*"] where outer_set contains ["inner_set*", "field3"] and inner_set contains ["field1", "field2"], the function returns ["field1", "field2", "field3"]. @test
• Given a field list with nested exclusions where outer_set contains ["inner_set*", "-field2"] and inner_set contains ["field1", "field2", "field3"], the function returns ["field1", "field3"]. @test

Error handling

• Given a field list referencing a non-existent set ["missing_set*"], the function throws an error indicating the set was not found. @test
• Given sets with circular references where set_a contains ["set_b*"] and set_b contains ["set_a*"], the function throws an error about circular references. @test
• Given a field list with more than one level of nesting where level3 contains ["level2*"], level2 contains ["level1*"], and level1 contains ["field1"], the function throws an error indicating only one level of nesting is allowed. @test

Implementation

@generates

API

/**
 * Represents a field set definition with a list of field names
 */
export interface FieldSetDefinition {
  fields: string[];
  description?: string;
}

/**
 * Represents a table with available field sets
 */
export interface Table {
  name: string;
  sets?: Record<string, FieldSetDefinition>;
}

/**
 * Expands field references that include set references (ending with *)
 * and applies exclusions (starting with -)
 *
 * Supports one level of set nesting: a set can reference another set,
 * but the referenced set cannot contain set references.
 *
 * @param fields - Array of field names, set references (ending with *), or exclusions (starting with -)
 * @param currentTable - The table containing the sets being expanded
 * @returns Expanded field list with exclusions applied and duplicates removed in order
 */
export function expandFieldsWithSets(
  fields: string[],
  currentTable: Table,
): string[];

Dependencies { .dependencies }

@lightdash/common { .dependency }

Provides the package containing field set expansion functionality for data modeling systems.

@satisfied-by