nic-structure

NIC architecture, resource processing pipeline, template systems, and key type definitions. Use when exploring the codebase, understanding data flow, debugging config generation, or working on controller logic.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

NIC Architecture and Structure

Name: nic-structure
Rating: 67.2 (1 reviews)
Author: nginx

Repository Layout

cmd/nginx-ingress/              Main binary entry point
pkg/apis/configuration/v1/
  types.go                      CRD struct definitions (source of truth)
  zz_generated.deepcopy.go      Auto-generated DeepCopy (never edit)
pkg/apis/configuration/validation/
  policy.go                     ValidatePolicy entry point
  virtualserver.go              VirtualServer/VSR validation
pkg/client/                     Auto-generated typed clients, informers, listers
internal/k8s/
  controller.go                 Informer setup, sync loop, task dispatch
  policy.go                     syncPolicy handler
  handlers.go                   Event handler factories
  configuration.go              In-memory resource state
  secrets/                      Secret store and validation
  policies/policy_refs.go       Policy reference conversion
internal/configs/
  configurator.go               Orchestrator: merge config, render, write, reload
  virtualserver.go              VirtualServer -> version2 config generation
  ingress.go                    Ingress -> version1 config generation
  transportserver.go            TransportServer -> version2 stream config generation
  policy.go                     generatePolicies() dispatcher + add*Config() methods
  annotations.go                Annotation constants + parseAnnotations()
  config_params.go              ConfigParams struct + defaults
  configmaps.go                 ConfigMap -> ConfigParams merge
  dos.go                        DoS protection config generation
  common.go                     Shared config utilities
  warnings.go                   Warning accumulation types
  validation_results.go         validationResults type (isError + warnings)
  commonhelpers/                Shared template helper functions (v1 + v2)
  oidc/                         OIDC config files (openid_connect.js, oidc_common.conf)
  njs/                          NJS scripts (apikey_auth.js)
  version1/                     Ingress template structs + .tmpl files
    __snapshots__/              Snapshot golden files
  version2/                     VirtualServer/TS template structs + .tmpl files
    __snapshots__/              Snapshot golden files
internal/nginx/                 NGINX process manager, reload, rollback, version detection
internal/metrics/               Prometheus metrics collectors and listeners
internal/telemetry/             Usage telemetry collection and export
internal/certmanager/           cert-manager integration controller
internal/externaldns/           ExternalDNS integration controller
charts/nginx-ingress/           Helm chart (values.yaml, schema, templates)
charts/tests/                   Helm snapshot tests (terratest + go-snaps)
tests/suite/                    Python integration tests (pytest)
tests/data/                     Test YAML manifests by feature
config/crd/bases/               Generated CRD YAML (from controller-gen)
deploy/                         Pre-built CRD YAML bundles (crds.yaml, crds-nap-*.yaml)
hack/                           update-codegen.sh, verify-codegen.sh

Architectural Layers

Each layer has a strict ownership boundary. Identify the correct layer before placing any change.

Layer	Package(s)	Owns
Data model	`pkg/apis/configuration/v1/`	CRD struct definitions, generated DeepCopy
Validation	`pkg/apis/configuration/validation/`, `internal/k8s/validation.go`	CRD field validation (kubebuilder markers), Ingress annotation validation
Controller	`internal/k8s/`	Event handling, in-memory state, secret resolution, sync handlers, status updates
Config generation	`internal/configs/`, `version1/`, `version2/`	Extended resource → NGINX config struct → template render → file write
Process management	`internal/nginx/`	NGINX process lifecycle, reload, rollback

Layer crossing rules — violations cause architectural drift:

Config generation (internal/configs/) must NOT call the k8s API or access SecretStore directly — it receives pre-resolved SecretReference{Path} via extended resources.
Controller (internal/k8s/) must NOT generate NGINX config text or render templates.
Data model (types.go) must NOT import internal/configs or internal/k8s.
Validation layer must NOT trigger NGINX reloads or update k8s status.

Resource Processing Pipeline

kubectl apply -f resource.yaml
  -> K8s API Server persists resource
  -> Informer detects Add/Update/Delete event
      [handlers.go: createXxxHandlers(); IsSupportedSecretType() gates secret events]
  -> Event handler enqueues task onto syncQueue
      [controller.go: AddSyncQueue()]
  -> Controller dispatches task
      [controller.go: sync() -> syncVirtualServer() / syncIngress() / syncSecret() / syncPolicy() / ...]
  -> Build / update in-memory state, returning []ResourceChange
      [configuration.go: AddOrUpdateVirtualServer() / AddOrUpdateIngress()]
      Validation (CRD fields):          pkg/apis/configuration/validation/
      Validation (Ingress annotations): internal/k8s/validation.go
  -> Find affected resources (fans out when a secret or policy changes)
      [configuration.go: FindResourcesForSecret() / FindResourcesForPolicy()]
  -> Resolve secret references  <-- controller layer resolves; configurator only consumes paths
      [controller.go: createVirtualServerEx() / createIngressEx() -> secretStore.GetSecret()]
      [secrets/store.go: GetSecret() lazily writes valid secret to filesystem via SecretFileManager]
  -> Build extended resources
      [controller.go: createVirtualServerEx()   -> VirtualServerEx]
                     [createIngressEx()          -> IngressEx]
                     [createTransportServerEx()  -> TransportServerEx]
  -> Configurator generates NGINX config  [internal/configs/configurator.go: AddOrUpdateVirtualServer()]
      HTTP path:   GenerateVirtualServerConfig() [virtualserver.go]    -> version2.VirtualServerConfig
                   generateNginxCfg()            [ingress.go]          -> version1.IngressNginxConfig
      Stream path: generateTransportServerConfig(...) [transportserver.go] -> *version2.TransportServerConfig
      Policies:    generatePolicies() -> add*Config() -> policiesCfg   [policy.go]
      OSS vs Plus: Configurator.isPlus flag; Plus-only policies = OIDC, WAF
                   Template level: nginx.virtualserver.tmpl vs nginx-plus.virtualserver.tmpl
  -> Template executor renders NGINX config text
      [version1.TemplateExecutor / version2.TemplateExecutorV2;
       TransportServer uses ExecuteTransportServerTemplate(...)]
  -> NginxManager writes files + reloads NGINX
      [internal/nginx/: Manager.CreateConfig() + Manager.Reload()]
  -> Update resource status + emit Kubernetes events  [happens AFTER reload returns]
      [controller.go: updateVirtualServerStatusAndEvents() / updateIngressStatusAndEvents()]
      [k8s/status.go: statusUpdater.UpdateVirtualServerStatus()]
      Startup optimisation: status updates deferred to pendingVSStatus slices during !isNginxReady;
      flushed in background via flushPendingStatusesAsync() after first reload.

Secret Store

The secret store (internal/k8s/secrets/) sits entirely in the controller layer and uses a two-phase model to avoid writing unreferenced files to the NGINX filesystem.

Phase 1 — in-memory validation (SecretStore.AddOrUpdateSecret()): Validates the secret via ValidateSecret() and stores SecretReference{Secret, Error} in memory. Does not write to disk unless a filesystem path already exists for that secret.

Phase 2 — lazy filesystem write (SecretStore.GetSecret()): On first reference during createVirtualServerEx() / createIngressEx(), materializes supported secrets under /etc/nginx/secrets/ via SecretFileManager (implemented by Configurator). Filenames are derived from <namespace>-<secretName> rather than a literal path. Some secret types create multiple files (for example, CA secrets), while OIDC and API key secrets are not written to disk, so their Path is empty. Returns SecretReference{Path, Error}.

Supported secret types (internal/k8s/secrets/validation.go):

Constant	Kubernetes type	Used for
—	`kubernetes.io/tls`	TLS server certs
`SecretTypeCA`	`nginx.org/ca`	CA cert (mTLS / upstream trust)
`SecretTypeJWK`	`nginx.org/jwk`	JWT validation keys
`SecretTypeOIDC`	`nginx.org/oidc`	OIDC client secret
`SecretTypeHtpasswd`	`nginx.org/htpasswd`	HTTP Basic auth
`SecretTypeAPIKey`	`nginx.org/apikey`	API key auth
`SecretTypeLicense`	`nginx.com/license`	NGINX Plus license

Special secrets (defaultServer TLS, wildcard TLS, license, mgmt client cert, mgmt trusted CA): handled by handleSpecialSecretUpdate() in the controller, which triggers an NGINX reload directly — independent of any resource re-sync.

Key invariant: The controller resolves secrets before they reach config generation. VirtualServerEx.SecretRefs / IngressEx.SecretRefs carry map[string]*secrets.SecretReference, and internal/configs/ may consume the pre-resolved data on those references, including .Path, .Secret.Type, and .Secret.Data. Do not add SecretStore.GetSecret() calls or any direct Kubernetes API access inside internal/configs/.

Two Template Systems

Pipeline	Resources	Package	Templates
Version 1	Ingress	`internal/configs/version1/`	`nginx.ingress.tmpl`, `nginx-plus.ingress.tmpl`
Version 2	VirtualServer, VSR, TS	`internal/configs/version2/`	`nginx.virtualserver.tmpl`, `nginx-plus.virtualserver.tmpl`

Version 1: IngressNginxConfig with multiple Server blocks per config
Version 2: VirtualServerConfig with single Server block per config
Main templates (nginx.tmpl, nginx-plus.tmpl) produce global nginx.conf
Both share generatePolicies() in internal/configs/policy.go

Policy System

Policies are mutually exclusive: each Policy CR has exactly ONE non-nil field in PolicySpec.

Types: AccessControl, RateLimit, JWTAuth, ExternalAuth, BasicAuth, IngressMTLS, EgressMTLS, OIDC, WAF, APIKey, Cache, CORS.

Application levels (VirtualServer):

spec.policies -- server-level (all routes unless overridden)
route.policies -- route-level (overrides spec-level)
subroute.policies -- VirtualServerRoute subroute-level

Ingress: Policies referenced via IngressEx.Policies map. Annotations are Ingress-only, never on VS/VSR.

Key Types

policiesCfg (internal/configs/policy.go): Aggregation struct holding resolved policies per context (Allow/Deny slices, RateLimit, JWTAuth, ExternalAuth, BasicAuth, IngressMTLS, EgressMTLS, OIDC, APIKey, WAF, Cache, CORSHeaders/CORSMap, Context, BundleValidator, ErrorReturn).

version2.VirtualServerConfig: Top-level struct with HTTP-level directives (Maps, LimitReqZones, CacheZones) and a single Server block.

version2.Location: Per-route struct with all policy fields (Allow, Deny, LimitReqs, JWTAuth, Cache, CORSEnabled, AddHeaders).

version1.IngressNginxConfig: Top-level Ingress struct with multiple Server blocks plus Maps, CORSHeaders, LimitReqZones.

ConfigParams (config_params.go): ~125 fields for tunable NGINX params. Flow: defaults -> ConfigMap -> Ingress annotations.

CRD Struct Pattern

// +kubebuilder:resource:shortName=pol
// +kubebuilder:subresource:status
// +kubebuilder:storageversion
type Policy struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata"`
    Spec              PolicySpec   `json:"spec"`
    Status            PolicyStatus `json:"status"`
}

Types: PascalCase singular. Spec/Status: <CRD>Spec, <CRD>Status. Lists: <CRD>List.
Short names: vs, vsr, ts, gc, pol. API group: k8s.nginx.org/v1.

Kubebuilder Markers

Marker	Purpose
`+kubebuilder:validation:Required`	Field must be present
`+kubebuilder:validation:Optional`	Field is optional
`+kubebuilder:validation:Pattern=` `regex`	Regex validation
`+kubebuilder:validation:Minimum=N`	Numeric minimum
`+kubebuilder:default=value`	Default value
`+kubebuilder:validation:XValidation:rule="CEL"`	Cross-field CEL validation

Error Handling

Warnings: map[runtime.Object][]string in internal/configs/warnings.go
validationResults: isError bool + warnings []string. When isError = true, policy dispatcher returns ErrorReturn: {Code: 500}
Validation errors: Kubernetes field.ErrorList from k8s.io/apimachinery/pkg/util/validation/field

Repository: nginx/kubernetes-ingress
Path: .github/skills/nic-structure/SKILL.md
Commit: 0eb3072

Last updated: 1 day ago
First committed: 3 months ago

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.