CtrlK
BlogDocsLog inGet started
Tessl Logo

nic-structure

NIC architecture, resource processing pipeline, template systems, and key type definitions. Use when exploring the codebase, understanding data flow, debugging config generation, or working on controller logic.

67

Quality

84%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

NIC Architecture and Structure

Repository Layout

cmd/nginx-ingress/              Main binary entry point
pkg/apis/configuration/v1/
  types.go                      CRD struct definitions (source of truth)
  zz_generated.deepcopy.go      Auto-generated DeepCopy (never edit)
pkg/apis/configuration/validation/
  policy.go                     ValidatePolicy entry point
  virtualserver.go              VirtualServer/VSR validation
pkg/client/                     Auto-generated typed clients, informers, listers
internal/k8s/
  controller.go                 Informer setup, sync loop, task dispatch
  policy.go                     syncPolicy handler
  handlers.go                   Event handler factories
  configuration.go              In-memory resource state
  secrets/                      Secret store and validation
  policies/policy_refs.go       Policy reference conversion
internal/configs/
  configurator.go               Orchestrator: merge config, render, write, reload
  virtualserver.go              VirtualServer -> version2 config generation
  ingress.go                    Ingress -> version1 config generation
  transportserver.go            TransportServer -> version2 stream config generation
  policy.go                     generatePolicies() dispatcher + add*Config() methods
  annotations.go                Annotation constants + parseAnnotations()
  config_params.go              ConfigParams struct + defaults
  configmaps.go                 ConfigMap -> ConfigParams merge
  dos.go                        DoS protection config generation
  common.go                     Shared config utilities
  warnings.go                   Warning accumulation types
  validation_results.go         validationResults type (isError + warnings)
  commonhelpers/                Shared template helper functions (v1 + v2)
  oidc/                         OIDC config files (openid_connect.js, oidc_common.conf)
  njs/                          NJS scripts (apikey_auth.js)
  version1/                     Ingress template structs + .tmpl files
    __snapshots__/              Snapshot golden files
  version2/                     VirtualServer/TS template structs + .tmpl files
    __snapshots__/              Snapshot golden files
internal/nginx/                 NGINX process manager, reload, rollback, version detection
internal/metrics/               Prometheus metrics collectors and listeners
internal/telemetry/             Usage telemetry collection and export
internal/certmanager/           cert-manager integration controller
internal/externaldns/           ExternalDNS integration controller
charts/nginx-ingress/           Helm chart (values.yaml, schema, templates)
charts/tests/                   Helm snapshot tests (terratest + go-snaps)
tests/suite/                    Python integration tests (pytest)
tests/data/                     Test YAML manifests by feature
config/crd/bases/               Generated CRD YAML (from controller-gen)
deploy/                         Pre-built CRD YAML bundles (crds.yaml, crds-nap-*.yaml)
hack/                           update-codegen.sh, verify-codegen.sh

Architectural Layers

Each layer has a strict ownership boundary. Identify the correct layer before placing any change.

LayerPackage(s)Owns
Data modelpkg/apis/configuration/v1/CRD struct definitions, generated DeepCopy
Validationpkg/apis/configuration/validation/, internal/k8s/validation.goCRD field validation (kubebuilder markers), Ingress annotation validation
Controllerinternal/k8s/Event handling, in-memory state, secret resolution, sync handlers, status updates
Config generationinternal/configs/, version1/, version2/Extended resource → NGINX config struct → template render → file write
Process managementinternal/nginx/NGINX process lifecycle, reload, rollback

Layer crossing rules — violations cause architectural drift:

  • Config generation (internal/configs/) must NOT call the k8s API or access SecretStore directly — it receives pre-resolved role-qualified SecretReference via extended resources. Controller-side WAF bundle resolution and the dedicated PLM KubeClientSecretSource are explicit exceptions to the normal extended-resource flow.
  • Controller (internal/k8s/) must NOT generate NGINX config text or render templates.
  • Data model (types.go) must NOT import internal/configs or internal/k8s.
  • Validation layer must NOT trigger NGINX reloads or update k8s status.

Generated Artifacts — never hand-edit

Every entry below is produced by a command. Regenerate and commit the output after changing the source.

ArtifactSourceCommandDiffed by CI
pkg/apis/**/zz_generated.deepcopy.go, pkg/client/**pkg/apis/**/types.gomake update-codegenyes (pkg/**)
config/crd/bases/*.yamlkubebuilder markers in pkg/apis/**make update-crdsyes
deploy/crds.yaml, deploy/crds-nap-*.yamlconfig/crd/** via kustomizemake update-crdsno
docs/crd/*.mdconfig/crd/bases via hack/generate-crd-docs.gomake update-crds (runs update-crd-docs)no
charts/nginx-ingress/crdssymlink to config/crd/bases/nothing — never editn/a
internal/telemetry/*_generated.go, data.avdlData / NICResourceCounts in internal/telemetry/exporter.gomake telemetry-schemayes
internal/configs/version1/__snapshots__/**version1/*.tmpl + fixtures in template_test.gomake test-update-snapsvia unit-tests
internal/configs/version2/__snapshots__/**version2/*.tmpl + fixtures in templates_test.gomake test-update-snapsvia unit-tests
charts/tests/__snapshots__/**chart templates + charts/tests/testdata/*.yamlmake test-update-snapsvia unit-tests

Two traps:

  • verify-codegen diffs only config/crd/bases after make update-crds. Uncommitted deploy/crds*.yaml or docs/crd/ changes pass CI silently.
  • Snapshot files only re-record the existing fixtures. A template change with no matching fixture produces an empty diff and zero coverage — see nic-testing for the required sequence.

Resource Processing Pipeline

kubectl apply -f resource.yaml
  -> K8s API Server persists resource
  -> Informer detects Add/Update/Delete event
      [handlers.go: createXxxHandlers()]
  -> Event handler enqueues task onto syncQueue
      [controller.go: AddSyncQueue()]
  -> Controller dispatches task
      [controller.go: sync() -> syncVirtualServer() / syncIngress() / syncSecret() / syncPolicy() / ...]
  -> Build / update in-memory state, returning []ResourceChange
      [configuration.go: AddOrUpdateVirtualServer() / AddOrUpdateIngress()]
      Validation (CRD fields):          pkg/apis/configuration/validation/
      Validation (Ingress annotations): internal/k8s/validation.go
  -> Find affected resources (fans out when a secret or policy changes)
      [configuration.go: FindResourcesForSecret() / FindResourcesForPolicy()]
  -> Resolve secret references  <-- controller layer resolves; configurator only consumes paths
      [controller.go: createVirtualServerEx() / createIngressEx() and add*SecretRefs() -> secretStore.GetSecret(key, role)]
      On a store miss, the resolver reads the namespace informer cache.
      Valid file-backed roles are materialized under /etc/nginx/secrets on first successful resolution. 
      Later secret updates revalidate and rewrite roles that have already been resolved.
  -> Build extended resources
      [controller.go: createVirtualServerEx()   -> VirtualServerEx]
                     [createIngressEx()          -> IngressEx]
                     [createTransportServerEx()  -> TransportServerEx]
  -> Configurator generates NGINX config  [internal/configs/configurator.go: AddOrUpdateVirtualServer()]
      HTTP path:   GenerateVirtualServerConfig() [virtualserver.go]    -> version2.VirtualServerConfig
                   generateNginxCfg()            [ingress.go]          -> version1.IngressNginxConfig
      Stream path: generateTransportServerConfig(...) [transportserver.go] -> *version2.TransportServerConfig
      Policies:    generatePolicies() -> add*Config() -> policiesCfg   [policy.go]
      OSS vs Plus: Configurator.isPlus flag; Plus-only policies = OIDC, WAF
                   Template level: nginx.virtualserver.tmpl vs nginx-plus.virtualserver.tmpl
  -> Template executor renders NGINX config text
      [version1.TemplateExecutor / version2.TemplateExecutorV2;
       TransportServer uses ExecuteTransportServerTemplate(...)]
  -> NginxManager writes files + reloads NGINX
      [internal/nginx/: Manager.CreateConfig() + Manager.Reload()]
  -> Update resource status + emit Kubernetes events  [happens AFTER reload returns]
      [controller.go: updateVirtualServerStatusAndEvents() / updateIngressStatusAndEvents()]
      [k8s/status.go: statusUpdater.UpdateVirtualServerStatus()]
      Startup optimisation: status updates deferred to pendingVSStatus slices during !isNginxReady;
      flushed in background via flushPendingStatusesAsync() after first reload.

Secret Store

The secret store (internal/k8s/secrets/) is role-driven. A reference site selects a SecretRole for each secret. Kubernetes Secret.type does not determine validation, materialization, or reload behavior.

Phase 1 — reference-gated caching (syncSecret() and SecretStore.AddOrUpdateSecret()): syncSecret() finds direct references and Policy references through policySecretIndex. Referenced and special Secrets are cached; unreferenced Secrets are evicted. preSyncSecrets() temporarily primes the store during startup, and the informer-backed resolver loads newly referenced Secrets on demand.

Phase 2 — lazy role resolution (SecretStore.GetSecret()): For an existing Secret, GetSecret() validates and caches the result by (namespace/name, role). Valid file-backed roles are materialized under /etc/nginx/secrets/ using role-specific filenames. Missing lookups return an error reference with the expected path but are not cached.

Secret roles (internal/k8s/secrets/validation.go): Kubernetes Secret.type is not used for validation, so any type is accepted. The Opaque type is recommended, or kubernetes.io/tls for TLS secrets. Legacy nginx.org/* and nginx.com/* types remain accepted.

RoleRequired or Recognized KeysUsed for
RoleTLStls.crt, tls.keyTLS server certs
RoleCAca.crt; optional ca.crlCA cert (mTLS / upstream trust)
RoleJWKjwkJWT validation keys
RoleHtpasswdhtpasswdHTTP Basic auth
RoleOIDCclient-secretOIDC client secret
RoleAPIKeyClient IDs and credentialsAPI key auth
RoleLicenselicense.jwtNGINX Plus license
RoleWAFBundletoken, or username and password; optional ca.crtBundle-fetch credentials

Special secrets Default-server TLS, wildcard TLS, license, management client certificate, and management trusted CA are selected by configured references. One Secret can satisfy multiple roles; NIC validates and writes every applicable representation and performs the strongest required reload once.

Key invariant: Extended resources carry map[secrets.SecretRefKey]*secrets.SecretReference, keyed by namespaced Secret and role. Standard config generation may consume Path, CRLPath, Error, and role-specific data, but must not inspect Secret.type or call SecretStore.GetSecret().


Two Template Systems

PipelineResourcesPackageTemplates
Version 1Ingressinternal/configs/version1/nginx.ingress.tmpl, nginx-plus.ingress.tmpl
Version 2VirtualServer, VSR, TSinternal/configs/version2/nginx.virtualserver.tmpl, nginx-plus.virtualserver.tmpl
  • Version 1: IngressNginxConfig with multiple Server blocks per config
  • Version 2: VirtualServerConfig with single Server block per config
  • Main templates (nginx.tmpl, nginx-plus.tmpl) produce global nginx.conf
  • Both share generatePolicies() in internal/configs/policy.go

Policy System

Policies are mutually exclusive: each Policy CR has exactly ONE non-nil field in PolicySpec.

Types: AccessControl, RateLimit, JWTAuth, ExternalAuth, BasicAuth, IngressMTLS, EgressMTLS, OIDC, WAF, APIKey, Cache, CORS.

Application levels (VirtualServer):

  • spec.policies -- server-level (all routes unless overridden)
  • route.policies -- route-level (overrides spec-level)
  • subroute.policies -- VirtualServerRoute subroute-level

Ingress: Policies referenced via IngressEx.Policies map. Annotations are Ingress-only, never on VS/VSR.


Key Types

policiesCfg (internal/configs/policy.go): Aggregation struct holding resolved policies per context (Allow/Deny slices, RateLimit, JWTAuth, ExternalAuth, BasicAuth, IngressMTLS, EgressMTLS, OIDC, APIKey, WAF, Cache, CORSHeaders/CORSMap, Context, BundleValidator, ErrorReturn).

version2.VirtualServerConfig: Top-level struct with HTTP-level directives (Maps, LimitReqZones, CacheZones) and a single Server block.

version2.Location: Per-route struct with all policy fields (Allow, Deny, LimitReqs, JWTAuth, Cache, CORSEnabled, AddHeaders).

version1.IngressNginxConfig: Top-level Ingress struct with multiple Server blocks plus Maps, CORSHeaders, LimitReqZones.

ConfigParams (config_params.go): ~125 fields for tunable NGINX params. Flow: defaults -> ConfigMap -> Ingress annotations.

CRD Struct Pattern

// +kubebuilder:resource:shortName=pol
// +kubebuilder:subresource:status
// +kubebuilder:storageversion
type Policy struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata"`
    Spec              PolicySpec   `json:"spec"`
    Status            PolicyStatus `json:"status"`
}
  • Types: PascalCase singular. Spec/Status: <CRD>Spec, <CRD>Status. Lists: <CRD>List.
  • Short names: vs, vsr, ts, gc, pol. API group: k8s.nginx.org/v1.

Kubebuilder Markers

MarkerPurpose
+kubebuilder:validation:RequiredField must be present
+kubebuilder:validation:OptionalField is optional
+kubebuilder:validation:Pattern= `regex`Regex validation
+kubebuilder:validation:Minimum=NNumeric minimum
+kubebuilder:default=valueDefault value
+kubebuilder:validation:XValidation:rule="CEL"Cross-field CEL validation

Error Handling

  • Warnings: map[runtime.Object][]string in internal/configs/warnings.go
  • validationResults: isError bool + warnings []string. When isError = true, policy dispatcher returns ErrorReturn: {Code: 500}
  • Validation errors: Kubernetes field.ErrorList from k8s.io/apimachinery/pkg/util/validation/field
Repository
nginx/kubernetes-ingress
Last updated
First committed

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.