Validate C4 Diagrams

Validate and update the C4 architecture diagrams (AIR-wide overview in docs/diagrams/, Air Automations diagrams in automations/docs/diagrams/C4/) to match the current codebase.

Steps

1. Gather Current State from Code

Read these sources to build a picture of the actual architecture:

• automations/docs/SERVICES.md — canonical service map with ports, URLs, environments, and external dependencies
• automations/apps/*/package.json — each service's name, dependencies, and tech stack (all five are NestJS on the Fastify adapter)
• automations/apps/*/src/config/env.config.ts — each service's external dependency URLs (the source of truth for dependency edges)
• automations/apps/*/deploy/values.yaml — deployment topology: ports, probes, gatewayPrefix/httpRoute exposure, Kafka, databases
• Route files and HTTP client imports — trace service-to-service relationships (generated client imports like @concierge/*-client, controllers)
• automations/CLAUDE.md — the Architecture section for canonical service descriptions
• cloud/backend config (Config.kt, application*.conf) — Air Cloud backend dependencies (for the AIR-wide overview)
• cloud/frontend/vite.config.ts — frontend API proxies → which backends the frontend calls
• docs/ARCHITECTURE.md — the repo-level system overview (Mermaid); must stay consistent with the diagrams

Build a structured summary:

Service: <name>
  Package: @concierge/<name> | @air/<name>
  Framework: NestJS | Ktor | React/Vite
  Port: <port>
  Status: active | deferred | deprecated
  Calls: [list of services it calls]
  Called by: [list of services that call it]

2. Read All Current Diagrams

Historical diagrams live in automations/docs/diagrams/archive/ — do not validate or update them. (Frozen snapshots and pre-Agent-Spawner concepts: Q1-DP container diagram, platform primitives, Merge Agent Handler design, agent workflow, cloud-function state diagram.)

For each diagram, extract:

• All declared elements (containers, systems, components) with their labels and tech stack
• All relationships with descriptions
• All tags (READY, WIP, DEFERRED, DEPRECATED, PLANNED)
• Notes and annotations

3. Validate Each Diagram Against Code

Compare and identify discrepancies:

AIR Services Overview (docs/diagrams/air-services-overview.puml):

• Are both products' services present and correctly described (cloud/frontend, cloud/backend, the five automations apps)?
• Do dependency edges match env.config.ts (automations) and Config.kt/application.conf (cloud backend)?
• Is it consistent with the Mermaid diagram in docs/ARCHITECTURE.md? (Update both together.)

Container diagram (container-diagram.puml):

• Are all services in automations/apps/ represented as containers?
• Are tech stack labels correct?
• Are port numbers consistent with SERVICES.md?
• Are service descriptions and relationship edges still accurate? Verify every edge against code — past drift example: a Backend → Launcher "Direct launch" edge that never existed in code.
• Are workstream assignments (S1/S2/S3) still correct?

Context diagram (context-diagram.puml):

• Are all external systems still relevant?
• Are inbound trigger sources accurate (currently YouTrack + GitHub webhook controllers in Gateway; others planned)?
• Are relationship descriptions current?

Deployment diagram (deployment-diagram.puml):

• Cloud provider and clusters: AWS EKS (eks-eu-west-1 / eks-staging-eu-west-1 / eks-prod-eu-west-1), namespaces jcp-air-automations-{stgn|preprod|prod} (SERVICES.md)
• Exposure: only services with gatewayPrefix + httpRoute in deploy/values.yaml are behind the JCP API Gateway; the rest are cluster-internal
• Data services: AWS RDS shared-postgres, Kafka shared-cluster (consumer groups in values.yaml)

Triggering subsystem (triggering-subsystem-diagram.puml):

• Does the event flow match actual Gateway controllers (apps/gateway/src/modules/{automations,events}/)? Routes are automation-id based.
• Scheduler calls Launcher directly (it does not go through Gateway).
• Are not-implemented components (orange) still not implemented?

Monitoring diagram (monitoring-architecture-diagram.puml):

• Do the service-relation edges match each app's env config and generated clients (apps/*/src/config/env.config.ts, apps/*/package.json)?
• Does the instrumentation description match packages/nestjs-otel/src/sdk.ts and apps/*/src/instrumentation.ts?
• Do metrics/traces config (ports, env vars, secrets) match apps/*/deploy/values.yaml and automations/docs/monitoring-and-logging.md?

Status tags across all diagrams:

• Are DEFERRED items still deferred, or have they been implemented?
• Are WIP items still in progress, or are they now READY?
• Are DEPRECATED relationships still marked, or should they be removed?
• Are PLANNED relationships still planned, or have they been implemented?

4. Report Discrepancies

Print a structured report grouped by diagram:

## Discrepancy Report

### container-diagram.puml
| Element | Diagram Says | Code Says | Suggested Change |
|---------|-------------|-----------|-----------------|
| Gateway | "Fastify" | NestJS (package.json) | Update tech label |
| ...     | ...         | ...       | ...             |

### No issues found:
- context-diagram.puml ✓

Cite file:line evidence for every claimed discrepancy. If no discrepancies are found in any diagram, report that all diagrams are up to date.

5. Apply Updates (With User Confirmation)

Before modifying any diagram:

• Show the specific changes that will be made
• Ask the user for confirmation

When updating .puml files, preserve:

• skinparam blocks (styling) — all current diagrams use Dpi 300; keep it that way
• Lay_* directives (layout hints)
• AddElementTag / AddRelTag definitions (color coding)
• SHOW_LEGEND() and footer lines
• Comment structure and section organization
• Workstream color coding conventions (EXT=Gray, S1=Blue, S2=Green, S3=Orange)

Only change:

• Element definitions (Container, System, Person, etc.) — labels, tech stack, descriptions
• Relationship definitions (Rel, Rel_R, Rel_D, etc.) — descriptions, tags
• Status tags on elements ($tags="...")
• Notes content

Readability rules (learned the hard way — keep diagrams scannable):

• Keep edge labels to 1–3 words; put details in element descriptions or the companion doc. Graphviz reserves horizontal space for every label — long labels are the main cause of overly wide renders.
• Group shared dependencies into labeled sub-boundaries (e.g. "Used by all five services") and draw ONE consolidated edge per consumer instead of an edge per target.
• Avoid note ... of <element> when the element is inside a boundary — notes are layout nodes and distort clusters (observed swings of 6000px+). Prefer prose in the companion doc; at most one legend-like note attached to a standalone element.
• Prefer Lay_D columns over long Lay_R rows for groups of 4+ elements.
• Rel_U creates reverse rank constraints that can force clusters side by side; use plain Rel for back-edges (e.g. Kafka consumption) when stacking matters.
• System / System_Ext have NO technology parameter (only Container does). A 4th positional string is silently parsed as a sprite name and renders as literal <$...> text inside the box. Fold the technology into the description: System_Ext(alias, "Label", "tech — description", $tags=...).

C4-PlantUML macro signatures — positional args only; pass $tags/$link as named args. An extra positional string never errors — it becomes a sprite and renders as literal <$...> text in the box:

After editing, lint macro arity before rendering — count positional strings per call and flag any Person/System*/*_Boundary call with >2, or Container*/Component*/Deployment_Node/Rel* with >3:

grep -nE 'System(_Ext|Db|Queue)?\(\w+, ("[^"]*", ){2,}"' <diagram>.puml   # systems with a 3rd string
grep -nE '(Person|_Boundary)\w*\(\w+, ("[^"]*", ){2,}"' <diagram>.puml    # persons/boundaries with a 3rd string
grep -nE '(Container|Component)\w*\(\w+, ("[^"]*", ){3,}"' <diagram>.puml # containers with a 4th string
grep -nE 'Rel\w*\(\w+, \w+, ("[^"]*", ){3,}"' <diagram>.puml              # rels with a 4th string

6. Re-render PNGs

Every .puml has a rendered PNG in a sibling renders/ directory; re-render after any source change.

Standard settings: all current diagrams use Dpi 300 in their skinparam block. At that DPI the wider diagrams exceed PlantUML's default 4096px canvas limit, and PlantUML silently clips oversized output instead of erroring — always render with PLANTUML_LIMIT_SIZE=20000.

1. Preferred (no local install needed) — the official Docker image bundles the C4-PlantUML stdlib:

   cd <directory containing the .puml>
   docker run --rm -e PLANTUML_LIMIT_SIZE=20000 -v "$PWD":/data plantuml/plantuml \
     -tpng -o renders/ /data/<diagram>.puml

2. With a local CLI (brew install plantuml):

   PLANTUML_LIMIT_SIZE=20000 plantuml -tpng -o renders/ <diagram>.puml

3. Verify nothing was clipped — PlantUML silently caps an oversized render at exactly 4096 px on one axis. Flag any render that hit the cap:

   for f in renders/*.png; do
     read w h < <(sips -g pixelWidth -g pixelHeight "$f" \
       | awk '/pixelWidth/{w=$2}/pixelHeight/{h=$2}END{print w, h}')
     if [ "$w" -eq 4096 ] || [ "$h" -eq 4096 ]; then
       echo "CLIPPED (hit 4096px) → re-render with PLANTUML_LIMIT_SIZE=20000: $f (${w}x${h})"
     else
       echo "ok: $f (${w}x${h})"
     fi
   done

Key Files

• docs/diagrams/*.puml — AIR-wide overview diagram (source of truth for the whole-context visualization)
• automations/docs/diagrams/C4/*.puml — Air Automations C4 diagrams
• */renders/*.png — rendered PNG outputs (always regenerate together with source changes)
• docs/ARCHITECTURE.md — repo-level overview (Mermaid) that must stay consistent with the diagrams
• automations/docs/SERVICES.md — canonical service map with ports and URLs
• automations/apps/*/package.json, automations/apps/*/src/config/env.config.ts — service tech stacks and dependencies
• automations/CLAUDE.md — architecture section with service descriptions