Fix Throughput Degradation in a High-Load API Gateway

Problem/Feature Description

ShipFast Logistics operates an API gateway written in Node.js that routes inbound HTTP requests to dozens of upstream microservices. Each request requires resolving the target service's hostname, reading a small routing config file from disk, and occasionally computing a request signature (PBKDF2-based HMAC) for authenticated endpoints.

Under normal load the gateway handles ~500 requests/second comfortably. But when traffic spikes to 1,500 requests/second during morning rush hours, p99 latency jumps from under 20ms to over 800ms and the team observes that DNS resolution becomes the biggest bottleneck. A monitoring probe shows that even a simple dns.lookup('localhost') is taking hundreds of milliseconds during these peaks — a classic symptom the infrastructure team recognises as resource exhaustion in a fixed-size worker pool.

Additionally, a nightly batch job reads a 200MB routing table dump from disk and loads it fully into memory to refresh the in-process cache. This job competes for the same internal resource pool as live traffic and makes the latency problem dramatically worse during the overlap window.

The team has been given the mandate to fix the gateway without replacing the Node.js runtime or switching to a different language. They need:

1. Proper runtime configuration so the internal pool is large enough for peak concurrency
2. DNS resolution rewritten to avoid the bottleneck pool entirely
3. DNS results cached so repeated calls to the same upstream don't re-resolve every time
4. The nightly routing-table reload converted to streaming so it doesn't hog pool threads for the duration of a full file read
5. Monitoring so the team can detect future pool saturation in production before users notice
6. CPU-intensive signature verification moved off the bottleneck path

Output Specification

Produce a refactored version of the gateway code in a directory called gateway/. Include:

• gateway/index.js — the refactored gateway entry point (or a clearly structured set of modules)
• gateway/start.sh — a shell script that starts the gateway with the correct environment settings
• gateway/monitor.js — a monitoring module that can be required to track pool saturation
• gateway/CHANGES.md — a brief explanation of each change made and why, including the reasoning behind the chosen pool size

The code does not need to be runnable end-to-end (no real upstream services are needed), but it must be structurally complete and demonstrate the correct patterns. Comments or inline documentation are welcome to explain the design.

Input Files

The following files are provided as inputs. Extract them before beginning.

=============== FILE: gateway-original/index.js =============== 'use strict';

const http = require('http'); const dns = require('dns'); const fs = require('fs'); const crypto = require('crypto'); const os = require('os');

// Load routing table (runs at startup and on a nightly timer) async function loadRoutingTable(path) { const data = await fs.promises.readFile(path, 'utf8'); return JSON.parse(data); }

// Resolve upstream host for a given service name async function resolveUpstream(serviceName, routingTable) { const entry = routingTable[serviceName]; if (!entry) throw new Error(Unknown service: ${serviceName});

// Resolve the hostname to an IP return new Promise((resolve, reject) => { dns.lookup(entry.host, (err, address) => { if (err) return reject(err); resolve({ address, port: entry.port }); }); }); }

// Compute request signature for authenticated endpoints function computeSignature(payload, secret) { // Deliberately using sync version — fast enough for now? return crypto.pbkdf2Sync(payload, secret, 100000, 32, 'sha512').toString('hex'); }

async function handleRequest(req, res, routingTable) { const serviceName = req.headers['x-target-service']; if (!serviceName) { res.writeHead(400); return res.end('Missing x-target-service header'); }

try { const upstream = await resolveUpstream(serviceName, routingTable); // ... forward request to upstream.address:upstream.port ... res.writeHead(200); res.end(JSON.stringify(upstream)); } catch (err) { res.writeHead(502); res.end(err.message); } }

async function main() { let routingTable = await loadRoutingTable('./routing-table.json');

// Reload every 24 hours setInterval(async () => { routingTable = await loadRoutingTable('./routing-table.json'); }, 24 * 60 * 60 * 1000);

const server = http.createServer((req, res) => { handleRequest(req, res, routingTable).catch(err => { res.writeHead(500); res.end('Internal error'); }); });

server.listen(3000, () => { console.log('Gateway listening on port 3000'); }); }

main().catch(console.error);

=============== FILE: gateway-original/routing-table-schema.md ===============

Routing Table Schema

The routing table is a JSON object where each key is a service name and the value is:

{
  "service-name": {
    "host": "service-hostname.internal",
    "port": 8080,
    "authenticated": true
  }
}

Example with ~500 services, the file can reach 150-250MB.