tessl/npm-langchain

TypeScript framework for building LLM-powered applications with agents, tools, middleware, and model interoperability

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Middleware Guide

Name: tessl/npm-langchain
Author: tessl

This guide covers using middleware to extend agent behavior with composable, reusable components.

What is Middleware?

Middleware provides a way to add cross-cutting concerns to agents:

Human-in-the-loop approval
PII detection and redaction
Retry logic for tools and models
Rate limiting
Logging and monitoring
Summarization
Custom business logic

Using Pre-built Middleware

Human-in-the-Loop

import { createAgent, humanInTheLoopMiddleware, tool } from "langchain";
import { z } from "zod";

const sendEmail = tool(
  async ({ to, subject, body }) => {
    await emailService.send({ to, subject, body });
    return `Email sent to ${to}`;
  },
  {
    name: "send_email",
    description: "Send an email",
    schema: z.object({
      to: z.string().email(),
      subject: z.string(),
      body: z.string(),
    }),
  }
);

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [sendEmail],
  middleware: [
    humanInTheLoopMiddleware({ interruptOn: "tools" }),
  ],
  checkpointer: myCheckpointer, // Required for interrupts
});

// Agent will pause before calling tools
const result = await agent.invoke(
  { messages: [{ role: "user", content: "Send email to john@example.com" }] },
  { configurable: { thread_id: "thread-1" } }
);

// Resume after human approval
await agent.invoke(
  { messages: [] },
  { configurable: { thread_id: "thread-1", /* decision */ } }
);

PII Detection

import { createAgent, piiMiddleware } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [],
  middleware: [
    piiMiddleware({
      builtInTypes: ["email", "credit_card", "ip"],
      strategy: "redact", // or "mask", "hash", "remove"
    }),
  ],
});

Tool Retry

import { createAgent, toolRetryMiddleware } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [unreliableTool],
  middleware: [
    toolRetryMiddleware({
      maxRetries: 3,
      initialDelay: 1000,
      backoffMultiplier: 2,
      retryableErrors: [/timeout/i, /network/i],
    }),
  ],
});

Model Retry

import { createAgent, modelRetryMiddleware } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [],
  middleware: [
    modelRetryMiddleware({
      maxRetries: 3,
      initialDelay: 1000,
      maxDelay: 10000,
      backoffMultiplier: 2,
    }),
  ],
});

Summarization

import { createAgent, summarizationMiddleware } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [],
  middleware: [
    summarizationMiddleware({
      threshold: 4000, // Token threshold
      model: "openai:gpt-4o-mini", // Model for summarization
    }),
  ],
});

Tool Call Limit

import { createAgent, toolCallLimitMiddleware } from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [tool1, tool2],
  middleware: [
    toolCallLimitMiddleware({
      threadLimit: 10, // Max per thread
      runLimit: 5, // Max per run
    }),
  ],
});

Combining Middleware

import {
  createAgent,
  humanInTheLoopMiddleware,
  piiMiddleware,
  toolCallLimitMiddleware,
  toolRetryMiddleware,
} from "langchain";

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [tools],
  middleware: [
    // Order matters! Middleware executes in array order
    piiMiddleware({ builtInTypes: ["email", "credit_card"] }),
    toolCallLimitMiddleware({ runLimit: 5 }),
    toolRetryMiddleware({ maxRetries: 3 }),
    humanInTheLoopMiddleware({ interruptOn: "tools" }),
  ],
  checkpointer: myCheckpointer,
});

Creating Custom Middleware

Basic Middleware

import { createMiddleware, createAgent } from "langchain";

const loggingMiddleware = createMiddleware({
  name: "logging",

  // Wrap tool calls
  wrapToolCall: async (request, handler, runtime) => {
    console.log(`Calling tool: ${request.toolName}`);
    console.log(`Arguments:`, request.args);

    const result = await handler(request);

    console.log(`Result:`, result);
    return result;
  },
});

const agent = createAgent({
  model: "openai:gpt-4o",
  tools: [],
  middleware: [loggingMiddleware],
});

Middleware with State

import { createMiddleware } from "langchain";
import { z } from "zod";

const sessionMiddleware = createMiddleware({
  name: "session",
  stateSchema: z.object({
    sessionId: z.string(),
    startTime: z.number(),
    toolCallCount: z.number().default(0),
  }),

  beforeAgent: async (state, runtime) => {
    return {
      ...state,
      sessionId: crypto.randomUUID(),
      startTime: Date.now(),
    };
  },

  wrapToolCall: async (request, handler, runtime) => {
    const result = await handler(request);
    // State is automatically updated
    return result;
  },

  afterAgent: async (state, runtime) => {
    const duration = Date.now() - state.startTime;
    console.log(`Session ${state.sessionId} completed in ${duration}ms`);
    console.log(`Tool calls: ${state.toolCallCount}`);
    return state;
  },
});

Middleware with Context

import { createMiddleware } from "langchain";
import { z } from "zod";

const analyticsMiddleware = createMiddleware({
  name: "analytics",
  contextSchema: z.object({
    userId: z.string(),
    requestId: z.string(),
  }),

  afterModel: async (state, runtime) => {
    // Access read-only context
    const { userId, requestId } = runtime.context;

    await analytics.track({
      userId,
      requestId,
      event: "model_call_completed",
      messageCount: state.messages.length,
    });

    return state;
  },
});

// Use with context
const result = await agent.invoke(
  { messages: [...] },
  {
    context: {
      userId: "user-123",
      requestId: "req-456",
    },
  }
);

Middleware with Additional Tools

import { createMiddleware, tool } from "langchain";
import { z } from "zod";

const cacheTool = tool(
  async ({ key }) => {
    const value = await cache.get(key);
    return value || "Not found";
  },
  {
    name: "get_cache",
    description: "Get value from cache",
    schema: z.object({
      key: z.string(),
    }),
  }
);

const cacheMiddleware = createMiddleware({
  name: "cache",
  tools: [cacheTool], // Additional tools provided by middleware
});

Middleware Hooks

Available Hooks

createMiddleware({
  name: "my_middleware",

  // Before agent starts
  beforeAgent: async (state, runtime) => {
    // Modify state before agent runs
    return state;
  },

  // Before model call
  beforeModel: async (state, runtime) => {
    // Modify state before model invocation
    return state;
  },

  // After model call
  afterModel: async (state, runtime) => {
    // Process model response
    return state;
  },

  // After agent completes
  afterAgent: async (state, runtime) => {
    // Final processing
    return state;
  },

  // Wrap tool calls
  wrapToolCall: async (request, handler, runtime) => {
    // Pre-processing
    const result = await handler(request);
    // Post-processing
    return result;
  },

  // Wrap model calls
  wrapModelCall: async (state, handler, runtime) => {
    // Pre-processing
    const newState = await handler(state);
    // Post-processing
    return newState;
  },
});

Hook Execution Order

beforeAgent - Runs once at start
For each iteration:
- beforeModel - Before model call
- wrapModelCall - Around model call
- afterModel - After model call
- wrapToolCall - Around each tool call
afterAgent - Runs once at end

Advanced Patterns

Conditional Logic

const conditionalMiddleware = createMiddleware({
  name: "conditional",
  wrapToolCall: async (request, handler, runtime) => {
    // Only apply to specific tools
    if (request.toolName === "dangerous_tool") {
      // Add extra validation
      if (!validateInput(request.args)) {
        throw new Error("Invalid input");
      }
    }

    return handler(request);
  },
});

Timing and Metrics

const metricsMiddleware = createMiddleware({
  name: "metrics",
  wrapModelCall: async (state, handler, runtime) => {
    const start = Date.now();

    try {
      const result = await handler(state);
      const duration = Date.now() - start;

      await metrics.record({
        type: "model_call",
        duration,
        success: true,
      });

      return result;
    } catch (error) {
      const duration = Date.now() - start;

      await metrics.record({
        type: "model_call",
        duration,
        success: false,
        error: error.message,
      });

      throw error;
    }
  },
});

Access Control

const aclMiddleware = createMiddleware({
  name: "acl",
  contextSchema: z.object({
    userId: z.string(),
    role: z.enum(["admin", "user"]),
  }),
  wrapToolCall: async (request, handler, runtime) => {
    const { role } = runtime.context;

    // Check permissions
    const requiredRole = getToolPermission(request.toolName);
    if (!hasPermission(role, requiredRole)) {
      return "Error: Insufficient permissions";
    }

    return handler(request);
  },
});

Best Practices

Middleware Order

Place validation middleware early
Place logging middleware early for complete logs
Place retry middleware before HITL
Place HITL middleware last for final approval

State Management

Use stateSchema for persisted data
Use contextSchema for read-only per-invocation data
Keep state minimal
Clean up state in afterAgent

Error Handling

Handle errors gracefully in hooks
Don't throw unless critical
Log errors for debugging
Provide fallbacks when possible

Performance

Keep middleware logic lightweight
Avoid expensive operations in hot paths
Cache when appropriate
Use async operations efficiently

See Middleware Overview and Built-in Middleware Catalog for more details.

tessl/npm-langchain

middleware.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/guides/

Middleware Guide

What is Middleware?

Using Pre-built Middleware

Human-in-the-Loop

PII Detection

Tool Retry

Model Retry

Summarization

Tool Call Limit

Combining Middleware

Creating Custom Middleware

Basic Middleware

Middleware with State

Middleware with Context

Middleware with Additional Tools

Middleware Hooks

Available Hooks

Hook Execution Order

Advanced Patterns

Conditional Logic

Timing and Metrics

Access Control

Best Practices

Middleware Order

State Management

Error Handling

Performance

middleware.mddocs/guides/