or run

tessl search

tessl/npm-langsmith

tessl install tessl/npm-langsmith@0.4.3

TypeScript client SDK for the LangSmith LLM tracing, evaluation, and monitoring platform.

Setup Guide

Complete setup instructions for the LangSmith TypeScript SDK.

Installation

Install via npm or yarn:

npm install langsmith

yarn add langsmith

Framework-Specific Installation

# With LangChain
npm install langsmith @langchain/core @langchain/openai

# With OpenAI SDK
npm install langsmith openai

# With Anthropic SDK
npm install langsmith @anthropic-ai/sdk

# With Vercel AI SDK
npm install langsmith ai @ai-sdk/openai

Environment Configuration

Get Your API Key

Sign up at smith.langchain.com
Navigate to Settings to find your API key
Optionally create a new API key for your project

Configure Environment Variables

Set the following environment variables:

# Required
export LANGCHAIN_API_KEY="your-api-key-here"

# Optional
export LANGCHAIN_PROJECT="my-project"
export LANGCHAIN_ENDPOINT="https://api.smith.langchain.com"
export LANGCHAIN_TRACING=true

Using .env File

For development, use a .env file:

# .env
LANGCHAIN_API_KEY=lsv2_pt_...
LANGCHAIN_PROJECT=my-first-project

Load in your application:

import * as dotenv from "dotenv";
dotenv.config();

Verify Configuration

Test your setup:

import { Client } from "langsmith";

const client = new Client();

// Test API connectivity
try {
  const config = Client.getDefaultClientConfig();
  console.log("API URL:", config.apiUrl);
  console.log("API Key configured:", !!config.apiKey);

  // Try creating a test project
  const project = await client.createProject({
    projectName: "test-connection",
    description: "Testing LangSmith connection"
  });

  console.log("Connection successful! Project ID:", project.id);
} catch (error) {
  console.error("Configuration error:", error.message);
}

Client Configuration

Default Client

import { Client } from "langsmith";

// Uses environment variables
const client = new Client();

Custom Configuration

import { Client } from "langsmith";

const client = new Client({
  apiUrl: "https://api.smith.langchain.com",
  apiKey: process.env.LANGCHAIN_API_KEY,
  timeout_ms: 10000,
  autoBatchTracing: true,
  tracingSamplingRate: 1.0  // Trace 100% of requests
});

Privacy Configuration

const client = new Client({
  // Hide all inputs
  hideInputs: true,

  // Hide all outputs
  hideOutputs: true,

  // Or use functions for selective hiding
  hideInputs: (inputs) => {
    const { apiKey, password, ...safe } = inputs;
    return safe;
  }
});

Production Configuration

const client = new Client({
  apiKey: process.env.LANGSMITH_API_KEY,

  // Batch traces for better performance
  autoBatchTracing: true,
  batchSizeBytesLimit: 20_000_000,

  // Sample traces in high-volume environments
  tracingSamplingRate: 0.1,  // 10% sampling

  // Protect sensitive data
  hideInputs: (inputs) => redactPII(inputs),
  hideOutputs: false,

  // Ensure traces upload before shutdown
  blockOnRootRunFinalization: false,

  // Custom timeout
  timeout_ms: 30000
});

// Always flush before shutdown
process.on("SIGTERM", async () => {
  await client.awaitPendingTraceBatches();
  process.exit(0);
});

Utility Functions

/**
 * Get default project name from environment or return default
 * @returns Default project name
 */
function getDefaultProjectName(): string;

/**
 * Override fetch implementation globally
 * @param fetchImpl - Custom fetch implementation
 */
function overrideFetchImplementation(fetchImpl: typeof fetch): void;

/**
 * Generate UUID v7 (time-ordered UUID)
 * @returns UUID v7 string
 */
function uuid7(): string;

/**
 * Generate UUID v7 from specific timestamp
 * @param timestamp - Timestamp in milliseconds
 * @returns UUID v7 string
 */
function uuid7FromTime(timestamp: number): string;

/**
 * LangSmith SDK version constant
 */
const __version__: string;

Usage Examples

import { getDefaultProjectName, uuid7, uuid7FromTime, __version__, overrideFetchImplementation } from "langsmith";

// Get default project name
const projectName = getDefaultProjectName();
console.log("Project:", projectName);

// Generate UUIDs
const id1 = uuid7();
const id2 = uuid7FromTime(Date.now());

// Check SDK version
console.log("LangSmith SDK version:", __version__);

// Override fetch for custom networking
overrideFetchImplementation(customFetch);

Caching

/**
 * Cache class for storing API responses
 */
class Cache {
  /**
   * Create cache with configuration
   * @param config - Cache configuration
   */
  constructor(config?: CacheConfig);

  /**
   * Get value from cache
   * @param key - Cache key
   * @returns Cached value or undefined
   */
  get<T>(key: string): Promise<T | undefined>;

  /**
   * Set value in cache
   * @param key - Cache key
   * @param value - Value to cache
   * @param ttl - Time to live in milliseconds
   */
  set(key: string, value: any, ttl?: number): Promise<void>;

  /**
   * Delete value from cache
   * @param key - Cache key
   */
  delete(key: string): Promise<void>;

  /**
   * Clear all cache entries
   */
  clear(): Promise<void>;
}

/**
 * Cache configuration interface
 */
interface CacheConfig {
  /** Maximum cache size in bytes */
  maxSize?: number;
  /** Default TTL in milliseconds */
  ttl?: number;
  /** Storage backend (memory, redis, etc.) */
  backend?: "memory" | "redis";
  /** Redis connection string (if backend is redis) */
  redisUrl?: string;
}

Usage Examples

import { Cache } from "langsmith";

// Create cache with config
const cache = new Cache({
  maxSize: 100_000_000,  // 100 MB
  ttl: 3600000,          // 1 hour
  backend: "memory"
});

// Use cache
await cache.set("key1", { data: "value" });
const value = await cache.get("key1");
await cache.delete("key1");
await cache.clear();

Configuration Options Reference

interface ClientConfig {
  /** API URL (default: https://api.smith.langchain.com) */
  apiUrl?: string;

  /** API key (default: LANGCHAIN_API_KEY env var) */
  apiKey?: string;

  /** Request timeout in milliseconds (default: 120000) */
  timeout_ms?: number;

  /** Web URL for UI links (default: derived from apiUrl) */
  webUrl?: string;

  /** Custom fetch implementation */
  fetchImplementation?: typeof fetch;

  /** Enable auto-batching (default: true) */
  autoBatchTracing?: boolean;

  /** Batch size limit in bytes (default: 20MB) */
  batchSizeBytesLimit?: number;

  /** Max operations per batch */
  batchSizeLimit?: number;

  /** Max memory for batch queues (default: 1GB) */
  maxIngestMemoryBytes?: number;

  /** Concurrent batch uploads (default: 5) */
  traceBatchConcurrency?: number;

  /** Block on root run finalization (default: false) */
  blockOnRootRunFinalization?: boolean;

  /** Hide inputs from traces */
  hideInputs?: boolean | ((inputs: KVMap) => KVMap | Promise<KVMap>);

  /** Hide outputs from traces */
  hideOutputs?: boolean | ((outputs: KVMap) => KVMap | Promise<KVMap>);

  /** Custom anonymizer function */
  anonymizer?: (values: KVMap) => KVMap | Promise<KVMap>;

  /** Omit runtime info from traces (default: false) */
  omitTracedRuntimeInfo?: boolean;

  /** Workspace ID (required for org-scoped API keys) */
  workspaceId?: string;

  /** Custom fetch options for all requests */
  fetchOptions?: RequestInit;

  /** Require manual .flush() calls (default: false) */
  manualFlushMode?: boolean;

  /** Sampling rate 0-1 (default: 1.0) */
  tracingSamplingRate?: number;

  /** Enable debug logging (default: false) */
  debug?: boolean;

  /** Caching configuration */
  cache?: Cache | boolean;
}

Your First Trace

The simplest way to trace an LLM call is using the traceable() decorator.

Basic Tracing

import { traceable } from "langsmith/traceable";

// Wrap any function with traceable
const greet = traceable(
  async (name: string) => {
    return `Hello, ${name}!`;
  },
  { name: "greet-user", run_type: "chain" }
);

// Call the function - automatically traced to LangSmith
const greeting = await greet("Alice");
console.log(greeting); // "Hello, Alice!"

Tracing an LLM Call

import { traceable } from "langsmith/traceable";
import OpenAI from "openai";

const openai = new OpenAI();

const generateAnswer = traceable(
  async (question: string) => {
    const completion = await openai.chat.completions.create({
      model: "gpt-4",
      messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: question }
      ],
      temperature: 0.7,
    });

    return completion.choices[0].message.content;
  },
  { name: "generate-answer", run_type: "llm" }
);

// Execute and trace
const answer = await generateAnswer("What is the capital of France?");
console.log("Answer:", answer);

// View your trace at: https://smith.langchain.com

Nested Tracing

Create hierarchical traces automatically:

import { traceable } from "langsmith/traceable";

// Child function
const retrieveDocs = traceable(
  async (query: string) => {
    // Simulate document retrieval
    return ["Doc 1 about Paris", "Doc 2 about France"];
  },
  { name: "retrieve-docs", run_type: "retriever" }
);

// Parent function that calls child
const ragPipeline = traceable(
  async (question: string) => {
    // This call is automatically traced as a child
    const docs = await retrieveDocs(question);

    const context = docs.join("\n");
    const answer = `Based on: ${context}\nAnswer: Paris is the capital.`;

    return answer;
  },
  { name: "rag-pipeline", run_type: "chain" }
);

// Execute - creates a parent trace with child traces
const result = await ragPipeline("What is the capital of France?");

View Your Traces

After running traced functions, view them in the LangSmith UI:

Go to smith.langchain.com
Navigate to your project (e.g., "my-first-project")
Click on a trace to see:
- Input and output data
- Execution timeline
- Nested run hierarchy
- Performance metrics (latency, tokens)
- Any errors or exceptions

Your First Evaluation

Evaluation helps you systematically test your LLM application against a dataset of examples.

Create a Dataset

First, create a dataset with test examples:

import { Client } from "langsmith";

const client = new Client();

// Create a dataset
const dataset = await client.createDataset({
  datasetName: "capital-cities-qa",
  description: "Questions about capital cities",
  dataType: "kv",
});

// Add test examples
await client.createExamples({
  datasetId: dataset.id,
  inputs: [
    { question: "What is the capital of France?" },
    { question: "What is the capital of Japan?" },
    { question: "What is the capital of Brazil?" },
  ],
  outputs: [
    { answer: "Paris" },
    { answer: "Tokyo" },
    { answer: "Brasília" },
  ],
});

console.log("Dataset created:", dataset.id);

Define Your Target Function

This is the function you want to evaluate:

import { traceable } from "langsmith/traceable";
import OpenAI from "openai";

const openai = new OpenAI();

const answerQuestion = traceable(
  async (inputs: { question: string }) => {
    const completion = await openai.chat.completions.create({
      model: "gpt-3.5-turbo",
      messages: [
        { role: "system", content: "Answer questions concisely." },
        { role: "user", content: inputs.question }
      ],
      temperature: 0,
    });

    return {
      answer: completion.choices[0].message.content
    };
  },
  { name: "answer-question", run_type: "chain" }
);

Create Custom Evaluators

Define how to score your results:

// Check if the answer matches the expected output
const correctnessEvaluator = ({ run, example }) => {
  const predicted = run.outputs?.answer || "";
  const expected = example?.outputs?.answer || "";

  // Simple exact match
  const isCorrect = predicted.toLowerCase().includes(expected.toLowerCase());

  return {
    key: "correctness",
    score: isCorrect ? 1 : 0,
    comment: isCorrect ? "Correct answer" : "Incorrect answer"
  };
};

// Check response length
const lengthEvaluator = ({ run }) => {
  const answer = run.outputs?.answer || "";
  const wordCount = answer.split(" ").length;

  // Prefer concise answers (under 50 words)
  const score = wordCount <= 50 ? 1 : 0;

  return {
    key: "conciseness",
    score: score,
    value: wordCount,
    comment: `${wordCount} words`
  };
};

Run Evaluation

Run your target function against the dataset:

import { evaluate } from "langsmith/evaluation";

// Run evaluation
const results = await evaluate(answerQuestion, {
  data: "capital-cities-qa", // Dataset name
  evaluators: [correctnessEvaluator, lengthEvaluator],
  experimentPrefix: "capital-cities-eval",
  metadata: {
    model: "gpt-3.5-turbo",
    temperature: 0,
  },
});

// View results
console.log("Evaluation complete!");
console.log("Results:", results.results.length);

// Calculate aggregate scores
let correctCount = 0;
let totalConcise = 0;

for (const row of results.results) {
  const correctness = row.evaluation_results.find(e => e.key === "correctness");
  const conciseness = row.evaluation_results.find(e => e.key === "conciseness");

  if (correctness?.score === 1) correctCount++;
  if (conciseness?.score === 1) totalConcise++;
}

const accuracy = correctCount / results.results.length;
const concisenessRate = totalConcise / results.results.length;

console.log(`Accuracy: ${(accuracy * 100).toFixed(1)}%`);
console.log(`Conciseness: ${(concisenessRate * 100).toFixed(1)}%`);

View Evaluation Results

Go to smith.langchain.com
Navigate to the Datasets tab
Find your dataset ("capital-cities-qa")
View the experiment run
See detailed results including:
- Individual example results
- Aggregate metrics
- Score distributions
- Comparison with previous runs

Common Patterns

Pattern 1: Trace and Iterate

Use tracing to debug and improve your application:

import { traceable } from "langsmith/traceable";

const pipeline = traceable(
  async (input: string) => {
    // Step 1: Preprocess
    const cleaned = input.trim().toLowerCase();

    // Step 2: Process with LLM
    const result = await callLLM(cleaned);

    // Step 3: Post-process
    const final = result.toUpperCase();

    return final;
  },
  {
    name: "my-pipeline",
    run_type: "chain",
    // Add metadata for debugging
    metadata: { version: "1.0" }
  }
);

// Run and check traces in UI
await pipeline("  Hello World  ");

Pattern 2: Compare Models

Evaluate different models on the same dataset:

import { evaluate } from "langsmith/evaluation";

// Evaluator
const qualityEvaluator = ({ run, example }) => ({
  key: "quality",
  score: run.outputs?.answer === example?.outputs?.answer ? 1 : 0
});

// Evaluate GPT-3.5
const gpt35Results = await evaluate(
  (input) => answerWithModel(input, "gpt-3.5-turbo"),
  {
    data: "capital-cities-qa",
    evaluators: [qualityEvaluator],
    experimentPrefix: "gpt-3.5",
  }
);

// Evaluate GPT-4
const gpt4Results = await evaluate(
  (input) => answerWithModel(input, "gpt-4"),
  {
    data: "capital-cities-qa",
    evaluators: [qualityEvaluator],
    experimentPrefix: "gpt-4",
  }
);

// Compare results in LangSmith UI

Pattern 3: Collect User Feedback

Gather feedback from users on production traces:

import { Client } from "langsmith";
import { traceable } from "langsmith/traceable";

const client = new Client();

const chatbot = traceable(
  async (message: string) => {
    // Your chatbot logic
    const response = await generateResponse(message);
    return response;
  },
  { name: "chatbot", run_type: "chain" }
);

// Execute chatbot and get run ID
const response = await chatbot("Hello!");

// Later, collect user feedback
await client.createFeedback(response.runId, // Captured from trace context
  "user-rating", {
  score: 1, // thumbs up,
  comment: "Great response!",
});

Pattern 4: Monitor Production

Use tracing to monitor production LLM applications:

import { traceable } from "langsmith/traceable";
import { Client } from "langsmith";

const client = new Client({
  projectName: "production-chatbot",
  // Sampling: only trace 10% of requests in production
  tracingSamplingRate: 0.1,
});

const productionBot = traceable(
  async (userInput: string) => {
    try {
      const response = await processInput(userInput);
      return { success: true, response };
    } catch (error) {
      // Errors are automatically captured in traces
      return { success: false, error: error.message };
    }
  },
  {
    name: "production-bot",
    run_type: "chain",
    client: client,
    metadata: {
      environment: "production",
      version: "2.1.0"
    }
  }
);

// Runs are sampled and traced
await productionBot("User query");

Troubleshooting

Common Issues

Issue: Traces not appearing in LangSmith UI

Check that LANGCHAIN_API_KEY is set correctly
Verify project name in environment variables or client config
Ensure await client.awaitPendingTraceBatches() is called before app shutdown
Check for network connectivity to api.smith.langchain.com

Issue: Import errors with traceable

// ✓ Correct
import { traceable } from "langsmith/traceable";

// ✗ Incorrect
import { traceable } from "langsmith"; // Won't work

Issue: Traces are batched and delayed

LangSmith batches traces for performance. To ensure immediate upload:

import { Client } from "langsmith";

const client = new Client({
  autoBatchTracing: false, // Disable batching for debugging
});

// Or wait for pending batches
await client.awaitPendingTraceBatches();

Issue: Missing environment variables in production

Make sure environment variables are set in your deployment platform:

// Verify at runtime
if (!process.env.LANGCHAIN_API_KEY) {
  console.warn("LANGCHAIN_API_KEY not set - tracing disabled");
}

Getting Help

Documentation: smith.langchain.com/docs
GitHub Issues: github.com/langchain-ai/langsmith-sdk
Discord Community: LangChain Discord

Complete Example

Here's a complete working example combining tracing and evaluation:

import { traceable } from "langsmith/traceable";
import { Client, evaluate } from "langsmith";
import OpenAI from "openai";
import * as dotenv from "dotenv";

// Load environment
dotenv.config();

const client = new Client();
const openai = new OpenAI();

// 1. Define your application
const qaBot = traceable(
  async (inputs: { question: string }) => {
    const completion = await openai.chat.completions.create({
      model: "gpt-3.5-turbo",
      messages: [{ role: "user", content: inputs.question }],
      temperature: 0,
    });

    return {
      answer: completion.choices[0].message.content
    };
  },
  { name: "qa-bot", run_type: "chain" }
);

// 2. Create test dataset
const dataset = await client.createDataset({
  datasetName: "qa-test-set",
  description: "Test questions for QA bot",
});

await client.createExamples({
  datasetId: dataset.id,
  inputs: [
    { question: "What is 2+2?" },
    { question: "What is the capital of Spain?" },
  ],
  outputs: [
    { answer: "4" },
    { answer: "Madrid" },
  ],
});

// 3. Create evaluator
const correctnessEval = ({ run, example }) => {
  const predicted = run.outputs?.answer || "";
  const expected = example?.outputs?.answer || "";
  return {
    key: "correctness",
    score: predicted.includes(expected) ? 1 : 0
  };
};

// 4. Run evaluation
const results = await evaluate(qaBot, {
  data: "qa-test-set",
  evaluators: [correctnessEval],
  experimentPrefix: "qa-bot-v1",
});

// 5. View results
console.log(`Evaluated ${results.results.length} examples`);
const accuracy = results.results.filter(r =>
  r.evaluation_results.find(e => e.key === "correctness")?.score === 1
).length / results.results.length;

console.log(`Accuracy: ${(accuracy * 100).toFixed(1)}%`);
console.log("View detailed results at: https://smith.langchain.com");

// 6. Cleanup - ensure traces are uploaded
await client.awaitPendingTraceBatches();

Run this example:

node --loader ts-node/esm example.ts

Next Steps

Quick Reference - Common patterns and code snippets
Core Concepts - Understanding projects, runs, and datasets
Tracing Guide - Start tracing your LLM applications

Version

tessl/npm-langsmith

setup.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/guides/

Setup Guide

Installation

Framework-Specific Installation

Environment Configuration

Get Your API Key

Configure Environment Variables

Using .env File

Verify Configuration

Client Configuration

Default Client

Custom Configuration

Privacy Configuration

Production Configuration

Utility Functions

Usage Examples

Caching

Usage Examples

Configuration Options Reference

Your First Trace

Basic Tracing

Tracing an LLM Call

Nested Tracing

View Your Traces

Your First Evaluation

Create a Dataset

Define Your Target Function

Create Custom Evaluators

Run Evaluation

View Evaluation Results

Common Patterns

Pattern 1: Trace and Iterate

Pattern 2: Compare Models

Pattern 3: Collect User Feedback

Pattern 4: Monitor Production

Troubleshooting

Common Issues

Getting Help

Complete Example

Next Steps

setup.mddocs/guides/