or run

npx @tessl/cli init

Datasets and Examples

LangSmith Datasets are collections of examples used for testing, evaluation, and fine-tuning. Each dataset contains examples with inputs, optional outputs, and metadata. Datasets support versioning, sharing, and can be exported for various use cases including OpenAI fine-tuning.

Core Imports

import { Client } from "langsmith";
import type {
  Dataset,
  Example,
  ExampleCreate,
  ExampleUpdate,
  DatasetVersion,
  DatasetDiffInfo,
} from "langsmith/schemas";

Basic Usage

import { Client } from "langsmith";

const client = new Client();

// Create a dataset
const dataset = await client.createDataset({
  name: "customer-support-qa",
  description: "Q&A pairs for customer support",
  dataType: "kv",
});

// Add examples to the dataset
await client.createExamples({
  inputs: [
    { question: "How do I reset my password?" },
    { question: "What are your business hours?" },
  ],
  outputs: [
    { answer: "Click 'Forgot Password' on the login page." },
    { answer: "We're open Monday-Friday, 9am-5pm EST." },
  ],
  datasetId: dataset.id,
});

// List all examples in the dataset
for await (const example of client.listExamples({
  datasetId: dataset.id,
})) {
  console.log(example.inputs, example.outputs);
}

Capabilities

Dataset Management

Create, read, update, and delete datasets with full CRUD operations.

/**
 * Create a new dataset
 * @param params - Dataset creation parameters
 * @returns Created dataset
 */
createDataset(params: CreateDatasetParams): Promise<Dataset>;

interface CreateDatasetParams {
  /** Dataset name (required) */
  name: string;
  /** Dataset description */
  description?: string;
  /** Data type: "kv" (key-value), "llm" (LLM format), or "chat" (chat format) */
  dataType?: DataType;
  /** Input keys schema */
  inputsSchema?: Record<string, any>;
  /** Output keys schema */
  outputsSchema?: Record<string, any>;
  /** Dataset ID (optional, auto-generated if not provided) */
  id?: string;
}

/**
 * Read a dataset by ID or name
 * @param params - Dataset identifier
 * @returns Dataset object
 */
readDataset(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<Dataset>;

/**
 * List all datasets with optional filtering
 * @param params - List parameters
 * @returns Async iterable of datasets
 */
listDatasets(params?: ListDatasetsParams): AsyncIterable<Dataset>;

interface ListDatasetsParams {
  /** Filter by dataset name */
  datasetName?: string;
  /** Filter by data type */
  dataType?: DataType;
  /** Limit number of results */
  limit?: number;
  /** Offset for pagination */
  offset?: number;
}

/**
 * Delete a dataset
 * @param params - Dataset identifier
 */
deleteDataset(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<void>;

/**
 * Update a dataset's properties
 * @param props - Dataset identifier and properties to update
 * @returns Promise resolving to updated dataset
 */
updateDataset(props: {
  datasetId?: string;
  datasetName?: string;
  name?: string;
  description?: string;
}): Promise<Dataset>;

/**
 * Check if a dataset exists
 * @param params - Dataset identifier
 * @returns Promise resolving to boolean
 */
hasDataset(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<boolean>;

/**
 * Update dataset tags
 * @param params - Dataset and tag update parameters
 * @returns Promise resolving to updated dataset
 */
updateDatasetTag(params: {
  datasetId?: string;
  datasetName?: string;
  tag: string;
}): Promise<Dataset>;

/**
 * Upload CSV file as a dataset
 * @param params - CSV upload parameters
 * @returns Promise resolving to created dataset
 */
uploadCsv(params: {
  csvFile: Blob;
  fileName: string;
  inputKeys: string[];
  outputKeys: string[];
  name?: string;
  description?: string;
  dataType?: DataType;
}): Promise<Dataset>;

/**
 * Index a dataset for search
 * @param params - Dataset identifier
 * @returns Promise resolving when indexing completes
 */
indexDataset(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<void>;

Usage Examples:

import { Client } from "langsmith";

const client = new Client();

// Create a dataset with schema
const dataset = await client.createDataset({
  name: "translation-eval",
  description: "Translation quality evaluation dataset",
  dataType: "kv",
  inputsSchema: {
    type: "object",
    properties: {
      text: { type: "string" },
      source_lang: { type: "string" },
      target_lang: { type: "string" },
    },
  },
  outputsSchema: {
    type: "object",
    properties: {
      translation: { type: "string" },
    },
  },
});

// Read by name
const ds = await client.readDataset({
  datasetName: "translation-eval",
});

// List all datasets
for await (const dataset of client.listDatasets()) {
  console.log(dataset.name, dataset.example_count);
}

// Delete dataset
await client.deleteDataset({
  datasetId: dataset.id,
});

Examples Management

Dataset Cloning

Create a copy of an existing dataset with all its examples, or clone a publicly shared dataset.

/**
 * Clone an existing dataset
 * @param params - Clone parameters
 * @returns New cloned dataset
 */
cloneDataset(params: CloneDatasetParams): Promise<Dataset>;

/**
 * Clone a publicly shared dataset to your tenant
 * @param tokenOrUrl - Share token or full share URL
 * @param options - Optional clone options
 * @returns Promise resolving when clone completes
 */
clonePublicDataset(
  tokenOrUrl: string,
  options?: {
    sourceApiUrl?: string;
    datasetName?: string;
  }
): Promise<void>;

interface CloneDatasetParams {
  /** Source dataset ID (required if not using dataset name) */
  sourceDatasetId?: string;
  /** Source dataset name (required if not using dataset ID) */
  sourceDatasetName?: string;
  /** New dataset name (optional, defaults to source name + suffix) */
  targetDatasetName?: string;
  /** Target dataset ID (optional, auto-generated if not provided) */
  targetDatasetId?: string;
  /** Target dataset description */
  targetDatasetDescription?: string;
  /** Example IDs to clone (optional, clones all if not specified) */
  exampleIds?: string[];
  /** Tags to filter source examples (optional) */
  tags?: string[];
}

Usage Examples:

import { Client } from "langsmith";

const client = new Client();

// Clone entire dataset
const clonedDataset = await client.cloneDataset({
  sourceDatasetName: "prod-eval-dataset",
  targetDatasetName: "staging-eval-dataset",
  targetDatasetDescription: "Staging copy of production dataset",
});

// Clone specific examples
const partialClone = await client.cloneDataset({
  sourceDatasetId: "abc123",
  targetDatasetName: "subset-dataset",
  exampleIds: ["ex1", "ex2", "ex3"],
});

// Clone examples with specific tags
const taggedClone = await client.cloneDataset({
  sourceDatasetName: "full-dataset",
  targetDatasetName: "edge-cases",
  tags: ["edge-case", "high-priority"],
});

// Clone a publicly shared dataset
await client.clonePublicDataset("https://smith.langchain.com/public/abc123/d", {
  datasetName: "cloned-public-dataset",
});

// Or use just the share token
await client.clonePublicDataset("share_token_abc123");

Dataset Sharing

Share datasets publicly via share tokens for collaboration.

/**
 * Share a dataset publicly
 * @param params - Share parameters
 * @returns Dataset share schema with share token
 */
shareDataset(params: ShareDatasetParams): Promise<DatasetShareSchema>;

interface ShareDatasetParams {
  /** Dataset ID to share */
  datasetId: string;
}

/**
 * Unshare a dataset
 * @param params - Unshare parameters
 */
unshareDataset(params: UnshareDatasetParams): Promise<void>;

interface UnshareDatasetParams {
  /** Dataset ID to unshare */
  datasetId: string;
}

/**
 * Read a shared dataset using share token
 * @param shareToken - Share token string
 * @returns Dataset object
 */
readSharedDataset(shareToken: string): Promise<Dataset>;

/**
 * List examples from a shared dataset
 * @param shareToken - Share token string
 * @param params - Optional list parameters
 * @returns Async iterable of examples
 */
listSharedExamples(
  shareToken: string,
  params?: { limit?: number; offset?: number }
): AsyncIterable<Example>;

/**
 * Read schema information from a shared dataset
 * @param shareToken - Share token string
 * @returns Dataset schema
 */
readDatasetSharedSchema(shareToken: string): Promise<{
  inputs_schema?: Record<string, any>;
  outputs_schema?: Record<string, any>;
}>;

Usage Examples:

import { Client } from "langsmith";

const client = new Client();

// Share a dataset
const shareInfo = await client.shareDataset({
  datasetId: "dataset-123",
});
console.log("Share URL:", shareInfo.url);
console.log("Share token:", shareInfo.share_token);

// Read shared dataset (no auth required)
const sharedDataset = await client.readSharedDataset(
  "share_token_abc123xyz"
);

// Unshare dataset
await client.unshareDataset({
  datasetId: "dataset-123",
});

Dataset Versioning

Create and manage dataset versions for tracking changes over time.

/**
 * Create a new dataset version snapshot
 * @param params - Version creation parameters
 * @returns Created dataset version
 */
createDatasetVersion(
  params: CreateDatasetVersionParams
): Promise<DatasetVersion>;

interface CreateDatasetVersionParams {
  /** Dataset ID or name */
  datasetId?: string;
  datasetName?: string;
  /** Version name/tag */
  name?: string;
  /** Version description */
  description?: string;
  /** Version tags */
  tags?: string[];
  /** Example IDs to include in this version (optional, includes all if not specified) */
  exampleIds?: string[];
}

/**
 * Update an existing dataset version
 * @param params - Version update parameters
 * @returns Updated dataset version
 */
updateDatasetVersion(
  params: UpdateDatasetVersionParams
): Promise<DatasetVersion>;

interface UpdateDatasetVersionParams {
  /** Dataset ID or name */
  datasetId?: string;
  datasetName?: string;
  /** Version ID or name to update */
  versionId?: string;
  versionName?: string;
  /** New version name */
  name?: string;
  /** New description */
  description?: string;
  /** New tags */
  tags?: string[];
}

/**
 * Read a specific dataset version
 * @param params - Version identifier
 * @returns Promise resolving to dataset version
 */
readDatasetVersion(params: {
  datasetId?: string;
  datasetName?: string;
  versionId?: string;
  versionName?: string;
}): Promise<DatasetVersion>;

/**
 * List dataset splits (train/test/val)
 * @param params - Dataset identifier
 * @returns Promise resolving to array of split names
 */
listDatasetSplits(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<string[]>;

/**
 * Update dataset splits assignments
 * @param params - Split update parameters
 * @returns Promise resolving when update completes
 */
updateDatasetSplits(params: {
  datasetId?: string;
  datasetName?: string;
  splits: Record<string, string[]>; // { 'train': ['ex1', 'ex2'], 'test': ['ex3'] }
}): Promise<void>;

/**
 * Compare two dataset versions
 * @param params - Diff parameters
 * @returns Dataset diff information
 */
diffDatasetVersions(
  params: DiffDatasetVersionsParams
): Promise<DatasetDiffInfo>;

interface DiffDatasetVersionsParams {
  /** Dataset ID or name */
  datasetId?: string;
  datasetName?: string;
  /** Source version ID or name */
  fromVersion: string;
  /** Target version ID or name */
  toVersion: string;
}

Usage Examples:

import { Client } from "langsmith";

const client = new Client();

// Create a version snapshot
const version = await client.createDatasetVersion({
  datasetName: "qa-dataset",
  name: "v1.0.0",
  description: "Initial release version",
  tags: ["production", "release"],
});

// Update version metadata
await client.updateDatasetVersion({
  datasetId: "dataset-123",
  versionName: "v1.0.0",
  description: "Production release - validated and approved",
  tags: ["production", "release", "validated"],
});

// Compare two versions
const diff = await client.diffDatasetVersions({
  datasetName: "qa-dataset",
  fromVersion: "v1.0.0",
  toVersion: "v1.1.0",
});

console.log("Examples added:", diff.examples_added.length);
console.log("Examples removed:", diff.examples_removed.length);
console.log("Examples modified:", diff.examples_modified.length);

Example Management

Create, update, and manage individual examples within datasets.

/**
 * Create a single example
 * @param example - Example creation data
 * @returns Created example
 */
createExample(example: ExampleCreate): Promise<Example>;

interface ExampleCreate {
  /** Dataset ID */
  datasetId: string;
  /** Input data */
  inputs: Record<string, any>;
  /** Output data (optional) */
  outputs?: Record<string, any>;
  /** Example metadata (optional) */
  metadata?: Record<string, any>;
  /** Example ID (optional, auto-generated if not provided) */
  id?: string;
  /** Split identifier (e.g., "train", "test", "validation") */
  split?: string | string[];
  /** Source run ID this example was created from (optional) */
  sourceRunId?: string;
}

/**
 * Bulk create multiple examples
 * @param params - Bulk creation parameters
 * @returns Array of created examples
 */
createExamples(params: CreateExamplesParams): Promise<Example[]>;

interface CreateExamplesParams {
  /** Dataset ID or name */
  datasetId?: string;
  datasetName?: string;
  /** Array of input objects */
  inputs: Record<string, any>[];
  /** Array of output objects (optional, must match inputs length) */
  outputs?: Record<string, any>[];
  /** Array of metadata objects (optional) */
  metadata?: Record<string, any>[];
  /** Array of example IDs (optional) */
  ids?: string[];
  /** Array of split identifiers (optional) */
  splits?: (string | string[])[];
  /** Array of source run IDs (optional) */
  sourceRunIds?: string[];
}

/**
 * Update a single example
 * @param example - Example update data
 * @returns Updated example
 */
updateExample(example: ExampleUpdate): Promise<Example>;

interface ExampleUpdate {
  /** Example ID to update */
  exampleId: string;
  /** Updated input data (optional) */
  inputs?: Record<string, any>;
  /** Updated output data (optional) */
  outputs?: Record<string, any>;
  /** Updated metadata (optional) */
  metadata?: Record<string, any>;
  /** Updated split (optional) */
  split?: string | string[];
  /** Dataset ID (optional) */
  datasetId?: string;
}

/**
 * Bulk update multiple examples
 * @param update - Array of example updates
 * @returns Promise resolving to update response object
 */
updateExamples(update: ExampleUpdate[]): Promise<object>;

/**
 * Read a single example
 * @param exampleId - Example ID
 * @returns Example object
 */
readExample(exampleId: string): Promise<Example>;

/**
 * List examples with filtering
 * @param params - List parameters
 * @returns Async iterable of examples
 */
listExamples(params?: ListExamplesParams): AsyncIterable<Example>;

interface ListExamplesParams {
  /** Filter by dataset ID or name */
  datasetId?: string;
  datasetName?: string;
  /** Filter by example IDs */
  exampleIds?: string[];
  /** Filter by splits */
  splits?: string[];
  /** Filter by metadata */
  metadata?: Record<string, any>;
  /** Limit number of results */
  limit?: number;
  /** Offset for pagination */
  offset?: number;
  /** Include run information */
  includeRuns?: boolean;
}

/**
 * Delete an example
 * @param exampleId - Example ID to delete
 */
deleteExample(exampleId: string): Promise<void>;

/**
 * Delete multiple examples at once
 * @param exampleIds - Array of example IDs to delete
 * @param options - Optional deletion options
 */
deleteExamples(
  exampleIds: string[],
  options?: { hardDelete?: boolean }
): Promise<void>;

/**
 * Create an LLM format example (for completion-style datasets)
 * @param input - Prompt/input text
 * @param generation - Expected completion/output text
 * @param options - Additional options
 * @returns Created example
 */
createLLMExample(
  input: string,
  generation: string | undefined,
  options: CreateExampleOptions
): Promise<Example>;

/**
 * Create a chat format example (for chat-style datasets)
 * @param inputs - Array of chat messages (input)
 * @param generations - Array of chat messages (output)
 * @param options - Additional options
 * @returns Created example
 */
createChatExample(
  inputs: Array<{ role: string; content: string }>,
  generations: Array<{ role: string; content: string }> | undefined,
  options: CreateExampleOptions
): Promise<Example>;

/**
 * Find similar examples using embeddings
 * @param inputs - Input to find similar examples for
 * @param datasetId - Dataset ID to search within
 * @param limit - Maximum number of results to return
 * @param options - Optional filter parameters
 * @returns Promise resolving to array of similar examples with scores
 */
similarExamples(
  inputs: Record<string, any>,
  datasetId: string,
  limit: number,
  options?: { filter?: string }
): Promise<Array<Example & { score?: number }>>;

/**
 * Upload examples using multipart (for large batches)
 * @param params - Multipart upload parameters
 * @returns Promise resolving to upload response
 */
uploadExamplesMultipart(params: {
  datasetId?: string;
  datasetName?: string;
  examples: Array<{
    inputs: Record<string, any>;
    outputs?: Record<string, any>;
    metadata?: Record<string, any>;
  }>;
}): Promise<UploadExamplesResponse>;

/**
 * Update examples using multipart (for large batches)
 * @param params - Multipart update parameters
 * @returns Promise resolving to update response
 */
updateExamplesMultipart(params: {
  exampleIds: string[];
  patches: Array<{
    inputs?: Record<string, any>;
    outputs?: Record<string, any>;
    metadata?: Record<string, any>;
  }>;
}): Promise<UpdateExamplesResponse>;

Usage Examples:

import { Client } from "langsmith";

const client = new Client();

// Create a single example
const example = await client.createExample({
  datasetId: "dataset-123",
  inputs: { question: "What is the capital of France?" },
  outputs: { answer: "Paris" },
  metadata: { category: "geography", difficulty: "easy" },
  split: "train",
});

// Bulk create examples
await client.createExamples({
  datasetName: "qa-dataset",
  inputs: [
    { question: "What is 2+2?" },
    { question: "What is the speed of light?" },
  ],
  outputs: [{ answer: "4" }, { answer: "299,792,458 m/s" }],
  metadata: [
    { category: "math", difficulty: "easy" },
    { category: "physics", difficulty: "medium" },
  ],
  splits: ["train", "test"],
});

// Update an example
await client.updateExample({
  exampleId: example.id,
  outputs: { answer: "Paris, France" },
  metadata: { ...example.metadata, verified: true },
});

// Read an example
const retrieved = await client.readExample(example.id);

// List examples with filters
for await (const ex of client.listExamples({
  datasetName: "qa-dataset",
  splits: ["train"],
  limit: 100,
})) {
  console.log(ex.inputs, ex.outputs);
}

// Bulk update examples
await client.updateExamples({
  exampleIds: ["ex1", "ex2", "ex3"],
  metadata: [
    { verified: true },
    { verified: true },
    { verified: false },
  ],
});

// Delete an example
await client.deleteExample(example.id);

OpenAI Fine-tuning Export

Export datasets in OpenAI fine-tuning format for model training.

/**
 * Export dataset in OpenAI fine-tuning format (JSONL)
 * @param params - Export parameters
 * @returns Array of formatted training examples
 */
readDatasetOpenaiFinetuning(params: {
  datasetId?: string;
  datasetName?: string;
}): Promise<Array<Record<string, any>>>;

Usage Examples:

import { Client } from "langsmith";
import { writeFile } from "fs/promises";

const client = new Client();

// Export dataset for OpenAI fine-tuning
const finetuningData = await client.readDatasetOpenaiFinetuning({
  datasetName: "chat-training-data",
});

// Save to JSONL file
const jsonl = finetuningData.map((item) => JSON.stringify(item)).join("\n");
await writeFile("training_data.jsonl", jsonl);

// Each item follows OpenAI's format:
// {
//   "messages": [
//     { "role": "system", "content": "..." },
//     { "role": "user", "content": "..." },
//     { "role": "assistant", "content": "..." }
//   ]
// }

Types and Interfaces

Dataset Schema

Complete dataset object structure.

interface Dataset {
  /** Dataset ID */
  id: string;
  /** Dataset name */
  name: string;
  /** Dataset description */
  description?: string;
  /** Creation timestamp */
  created_at: string;
  /** Data type */
  data_type?: DataType;
  /** Number of examples in dataset */
  example_count?: number;
  /** Input keys schema */
  inputs_schema?: Record<string, any>;
  /** Output keys schema */
  outputs_schema?: Record<string, any>;
  /** Last session ID that used this dataset */
  last_session_id?: string;
  /** Tenant ID */
  tenant_id?: string;
  /** Modified timestamp */
  modified_at?: string;
}

type DataType = "kv" | "llm" | "chat";

Example Schema

Complete example object structure.

interface Example {
  /** Example ID */
  id: string;
  /** Dataset ID */
  dataset_id: string;
  /** Input data */
  inputs: Record<string, any>;
  /** Output data */
  outputs?: Record<string, any>;
  /** Example metadata */
  metadata?: Record<string, any>;
  /** Creation timestamp */
  created_at: string;
  /** Modified timestamp */
  modified_at?: string;
  /** Split identifiers */
  split?: string | string[];
  /** Source run ID */
  source_run_id?: string;
  /** Associated runs */
  runs?: Run[];
}

Dataset Version Schema

Dataset version information for tracking changes.

interface DatasetVersion {
  /** Version ID */
  id: string;
  /** Dataset ID */
  dataset_id: string;
  /** Version name/tag */
  name?: string;
  /** Version description */
  description?: string;
  /** Creation timestamp */
  created_at: string;
  /** Version tags */
  tags?: string[];
  /** Example IDs included in this version */
  example_ids?: string[];
}

Dataset Share Schema

Dataset sharing configuration and tokens.

interface DatasetShareSchema {
  /** Dataset ID */
  dataset_id: string;
  /** Share token for public access */
  share_token: string;
  /** Public share URL */
  url: string;
}

Dataset Diff Schema

Information about differences between dataset versions.

interface DatasetDiffInfo {
  /** Examples added in target version */
  examples_added: Example[];
  /** Examples removed in target version */
  examples_removed: Example[];
  /** Examples modified between versions */
  examples_modified: Array<{
    /** Example in source version */
    before: Example;
    /** Example in target version */
    after: Example;
  }>;
}

Advanced Patterns

Dataset Workflow

Complete workflow for managing dataset lifecycle.

import { Client } from "langsmith";

const client = new Client();

// 1. Create dataset
const dataset = await client.createDataset({
  name: "customer-feedback-v1",
  description: "Customer feedback classification dataset",
  dataType: "kv",
});

// 2. Add examples in batches
const feedbackData = [
  {
    inputs: { text: "Great product, very satisfied!" },
    outputs: { sentiment: "positive" },
  },
  {
    inputs: { text: "Terrible experience, would not recommend." },
    outputs: { sentiment: "negative" },
  },
  // ... more examples
];

await client.createExamples({
  datasetId: dataset.id,
  inputs: feedbackData.map((d) => d.inputs),
  outputs: feedbackData.map((d) => d.outputs),
});

// 3. Create version snapshot
const v1 = await client.createDatasetVersion({
  datasetId: dataset.id,
  name: "v1.0",
  description: "Initial dataset version",
});

// 4. Update examples
await client.updateExamples({
  exampleIds: ["ex1", "ex2"],
  metadata: [{ reviewed: true }, { reviewed: true }],
});

// 5. Create new version
const v2 = await client.createDatasetVersion({
  datasetId: dataset.id,
  name: "v1.1",
  description: "Updated with reviewed examples",
});

// 6. Compare versions
const diff = await client.diffDatasetVersions({
  datasetId: dataset.id,
  fromVersion: "v1.0",
  toVersion: "v1.1",
});

console.log(`Modified examples: ${diff.examples_modified.length}`);

// 7. Share dataset
const shareInfo = await client.shareDataset({
  datasetId: dataset.id,
});
console.log(`Share at: ${shareInfo.url}`);

Converting Runs to Examples

Create datasets from production traces.

import { Client } from "langsmith";

const client = new Client();

// Create dataset for storing examples
const dataset = await client.createDataset({
  name: "production-traces-2024",
  description: "High-quality production runs converted to examples",
});

// Fetch runs with specific criteria
const runs = [];
for await (const run of client.listRuns({
  projectName: "production-chat",
  filter: 'eq(feedback.score, 1) and eq(feedback.key, "quality")',
  limit: 100,
})) {
  runs.push(run);
}

// Convert runs to examples
await client.createExamples({
  datasetId: dataset.id,
  inputs: runs.map((run) => run.inputs),
  outputs: runs.map((run) => run.outputs),
  metadata: runs.map((run) => ({
    source: "production",
    originalRunId: run.id,
    feedback: run.feedback_stats,
  })),
  sourceRunIds: runs.map((run) => run.id),
});

console.log(`Created ${runs.length} examples from production runs`);

Dataset Splits for Train/Test/Val

Organize examples into training splits.

import { Client } from "langsmith";

const client = new Client();

const dataset = await client.createDataset({
  name: "ml-training-data",
  dataType: "kv",
});

// Add examples with different splits
const allData = loadYourData(); // Your data loading function

// Split data 70/15/15
const trainSize = Math.floor(allData.length * 0.7);
const valSize = Math.floor(allData.length * 0.15);

const trainData = allData.slice(0, trainSize);
const valData = allData.slice(trainSize, trainSize + valSize);
const testData = allData.slice(trainSize + valSize);

// Create examples with split labels
await client.createExamples({
  datasetId: dataset.id,
  inputs: trainData.map((d) => d.inputs),
  outputs: trainData.map((d) => d.outputs),
  splits: trainData.map(() => "train"),
});

await client.createExamples({
  datasetId: dataset.id,
  inputs: valData.map((d) => d.inputs),
  outputs: valData.map((d) => d.outputs),
  splits: valData.map(() => "validation"),
});

await client.createExamples({
  datasetId: dataset.id,
  inputs: testData.map((d) => d.inputs),
  outputs: testData.map((d) => d.outputs),
  splits: testData.map(() => "test"),
});

// Query specific splits
for await (const example of client.listExamples({
  datasetId: dataset.id,
  splits: ["train"],
})) {
  // Train model on training examples
}

Metadata-based Filtering

Use metadata for organizing and querying examples.

import { Client } from "langsmith";

const client = new Client();

// Create examples with rich metadata
await client.createExamples({
  datasetName: "multi-domain-qa",
  inputs: [
    { question: "What is photosynthesis?" },
    { question: "Explain quantum entanglement" },
  ],
  outputs: [{ answer: "..." }, { answer: "..." }],
  metadata: [
    {
      domain: "biology",
      difficulty: "medium",
      source: "textbook",
      verified: true,
      tags: ["science", "education"],
    },
    {
      domain: "physics",
      difficulty: "hard",
      source: "research_paper",
      verified: true,
      tags: ["science", "quantum"],
    },
  ],
});

// Query examples by metadata
for await (const example of client.listExamples({
  datasetName: "multi-domain-qa",
  metadata: { domain: "biology", verified: true },
})) {
  console.log(example.inputs.question);
}

Integration with Evaluation

Datasets are primarily used for evaluation workflows.

import { Client, evaluate } from "langsmith";

const client = new Client();

// Target function to evaluate
async function myLLMChain(inputs: { question: string }) {
  // Your LLM logic here
  return { answer: "..." };
}

// Run evaluation against dataset
const results = await evaluate(myLLMChain, {
  data: "qa-dataset", // Dataset name
  evaluators: [
    // Your evaluators
  ],
  experimentPrefix: "qa-eval",
});

console.log("Evaluation complete:", results);

Evaluation Documentation

Version

Tile

Files

datasets.mddocs/

Datasets and Examples

Core Imports

Basic Usage

Capabilities

Dataset Management

Dataset Cloning

Dataset Sharing

Dataset Versioning

Example Management

OpenAI Fine-tuning Export

Types and Interfaces

Dataset Schema

Example Schema

Dataset Version Schema

Dataset Share Schema

Dataset Diff Schema

Advanced Patterns

Dataset Workflow

Converting Runs to Examples

Dataset Splits for Train/Test/Val

Metadata-based Filtering

Integration with Evaluation

See Also

Version

Tile

Files

datasets.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Datasets and Examples

Core Imports

Basic Usage

Capabilities

Dataset Management

Dataset Cloning

Dataset Sharing

Dataset Versioning

Example Management

OpenAI Fine-tuning Export

Types and Interfaces

Dataset Schema

Example Schema

Dataset Version Schema

Dataset Share Schema

Dataset Diff Schema

Advanced Patterns

Dataset Workflow

Converting Runs to Examples

Dataset Splits for Train/Test/Val

Metadata-based Filtering

Integration with Evaluation

See Also

datasets.mddocs/