or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/langsmith@0.6.x
tile.json

tessl/pypi-langsmith

tessl install tessl/pypi-langsmith@0.6.1

Python SDK for LangSmith Observability and Evaluation Platform

index.mddocs/

LangSmith Python SDK

A comprehensive Python SDK for LangSmith, providing observability, evaluation, and testing capabilities for LLM applications. LangSmith enables developers to trace, debug, evaluate, and monitor AI applications through a unified API, with full support for synchronous and asynchronous operations.

Package Information

  • Package Name: langsmith
  • Version: 0.6.2
  • Installation: pip install langsmith
  • Language: Python

Quick Start

Installation

pip install langsmith

Basic Imports

import langsmith as ls
from langsmith import Client, traceable

First Trace

import langsmith as ls
from langsmith import traceable

# Configure at startup
ls.configure(
    enabled=True,
    project_name="my-project"
)

# Trace a function
@traceable
def process_query(query: str) -> str:
    """Process a user query."""
    result = f"Processed: {query}"
    return result

# Use it
result = process_query("What is LangSmith?")

Get Started →

How Do I...?

Quick links to common tasks:

Tracing

Data Management

Evaluation

Testing

Client Operations

Advanced

💡 Navigation Tip: This index provides three ways to find what you need:

  1. "How Do I?" above - Task-oriented links for common operations
  2. "Quick API Reference" below - API signatures for immediate copy-paste use
  3. "API Directory" further below - Links to complete documentation with examples and patterns

Quick API Reference

Essential API signatures for immediate use. For detailed documentation, examples, and usage patterns, see the API Directory below.

Tracing

# Automatic tracing decorator
@traceable(
    run_type: Literal["chain", "llm", "tool", "retriever", "prompt"] = "chain",
    name: Optional[str] = None,
    metadata: Optional[Mapping[str, Any]] = None,
    tags: Optional[list[str]] = None,
    client: Optional[Client] = None,
    project_name: Optional[str] = None,
)

# Manual tracing context manager
with trace(
    name: str,
    run_type: str = "chain",
    inputs: Optional[dict] = None,
    metadata: Optional[Mapping[str, Any]] = None,
    tags: Optional[list[str]] = None,
    project_name: Optional[str] = None,
) as run:
    run.end(outputs=dict, error=str)

# Context utilities
get_current_run_tree() -> Optional[RunTree]
set_run_metadata(**metadata: Any) -> None

# Global configuration
ls.configure(
    enabled: Optional[bool] = ...,
    project_name: Optional[str] = ...,
    client: Optional[Client] = ...,
)

→ Full Tracing Documentation

Client

# Initialize client
client = Client(
    api_key: Optional[str] = None,
    api_url: Optional[str] = None,
    timeout_ms: Optional[int] = None,
)

# Run operations
client.create_run(name: str, inputs: dict, run_type: str, ...)
client.read_run(run_id: Union[str, UUID]) -> Run
client.list_runs(project_name: str = None, ...) -> Iterator[Run]
client.share_run(run_id: Union[str, UUID]) -> str

# Project operations
client.create_project(project_name: str, ...) -> TracerSession
client.list_projects(...) -> Iterator[TracerSession]

# Dataset operations
client.create_dataset(dataset_name: str, ...) -> Dataset
client.create_example(inputs: dict, dataset_id: UUID, outputs: dict = None, ...) -> Example
client.list_examples(dataset_id: UUID, ...) -> Iterator[Example]

# Feedback operations
client.create_feedback(run_id: UUID, key: str, score: float = None, ...) -> Feedback

→ Full Client Documentation

Evaluation

# Evaluate a target function
results = evaluate(
    target: Union[Callable, Runnable, str, UUID],
    data: Union[str, UUID, Iterable[Example]],
    evaluators: Optional[Sequence[Union[RunEvaluator, Callable]]] = None,
    summary_evaluators: Optional[Sequence[Callable]] = None,
    experiment_prefix: Optional[str] = None,
    max_concurrency: Optional[int] = None,
) -> ExperimentResults

# Async evaluation
results = await aevaluate(target, data, evaluators, ...)

# Evaluate existing experiment
results = evaluate_existing(
    experiment: Union[str, UUID],
    evaluators: Sequence[RunEvaluator],
) -> ExperimentResults

# Custom evaluator protocol
class MyEvaluator(RunEvaluator):
    def evaluate_run(self, run: Run, example: Example) -> EvaluationResult:
        return EvaluationResult(key="metric", score=0.9)

→ Full Evaluation Documentation

Testing

# Trace pytest tests
@test(output_keys=["answer"], ...)
def test_my_function():
    result = my_function()
    assert result["answer"] == "expected"

# Use expectations API
@test
def test_with_expectations():
    result = my_function()
    assert expect.score(0.9, key="accuracy").compare(result)
    assert expect.embedding_distance("predicted", "reference").compare(result)

→ Full Testing Documentation

Async Client

# Initialize async client
async with AsyncClient(api_key=..., api_url=...) as client:
    await client.create_run(...)
    run = await client.read_run(run_id)
    async for run in client.list_runs(...):
        process(run)

→ Full Async Client Documentation

Caching

# Synchronous cache
cache = Cache(max_size=100, ttl_seconds=3600)
cache.set(key, value)
value = cache.get(key)

# Asynchronous cache
async_cache = AsyncCache(max_size=100, ttl_seconds=3600)
await async_cache.start()
async_cache.set(key, value)
value = async_cache.get(key)

# Use with client
client = Client(cache=True)  # Use default cache

→ Full Caching Documentation

API Directory

Core APIs

Client & AsyncClient

Synchronous and asynchronous clients for the LangSmith API.

  • Client - Synchronous client with full CRUD operations for runs, projects, datasets, examples, and feedback
  • AsyncClient - Async/await client for high-performance concurrent operations

Tracing APIs

Automatic and manual tracing of functions and code blocks.

Configuration

Global configuration for tracing behavior.

Data APIs

Projects

Organize and manage trace collections (also called sessions).

Datasets

Collections of examples for evaluation and testing.

Examples

Individual records within datasets.

Feedback

Metrics and annotations on runs.

Evaluation APIs

Run evaluations and create custom metrics.

Testing APIs

Pytest integration for trace-aware testing.

Advanced Features

Caching

LRU cache with TTL and background refresh.

  • Cache - Synchronous cache implementation
  • AsyncCache - Asynchronous cache implementation

Utilities

Helper functions and data anonymization.

Prompt Management

Version and share prompts.

Annotation Queues

Human review workflows.

Reference

Complete API reference and data schemas.

  • Data Schemas - Core data types (Run, Example, Dataset, Feedback, etc.)

Architecture Overview

LangSmith's architecture centers on runs (trace spans) organized in tree structures:

  1. Tracing System: Uses context variables to implicitly track parent-child relationships between runs, enabling automatic trace tree construction

  2. Client Architecture: The Client and AsyncClient provide comprehensive API access for managing runs, projects, datasets, examples, and feedback

  3. RunTree: Represents individual trace spans with metadata, inputs, outputs, and timing information

  4. Evaluation: Built on top of tracing infrastructure, allowing runs to be associated with dataset examples and evaluated by custom or built-in evaluators

  5. Testing Framework: Integrates with pytest to provide trace-aware test execution with expectations and assertions

Common Patterns

Basic Tracing Workflow

import langsmith as ls

# 1. Configure once at startup
ls.configure(enabled=True, project_name="my-app")

# 2. Trace your functions
@traceable
def my_chain(input_text):
    return process(input_text)

# 3. Run and traces are automatically sent
result = my_chain("test input")

Evaluation Workflow

from langsmith import evaluate

# 1. Define your target function
def my_app(inputs):
    return {"answer": process(inputs["question"])}

# 2. Define evaluator
def correctness(run, example):
    return {
        "key": "correct",
        "score": run.outputs["answer"] == example.outputs["expected"]
    }

# 3. Run evaluation
results = evaluate(
    my_app,
    data="my-dataset",
    evaluators=[correctness]
)

Async Pattern

from langsmith import AsyncClient, traceable

@traceable
async def async_process(input_data):
    result = await async_operation(input_data)
    return result

# Use with async client
async with AsyncClient() as client:
    await client.create_run(...)

Environment Variables

LangSmith respects the following environment variables:

  • LANGSMITH_API_KEY - API key for authentication
  • LANGSMITH_ENDPOINT - API endpoint URL (default: https://api.smith.langchain.com)
  • LANGSMITH_PROJECT - Default project name
  • LANGSMITH_TRACING - Enable/disable tracing ("true"/"false")

Error Handling

LangSmith is designed to be non-blocking - tracing errors won't crash your application:

from langsmith import Client

# Tracing errors are logged but don't raise
client = Client()

@traceable
def my_function():
    # Even if tracing fails, your function runs
    return "result"

To handle tracing errors explicitly:

def error_handler(error: Exception):
    print(f"Tracing error: {error}")

client = Client(tracing_error_callback=error_handler)

Next Steps