or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

cache-testing.mdconfiguration.mdindex.mdintegration-tests.mdkey-value-stores.mdunit-tests.mdvector-stores.md
tile.json

tessl/pypi-langchain-tests

Standard tests for LangChain implementations

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/langchain-tests@0.3.x

To install, run

npx @tessl/cli install tessl/pypi-langchain-tests@0.3.0

index.mddocs/

LangChain Tests

A comprehensive Python testing framework for LangChain implementations, providing standardized test classes and fixtures for different LangChain components like ChatModels, Embeddings, Tools, VectorStores, and more. This package enables integration developers to ensure their implementations meet LangChain's interface standards through both unit and integration test suites.

Package Information

  • Package Name: langchain-tests
  • Language: Python
  • Installation: pip install langchain-tests
  • PyPI: https://pypi.org/project/langchain-tests/
  • Source: https://github.com/langchain-ai/langchain/tree/master/libs/standard-tests

Core Imports

# Unit test classes
from langchain_tests.unit_tests import (
    ChatModelUnitTests,
    EmbeddingsUnitTests,
    ToolsUnitTests
)

# Integration test classes
from langchain_tests.integration_tests import (
    ChatModelIntegrationTests,
    EmbeddingsIntegrationTests,
    ToolsIntegrationTests,
    VectorStoreIntegrationTests,
    RetrieversIntegrationTests,
    SyncCacheTestSuite,
    AsyncCacheTestSuite,
    BaseStoreSyncTests,
    BaseStoreAsyncTests
)

Basic Usage

Testing a Chat Model

import pytest
from langchain_tests.unit_tests import ChatModelUnitTests
from langchain_tests.integration_tests import ChatModelIntegrationTests
from my_integration import MyChatModel

class TestMyChatModelUnit(ChatModelUnitTests):
    @property
    def chat_model_class(self):
        return MyChatModel
    
    @property
    def chat_model_params(self):
        return {"api_key": "test-key", "model": "test-model"}

class TestMyChatModelIntegration(ChatModelIntegrationTests):
    @property
    def chat_model_class(self):
        return MyChatModel
    
    @property
    def chat_model_params(self):
        return {"api_key": "real-api-key", "model": "production-model"}
    
    # Configure capabilities
    @property
    def has_tool_calling(self):
        return True
    
    @property
    def supports_structured_output(self):
        return True

Testing a Vector Store

from langchain_tests.integration_tests import VectorStoreIntegrationTests
from my_integration import MyVectorStore

class TestMyVectorStore(VectorStoreIntegrationTests):
    @pytest.fixture
    def vectorstore(self):
        return MyVectorStore(connection_url="test://localhost")
    
    @property
    def has_sync(self):
        return True
    
    @property
    def has_async(self):
        return False

Architecture

The langchain-tests framework is built around inheritance-based testing patterns:

  • Base Classes: Abstract test classes that define the testing interface
  • Configuration Properties: Boolean flags and parameter dictionaries to configure test behavior
  • Fixture System: pytest fixtures for test isolation and setup
  • Standard Test Methods: Pre-defined test methods that verify LangChain interface compliance

Test Class Hierarchy

BaseStandardTests (base override protection)
├── ChatModelTests (base chat model testing)
│   ├── ChatModelUnitTests (unit tests)
│   └── ChatModelIntegrationTests (integration tests)
├── EmbeddingsTests (base embeddings testing)
│   ├── EmbeddingsUnitTests (unit tests)
│   └── EmbeddingsIntegrationTests (integration tests)
├── ToolsTests (base tool testing)
│   ├── ToolsUnitTests (unit tests)
│   └── ToolsIntegrationTests (integration tests)
├── VectorStoreIntegrationTests (vector store testing)
├── RetrieversIntegrationTests (retriever testing)
├── SyncCacheTestSuite / AsyncCacheTestSuite (cache testing)
└── BaseStoreSyncTests / BaseStoreAsyncTests (key-value store testing)

Capabilities

Unit Testing

Lightweight tests for basic functionality verification, initialization, serialization, and environment variable configuration. Unit tests focus on object creation, parameter validation, and basic method availability without external dependencies.

class ChatModelUnitTests(ChatModelTests):
    """Unit tests for chat models."""
    
    @property
    def init_from_env_params(self) -> dict:
        """Parameters for environment variable initialization test."""
    
    def test_init(self) -> None:
        """Test model initialization."""
    
    def test_init_from_env(self) -> None:
        """Test initialization from environment variables."""
    
    def test_serdes(self) -> None:
        """Test serialization/deserialization."""

Unit Testing

Integration Testing

Comprehensive tests for full functionality including API calls, streaming, tool calling, structured output, and multimodal inputs. Integration tests verify real-world usage patterns and complete feature sets.

class ChatModelIntegrationTests(ChatModelTests):
    """Integration tests for chat models with 40+ test methods."""
    
    def test_invoke(self) -> None:
        """Test basic invocation."""
    
    def test_stream(self) -> None:
        """Test streaming responses."""
    
    def test_tool_calling(self) -> None:
        """Test tool calling functionality."""
    
    def test_structured_output(self) -> None:
        """Test structured output generation."""

Integration Testing

Vector Store Testing

Specialized testing suite for vector store implementations with comprehensive CRUD operations, async support, and bulk operations testing.

class VectorStoreIntegrationTests(BaseStandardTests):
    """Integration tests for vector stores with 30+ test methods."""
    
    @property
    def has_sync(self) -> bool:
        """Whether the vector store supports synchronous operations."""
    
    @property
    def has_async(self) -> bool:
        """Whether the vector store supports asynchronous operations."""
    
    def get_embeddings(self):
        """Returns deterministic fake embeddings for testing."""

Vector Store Testing

Cache Testing

Testing suites for LangChain cache implementations with both synchronous and asynchronous support, covering cache hits, misses, updates, and clearing operations.

class SyncCacheTestSuite(BaseStandardTests):
    """Synchronous cache testing suite."""
    
    @pytest.fixture
    def cache(self):
        """Empty cache instance for testing."""
    
    def test_cache_hit(self) -> None:
        """Test cache hit behavior."""
    
    def test_cache_miss(self) -> None:
        """Test cache miss handling."""

Cache Testing

Configuration and Utilities

VCR (Video Cassette Recorder) integration for HTTP call recording/playback, pytest fixtures for test isolation, Pydantic utilities, and custom serialization for consistent testing environments.

class CustomSerializer:
    """Custom VCR cassette serializer using YAML and gzip compression."""
    
    def serialize(self, cassette_dict: dict) -> bytes:
        """Convert cassette to compressed YAML."""
    
    def deserialize(self, data: bytes) -> dict:
        """Decompress and convert from YAML."""

Configuration and Utilities

Key-Value Store Testing

Generic testing suites for key-value store implementations with comprehensive CRUD operations, bulk operations, and both synchronous and asynchronous support.

from typing import Generic, TypeVar

V = TypeVar('V')

class BaseStoreSyncTests(BaseStandardTests, Generic[V]):
    """Synchronous key-value store testing suite."""
    
    @pytest.fixture
    def kv_store(self):
        """Empty key-value store instance."""
    
    @pytest.fixture  
    def three_values(self) -> List[V]:
        """Three example values for testing."""

Key-Value Store Testing

Error Handling

The framework provides comprehensive error handling patterns:

  • Validation Errors: Parameter validation and type checking
  • Connection Errors: Network and API connectivity issues
  • Configuration Errors: Missing or invalid configuration parameters
  • Tool Calling Errors: Structured error responses for tool failures
  • Timeout Errors: Async operation timeout handling

Standard test methods verify that implementations handle these error conditions appropriately and return meaningful error messages.