0
# LangChain Tests
1

2
A comprehensive Python testing framework for LangChain implementations, providing standardized test classes and fixtures for different LangChain components like ChatModels, Embeddings, Tools, VectorStores, and more. This package enables integration developers to ensure their implementations meet LangChain's interface standards through both unit and integration test suites.
3

4
## Package Information
5

6
- **Package Name**: langchain-tests
7
- **Language**: Python
8
- **Installation**: `pip install langchain-tests`
9
- **PyPI**: https://pypi.org/project/langchain-tests/
10
- **Source**: https://github.com/langchain-ai/langchain/tree/master/libs/standard-tests
11

12
## Core Imports
13

14
```python
15
# Unit test classes
16
from langchain_tests.unit_tests import (
17
    ChatModelUnitTests,
18
    EmbeddingsUnitTests,
19
    ToolsUnitTests
20
)
21

22
# Integration test classes
23
from langchain_tests.integration_tests import (
24
    ChatModelIntegrationTests,
25
    EmbeddingsIntegrationTests,
26
    ToolsIntegrationTests,
27
    VectorStoreIntegrationTests,
28
    RetrieversIntegrationTests,
29
    SyncCacheTestSuite,
30
    AsyncCacheTestSuite,
31
    BaseStoreSyncTests,
32
    BaseStoreAsyncTests
33
)
34
```
35

36
## Basic Usage
37

38
### Testing a Chat Model
39

40
```python
41
import pytest
42
from langchain_tests.unit_tests import ChatModelUnitTests
43
from langchain_tests.integration_tests import ChatModelIntegrationTests
44
from my_integration import MyChatModel
45

46
class TestMyChatModelUnit(ChatModelUnitTests):
47
    @property
48
    def chat_model_class(self):
49
        return MyChatModel
50
    
51
    @property
52
    def chat_model_params(self):
53
        return {"api_key": "test-key", "model": "test-model"}
54

55
class TestMyChatModelIntegration(ChatModelIntegrationTests):
56
    @property
57
    def chat_model_class(self):
58
        return MyChatModel
59
    
60
    @property
61
    def chat_model_params(self):
62
        return {"api_key": "real-api-key", "model": "production-model"}
63
    
64
    # Configure capabilities
65
    @property
66
    def has_tool_calling(self):
67
        return True
68
    
69
    @property
70
    def supports_structured_output(self):
71
        return True
72
```
73

74
### Testing a Vector Store
75

76
```python
77
from langchain_tests.integration_tests import VectorStoreIntegrationTests
78
from my_integration import MyVectorStore
79

80
class TestMyVectorStore(VectorStoreIntegrationTests):
81
    @pytest.fixture
82
    def vectorstore(self):
83
        return MyVectorStore(connection_url="test://localhost")
84
    
85
    @property
86
    def has_sync(self):
87
        return True
88
    
89
    @property
90
    def has_async(self):
91
        return False
92
```
93

94
## Architecture
95

96
The langchain-tests framework is built around inheritance-based testing patterns:
97

98
- **Base Classes**: Abstract test classes that define the testing interface
99
- **Configuration Properties**: Boolean flags and parameter dictionaries to configure test behavior
100
- **Fixture System**: pytest fixtures for test isolation and setup
101
- **Standard Test Methods**: Pre-defined test methods that verify LangChain interface compliance
102

103
### Test Class Hierarchy
104

105
```
106
BaseStandardTests (base override protection)
107
├── ChatModelTests (base chat model testing)
108
│   ├── ChatModelUnitTests (unit tests)
109
│   └── ChatModelIntegrationTests (integration tests)
110
├── EmbeddingsTests (base embeddings testing)
111
│   ├── EmbeddingsUnitTests (unit tests)
112
│   └── EmbeddingsIntegrationTests (integration tests)
113
├── ToolsTests (base tool testing)
114
│   ├── ToolsUnitTests (unit tests)
115
│   └── ToolsIntegrationTests (integration tests)
116
├── VectorStoreIntegrationTests (vector store testing)
117
├── RetrieversIntegrationTests (retriever testing)
118
├── SyncCacheTestSuite / AsyncCacheTestSuite (cache testing)
119
└── BaseStoreSyncTests / BaseStoreAsyncTests (key-value store testing)
120
```
121

122
## Capabilities
123

124
### Unit Testing
125

126
Lightweight tests for basic functionality verification, initialization, serialization, and environment variable configuration. Unit tests focus on object creation, parameter validation, and basic method availability without external dependencies.
127

128
```python { .api }
129
class ChatModelUnitTests(ChatModelTests):
130
    """Unit tests for chat models."""
131
    
132
    @property
133
    def init_from_env_params(self) -> dict:
134
        """Parameters for environment variable initialization test."""
135
    
136
    def test_init(self) -> None:
137
        """Test model initialization."""
138
    
139
    def test_init_from_env(self) -> None:
140
        """Test initialization from environment variables."""
141
    
142
    def test_serdes(self) -> None:
143
        """Test serialization/deserialization."""
144
```
145

146
[Unit Testing](./unit-tests.md)
147

148
### Integration Testing
149

150
Comprehensive tests for full functionality including API calls, streaming, tool calling, structured output, and multimodal inputs. Integration tests verify real-world usage patterns and complete feature sets.
151

152
```python { .api }
153
class ChatModelIntegrationTests(ChatModelTests):
154
    """Integration tests for chat models with 40+ test methods."""
155
    
156
    def test_invoke(self) -> None:
157
        """Test basic invocation."""
158
    
159
    def test_stream(self) -> None:
160
        """Test streaming responses."""
161
    
162
    def test_tool_calling(self) -> None:
163
        """Test tool calling functionality."""
164
    
165
    def test_structured_output(self) -> None:
166
        """Test structured output generation."""
167
```
168

169
[Integration Testing](./integration-tests.md)
170

171
### Vector Store Testing
172

173
Specialized testing suite for vector store implementations with comprehensive CRUD operations, async support, and bulk operations testing.
174

175
```python { .api }
176
class VectorStoreIntegrationTests(BaseStandardTests):
177
    """Integration tests for vector stores with 30+ test methods."""
178
    
179
    @property
180
    def has_sync(self) -> bool:
181
        """Whether the vector store supports synchronous operations."""
182
    
183
    @property
184
    def has_async(self) -> bool:
185
        """Whether the vector store supports asynchronous operations."""
186
    
187
    def get_embeddings(self):
188
        """Returns deterministic fake embeddings for testing."""
189
```
190

191
[Vector Store Testing](./vector-stores.md)
192

193
### Cache Testing
194

195
Testing suites for LangChain cache implementations with both synchronous and asynchronous support, covering cache hits, misses, updates, and clearing operations.
196

197
```python { .api }
198
class SyncCacheTestSuite(BaseStandardTests):
199
    """Synchronous cache testing suite."""
200
    
201
    @pytest.fixture
202
    def cache(self):
203
        """Empty cache instance for testing."""
204
    
205
    def test_cache_hit(self) -> None:
206
        """Test cache hit behavior."""
207
    
208
    def test_cache_miss(self) -> None:
209
        """Test cache miss handling."""
210
```
211

212
[Cache Testing](./cache-testing.md)
213

214
### Configuration and Utilities
215

216
VCR (Video Cassette Recorder) integration for HTTP call recording/playback, pytest fixtures for test isolation, Pydantic utilities, and custom serialization for consistent testing environments.
217

218
```python { .api }
219
class CustomSerializer:
220
    """Custom VCR cassette serializer using YAML and gzip compression."""
221
    
222
    def serialize(self, cassette_dict: dict) -> bytes:
223
        """Convert cassette to compressed YAML."""
224
    
225
    def deserialize(self, data: bytes) -> dict:
226
        """Decompress and convert from YAML."""
227
```
228

229
[Configuration and Utilities](./configuration.md)
230

231
### Key-Value Store Testing
232

233
Generic testing suites for key-value store implementations with comprehensive CRUD operations, bulk operations, and both synchronous and asynchronous support.
234

235
```python { .api }
236
from typing import Generic, TypeVar
237

238
V = TypeVar('V')
239

240
class BaseStoreSyncTests(BaseStandardTests, Generic[V]):
241
    """Synchronous key-value store testing suite."""
242
    
243
    @pytest.fixture
244
    def kv_store(self):
245
        """Empty key-value store instance."""
246
    
247
    @pytest.fixture  
248
    def three_values(self) -> List[V]:
249
        """Three example values for testing."""
250
```
251

252
[Key-Value Store Testing](./key-value-stores.md)
253

254
## Error Handling
255

256
The framework provides comprehensive error handling patterns:
257

258
- **Validation Errors**: Parameter validation and type checking
259
- **Connection Errors**: Network and API connectivity issues  
260
- **Configuration Errors**: Missing or invalid configuration parameters
261
- **Tool Calling Errors**: Structured error responses for tool failures
262
- **Timeout Errors**: Async operation timeout handling
263

264
Standard test methods verify that implementations handle these error conditions appropriately and return meaningful error messages.