0
# FlagEmbedding
1
2
FlagEmbedding (BGE - BAAI General Embedding) is a comprehensive Python library focused on retrieval-augmented language models and embedding technologies. The package provides state-of-the-art text embedding models, rerankers, and multimodal embedding capabilities for search and RAG applications. It includes tools for both inference and fine-tuning of embedding models, evaluation frameworks, and supports various embedding tasks including text-to-text, text-to-image, and image-to-text retrieval.
3
4
## Package Information
5
6
- **Package Name**: FlagEmbedding
7
- **Language**: Python
8
- **Installation**: `pip install FlagEmbedding`
9
10
## Core Imports
11
12
```python
13
from FlagEmbedding import FlagAutoModel, FlagAutoReranker
14
```
15
16
For direct model access:
17
18
```python
19
from FlagEmbedding import FlagModel, BGEM3FlagModel, FlagReranker
20
from FlagEmbedding import FlagLLMModel, FlagICLModel, FlagLLMReranker
21
```
22
23
For model class enumeration:
24
25
```python
26
from FlagEmbedding import EmbedderModelClass, RerankerModelClass
27
```
28
29
## Basic Usage
30
31
```python
32
from FlagEmbedding import FlagAutoModel, FlagAutoReranker
33
34
# Initialize embedder with automatic model selection
35
embedder = FlagAutoModel.from_finetuned('bge-large-en-v1.5', use_fp16=True)
36
37
# Encode queries and documents
38
queries = ["What is machine learning?", "How does neural networks work?"]
39
documents = [
40
"Machine learning is a subset of artificial intelligence...",
41
"Neural networks are computing systems inspired by biological neural networks..."
42
]
43
44
query_embeddings = embedder.encode_queries(queries)
45
doc_embeddings = embedder.encode_corpus(documents)
46
47
# Initialize reranker for scoring
48
reranker = FlagAutoReranker.from_finetuned('bge-reranker-base')
49
50
# Score query-document pairs
51
pairs = [("What is machine learning?", "Machine learning is a subset of artificial intelligence...")]
52
scores = reranker.compute_score(pairs)
53
54
print(f"Similarity score: {scores[0]}")
55
```
56
57
## Architecture
58
59
FlagEmbedding is built around a hierarchical architecture that supports multiple model types and architectures:
60
61
- **Abstract Base Classes**: `AbsEmbedder` and `AbsReranker` define the interface contracts for all embedding/reranking models
62
- **Auto Models**: Factory classes (`FlagAutoModel`, `FlagAutoReranker`) that automatically select appropriate model implementations
63
- **Concrete Implementations**: Specialized classes for encoder-only models (BERT-like), decoder-only models (LLM-like), and hybrid approaches
64
- **Multi-device Support**: Built-in parallelization across multiple GPUs/devices for scalable inference
65
66
## Capabilities
67
68
### Auto Model Factory
69
70
Automatically selects and initializes the appropriate embedder or reranker class based on the model name, providing the simplest way to use FlagEmbedding with any supported model.
71
72
```python { .api }
73
class FlagAutoModel:
74
@classmethod
75
def from_finetuned(
76
cls,
77
model_name_or_path: str,
78
model_class: Optional[str] = None,
79
normalize_embeddings: bool = True,
80
use_fp16: bool = True,
81
query_instruction_for_retrieval: Optional[str] = None,
82
devices: Optional[List[str]] = None,
83
pooling_method: Optional[str] = None,
84
trust_remote_code: Optional[bool] = None,
85
**kwargs
86
) -> AbsEmbedder: ...
87
88
class FlagAutoReranker:
89
@classmethod
90
def from_finetuned(
91
cls,
92
model_name_or_path: str,
93
model_class: Optional[str] = None,
94
use_fp16: bool = False,
95
trust_remote_code: Optional[bool] = None,
96
**kwargs
97
) -> AbsReranker: ...
98
```
99
100
[Auto Models](./auto-models.md)
101
102
### Encoder-Only Embedders
103
104
Embedders designed for encoder-only transformer models (BERT-like architectures), including specialized implementations for BGE-M3 models with dense, sparse, and ColBERT support.
105
106
```python { .api }
107
class FlagModel(AbsEmbedder):
108
def __init__(
109
self,
110
model_name_or_path: str,
111
pooling_method: str = "cls",
112
normalize_embeddings: bool = True,
113
use_fp16: bool = True,
114
trust_remote_code: bool = False,
115
**kwargs
116
): ...
117
118
class BGEM3FlagModel(AbsEmbedder):
119
def __init__(
120
self,
121
model_name_or_path: str,
122
pooling_method: str = "cls",
123
normalize_embeddings: bool = True,
124
use_fp16: bool = True,
125
colbert_dim: int = -1,
126
return_dense: bool = True,
127
return_sparse: bool = False,
128
return_colbert_vecs: bool = False,
129
**kwargs
130
): ...
131
```
132
133
[Encoder-Only Embedders](./encoder-embedders.md)
134
135
### Decoder-Only Embedders
136
137
Embedders for decoder-only transformer models (LLM-like architectures), including support for in-context learning approaches.
138
139
```python { .api }
140
class FlagLLMModel(AbsEmbedder):
141
def __init__(
142
self,
143
model_name_or_path: str,
144
pooling_method: str = "last_token",
145
normalize_embeddings: bool = True,
146
use_fp16: bool = True,
147
query_instruction_format: str = "Instruct: {}\\nQuery: {}",
148
**kwargs
149
): ...
150
151
class FlagICLModel(AbsEmbedder):
152
def __init__(
153
self,
154
model_name_or_path: str,
155
pooling_method: str = "last_token",
156
normalize_embeddings: bool = True,
157
use_fp16: bool = True,
158
**kwargs
159
): ...
160
```
161
162
[Decoder-Only Embedders](./decoder-embedders.md)
163
164
### Reranking Models
165
166
Reranking models for scoring query-document pairs, available in both encoder-only and decoder-only variants with specialized implementations for different use cases.
167
168
```python { .api }
169
class FlagReranker(AbsReranker):
170
def __init__(
171
self,
172
model_name_or_path: str,
173
use_fp16: bool = False,
174
trust_remote_code: bool = False,
175
**kwargs
176
): ...
177
178
class FlagLLMReranker(AbsReranker):
179
def __init__(
180
self,
181
model_name_or_path: str,
182
use_fp16: bool = False,
183
**kwargs
184
): ...
185
```
186
187
[Rerankers](./rerankers.md)
188
189
### Base Classes and Interfaces
190
191
Abstract base classes that define the core interface contracts for embedders and rerankers, providing multi-device support and consistent API patterns.
192
193
```python { .api }
194
class AbsEmbedder:
195
def encode_queries(
196
self,
197
queries: List[str],
198
batch_size: Optional[int] = None,
199
max_length: Optional[int] = None,
200
convert_to_numpy: Optional[bool] = None,
201
**kwargs
202
) -> Union[torch.Tensor, np.ndarray]: ...
203
204
class AbsReranker:
205
def compute_score(
206
self,
207
sentence_pairs: List[Tuple[str, str]],
208
**kwargs
209
) -> np.ndarray: ...
210
```
211
212
[Base Classes](./base-classes.md)
213
214
### Model Enumerations and Utilities
215
216
Enumerations for supported model classes and utility functions for discovering available models and their capabilities.
217
218
```python { .api }
219
class EmbedderModelClass(Enum):
220
ENCODER_ONLY_BASE = "encoder-only-base"
221
ENCODER_ONLY_M3 = "encoder-only-m3"
222
DECODER_ONLY_BASE = "decoder-only-base"
223
DECODER_ONLY_ICL = "decoder-only-icl"
224
225
class RerankerModelClass(Enum):
226
ENCODER_ONLY_BASE = "encoder-only-base"
227
DECODER_ONLY_BASE = "decoder-only-base"
228
DECODER_ONLY_LAYERWISE = "decoder-only-layerwise"
229
DECODER_ONLY_LIGHTWEIGHT = "decoder-only-lightweight"
230
231
def support_model_list() -> List[str]: ...
232
def support_native_bge_model_list() -> List[str]: ...
233
```
234
235
[Model Types and Utilities](./model-types.md)
236
237
## Supported Models
238
239
FlagEmbedding supports a comprehensive range of pre-trained models:
240
241
- **BGE Models**: bge-m3, bge-large-en-v1.5, bge-base-en-v1.5, bge-small-en-v1.5, bge-large-zh-v1.5, bge-multilingual-gemma2, bge-en-icl
242
- **E5 Models**: e5-mistral-7b-instruct, e5-large-v2, e5-base-v2, multilingual-e5-large-instruct, multilingual-e5-large
243
- **GTE Models**: gte-Qwen2-7B-instruct, gte-Qwen2-1.5B-instruct, gte-large-en-v1.5, gte-base-en-v1.5, gte-multilingual-base
244
- **Reranker Models**: bge-reranker-base, bge-reranker-large, bge-reranker-v2-m3, bge-reranker-v2-gemma, bge-reranker-v2-minicpm-layerwise
245
246
## Common Types
247
248
```python { .api }
249
from typing import List, Union, Optional, Dict, Any, Tuple
250
import torch
251
import numpy as np
252
253
# Core types used across the API
254
QueryType = Union[str, List[str]]
255
CorpusType = Union[str, List[str]]
256
EmbeddingOutput = Union[torch.Tensor, np.ndarray]
257
SentencePair = Tuple[str, str]
258
DeviceSpec = Union[str, List[str]]
259
```