0
# Auto Models
1

2
Auto model factory classes that automatically select and initialize the appropriate embedder or reranker implementation based on the model name. These provide the simplest way to use FlagEmbedding with any supported model without needing to know the specific model architecture or implementation details.
3

4
## Capabilities
5

6
### FlagAutoModel Factory
7

8
Automatically selects the appropriate embedder class based on the model name and returns an initialized instance. Supports all embedder model types including encoder-only, decoder-only, and specialized variants.
9

10
```python { .api }
11
from typing import Union
12

13
class FlagAutoModel:
14
    @classmethod
15
    def from_finetuned(
16
        cls,
17
        model_name_or_path: str,
18
        model_class: Optional[Union[str, EmbedderModelClass]] = None,
19
        normalize_embeddings: bool = True,
20
        use_fp16: bool = True,
21
        query_instruction_for_retrieval: Optional[str] = None,
22
        devices: Optional[Union[str, List[str]]] = None,
23
        pooling_method: Optional[str] = None,
24
        trust_remote_code: Optional[bool] = None,
25
        query_instruction_format: Optional[str] = None,
26
        **kwargs
27
    ) -> AbsEmbedder:
28
        """
29
        Automatically create embedder instance from model name.
30
        
31
        Args:
32
            model_name_or_path: Path to model or HuggingFace model name
33
            model_class: Force specific model class (optional)
34
            normalize_embeddings: Whether to normalize output embeddings
35
            use_fp16: Use half precision for inference
36
            query_instruction_for_retrieval: Instruction prepended to queries
37
            devices: List of devices for multi-GPU inference
38
            pooling_method: Pooling strategy ("cls", "mean", "last_token")
39
            trust_remote_code: Allow custom model code execution
40
            query_instruction_format: Format string for query instructions
41
            **kwargs: Additional model-specific parameters
42
            
43
        Returns:
44
            Initialized AbsEmbedder instance
45
        """
46
```
47

48
### FlagAutoReranker Factory
49

50
Automatically selects the appropriate reranker class based on the model name and returns an initialized instance. Supports all reranker model types including encoder-only, decoder-only, and specialized variants.
51

52
```python { .api }
53
class FlagAutoReranker:
54
    @classmethod
55
    def from_finetuned(
56
        cls,
57
        model_name_or_path: str,
58
        model_class: Optional[Union[str, RerankerModelClass]] = None,
59
        use_fp16: bool = False,
60
        trust_remote_code: Optional[bool] = None,
61
        **kwargs
62
    ) -> AbsReranker:
63
        """
64
        Automatically create reranker instance from model name.
65
        
66
        Args:
67
            model_name_or_path: Path to model or HuggingFace model name
68
            model_class: Force specific model class (optional)
69
            use_fp16: Use half precision for inference
70
            trust_remote_code: Allow custom model code execution
71
            **kwargs: Additional model-specific parameters
72
            
73
        Returns:
74
            Initialized AbsReranker instance
75
        """
76
```
77

78
## Usage Examples
79

80
### Basic Auto Embedder Usage
81

82
```python
83
from FlagEmbedding import FlagAutoModel
84

85
# Automatic model selection
86
embedder = FlagAutoModel.from_finetuned('bge-large-en-v1.5')
87

88
# Encode single query
89
embedding = embedder.encode_queries("What is machine learning?")
90

91
# Encode multiple documents
92
docs = ["ML is a subset of AI", "Neural networks are computing systems"]
93
doc_embeddings = embedder.encode_corpus(docs)
94
```
95

96
### Multi-GPU Auto Embedder
97

98
```python
99
from FlagEmbedding import FlagAutoModel
100

101
# Use multiple GPUs for faster inference
102
embedder = FlagAutoModel.from_finetuned(
103
    'bge-large-en-v1.5',
104
    devices=['cuda:0', 'cuda:1', 'cuda:2'],
105
    use_fp16=True
106
)
107

108
# Process large batch across multiple devices
109
large_corpus = [f"Document {i}" for i in range(10000)]
110
embeddings = embedder.encode_corpus(large_corpus, batch_size=512)
111
```
112

113
### Custom Instructions and Formatting
114

115
```python
116
from FlagEmbedding import FlagAutoModel
117

118
# Use custom query instructions
119
embedder = FlagAutoModel.from_finetuned(
120
    'bge-large-en-v1.5',
121
    query_instruction_for_retrieval="Represent this query for retrieving relevant documents: ",
122
    query_instruction_format="{}{}",
123
    normalize_embeddings=True
124
)
125

126
queries = ["How to implement neural networks"]
127
embeddings = embedder.encode_queries(queries)
128
```
129

130
### Auto Reranker Usage
131

132
```python
133
from FlagEmbedding import FlagAutoReranker
134

135
# Initialize reranker
136
reranker = FlagAutoReranker.from_finetuned('bge-reranker-base')
137

138
# Score query-document pairs
139
pairs = [
140
    ("machine learning query", "ML is a subset of artificial intelligence"),
141
    ("machine learning query", "Cooking recipes for pasta dishes")
142
]
143

144
scores = reranker.compute_score(pairs)
145
print(f"Relevance scores: {scores}")  # Higher scores = more relevant
146
```
147

148
### Forced Model Class Selection
149

150
```python
151
from FlagEmbedding import FlagAutoModel, EmbedderModelClass
152

153
# Force specific model class even if auto-detection would choose different
154
embedder = FlagAutoModel.from_finetuned(
155
    'custom-model-name',
156
    model_class=EmbedderModelClass.ENCODER_ONLY_BASE,
157
    pooling_method="mean",
158
    trust_remote_code=True
159
)
160
```
161

162
## Model Selection Logic
163

164
The auto factories use the following logic to select model implementations:
165

166
1. **Model Name Matching**: Checks model name against known model patterns
167
2. **Architecture Detection**: Analyzes model configuration to determine architecture type
168
3. **Fallback Selection**: Uses sensible defaults based on model characteristics
169
4. **Override Support**: Allows manual override via `model_class` parameter
170

171
## Error Handling
172

173
```python
174
from FlagEmbedding import FlagAutoModel
175

176
try:
177
    embedder = FlagAutoModel.from_finetuned('unknown-model')
178
except ValueError as e:
179
    print(f"Model not supported: {e}")
180
    
181
try:
182
    # Invalid device specification
183
    embedder = FlagAutoModel.from_finetuned('bge-base-en', devices=['invalid:0'])
184
except RuntimeError as e:
185
    print(f"Device error: {e}")
186
```
187

188
## Types
189

190
```python { .api }
191
from typing import Optional, List
192
from FlagEmbedding import AbsEmbedder, AbsReranker, EmbedderModelClass, RerankerModelClass
193

194
# Type aliases for auto model factory methods
195
AutoEmbedderFactory = type(FlagAutoModel.from_finetuned)
196
AutoRerankerFactory = type(FlagAutoReranker.from_finetuned)
197
```