0
# Model Interface
1
2
The Model interface provides direct access to pre-trained models with fine-grained control over the inference process. This lower-level interface allows for custom workflows, advanced model manipulation, and integration into custom pipelines.
3
4
## Capabilities
5
6
### Base Model Class
7
8
Abstract base class providing the foundation for all ModelScope models.
9
10
```python { .api }
11
class Model:
12
"""
13
Base model interface for all ModelScope models.
14
"""
15
16
@classmethod
17
def from_pretrained(
18
cls,
19
model_name_or_path: str,
20
revision: Optional[str] = DEFAULT_MODEL_REVISION,
21
cfg_dict: Config = None,
22
device: str = None,
23
trust_remote_code: Optional[bool] = False,
24
**kwargs
25
):
26
"""
27
Load a pre-trained model from ModelScope Hub or local path.
28
29
Parameters:
30
- model_name_or_path: Model identifier on ModelScope Hub or local directory path
31
- revision: Model revision/version to load (default: DEFAULT_MODEL_REVISION)
32
- cfg_dict: Configuration dictionary for the model
33
- device: Target device ('cpu', 'cuda', 'gpu')
34
- trust_remote_code: Whether to trust and execute remote code in the model
35
- **kwargs: Additional model-specific parameters
36
37
Returns:
38
Initialized model instance
39
"""
40
41
def forward(self, inputs):
42
"""
43
Run forward pass through the model.
44
45
Parameters:
46
- inputs: Model inputs (format depends on model type)
47
48
Returns:
49
Model outputs
50
"""
51
52
def __call__(self, inputs):
53
"""
54
Callable interface for model inference.
55
56
Parameters:
57
- inputs: Model inputs
58
59
Returns:
60
Model outputs
61
"""
62
63
def postprocess(self, inputs):
64
"""
65
Post-process model outputs into user-friendly format.
66
67
Parameters:
68
- inputs: Raw model outputs
69
70
Returns:
71
Processed outputs
72
"""
73
74
def to(self, device: str):
75
"""
76
Move model to specified device.
77
78
Parameters:
79
- device: Target device ('cpu', 'cuda', 'gpu')
80
81
Returns:
82
Self (for method chaining)
83
"""
84
85
def eval(self):
86
"""
87
Set model to evaluation mode.
88
89
Returns:
90
Self (for method chaining)
91
"""
92
93
def train(self):
94
"""
95
Set model to training mode.
96
97
Returns:
98
Self (for method chaining)
99
"""
100
```
101
102
### PyTorch Model Class
103
104
PyTorch-specific model implementation with additional PyTorch features.
105
106
```python { .api }
107
class TorchModel(Model):
108
"""
109
PyTorch-specific model implementation extending base Model class.
110
"""
111
112
def __init__(self, model_dir: str, device: str = None, **kwargs):
113
"""
114
Initialize PyTorch model.
115
116
Parameters:
117
- model_dir: Directory containing model files
118
- device: Target device for model
119
- **kwargs: Additional PyTorch-specific parameters
120
"""
121
122
def save_pretrained(self, save_directory: str):
123
"""
124
Save model to local directory.
125
126
Parameters:
127
- save_directory: Directory to save model files
128
"""
129
130
def load_state_dict(self, state_dict: dict):
131
"""
132
Load model weights from state dictionary.
133
134
Parameters:
135
- state_dict: PyTorch state dictionary
136
"""
137
138
def state_dict(self) -> dict:
139
"""
140
Get model state dictionary.
141
142
Returns:
143
PyTorch state dictionary containing model weights
144
"""
145
```
146
147
### Model Builder
148
149
Factory functions for creating models dynamically based on configuration.
150
151
```python { .api }
152
def build_model(cfg: dict, default_args: dict = None) -> Model:
153
"""
154
Build model from configuration dictionary.
155
156
Parameters:
157
- cfg: Configuration dictionary specifying model architecture and parameters
158
- default_args: Default arguments to merge with configuration
159
160
Returns:
161
Initialized model instance
162
"""
163
164
# Model registries for different components
165
MODELS: dict # Registry of available model architectures
166
BACKBONES: dict # Registry of backbone networks
167
HEADS: dict # Registry of model heads
168
```
169
170
### Model Heads
171
172
Specialized model heads for different tasks and architectures.
173
174
```python { .api }
175
class Head:
176
"""
177
Base class for model heads (task-specific output layers).
178
"""
179
180
def __init__(self, **kwargs):
181
"""Initialize model head with task-specific parameters."""
182
183
def forward(self, features):
184
"""
185
Process backbone features through the head.
186
187
Parameters:
188
- features: Feature tensors from backbone network
189
190
Returns:
191
Task-specific outputs
192
"""
193
194
class TorchHead(Head):
195
"""
196
PyTorch-specific model head implementation.
197
"""
198
pass
199
```
200
201
## Usage Examples
202
203
### Basic Model Loading and Inference
204
205
```python
206
from modelscope import Model
207
208
# Load model from ModelScope Hub
209
model = Model.from_pretrained('damo/nlp_structbert_sentence-similarity_chinese')
210
211
# Set to evaluation mode
212
model.eval()
213
214
# Run inference
215
inputs = "这是一个测试文本"
216
outputs = model(inputs)
217
print(outputs)
218
219
# Move to GPU if available
220
model.to('cuda')
221
```
222
223
### Custom Model Configuration
224
225
```python
226
from modelscope import Model
227
228
# Load model with custom configuration
229
model = Model.from_pretrained(
230
'model_name',
231
device='cuda',
232
torch_dtype='float16', # Use half precision
233
trust_remote_code=True, # Allow custom model code
234
revision='v1.0.0' # Specific model version
235
)
236
237
# Custom preprocessing
238
def preprocess_inputs(text):
239
# Custom preprocessing logic
240
return processed_text
241
242
# Run inference with preprocessing
243
raw_input = "原始文本"
244
processed_input = preprocess_inputs(raw_input)
245
output = model(processed_input)
246
```
247
248
### Model Fine-tuning Setup
249
250
```python
251
from modelscope import TorchModel
252
253
# Load model for fine-tuning
254
model = TorchModel.from_pretrained('base_model_name')
255
256
# Set to training mode
257
model.train()
258
259
# Access model parameters for optimizer
260
parameters = model.parameters()
261
262
# Example training loop setup
263
import torch.optim as optim
264
optimizer = optim.Adam(parameters, lr=1e-5)
265
266
# Save fine-tuned model
267
model.save_pretrained('./fine_tuned_model')
268
```
269
270
### Building Custom Models
271
272
```python
273
from modelscope import build_model, MODELS
274
275
# Define model configuration
276
model_config = {
277
'type': 'BertModel',
278
'vocab_size': 30000,
279
'hidden_size': 768,
280
'num_hidden_layers': 12,
281
'num_attention_heads': 12
282
}
283
284
# Build model from configuration
285
model = build_model(model_config)
286
287
# Register custom model architecture
288
@MODELS.register_module()
289
class CustomModel(Model):
290
def __init__(self, **kwargs):
291
super().__init__()
292
# Custom model implementation
293
294
def forward(self, inputs):
295
# Custom forward pass
296
return outputs
297
```
298
299
### Multi-GPU and Distributed Inference
300
301
```python
302
from modelscope import TorchModel
303
import torch
304
305
# Load model
306
model = TorchModel.from_pretrained('large_model_name')
307
308
# Data parallel across multiple GPUs
309
if torch.cuda.device_count() > 1:
310
model = torch.nn.DataParallel(model)
311
312
# Move to GPU
313
model.to('cuda')
314
315
# Distributed inference
316
model.eval()
317
with torch.no_grad():
318
outputs = model(batch_inputs)
319
```
320
321
### Model Inspection and Analysis
322
323
```python
324
from modelscope import Model
325
326
# Load model
327
model = Model.from_pretrained('model_name')
328
329
# Inspect model architecture
330
print(f"Model type: {type(model)}")
331
print(f"Model parameters: {sum(p.numel() for p in model.parameters())}")
332
333
# Get model configuration
334
config = model.config
335
print(f"Model config: {config}")
336
337
# Access specific model components
338
if hasattr(model, 'backbone'):
339
print(f"Backbone: {model.backbone}")
340
if hasattr(model, 'head'):
341
print(f"Head: {model.head}")
342
```
343
344
### Model Export and Conversion
345
346
```python
347
from modelscope import TorchModel
348
349
# Load PyTorch model
350
model = TorchModel.from_pretrained('model_name')
351
352
# Export to ONNX
353
model.eval()
354
torch.onnx.export(
355
model,
356
example_input,
357
'model.onnx',
358
export_params=True,
359
opset_version=11
360
)
361
362
# Save model state for later loading
363
torch.save(model.state_dict(), 'model_weights.pth')
364
365
# Load weights later
366
new_model = TorchModel.from_pretrained('model_name')
367
new_model.load_state_dict(torch.load('model_weights.pth'))
368
```
369
370
### Conditional Model Loading
371
372
```python
373
from modelscope import Model
374
from modelscope.utils.import_utils import is_torch_available
375
376
# Conditional loading based on available frameworks
377
if is_torch_available():
378
from modelscope import TorchModel
379
model = TorchModel.from_pretrained('pytorch_model')
380
else:
381
# Fallback to base model or alternative implementation
382
model = Model.from_pretrained('base_model')
383
384
# Handle different model formats
385
try:
386
model = Model.from_pretrained('model_name', format='pytorch')
387
except Exception:
388
try:
389
model = Model.from_pretrained('model_name', format='tensorflow')
390
except Exception:
391
model = Model.from_pretrained('model_name') # Default format
392
```
393
394
### Model Caching and Optimization
395
396
```python
397
from modelscope import Model
398
399
# Enable model caching for faster loading
400
model = Model.from_pretrained(
401
'model_name',
402
cache_dir='./model_cache',
403
local_files_only=False # Allow downloading if not cached
404
)
405
406
# Load with optimization flags
407
model = Model.from_pretrained(
408
'model_name',
409
torch_dtype='float16', # Half precision
410
low_cpu_mem_usage=True, # Optimize CPU memory usage
411
device_map='auto' # Automatic device mapping
412
)
413
414
# Compile model for better performance (PyTorch 2.0+)
415
if hasattr(torch, 'compile'):
416
model = torch.compile(model)
417
```