0
# Pipeline Interface
1

2
The Pipeline interface provides the highest-level and most convenient way to use pre-trained models in ModelScope. Pipelines abstract away the complexity of preprocessing, model inference, and postprocessing, offering a simple, task-oriented interface for various AI capabilities.
3

4
## Capabilities
5

6
### Pipeline Factory
7

8
The main pipeline factory function that creates task-specific pipeline instances.
9

10
```python { .api }
11
def pipeline(
12
    task: str = None,
13
    model: Union[str, List[str], Model, List[Model]] = None,
14
    preprocessor = None,
15
    config_file: str = None,
16
    pipeline_name: str = None,
17
    framework: str = None,
18
    device: str = None,
19
    model_revision: Optional[str] = DEFAULT_MODEL_REVISION,
20
    ignore_file_pattern: List[str] = None,
21
    **kwargs
22
) -> Pipeline:
23
    """
24
    Create a pipeline for a specific AI task.
25
    
26
    Parameters:
27
    - task: Task identifier (e.g., 'text-classification', 'image-classification')
28
    - model: Model identifier(s), local path(s), or Model instance(s) - supports single values or lists
29
    - preprocessor: Custom preprocessor instance
30
    - config_file: Path to configuration file
31
    - pipeline_name: Name for the pipeline
32
    - framework: ML framework to use ('pytorch', 'tensorflow', etc.)
33
    - device: Device to run inference on ('cpu', 'cuda', 'gpu')
34
    - model_revision: Specific model version/revision (default: DEFAULT_MODEL_REVISION)
35
    - ignore_file_pattern: List of file patterns to ignore during model loading
36
    - **kwargs: Additional task-specific parameters
37
    
38
    Returns:
39
    Task-specific pipeline instance
40
    """
41
```
42

43
### Base Pipeline Class
44

45
Abstract base class for all pipelines providing common interface and functionality.
46

47
```python { .api }
48
class Pipeline:
49
    """
50
    Base pipeline class providing common interface for all AI tasks.
51
    """
52
    
53
    def __init__(
54
        self,
55
        model,
56
        preprocessor = None,
57
        config_file: str = None,
58
        device: str = None,
59
        auto_collate: bool = True,
60
        **kwargs
61
    ):
62
        """
63
        Initialize pipeline with model and preprocessing components.
64
        
65
        Parameters:
66
        - model: Model instance or identifier
67
        - preprocessor: Preprocessor instance
68
        - config_file: Path to configuration file
69
        - device: Target device for inference
70
        - auto_collate: Whether to automatically collate batch inputs
71
        """
72
    
73
    def __call__(self, inputs, **kwargs):
74
        """
75
        Process inputs through the complete pipeline.
76
        
77
        Parameters:
78
        - inputs: Input data (format varies by task)
79
        - **kwargs: Task-specific parameters
80
        
81
        Returns:
82
        Processed outputs in task-specific format
83
        """
84
    
85
    def preprocess(self, inputs):
86
        """
87
        Preprocess raw inputs for model consumption.
88
        
89
        Parameters:
90
        - inputs: Raw input data
91
        
92
        Returns:
93
        Preprocessed data ready for model inference
94
        """
95
    
96
    def forward(self, inputs):
97
        """
98
        Run model inference on preprocessed inputs.
99
        
100
        Parameters:
101
        - inputs: Preprocessed input data
102
        
103
        Returns:
104
        Raw model outputs
105
        """
106
    
107
    def postprocess(self, inputs):
108
        """
109
        Process model outputs into final user-friendly format.
110
        
111
        Parameters:
112
        - inputs: Raw model outputs
113
        
114
        Returns:
115
        Final processed outputs
116
        """
117
```
118

119
## Task Categories and Examples
120

121
### Natural Language Processing
122

123
Text processing tasks including classification, generation, translation, and analysis.
124

125
```python
126
# Text Classification
127
from modelscope import pipeline
128

129
classifier = pipeline('text-classification', 
130
                     model='damo/nlp_structbert_sentence-similarity_chinese')
131
result = classifier('这是一个积极的评论')
132

133
# Text Generation  
134
generator = pipeline('text-generation',
135
                    model='damo/nlp_gpt3_text-generation_chinese')
136
result = generator('人工智能的未来发展', max_length=100)
137

138
# Named Entity Recognition
139
ner = pipeline('named-entity-recognition',
140
              model='damo/nlp_raner_named-entity-recognition_chinese')
141
result = ner('张三在北京工作')
142

143
# Text Summarization
144
summarizer = pipeline('summarization',
145
                     model='damo/nlp_pegasus_summarization_chinese')
146
result = summarizer('长篇文本内容...')
147
```
148

149
### Computer Vision
150

151
Image and video processing tasks including classification, detection, and generation.
152

153
```python
154
# Image Classification
155
classifier = pipeline('image-classification',
156
                     model='damo/cv_resnet50_image-classification_imagenet')
157
result = classifier('path/to/image.jpg')
158

159
# Object Detection
160
detector = pipeline('object-detection',
161
                   model='damo/cv_yolox-s_detection_coco')
162
result = detector('path/to/image.jpg')
163

164
# Image Segmentation
165
segmenter = pipeline('image-segmentation',
166
                    model='damo/cv_resnet101_image-multiple-human-parsing')
167
result = segmenter('path/to/image.jpg')
168

169
# Face Recognition
170
face_recognition = pipeline('face-recognition',
171
                           model='damo/cv_ir50_face-recognition_faceid')
172
result = face_recognition('path/to/face.jpg')
173
```
174

175
### Audio Processing
176

177
Speech and audio processing tasks including recognition, synthesis, and enhancement.
178

179
```python
180
# Speech Recognition
181
asr = pipeline('automatic-speech-recognition',
182
              model='damo/speech_paraformer-large_asr_nat-zh-cn')
183
result = asr('path/to/audio.wav')
184

185
# Text-to-Speech
186
tts = pipeline('text-to-speech',
187
              model='damo/speech_sambert-hifigan_tts_zh-cn_16k')
188
result = tts('你好，欢迎使用ModelScope')
189

190
# Audio Classification
191
audio_classifier = pipeline('audio-classification',
192
                           model='damo/speech_xvector_sv-zh-cn-cnceleb-16k')
193
result = audio_classifier('path/to/audio.wav')
194
```
195

196
### Multi-Modal
197

198
Tasks that work with multiple modalities like text, images, and audio.
199

200
```python
201
# Image Captioning
202
captioner = pipeline('image-captioning',
203
                    model='damo/multi-modal_clip-vit-base-patch16_zh')
204
result = captioner('path/to/image.jpg')
205

206
# Visual Question Answering
207
vqa = pipeline('visual-question-answering',
208
              model='damo/multi-modal_clip-vit-base-patch16_zh')
209
result = vqa(image='path/to/image.jpg', question='图片中有什么？')
210

211
# Text-to-Image Generation
212
text2img = pipeline('text-to-image-synthesis',
213
                   model='damo/multi-modal_latent-diffusion_text-to-image-synthesis')
214
result = text2img('一只可爱的小猫')
215
```
216

217
## Pipeline Configuration
218

219
### Device Management
220

221
```python
222
from modelscope import pipeline
223

224
# CPU inference
225
pipe_cpu = pipeline('text-classification', 
226
                   model='model_name',
227
                   device='cpu')
228

229
# GPU inference
230
pipe_gpu = pipeline('text-classification',
231
                   model='model_name', 
232
                   device='cuda')
233

234
# Automatic device selection
235
pipe_auto = pipeline('text-classification',
236
                    model='model_name')  # Automatically selects best available device
237
```
238

239
### Custom Preprocessors
240

241
```python
242
from modelscope import pipeline, Preprocessor
243

244
# Custom preprocessor
245
class MyPreprocessor(Preprocessor):
246
    def __call__(self, data):
247
        # Custom preprocessing logic
248
        return processed_data
249

250
# Use custom preprocessor
251
pipe = pipeline('text-classification',
252
               model='model_name',
253
               preprocessor=MyPreprocessor())
254
```
255

256
### Batch Processing
257

258
```python
259
# Single input
260
result = pipe('single input text')
261

262
# Batch inputs
263
results = pipe(['input 1', 'input 2', 'input 3'])
264

265
# Large batch processing with automatic collation
266
pipe = pipeline('text-classification', 
267
               model='model_name',
268
               auto_collate=True)
269
results = pipe(large_input_list)
270
```
271

272
## Task-Specific Parameters
273

274
Different pipeline tasks support specific parameters to customize behavior:
275

276
### Text Generation Parameters
277

278
```python
279
generator = pipeline('text-generation', model='model_name')
280

281
result = generator(
282
    'Input text',
283
    max_length=100,          # Maximum output length
284
    min_length=10,           # Minimum output length
285
    temperature=0.7,         # Sampling temperature
286
    top_k=50,               # Top-k sampling
287
    top_p=0.9,              # Top-p (nucleus) sampling
288
    repetition_penalty=1.2,  # Repetition penalty
289
    do_sample=True          # Enable sampling
290
)
291
```
292

293
### Image Processing Parameters
294

295
```python
296
detector = pipeline('object-detection', model='model_name')
297

298
result = detector(
299
    'path/to/image.jpg',
300
    score_thresh=0.5,        # Confidence threshold
301
    nms_thresh=0.3,          # Non-maximum suppression threshold
302
    max_det=100             # Maximum detections
303
)
304
```
305

306
### Audio Processing Parameters
307

308
```python
309
asr = pipeline('automatic-speech-recognition', model='model_name')
310

311
result = asr(
312
    'path/to/audio.wav',
313
    sample_rate=16000,       # Audio sample rate
314
    chunk_size=6000,         # Chunk size for long audio
315
    merge_chunks=True        # Merge chunk results
316
)
317
```
318

319
## Error Handling and Validation
320

321
```python
322
from modelscope import pipeline
323

324
try:
325
    # Create pipeline
326
    pipe = pipeline('text-classification', model='invalid_model')
327
except Exception as e:
328
    print(f"Pipeline creation failed: {e}")
329

330
try:
331
    # Process inputs
332
    result = pipe('input text')
333
except Exception as e:
334
    print(f"Processing failed: {e}")
335
```
336

337
## Advanced Usage
338

339
### Pipeline Chaining
340

341
```python
342
# Create multiple pipelines for complex workflows
343
preprocessor_pipe = pipeline('text-preprocessing', model='model1')
344
classifier_pipe = pipeline('text-classification', model='model2')
345

346
# Chain pipelines
347
preprocessed = preprocessor_pipe(raw_text)
348
final_result = classifier_pipe(preprocessed)
349
```
350

351
### Custom Pipeline Implementation
352

353
```python
354
from modelscope import Pipeline, Model, Preprocessor
355

356
class CustomTextPipeline(Pipeline):
357
    def __init__(self, model, preprocessor=None, **kwargs):
358
        super().__init__(model, preprocessor, **kwargs)
359
    
360
    def preprocess(self, inputs):
361
        # Custom preprocessing logic
362
        return processed_inputs
363
    
364
    def postprocess(self, inputs):
365
        # Custom postprocessing logic
366
        return final_outputs
367

368
# Register and use custom pipeline
369
pipeline = CustomTextPipeline(model=my_model, preprocessor=my_preprocessor)
370
result = pipeline('input text')
371
```