0
# Hub Operations
1

2
ModelScope Hub integration provides seamless access to the ModelScope ecosystem for downloading, uploading, and managing models and datasets. The Hub API enables model discovery, version control, and collaborative development workflows.
3

4
## Capabilities
5

6
### Model Download
7

8
Download complete model repositories or individual files from ModelScope Hub.
9

10
```python { .api }
11
def snapshot_download(
12
    model_id: str = None,
13
    revision: Optional[str] = None,
14
    cache_dir: Union[str, Path, None] = None,
15
    user_agent: Optional[Union[Dict, str]] = None,
16
    local_files_only: Optional[bool] = False,
17
    cookies: Optional[CookieJar] = None,
18
    ignore_file_pattern: Optional[Union[str, List[str]]] = None,
19
    allow_file_pattern: Optional[Union[str, List[str]]] = None,
20
    local_dir: Optional[str] = None,
21
    allow_patterns: Optional[Union[List[str], str]] = None,
22
    ignore_patterns: Optional[Union[List[str], str]] = None,
23
    max_workers: int = 8,
24
    repo_id: str = None,
25
    repo_type: Optional[str] = REPO_TYPE_MODEL,
26
    enable_file_lock: Optional[bool] = None,
27
    progress_callbacks: List[Type[ProgressCallback]] = None,
28
) -> str:
29
    """
30
    Download a complete model repository from ModelScope Hub.
31
    
32
    Parameters:
33
    - model_id: ModelScope model identifier (e.g., 'damo/nlp_structbert_sentence-similarity_chinese')
34
    - revision: Model version/revision to download (default: latest)
35
    - cache_dir: Local cache directory path (Union[str, Path, None])
36
    - user_agent: User agent for HTTP requests (dict or string)
37
    - local_files_only: Only use local cached files
38
    - cookies: HTTP cookies for authenticated requests
39
    - ignore_file_pattern: File patterns to ignore during download
40
    - allow_file_pattern: File patterns to allow during download
41
    - local_dir: Local directory to download files to
42
    - allow_patterns: List of file patterns to include (Union[List[str], str])
43
    - ignore_patterns: List of file patterns to exclude (Union[List[str], str])
44
    - max_workers: Maximum number of concurrent download workers
45
    - repo_id: Repository identifier
46
    - repo_type: Type of repository (REPO_TYPE_MODEL by default)
47
    - enable_file_lock: Enable file locking during download
48
    - progress_callbacks: List of progress callback functions
49
    
50
    Returns:
51
    Local path to downloaded model directory
52
    """
53

54
def model_file_download(
55
    model_id: str,
56
    file_path: str,
57
    cache_dir: str = None,
58
    local_files_only: bool = False,
59
    **kwargs
60
) -> str:
61
    """
62
    Download a single file from a model repository.
63
    
64
    Parameters:
65
    - model_id: ModelScope model identifier
66
    - file_path: Path to file within the model repository
67
    - cache_dir: Local cache directory path
68
    - local_files_only: Only use local cached files
69
    
70
    Returns:
71
    Local path to downloaded file
72
    """
73
```
74

75
### Dataset Download
76

77
Download datasets from ModelScope Dataset Hub.
78

79
```python { .api }
80
def dataset_snapshot_download(
81
    dataset_id: str,
82
    revision: str = None,
83
    cache_dir: str = None,
84
    local_files_only: bool = False,
85
    **kwargs
86
) -> str:
87
    """
88
    Download a complete dataset repository from ModelScope Hub.
89
    
90
    Parameters:
91
    - dataset_id: ModelScope dataset identifier
92
    - revision: Dataset version/revision to download
93
    - cache_dir: Local cache directory path
94
    - local_files_only: Only use local cached files
95
    
96
    Returns:
97
    Local path to downloaded dataset directory
98
    """
99

100
def dataset_file_download(
101
    dataset_id: str,
102
    file_path: str,
103
    cache_dir: str = None,
104
    **kwargs
105
) -> str:
106
    """
107
    Download a single file from a dataset repository.
108
    
109
    Parameters:
110
    - dataset_id: ModelScope dataset identifier
111
    - file_path: Path to file within the dataset repository
112
    - cache_dir: Local cache directory path
113
    
114
    Returns:
115
    Local path to downloaded file
116
    """
117
```
118

119
### Model Upload
120

121
Upload models and datasets to ModelScope Hub.
122

123
```python { .api }
124
def push_to_hub(
125
    repo_name: str,
126
    output_dir: str,
127
    token: str = None,
128
    private: bool = True,
129
    retry: int = 3,
130
    commit_message: str = '',
131
    tag: str = None,
132
    source_repo: str = '',
133
    ignore_file_pattern: list = None,
134
    revision: str = DEFAULT_REPOSITORY_REVISION
135
):
136
    """
137
    Upload a model or dataset to ModelScope Hub.
138
    
139
    Parameters:
140
    - repo_name: Repository name on ModelScope Hub
141
    - output_dir: Local directory containing files to upload
142
    - token: Authentication token (optional)
143
    - private: Whether to create a private repository (default: True)
144
    - retry: Number of retry attempts on failure (default: 3)
145
    - commit_message: Commit message for the upload (default: empty string)
146
    - tag: Git tag for the upload (optional)
147
    - source_repo: Source repository URL (default: empty string)
148
    - ignore_file_pattern: List of file patterns to ignore during upload
149
    - revision: Repository revision (default: DEFAULT_REPOSITORY_REVISION)
150
    """
151

152
def push_to_hub_async(
153
    repo_name: str,
154
    output_dir: str,
155
    **kwargs
156
):
157
    """
158
    Asynchronously upload a model or dataset to ModelScope Hub.
159
    
160
    Parameters:
161
    - repo_name: Repository name on ModelScope Hub
162
    - output_dir: Local directory containing files to upload
163
    """
164
```
165

166
### Hub API
167

168
Comprehensive API interface for advanced Hub operations.
169

170
```python { .api }
171
class HubApi:
172
    """
173
    Main API interface for ModelScope Hub operations.
174
    """
175
    
176
    def __init__(
177
        self,
178
        endpoint: Optional[str] = None,
179
        timeout = API_HTTP_CLIENT_TIMEOUT,
180
        max_retries = API_HTTP_CLIENT_MAX_RETRIES
181
    ):
182
        """
183
        Initialize Hub API client.
184
        
185
        Parameters:
186
        - endpoint: ModelScope Hub endpoint URL (optional)
187
        - timeout: HTTP client timeout duration
188
        - max_retries: Maximum number of HTTP request retries
189
        """
190
    
191
    def login(
192
        self,
193
        access_token: Optional[str] = None,
194
        endpoint: Optional[str] = None
195
    ):
196
        """
197
        Authenticate with ModelScope Hub.
198
        
199
        Parameters:
200
        - access_token: Authentication access token (optional)
201
        - endpoint: Hub endpoint URL (optional)
202
        """
203
    
204
    def get_cookies(
205
        self,
206
        access_token: str,
207
        cookies_required: Optional[bool] = False
208
    ):
209
        """
210
        Get authentication cookies from access token.
211
        
212
        Parameters:
213
        - access_token: Authentication access token
214
        - cookies_required: Whether cookies are required for the operation
215
        """
216
    
217
    def logout(self):
218
        """
219
        Log out from ModelScope Hub.
220
        """
221
    
222
    def create_model(
223
        self,
224
        model_name: str,
225
        visibility: int = 1,
226
        license: str = None,
227
        **kwargs
228
    ):
229
        """
230
        Create a new model repository.
231
        
232
        Parameters:
233
        - model_name: Name of the model repository
234
        - visibility: Repository visibility (1=public, 0=private)
235
        - license: License type for the model
236
        """
237
    
238
    def create_dataset(
239
        self,
240
        dataset_name: str,
241
        visibility: int = 1,
242
        **kwargs
243
    ):
244
        """
245
        Create a new dataset repository.
246
        
247
        Parameters:
248
        - dataset_name: Name of the dataset repository
249
        - visibility: Repository visibility (1=public, 0=private)
250
        """
251
    
252
    def push_model(
253
        self,
254
        model_id: str,
255
        model_dir: str,
256
        **kwargs
257
    ):
258
        """
259
        Push model files to repository.
260
        
261
        Parameters:
262
        - model_id: Target model repository identifier
263
        - model_dir: Local directory containing model files
264
        """
265
    
266
    def list_models(
267
        self,
268
        owner: str = None,
269
        page_number: int = 1,
270
        page_size: int = 10,
271
        **kwargs
272
    ) -> list:
273
        """
274
        List models from ModelScope Hub.
275
        
276
        Parameters:
277
        - owner: Filter by model owner
278
        - page_number: Page number for pagination
279
        - page_size: Number of models per page
280
        
281
        Returns:
282
        List of model information dictionaries
283
        """
284
    
285
    def get_model_info(self, model_id: str) -> dict:
286
        """
287
        Get detailed information about a model.
288
        
289
        Parameters:
290
        - model_id: ModelScope model identifier
291
        
292
        Returns:
293
        Dictionary containing model metadata
294
        """
295
```
296

297
### Model Validation
298

299
Utilities for validating model identifiers and checking model status.
300

301
```python { .api }
302
def check_model_is_id(model_id: str, token: str = None) -> bool:
303
    """
304
    Validate if a string is a valid ModelScope model identifier.
305
    
306
    Parameters:
307
    - model_id: String to validate as model identifier
308
    - token: Authentication token (optional)
309
    
310
    Returns:
311
    True if valid model ID, False otherwise
312
    """
313

314
def check_local_model_is_latest(
315
    local_path: str,
316
    model_id: str,
317
    token: str = None
318
) -> bool:
319
    """
320
    Check if local model version matches the latest version on Hub.
321
    
322
    Parameters:
323
    - local_path: Path to local model directory
324
    - model_id: ModelScope model identifier
325
    - token: Authentication token (optional)
326
    
327
    Returns:
328
    True if local model is up to date, False otherwise
329
    """
330
```
331

332
### Configuration and Repository Management
333

334
Classes for configuration management and repository operations.
335

336
```python { .api }
337
class ModelScopeConfig:
338
    """
339
    Configuration management for ModelScope Hub operations.
340
    """
341
    
342
    def __init__(self):
343
        """Initialize configuration manager."""
344
    
345
    def get_token(self) -> str:
346
        """Get authentication token."""
347
    
348
    def save_token(self, token: str):
349
        """Save authentication token."""
350

351
class Repository:
352
    """
353
    Git repository management for ModelScope repositories.
354
    """
355
    
356
    def __init__(self, local_dir: str, clone_from: str = None):
357
        """
358
        Initialize repository manager.
359
        
360
        Parameters:
361
        - local_dir: Local directory for the repository
362
        - clone_from: Remote repository URL to clone from
363
        """
364
    
365
    def git_add(self, pattern: str = None):
366
        """Add files to git staging area."""
367
    
368
    def git_commit(self, message: str):
369
        """Commit changes to repository."""
370
    
371
    def git_push(self):
372
        """Push changes to remote repository."""
373

374
class DatasetRepository(Repository):
375
    """
376
    Dataset-specific repository operations.
377
    """
378
    pass
379
```
380

381
### Constants and Enums
382

383
```python { .api }
384
class ModelVisibility:
385
    PUBLIC = 5
386
    ORGANIZATION = 3
387
    PRIVATE = 1
388

389
class DatasetVisibility:
390
    PUBLIC = 5
391
    ORGANIZATION = 3  
392
    PRIVATE = 1
393

394
class Licenses:
395
    APACHE_2_0 = "Apache License 2.0"
396
    MIT = "MIT"
397
    GPL_3_0 = "GPL-3.0"
398
    # Additional license constants available
399
```
400

401
## Usage Examples
402

403
### Basic Model Download
404

405
```python
406
from modelscope import snapshot_download
407

408
# Download a text classification model
409
model_dir = snapshot_download('damo/nlp_structbert_sentence-similarity_chinese')
410
print(f"Model downloaded to: {model_dir}")
411

412
# Download specific version
413
model_dir = snapshot_download(
414
    'damo/nlp_structbert_sentence-similarity_chinese',
415
    revision='v1.0.0'
416
)
417
```
418

419
### Upload Custom Model
420

421
```python
422
from modelscope import HubApi, push_to_hub
423

424
# Initialize Hub API and login
425
api = HubApi()
426
api.login('your_token_here')
427

428
# Create model repository
429
api.create_model('my-awesome-model', visibility=1)
430

431
# Upload model files
432
push_to_hub(
433
    repo_name='my-awesome-model',
434
    output_dir='./my_model_directory',
435
    commit_message='Initial model upload'
436
)
437
```
438

439
### Check Model Status
440

441
```python
442
from modelscope import check_model_is_id, check_local_model_is_latest
443

444
# Validate model ID
445
is_valid = check_model_is_id('damo/nlp_structbert_sentence-similarity_chinese')
446

447
# Check if local model is up to date
448
is_latest = check_local_model_is_latest(
449
    './local_model_dir',
450
    'damo/nlp_structbert_sentence-similarity_chinese'
451
)
452
```