or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

datasets.mdexport.mdhub.mdindex.mdmetrics.mdmodels.mdpipelines.mdpreprocessors.mdtraining.mdutilities.md
tile.json

tessl/pypi-modelscope

ModelScope brings the notion of Model-as-a-Service to life with unified interfaces for state-of-the-art machine learning models.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/modelscope@1.29.x

To install, run

npx @tessl/cli install tessl/pypi-modelscope@1.29.0

index.mddocs/

ModelScope

ModelScope is a comprehensive Python library that implements Model-as-a-Service (MaaS) architecture, providing unified interfaces for exploring, using, and integrating state-of-the-art machine learning models across multiple domains including Computer Vision, Natural Language Processing, Speech, Multi-Modality, and Scientific Computing.

Package Information

  • Package Name: modelscope
  • Package Type: pypi
  • Language: Python
  • Installation: pip install modelscope

Core Imports

import modelscope
from modelscope import pipeline, Model, snapshot_download

Common patterns for specific functionality:

# Hub operations
from modelscope import HubApi, snapshot_download, push_to_hub

# Pipeline usage (recommended approach)
from modelscope import pipeline

# Model and training
from modelscope import Model, EpochBasedTrainer, TrainingArgs

# Preprocessors and metrics
from modelscope import Preprocessor, Metric

# Dataset operations
from modelscope import MsDataset

Basic Usage

Simple Pipeline Usage (Recommended)

from modelscope import pipeline

# Create a pipeline for a specific task
pipe = pipeline('text-classification', model='nlp_structbert_sentence-similarity_chinese')

# Use the pipeline
result = pipe('这是一个测试文本')
print(result)

# Computer vision example
cv_pipe = pipeline('image-classification', model='damo/cv_resnet50_image-classification_imagenet')
result = cv_pipe('path/to/image.jpg')

Model Download and Usage

from modelscope import snapshot_download, Model

# Download a model
model_dir = snapshot_download('nlp_structbert_sentence-similarity_chinese')

# Load the model
model = Model.from_pretrained(model_dir)

# Use the model
output = model(input_data)

Training a Model

from modelscope import EpochBasedTrainer, TrainingArgs
from modelscope import build_dataset_from_file

# Prepare dataset
dataset = build_dataset_from_file('path/to/train.json')

# Configure training
training_args = TrainingArgs(
    output_dir='./output',
    max_epochs=10,
    eval_strategy='epoch'
)

# Create trainer
trainer = EpochBasedTrainer(
    model=model,
    args=training_args,
    train_dataset=dataset
)

# Start training
trainer.train()

Architecture

ModelScope is built on a layered architecture that provides multiple levels of abstraction:

  • Pipeline Layer: High-level task-oriented interfaces that handle preprocessing, inference, and postprocessing automatically
  • Model Layer: Direct model interfaces for custom workflows and fine-grained control
  • Component Layer: Individual components (preprocessors, models, postprocessors) for maximum customization
  • Hub Layer: Integration with ModelScope backend services for model and dataset management

The library seamlessly integrates with ModelScope Hub for model discovery, version control, and collaborative development, while also supporting local model development and custom implementations.

Capabilities

Hub Operations

ModelScope Hub integration for downloading, uploading, and managing models and datasets from the ModelScope ecosystem.

def snapshot_download(model_id: str, revision: str = None, cache_dir: str = None, **kwargs) -> str: ...
def push_to_hub(repo_name: str, output_dir: str, **kwargs): ...
def model_file_download(model_id: str, file_path: str, **kwargs) -> str: ...

class HubApi:
    def login(self, token: str): ...
    def create_model(self, model_name: str, **kwargs): ...
    def list_models(self, **kwargs): ...

Hub Operations

Pipeline Interface

High-level task-oriented pipeline interface that provides the easiest way to use pre-trained models for inference across different domains.

def pipeline(task: str, model: str = None, preprocessor=None, **kwargs) -> Pipeline: ...

class Pipeline:
    def __call__(self, inputs, **kwargs): ...
    def preprocess(self, inputs): ...
    def forward(self, inputs): ...
    def postprocess(self, inputs): ...

Pipeline Interface

Model Interface

Direct model interfaces for loading, using, and customizing pre-trained models with fine-grained control over the inference process.

class Model:
    @classmethod
    def from_pretrained(cls, model_name_or_path: str, **kwargs): ...
    def forward(self, inputs): ...
    def __call__(self, inputs): ...

class TorchModel(Model):
    # PyTorch-specific model implementation
    pass

Model Interface

Training Framework

Comprehensive training framework supporting epoch-based training with hooks, metrics, and evaluation capabilities.

class EpochBasedTrainer:
    def __init__(self, model, args: TrainingArgs, **kwargs): ...
    def train(self): ...
    def evaluate(self): ...
    def save_checkpoint(self, checkpoint_dir: str): ...

class TrainingArgs:
    def __init__(self, output_dir: str, max_epochs: int = 1, **kwargs): ...

def build_dataset_from_file(data_files, **kwargs): ...

Training Framework

Metrics and Evaluation

Comprehensive metrics framework for evaluating model performance across different tasks and domains.

class Metric:
    def add(self, outputs, inputs): ...
    def evaluate(self): ...
    def merge(self, other): ...

def task_default_metrics(task: str) -> list: ...

# Specialized metrics
class AccuracyMetric(Metric): ...
class BleuMetric(Metric): ...
class ImageQualityAssessmentMetric(Metric): ...

Metrics and Evaluation

Preprocessors

Data preprocessing components for different modalities and tasks, providing consistent data preparation pipelines.

class Preprocessor:
    def __call__(self, data): ...
    def forward(self, data): ...

# Common preprocessors
class Compose(Preprocessor): ...
class ToTensor(Preprocessor): ...
class LoadImage(Preprocessor): ...

Preprocessors

Datasets

Dataset handling and manipulation utilities for working with ModelScope datasets and local data.

class MsDataset:
    @classmethod
    def load(cls, dataset_name: str, subset_name: str = None, **kwargs): ...
    def to_hf_dataset(self): ...
    def map(self, function): ...
    def filter(self, function): ...
    def split(self, test_size: float): ...

Datasets

Model Export

Model export capabilities for deploying models to different formats and target platforms.

class Exporter:
    def export(self, model, output_dir: str): ...

class TorchModelExporter(Exporter): ...
class TfModelExporter(Exporter): ...

Model Export

Utilities and Constants

Core utilities, logging, configuration management, and task constants used throughout the ModelScope ecosystem.

class Tasks:
    # Task constants for different domains
    pass

def get_logger(log_file: str = None, log_level=INFO) -> Logger: ...
def read_config(path: str) -> dict: ...

Utilities