tessl/npm-xenova--transformers

State-of-the-art Machine Learning for the web that runs Transformers directly in browsers with no server needed.

Overview

Eval results

Files

Models and Tokenizers

Name: tessl/npm-xenova--transformers
Author: tessl

This module provides Auto classes for automatic model and tokenizer selection, as well as direct access to specific model implementations for fine-grained control over model loading and inference.

Capabilities

Auto Classes - Automatic Selection

Auto classes automatically select the appropriate model or tokenizer implementation based on the model configuration, providing the most convenient interface for most use cases.

AutoModel

Loads the appropriate model architecture automatically based on the model configuration.

/**
 * Instantiate a pretrained model automatically based on the model type
 * @param pretrained_model_name_or_path - Model identifier or path
 * @param options - Configuration options for model loading
 * @returns Promise resolving to the appropriate model instance
 */
class AutoModel {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

interface ModelOptions {
  /** Use quantized version of the model (default: true) */
  quantized?: boolean;
  /** Callback to track model download progress */
  progress_callback?: (progress: any) => void;
  /** Custom model configuration */
  config?: any;
  /** Directory to cache downloaded models */
  cache_dir?: string;
  /** Only use local files, don't download from remote */
  local_files_only?: boolean;
  /** Model revision/branch to use (default: 'main') */
  revision?: string;
  /** Specific model file name to use */
  model_file_name?: string;
}

Usage Example:

import { AutoModel, AutoTokenizer } from "@xenova/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/bert-base-uncased");
const model = await AutoModel.from_pretrained("Xenova/bert-base-uncased");

const inputs = await tokenizer("I love transformers!");
const outputs = await model(inputs);

AutoConfig

Automatically loads model configuration from pretrained models.

/**
 * Load model configuration automatically
 * @param pretrained_model_name_or_path - Model identifier or path
 * @param options - Configuration options for loading
 * @returns Promise resolving to model configuration
 */
class AutoConfig {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ConfigOptions
  ): Promise<PretrainedConfig>;
}

interface ConfigOptions {
  /** Directory to cache downloaded files */
  cache_dir?: string;
  /** Only use local files, don't download from remote */
  local_files_only?: boolean;
  /** Model revision/branch to use (default: 'main') */
  revision?: string;
}

interface PretrainedConfig {
  model_type: string;
  architectures?: string[];
  vocab_size?: number;
  hidden_size?: number;
  num_attention_heads?: number;
  num_hidden_layers?: number;
  max_position_embeddings?: number;
  [key: string]: any;
}

Usage Example:

import { AutoConfig } from "@xenova/transformers";

const config = await AutoConfig.from_pretrained("Xenova/bert-base-uncased");
console.log(config.model_type); // "bert"
console.log(config.vocab_size); // 30522

AutoTokenizer

Automatically selects and loads the appropriate tokenizer based on the tokenizer configuration.

/**
 * Instantiate a tokenizer automatically based on the tokenizer type
 * @param pretrained_model_name_or_path - Model identifier or path
 * @param options - Configuration options for tokenizer loading
 * @returns Promise resolving to the appropriate tokenizer instance
 */
class AutoTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<PreTrainedTokenizer>;
}

interface TokenizerOptions {
  /** Use quantized version (default: true) */
  quantized?: boolean;
  /** Callback to track download progress */
  progress_callback?: (progress: any) => void;
  /** Custom configuration */
  config?: any;
  /** Directory to cache downloaded files */
  cache_dir?: string;
  /** Only use local files, don't download from remote */
  local_files_only?: boolean;
  /** Model revision/branch to use (default: 'main') */
  revision?: string;
  /** Whether to use legacy tokenizer behavior */
  legacy?: boolean;
}

Usage Example:

import { AutoTokenizer } from "@xenova/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/t5-small");
const inputs = await tokenizer("translate English to German: Hello world");
const decoded = tokenizer.decode(inputs.input_ids[0]);

Task-Specific Auto Model Classes

These classes automatically load models optimized for specific tasks:

Text Processing Models

class AutoModelForSequenceClassification {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForTokenClassification {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForQuestionAnswering {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForMaskedLM {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForCausalLM {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

Sequence-to-Sequence Models

class AutoModelForSeq2SeqLM {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForVision2Seq {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

Vision Models

class AutoModelForImageClassification {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForImageSegmentation {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForSemanticSegmentation {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForObjectDetection {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForZeroShotObjectDetection {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForDepthEstimation {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForImageToImage {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForImageFeatureExtraction {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

Audio Models

class AutoModelForAudioClassification {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForSpeechSeq2Seq {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForCTC {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForAudioFrameClassification {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForXVector {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForTextToWaveform {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForTextToSpectrogram {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

Multimodal Models

class AutoModelForDocumentQuestionAnswering {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForImageMatting {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

class AutoModelForMaskGeneration {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: ModelOptions
  ): Promise<PreTrainedModel>;
}

Base Model Classes

PreTrainedModel

Base class for all model implementations providing core functionality for inference and resource management.

/**
 * Base class for all pretrained models
 */
class PreTrainedModel {
  /** Model configuration object */
  config: any;
  
  /**
   * Run forward pass through the model
   * @param model_inputs - Tokenized inputs or tensors
   * @returns Promise resolving to model outputs
   */
  async forward(model_inputs: any): Promise<any>;
  
  /**
   * Generate text using the model (for generation models)
   * @param inputs - Input token IDs
   * @param generation_config - Generation parameters
   * @returns Promise resolving to generated token sequences
   */
  async generate(
    inputs: Tensor,
    generation_config?: GenerationConfig
  ): Promise<Tensor[]>;
  
  /**
   * Dispose of model resources
   */
  async dispose(): Promise<void>;
}

interface GenerationConfig {
  /** Maximum number of new tokens to generate */
  max_new_tokens?: number;
  /** Maximum total length of generated sequence */
  max_length?: number;
  /** Minimum number of new tokens to generate */
  min_new_tokens?: number;
  /** Whether to use sampling for generation */
  do_sample?: boolean;
  /** Sampling temperature (0.0 to 1.0) */
  temperature?: number;
  /** Top-k sampling parameter */
  top_k?: number;
  /** Top-p (nucleus) sampling parameter */
  top_p?: number;
  /** Repetition penalty to avoid repetitive text */
  repetition_penalty?: number;
  /** Number of beams for beam search */
  num_beams?: number;
  /** Whether to use early stopping in beam search */
  early_stopping?: boolean;
}

PreTrainedTokenizer

Base class for all tokenizer implementations providing text encoding and decoding functionality.

/**
 * Base class for all pretrained tokenizers
 */
class PreTrainedTokenizer {
  /**
   * Tokenize and encode text input
   * @param text - Input text to tokenize
   * @param options - Tokenization options
   * @returns Tokenized output with input_ids and attention_mask
   */
  async encode(
    text: string | string[],
    options?: TokenizeOptions
  ): Promise<{
    input_ids: Tensor;
    attention_mask: Tensor;
    [key: string]: Tensor;
  }>;
  
  /**
   * Tokenize text (alias for encode)
   * @param text - Input text to tokenize
   * @param options - Tokenization options
   * @returns Tokenized output
   */
  async __call__(
    text: string | string[],
    options?: TokenizeOptions
  ): Promise<{
    input_ids: Tensor;
    attention_mask: Tensor;
    [key: string]: Tensor;
  }>;
  
  /**
   * Decode token IDs back to text
   * @param token_ids - Token IDs to decode
   * @param options - Decoding options
   * @returns Decoded text string
   */
  decode(
    token_ids: number[] | Tensor,
    options?: DecodeOptions
  ): string;
  
  /**
   * Decode multiple sequences of token IDs
   * @param sequences - Array of token ID sequences
   * @param options - Decoding options
   * @returns Array of decoded text strings
   */
  batch_decode(
    sequences: number[][] | Tensor[],
    options?: DecodeOptions
  ): string[];
  
  /**
   * Get the vocabulary size
   */
  get vocab_size(): number;
  
  /**
   * Dispose of tokenizer resources
   */
  async dispose(): Promise<void>;
}

interface TokenizeOptions {
  /** Add special tokens like [CLS], [SEP] */
  add_special_tokens?: boolean;
  /** Return attention mask */
  return_attention_mask?: boolean;
  /** Return token type IDs */
  return_token_type_ids?: boolean;
  /** Return tensor type ('pt' for PyTorch-like) */
  return_tensors?: string;
  /** Truncate to maximum length */
  truncation?: boolean;
  /** Maximum sequence length */
  max_length?: number;
  /** Padding strategy ('max_length', 'longest', etc.) */
  padding?: boolean | string;
}

interface DecodeOptions {
  /** Skip special tokens in decoded output */
  skip_special_tokens?: boolean;
  /** Clean up tokenization spaces */
  clean_up_tokenization_spaces?: boolean;
}

Specific Tokenizer Classes

The library includes numerous specific tokenizer implementations for different model architectures:

// BERT family tokenizers
class BertTokenizer extends PreTrainedTokenizer {}
class DistilBertTokenizer extends PreTrainedTokenizer {}
class RobertaTokenizer extends PreTrainedTokenizer {}
class AlbertTokenizer extends PreTrainedTokenizer {}

// Transformer tokenizers
class T5Tokenizer extends PreTrainedTokenizer {}
class GPT2Tokenizer extends PreTrainedTokenizer {}
class BartTokenizer extends PreTrainedTokenizer {}

// Multilingual tokenizers
class XLMTokenizer extends PreTrainedTokenizer {}
class XLMRobertaTokenizer extends PreTrainedTokenizer {}
class MBartTokenizer extends PreTrainedTokenizer {}
class MBart50Tokenizer extends PreTrainedTokenizer {}

// Audio tokenizers
class WhisperTokenizer extends PreTrainedTokenizer {}
class Wav2Vec2CTCTokenizer extends PreTrainedTokenizer {}
class SpeechT5Tokenizer extends PreTrainedTokenizer {}

// Vision-language tokenizers
class CLIPTokenizer extends PreTrainedTokenizer {}

// And many more specific implementations...

Usage Patterns

Basic Model and Tokenizer Usage

import { AutoModel, AutoTokenizer } from "@xenova/transformers";

// Load model and tokenizer
const tokenizer = await AutoTokenizer.from_pretrained("Xenova/bert-base-uncased");
const model = await AutoModel.from_pretrained("Xenova/bert-base-uncased");

// Tokenize input
const inputs = await tokenizer("Hello, world!");
console.log(inputs.input_ids); // Tensor with token IDs

// Run inference
const outputs = await model(inputs);
console.log(outputs.last_hidden_state); // Model embeddings

Text Generation

import { AutoModelForCausalLM, AutoTokenizer } from "@xenova/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/gpt2");
const model = await AutoModelForCausalLM.from_pretrained("Xenova/gpt2");

const inputs = await tokenizer("The future of AI is");
const outputs = await model.generate(inputs.input_ids, {
  max_new_tokens: 50,
  do_sample: true,
  temperature: 0.7,
});

const generated_text = tokenizer.decode(outputs[0], {
  skip_special_tokens: true,
});
console.log(generated_text);

Sequence-to-Sequence Tasks

import { AutoModelForSeq2SeqLM, AutoTokenizer } from "@xenova/transformers";

const tokenizer = await AutoTokenizer.from_pretrained("Xenova/t5-small");
const model = await AutoModelForSeq2SeqLM.from_pretrained("Xenova/t5-small");

const inputs = await tokenizer("translate English to German: I love transformers!");
const outputs = await model.generate(inputs.input_ids);
const translation = tokenizer.decode(outputs[0], {
  skip_special_tokens: true,
});
console.log(translation); // "Ich liebe Transformatoren!"

Resource Management

// Always dispose of models and tokenizers when done
await model.dispose();
await tokenizer.dispose();

Specific Tokenizer Classes

For advanced use cases that require direct access to specific tokenizer implementations, Transformers.js exports individual tokenizer classes.

Common Tokenizer Classes

Popular tokenizer implementations for direct instantiation when you need fine-grained control.

/**
 * BERT tokenizer with WordPiece tokenization
 */
class BertTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<BertTokenizer>;
}

/**
 * GPT-2 tokenizer with BPE tokenization and chat template support
 */
class GPT2Tokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<GPT2Tokenizer>;
  
  /** Default chat template for conversation formatting */
  get default_chat_template(): string;
}

/**
 * T5 tokenizer for encoder-decoder models
 */
class T5Tokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<T5Tokenizer>;
}

Language Model Tokenizers

Specialized tokenizers for modern language models.

/**
 * LLaMA tokenizer with chat template and legacy support
 */
class LlamaTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<LlamaTokenizer>;
  
  /** Whether to use legacy behavior */
  readonly legacy: boolean;
  
  /** Whether to use default system prompt */
  readonly use_default_system_prompt: boolean;
  
  /** Get dynamic chat template with system prompt support */
  get default_chat_template(): string;
  
  /** Default system prompt for chat */
  static readonly DEFAULT_SYSTEM_PROMPT: string;
}

/**
 * Code Llama tokenizer (extends LlamaTokenizer)
 */
class CodeLlamaTokenizer extends LlamaTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<CodeLlamaTokenizer>;
}

/**
 * Gemma tokenizer with chat template support
 */
class GemmaTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<GemmaTokenizer>;
  
  /** Default chat template for conversation formatting */
  get default_chat_template(): string;
}

Audio and Speech Tokenizers

Tokenizers specialized for audio and speech processing models.

/**
 * Whisper tokenizer for automatic speech recognition
 */
class WhisperTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<WhisperTokenizer>;
  
  /**
   * Decode ASR sequences with timestamp support
   * @param sequences - Token sequences to decode
   * @param options - Decoding options including timestamp handling
   */
  _decode_asr(
    sequences: number[][],
    options?: {
      time_precision?: number;
      return_timestamps?: boolean;
      return_language?: boolean;
    }
  ): string[];
  
  /**
   * Get decoder prompt IDs for language/task specification
   * @param options - Language and task options
   */
  get_decoder_prompt_ids(options?: {
    language?: string;
    task?: string;
    no_timestamps?: boolean;
  }): number[];
  
  /**
   * Combine tokens into words with language-specific boundary detection
   * @param tokens - Array of token objects with timestamps
   * @param language - Language code for boundary detection
   */
  combineTokensIntoWords(
    tokens: Array<{ text: string; timestamp: [number, number] }>,
    language?: string
  ): Array<{ word: string; timestamp: [number, number] }>;
}

/**
 * Wav2Vec2 CTC tokenizer for connectionist temporal classification
 */
class Wav2Vec2CTCTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<Wav2Vec2CTCTokenizer>;
}

Multilingual and Translation Tokenizers

Tokenizers for multilingual models and translation tasks.

/**
 * XLM-RoBERTa multilingual tokenizer
 */
class XLMRobertaTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<XLMRobertaTokenizer>;
}

/**
 * mBART tokenizer for multilingual translation
 */
class MBartTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<MBartTokenizer>;
  
  /** Supported language codes */
  readonly language_codes: string[];
  
  /** Language code to token mapping */
  readonly lang_to_token: Record<string, string>;
  
  /**
   * Build translation inputs with language tokens
   * @param raw_inputs - Input text(s)
   * @param tokenizer_options - Tokenization options
   * @param generate_kwargs - Generation parameters including src_lang/tgt_lang
   */
  _build_translation_inputs(
    raw_inputs: string | string[],
    tokenizer_options: any,
    generate_kwargs: { src_lang?: string; tgt_lang?: string }
  ): any;
}

/**
 * NLLB tokenizer for No Language Left Behind translation
 */
class NllbTokenizer extends PreTrainedTokenizer {
  static async from_pretrained(
    pretrained_model_name_or_path: string,
    options?: TokenizerOptions
  ): Promise<NllbTokenizer>;
  
  /**
   * Build translation inputs with NLLB language codes
   */
  _build_translation_inputs(
    raw_inputs: string | string[],
    tokenizer_options: any,
    generate_kwargs: { src_lang?: string; tgt_lang?: string }
  ): any;
}

Usage Examples:

import { 
  WhisperTokenizer, 
  LlamaTokenizer, 
  BertTokenizer 
} from "@xenova/transformers";

// Direct tokenizer instantiation
const whisperTokenizer = await WhisperTokenizer.from_pretrained("openai/whisper-base");
const llamaTokenizer = await LlamaTokenizer.from_pretrained("meta-llama/Llama-2-7b-hf");

// Access specialized methods
const promptIds = whisperTokenizer.get_decoder_prompt_ids({
  language: "english",
  task: "transcribe"
});

// Use chat templates
const chatTemplate = llamaTokenizer.default_chat_template;
const conversation = [
  { role: "user", content: "Hello!" },
  { role: "assistant", content: "Hi there!" }
];
const formatted = await llamaTokenizer.apply_chat_template(conversation);

Types

interface ModelOutput {
  last_hidden_state?: Tensor;
  logits?: Tensor;
  hidden_states?: Tensor[];
  attentions?: Tensor[];
  [key: string]: any;
}

interface Tensor {
  data: TypedArray;
  dims: number[];
  type: string;
  size: number;
}

Install with Tessl CLI

npx tessl i tessl/npm-xenova--transformers

docs

tessl/npm-xenova--transformers

models-tokenizers.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Models and Tokenizers

Capabilities

Auto Classes - Automatic Selection

AutoModel

AutoConfig

AutoTokenizer

Task-Specific Auto Model Classes

Text Processing Models

Sequence-to-Sequence Models

Vision Models

Audio Models

Multimodal Models

Base Model Classes

PreTrainedModel

PreTrainedTokenizer

Specific Tokenizer Classes

Usage Patterns

Basic Model and Tokenizer Usage

Text Generation

Sequence-to-Sequence Tasks

Resource Management

Specific Tokenizer Classes

Common Tokenizer Classes

Language Model Tokenizers

Audio and Speech Tokenizers

Multilingual and Translation Tokenizers

Types

models-tokenizers.mddocs/