CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-vercel--blob

The Vercel Blob JavaScript API client for cloud blob storage operations

—

Pending
Overview
Eval results
Files

multipart-uploads.mddocs/

Multipart Uploads

Large file upload system supporting files up to 5TB with parallel upload, retry logic, and progress tracking.

Overview

The multipart upload system is designed for large files that exceed single-request upload limits. It splits files into parts (minimum 5MB each, except the last part), uploads them in parallel, and then combines them into a single blob. This approach provides better reliability, progress tracking, and support for very large files.

Key Features

  • Large File Support: Upload files up to 5TB
  • Parallel Uploads: Upload multiple parts simultaneously for faster transfers
  • Progress Tracking: Monitor upload progress with detailed callbacks
  • Resumable Uploads: Restart failed uploads from where they left off
  • Automatic Retry: Built-in retry logic for failed parts
  • Cross-Platform: Works in Node.js, Edge Runtime, and browsers

Capabilities

Multipart Upload Lifecycle

createMultipartUpload

Initiates a multipart upload session.

/**
 * Initiates a multipart upload process
 * @param pathname - The pathname to upload the blob to
 * @param options - Configuration options
 * @returns Promise resolving to upload session information
 */
function createMultipartUpload(pathname: string, options: CommonCreateBlobOptions): Promise<MultipartUploadInfo>;

interface MultipartUploadInfo {
  key: string;
  uploadId: string;
}

Usage Examples:

import { createMultipartUpload } from '@vercel/blob';

// Start multipart upload
const { key, uploadId } = await createMultipartUpload('videos/large-video.mp4', {
  access: 'public',
  contentType: 'video/mp4',
});

console.log('Upload session created:', { key, uploadId });

uploadPart

Uploads a single part of a multipart upload.

/**
 * Uploads a part in a multipart upload
 * @param pathname - Same pathname used in createMultipartUpload
 * @param body - Part content (minimum 5MB except for the last part)
 * @param options - Part upload options
 * @returns Promise resolving to part information
 */
function uploadPart(pathname: string, body: PutBody, options: UploadPartCommandOptions): Promise<Part>;

interface UploadPartCommandOptions extends BlobCommandOptions {
  key: string;
  uploadId: string;
  partNumber: number;
}

interface Part {
  etag: string;
  partNumber: number;
}

Usage Examples:

import { uploadPart } from '@vercel/blob';

// Upload a single part
const part = await uploadPart('videos/large-video.mp4', chunkData, {
  key: 'abc123',
  uploadId: 'xyz789',
  partNumber: 1,
});

console.log('Part uploaded:', part);

completeMultipartUpload

Completes a multipart upload by combining all parts.

/**
 * Completes a multipart upload by combining all parts
 * @param pathname - Same pathname used in createMultipartUpload
 * @param parts - Array of uploaded parts in correct order
 * @param options - Completion options
 * @returns Promise resolving to blob information
 */
function completeMultipartUpload(pathname: string, parts: Part[], options: CompleteMultipartUploadCommandOptions): Promise<PutBlobResult>;

interface CompleteMultipartUploadCommandOptions extends BlobCommandOptions {
  key: string;
  uploadId: string;
}

Usage Examples:

import { completeMultipartUpload } from '@vercel/blob';

// Complete the upload
const result = await completeMultipartUpload('videos/large-video.mp4', parts, {
  key: 'abc123',
  uploadId: 'xyz789',
});

console.log('Upload completed:', result.url);

Simplified Multipart Interface

createMultipartUploader

Creates a simplified wrapper for multipart uploads that handles the lifecycle automatically.

/**
 * Creates a simplified multipart uploader wrapper
 * @param pathname - The pathname to upload the blob to
 * @param options - Configuration options
 * @returns Promise resolving to multipart uploader instance
 */
function createMultipartUploader(pathname: string, options: CommonCreateBlobOptions): Promise<MultipartUploader>;

interface MultipartUploader {
  /** Upload a single part by part number */
  uploadPart(partNumber: number, body: PutBody): Promise<Part>;
  /** Complete the multipart upload with all parts */
  complete(parts: Part[]): Promise<PutBlobResult>;
  /** Abort the multipart upload */
  abort(): Promise<void>;
}

Usage Examples:

import { createMultipartUploader } from '@vercel/blob';

// Create uploader instance
const uploader = await createMultipartUploader('videos/large-video.mp4', {
  access: 'public',
  contentType: 'video/mp4',
});

// Upload parts
const parts = [];
for (let i = 0; i < chunks.length; i++) {
  const part = await uploader.uploadPart(i + 1, chunks[i]);
  parts.push(part);
}

// Complete upload
const result = await uploader.complete(parts);
console.log('Upload completed:', result.url);

Complete Multipart Upload Examples

Basic Multipart Upload

import { 
  createMultipartUpload, 
  uploadPart, 
  completeMultipartUpload 
} from '@vercel/blob';

async function uploadLargeFile(file: File, pathname: string) {
  // 1. Create multipart upload
  const { key, uploadId } = await createMultipartUpload(pathname, {
    access: 'public',
    contentType: file.type,
  });

  // 2. Split file into parts (minimum 5MB each)
  const partSize = 5 * 1024 * 1024; // 5MB
  const parts: Part[] = [];

  for (let i = 0; i < Math.ceil(file.size / partSize); i++) {
    const start = i * partSize;
    const end = Math.min(start + partSize, file.size);
    const chunk = file.slice(start, end);
    
    const part = await uploadPart(pathname, chunk, {
      key,
      uploadId,
      partNumber: i + 1,
    });
    
    parts.push(part);
  }

  // 3. Complete multipart upload
  const result = await completeMultipartUpload(pathname, parts, {
    key,
    uploadId,
  });

  return result;
}

Parallel Multipart Upload with Progress

import { 
  createMultipartUpload, 
  uploadPart, 
  completeMultipartUpload 
} from '@vercel/blob';

interface UploadProgress {
  totalParts: number;
  completedParts: number;
  percentage: number;
}

async function uploadLargeFileWithProgress(
  file: File, 
  pathname: string,
  onProgress?: (progress: UploadProgress) => void
) {
  // Create multipart upload
  const { key, uploadId } = await createMultipartUpload(pathname, {
    access: 'public',
    contentType: file.type,
  });

  // Split file into parts
  const partSize = 5 * 1024 * 1024; // 5MB
  const totalParts = Math.ceil(file.size / partSize);
  const chunks: Blob[] = [];

  for (let i = 0; i < totalParts; i++) {
    const start = i * partSize;
    const end = Math.min(start + partSize, file.size);
    chunks.push(file.slice(start, end));
  }

  // Upload parts in parallel with progress tracking
  let completedParts = 0;
  
  const uploadPromises = chunks.map(async (chunk, index) => {
    const part = await uploadPart(pathname, chunk, {
      key,
      uploadId,
      partNumber: index + 1,
    });
    
    completedParts++;
    onProgress?.({
      totalParts,
      completedParts,
      percentage: Math.round((completedParts / totalParts) * 100),
    });
    
    return part;
  });

  // Wait for all parts to complete
  const parts = await Promise.all(uploadPromises);

  // Complete multipart upload
  const result = await completeMultipartUpload(pathname, parts, {
    key,
    uploadId,
  });

  return result;
}

// Usage
const result = await uploadLargeFileWithProgress(
  largeVideoFile,
  'videos/my-video.mp4',
  (progress) => {
    console.log(`Upload progress: ${progress.percentage}% (${progress.completedParts}/${progress.totalParts} parts)`);
  }
);

Resumable Multipart Upload

import { 
  createMultipartUpload, 
  uploadPart, 
  completeMultipartUpload 
} from '@vercel/blob';

interface UploadState {
  key: string;
  uploadId: string;
  completedParts: Part[];
  totalParts: number;
}

async function resumableUpload(
  file: File, 
  pathname: string,
  savedState?: UploadState
) {
  let state: UploadState;

  if (savedState) {
    // Resume existing upload
    state = savedState;
  } else {
    // Start new upload
    const { key, uploadId } = await createMultipartUpload(pathname, {
      access: 'public',
      contentType: file.type,
    });

    state = {
      key,
      uploadId,
      completedParts: [],
      totalParts: Math.ceil(file.size / (5 * 1024 * 1024)),
    };
  }

  const partSize = 5 * 1024 * 1024; // 5MB
  const allParts: Part[] = [...state.completedParts];

  // Upload remaining parts
  for (let i = state.completedParts.length; i < state.totalParts; i++) {
    try {
      const start = i * partSize;
      const end = Math.min(start + partSize, file.size);
      const chunk = file.slice(start, end);

      const part = await uploadPart(pathname, chunk, {
        key: state.key,
        uploadId: state.uploadId,
        partNumber: i + 1,
      });

      allParts.push(part);
      state.completedParts.push(part);

      // Save state for resumption (implement your own storage)
      await saveUploadState(pathname, state);
      
    } catch (error) {
      console.error(`Failed to upload part ${i + 1}:`, error);
      // Save current state and allow for resumption
      await saveUploadState(pathname, state);
      throw error;
    }
  }

  // Complete the upload
  const result = await completeMultipartUpload(pathname, allParts, {
    key: state.key,
    uploadId: state.uploadId,
  });

  // Clear saved state
  await clearUploadState(pathname);

  return result;
}

// Helper functions (implement based on your storage needs)
async function saveUploadState(pathname: string, state: UploadState) {
  localStorage.setItem(`upload_${pathname}`, JSON.stringify(state));
}

async function clearUploadState(pathname: string) {
  localStorage.removeItem(`upload_${pathname}`);
}

async function getUploadState(pathname: string): Promise<UploadState | undefined> {
  const saved = localStorage.getItem(`upload_${pathname}`);
  return saved ? JSON.parse(saved) : undefined;
}

Using the Simplified Uploader

import { createMultipartUploader } from '@vercel/blob';

async function uploadWithSimplifiedInterface(file: File, pathname: string) {
  // Create uploader
  const uploader = await createMultipartUploader(pathname, {
    access: 'public',
    contentType: file.type,
  });

  try {
    // Split and upload parts
    const partSize = 5 * 1024 * 1024; // 5MB
    const parts: Part[] = [];

    for (let i = 0; i < Math.ceil(file.size / partSize); i++) {
      const start = i * partSize;
      const end = Math.min(start + partSize, file.size);
      const chunk = file.slice(start, end);
      
      const part = await uploader.uploadPart(i + 1, chunk);
      parts.push(part);
    }

    // Complete upload
    const result = await uploader.complete(parts);
    return result;
    
  } catch (error) {
    // Abort upload on error
    await uploader.abort();
    throw error;
  }
}

Common Types

Part Input for Custom Uploaders

interface PartInput {
  partNumber: number;
  blob: PutBody;
}

Multipart Helper Types

interface MultipartUploadInfo {
  key: string;
  uploadId: string;
}

interface Part {
  etag: string;
  partNumber: number;
}

interface MultipartUploader {
  uploadPart(partNumber: number, body: PutBody): Promise<Part>;
  complete(parts: Part[]): Promise<PutBlobResult>;
  abort(): Promise<void>;
}

Best Practices

File Size Considerations

  • Small files (< 4.5MB): Use regular put() function for simplicity
  • Medium files (4.5MB - 100MB): Use put() with multipart: true option
  • Large files (> 100MB): Use dedicated multipart upload functions for better control

Part Size Guidelines

  • Minimum part size: 5MB (except for the last part)
  • Maximum part size: 5GB
  • Recommended part size: 10-100MB for optimal performance
  • Total parts limit: 10,000 parts per upload

Error Handling

import { 
  createMultipartUpload, 
  uploadPart, 
  completeMultipartUpload,
  BlobError 
} from '@vercel/blob';

async function robustMultipartUpload(file: File, pathname: string) {
  let uploadId: string;
  let key: string;

  try {
    // Create upload
    const result = await createMultipartUpload(pathname, {
      access: 'public',
    });
    uploadId = result.uploadId;
    key = result.key;

    // Upload parts with retry logic
    const parts = await uploadPartsWithRetry(file, pathname, key, uploadId);

    // Complete upload
    return await completeMultipartUpload(pathname, parts, {
      key,
      uploadId,
    });

  } catch (error) {
    if (error instanceof BlobError) {
      console.error('Blob service error:', error.message);
    }
    
    // Clean up failed upload if possible
    if (uploadId && key) {
      try {
        // Note: There's no explicit abort function in the public API
        // The upload will be cleaned up automatically after some time
        console.log('Upload failed, will be cleaned up automatically');
      } catch (cleanupError) {
        console.error('Cleanup failed:', cleanupError);
      }
    }
    
    throw error;
  }
}

async function uploadPartsWithRetry(
  file: File, 
  pathname: string, 
  key: string, 
  uploadId: string,
  maxRetries = 3
): Promise<Part[]> {
  const partSize = 5 * 1024 * 1024;
  const parts: Part[] = [];

  for (let i = 0; i < Math.ceil(file.size / partSize); i++) {
    const start = i * partSize;
    const end = Math.min(start + partSize, file.size);
    const chunk = file.slice(start, end);
    
    let attempts = 0;
    while (attempts < maxRetries) {
      try {
        const part = await uploadPart(pathname, chunk, {
          key,
          uploadId,
          partNumber: i + 1,
        });
        parts.push(part);
        break;
      } catch (error) {
        attempts++;
        if (attempts >= maxRetries) {
          throw new Error(`Failed to upload part ${i + 1} after ${maxRetries} attempts: ${error}`);
        }
        // Exponential backoff
        await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempts) * 1000));
      }
    }
  }

  return parts;
}

Install with Tessl CLI

npx tessl i tessl/npm-vercel--blob

docs

client-api.md

errors.md

index.md

multipart-uploads.md

server-api.md

tile.json