tessl/npm-pnpm--worker

A worker for extracting package tarballs to the store using multi-threaded worker pools for high-performance package management operations

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview

Eval results

Files

Tarball Processing

Name: tessl/npm-pnpm--worker
Author: tessl

Extract and process package tarballs with integrity verification and CAFS integration for efficient package storage and management.

Capabilities

Add Files From Tarball

Extracts and adds files from tarball buffer to CAFS store with optional integrity verification. This is the primary function for processing downloaded package tarballs.

/**
 * Extracts and adds files from tarball buffer to CAFS store
 * @param opts - Configuration options for tarball extraction
 * @returns Promise resolving to extraction results with file mappings and metadata
 */
function addFilesFromTarball(opts: AddFilesFromTarballOptions): Promise<AddFilesResult>;

type AddFilesFromTarballOptions = Pick<TarballExtractMessage, 'buffer' | 'storeDir' | 'filesIndexFile' | 'integrity' | 'readManifest' | 'pkg'> & {
  /** Source URL for error reporting */
  url: string;
};

interface TarballExtractMessage {
  type: 'extract';
  /** Tarball buffer data to extract */
  buffer: Buffer;
  /** Store directory path where files will be stored */
  storeDir: string;
  /** Integrity hash for verification (format: algorithm-hash) */
  integrity?: string;
  /** Path to the files index file for metadata storage */
  filesIndexFile: string;
  /** Whether to read and parse the package manifest */
  readManifest?: boolean;
  /** Package name and version information */
  pkg?: PkgNameVersion;
}

interface AddFilesResult {
  /** Mapping of file paths to their CAFS store locations */
  filesIndex: Record<string, string>;
  /** Parsed package manifest (package.json) */
  manifest: DependencyManifest;
  /** Whether the package requires a build step */
  requiresBuild: boolean;
}

Usage Examples:

import { addFilesFromTarball } from "@pnpm/worker";
import fs from "fs";

// Basic tarball extraction
const tarballBuffer = fs.readFileSync("package.tgz");
const result = await addFilesFromTarball({
  buffer: tarballBuffer,
  storeDir: "/path/to/pnpm/store",
  filesIndexFile: "/path/to/index.json",
  url: "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz"
});

console.log("Extracted files:", Object.keys(result.filesIndex));
console.log("Package name:", result.manifest.name);
console.log("Requires build:", result.requiresBuild);

// With integrity verification
const resultWithIntegrity = await addFilesFromTarball({
  buffer: tarballBuffer,
  storeDir: "/path/to/pnpm/store",
  filesIndexFile: "/path/to/index.json",
  integrity: "sha512-9h7e6r3a5x5b2b5f3fdc2c7d8e1f6r7a9e7q8d4f1d7a8c6b7e8d9f0a1b2c3d4e5f6",
  readManifest: true,
  pkg: { name: "lodash", version: "4.17.21" },
  url: "https://registry.npmjs.org/lodash/-/lodash-4.17.21.tgz"
});

Integrity Verification

When the integrity parameter is provided, the function performs checksum verification:

Hash Calculation: Calculates hash of the tarball buffer using the specified algorithm
Format: Expects SRI (Subresource Integrity) format: algorithm-base64hash
Verification: Compares calculated hash with expected hash
Error Handling: Throws TarballIntegrityError if verification fails

Supported Hash Algorithms:

sha1 - SHA-1 (legacy)
sha256 - SHA-256 (recommended)
sha512 - SHA-512 (most secure)

Error Handling

Tarball Integrity Error

Specialized error class for integrity verification failures:

class TarballIntegrityError extends Error {
  /** Actual hash found during verification */
  readonly found: string;
  /** Expected hash from integrity parameter */
  readonly expected: string;
  /** Hash algorithm used (sha1, sha256, sha512) */
  readonly algorithm: string;
  /** Original SRI string */
  readonly sri: string;
  /** Source URL where tarball was downloaded */
  readonly url: string;
  
  constructor(opts: {
    attempts?: number;
    found: string;
    expected: string;
    algorithm: string;
    sri: string;
    url: string;
  });
}

Error Handling Example:

import { addFilesFromTarball, TarballIntegrityError } from "@pnpm/worker";

try {
  const result = await addFilesFromTarball({
    buffer: tarballBuffer,
    storeDir: "/path/to/store",
    filesIndexFile: "/path/to/index.json",
    integrity: "sha512-invalid-hash",
    url: "https://registry.npmjs.org/package/-/package-1.0.0.tgz"
  });
} catch (error) {
  if (error instanceof TarballIntegrityError) {
    console.error(`Integrity check failed for ${error.url}`);
    console.error(`Expected: ${error.expected}, Found: ${error.found}`);
    console.error(`Algorithm: ${error.algorithm}`);
    
    // Recommended action: clear cache and retry
    console.log("Try running: pnpm store prune");
  } else {
    console.error("Other extraction error:", error.message);
  }
}