tessl/npm-pnpm--worker

A worker for extracting package tarballs to the store using multi-threaded worker pools for high-performance package management operations

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview

Eval results

Files

Directory Operations

Name: tessl/npm-pnpm--worker
Author: tessl

Add files from local directories to the CAFS store and manage directory-based package operations for local development and file system integration.

Capabilities

Add Files From Directory

Adds files from a local directory to the CAFS store. This is useful for processing local packages or directories that aren't packaged as tarballs.

/**
 * Adds files from a directory to the CAFS store
 * @param opts - Configuration options for directory processing
 * @returns Promise resolving to processing results with file mappings and metadata
 */
function addFilesFromDir(opts: AddFilesFromDirOptions): Promise<AddFilesResult>;

type AddFilesFromDirOptions = Pick<AddDirToStoreMessage, 'storeDir' | 'dir' | 'filesIndexFile' | 'sideEffectsCacheKey' | 'readManifest' | 'pkg' | 'files'>;

interface AddDirToStoreMessage {
  type: 'add-dir';
  /** Store directory path where files will be stored */
  storeDir: string;
  /** Source directory path to process */
  dir: string;
  /** Path to the files index file for metadata storage */
  filesIndexFile: string;
  /** Optional cache key for side effects tracking */
  sideEffectsCacheKey?: string;
  /** Whether to read and parse the package manifest */
  readManifest?: boolean;
  /** Package name and version information */
  pkg?: PkgNameVersion;
  /** Specific files to include (if not provided, includes all files) */
  files?: string[];
}

interface AddFilesResult {
  /** Mapping of file paths to their CAFS store locations */
  filesIndex: Record<string, string>;
  /** Parsed package manifest (package.json) if available */
  manifest: DependencyManifest;
  /** Whether the package requires a build step */
  requiresBuild: boolean;
}

Usage Examples:

import { addFilesFromDir } from "@pnpm/worker";

// Process entire directory
const result = await addFilesFromDir({
  storeDir: "/path/to/pnpm/store",
  dir: "/path/to/local/package",
  filesIndexFile: "/path/to/index.json",
  readManifest: true
});

console.log("Processed files:", Object.keys(result.filesIndex));
console.log("Package name:", result.manifest?.name);

// Process specific files with side effects caching
const specificFilesResult = await addFilesFromDir({
  storeDir: "/path/to/pnpm/store",
  dir: "/path/to/package",
  filesIndexFile: "/path/to/index.json",
  sideEffectsCacheKey: "build-artifacts",
  files: ["lib/index.js", "lib/utils.js", "package.json"],
  pkg: { name: "my-package", version: "1.0.0" }
});

Hard Link Directory

Creates hard links from a source directory to multiple destination directories. This is an efficient way to duplicate directory structures without copying file contents.

/**
 * Creates hard links from source directory to multiple destination directories
 * @param src - Source directory path to link from
 * @param destDirs - Array of destination directory paths
 * @returns Promise that resolves when all hard links are created
 */
function hardLinkDir(src: string, destDirs: string[]): Promise<void>;

Usage Examples:

import { hardLinkDir } from "@pnpm/worker";

// Link one directory to multiple locations
await hardLinkDir(
  "/path/to/source/package",
  [
    "/path/to/project1/node_modules/package",
    "/path/to/project2/node_modules/package",
    "/path/to/project3/node_modules/package"
  ]
);

console.log("Hard links created successfully");

// Single destination
await hardLinkDir(
  "/pnpm/store/files/abc123",
  ["/project/node_modules/lodash"]
);

Initialize Store Directory

Initializes the store directory structure for CAFS. This creates the necessary subdirectory structure used by pnpm's content-addressable file system.

/**
 * Initializes the store directory structure for CAFS
 * @param storeDir - Store directory path to initialize
 * @returns Promise that resolves when store structure is created
 */
function initStoreDir(storeDir: string): Promise<void>;

Usage Examples:

import { initStoreDir } from "@pnpm/worker";

// Initialize a new pnpm store
await initStoreDir("/path/to/new/pnpm/store");
console.log("Store directory initialized");

// Initialize before other operations
const storeDir = "/path/to/pnpm/store";
await initStoreDir(storeDir);

// Now ready for other operations
const result = await addFilesFromDir({
  storeDir,
  dir: "/path/to/package",
  filesIndexFile: "/path/to/index.json"
});

Side Effects Caching

The sideEffectsCacheKey parameter in addFilesFromDir enables tracking of build artifacts and other side effects:

How Side Effects Work

Base Files: Initial files added to the store
Side Effects: Files created or modified after processing (e.g., build outputs)
Diff Calculation: Compares base files with side effects to track changes
Cache Storage: Stores differences in the files index for future reference

Side Effects Cache Key Usage

// Initial package processing
await addFilesFromDir({
  storeDir: "/store",
  dir: "/package",
  filesIndexFile: "/index.json",
  readManifest: true
});

// After build step - cache the build artifacts
await addFilesFromDir({
  storeDir: "/store", 
  dir: "/package",
  filesIndexFile: "/index.json",
  sideEffectsCacheKey: "post-build",
  readManifest: false
});

The side effects cache tracks:

Added files: New files created during processing
Modified files: Existing files with changed content or permissions
Deleted files: Files removed during processing

Store Directory Structure

The initStoreDir function creates the following structure:

store/
├── files/
│   ├── 00/
│   ├── 01/
│   ├── ...
│   └── ff/
└── index/
    ├── 00/
    ├── 01/
    ├── ...
    └── ff/

files/: Contains actual file content, organized by content hash prefixes
index/: Contains metadata and index files for fast lookups
Subdirectories: 256 subdirectories (00-ff) for efficient file system performance

Install with Tessl CLI