or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/npm-pnpm--directory-fetcher

A fetcher for local directory packages within the pnpm ecosystem

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/@pnpm/directory-fetcher@1000.1.x

To install, run

npx @tessl/cli install tessl/npm-pnpm--directory-fetcher@1000.1.0

index.mddocs/

@pnpm/directory-fetcher

@pnpm/directory-fetcher is a TypeScript library that provides a fetcher for local directory packages within the pnpm ecosystem. It handles fetching and processing of packages from local directories, supporting both package-file-only inclusion and complete directory content retrieval with configurable symlink resolution.

Package Information

  • Package Name: @pnpm/directory-fetcher
  • Package Type: npm
  • Language: TypeScript
  • Installation: pnpm add @pnpm/directory-fetcher

Core Imports

import { createDirectoryFetcher, fetchFromDir } from "@pnpm/directory-fetcher";
import type { CreateDirectoryFetcherOptions, FetchResult } from "@pnpm/directory-fetcher";

For CommonJS:

const { createDirectoryFetcher, fetchFromDir } = require("@pnpm/directory-fetcher");

Basic Usage

import { createDirectoryFetcher } from "@pnpm/directory-fetcher";

// Create a basic directory fetcher
const fetcher = createDirectoryFetcher();

// Use the fetcher with pnpm's CAFS system
const result = await fetcher.directory(cafs, {
  type: 'directory',
  directory: './packages/my-package'
}, {
  lockfileDir: process.cwd()
});

// Result contains file index and package metadata
console.log(result.filesIndex);
console.log(result.manifest);
console.log(result.requiresBuild);

Architecture

@pnpm/directory-fetcher is built around several key components:

  • Directory Fetcher Factory: createDirectoryFetcher function that creates configured fetcher instances
  • Direct Fetch Function: fetchFromDir for direct directory fetching without CAFS integration
  • File Processing: Support for both complete directory scanning and package-file-only inclusion
  • Symlink Resolution: Configurable symlink handling for flexible file system access
  • Build Detection: Automatic analysis using @pnpm/exec.pkg-requires-build to determine if packages need compilation
  • Manifest Integration: Safe package.json reading with fallback handling
  • File Filtering: Uses @pnpm/fs.packlist for npm-compatible file inclusion when includeOnlyPackageFiles is enabled

Capabilities

Directory Fetcher Creation

Creates a configured directory fetcher instance with customizable options for file inclusion and symlink handling.

/**
 * Creates a directory fetcher with configurable options
 * @param opts - Optional configuration for the fetcher
 * @returns Object containing the directory fetcher function
 */
function createDirectoryFetcher(
  opts?: CreateDirectoryFetcherOptions
): { directory: DirectoryFetcher };

interface CreateDirectoryFetcherOptions {
  /** Only include files that would be published (uses npm's file inclusion rules) */
  includeOnlyPackageFiles?: boolean;
  /** Resolve symlinks to their real file system paths */
  resolveSymlinks?: boolean;
}

type DirectoryFetcher = (
  cafs: Cafs,
  resolution: DirectoryResolution,
  opts: DirectoryFetcherOptions
) => Promise<FetchResult>;

/**
 * Content Addressable File System interface for pnpm
 */
interface Cafs {
  /** Content-addressable file system interface (imported from @pnpm/fetcher-base) */
  [key: string]: any;
}

Direct Directory Fetching

Directly fetches files from a directory with specified options, bypassing the CAFS integration layer.

/**
 * Directly fetch files from a directory with specified options
 * @param dir - The directory path to fetch from
 * @param opts - Fetch configuration options
 * @returns Promise resolving to fetch result with file index and metadata
 */
function fetchFromDir(
  dir: string, 
  opts: FetchFromDirOptions
): Promise<FetchResult>;

type FetchFromDirOptions = Omit<DirectoryFetcherOptions, 'lockfileDir'> & CreateDirectoryFetcherOptions;

Types

Core Result Types

interface FetchResult {
  /** Always true for directory fetcher */
  local: true;
  /** Map of relative file paths to absolute file system paths */
  filesIndex: Record<string, string>;
  /** File statistics (only present when not using includeOnlyPackageFiles) */
  filesStats?: Record<string, Stats | null>;
  /** Always 'hardlink' for directory packages */
  packageImportMethod: 'hardlink';
  /** Package manifest (package.json content), may be undefined if no package.json exists */
  manifest: DependencyManifest | undefined;
  /** Whether the package requires build steps */
  requiresBuild: boolean;
}

Integration Types

interface DirectoryFetcherOptions {
  /** Directory containing the lockfile (required) */
  lockfileDir: string;
  /** Whether to read package manifest (default: depends on implementation) */
  readManifest?: boolean;
}

interface DirectoryResolution {
  /** Resolution type identifier */
  type: 'directory';
  /** Relative path to the directory */
  directory: string;
}

interface DependencyManifest {
  /** Package name */
  name: string;
  /** Package version */
  version: string;
  /** Package dependencies */
  dependencies?: Record<string, string>;
  /** Development dependencies */
  devDependencies?: Record<string, string>;
  /** Package scripts */
  scripts?: Record<string, string>;
  /** Node.js engine requirements */
  engines?: Record<string, string>;
  /** Additional package.json fields */
  [key: string]: any;
}

File System Types

// Uses Node.js built-in fs.Stats interface for file system information
import type { Stats } from 'fs';

Usage Examples

Package Files Only

import { createDirectoryFetcher } from "@pnpm/directory-fetcher";

// Only include files that would be published to npm
const fetcher = createDirectoryFetcher({ 
  includeOnlyPackageFiles: true 
});

const result = await fetcher.directory(cafs, {
  type: 'directory',
  directory: './my-package'
}, {
  lockfileDir: process.cwd()
});

// result.filesIndex only contains publishable files
// No result.filesStats property

Symlink Resolution

The fetcher provides two symlink handling modes:

import { createDirectoryFetcher } from "@pnpm/directory-fetcher";

// Default behavior: preserve symlink paths
const defaultFetcher = createDirectoryFetcher();

// Resolve symlinks to their real paths
const resolvingFetcher = createDirectoryFetcher({ 
  resolveSymlinks: true 
});

const result = await resolvingFetcher.directory(cafs, {
  type: 'directory',
  directory: './symlinked-package'
}, {
  lockfileDir: process.cwd()
});

// With resolveSymlinks: true
// - result.filesIndex contains real file paths, not symlink paths
// - Broken symlinks are gracefully skipped with debug logging
// - Uses fs.realpath() and fs.stat() for resolved paths

// With resolveSymlinks: false (default)
// - result.filesIndex contains original symlink paths
// - Uses fs.stat() directly on symlink paths
// - Broken symlinks still cause debug logging but preserve original paths

Direct Directory Fetching

import { fetchFromDir } from "@pnpm/directory-fetcher";

// Fetch directly without CAFS integration
const result = await fetchFromDir('/path/to/package', {
  includeOnlyPackageFiles: false,
  resolveSymlinks: true,
  readManifest: true
});

// Full FetchResult with all files and statistics
console.log(result.filesIndex);
console.log(result.filesStats);
console.log(result.manifest?.name); // manifest may be undefined

Handling Packages Without package.json

The directory fetcher gracefully handles directories that don't contain a package.json file:

import { createDirectoryFetcher } from "@pnpm/directory-fetcher";

const fetcher = createDirectoryFetcher();

// Fetch from a directory without package.json (e.g., Bit workspace components)
const result = await fetcher.directory(cafs, {
  type: 'directory',
  directory: './component-without-manifest'
}, {
  lockfileDir: process.cwd(),
  readManifest: true
});

// result.manifest will be undefined
console.log(result.manifest); // undefined
console.log(result.filesIndex); // Still contains all files
console.log(result.requiresBuild); // Determined without manifest

Error Handling

The directory fetcher includes robust error handling for common file system issues:

import { createDirectoryFetcher } from "@pnpm/directory-fetcher";

const fetcher = createDirectoryFetcher();

try {
  const result = await fetcher.directory(cafs, {
    type: 'directory',
    directory: './nonexistent-package'
  }, {
    lockfileDir: process.cwd()
  });
} catch (error) {
  // Handle directory access errors, permission issues, etc.
  if (error.code === 'ENOENT') {
    console.error('Directory not found:', error.path);
  } else if (error.code === 'EACCES') {
    console.error('Permission denied:', error.path);
  } else {
    console.error('Failed to fetch directory:', error.message);
  }
}

Error Handling Behavior:

  • Broken symlinks: Gracefully skipped with debug logging, do not cause failures
  • Missing package.json: Results in undefined manifest using safeReadProjectManifestOnly, fetch continues
  • Directory access errors: Thrown as standard Node.js file system errors
  • Permission errors: Thrown with appropriate error codes (EACCES, etc.)
  • ENOENT errors on symlinks: Caught internally, logged as debug messages, symlink skipped
  • node_modules directories: Automatically filtered out during directory traversal

Debug Logging: The fetcher uses @pnpm/logger with category 'directory-fetcher' for debug information:

// Broken symlinks are logged as:
// { brokenSymlink: '/absolute/path/to/broken/symlink' }