0
# @pnpm/directory-fetcher
1

2
@pnpm/directory-fetcher is a TypeScript library that provides a fetcher for local directory packages within the pnpm ecosystem. It handles fetching and processing of packages from local directories, supporting both package-file-only inclusion and complete directory content retrieval with configurable symlink resolution.
3

4
## Package Information
5

6
- **Package Name**: @pnpm/directory-fetcher
7
- **Package Type**: npm
8
- **Language**: TypeScript
9
- **Installation**: `pnpm add @pnpm/directory-fetcher`
10

11
## Core Imports
12

13
```typescript
14
import { createDirectoryFetcher, fetchFromDir } from "@pnpm/directory-fetcher";
15
import type { CreateDirectoryFetcherOptions, FetchResult } from "@pnpm/directory-fetcher";
16
```
17

18
For CommonJS:
19

20
```javascript
21
const { createDirectoryFetcher, fetchFromDir } = require("@pnpm/directory-fetcher");
22
```
23

24
## Basic Usage
25

26
```typescript
27
import { createDirectoryFetcher } from "@pnpm/directory-fetcher";
28

29
// Create a basic directory fetcher
30
const fetcher = createDirectoryFetcher();
31

32
// Use the fetcher with pnpm's CAFS system
33
const result = await fetcher.directory(cafs, {
34
  type: 'directory',
35
  directory: './packages/my-package'
36
}, {
37
  lockfileDir: process.cwd()
38
});
39

40
// Result contains file index and package metadata
41
console.log(result.filesIndex);
42
console.log(result.manifest);
43
console.log(result.requiresBuild);
44
```
45

46
## Architecture
47

48
@pnpm/directory-fetcher is built around several key components:
49

50
- **Directory Fetcher Factory**: `createDirectoryFetcher` function that creates configured fetcher instances
51
- **Direct Fetch Function**: `fetchFromDir` for direct directory fetching without CAFS integration
52
- **File Processing**: Support for both complete directory scanning and package-file-only inclusion
53
- **Symlink Resolution**: Configurable symlink handling for flexible file system access
54
- **Build Detection**: Automatic analysis using `@pnpm/exec.pkg-requires-build` to determine if packages need compilation
55
- **Manifest Integration**: Safe package.json reading with fallback handling
56
- **File Filtering**: Uses `@pnpm/fs.packlist` for npm-compatible file inclusion when `includeOnlyPackageFiles` is enabled
57

58
## Capabilities
59

60
### Directory Fetcher Creation
61

62
Creates a configured directory fetcher instance with customizable options for file inclusion and symlink handling.
63

64
```typescript { .api }
65
/**
66
 * Creates a directory fetcher with configurable options
67
 * @param opts - Optional configuration for the fetcher
68
 * @returns Object containing the directory fetcher function
69
 */
70
function createDirectoryFetcher(
71
  opts?: CreateDirectoryFetcherOptions
72
): { directory: DirectoryFetcher };
73

74
interface CreateDirectoryFetcherOptions {
75
  /** Only include files that would be published (uses npm's file inclusion rules) */
76
  includeOnlyPackageFiles?: boolean;
77
  /** Resolve symlinks to their real file system paths */
78
  resolveSymlinks?: boolean;
79
}
80

81
type DirectoryFetcher = (
82
  cafs: Cafs,
83
  resolution: DirectoryResolution,
84
  opts: DirectoryFetcherOptions
85
) => Promise<FetchResult>;
86

87
/**
88
 * Content Addressable File System interface for pnpm
89
 */
90
interface Cafs {
91
  /** Content-addressable file system interface (imported from @pnpm/fetcher-base) */
92
  [key: string]: any;
93
}
94
```
95

96
### Direct Directory Fetching
97

98
Directly fetches files from a directory with specified options, bypassing the CAFS integration layer.
99

100
```typescript { .api }
101
/**
102
 * Directly fetch files from a directory with specified options
103
 * @param dir - The directory path to fetch from
104
 * @param opts - Fetch configuration options
105
 * @returns Promise resolving to fetch result with file index and metadata
106
 */
107
function fetchFromDir(
108
  dir: string, 
109
  opts: FetchFromDirOptions
110
): Promise<FetchResult>;
111

112
type FetchFromDirOptions = Omit<DirectoryFetcherOptions, 'lockfileDir'> & CreateDirectoryFetcherOptions;
113
```
114

115
## Types
116

117
### Core Result Types
118

119
```typescript { .api }
120
interface FetchResult {
121
  /** Always true for directory fetcher */
122
  local: true;
123
  /** Map of relative file paths to absolute file system paths */
124
  filesIndex: Record<string, string>;
125
  /** File statistics (only present when not using includeOnlyPackageFiles) */
126
  filesStats?: Record<string, Stats | null>;
127
  /** Always 'hardlink' for directory packages */
128
  packageImportMethod: 'hardlink';
129
  /** Package manifest (package.json content), may be undefined if no package.json exists */
130
  manifest: DependencyManifest | undefined;
131
  /** Whether the package requires build steps */
132
  requiresBuild: boolean;
133
}
134
```
135

136
### Integration Types
137

138
```typescript { .api }
139
interface DirectoryFetcherOptions {
140
  /** Directory containing the lockfile (required) */
141
  lockfileDir: string;
142
  /** Whether to read package manifest (default: depends on implementation) */
143
  readManifest?: boolean;
144
}
145

146
interface DirectoryResolution {
147
  /** Resolution type identifier */
148
  type: 'directory';
149
  /** Relative path to the directory */
150
  directory: string;
151
}
152

153
interface DependencyManifest {
154
  /** Package name */
155
  name: string;
156
  /** Package version */
157
  version: string;
158
  /** Package dependencies */
159
  dependencies?: Record<string, string>;
160
  /** Development dependencies */
161
  devDependencies?: Record<string, string>;
162
  /** Package scripts */
163
  scripts?: Record<string, string>;
164
  /** Node.js engine requirements */
165
  engines?: Record<string, string>;
166
  /** Additional package.json fields */
167
  [key: string]: any;
168
}
169
```
170

171
### File System Types
172

173
```typescript { .api }
174
// Uses Node.js built-in fs.Stats interface for file system information
175
import type { Stats } from 'fs';
176
```
177

178
## Usage Examples
179

180
### Package Files Only
181

182
```typescript
183
import { createDirectoryFetcher } from "@pnpm/directory-fetcher";
184

185
// Only include files that would be published to npm
186
const fetcher = createDirectoryFetcher({ 
187
  includeOnlyPackageFiles: true 
188
});
189

190
const result = await fetcher.directory(cafs, {
191
  type: 'directory',
192
  directory: './my-package'
193
}, {
194
  lockfileDir: process.cwd()
195
});
196

197
// result.filesIndex only contains publishable files
198
// No result.filesStats property
199
```
200

201
### Symlink Resolution
202

203
The fetcher provides two symlink handling modes:
204

205
```typescript
206
import { createDirectoryFetcher } from "@pnpm/directory-fetcher";
207

208
// Default behavior: preserve symlink paths
209
const defaultFetcher = createDirectoryFetcher();
210

211
// Resolve symlinks to their real paths
212
const resolvingFetcher = createDirectoryFetcher({ 
213
  resolveSymlinks: true 
214
});
215

216
const result = await resolvingFetcher.directory(cafs, {
217
  type: 'directory',
218
  directory: './symlinked-package'
219
}, {
220
  lockfileDir: process.cwd()
221
});
222

223
// With resolveSymlinks: true
224
// - result.filesIndex contains real file paths, not symlink paths
225
// - Broken symlinks are gracefully skipped with debug logging
226
// - Uses fs.realpath() and fs.stat() for resolved paths
227

228
// With resolveSymlinks: false (default)
229
// - result.filesIndex contains original symlink paths
230
// - Uses fs.stat() directly on symlink paths
231
// - Broken symlinks still cause debug logging but preserve original paths
232
```
233

234
### Direct Directory Fetching
235

236
```typescript
237
import { fetchFromDir } from "@pnpm/directory-fetcher";
238

239
// Fetch directly without CAFS integration
240
const result = await fetchFromDir('/path/to/package', {
241
  includeOnlyPackageFiles: false,
242
  resolveSymlinks: true,
243
  readManifest: true
244
});
245

246
// Full FetchResult with all files and statistics
247
console.log(result.filesIndex);
248
console.log(result.filesStats);
249
console.log(result.manifest?.name); // manifest may be undefined
250
```
251

252
### Handling Packages Without package.json
253

254
The directory fetcher gracefully handles directories that don't contain a package.json file:
255

256
```typescript
257
import { createDirectoryFetcher } from "@pnpm/directory-fetcher";
258

259
const fetcher = createDirectoryFetcher();
260

261
// Fetch from a directory without package.json (e.g., Bit workspace components)
262
const result = await fetcher.directory(cafs, {
263
  type: 'directory',
264
  directory: './component-without-manifest'
265
}, {
266
  lockfileDir: process.cwd(),
267
  readManifest: true
268
});
269

270
// result.manifest will be undefined
271
console.log(result.manifest); // undefined
272
console.log(result.filesIndex); // Still contains all files
273
console.log(result.requiresBuild); // Determined without manifest
274
```
275

276
### Error Handling
277

278
The directory fetcher includes robust error handling for common file system issues:
279

280
```typescript
281
import { createDirectoryFetcher } from "@pnpm/directory-fetcher";
282

283
const fetcher = createDirectoryFetcher();
284

285
try {
286
  const result = await fetcher.directory(cafs, {
287
    type: 'directory',
288
    directory: './nonexistent-package'
289
  }, {
290
    lockfileDir: process.cwd()
291
  });
292
} catch (error) {
293
  // Handle directory access errors, permission issues, etc.
294
  if (error.code === 'ENOENT') {
295
    console.error('Directory not found:', error.path);
296
  } else if (error.code === 'EACCES') {
297
    console.error('Permission denied:', error.path);
298
  } else {
299
    console.error('Failed to fetch directory:', error.message);
300
  }
301
}
302
```
303

304
**Error Handling Behavior:**
305
- **Broken symlinks**: Gracefully skipped with debug logging, do not cause failures
306
- **Missing package.json**: Results in `undefined` manifest using `safeReadProjectManifestOnly`, fetch continues
307
- **Directory access errors**: Thrown as standard Node.js file system errors
308
- **Permission errors**: Thrown with appropriate error codes (EACCES, etc.)
309
- **ENOENT errors on symlinks**: Caught internally, logged as debug messages, symlink skipped
310
- **node_modules directories**: Automatically filtered out during directory traversal
311

312
**Debug Logging:**
313
The fetcher uses `@pnpm/logger` with category 'directory-fetcher' for debug information:
314
```typescript
315
// Broken symlinks are logged as:
316
// { brokenSymlink: '/absolute/path/to/broken/symlink' }
317
```