0
# @ipld/car
1
2
@ipld/car is a JavaScript implementation of the Content Addressable aRchive (CAR) format, enabling developers to read and write CAR files for IPLD (InterPlanetary Linked Data) applications. It offers comprehensive functionality for creating CAR archives with single or multiple roots, reading from streams or buffers, indexing for efficient random access, and iterating through blocks within archives.
3
4
## Package Information
5
6
- **Package Name**: @ipld/car
7
- **Package Type**: npm
8
- **Language**: TypeScript/JavaScript
9
- **Installation**: `npm install @ipld/car`
10
11
## Core Imports
12
13
```typescript
14
import { CarReader, CarWriter, CarIndexer, CarBlockIterator, CarCIDIterator, CarIndexedReader, CarBufferWriter } from "@ipld/car";
15
```
16
17
For specific modules (smaller bundle sizes):
18
19
```typescript
20
import { CarReader } from "@ipld/car/reader";
21
import { CarWriter } from "@ipld/car/writer";
22
import { CarIndexer } from "@ipld/car/indexer";
23
import { CarBlockIterator, CarCIDIterator } from "@ipld/car/iterator";
24
import { CarIndexedReader } from "@ipld/car/indexed-reader"; // Node.js only
25
import * as CarBufferWriter from "@ipld/car/buffer-writer"; // Node.js only
26
import { readHeader, readBlockHead, createDecoder, bytesReader, asyncIterableReader, chunkReader, limitReader } from "@ipld/car/decoder";
27
```
28
29
CommonJS:
30
31
```javascript
32
const { CarReader, CarWriter, CarIndexer } = require("@ipld/car");
33
const { CarReader } = require("@ipld/car/reader");
34
```
35
36
## Basic Usage
37
38
```typescript
39
import fs from 'fs';
40
import { Readable } from 'stream';
41
import { CarReader, CarWriter } from '@ipld/car';
42
import * as raw from 'multiformats/codecs/raw';
43
import { CID } from 'multiformats/cid';
44
import { sha256 } from 'multiformats/hashes/sha2';
45
46
// Create a simple CAR file
47
const bytes = new TextEncoder().encode('hello world');
48
const hash = await sha256.digest(raw.encode(bytes));
49
const cid = CID.create(1, raw.code, hash);
50
51
// Write CAR file
52
const { writer, out } = CarWriter.create([cid]);
53
Readable.from(out).pipe(fs.createWriteStream('hello.car'));
54
await writer.put({ cid, bytes });
55
await writer.close();
56
57
// Read CAR file
58
const inStream = fs.createReadStream('hello.car');
59
const reader = await CarReader.fromIterable(inStream);
60
const roots = await reader.getRoots();
61
const block = await reader.get(roots[0]);
62
console.log(new TextDecoder().decode(block.bytes)); // "hello world"
63
```
64
65
## Architecture
66
67
@ipld/car is built around several key components:
68
69
- **CAR Format**: Implements the CAR (Content Addressable aRchive) specification for storing IPLD blocks
70
- **Reading Interface**: Multiple reader implementations for different use cases (in-memory, streaming, indexed)
71
- **Writing Interface**: Writer implementations for creating CAR archives (streaming, buffer-based)
72
- **Iterator Pattern**: Efficient iteration over blocks and CIDs without loading entire archive into memory
73
- **Platform Support**: Browser and Node.js variants with platform-specific optimizations
74
- **Type Safety**: Full TypeScript definitions for all APIs and data structures
75
76
## Capabilities
77
78
### CAR Reading
79
80
Core functionality for reading CAR archives with different performance characteristics. CarReader loads entire archive into memory, while iterators provide streaming access.
81
82
```typescript { .api }
83
class CarReader {
84
readonly version: number;
85
getRoots(): Promise<CID[]>;
86
has(key: CID): Promise<boolean>;
87
get(key: CID): Promise<Block | undefined>;
88
blocks(): AsyncGenerator<Block>;
89
cids(): AsyncGenerator<CID>;
90
static fromBytes(bytes: Uint8Array): Promise<CarReader>;
91
static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarReader>;
92
}
93
```
94
95
[CAR Reading](./reading.md)
96
97
### CAR Writing
98
99
Streaming writer interface for creating CAR archives with backpressure support and flexible root management.
100
101
```typescript { .api }
102
class CarWriter {
103
put(block: Block): Promise<void>;
104
close(): Promise<void>;
105
static create(roots?: CID[] | CID): WriterChannel;
106
static createAppender(): WriterChannel;
107
static updateRootsInBytes(bytes: Uint8Array, roots: CID[]): Promise<Uint8Array>;
108
}
109
110
interface WriterChannel {
111
writer: CarWriter;
112
out: AsyncIterable<Uint8Array>;
113
}
114
```
115
116
[CAR Writing](./writing.md)
117
118
### CAR Indexing
119
120
Efficient indexing functionality for creating block indices and enabling random access to large CAR files.
121
122
```typescript { .api }
123
class CarIndexer {
124
readonly version: number;
125
getRoots(): Promise<CID[]>;
126
[Symbol.asyncIterator](): AsyncIterator<BlockIndex>;
127
static fromBytes(bytes: Uint8Array): Promise<CarIndexer>;
128
static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarIndexer>;
129
}
130
131
interface BlockIndex {
132
cid: CID;
133
length: number;
134
blockLength: number;
135
offset: number;
136
blockOffset: number;
137
}
138
```
139
140
[CAR Indexing](./indexing.md)
141
142
### Block and CID Iteration
143
144
Memory-efficient iteration over CAR contents without loading entire archive. Ideal for processing large archives or streaming scenarios.
145
146
```typescript { .api }
147
class CarBlockIterator {
148
readonly version: number;
149
getRoots(): Promise<CID[]>;
150
[Symbol.asyncIterator](): AsyncIterator<Block>;
151
static fromBytes(bytes: Uint8Array): Promise<CarBlockIterator>;
152
static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarBlockIterator>;
153
}
154
155
class CarCIDIterator {
156
readonly version: number;
157
getRoots(): Promise<CID[]>;
158
[Symbol.asyncIterator](): AsyncIterator<CID>;
159
static fromBytes(bytes: Uint8Array): Promise<CarCIDIterator>;
160
static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarCIDIterator>;
161
}
162
```
163
164
[Block and CID Iteration](./iteration.md)
165
166
### Indexed Reading (Node.js)
167
168
File-based indexed reading with random access capabilities. Pre-indexes CAR files for efficient block retrieval by CID.
169
170
```typescript { .api }
171
class CarIndexedReader {
172
readonly version: number;
173
getRoots(): Promise<CID[]>;
174
has(key: CID): Promise<boolean>;
175
get(key: CID): Promise<Block | undefined>;
176
blocks(): AsyncGenerator<Block>;
177
cids(): AsyncGenerator<CID>;
178
close(): Promise<void>;
179
static fromFile(path: string): Promise<CarIndexedReader>;
180
}
181
```
182
183
[Indexed Reading](./indexed-reading.md)
184
185
### Buffer Writing (Node.js)
186
187
Synchronous buffer-based CAR writing for performance-critical scenarios where the final CAR size is known in advance.
188
189
```typescript { .api }
190
interface CarBufferWriter {
191
addRoot(root: CID, options?: { resize?: boolean }): CarBufferWriter;
192
write(block: Block): CarBufferWriter;
193
close(options?: { resize?: boolean }): Uint8Array;
194
}
195
196
function createWriter(buffer: ArrayBuffer, options?: CarBufferWriterOptions): CarBufferWriter;
197
function blockLength(block: Block): number;
198
function headerLength(options: { roots: CID[] }): number;
199
function estimateHeaderLength(rootCount: number, rootByteLength?: number): number;
200
```
201
202
[Buffer Writing](./buffer-writing.md)
203
204
### Low-Level Decoding Utilities
205
206
Low-level functions for custom CAR decoding implementations and advanced use cases requiring direct control over the decoding process.
207
208
```typescript { .api }
209
function readHeader(reader: BytesReader, strictVersion?: number): Promise<CarHeader | CarV2Header>;
210
function readBlockHead(reader: BytesReader): Promise<BlockHeader>;
211
function createDecoder(reader: BytesReader): CarDecoder;
212
function bytesReader(bytes: Uint8Array): BytesReader;
213
function asyncIterableReader(asyncIterable: AsyncIterable<Uint8Array>): BytesReader;
214
function chunkReader(readChunk: () => Promise<Uint8Array | null>): BytesReader;
215
function limitReader(reader: BytesReader, byteLimit: number): BytesReader;
216
```
217
218
These utilities are used internally by other classes but can be imported directly for custom CAR processing implementations.
219
220
## Core Types
221
222
```typescript { .api }
223
interface Block {
224
cid: CID;
225
bytes: Uint8Array;
226
}
227
228
interface BlockHeader {
229
cid: CID;
230
length: number;
231
blockLength: number;
232
}
233
234
interface CarBufferWriterOptions {
235
roots?: CID[];
236
byteOffset?: number;
237
byteLength?: number;
238
headerSize?: number;
239
}
240
241
interface BytesReader {
242
upTo(length: number): Promise<Uint8Array>;
243
exactly(length: number): Promise<Uint8Array>;
244
seek(length: number): void;
245
pos: number;
246
}
247
248
interface CarHeader {
249
version: 1;
250
roots: CID[];
251
}
252
253
interface CarV2Header extends CarHeader {
254
version: 2;
255
characteristics: [bigint, bigint];
256
dataOffset: number;
257
dataSize: number;
258
indexOffset: number;
259
}
260
261
interface CarDecoder {
262
header(): Promise<CarHeader | CarV2Header>;
263
blocks(): AsyncGenerator<Block>;
264
blocksIndex(): AsyncGenerator<BlockIndex>;
265
}
266
```