or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

buffer-writing.mdindex.mdindexed-reading.mdindexing.mditeration.mdreading.mdwriting.md
tile.json

tessl/npm-ipld--car

Content Addressable aRchive format reader and writer for IPLD data structures.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/@ipld/car@5.4.x

To install, run

npx @tessl/cli install tessl/npm-ipld--car@5.4.0

index.mddocs/

@ipld/car

@ipld/car is a JavaScript implementation of the Content Addressable aRchive (CAR) format, enabling developers to read and write CAR files for IPLD (InterPlanetary Linked Data) applications. It offers comprehensive functionality for creating CAR archives with single or multiple roots, reading from streams or buffers, indexing for efficient random access, and iterating through blocks within archives.

Package Information

  • Package Name: @ipld/car
  • Package Type: npm
  • Language: TypeScript/JavaScript
  • Installation: npm install @ipld/car

Core Imports

import { CarReader, CarWriter, CarIndexer, CarBlockIterator, CarCIDIterator, CarIndexedReader, CarBufferWriter } from "@ipld/car";

For specific modules (smaller bundle sizes):

import { CarReader } from "@ipld/car/reader";
import { CarWriter } from "@ipld/car/writer";
import { CarIndexer } from "@ipld/car/indexer";
import { CarBlockIterator, CarCIDIterator } from "@ipld/car/iterator";
import { CarIndexedReader } from "@ipld/car/indexed-reader"; // Node.js only
import * as CarBufferWriter from "@ipld/car/buffer-writer"; // Node.js only
import { readHeader, readBlockHead, createDecoder, bytesReader, asyncIterableReader, chunkReader, limitReader } from "@ipld/car/decoder";

CommonJS:

const { CarReader, CarWriter, CarIndexer } = require("@ipld/car");
const { CarReader } = require("@ipld/car/reader");

Basic Usage

import fs from 'fs';
import { Readable } from 'stream';
import { CarReader, CarWriter } from '@ipld/car';
import * as raw from 'multiformats/codecs/raw';
import { CID } from 'multiformats/cid';
import { sha256 } from 'multiformats/hashes/sha2';

// Create a simple CAR file
const bytes = new TextEncoder().encode('hello world');
const hash = await sha256.digest(raw.encode(bytes));
const cid = CID.create(1, raw.code, hash);

// Write CAR file
const { writer, out } = CarWriter.create([cid]);
Readable.from(out).pipe(fs.createWriteStream('hello.car'));
await writer.put({ cid, bytes });
await writer.close();

// Read CAR file
const inStream = fs.createReadStream('hello.car');
const reader = await CarReader.fromIterable(inStream);
const roots = await reader.getRoots();
const block = await reader.get(roots[0]);
console.log(new TextDecoder().decode(block.bytes)); // "hello world"

Architecture

@ipld/car is built around several key components:

  • CAR Format: Implements the CAR (Content Addressable aRchive) specification for storing IPLD blocks
  • Reading Interface: Multiple reader implementations for different use cases (in-memory, streaming, indexed)
  • Writing Interface: Writer implementations for creating CAR archives (streaming, buffer-based)
  • Iterator Pattern: Efficient iteration over blocks and CIDs without loading entire archive into memory
  • Platform Support: Browser and Node.js variants with platform-specific optimizations
  • Type Safety: Full TypeScript definitions for all APIs and data structures

Capabilities

CAR Reading

Core functionality for reading CAR archives with different performance characteristics. CarReader loads entire archive into memory, while iterators provide streaming access.

class CarReader {
  readonly version: number;
  getRoots(): Promise<CID[]>;
  has(key: CID): Promise<boolean>;
  get(key: CID): Promise<Block | undefined>;
  blocks(): AsyncGenerator<Block>;
  cids(): AsyncGenerator<CID>;
  static fromBytes(bytes: Uint8Array): Promise<CarReader>;
  static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarReader>;
}

CAR Reading

CAR Writing

Streaming writer interface for creating CAR archives with backpressure support and flexible root management.

class CarWriter {
  put(block: Block): Promise<void>;
  close(): Promise<void>;
  static create(roots?: CID[] | CID): WriterChannel;
  static createAppender(): WriterChannel;
  static updateRootsInBytes(bytes: Uint8Array, roots: CID[]): Promise<Uint8Array>;
}

interface WriterChannel {
  writer: CarWriter;
  out: AsyncIterable<Uint8Array>;
}

CAR Writing

CAR Indexing

Efficient indexing functionality for creating block indices and enabling random access to large CAR files.

class CarIndexer {
  readonly version: number;
  getRoots(): Promise<CID[]>;
  [Symbol.asyncIterator](): AsyncIterator<BlockIndex>;
  static fromBytes(bytes: Uint8Array): Promise<CarIndexer>;
  static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarIndexer>;
}

interface BlockIndex {
  cid: CID;
  length: number;
  blockLength: number;
  offset: number;
  blockOffset: number;
}

CAR Indexing

Block and CID Iteration

Memory-efficient iteration over CAR contents without loading entire archive. Ideal for processing large archives or streaming scenarios.

class CarBlockIterator {
  readonly version: number;
  getRoots(): Promise<CID[]>;
  [Symbol.asyncIterator](): AsyncIterator<Block>;
  static fromBytes(bytes: Uint8Array): Promise<CarBlockIterator>;
  static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarBlockIterator>;
}

class CarCIDIterator {
  readonly version: number;
  getRoots(): Promise<CID[]>;
  [Symbol.asyncIterator](): AsyncIterator<CID>;
  static fromBytes(bytes: Uint8Array): Promise<CarCIDIterator>;
  static fromIterable(asyncIterable: AsyncIterable<Uint8Array>): Promise<CarCIDIterator>;
}

Block and CID Iteration

Indexed Reading (Node.js)

File-based indexed reading with random access capabilities. Pre-indexes CAR files for efficient block retrieval by CID.

class CarIndexedReader {
  readonly version: number;
  getRoots(): Promise<CID[]>;
  has(key: CID): Promise<boolean>;
  get(key: CID): Promise<Block | undefined>;
  blocks(): AsyncGenerator<Block>;
  cids(): AsyncGenerator<CID>;
  close(): Promise<void>;
  static fromFile(path: string): Promise<CarIndexedReader>;
}

Indexed Reading

Buffer Writing (Node.js)

Synchronous buffer-based CAR writing for performance-critical scenarios where the final CAR size is known in advance.

interface CarBufferWriter {
  addRoot(root: CID, options?: { resize?: boolean }): CarBufferWriter;
  write(block: Block): CarBufferWriter;
  close(options?: { resize?: boolean }): Uint8Array;
}

function createWriter(buffer: ArrayBuffer, options?: CarBufferWriterOptions): CarBufferWriter;
function blockLength(block: Block): number;
function headerLength(options: { roots: CID[] }): number;
function estimateHeaderLength(rootCount: number, rootByteLength?: number): number;

Buffer Writing

Low-Level Decoding Utilities

Low-level functions for custom CAR decoding implementations and advanced use cases requiring direct control over the decoding process.

function readHeader(reader: BytesReader, strictVersion?: number): Promise<CarHeader | CarV2Header>;
function readBlockHead(reader: BytesReader): Promise<BlockHeader>;
function createDecoder(reader: BytesReader): CarDecoder;
function bytesReader(bytes: Uint8Array): BytesReader;
function asyncIterableReader(asyncIterable: AsyncIterable<Uint8Array>): BytesReader;
function chunkReader(readChunk: () => Promise<Uint8Array | null>): BytesReader;
function limitReader(reader: BytesReader, byteLimit: number): BytesReader;

These utilities are used internally by other classes but can be imported directly for custom CAR processing implementations.

Core Types

interface Block {
  cid: CID;
  bytes: Uint8Array;
}

interface BlockHeader {
  cid: CID;
  length: number;
  blockLength: number;
}

interface CarBufferWriterOptions {
  roots?: CID[];
  byteOffset?: number;
  byteLength?: number;
  headerSize?: number;
}

interface BytesReader {
  upTo(length: number): Promise<Uint8Array>;
  exactly(length: number): Promise<Uint8Array>;
  seek(length: number): void;
  pos: number;
}

interface CarHeader {
  version: 1;
  roots: CID[];
}

interface CarV2Header extends CarHeader {
  version: 2;
  characteristics: [bigint, bigint];
  dataOffset: number;
  dataSize: number;
  indexOffset: number;
}

interface CarDecoder {
  header(): Promise<CarHeader | CarV2Header>;
  blocks(): AsyncGenerator<Block>;
  blocksIndex(): AsyncGenerator<BlockIndex>;
}