CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-orbitdb--core

Distributed p2p database on IPFS with automatic peer synchronization and conflict-free writes

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

oplog-system.mddocs/

OpLog System

The OpLog (Operation Log) is OrbitDB's core data structure - an immutable, cryptographically verifiable, operation-based CRDT (Conflict-free Replicated Data Type) that enables distributed consensus without coordination. All OrbitDB databases are built on top of the OpLog.

Capabilities

Log Operations

The Log provides the fundamental operations for building distributed data structures.

/**
 * Appends an entry to the log
 * @param entry The entry to append
 * @returns Promise resolving to the entry hash
 */
append(entry: Entry): Promise<string>;

/**
 * Joins this log with another log, merging their histories
 * @param log The log to join with
 * @returns Promise resolving to the merged log
 */
join(log: Log): Promise<Log>;

interface Log {
  /** Current heads of the log (latest entries) */
  heads: Entry[];
  /** All entries in the log */
  entries: Entry[];
  /** Number of entries in the log */
  length: number;
  /** Unique identifier for this log */
  id: string;
  /** Access controller for this log */
  accessController: AccessController;
}

Usage Examples:

import { Log } from '@orbitdb/core';

// Logs are typically accessed through database instances
const ipfs = await createHelia();
const orbitdb = await createOrbitDB({ ipfs });
const db = await orbitdb.open('my-events');

// Access the underlying log
const log = db.log;

console.log('Log heads:', log.heads);
console.log('Total entries:', log.length);

// Add data (which creates log entries)
await db.add('Hello World');
console.log('New length:', log.length);

Entry Structure

Log entries are the atomic units of data in OrbitDB, containing the operation, metadata, and cryptographic proofs.

interface Entry {
  /** Unique hash identifying this entry */
  hash: string;
  /** The operation payload */
  payload: {
    /** Operation type (ADD, PUT, DEL, etc.) */
    op: string;
    /** Operation key (null for some operations) */
    key: string | null;
    /** Operation value */
    value: any;
  };
  /** Identity that created this entry */
  identity: string;
  /** Cryptographic signature */
  sig: string;
  /** Logical clock for ordering */
  clock: Clock;
  /** References to previous entries */
  refs: string[];
  /** Additional entry metadata */
  meta?: object;
}

Usage Examples:

// Entries are created automatically when you perform database operations
const db = await orbitdb.open('my-events');
const hash = await db.add('My event data');

// Find the entry by hash
const entries = db.log.entries;
const entry = entries.find(e => e.hash === hash);

console.log('Entry hash:', entry.hash);
console.log('Operation:', entry.payload.op);
console.log('Value:', entry.payload.value);
console.log('Creator:', entry.identity);
console.log('Clock:', entry.clock);

Entry Creation and Verification

Entries can be created and verified independently of the database context.

/**
 * Creates a new log entry
 * @param identity The identity creating the entry
 * @param operation The operation data
 * @param clock Optional logical clock
 * @returns Promise resolving to new Entry
 */
Entry.create(
  identity: Identity, 
  operation: { op: string; key?: string; value?: any }, 
  clock?: Clock
): Promise<Entry>;

/**
 * Verifies an entry's signature and integrity
 * @param entry The entry to verify
 * @returns Promise resolving to true if valid
 */
Entry.verify(entry: Entry): Promise<boolean>;

Usage Examples:

import { Entry } from '@orbitdb/core';

// Create a custom entry (typically done internally)
const identity = orbitdb.identity;
const operation = { op: 'ADD', value: 'Custom data' };
const entry = await Entry.create(identity, operation);

// Verify entry integrity
const isValid = await Entry.verify(entry);
console.log('Entry is valid:', isValid);

Logical Clocks

Logical clocks provide causal ordering for distributed operations without requiring synchronized time.

interface Clock {
  /** Unique identifier for the clock (usually identity ID) */
  id: string;
  /** Logical time value */
  time: number;
}

/**
 * Creates a new clock
 * @param id Clock identifier
 * @param time Initial time (default: 0)
 * @returns New Clock instance
 */
Clock.create(id: string, time?: number): Clock;

/**
 * Compares two clocks for ordering
 * @param a First clock
 * @param b Second clock
 * @returns -1, 0, or 1 for less than, equal, or greater than
 */
Clock.compare(a: Clock, b: Clock): number;

/**
 * Advances a clock
 * @param clock Clock to advance
 * @param other Optional other clock to sync with
 * @returns New advanced clock
 */
Clock.tick(clock: Clock, other?: Clock): Clock;

Usage Examples:

import { Clock } from '@orbitdb/core';

// Clocks are typically managed automatically
const db = await orbitdb.open('my-events');
await db.add('First event');
await db.add('Second event');

// Examine the logical clocks in entries
const entries = db.log.entries;
entries.forEach((entry, index) => {
  console.log(`Entry ${index}:`, entry.clock);
});

// Create custom clocks (advanced usage)
const clock1 = Clock.create('peer1', 0);
const clock2 = Clock.create('peer2', 0);
const clock3 = Clock.tick(clock1); // Advances time to 1

Default Access Controller

The OpLog includes a default access controller that allows all operations.

/**
 * Creates a default access controller that allows all operations
 * @returns Promise resolving to access controller instance
 */
function DefaultAccessController(): Promise<AccessController>;

interface AccessController {
  /** Checks if an entry can be appended (always returns true for default) */
  canAppend(entry: Entry): Promise<boolean>;
}

Usage Examples:

import { DefaultAccessController } from '@orbitdb/core';

// The default access controller is used automatically
// when no specific access controller is provided
const defaultAC = await DefaultAccessController();
console.log(await defaultAC.canAppend(someEntry)); // Always true

Conflict Resolution

The OpLog system automatically handles conflicts when merging concurrent operations from different peers.

interface ConflictResolution {
  /** Resolves conflicts between concurrent entries */
  resolve(entries: Entry[]): Entry[];
}

Usage Examples:

// Conflict resolution happens automatically during log joins
const db1 = await orbitdb1.open('/orbitdb/shared-address');
const db2 = await orbitdb2.open('/orbitdb/shared-address');

// Both peers add concurrent entries
await db1.add('From peer 1');
await db2.add('From peer 2');

// When peers sync, conflicts are resolved automatically
await db1.sync(db2.log.heads);
const allEntries = await db1.all();
console.log('Merged data:', allEntries); // Contains both entries

Log Synchronization

Logs can be synchronized between peers to maintain eventual consistency.

/**
 * Synchronizes this log with heads from another log
 * @param heads Array of head entries from another log
 * @returns Promise resolving when sync is complete
 */
sync(heads: Entry[]): Promise<void>;

/**
 * Gets the difference between this log and another
 * @param other Another log to compare with
 * @returns Array of entries that are in other but not in this log
 */
difference(other: Log): Entry[];

Usage Examples:

// Manual synchronization between database instances
const db1 = await orbitdb1.open('shared-db');
const db2 = await orbitdb2.open('shared-db');

// Add data to first database
await db1.add('Data from peer 1');

// Sync second database with first
await db2.sync(db1.log.heads);

// Both databases now have the same data
const data1 = await db1.all();
const data2 = await db2.all();
console.log('Synchronized:', JSON.stringify(data1) === JSON.stringify(data2));

Advanced Log Operations

Access lower-level log operations for custom database implementations.

interface Log {
  /** Traverse log entries with optional filtering */
  traverse(options?: {
    amount?: number;
    gt?: string;
    gte?: string;
    lt?: string;
    lte?: string;
  }): AsyncIterable<Entry>;
  
  /** Get entries by their hashes */
  get(hashes: string[]): Entry[];
  
  /** Check if log contains specific entries */
  has(hash: string): boolean;
  
  /** Get log statistics */
  stats(): {
    length: number;
    heads: number;
    unique: number;
  };
}

Usage Examples:

const db = await orbitdb.open('my-events');

// Add some data
await db.add('Event 1');
await db.add('Event 2');
await db.add('Event 3');

const log = db.log;

// Traverse recent entries
for await (const entry of log.traverse({ amount: 2 })) {
  console.log('Recent entry:', entry.payload.value);
}

// Check log statistics
const stats = log.stats();
console.log('Log stats:', stats);

// Check if specific entry exists
const hasEntry = log.has(someEntryHash);
console.log('Entry exists:', hasEntry);

Custom Operations

When building custom database types, you can define custom operation types.

// Example: Custom counter database operations
const CounterOperations = {
  INCREMENT: 'INCREMENT',
  DECREMENT: 'DECREMENT',
  RESET: 'RESET'
};

// Custom database implementation
const CounterDB = () => async (params) => {
  const database = await Database(params);
  const { addOperation } = database;
  
  const increment = async () => {
    return addOperation({
      op: CounterOperations.INCREMENT,
      key: null,
      value: 1
    });
  };
  
  const decrement = async () => {
    return addOperation({
      op: CounterOperations.DECREMENT,
      key: null,
      value: -1
    });
  };
  
  const reset = async () => {
    return addOperation({
      op: CounterOperations.RESET,
      key: null,
      value: 0
    });
  };
  
  return {
    ...database,
    increment,
    decrement,
    reset
  };
};

Error Handling

Handle common OpLog-related errors:

try {
  const db = await orbitdb.open('my-db');
  await db.add('Some data');
} catch (error) {
  if (error.message.includes('Access denied')) {
    console.error('No permission to write to this database');
  } else if (error.message.includes('Invalid entry')) {
    console.error('Entry validation failed');
  } else {
    console.error('Unexpected error:', error.message);
  }
}

The OpLog system provides the foundation for all OrbitDB operations, ensuring data integrity, conflict resolution, and eventual consistency across distributed peers.

Install with Tessl CLI

npx tessl i tessl/npm-orbitdb--core

docs

access-controllers.md

address-management.md

database-types.md

identity-system.md

index.md

oplog-system.md

orbitdb-factory.md

storage-backends.md

tile.json