Distributed p2p database on IPFS with automatic peer synchronization and conflict-free writes
—
Quality
Pending
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
The OpLog (Operation Log) is OrbitDB's core data structure - an immutable, cryptographically verifiable, operation-based CRDT (Conflict-free Replicated Data Type) that enables distributed consensus without coordination. All OrbitDB databases are built on top of the OpLog.
The Log provides the fundamental operations for building distributed data structures.
/**
* Appends an entry to the log
* @param entry The entry to append
* @returns Promise resolving to the entry hash
*/
append(entry: Entry): Promise<string>;
/**
* Joins this log with another log, merging their histories
* @param log The log to join with
* @returns Promise resolving to the merged log
*/
join(log: Log): Promise<Log>;
interface Log {
/** Current heads of the log (latest entries) */
heads: Entry[];
/** All entries in the log */
entries: Entry[];
/** Number of entries in the log */
length: number;
/** Unique identifier for this log */
id: string;
/** Access controller for this log */
accessController: AccessController;
}Usage Examples:
import { Log } from '@orbitdb/core';
// Logs are typically accessed through database instances
const ipfs = await createHelia();
const orbitdb = await createOrbitDB({ ipfs });
const db = await orbitdb.open('my-events');
// Access the underlying log
const log = db.log;
console.log('Log heads:', log.heads);
console.log('Total entries:', log.length);
// Add data (which creates log entries)
await db.add('Hello World');
console.log('New length:', log.length);Log entries are the atomic units of data in OrbitDB, containing the operation, metadata, and cryptographic proofs.
interface Entry {
/** Unique hash identifying this entry */
hash: string;
/** The operation payload */
payload: {
/** Operation type (ADD, PUT, DEL, etc.) */
op: string;
/** Operation key (null for some operations) */
key: string | null;
/** Operation value */
value: any;
};
/** Identity that created this entry */
identity: string;
/** Cryptographic signature */
sig: string;
/** Logical clock for ordering */
clock: Clock;
/** References to previous entries */
refs: string[];
/** Additional entry metadata */
meta?: object;
}Usage Examples:
// Entries are created automatically when you perform database operations
const db = await orbitdb.open('my-events');
const hash = await db.add('My event data');
// Find the entry by hash
const entries = db.log.entries;
const entry = entries.find(e => e.hash === hash);
console.log('Entry hash:', entry.hash);
console.log('Operation:', entry.payload.op);
console.log('Value:', entry.payload.value);
console.log('Creator:', entry.identity);
console.log('Clock:', entry.clock);Entries can be created and verified independently of the database context.
/**
* Creates a new log entry
* @param identity The identity creating the entry
* @param operation The operation data
* @param clock Optional logical clock
* @returns Promise resolving to new Entry
*/
Entry.create(
identity: Identity,
operation: { op: string; key?: string; value?: any },
clock?: Clock
): Promise<Entry>;
/**
* Verifies an entry's signature and integrity
* @param entry The entry to verify
* @returns Promise resolving to true if valid
*/
Entry.verify(entry: Entry): Promise<boolean>;Usage Examples:
import { Entry } from '@orbitdb/core';
// Create a custom entry (typically done internally)
const identity = orbitdb.identity;
const operation = { op: 'ADD', value: 'Custom data' };
const entry = await Entry.create(identity, operation);
// Verify entry integrity
const isValid = await Entry.verify(entry);
console.log('Entry is valid:', isValid);Logical clocks provide causal ordering for distributed operations without requiring synchronized time.
interface Clock {
/** Unique identifier for the clock (usually identity ID) */
id: string;
/** Logical time value */
time: number;
}
/**
* Creates a new clock
* @param id Clock identifier
* @param time Initial time (default: 0)
* @returns New Clock instance
*/
Clock.create(id: string, time?: number): Clock;
/**
* Compares two clocks for ordering
* @param a First clock
* @param b Second clock
* @returns -1, 0, or 1 for less than, equal, or greater than
*/
Clock.compare(a: Clock, b: Clock): number;
/**
* Advances a clock
* @param clock Clock to advance
* @param other Optional other clock to sync with
* @returns New advanced clock
*/
Clock.tick(clock: Clock, other?: Clock): Clock;Usage Examples:
import { Clock } from '@orbitdb/core';
// Clocks are typically managed automatically
const db = await orbitdb.open('my-events');
await db.add('First event');
await db.add('Second event');
// Examine the logical clocks in entries
const entries = db.log.entries;
entries.forEach((entry, index) => {
console.log(`Entry ${index}:`, entry.clock);
});
// Create custom clocks (advanced usage)
const clock1 = Clock.create('peer1', 0);
const clock2 = Clock.create('peer2', 0);
const clock3 = Clock.tick(clock1); // Advances time to 1The OpLog includes a default access controller that allows all operations.
/**
* Creates a default access controller that allows all operations
* @returns Promise resolving to access controller instance
*/
function DefaultAccessController(): Promise<AccessController>;
interface AccessController {
/** Checks if an entry can be appended (always returns true for default) */
canAppend(entry: Entry): Promise<boolean>;
}Usage Examples:
import { DefaultAccessController } from '@orbitdb/core';
// The default access controller is used automatically
// when no specific access controller is provided
const defaultAC = await DefaultAccessController();
console.log(await defaultAC.canAppend(someEntry)); // Always trueThe OpLog system automatically handles conflicts when merging concurrent operations from different peers.
interface ConflictResolution {
/** Resolves conflicts between concurrent entries */
resolve(entries: Entry[]): Entry[];
}Usage Examples:
// Conflict resolution happens automatically during log joins
const db1 = await orbitdb1.open('/orbitdb/shared-address');
const db2 = await orbitdb2.open('/orbitdb/shared-address');
// Both peers add concurrent entries
await db1.add('From peer 1');
await db2.add('From peer 2');
// When peers sync, conflicts are resolved automatically
await db1.sync(db2.log.heads);
const allEntries = await db1.all();
console.log('Merged data:', allEntries); // Contains both entriesLogs can be synchronized between peers to maintain eventual consistency.
/**
* Synchronizes this log with heads from another log
* @param heads Array of head entries from another log
* @returns Promise resolving when sync is complete
*/
sync(heads: Entry[]): Promise<void>;
/**
* Gets the difference between this log and another
* @param other Another log to compare with
* @returns Array of entries that are in other but not in this log
*/
difference(other: Log): Entry[];Usage Examples:
// Manual synchronization between database instances
const db1 = await orbitdb1.open('shared-db');
const db2 = await orbitdb2.open('shared-db');
// Add data to first database
await db1.add('Data from peer 1');
// Sync second database with first
await db2.sync(db1.log.heads);
// Both databases now have the same data
const data1 = await db1.all();
const data2 = await db2.all();
console.log('Synchronized:', JSON.stringify(data1) === JSON.stringify(data2));Access lower-level log operations for custom database implementations.
interface Log {
/** Traverse log entries with optional filtering */
traverse(options?: {
amount?: number;
gt?: string;
gte?: string;
lt?: string;
lte?: string;
}): AsyncIterable<Entry>;
/** Get entries by their hashes */
get(hashes: string[]): Entry[];
/** Check if log contains specific entries */
has(hash: string): boolean;
/** Get log statistics */
stats(): {
length: number;
heads: number;
unique: number;
};
}Usage Examples:
const db = await orbitdb.open('my-events');
// Add some data
await db.add('Event 1');
await db.add('Event 2');
await db.add('Event 3');
const log = db.log;
// Traverse recent entries
for await (const entry of log.traverse({ amount: 2 })) {
console.log('Recent entry:', entry.payload.value);
}
// Check log statistics
const stats = log.stats();
console.log('Log stats:', stats);
// Check if specific entry exists
const hasEntry = log.has(someEntryHash);
console.log('Entry exists:', hasEntry);When building custom database types, you can define custom operation types.
// Example: Custom counter database operations
const CounterOperations = {
INCREMENT: 'INCREMENT',
DECREMENT: 'DECREMENT',
RESET: 'RESET'
};
// Custom database implementation
const CounterDB = () => async (params) => {
const database = await Database(params);
const { addOperation } = database;
const increment = async () => {
return addOperation({
op: CounterOperations.INCREMENT,
key: null,
value: 1
});
};
const decrement = async () => {
return addOperation({
op: CounterOperations.DECREMENT,
key: null,
value: -1
});
};
const reset = async () => {
return addOperation({
op: CounterOperations.RESET,
key: null,
value: 0
});
};
return {
...database,
increment,
decrement,
reset
};
};Handle common OpLog-related errors:
try {
const db = await orbitdb.open('my-db');
await db.add('Some data');
} catch (error) {
if (error.message.includes('Access denied')) {
console.error('No permission to write to this database');
} else if (error.message.includes('Invalid entry')) {
console.error('Entry validation failed');
} else {
console.error('Unexpected error:', error.message);
}
}The OpLog system provides the foundation for all OrbitDB operations, ensuring data integrity, conflict resolution, and eventual consistency across distributed peers.
Install with Tessl CLI
npx tessl i tessl/npm-orbitdb--core