0
# Storage Metadata
1

2
Hash-based and sequence-based metadata system for tracking payloads in blockchain storage, including data hash tracking, storage hash management, and sequence parsing for temporal ordering in the XYO network.
3

4
## Capabilities
5

6
### Storage Metadata Interface
7

8
Complete storage metadata combining hash and sequence information for comprehensive payload tracking.
9

10
```typescript { .api }
11
/**
12
 * Import Hash type from @xylabs/hex
13
 */
14
import type { Hash } from '@xylabs/hex';
15

16
/**
17
 * Complete storage metadata interface combining hash and sequence tracking
18
 */
19
interface StorageMeta extends SequenceStorageMeta, HashStorageMeta {
20
  _hash: Hash;
21
  _dataHash: Hash;
22
  _sequence: Sequence;
23
}
24

25
/**
26
 * Add storage metadata to payload type
27
 */
28
type WithStorageMeta<T extends Payload = Payload> = T & StorageMeta;
29

30
/**
31
 * Add partial storage metadata to payload type
32
 */
33
type WithPartialStorageMeta<T extends Payload = Payload> = T & Partial<StorageMeta>;
34

35
/**
36
 * Type guard for complete storage metadata
37
 */
38
function isStorageMeta(value: unknown): value is StorageMeta;
39

40
/**
41
 * Type assertion for storage metadata
42
 */
43
function asStorageMeta(value: unknown): StorageMeta;
44

45
/**
46
 * Optional type assertion for storage metadata
47
 */
48
function asOptionalStorageMeta(value: unknown): StorageMeta | undefined;
49
```
50

51
**Usage Examples:**
52

53
```typescript
54
import { 
55
  StorageMeta, 
56
  WithStorageMeta, 
57
  isStorageMeta,
58
  Payload 
59
} from "@xyo-network/payload-model";
60

61
// Payload with complete storage metadata
62
interface UserPayloadWithStorage extends WithStorageMeta<Payload> {
63
  schema: "network.example.user";
64
  name: string;
65
  email: string;
66
}
67

68
const userWithStorage: UserPayloadWithStorage = {
69
  schema: "network.example.user",
70
  name: "Alice",
71
  email: "alice@example.com",
72
  _hash: "0x123...",
73
  _dataHash: "0x456...",
74
  _sequence: "1234567890abcdef"
75
};
76

77
// Validate storage metadata
78
if (isStorageMeta(userWithStorage)) {
79
  console.log("Hash:", userWithStorage._hash);
80
  console.log("Data Hash:", userWithStorage._dataHash);
81
  console.log("Sequence:", userWithStorage._sequence);
82
}
83

84
// Process payloads with storage metadata
85
function processStoredPayload(payload: WithStorageMeta<Payload>) {
86
  return {
87
    schema: payload.schema,
88
    hash: payload._hash,
89
    dataHash: payload._dataHash,
90
    sequence: payload._sequence
91
  };
92
}
93
```
94

95
### Hash Storage Metadata
96

97
Hash-based tracking system for payload storage including both content hash and storage hash.
98

99
```typescript { .api }
100
/**
101
 * Data hash storage metadata
102
 */
103
interface DataHashStorageMeta {
104
  _dataHash: Hash;
105
}
106

107
/**
108
 * Complete hash storage metadata extending data hash with storage hash
109
 */
110
interface HashStorageMeta extends DataHashStorageMeta {
111
  _hash: Hash;
112
  _dataHash: Hash;
113
}
114

115
/**
116
 * Add data hash metadata to payload type
117
 */
118
type WithDataHashStorageMeta<T extends Payload = Payload> = T & DataHashStorageMeta;
119

120
/**
121
 * Add hash metadata to payload type
122
 */
123
type WithHashStorageMeta<T extends Payload = Payload> = T & HashStorageMeta;
124

125
/**
126
 * Type guard for data hash metadata
127
 */
128
function isDataHashStorageMeta(value: unknown): value is DataHashStorageMeta;
129

130
/**
131
 * Type guard for complete hash metadata
132
 */
133
function isHashStorageMeta(value: unknown): value is HashStorageMeta;
134

135
/**
136
 * Type assertion for data hash metadata
137
 */
138
function asDataHashStorageMeta(value: unknown): DataHashStorageMeta;
139

140
/**
141
 * Type assertion for hash metadata
142
 */
143
function asHashStorageMeta(value: unknown): HashStorageMeta;
144
```
145

146
**Usage Examples:**
147

148
```typescript
149
import { 
150
  HashStorageMeta,
151
  DataHashStorageMeta,
152
  WithHashStorageMeta,
153
  isHashStorageMeta,
154
  isDataHashStorageMeta
155
} from "@xyo-network/payload-model";
156

157
// Payload with hash metadata
158
interface TrackedPayload extends WithHashStorageMeta<Payload> {
159
  schema: "network.example.tracked";
160
  data: string;
161
}
162

163
const trackedPayload: TrackedPayload = {
164
  schema: "network.example.tracked",
165
  data: "important data",
166
  _hash: "0x789...",      // Storage hash
167
  _dataHash: "0xabc..."   // Content hash
168
};
169

170
// Validate hash metadata
171
if (isHashStorageMeta(trackedPayload)) {
172
  console.log("Storage hash:", trackedPayload._hash);
173
  console.log("Content hash:", trackedPayload._dataHash);
174
}
175

176
// Check for data hash only
177
const partialPayload = {
178
  schema: "network.example.partial",
179
  data: "some data",
180
  _dataHash: "0xdef..."
181
};
182

183
if (isDataHashStorageMeta(partialPayload)) {
184
  console.log("Has data hash:", partialPayload._dataHash);
185
}
186
```
187

188
### Sequence Storage Metadata
189

190
Sequence-based metadata for temporal ordering and tracking in the XYO network.
191

192
```typescript { .api }
193
/**
194
 * Sequence storage metadata
195
 */
196
interface SequenceStorageMeta {
197
  _sequence: Sequence;
198
}
199

200
/**
201
 * Add sequence metadata to payload type
202
 */
203
type WithSequenceStorageMeta<T extends Payload = Payload> = T & SequenceStorageMeta;
204

205
/**
206
 * Type guard for sequence metadata
207
 */
208
function isSequenceStorageMeta(value: unknown): value is SequenceStorageMeta;
209

210
/**
211
 * Type assertion for sequence metadata
212
 */
213
function asSequenceStorageMeta(value: unknown): SequenceStorageMeta;
214

215
/**
216
 * Optional type assertion for sequence metadata
217
 */
218
function asOptionalSequenceStorageMeta(value: unknown): SequenceStorageMeta | undefined;
219
```
220

221
### Sequence Types and Constants
222

223
Comprehensive sequence type system with local and qualified sequences for XYO network ordering.
224

225
```typescript { .api }
226
/**
227
 * Import types from @xylabs/hex for branded hex types
228
 */
229
import type { Address, Brand, Hex } from '@xylabs/hex';
230

231
/**
232
 * Local sequence type (epoch + nonce)
233
 */
234
type LocalSequence = Brand<Hex, { __localSequence: true }>;
235

236
/**
237
 * Qualified sequence type (local sequence + address)
238
 */
239
type QualifiedSequence = Brand<Hex, { __qualifiedSequence: true }>;
240

241
/**
242
 * Union of local and qualified sequences
243
 */
244
type Sequence = LocalSequence | QualifiedSequence;
245

246
/**
247
 * Epoch component of sequence
248
 */
249
type Epoch = Brand<Hex, { __epoch: true }>;
250

251
/**
252
 * Nonce component of sequence
253
 */
254
type Nonce = Brand<Hex, { __nonce: true }>;
255

256
/**
257
 * Type guard for epoch
258
 */
259
function isEpoch(value: unknown): value is Epoch;
260

261
/**
262
 * Type guard for nonce
263
 */
264
function isNonce(value: unknown): value is Nonce;
265

266
/**
267
 * Type guard for local sequence
268
 */
269
function isLocalSequence(value: unknown): value is LocalSequence;
270

271
/**
272
 * Type guard for qualified sequence
273
 */
274
function isQualifiedSequence(value: unknown): value is QualifiedSequence;
275

276
/**
277
 * Type guard for any sequence type
278
 */
279
function isSequence(value: unknown): value is Sequence;
280

281
/**
282
 * Constants for sequence component lengths and limits
283
 */
284
const SequenceConstants: {
285
  // Component lengths in bytes
286
  epochBytes: 8;
287
  nonceBytes: 8;
288
  nonceIndexBytes: 4;
289
  nonceHashBytes: 4;
290
  addressBytes: 20;
291
  localSequenceBytes: 16;  // epochBytes + nonceBytes
292
  qualifiedSequenceBytes: 36; // localSequenceBytes + addressBytes
293
  
294
  // Min/max values for components
295
  minEpoch: Epoch;
296
  maxEpoch: Epoch;
297
  minNonce: Nonce;
298
  maxNonce: Nonce;
299
  minAddress: Address;
300
  maxAddress: Address;
301
  minLocalSequence: LocalSequence;
302
  maxLocalSequence: LocalSequence;
303
  minQualifiedSequence: QualifiedSequence;
304
  maxQualifiedSequence: QualifiedSequence;
305
};
306

307
/**
308
 * Local sequence specific constants
309
 */
310
const LocalSequenceConstants: typeof SequenceConstants & {
311
  localSequenceBytes: 16;
312
  minLocalSequence: LocalSequence;
313
  maxLocalSequence: LocalSequence;
314
};
315

316
/**
317
 * Qualified sequence specific constants  
318
 */
319
const QualifiedSequenceConstants: {
320
  qualifiedSequenceBytes: 36;
321
  minQualifiedSequence: QualifiedSequence;
322
  maxQualifiedSequence: QualifiedSequence;
323
};
324
```
325

326
**Usage Examples:**
327

328
```typescript
329
import { 
330
  Sequence,
331
  LocalSequence,
332
  QualifiedSequence,
333
  isSequence,
334
  isLocalSequence,
335
  isQualifiedSequence,
336
  SequenceConstants
337
} from "@xyo-network/payload-model";
338

339
// Work with different sequence types
340
const localSeq: LocalSequence = "1234567890abcdef12345678" as LocalSequence;
341
const qualifiedSeq: QualifiedSequence = "1234567890abcdef12345678901234567890abcdef12345678901234567890abcdef" as QualifiedSequence;
342

343
// Validate sequence types
344
function processSequence(seq: unknown) {
345
  if (isLocalSequence(seq)) {
346
    console.log("Local sequence:", seq);
347
  } else if (isQualifiedSequence(seq)) {
348
    console.log("Qualified sequence:", seq);
349
  } else if (isSequence(seq)) {
350
    console.log("Unknown sequence type:", seq);
351
  } else {
352
    console.log("Not a sequence");
353
  }
354
}
355

356
// Use sequence constants
357
console.log("Local sequence bytes:", SequenceConstants.localSequenceBytes);
358
console.log("Qualified sequence bytes:", SequenceConstants.qualifiedSequenceBytes);
359
console.log("Min local sequence:", SequenceConstants.minLocalSequence);
360
```
361

362
### Sequence Parser
363

364
Advanced sequence parsing and manipulation utilities for extracting components and creating sequences.
365

366
```typescript { .api }
367
/**
368
 * Sequence parser class for extracting and manipulating sequence components
369
 */
370
class SequenceParser {
371
  /**
372
   * Create parser from existing sequence
373
   */
374
  static from(sequence: Sequence, address?: Address): SequenceParser;
375
  
376
  /**
377
   * Create parser from timestamp and hash components
378
   */
379
  static from(timestamp: Hex, hash: Hash, address?: Address): SequenceParser;
380
  static from(timestamp: Hex, hash: Hex, address?: Address): SequenceParser;
381
  static from(timestamp: Hex, nonce: Nonce, address?: Address): SequenceParser;
382
  static from(timestamp: number, hash: Hash, address?: Address): SequenceParser;
383
  static from(timestamp: number, hash: Hex, address?: Address): SequenceParser;
384
  static from(timestamp: number, nonce: Nonce, address?: Address): SequenceParser;
385
  
386
  /**
387
   * Create parser from timestamp, hash, and index components
388
   */
389
  static from(timestamp: Hex, hash: Hash, index?: number, address?: Address): SequenceParser;
390
  static from(timestamp: Hex, hash: Hex, index?: number, address?: Address): SequenceParser;
391
  static from(timestamp: Hex, nonce: Nonce, index?: number, address?: Address): SequenceParser;
392
  static from(timestamp: number, hash: Hash, index?: number, address?: Address): SequenceParser;
393
  static from(timestamp: number, hash: Hex, index?: number, address?: Address): SequenceParser;
394
  static from(timestamp: number, nonce: Nonce, index?: number, address?: Address): SequenceParser;
395

396
  /**
397
   * Parse sequence from hex, string, or ArrayBuffer
398
   */
399
  static parse(value: Hex | string | ArrayBufferLike): SequenceParser;
400

401
  /**
402
   * Convert value to epoch component
403
   */
404
  static toEpoch(value: number | Hex | Epoch): Epoch;
405

406
  /**
407
   * Convert value to nonce component
408
   */
409
  static toNonce(value: Hash | Hex, index?: number): Nonce;
410

411
  /**
412
   * Extract address component from qualified sequence
413
   */
414
  get address(): Address;
415

416
  /**
417
   * Extract epoch component
418
   */
419
  get epoch(): Epoch;
420

421
  /**
422
   * Extract local sequence (epoch + nonce)
423
   */
424
  get localSequence(): LocalSequence;
425

426
  /**
427
   * Extract nonce component
428
   */
429
  get nonce(): Nonce;
430

431
  /**
432
   * Extract qualified sequence (epoch + nonce + address)
433
   */
434
  get qualifiedSequence(): QualifiedSequence;
435
}
436
```
437

438
**Usage Examples:**
439

440
```typescript
441
import { SequenceParser } from "@xyo-network/payload-model";
442

443
// Create sequence from timestamp and hash
444
const timestamp = Date.now();
445
const hash = "0x123456789abcdef";
446
const address = "0x742d35Cc6065C6EaABf23bA0aC21e0017E3BB26C";
447

448
const parser = SequenceParser.from(timestamp, hash, address);
449

450
// Extract components
451
console.log("Epoch:", parser.epoch);
452
console.log("Nonce:", parser.nonce);
453
console.log("Address:", parser.address);
454
console.log("Local Sequence:", parser.localSequence);
455
console.log("Qualified Sequence:", parser.qualifiedSequence);
456

457
// Parse existing sequence
458
const existingSequence = "1234567890abcdef12345678901234567890abcdef12345678901234567890abcdef";
459
const existingParser = SequenceParser.parse(existingSequence);
460

461
console.log("Parsed epoch:", existingParser.epoch);
462
console.log("Parsed nonce:", existingParser.nonce);
463

464
// Create epoch and nonce utilities
465
const epoch = SequenceParser.toEpoch(timestamp);
466
const nonce = SequenceParser.toNonce(hash, 0);
467

468
console.log("Created epoch:", epoch);
469
console.log("Created nonce:", nonce);
470
```
471

472
### Sequence Comparison
473

474
Utilities for comparing and ordering sequences in the XYO network.
475

476
```typescript { .api }
477
/**
478
 * Sequence comparison utilities
479
 */
480
const SequenceComparer: {
481
  /**
482
   * Compare sequences by local sequence component only
483
   */
484
  local: (a: Sequence, b: Sequence) => number;
485

486
  /**
487
   * Compare sequences by qualified sequence (includes address)
488
   */
489
  qualified: (a: Sequence, b: Sequence) => number;
490
};
491
```
492

493
**Usage Examples:**
494

495
```typescript
496
import { SequenceComparer, Sequence } from "@xyo-network/payload-model";
497

498
const sequences: Sequence[] = [
499
  "1234567890abcdef12345678" as Sequence,
500
  "2345678901bcdef123456789" as Sequence,
501
  "0123456789abcdef12345678" as Sequence
502
];
503

504
// Sort by local sequence
505
const sortedByLocal = sequences.sort(SequenceComparer.local);
506
console.log("Sorted by local:", sortedByLocal);
507

508
// Sort by qualified sequence
509
const qualifiedSequences: Sequence[] = [
510
  "1234567890abcdef12345678901234567890abcdef12345678901234567890abcdef" as Sequence,
511
  "2345678901bcdef123456789012345678901bcdef123456789012345678901bcdef1" as Sequence
512
];
513

514
const sortedByQualified = qualifiedSequences.sort(SequenceComparer.qualified);
515
console.log("Sorted by qualified:", sortedByQualified);
516

517
// Use in payload sorting
518
interface PayloadWithSequence {
519
  schema: string;
520
  data: any;
521
  _sequence: Sequence;
522
}
523

524
function sortPayloadsBySequence(payloads: PayloadWithSequence[]) {
525
  return payloads.sort((a, b) => SequenceComparer.local(a._sequence, b._sequence));
526
}
527
```
528

529
### Zod Integration for Storage Metadata
530

531
Runtime validation schemas for storage metadata using Zod integration.
532

533
```typescript { .api }
534
/**
535
 * Zod schema for storage metadata validation
536
 */
537
const StorageMetaZod: ZodType<{
538
  _hash: Hash;
539
  _dataHash: Hash; 
540
  _sequence: Sequence;
541
}>;
542

543
/**
544
 * Zod schema for payload with storage metadata
545
 */
546
const PayloadWithStorageMetaZod: ZodType<PayloadWithStorageMeta>;
547

548
/**
549
 * Sequence validation schemas
550
 */
551
const LocalSequenceToStringZod: ZodType<string>;
552
const LocalSequenceFromStringZod: ZodType<LocalSequence>;
553
const QualifiedSequenceToStringZod: ZodType<string>;
554
const QualifiedSequenceFromStringZod: ZodType<QualifiedSequence>;
555
const SequenceToStringZod: ZodType<string>;
556
const SequenceFromStringZod: ZodType<Sequence>;
557

558
/**
559
 * Helper to add storage metadata validation to existing Zod schema
560
 */
561
function WithStorageMetaZod<T extends ZodType<any>>(
562
  valueZod: T
563
): ZodType<StorageMeta & T>;
564
```
565

566
**Usage Examples:**
567

568
```typescript
569
import { 
570
  StorageMetaZod,
571
  PayloadWithStorageMetaZod,
572
  SequenceFromStringZod,
573
  WithStorageMetaZod
574
} from "@xyo-network/payload-model";
575
import * as z from "zod";
576

577
// Validate storage metadata
578
const storageData = {
579
  _hash: "0x123...",
580
  _dataHash: "0x456...",
581
  _sequence: "1234567890abcdef12345678"
582
};
583

584
const validatedStorage = StorageMetaZod.parse(storageData);
585
console.log("Valid storage metadata:", validatedStorage);
586

587
// Validate payload with storage metadata
588
const payloadData = {
589
  schema: "network.example.test",
590
  data: "test data",
591
  _hash: "0x123...",
592
  _dataHash: "0x456...",
593
  _sequence: "1234567890abcdef12345678"
594
};
595

596
const validatedPayload = PayloadWithStorageMetaZod.parse(payloadData);
597

598
// Create custom schema with storage metadata
599
const CustomPayloadZod = z.object({
600
  schema: z.literal("network.example.custom"),
601
  message: z.string(),
602
  timestamp: z.number()
603
});
604

605
const CustomPayloadWithStorageZod = WithStorageMetaZod(CustomPayloadZod);
606

607
// Validate sequence
608
const sequenceString = "1234567890abcdef12345678";
609
const validatedSequence = SequenceFromStringZod.parse(sequenceString);
610
console.log("Valid sequence:", validatedSequence);
611
```
612

613
## Types Reference
614

615
### Storage Metadata Types
616

617
- **`StorageMeta`**: Complete storage metadata interface
618
- **`DataHashStorageMeta`**: Data hash metadata interface
619
- **`HashStorageMeta`**: Complete hash metadata interface
620
- **`SequenceStorageMeta`**: Sequence metadata interface
621

622
### Sequence Types
623

624
- **`Sequence`**: Union of LocalSequence and QualifiedSequence
625
- **`LocalSequence`**: Epoch + Nonce sequence type
626
- **`QualifiedSequence`**: LocalSequence + Address sequence type
627
- **`Epoch`**: Timestamp component of sequence
628
- **`Nonce`**: Hash + index component of sequence
629

630
### Utility Types
631

632
- **`WithStorageMeta<T>`**: Add complete storage metadata to type
633
- **`WithHashStorageMeta<T>`**: Add hash metadata to type
634
- **`WithSequenceStorageMeta<T>`**: Add sequence metadata to type
635

636
### Constants
637

638
- **`SequenceConstants`**: Sequence component lengths and limits
639
- **`LocalSequenceConstants`**: Local sequence specific constants
640
- **`QualifiedSequenceConstants`**: Qualified sequence specific constants
641

642
### Classes
643

644
- **`SequenceParser`**: Sequence parsing and manipulation utility class
645

646
### Comparison Utilities
647

648
- **`SequenceComparer.local`**: Compare sequences by local component
649
- **`SequenceComparer.qualified`**: Compare sequences by full qualified sequence