or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

buffer-management.md graph-processing.md index.md input-output-streams.md serialization-utilities.md

tile.json

Graph Processing

Graph processing capabilities enable serialization of complex object graphs containing references, shared objects, and circular dependencies. Standard Protocol Buffer format cannot handle object references, but protostuff-core provides specialized utilities for these advanced scenarios.

GraphIOUtil

Main utility class for serializing and deserializing object graphs with reference tracking and cycle detection.

public final class GraphIOUtil {
    public static <T> byte[] toByteArray(T root, Schema<T> schema, LinkedBuffer buffer);
    public static <T> void writeTo(OutputStream out, T root, Schema<T> schema, LinkedBuffer buffer) throws IOException;
    public static <T> void writeTo(DataOutput out, T root, Schema<T> schema, LinkedBuffer buffer) throws IOException;
    public static <T> void mergeFrom(byte[] data, T message, Schema<T> schema) throws IOException;
    public static <T> void mergeFrom(InputStream in, T message, Schema<T> schema) throws IOException;
    public static <T> void mergeFrom(DataInput in, T message, Schema<T> schema) throws IOException;
}

Key Features

Reference Tracking: Maintains object identity across serialization/deserialization
Cycle Detection: Handles circular references without infinite loops
Shared Objects: Serializes shared object instances only once
Memory Efficiency: Optimizes storage for graphs with many references

Usage Examples

Serializing Object Graphs

// Object graph with circular references
Person john = new Person("John");
Person jane = new Person("Jane");
john.spouse = jane;
jane.spouse = john; // circular reference

// Serialize the graph
LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
byte[] graphData = GraphIOUtil.toByteArray(john, personSchema, buffer);

Deserializing Object Graphs

// Deserialize maintaining object references
Person deserializedJohn = personSchema.newMessage();
GraphIOUtil.mergeFrom(graphData, deserializedJohn, personSchema);

// Verify circular reference is preserved
assert deserializedJohn.spouse.spouse == deserializedJohn;

Shared Objects

// Graph with shared objects
Company company = new Company("TechCorp");
Person employee1 = new Person("Alice", company);
Person employee2 = new Person("Bob", company);

// Both employees reference the same company instance
assert employee1.company == employee2.company;

// Serialize preserving shared reference
byte[] data = GraphIOUtil.toByteArray(Arrays.asList(employee1, employee2), listSchema, buffer);

Graph Input Classes

GraphCodedInput

Wraps any CodedInput to add graph processing capabilities.

public final class GraphCodedInput extends CodedInput {
    public GraphCodedInput(CodedInput input);
}

Features:

Reference resolution during deserialization
Maintains object identity mapping
Handles forward references
Compatible with any underlying CodedInput implementation

Usage:

ByteArrayInput baseInput = new ByteArrayInput(graphData, false);
GraphCodedInput graphInput = new GraphCodedInput(baseInput);

// Use with schema for graph deserialization
schema.mergeFrom(graphInput, message);

GraphByteArrayInput

Specialized byte array input optimized for graph processing.

public final class GraphByteArrayInput extends ByteArrayInput {
    public GraphByteArrayInput(byte[] buffer, boolean decodeNestedMessageAsGroup);
}

Combines the efficiency of ByteArrayInput with built-in graph processing capabilities.

Graph Output Classes

GraphProtostuffOutput

Output implementation that tracks object references during serialization.

public final class GraphProtostuffOutput implements Output {
    public GraphProtostuffOutput(LinkedBuffer buffer);
}

Features:

Automatic reference detection
Cycle prevention
Shared object optimization
Compatible with Protostuff format

Internal Process:

Tracks all objects during traversal
Assigns unique IDs to objects
Replaces duplicate objects with reference IDs
Maintains reference table for deserialization

When to Use Graph Processing

Appropriate Use Cases

Complex Domain Models:

// E-commerce domain with bidirectional relationships
Customer customer = new Customer("John Doe");
Order order = new Order(customer);
customer.orders.add(order); // bidirectional reference

// Graph serialization preserves relationships
byte[] data = GraphIOUtil.toByteArray(customer, customerSchema, buffer);

Shared Configuration Objects:

// Multiple services sharing configuration
DatabaseConfig dbConfig = new DatabaseConfig("localhost", 5432);
UserService userService = new UserService(dbConfig);
OrderService orderService = new OrderService(dbConfig);

// Serialize service cluster
ServiceCluster cluster = new ServiceCluster(userService, orderService);
byte[] clusterData = GraphIOUtil.toByteArray(cluster, clusterSchema, buffer);

Object Hierarchies with Cross-References:

// Organization chart with cross-department references
Department engineering = new Department("Engineering");
Department marketing = new Department("Marketing");
Employee manager = new Employee("Alice", engineering);
Project project = new Project("Launch", manager, Arrays.asList(engineering, marketing));

// Complex relationships preserved
byte[] orgData = GraphIOUtil.toByteArray(project, projectSchema, buffer);

When NOT to Use Graph Processing

Simple Tree Structures:

Standard protobuf serialization is more efficient
No circular references or shared objects
Clear parent-child hierarchies

Performance-Critical Scenarios:

Graph processing adds overhead
Reference tracking requires additional memory
Standard serialization is faster for simple objects

Cross-Platform Compatibility:

Graph format is protostuff-specific
Other protobuf implementations cannot read graph data
Use standard format for interoperability

Graph Format Details

Reference Encoding

The graph format extends standard protostuff encoding with:

Object Registry: Maps objects to unique IDs
Reference Fields: Special fields containing object IDs instead of data
Definition Phase: First occurrence contains full object data
Reference Phase: Subsequent occurrences contain only ID references

Wire Format Example

// First occurrence: full object definition
field_id: 1, object_id: 100, data: { name: "John", age: 30 }

// Later reference: just the ID
field_id: 2, reference_id: 100

Error Handling

Graph processing can throw additional exceptions:

// Custom exceptions for graph processing
public class GraphException extends RuntimeException {
    public GraphException(String message);
    public GraphException(String message, Throwable cause);
}

Common Error Scenarios:

Unresolvable forward references
Corrupted reference tables
Incompatible schema versions
Memory limits exceeded during large graph processing

Performance Considerations

Memory Usage

Reference tracking requires additional memory overhead
Large graphs may need increased heap space
Consider streaming for very large object graphs

Serialization Performance

Graph processing is slower than standard serialization
Time complexity increases with graph connectivity
Profile before using in performance-critical paths

Optimization Tips

Buffer Sizing:

// Use larger buffers for complex graphs
LinkedBuffer buffer = LinkedBuffer.allocate(64 * 1024); // 64KB

Schema Design:

// Minimize deep nesting in graph schemas
// Use flat structures with references when possible

Selective Graph Processing:

// Only use graph utilities for objects that actually need it
if (hasCircularReferences(object)) {
    GraphIOUtil.toByteArray(object, schema, buffer);
} else {
    ProtostuffIOUtil.toByteArray(object, schema, buffer);
}

Compatibility

Graph-serialized data can only be read by protostuff-core
Not compatible with standard Protocol Buffer parsers
Use standard utilities for cross-platform scenarios
Graph format is stable across protostuff versions

Version

Tile

Files