or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

cpp-core.mdflexbuffers.mdgo.mdindex.mdjava.mdjavascript.mdpython.mdrust.mdschema-compiler.mdverification.md
tile.json

tessl/npm-flatbuffers

Memory efficient cross-platform serialization library with zero-copy deserialization supporting 15+ programming languages.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
npmpkg:npm/flatbuffers@25.2.x

To install, run

npx @tessl/cli install tessl/npm-flatbuffers@25.2.0

index.mddocs/

FlatBuffers

FlatBuffers is a cross-platform serialization library architected for maximum memory efficiency. It allows direct access to serialized data without parsing or unpacking, while maintaining excellent forward and backward compatibility. The library supports code generation for 15+ programming languages and provides zero-copy deserialization for high-performance applications.

Package Information

  • Package Name: flatbuffers
  • Package Type: cross-platform library
  • Language: C++ (core), with bindings for JavaScript, Python, Java, Go, Rust, C#, Swift, Kotlin, Dart, PHP, Lua, and more
  • Installation:
    • C++: Build from source or package managers
    • JavaScript: npm install flatbuffers
    • Python: pip install flatbuffers
    • Rust: cargo add flatbuffers
    • Java: Maven/Gradle dependency
    • Go: go get github.com/google/flatbuffers/go

Core Imports

C++:

#include "flatbuffers/flatbuffers.h"
using namespace flatbuffers;

JavaScript/TypeScript:

import { Builder, ByteBuffer, Encoding } from 'flatbuffers';

// For FlexBuffers (schema-less)
import * as flexbuffers from 'flatbuffers/flexbuffers';

Python:

import flatbuffers
from flatbuffers import Builder

Java:

import com.google.flatbuffers.FlatBufferBuilder;
import com.google.flatbuffers.Table;

Go:

import flatbuffers "github.com/google/flatbuffers/go"

Rust:

use flatbuffers::{FlatBufferBuilder, Vector, WIPOffset};

Basic Usage

FlatBuffers follows a schema-first approach where you define data structures in .fbs schema files, compile them with flatc to generate language-specific code, then use the generated code to serialize and deserialize data.

1. Define Schema (example.fbs):

namespace MyGame;

table Monster {
  name: string;
  hp: int = 100;
  mana: int = 50;
  pos: Vec3;
}

struct Vec3 {
  x: float;
  y: float;
  z: float;
}

root_type Monster;

2. Generate Code:

flatc --cpp --python --js example.fbs

3. Serialize Data (C++):

#include "example_generated.h"
using namespace MyGame;

FlatBufferBuilder builder(1024);

// Create position
Vec3 pos(1.0f, 2.0f, 3.0f);

// Create monster
auto name = builder.CreateString("Dragon");
auto monster = CreateMonster(builder, name, 150, 75, &pos);

// Finish buffer
builder.Finish(monster);

// Access serialized data
uint8_t *buf = builder.GetBufferPointer();
int size = builder.GetSize();

4. Deserialize Data (C++):

// Get monster from buffer
const Monster* monster = GetMonster(buf);

// Access fields directly (zero-copy)
std::cout << "Name: " << monster->name()->c_str() << std::endl;
std::cout << "HP: " << monster->hp() << std::endl;
std::cout << "Position: " << monster->pos()->x() << ", " 
          << monster->pos()->y() << ", " << monster->pos()->z() << std::endl;

JavaScript/TypeScript Example:

3. Serialize Data (JavaScript/TypeScript):

import { Builder } from 'flatbuffers';
import { Monster, Vec3 } from './example_generated';

const builder = new Builder(1024);

// Create position
const pos = new Vec3(1.0, 2.0, 3.0);
const posOffset = Vec3.pack(builder, pos);

// Create monster
const nameOffset = builder.createString("Dragon");
const monster = Monster.createMonster(
  builder,
  posOffset,  // pos
  75,         // mana
  150,        // hp
  nameOffset  // name
);

// Finish buffer
builder.finish(monster);

// Access serialized data
const buffer = builder.asUint8Array();

4. Deserialize Data (JavaScript/TypeScript):

import { ByteBuffer } from 'flatbuffers';
import { Monster } from './example_generated';

// Create ByteBuffer from serialized data
const bb = new ByteBuffer(buffer);

// Get monster from buffer
const monster = Monster.getRootAsMonster(bb);

// Access fields directly (zero-copy)
console.log(`Name: ${monster.name()}`);
console.log(`HP: ${monster.hp()}`);
const position = monster.pos();
if (position) {
  console.log(`Position: ${position.x}, ${position.y}, ${position.z}`);
}

Architecture

FlatBuffers is built around several key architectural principles:

  • Schema-First Design: All data structures are defined in .fbs files and compiled to generated code
  • Zero-Copy Deserialization: Data can be accessed directly from serialized buffers without parsing
  • Memory Layout Control: Careful buffer layout ensures optimal memory usage and access patterns
  • Cross-Language Compatibility: Identical wire format across all supported languages
  • Forward/Backward Compatibility: VTable-based field access allows schema evolution
  • Code Generation: The flatc compiler generates idiomatic code for each target language

Key components:

  • flatc Compiler: Schema compiler generating language-specific code
  • Builder Classes: Construct FlatBuffer data (FlatBufferBuilder, Builder, etc.)
  • Table/Struct Access: Generated classes for zero-copy data access
  • Verifier Systems: Optional buffer integrity validation
  • Vector Handling: Efficient array-like data structures
  • String Management: Zero-copy string access with encoding support

Capabilities

C++ Core API

The foundational C++ API providing core functionality for buffer creation, data access, verification, and memory management. This forms the basis for all language-specific implementations.

class FlatBufferBuilder {
  FlatBufferBuilder(size_t initial_size = 1024);
  void Clear();
  Offset<String> CreateString(const char *str);
  template<typename T> Offset<Vector<T>> CreateVector(const T *v, size_t len);
  void StartTable(uoffset_t num_fields);
  Offset<T> EndTable();
  void Finish(Offset<T> root);
  uint8_t* GetBufferPointer();
  size_t GetSize();
};

class Table {
  template<typename T> T GetField(voffset_t field, T defaultval) const;
  template<typename P> P GetPointer(voffset_t field);
  bool CheckField(voffset_t field) const;
};

class Vector<T> {
  uoffset_t size() const;
  const T* Get(uoffset_t i) const;
  const T* operator[](uoffset_t i) const;
  const T* Data() const;
};

C++ Core API

Schema Compiler (flatc)

Command-line compiler that transforms .fbs schema files into language-specific generated code. Supports 15+ programming languages and various output formats including binary serialization.

flatc [options] file1.fbs file2.fbs...

# Language generation options:
--cpp              # Generate C++ headers
--java             # Generate Java classes  
--python           # Generate Python files
--js               # Generate JavaScript
--ts               # Generate TypeScript
--go               # Generate Go files
--rust             # Generate Rust files
--csharp           # Generate C# classes
--swift            # Generate Swift files
--kotlin           # Generate Kotlin files
--dart             # Generate Dart files
--php              # Generate PHP files
--lua              # Generate Lua files

# Output format options:
--binary           # Generate binary data files
--json             # Generate JSON representation
--schema           # Generate schema files

Schema Compiler

JavaScript/TypeScript Binding

JavaScript and TypeScript implementation providing browser and Node.js compatibility with the same zero-copy deserialization benefits adapted for JavaScript's memory model.

class Builder {
  constructor(initialSize?: number);
  clear(): void;
  createString(s: string): number;
  startObject(numFields: number): void;
  endObject(): number;
  finish(rootTable: number): void;
  asUint8Array(): Uint8Array;
}

class ByteBuffer {
  constructor(bytes: Uint8Array);
  readInt32(offset: number): number;
  readFloat64(offset: number): number;
  __string(offset: number): string;
}

JavaScript/TypeScript

Python Binding

Python implementation providing Pythonic API while maintaining the performance benefits of zero-copy access where possible within Python's memory management model.

class Builder:
    def __init__(self, initialSize: int = 1024) -> None: ...
    def Clear(self) -> None: ...
    def CreateString(self, s: str) -> int: ...
    def StartObject(self, numFields: int) -> None: ...
    def EndObject(self) -> int: ...
    def Finish(self, rootTable: int) -> None: ...
    def Bytes(self) -> bytes: ...

class Table:
    def Get(self, slot: int) -> int: ...
    def String(self, off: int) -> str: ...
    def Vector(self, off: int) -> int: ...

Python

Java Binding

Java implementation providing type-safe access to FlatBuffer data with integration into Java's memory model and garbage collection system.

public class FlatBufferBuilder {
    public FlatBufferBuilder(int initialSize);
    public void clear();
    public int createString(String s);
    public void startObject(int numFields);
    public int endObject();
    public void finish(int rootTable);
    public byte[] sizedByteArray();
}

public class Table {
    protected int __offset(int vtableOffset);
    protected String __string(int offset);
    protected int __vector_len(int offset);
}

Java

Go Binding

Go implementation leveraging Go's memory management and providing idiomatic Go interfaces for FlatBuffer operations.

type Builder struct { ... }

func NewBuilder(initialSize int) *Builder
func (b *Builder) Reset()
func (b *Builder) CreateString(s string) UOffsetT
func (b *Builder) StartObject(numFields int)
func (b *Builder) EndObject() UOffsetT
func (b *Builder) Finish(rootTable UOffsetT)
func (b *Builder) FinishedBytes() []byte

Go

Rust Binding

Rust implementation providing memory-safe access with zero-cost abstractions and integration with Rust's ownership model and type system.

pub struct FlatBufferBuilder<'a> {
    // Implementation
}

impl<'a> FlatBufferBuilder<'a> {
    pub fn new() -> Self;
    pub fn reset(&mut self);
    pub fn create_string(&mut self, s: &str) -> WIPOffset<&str>;
    pub fn start_table(&mut self, num_fields: usize);
    pub fn end_table(&mut self) -> WIPOffset<Table>;
    pub fn finish<T>(&mut self, root: WIPOffset<T>, file_identifier: Option<&str>);
    pub fn finished_data(&self) -> &[u8];
}

Rust

Buffer Verification

Optional verification system for validating FlatBuffer integrity, detecting corruption, and ensuring safe access to potentially untrusted data.

class Verifier {
  struct Options {
    uoffset_t max_depth = 64;
    uoffset_t max_tables = 1000000;
    bool check_alignment = true;
    bool check_nested_flatbuffers = true; 
  };
  
  Verifier(const uint8_t *buf, size_t buf_len, const Options &opts = {});
  bool Verify();
  template<typename T> bool VerifyTable(const T *table);
  bool VerifyString(const String *str);
};

// Usage
bool VerifyMonsterBuffer(const uint8_t *buf, size_t size) {
  Verifier verifier(buf, size);
  return verifier.VerifyBuffer<Monster>(nullptr);
}

Verification

FlexBuffers

Schema-less variant of FlatBuffers for dynamic data structures where the schema is not known at compile time, providing JSON-like flexibility with FlatBuffers efficiency.

namespace flexbuffers {
  class Builder {
    Builder(size_t initial_size = 256);
    void Int(int64_t i);
    void Float(double f);  
    void String(const char *str);
    void StartVector();
    void EndVector();
    void StartMap();
    void EndMap();
    std::vector<uint8_t> GetBuffer();
  };
  
  class Reference {
    int64_t AsInt64() const;
    double AsDouble() const;
    const char *AsString() const;
    Reference operator[](size_t i) const;
    Reference operator[](const char *key) const;
  };
}

FlexBuffers