or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

examples

edge-cases.mdreal-world-scenarios.md
index.md
tile.json

tuple-layer.mddocs/reference/

Tuple Layer

The Tuple Layer provides type-safe encoding and decoding of structured data as database keys in FoundationDB. It allows you to work with tuples of typed values (integers, strings, bytes, floats, booleans, nested tuples, UUIDs, and versionstamps) while maintaining proper ordering semantics for range queries. The Tuple Layer is essential for creating hierarchical and structured key spaces, enabling you to build complex data models on top of FoundationDB's ordered key-value store.

Package Information

  • Module/Package:
    • Python: fdb.tuple
    • Go: github.com/apple/foundationdb/bindings/go/src/fdb/tuple
    • Java: com.apple.foundationdb.tuple
    • Ruby: FDB::Tuple (module)
  • Purpose: Type-safe key encoding with proper ordering
  • Layer Type: Data model abstraction built on key-value layer
  • Status: Standard across all language bindings

Core Imports

Python

from fdb import tuple
# or
import fdb.tuple

Go

import "github.com/apple/foundationdb/bindings/go/src/fdb/tuple"

Java

import com.apple.foundationdb.tuple.Tuple;
import com.apple.foundationdb.tuple.Versionstamp;

Ruby

require 'fdb'
# Tuple functions are in FDB::Tuple module

Core Concepts

Type-Safe Key Encoding

The Tuple Layer encodes tuples of typed values into byte strings suitable for use as FoundationDB keys. Each element type has a specific encoding that preserves ordering semantics, meaning that if tuple A sorts before tuple B semantically, then the encoded bytes of A will sort before the encoded bytes of B lexicographically.

Ordering Semantics

Tuples are ordered lexicographically by their elements. The ordering rules are:

  1. Elements are compared left to right
  2. Types are ordered: None/null < bytes < strings < integers < floats < false < true < UUID < versionstamps < nested tuples
  3. Within the same type, natural ordering applies (numeric order for numbers, lexicographic for strings, etc.)
  4. Shorter tuples sort before longer tuples with the same prefix

Type Support

The Tuple Layer supports the following types:

  • Null values: None (Python), nil (Go/Ruby), null (Java)
  • Byte strings: Raw binary data
  • Unicode strings: UTF-8 encoded text
  • Integers: Signed integers of arbitrary precision (language-dependent limits)
  • Floating point: IEEE 754 single (32-bit) and double (64-bit) precision
  • Booleans: true and false values
  • UUID: RFC 4122 universally unique identifiers
  • Versionstamps: 96-bit transaction commit version stamps
  • Nested tuples: Tuples containing other tuples

Encoding Efficiency

  • Integers use variable-length encoding (1-8 bytes based on magnitude)
  • Strings are null-terminated with escaping for embedded nulls
  • Floats use IEEE binary representation with bit flipping for ordering
  • Null values and booleans use zero-byte representations (type code only)

Capabilities

Tuple Encoding (Pack)

Convert structured data into bytes suitable for database keys.

# Python
def pack(t: tuple, prefix: bytes = b'') -> bytes:
    """
    Pack a tuple of items into a byte string.

    Args:
        t: Tuple of values to encode
        prefix: Optional prefix to prepend to the result

    Returns:
        Encoded byte string suitable as a key
    """
    ...
// Go
type Tuple []TupleElement

func (t Tuple) Pack() []byte
// Pack tuple into byte string

// Usage: tuple.Tuple{"user", 12345, "profile"}.Pack()
// Java
public class Tuple {
    public static Tuple from(Object... items);
    public byte[] pack();

    // Usage: Tuple.from("user", 12345, "profile").pack()
}
# Ruby
module FDB::Tuple
  def self.pack(t)
    # Pack array of elements into byte string
    # t: Array of values to encode
    # Returns: Encoded String
  end
end

Example: Basic Tuple Packing

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

# Pack simple values
key1 = tuple.pack(("users", 12345))
# b'\x02users\x00\x150\x39'

# Pack with prefix
key2 = tuple.pack(("profile", "email"), prefix=b'\x01app\x00')
# b'\x01app\x00\x02profile\x00\x02email\x00'

# Pack nested tuple
key3 = tuple.pack(("users", 12345, ("metadata", "created")))
# Nested tuple encoded with type 0x05

# Use in transaction
@fdb.transactional
def store_user_data(tr, user_id, field, value):
    key = tuple.pack(("users", user_id, field))
    tr[key] = value

store_user_data(db, 12345, "name", b"Alice")

Tuple Decoding (Unpack)

Decode byte strings back into structured tuples.

# Python
def unpack(key: bytes, prefix_len: int = 0) -> tuple:
    """
    Unpack a byte string into a tuple of items.

    Args:
        key: Encoded byte string to decode
        prefix_len: Number of bytes to skip before decoding

    Returns:
        Tuple of decoded values
    """
    ...
// Go
func Unpack(b []byte) (Tuple, error)
// Unpack byte string into tuple

// Returns tuple and error if decoding fails
// Java
public class Tuple {
    public static Tuple fromBytes(byte[] bytes);
    public List<?> getItems();

    // Usage: Tuple.fromBytes(encodedKey).getItems()
}
# Ruby
module FDB::Tuple
  def self.unpack(key)
    # Unpack byte string into array
    # key: Encoded String to decode
    # Returns: Array of decoded values
  end
end

Example: Tuple Unpacking

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def get_user_fields(tr, user_id):
    # Create key range for all fields of a user
    prefix = tuple.pack(("users", user_id))

    results = {}
    for key, value in tr.get_range_startswith(prefix):
        # Unpack the key to get the field name
        unpacked = tuple.unpack(key)
        # unpacked = ("users", 12345, "name")
        field_name = unpacked[2]
        results[field_name] = value

    return results

user_data = get_user_fields(db, 12345)
# {'name': b'Alice', 'email': b'alice@example.com', 'age': b'30'}

Range Generation for Tuple Prefixes

Generate key ranges for tuple prefix queries, essential for hierarchical data access.

# Python
def range(t: tuple = ()) -> tuple[bytes, bytes]:
    """
    Get key range for a tuple prefix.

    Args:
        t: Tuple prefix to generate range for

    Returns:
        Tuple of (begin_key, end_key) for range queries
    """
    ...
// Go
// Tuple implements FDBRangeKeys interface
func (t Tuple) FDBRangeKeys() (fdb.KeyConvertible, fdb.KeyConvertible)
// Returns begin and end keys for range

// Can be used directly in GetRange calls
// Java
public class Tuple {
    public static Range range(byte[] prefix);

    // Returns Range object for getRange queries
    // Usage: Tuple.from("users", 12345).range()
}
# Ruby
module FDB::Tuple
  def self.range(tuple = [])
    # Get key range for tuple prefix
    # tuple: Array prefix to generate range for
    # Returns: Array [begin_key, end_key]
  end
end

Example: Tuple Range Queries

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def get_all_user_posts(tr, user_id):
    # Get range for all posts by a user
    begin, end = tuple.range(("posts", user_id))

    posts = []
    for key, value in tr.get_range(begin, end):
        # Each key is like ("posts", user_id, post_id)
        post_id = tuple.unpack(key)[2]
        posts.append((post_id, value))

    return posts

posts = get_all_user_posts(db, 12345)
package main

import (
    "fmt"
    "github.com/apple/foundationdb/bindings/go/src/fdb"
    "github.com/apple/foundationdb/bindings/go/src/fdb/tuple"
)

func main() {
    fdb.MustAPIVersion(740)
    db := fdb.MustOpenDefault()

    // Query all posts for a user
    userID := 12345
    prefix := tuple.Tuple{"posts", userID}

    result, err := db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
        // Use tuple directly as range
        ri := tr.GetRange(prefix, fdb.RangeOptions{}).Iterator()

        var posts []string
        for ri.Advance() {
            kv, err := ri.Get()
            if err != nil {
                return nil, err
            }

            // Unpack key to get post ID
            t, err := tuple.Unpack(kv.Key)
            if err != nil {
                return nil, err
            }
            postID := t[2].(int64)
            posts = append(posts, fmt.Sprintf("Post %d: %s", postID, kv.Value))
        }

        return posts, nil
    })

    if err != nil {
        panic(err)
    }

    for _, post := range result.([]string) {
        fmt.Println(post)
    }
}

Versionstamp Support

Versionstamps embed transaction commit versions into keys or values, enabling conflict-free monotonic key generation and time-based ordering.

Incomplete Versionstamps

Incomplete versionstamps are placeholders that get filled in by FoundationDB at commit time with the actual transaction version.

# Python
def pack_with_versionstamp(items: tuple, prefix: bytes = b'') -> bytes:
    """
    Pack tuple containing an incomplete Versionstamp.

    The tuple must contain exactly one incomplete Versionstamp.
    Returns bytes with a placeholder that will be filled at commit.

    Args:
        items: Tuple containing values including one Versionstamp
        prefix: Optional prefix

    Returns:
        Encoded bytes with versionstamp placeholder
    """
    ...

class Versionstamp:
    def __init__(self, tr_version: Optional[bytes] = None, user_version: int = 0):
        """
        Create a versionstamp.

        Args:
            tr_version: None for incomplete, or 10 bytes for complete
            user_version: 16-bit user-defined ordering value
        """
        ...

    def is_complete(self) -> bool:
        """Check if versionstamp is complete (has transaction version)"""
        ...
// Go
func (t Tuple) PackWithVersionstamp(prefix []byte) ([]byte, error)
// Pack tuple with incomplete versionstamp
// Returns error if tuple doesn't contain exactly one incomplete versionstamp

type Versionstamp struct {
    TransactionVersion [10]byte
    UserVersion        uint16
}

func (v Versionstamp) IsComplete() bool
// Check if versionstamp has been set by database
// Java
public class Tuple {
    public byte[] packWithVersionstamp(byte[] prefix);
    // Pack tuple containing incomplete versionstamp
}

public class Versionstamp {
    // Create incomplete versionstamp
    public static Versionstamp incomplete(int userVersion);

    // Create complete versionstamp
    public Versionstamp(byte[] transactionVersion, int userVersion);

    public boolean isComplete();
}
# Ruby
# Note: The Ruby binding does not provide a FDB::Tuple::Versionstamp class.
# For versionstamped keys and values, use raw byte strings with the incomplete
# versionstamp placeholder (10 bytes of \xFF) and the set_versionstamped_key
# or set_versionstamped_value transaction methods.

Example: Versionstamp Usage

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def append_event(tr, event_type, data):
    # Create key with versionstamp for monotonic ordering
    # The versionstamp will be filled in at commit time
    incomplete_vs = tuple.Versionstamp()

    # Pack tuple with incomplete versionstamp
    key_template = tuple.pack_with_versionstamp(
        ("events", event_type, incomplete_vs)
    )

    # Use versionstamped key mutation
    tr.set_versionstamped_key(key_template, data)

# Each call gets a unique, monotonically increasing key
append_event(db, "login", b"user_id:12345")
append_event(db, "login", b"user_id:67890")

@fdb.transactional
def read_recent_events(tr, event_type, limit=10):
    # Read events in reverse chronological order
    begin, end = tuple.range(("events", event_type))

    events = []
    for key, value in tr.get_range(begin, end, limit=limit, reverse=True):
        unpacked = tuple.unpack(key)
        vs = unpacked[2]  # Versionstamp object
        events.append({
            'version': vs.tr_version.hex() if vs.is_complete() else 'pending',
            'user_version': vs.user_version,
            'data': value
        })

    return events

recent_logins = read_recent_events(db, "login", limit=5)
import com.apple.foundationdb.*;
import com.apple.foundationdb.tuple.*;

public class VersionstampExample {
    public static void main(String[] args) {
        FDB fdb = FDB.selectAPIVersion(740);

        try (Database db = fdb.open()) {
            // Append event with versionstamp
            db.run(tr -> {
                // Create incomplete versionstamp
                Versionstamp vs = Versionstamp.incomplete(0);

                // Pack tuple with versionstamp
                byte[] keyTemplate = Tuple.from("events", "login", vs)
                    .packWithVersionstamp(new byte[0]);

                // Use versionstamped key mutation
                tr.mutate(MutationType.SET_VERSIONSTAMPED_KEY,
                         keyTemplate,
                         "user_id:12345".getBytes());

                return null;
            });

            // Read recent events
            List<String> events = db.read(tr -> {
                Range range = Tuple.from("events", "login").range(new byte[0]);

                List<String> result = new ArrayList<>();
                for (KeyValue kv : tr.getRange(range, 10, true)) {
                    Tuple t = Tuple.fromBytes(kv.getKey());
                    Versionstamp vs = (Versionstamp) t.get(2);
                    result.add("Event at version: " + vs);
                }

                return result;
            });

            events.forEach(System.out::println);
        }
    }
}

Complete Versionstamps

Complete versionstamps contain the actual transaction version from a committed transaction.

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def get_commit_version(tr):
    # Perform some writes
    tr[b'key'] = b'value'

    # Get the commit version after committing
    tr.commit().wait()
    vs_bytes = tr.get_versionstamp().wait()

    # Create complete versionstamp from the 10-byte version
    complete_vs = tuple.Versionstamp(vs_bytes[:10], user_version=0)

    return complete_vs

vs = get_commit_version(db)
print(f"Transaction committed at version: {vs.tr_version.hex()}")

Special Types

UUID Support

Store and order RFC 4122 UUIDs efficiently.

# Python
import uuid

# UUIDs are automatically supported in tuples
my_uuid = uuid.UUID('550e8400-e29b-41d4-a716-446655440000')
key = tuple.pack(("objects", my_uuid, "metadata"))
// Go
type UUID [16]byte

// Create UUID from string
uuid, _ := uuid.Parse("550e8400-e29b-41d4-a716-446655440000")

// Use in tuple
t := tuple.Tuple{"objects", tuple.UUID(uuid), "metadata"}
key := t.Pack()
// Java
import java.util.UUID;

UUID uuid = UUID.fromString("550e8400-e29b-41d4-a716-446655440000");
byte[] key = Tuple.from("objects", uuid, "metadata").pack();
# Ruby
# Note: Ruby binding may have limited UUID support
# Check specific binding documentation

SingleFloat Support

Use single-precision (32-bit) floats instead of doubles for space efficiency.

# Python
class SingleFloat:
    def __init__(self, value: float):
        """Wrap a float to encode as 32-bit instead of 64-bit"""
        ...

# Usage
key = tuple.pack(("measurements", tuple.SingleFloat(3.14159), "temperature"))
// Go
// Single floats are encoded automatically based on Go float32 type
key := tuple.Tuple{"measurements", float32(3.14159), "temperature"}.Pack()
// Java
// Single floats are encoded automatically based on Java Float type
byte[] key = Tuple.from("measurements", Float.valueOf(3.14159f), "temperature").pack();
# Ruby
class FDB::Tuple::SingleFloat
  def initialize(value)
    # Wrap float for 32-bit encoding
  end
end

Supported Data Types

TypePythonGoJavaRubyEncoding SizeTypecode
NullNonenilnullnil1 byte (type only)0x00
Bytesbytes[]bytebyte[]String (binary)Variable + 2 bytes overhead0x01
StringstrstringStringStringVariable + 2 bytes overhead0x02
Nested TupletupleTupleTuple / ListArrayVariable + 2 bytes overhead0x05
Integerintint64, intLong, Integer, BigIntegerInteger, Bignum1-9+ bytes0x0c-0x1c
Float (32-bit)SingleFloatfloat32FloatSingleFloat5 bytes0x20
Float (64-bit)floatfloat64DoubleFloat9 bytes0x21
FalseFalsefalseBoolean (false)false1 byte (type only)0x26
TrueTruetrueBoolean (true)true1 byte (type only)0x27
UUIDuuid.UUIDtuple.UUIDjava.util.UUIDLimited support17 bytes0x30
Versionstamptuple.Versionstamptuple.VersionstampVersionstampVersionstamp13 bytes0x33

Type Ordering

Types are ordered in the following sequence (from lowest to highest):

  1. Null (None, nil, null)
  2. Byte strings
  3. Unicode strings
  4. Nested tuples
  5. Integers (negative to positive)
  6. Floats (negative to positive, with special NaN handling)
  7. False
  8. True
  9. UUID
  10. Versionstamp

Within each type, natural ordering applies:

  • Strings: Lexicographic order (UTF-8 byte order for unicode)
  • Integers: Numeric order
  • Floats: IEEE 754 total ordering (with -0.0 < 0.0)
  • Nested tuples: Lexicographic comparison of elements

Comprehensive Examples

Example 1: Hierarchical Data Model

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

# Build a hierarchical data model for a blog application
# Structure: ("blog", blog_id, "posts", post_id, "comments", comment_id)

@fdb.transactional
def create_blog(tr, blog_id, title, author):
    # Store blog metadata
    tr[tuple.pack(("blogs", blog_id, "title"))] = title
    tr[tuple.pack(("blogs", blog_id, "author"))] = author

@fdb.transactional
def create_post(tr, blog_id, post_id, title, content):
    # Store post under blog
    tr[tuple.pack(("blogs", blog_id, "posts", post_id, "title"))] = title
    tr[tuple.pack(("blogs", blog_id, "posts", post_id, "content"))] = content

@fdb.transactional
def create_comment(tr, blog_id, post_id, comment_id, author, text):
    # Store comment under post
    prefix = ("blogs", blog_id, "posts", post_id, "comments", comment_id)
    tr[tuple.pack(prefix + ("author",))] = author
    tr[tuple.pack(prefix + ("text",))] = text

@fdb.transactional
def get_all_posts(tr, blog_id):
    # Get all posts for a blog
    begin, end = tuple.range(("blogs", blog_id, "posts"))

    posts = {}
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        # parts = ("blogs", blog_id, "posts", post_id, field)
        post_id = parts[3]
        field = parts[4]

        if post_id not in posts:
            posts[post_id] = {}
        posts[post_id][field] = value

    return posts

@fdb.transactional
def get_post_comments(tr, blog_id, post_id):
    # Get all comments for a post
    begin, end = tuple.range(("blogs", blog_id, "posts", post_id, "comments"))

    comments = {}
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        # parts = ("blogs", blog_id, "posts", post_id, "comments", comment_id, field)
        comment_id = parts[5]
        field = parts[6]

        if comment_id not in comments:
            comments[comment_id] = {}
        comments[comment_id][field] = value

    return comments

# Usage
create_blog(db, 1, b"My Tech Blog", b"Alice")
create_post(db, 1, 101, b"First Post", b"Hello world!")
create_post(db, 1, 102, b"Second Post", b"More content here")
create_comment(db, 1, 101, 1001, b"Bob", b"Great post!")
create_comment(db, 1, 101, 1002, b"Charlie", b"Thanks for sharing")

posts = get_all_posts(db, 1)
comments = get_post_comments(db, 1, 101)

print("Posts:", posts)
print("Comments:", comments)

Example 2: Time-Series Data with Versionstamps

import fdb
from fdb import tuple
import json

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def record_metric(tr, service, metric_name, value):
    """Record a metric with automatic timestamp from versionstamp"""
    vs = tuple.Versionstamp()  # Incomplete versionstamp

    # Create key: ("metrics", service, metric_name, versionstamp)
    key_template = tuple.pack_with_versionstamp(
        ("metrics", service, metric_name, vs)
    )

    # Store the metric value
    data = json.dumps({"value": value}).encode()
    tr.set_versionstamped_key(key_template, data)

@fdb.transactional
def get_recent_metrics(tr, service, metric_name, limit=100):
    """Get most recent metrics for a service"""
    begin, end = tuple.range(("metrics", service, metric_name))

    metrics = []
    for key, value in tr.get_range(begin, end, limit=limit, reverse=True):
        parts = tuple.unpack(key)
        vs = parts[3]  # Versionstamp object
        data = json.loads(value)

        metrics.append({
            'timestamp': vs.tr_version.hex(),
            'value': data['value']
        })

    return metrics

@fdb.transactional
def get_metrics_range(tr, service, metric_name, start_version, end_version):
    """Get metrics within a version range"""
    # Create complete versionstamps for range boundaries
    start_vs = tuple.Versionstamp(start_version, 0)
    end_vs = tuple.Versionstamp(end_version, 0)

    begin = tuple.pack(("metrics", service, metric_name, start_vs))
    end = tuple.pack(("metrics", service, metric_name, end_vs))

    metrics = []
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        vs = parts[3]
        data = json.loads(value)

        metrics.append({
            'timestamp': vs.tr_version.hex(),
            'value': data['value']
        })

    return metrics

# Record some metrics
record_metric(db, "web-server", "cpu_usage", 45.2)
record_metric(db, "web-server", "cpu_usage", 48.7)
record_metric(db, "web-server", "cpu_usage", 52.1)
record_metric(db, "web-server", "memory_mb", 2048)
record_metric(db, "web-server", "memory_mb", 2156)

# Retrieve recent CPU metrics
cpu_metrics = get_recent_metrics(db, "web-server", "cpu_usage", limit=10)
print("Recent CPU usage:", cpu_metrics)

Example 3: Complex Nested Structures

import fdb
from fdb import tuple

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def store_user_profile(tr, user_id):
    """Store user profile with nested tuple structures"""

    # Use nested tuples for complex structures
    # Key: ("users", user_id, ("address", "city"))
    tr[tuple.pack(("users", user_id, ("address", "street")))] = b"123 Main St"
    tr[tuple.pack(("users", user_id, ("address", "city")))] = b"San Francisco"
    tr[tuple.pack(("users", user_id, ("address", "zip")))] = b"94102"

    # Store preferences as nested structure
    tr[tuple.pack(("users", user_id, ("prefs", "theme")))] = b"dark"
    tr[tuple.pack(("users", user_id, ("prefs", "language")))] = b"en"

@fdb.transactional
def get_user_section(tr, user_id, section):
    """Get all fields in a user section (e.g., address, prefs)"""
    begin, end = tuple.range(("users", user_id, (section,)))

    data = {}
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        # parts = ("users", user_id, (section, field))
        nested = parts[2]  # This is a tuple like ("address", "city")
        field = nested[1]
        data[field] = value

    return data

store_user_profile(db, 42)
address = get_user_section(db, 42, "address")
prefs = get_user_section(db, 42, "prefs")

print("Address:", address)
print("Preferences:", prefs)

Example 4: Mixed Type Ordering

import fdb
from fdb import tuple
import uuid

fdb.api_version(740)
db = fdb.open()

@fdb.transactional
def demonstrate_ordering(tr):
    """Show how different types are ordered in tuples"""

    # Store keys with different types
    keys = [
        tuple.pack(("demo", None, "null")),
        tuple.pack(("demo", b"bytes", "bytes")),
        tuple.pack(("demo", "string", "string")),
        tuple.pack(("demo", -100, "negative int")),
        tuple.pack(("demo", 0, "zero")),
        tuple.pack(("demo", 100, "positive int")),
        tuple.pack(("demo", -3.14, "negative float")),
        tuple.pack(("demo", 3.14, "positive float")),
        tuple.pack(("demo", False, "false")),
        tuple.pack(("demo", True, "true")),
        tuple.pack(("demo", uuid.UUID('00000000-0000-0000-0000-000000000000'), "uuid")),
        tuple.pack(("demo", (1, 2, 3), "nested tuple")),
    ]

    # Store with descriptive values
    for key in keys:
        parts = tuple.unpack(key)
        tr[key] = str(parts[2]).encode()

    # Read back in order
    begin, end = tuple.range(("demo",))
    print("\nOrdering demonstration:")
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        print(f"  {parts[1]!r:30} -> {value.decode()}")

demonstrate_ordering(db)

# Output shows ordering:
#   None                           -> null
#   b'bytes'                       -> bytes
#   'string'                       -> string
#   (1, 2, 3)                      -> nested tuple
#   -100                           -> negative int
#   0                              -> zero
#   100                            -> positive int
#   -3.14                          -> negative float
#   3.14                           -> positive float
#   False                          -> false
#   True                           -> true
#   UUID('...')                    -> uuid

Example 5: Range Queries with Tuple Prefixes (Go)

package main

import (
    "fmt"
    "github.com/apple/foundationdb/bindings/go/src/fdb"
    "github.com/apple/foundationdb/bindings/go/src/fdb/tuple"
)

func main() {
    fdb.MustAPIVersion(740)
    db := fdb.MustOpenDefault()

    // Populate data for different users and categories
    _, err := db.Transact(func(tr fdb.Transaction) (interface{}, error) {
        // Store: ("inventory", warehouse_id, category, item_id)
        data := []struct {
            warehouse int
            category  string
            itemID    int
            quantity  int
        }{
            {1, "electronics", 101, 50},
            {1, "electronics", 102, 30},
            {1, "furniture", 201, 15},
            {1, "furniture", 202, 8},
            {2, "electronics", 103, 75},
            {2, "electronics", 104, 60},
        }

        for _, d := range data {
            key := tuple.Tuple{"inventory", d.warehouse, d.category, d.itemID}.Pack()
            value := []byte(fmt.Sprintf("%d", d.quantity))
            tr.Set(fdb.Key(key), value)
        }

        return nil, nil
    })

    if err != nil {
        panic(err)
    }

    // Query 1: All items in warehouse 1
    fmt.Println("\nAll items in warehouse 1:")
    queryByPrefix(db, tuple.Tuple{"inventory", 1})

    // Query 2: All electronics in warehouse 1
    fmt.Println("\nElectronics in warehouse 1:")
    queryByPrefix(db, tuple.Tuple{"inventory", 1, "electronics"})

    // Query 3: All electronics across all warehouses
    fmt.Println("\nAll electronics:")
    queryByCategory(db, "electronics")
}

func queryByPrefix(db fdb.Database, prefix tuple.Tuple) {
    _, err := db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
        // Tuple can be used directly as a range
        ri := tr.GetRange(prefix, fdb.RangeOptions{}).Iterator()

        for ri.Advance() {
            kv, err := ri.Get()
            if err != nil {
                return nil, err
            }

            t, err := tuple.Unpack(kv.Key)
            if err != nil {
                return nil, err
            }

            fmt.Printf("  %v -> %s\n", t, kv.Value)
        }

        return nil, nil
    })

    if err != nil {
        panic(err)
    }
}

func queryByCategory(db fdb.Database, category string) {
    _, err := db.ReadTransact(func(tr fdb.ReadTransaction) (interface{}, error) {
        // For non-contiguous prefixes, we need to query each warehouse
        // In practice, you'd structure your keys differently or maintain an index

        // Here we demonstrate querying all warehouses
        for warehouseID := 1; warehouseID <= 2; warehouseID++ {
            prefix := tuple.Tuple{"inventory", warehouseID, category}
            ri := tr.GetRange(prefix, fdb.RangeOptions{}).Iterator()

            for ri.Advance() {
                kv, err := ri.Get()
                if err != nil {
                    return nil, err
                }

                t, err := tuple.Unpack(kv.Key)
                if err != nil {
                    return nil, err
                }

                fmt.Printf("  %v -> %s\n", t, kv.Value)
            }
        }

        return nil, nil
    })

    if err != nil {
        panic(err)
    }
}

Example 6: Tuple Layer with Subspace Integration

import fdb
from fdb import tuple
from fdb.subspace_impl import Subspace

fdb.api_version(740)
db = fdb.open()

# Combine Tuple Layer with Subspace for better organization
app_subspace = Subspace(('myapp',))
users_space = app_subspace.subspace(('users',))
posts_space = app_subspace.subspace(('posts',))

@fdb.transactional
def create_user_with_subspace(tr, user_id, username, email):
    # Using subspace automatically adds prefix
    tr[users_space.pack((user_id, "username"))] = username
    tr[users_space.pack((user_id, "email"))] = email

@fdb.transactional
def get_user_data(tr, user_id):
    # Use subspace range for cleaner code
    begin, end = users_space.range((user_id,))

    data = {}
    for key, value in tr.get_range(begin, end):
        # Unpack relative to subspace
        parts = users_space.unpack(key)
        field = parts[1]
        data[field] = value

    return data

create_user_with_subspace(db, 1, b"alice", b"alice@example.com")
user_data = get_user_data(db, 1)
print("User data:", user_data)

Ordering Semantics in Detail

Byte String vs Unicode String Ordering

Byte strings and unicode strings are different types and ordered separately:

import fdb
from fdb import tuple

# These will sort in this order:
keys = [
    tuple.pack((b"abc",)),      # Byte string (type 0x01)
    tuple.pack((b"xyz",)),      # Byte string (type 0x01)
    tuple.pack(("abc",)),       # Unicode string (type 0x02)
    tuple.pack(("xyz",)),       # Unicode string (type 0x02)
]

# Order: b"abc" < b"xyz" < "abc" < "xyz"
# All byte strings sort before all unicode strings

Integer Encoding and Ordering

Integers use variable-length encoding based on magnitude:

# Integer encodings by size:
# -2^64 to -2^56   : 9 bytes (type 0x0c + 8 bytes one's complement)
# -2^32 to -2^24   : 5 bytes (type 0x10 + 4 bytes one's complement)
# -128 to -1       : 2 bytes (type 0x13 + 1 byte one's complement)
# 0                : 1 byte  (type 0x14)
# 1 to 127         : 2 bytes (type 0x15 + 1 byte)
# 2^24 to 2^32-1   : 5 bytes (type 0x18 + 4 bytes)
# 2^56 to 2^64-1   : 9 bytes (type 0x1c + 8 bytes)

tuple.pack((0,))           # b'\x14'
tuple.pack((1,))           # b'\x15\x01'
tuple.pack((255,))         # b'\x16\x00\xff'
tuple.pack((-1,))          # b'\x13\xfe'

Float Ordering with Special Values

IEEE 754 floats are ordered according to the total ordering specification:

import fdb
from fdb import tuple
import math

# Float ordering:
values = [
    float('-inf'),    # Negative infinity
    -1000.0,          # Negative numbers
    -1.0,
    -0.0,             # Negative zero (sorts before positive zero!)
    0.0,              # Positive zero
    1.0,              # Positive numbers
    1000.0,
    float('inf'),     # Positive infinity
    float('nan'),     # NaN (multiple representations, ordered by bits)
]

for v in values:
    key = tuple.pack((v,))
    print(f"{v:15} -> {key.hex()}")

# Note: -0.0 and 0.0 are distinct in tuple encoding and have different sort orders

Nested Tuple Ordering

Nested tuples are compared element-by-element:

import fdb
from fdb import tuple

# These nested tuples will sort in this order:
keys = [
    tuple.pack((("a",),)),              # Shorter tuple
    tuple.pack((("a", 1),)),            # Longer tuple with same prefix
    tuple.pack((("a", 2),)),            # Same length, different element
    tuple.pack((("b",),)),              # Different first element
]

# Ordering follows lexicographic comparison:
# ("a",) < ("a", 1) < ("a", 2) < ("b",)

Best Practices

1. Design Keys for Range Queries

Structure tuple keys with the most general components first:

  • Good: ("users", user_id, "posts", post_id)
  • Bad: ("posts", post_id, "users", user_id)

The first structure allows you to query all posts for a user efficiently.

2. Use Consistent Types

Keep the same tuple position consistent in type across all keys:

  • Good: Always use integers for IDs at position 1
  • Bad: Mix integers and strings for the same semantic ID

3. Leverage Versionstamps for Monotonic Keys

Use versionstamps instead of timestamps for guaranteed monotonic ordering:

  • Versionstamps are guaranteed unique and monotonic
  • Timestamps can have collisions and clock skew issues

4. Consider Key Size

Tuples add encoding overhead:

  • Strings: 2-3 bytes overhead per string
  • Integers: 1-2 bytes overhead per integer
  • Plan key structures to minimize total size

5. Combine with Subspace Layer

Use the Subspace layer for automatic prefix management:

  • Cleaner code
  • Better namespace isolation
  • Easier to refactor

6. Test Ordering Assumptions

Always verify tuple ordering matches your expectations, especially with:

  • Mixed types
  • Negative numbers
  • Nested tuples
  • Float special values (NaN, infinities, -0.0 vs 0.0)

7. Document Tuple Schemas

Maintain documentation of your tuple key structures:

# Schema documentation
# Users: ("users", user_id: int, field: str)
# Posts: ("users", user_id: int, "posts", post_id: int, field: str)
# Comments: ("users", user_id: int, "posts", post_id: int, "comments", comment_id: int, field: str)

Common Patterns

Pattern 1: Secondary Indexes

@fdb.transactional
def create_user_with_index(tr, user_id, username, email):
    # Primary data
    tr[tuple.pack(("users", user_id, "username"))] = username
    tr[tuple.pack(("users", user_id, "email"))] = email

    # Secondary indexes
    tr[tuple.pack(("users_by_username", username, user_id))] = b''
    tr[tuple.pack(("users_by_email", email, user_id))] = b''

@fdb.transactional
def find_user_by_username(tr, username):
    # Look up in secondary index
    begin, end = tuple.range(("users_by_username", username))

    for key, _ in tr.get_range(begin, end, limit=1):
        parts = tuple.unpack(key)
        user_id = parts[2]
        return user_id

    return None

Pattern 2: Multi-Level Aggregation

@fdb.transactional
def increment_counter(tr, dimensions):
    """Increment counters at multiple aggregation levels"""
    # dimensions = ("country", "state", "city")

    for i in range(len(dimensions) + 1):
        # Increment at each level
        prefix = ("stats",) + dimensions[:i]
        key = tuple.pack(prefix)

        current = tr[key]
        current_val = int(current) if current.present() else 0
        tr[key] = str(current_val + 1).encode()

# Usage:
# increment_counter(db, ("US", "CA", "SF")) increments:
# - ("stats",) - total count
# - ("stats", "US") - US count
# - ("stats", "US", "CA") - California count
# - ("stats", "US", "CA", "SF") - San Francisco count

Pattern 3: Versioned Data

@fdb.transactional
def store_versioned_document(tr, doc_id, version, content):
    """Store multiple versions of a document"""
    key = tuple.pack(("documents", doc_id, "versions", version))
    tr[key] = content

    # Update latest version pointer
    latest_key = tuple.pack(("documents", doc_id, "latest"))
    tr[latest_key] = str(version).encode()

@fdb.transactional
def get_document_history(tr, doc_id):
    """Get all versions of a document"""
    begin, end = tuple.range(("documents", doc_id, "versions"))

    versions = []
    for key, value in tr.get_range(begin, end):
        parts = tuple.unpack(key)
        version = parts[3]
        versions.append((version, value))

    return versions

Performance Considerations

Encoding Cost

  • Tuple encoding/decoding has CPU overhead
  • Cache packed keys when possible for repeated use
  • Consider raw bytes for performance-critical paths

Key Size Impact

  • Larger keys reduce effective database capacity
  • Tuple encoding adds overhead (1-3 bytes per element typically)
  • Balance readability vs efficiency

Range Query Efficiency

  • Well-structured tuple keys enable efficient range queries
  • Poor key design can require full scans
  • Test query patterns with representative data

Resources

  • Official Tuple Layer Documentation: https://apple.github.io/foundationdb/data-modeling.html#tuples
  • Tuple Encoding Specification: https://github.com/apple/foundationdb/blob/main/design/tuple.md
  • Data Modeling Guide: https://apple.github.io/foundationdb/data-modeling.html
  • Python API Reference: python-api.md
  • Go API Reference: go-api.md
  • Java API Reference: java-api.md
  • Ruby API Reference: ruby-api.md

Related Layers