or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

compiler-protogen.mdcore.mdencoding.mdindex.mdreflection.mdruntime.mdtesting.mdtypes-descriptorpb.mdtypes-dynamicpb.mdtypes-known.mdtypes-pluginpb.md
tile.json

index.mddocs/

Protocol Buffers Go

Protocol Buffers Go (protobuf-go) is Google's official Go implementation for Protocol Buffers - a language-neutral, platform-neutral mechanism for serializing structured data. It provides comprehensive support for generating Go code from .proto files and runtime libraries for message manipulation, serialization in multiple formats (wire, JSON, text), reflection capabilities, and type registration.

Package Information

  • Package Name: google.golang.org/protobuf
  • Package Type: golang
  • Language: Go
  • Installation: go get google.golang.org/protobuf@v1.36.10

Core Imports

import (
    "google.golang.org/protobuf/proto"
    "google.golang.org/protobuf/encoding/protojson"
    "google.golang.org/protobuf/encoding/prototext"
    "google.golang.org/protobuf/types/known/timestamppb"
    "google.golang.org/protobuf/reflect/protoreflect"
)

Basic Usage

import (
    "fmt"
    "google.golang.org/protobuf/proto"
    pb "example.com/myproject/mypb" // Your generated protobuf code
)

// Creating a message
msg := &pb.MyMessage{
    Name:  "example",
    Value: 42,
}

// Marshal to wire format
data, err := proto.Marshal(msg)
if err != nil {
    fmt.Println("marshaling error:", err)
}

// Unmarshal from wire format
msg2 := &pb.MyMessage{}
err = proto.Unmarshal(data, msg2)
if err != nil {
    fmt.Println("unmarshaling error:", err)
}

// Clone a message
cloned := proto.Clone(msg).(*pb.MyMessage)

// Check equality
equal := proto.Equal(msg, msg2)

Architecture

The protobuf-go module is organized into several functional areas:

  • Core Operations - Message marshaling, unmarshaling, cloning, merging, and equality checking
  • Encoding Formats - Support for wire format, JSON, text format, and delimited streams
  • Reflection - Dynamic message manipulation, descriptor introspection, and path-based traversal
  • Runtime - Internal interfaces and implementation details for generated code
  • Type System - Descriptor types, dynamic messages, and well-known types (Any, Timestamp, Duration, etc.)
  • Code Generation - Compiler plugin API for generating Go code from .proto files
  • Testing - Protocol buffer comparison utilities and test helpers

Capabilities

Core Protocol Buffer Operations

Essential operations for working with protocol buffer messages including marshaling, unmarshaling, cloning, merging, equality checking, and field manipulation. Supports both wire format and type-safe operations with full extension field support.

// Message is the top-level interface for all protocol buffer messages
type Message = protoreflect.ProtoMessage

// Marshal returns the wire-format encoding of m
func Marshal(m Message) ([]byte, error)

// Unmarshal parses the wire-format message in b and places the result in m
func Unmarshal(b []byte, m Message) error

// Clone returns a deep copy of m
func Clone(m Message) Message

// Equal reports whether two messages are equal
func Equal(x, y Message) bool

// Merge merges src into dst
func Merge(dst, src Message)

// Size returns the size in bytes of the wire-format encoding
func Size(m Message) int

// Reset clears every field in the message
func Reset(m Message)

// CheckInitialized returns an error if any required fields are not set
func CheckInitialized(m Message) error

Core Protocol Buffer Operations

JSON and Text Encoding

Convert protocol buffer messages to and from JSON and text formats with customizable formatting options. JSON encoding follows the official protobuf JSON mapping specification, while text format provides a human-readable representation.

// JSON Marshal
func protojson.Marshal(m proto.Message) ([]byte, error)

// JSON Unmarshal
func protojson.Unmarshal(b []byte, m proto.Message) error

// Text Marshal
func prototext.Marshal(m proto.Message) ([]byte, error)

// Text Unmarshal
func prototext.Unmarshal(b []byte, m proto.Message) error

JSON and Text Encoding

Wire Format Encoding

Low-level wire format encoding and decoding for protocol buffer messages. Provides functions to parse and construct wire-format data, including varint encoding, fixed-width integers, length-prefixed data, and groups.

// AppendVarint appends v to b as a varint-encoded uint64
func protowire.AppendVarint(b []byte, v uint64) []byte

// ConsumeVarint parses b as a varint-encoded uint64
func protowire.ConsumeVarint(b []byte) (v uint64, n int)

// AppendTag encodes num and typ as a varint-encoded tag and appends it to b
func protowire.AppendTag(b []byte, num Number, typ Type) []byte

// ConsumeTag parses b as a varint-encoded tag
func protowire.ConsumeTag(b []byte) (Number, Type, int)

JSON and Text Encoding

Delimited Message Encoding

Marshal and unmarshal varint size-delimited messages for streaming protocol buffer messages. Each message is prefixed with its size as a varint, enabling efficient reading and writing of multiple messages in sequence.

// MarshalTo writes a varint size-delimited wire-format message to w
func protodelim.MarshalTo(w io.Writer, m proto.Message) (int, error)

// UnmarshalFrom parses and consumes a varint size-delimited wire-format message from r
func protodelim.UnmarshalFrom(r Reader, m proto.Message) error

JSON and Text Encoding

Message Reflection

Dynamically introspect and manipulate protocol buffer messages at runtime. The reflection API provides access to message descriptors, field values, and type information without compile-time knowledge of message structures.

// Message is the interface for a protobuf message
type protoreflect.Message interface {
    Descriptor() MessageDescriptor
    Type() MessageType
    New() Message
    Interface() ProtoMessage
    Range(func(FieldDescriptor, Value) bool)
    Has(FieldDescriptor) bool
    Get(FieldDescriptor) Value
    Set(FieldDescriptor, Value)
    Clear(FieldDescriptor)
    // ... additional methods
}

// MessageDescriptor describes a message type
type protoreflect.MessageDescriptor interface {
    Descriptor
    Fields() FieldDescriptors
    Oneofs() OneofDescriptors
    Extensions() ExtensionDescriptors
    IsMapEntry() bool
}

Message Reflection

Descriptor Construction and Conversion

Create file descriptors from FileDescriptorProto messages and convert between descriptor representations. Enables building structured descriptor hierarchies from serialized proto definitions.

// NewFile creates a new FileDescriptor from a FileDescriptorProto
func protodesc.NewFile(fd *descriptorpb.FileDescriptorProto, r Resolver) (protoreflect.FileDescriptor, error)

// NewFiles creates a new Files registry from a FileDescriptorSet
func protodesc.NewFiles(fd *descriptorpb.FileDescriptorSet) (*protoregistry.Files, error)

// ToFileDescriptorProto converts a FileDescriptor to a FileDescriptorProto
func protodesc.ToFileDescriptorProto(file protoreflect.FileDescriptor) *descriptorpb.FileDescriptorProto

Message Reflection

Path-Based Message Traversal

Represent and manipulate paths through message structures, enabling precise navigation to nested fields, list elements, and map entries. Supports structured access patterns for dynamic message manipulation.

// Path is a sequence of protobuf reflection steps
type protopath.Path []Step

// Range performs a depth-first traversal over reachable values in a message
func protorange.Range(m protoreflect.Message, f func(protopath.Values) error) error

Message Reflection

Type Registry

Register and look up message types, extension types, and enum types at runtime. The global registry enables resolution of types by name or number, supporting dynamic message creation and extension field handling.

// GlobalTypes is a global registry of types
var protoregistry.GlobalTypes *Types

// Files is a registry for looking up file descriptors
type protoregistry.Files struct { /* ... */ }

// Types is a registry for looking up types
type protoregistry.Types struct { /* ... */ }

Message Reflection

Runtime Interfaces

Low-level interfaces and implementation details used by generated protocol buffer code. These interfaces define the contract between generated code and the runtime library.

// MessageV1 is the v1 proto.Message interface
type protoiface.MessageV1 interface {
    Reset()
    String() string
    ProtoMessage()
}

// Methods provides access to optional fast-path implementations
type protoiface.Methods struct {
    Flags          SupportFlags
    Size           func(MarshalInput) MarshalOutput
    Marshal        func(MarshalInput) (MarshalOutput, error)
    Unmarshal      func(UnmarshalInput) (UnmarshalOutput, error)
    Merge          func(MergeInput) MergeOutput
    CheckInitialized func(CheckInitializedInput) (CheckInitializedOutput, error)
}

Runtime Interfaces

Generated Code Support

Helper functions and types used by generated protocol buffer code for message implementation, enum support, descriptor management, and extension handling.

// EnforceVersion ensures compatibility between generated code and runtime
func protoimpl.EnforceVersion(gen, min, max int)

// X contains exported functionality for generated code
type protoimpl.X struct{}

// MessageInfo provides infrastructure for generated message types
type protoimpl.MessageInfo struct { /* ... */ }

// ExtensionInfo provides infrastructure for extension fields
type protoimpl.ExtensionInfo struct { /* ... */ }

Runtime Interfaces

Descriptor Types

Complete set of generated types representing protocol buffer descriptors as defined in descriptor.proto. These types describe the structure of .proto files, messages, fields, enums, services, and more.

// FileDescriptorProto describes a complete .proto file
type descriptorpb.FileDescriptorProto struct { /* ... */ }

// DescriptorProto describes a message type
type descriptorpb.DescriptorProto struct { /* ... */ }

// FieldDescriptorProto describes a message field
type descriptorpb.FieldDescriptorProto struct { /* ... */ }

// EnumDescriptorProto describes an enum type
type descriptorpb.EnumDescriptorProto struct { /* ... */ }

// ServiceDescriptorProto describes a service
type descriptorpb.ServiceDescriptorProto struct { /* ... */ }

Descriptor Types

Dynamic Messages

Create and manipulate protocol buffer messages at runtime without generated code. Dynamic messages work with descriptors to provide full message functionality for types only known at runtime.

// NewMessage creates a new dynamic message
func dynamicpb.NewMessage(md protoreflect.MessageDescriptor) *Message

// NewMessageType creates a new dynamic message type
func dynamicpb.NewMessageType(md protoreflect.MessageDescriptor) protoreflect.MessageType

// NewExtensionType creates a new dynamic extension type
func dynamicpb.NewExtensionType(xd protoreflect.ExtensionDescriptor) protoreflect.ExtensionType

// NewEnumType creates a new dynamic enum type
func dynamicpb.NewEnumType(ed protoreflect.EnumDescriptor) protoreflect.EnumType

Dynamic Messages

Well-Known Types

Google's well-known protocol buffer types including Any, Timestamp, Duration, Struct, and wrapper types. These types provide standardized representations for common data structures.

// Any can hold any message type
type anypb.Any struct { /* ... */ }

// Timestamp represents a point in time
type timestamppb.Timestamp struct { /* ... */ }

// Duration represents a signed span of time
type durationpb.Duration struct { /* ... */ }

// Struct represents a JSON object
type structpb.Struct struct { /* ... */ }

// Value represents a JSON value
type structpb.Value struct { /* ... */ }

Well-Known Types

Plugin Types

Types for protocol buffer compiler plugins, enabling development of custom code generators that integrate with protoc.

// CodeGeneratorRequest is the input to a code generator plugin
type pluginpb.CodeGeneratorRequest struct { /* ... */ }

// CodeGeneratorResponse is the output from a code generator plugin
type pluginpb.CodeGeneratorResponse struct { /* ... */ }

Plugin Types

Code Generation

High-level API for writing protoc plugins that generate Go code from .proto files. Provides structured access to file descriptors, message types, and code generation utilities.

// Options configures a code generation request
type protogen.Options struct { /* ... */ }

// Run executes a code generation function
func (opts Options) Run(f func(*Plugin) error) error

// Plugin provides information about the code generation request
type protogen.Plugin struct { /* ... */ }

// GeneratedFile represents a generated Go source file
type protogen.GeneratedFile struct { /* ... */ }

Code Generation with Protogen

Protocol Buffer Comparison

Compare protocol buffer messages in tests with detailed diff reporting. Integrates with the go-cmp package to provide human-readable comparisons.

// Transform returns a cmp.Option for comparing protocol buffer messages
func protocmp.Transform() cmp.Option

// IgnoreFields returns an option that ignores specified fields
func protocmp.IgnoreFields(m proto.Message, fieldNames ...string) cmp.Option

// IgnoreEmptyMessages returns an option that ignores empty messages
func protocmp.IgnoreEmptyMessages() cmp.Option

// SortRepeated returns an option that sorts repeated fields
func protocmp.SortRepeated(lessFunc interface{}) cmp.Option

Testing Utilities

Wire Format Construction for Testing

Manually construct and decode wire-format protocol buffer data for testing purposes. Enables creation of test cases with precise control over wire encoding.

// Message represents a protocol buffer message
type protopack.Message []protopack.Token

// Marshal converts a Message to wire format
func (m Message) Marshal() []byte

// Unmarshal converts wire format to a Message
func Unmarshal(b []byte) (Message, error)

// Tag creates a field tag token
func Tag(num protowire.Number, typ protowire.Type) Token

// Varint creates a varint token
func Varint(v uint64) Token

Testing Utilities

Test Validation Utilities

Assert message equality and perform validation checks in tests. Provides helpers for common test scenarios with clear error messages.

// Assert returns an error if two messages are not equal
func prototest.Assert(x, y proto.Message) error

// AssertMessage returns an error if the message does not match expected state
func prototest.AssertMessage(m proto.Message, want string) error

Testing Utilities

Version Compatibility

This documentation covers protobuf-go version 1.36.10. The module follows semantic versioning and maintains backward compatibility within major versions. Generated code requires runtime library version compatibility enforced through build-time checks.

Additional Resources