tessl/pypi-proto-plus

Beautiful, Pythonic protocol buffers that make protocol buffer message classes behave like native Python types

—

Pending

Overview

Eval results

Files

Module System

Name: tessl/pypi-proto-plus
Author: tessl

The Module system provides organization and management utilities for protocol buffer definitions across multiple files and packages. It enables structured organization of proto-plus messages and enums within logical modules while managing protocol buffer file generation and package hierarchies.

Capabilities

Module Definition

Define protocol buffer modules with specific package organization and marshal management.

def define_module(*, package: str, marshal: str = None, 
                  manifest: Set[str] = frozenset()) -> _ProtoModule:
    """
    Define a protocol buffers module.
    
    The settings defined here are used for all protobuf messages
    declared in the module of the given name.
    
    Args:
        package (str): The proto package name.
        marshal (str): The name of the marshal to use. It is recommended
            to use one marshal per Python library (e.g. package on PyPI).
            If not provided, defaults to the package name.
        manifest (Set[str]): A set of messages and enums to be created. Setting
            this adds a slight efficiency in piecing together proto
            descriptors under the hood.
            
    Returns:
        _ProtoModule: A named tuple containing package, marshal, and manifest settings.
    """

The function is exported as module in the main proto package:

import proto

# Access the function as proto.module
my_module = proto.module(package="com.example.api", marshal="example_api")

Example usage:

import proto

# Define a module for an API package
api_module = proto.module(
    package="com.example.api",
    marshal="example_api",
    manifest={"User", "Order", "Product", "OrderStatus"}
)

# Define a module for internal services
internal_module = proto.module(
    package="com.example.internal", 
    marshal="internal_services"
)

# Simple module definition (marshal defaults to package name)
simple_module = proto.module(package="com.example.simple")

Module Integration with Messages

Use defined modules in message and enum definitions by setting the module attribute:

import proto

# Define the module
api_module = proto.module(
    package="com.example.api",
    marshal="api_marshal"
)

# Use the module in message definitions
class User(proto.Message):
    __module__ = api_module
    
    user_id = proto.Field(proto.STRING, number=1)
    email = proto.Field(proto.STRING, number=2)
    created_at = proto.Field(proto.INT64, number=3)

class Order(proto.Message):
    __module__ = api_module
    
    order_id = proto.Field(proto.STRING, number=1)
    user = proto.Field(User, number=2)
    total = proto.Field(proto.FLOAT, number=3)

class OrderStatus(proto.Enum):
    __module__ = api_module
    
    PENDING = 0
    CONFIRMED = 1
    SHIPPED = 2
    DELIVERED = 3

Package Organization

Organize messages into logical package hierarchies:

import proto

# Core API types
core_module = proto.module(package="com.example.core")

class BaseEntity(proto.Message):
    __module__ = core_module
    
    id = proto.Field(proto.STRING, number=1)
    created_at = proto.Field(proto.INT64, number=2)
    updated_at = proto.Field(proto.INT64, number=3)

# User management API
user_module = proto.module(package="com.example.user")

class User(proto.Message):
    __module__ = user_module
    
    base = proto.Field(BaseEntity, number=1)
    username = proto.Field(proto.STRING, number=2)
    email = proto.Field(proto.STRING, number=3)

# Order management API  
order_module = proto.module(package="com.example.order")

class Order(proto.Message):
    __module__ = order_module
    
    base = proto.Field(BaseEntity, number=1)
    user_id = proto.Field(proto.STRING, number=2)
    items = proto.RepeatedField(OrderItem, number=3)

Marshal Management

Control marshal instances across modules for shared type conversion:

import proto

# Shared marshal for the entire application
app_marshal = "my_application"

# All modules use the same marshal
user_module = proto.module(
    package="com.myapp.user",
    marshal=app_marshal
)

order_module = proto.module(
    package="com.myapp.order", 
    marshal=app_marshal
)

inventory_module = proto.module(
    package="com.myapp.inventory",
    marshal=app_marshal
)

# Messages in different modules share type conversion rules
class User(proto.Message):
    __module__ = user_module
    # ... fields ...

class Order(proto.Message):
    __module__ = order_module  
    user = proto.Field(User, number=1)  # Cross-module reference
    # ... other fields ...

Manifest Optimization

Use manifests to optimize protocol buffer descriptor generation:

import proto

# Define module with manifest for better performance
api_module = proto.module(
    package="com.example.api",
    marshal="api",
    manifest={
        # Messages
        "User", "Order", "Product", "Category",
        "Address", "PaymentMethod",
        
        # Enums  
        "UserStatus", "OrderStatus", "ProductType",
        "PaymentType"
    }
)

# Define the messages and enums listed in manifest
class User(proto.Message):
    __module__ = api_module
    # ... fields ...

class UserStatus(proto.Enum):
    __module__ = api_module
    # ... values ...

# All other messages and enums follow...

Cross-Module References

Reference messages and enums across different modules:

import proto

# Define modules
common_module = proto.module(package="com.example.common")
user_module = proto.module(package="com.example.user")  
order_module = proto.module(package="com.example.order")

# Common types
class Address(proto.Message):
    __module__ = common_module
    
    street = proto.Field(proto.STRING, number=1)
    city = proto.Field(proto.STRING, number=2)
    postal_code = proto.Field(proto.STRING, number=3)

class Currency(proto.Enum):
    __module__ = common_module
    
    USD = 1
    EUR = 2
    GBP = 3

# User module references common types
class User(proto.Message):
    __module__ = user_module
    
    name = proto.Field(proto.STRING, number=1)
    billing_address = proto.Field(Address, number=2)  # Cross-module reference

# Order module references both common and user types  
class Order(proto.Message):
    __module__ = order_module
    
    order_id = proto.Field(proto.STRING, number=1)
    user = proto.Field(User, number=2)              # Cross-module reference
    currency = proto.Field(Currency, number=3)      # Cross-module reference
    shipping_address = proto.Field(Address, number=4)  # Cross-module reference

Module System Features

Automatic File Organization

The module system automatically organizes protocol buffer definitions into appropriate file structures:

# Messages with the same module are grouped together
# in the generated protocol buffer files

core_module = proto.module(package="com.example.core")

# These messages will be in the same proto file
class Entity(proto.Message):
    __module__ = core_module
    # ...

class Metadata(proto.Message):  
    __module__ = core_module
    # ...

class Status(proto.Enum):
    __module__ = core_module
    # ...

Package Namespacing

Modules provide proper protocol buffer package namespacing:

# Different modules create separate protocol buffer packages
auth_module = proto.module(package="com.myapp.auth")
data_module = proto.module(package="com.myapp.data")

# These create different protobuf packages:
# com.myapp.auth.User vs com.myapp.data.User
class User(proto.Message):  # auth user
    __module__ = auth_module
    username = proto.Field(proto.STRING, number=1)

class User(proto.Message):  # data user  
    __module__ = data_module
    profile_data = proto.Field(proto.BYTES, number=1)

Import Management

The module system handles protocol buffer imports automatically:

base_module = proto.module(package="com.example.base")
extended_module = proto.module(package="com.example.extended")

class BaseMessage(proto.Message):
    __module__ = base_module
    id = proto.Field(proto.STRING, number=1)

class ExtendedMessage(proto.Message):
    __module__ = extended_module
    base = proto.Field(BaseMessage, number=1)  # Automatic import handling
    extra_data = proto.Field(proto.STRING, number=2)

Module Information Access

Access module configuration information:

import proto

# Create module
my_module = proto.module(
    package="com.example.test",
    marshal="test_marshal", 
    manifest={"TestMessage"}
)

# Access module properties
print(my_module.package)   # "com.example.test"
print(my_module.marshal)   # "test_marshal"  
print(my_module.manifest)  # frozenset({"TestMessage"})

Install with Tessl CLI