or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

events.mdindex.mdinstrumentation.mdmetrics.mdutilities.md
tile.json

tessl/pypi-opentelemetry-instrumentation-bedrock

OpenTelemetry instrumentation for AWS Bedrock runtime services providing automatic tracing, metrics, and event emission for AI model invocations

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/opentelemetry-instrumentation-bedrock@0.46.x

To install, run

npx @tessl/cli install tessl/pypi-opentelemetry-instrumentation-bedrock@0.46.0

index.mddocs/

OpenTelemetry Bedrock Instrumentation

OpenTelemetry instrumentation for AWS Bedrock runtime services providing automatic tracing, metrics, and event emission for AI model invocations. This package automatically instruments boto3 clients to capture comprehensive telemetry data for Amazon Bedrock's generative AI services including prompts, completions, model parameters, performance metrics, guardrail interactions, and prompt caching.

Package Information

  • Package Name: opentelemetry-instrumentation-bedrock
  • Package Type: PyPI
  • Language: Python (>=3.9,<4)
  • Installation: pip install opentelemetry-instrumentation-bedrock
  • Dependencies: boto3 >=1.28.57, opentelemetry-api ^1.28.0, anthropic >=0.17.0

Core Imports

from opentelemetry.instrumentation.bedrock import BedrockInstrumentor

Common imports for configuration and types:

from opentelemetry.instrumentation.bedrock.config import Config
from opentelemetry.instrumentation.bedrock.event_models import MessageEvent, ChoiceEvent, Roles
from opentelemetry.instrumentation.bedrock.utils import should_send_prompts, should_emit_events
from opentelemetry.instrumentation.bedrock.guardrail import is_guardrail_activated, GuardrailAttributes, Type
from opentelemetry.instrumentation.bedrock.prompt_caching import CachingHeaders, CacheSpanAttrs

Basic Usage

import boto3
from opentelemetry.instrumentation.bedrock import BedrockInstrumentor

# Basic instrumentation - automatically instruments all bedrock-runtime clients
BedrockInstrumentor().instrument()

# Create a Bedrock client (will be automatically instrumented)
bedrock_client = boto3.client(
    service_name='bedrock-runtime',
    region_name='us-east-1'
)

# Use the client normally - instrumentation happens automatically
response = bedrock_client.invoke_model(
    modelId='anthropic.claude-3-sonnet-20240229-v1:0',
    body=json.dumps({
        "messages": [{"role": "user", "content": "Hello, world!"}],
        "max_tokens": 100,
        "anthropic_version": "bedrock-2023-05-31"
    })
)

# Advanced configuration with custom options
BedrockInstrumentor(
    enrich_token_usage=True,
    use_legacy_attributes=False,
    exception_logger=my_exception_handler
).instrument()

Architecture

The instrumentation follows OpenTelemetry's auto-instrumentation pattern:

  • Automatic Wrapping: Intercepts boto3 client creation for bedrock-runtime services
  • Span Creation: Creates spans for invoke_model, invoke_model_with_response_stream, converse, and converse_stream operations
  • Metrics Collection: Tracks token usage, latency, errors, and guardrail interactions
  • Event Emission: Emits structured events for input/output messages (when semantic conventions are enabled)
  • Multi-Model Support: Handles all Bedrock-supported AI model families (Anthropic, Cohere, AI21, Meta, Amazon, Nova, imported models)

Capabilities

Core Instrumentation

Primary instrumentor class for enabling automatic tracing of Bedrock API calls with configuration options for token enrichment, legacy attributes, and exception handling.

class BedrockInstrumentor(BaseInstrumentor):
    def __init__(
        self,
        enrich_token_usage: bool = False,
        exception_logger = None,
        use_legacy_attributes: bool = True
    ): ...
    
    def instrument(self, **kwargs): ...
    def uninstrument(self, **kwargs): ...

Core Instrumentation

Event System

Comprehensive event models and emission functions for capturing AI model interactions as structured OpenTelemetry events, supporting both input messages and completion responses.

@dataclass
class MessageEvent:
    content: Any
    role: str = "user"
    tool_calls: List[ToolCall] | None = None

@dataclass  
class ChoiceEvent:
    index: int
    message: CompletionMessage
    finish_reason: str = "unknown"
    tool_calls: List[ToolCall] | None = None

def emit_message_events(event_logger, kwargs): ...
def emit_choice_events(event_logger, response): ...

Event System

Metrics and Monitoring

Advanced metrics collection including token usage, request duration, error tracking, guardrail interactions, and prompt caching with support for all major Bedrock model families.

class MetricParams:
    def __init__(
        self,
        token_histogram: Histogram,
        choice_counter: Counter,
        duration_histogram: Histogram,
        exception_counter: Counter,
        guardrail_activation: Counter,
        guardrail_latency_histogram: Histogram,
        # ... additional guardrail and caching metrics
    ): ...

def is_metrics_enabled() -> bool: ...

class GuardrailMeters:
    LLM_BEDROCK_GUARDRAIL_ACTIVATION = "gen_ai.bedrock.guardrail.activation"
    LLM_BEDROCK_GUARDRAIL_LATENCY = "gen_ai.bedrock.guardrail.latency"
    # ... additional constants

Metrics and Monitoring

Utilities and Streaming

Utility functions, streaming response handling, and helper classes for managing configuration, error handling, and reusable streaming bodies.

def dont_throw(func): ...
def should_send_prompts() -> bool: ...  
def should_emit_events() -> bool: ...

class StreamingWrapper(ObjectProxy):
    def __init__(self, response, stream_done_callback): ...
    def __iter__(self): ...

class ReusableStreamingBody(StreamingBody):
    def __init__(self, raw_stream, content_length): ...
    def read(self, amt=None): ...

Utilities and Streaming

Configuration

Environment Variables

Control instrumentation behavior through environment variables:

  • TRACELOOP_METRICS_ENABLED: Enable/disable metrics collection (default: "true")
  • TRACELOOP_TRACE_CONTENT: Enable/disable content tracing (default: "true")

Global Configuration

class Config:
    enrich_token_usage: bool
    exception_logger: Any
    use_legacy_attributes: bool

Type Definitions

Guardrail Types

class GuardrailAttributes:
    """Constants for guardrail span attributes in OpenTelemetry instrumentation."""
    GUARDRAIL = "gen_ai.guardrail"
    TYPE = "gen_ai.guardrail.type"
    PII = "gen_ai.guardrail.pii"
    PATTERN = "gen_ai.guardrail.pattern"
    TOPIC = "gen_ai.guardrail.topic"
    CONTENT = "gen_ai.guardrail.content"
    CONFIDENCE = "gen_ai.guardrail.confidence"
    MATCH = "gen_ai.guardrail.match"

from enum import Enum

class Type(Enum):
    """Enum for guardrail assessment types."""
    INPUT = "input"
    OUTPUT = "output"

Prompt Caching Types

class CachingHeaders:
    """HTTP headers for Bedrock prompt caching functionality."""
    READ = "x-amzn-bedrock-cache-read-input-token-count"
    WRITE = "x-amzn-bedrock-cache-write-input-token-count"

class CacheSpanAttrs:
    """Span attributes for prompt caching operations."""
    TYPE = "gen_ai.cache.type"
    CACHED = "gen_ai.prompt_caching"

Event Role Types

from enum import Enum

class Roles(Enum):
    """Valid roles for message events in conversation."""
    USER = "user"
    ASSISTANT = "assistant"
    SYSTEM = "system"
    TOOL = "tool"

Supported Models

The instrumentation supports all AI model vendors available through Amazon Bedrock:

  • Anthropic: Claude models (claude-3-sonnet, claude-3-haiku, claude-3-opus, claude-instant)
  • Cohere: Command models (command, command-light, command-r, command-r-plus)
  • AI21: Jurassic models (j2-mid, j2-ultra)
  • Meta: Llama models (llama2-7b, llama2-13b, llama2-70b, llama3-8b, llama3-70b)
  • Amazon: Titan models (titan-text-express, titan-text-lite, titan-embed)
  • Nova: Amazon's foundational models (nova-micro, nova-lite, nova-pro)
  • Imported Models: Custom and third-party models deployed through Bedrock

Key Features

  1. Zero-Code Instrumentation: Automatic instrumentation of all Bedrock API calls
  2. Comprehensive Telemetry: Spans, metrics, and events for complete observability
  3. Multi-API Support: Handles both legacy invoke_model and modern converse APIs
  4. Streaming Support: Full instrumentation for streaming responses
  5. Guardrail Integration: Detailed tracking of Bedrock Guardrails interactions
  6. Prompt Caching: Metrics and attributes for prompt caching features
  7. Token Usage Tracking: Detailed token consumption metrics with model-specific counting
  8. Error Handling: Comprehensive exception tracking and custom error handling
  9. Privacy Controls: Configurable content tracing for sensitive data management
  10. Production Ready: Designed for high-volume production AI applications