CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-tavern

Simple testing of RESTful APIs

Overview
Eval results
Files

exceptions.mddocs/

Exception Handling

Comprehensive exception hierarchy for detailed error reporting and debugging, covering schema validation, key mismatches, protocol errors, and configuration issues.

Capabilities

Base Exception Class

Root exception class for all Tavern-specific errors with contextual information about test execution.

class TavernException(Exception):
    """
    Base exception class for all Tavern errors.
    
    Attributes:
    - stage: Optional dictionary containing current test stage information
    - test_block_config: Optional test configuration object
    - is_final: Boolean indicating if this is a final error (default: False)
    """
    
    stage: Optional[dict]
    test_block_config: Optional["TestConfig"]
    is_final: bool = False
    
    def __init__(self, message: str, **kwargs):
        super().__init__(message)
        self.stage = kwargs.get('stage')
        self.test_block_config = kwargs.get('test_block_config')
        self.is_final = kwargs.get('is_final', False)

Validation Exceptions

Schema Validation Errors

Exceptions related to schema validation and test specification format issues.

class BadSchemaError(TavernException):
    """
    Raised when test schema is invalid or malformed.
    
    Common causes:
    - Invalid YAML structure
    - Missing required fields
    - Incorrect data types in test specification
    - Malformed validation functions
    """

class TestFailError(TavernException):
    """
    Raised when a test fails validation or execution.
    
    Common causes:
    - Response validation failures
    - Assertion errors
    - Test execution timeouts
    - Protocol-specific failures
    """

Key Matching Errors

Exceptions for response validation and key matching issues.

class KeyMismatchError(TavernException):
    """
    Raised when response keys don't match expected values.
    
    Common causes:
    - JSON response structure differs from expected
    - Missing keys in response
    - Type mismatches in response values
    """

class UnexpectedKeysError(TavernException):
    """
    Raised when request specification contains unexpected keys.
    
    Common causes:
    - Typos in request specification
    - Using unsupported request parameters
    - Plugin-specific parameter errors
    """

class MissingKeysError(TavernException):
    """
    Raised when required keys are missing from request specification.
    
    Common causes:
    - Incomplete request specifications
    - Missing required parameters for protocol
    - Configuration errors
    """

Template and Variable Errors

Exceptions related to variable templating and string formatting.

class MissingFormatError(TavernException):
    """
    Raised when template variable is not found during string formatting.
    
    Common causes:
    - Reference to undefined variable in template
    - Typos in variable names
    - Variable not saved from previous stage
    - Missing global configuration variables
    """

Protocol-Specific Exceptions

HTTP/REST Exceptions

Exceptions specific to HTTP/REST API testing.

class RestRequestException(TavernException):
    """
    Raised when HTTP request execution fails.
    
    Common causes:
    - Network connectivity issues
    - Invalid URLs
    - HTTP client configuration errors
    - Request timeout
    - SSL/TLS certificate issues
    """

MQTT Exceptions

Exceptions specific to MQTT protocol testing.

class MQTTError(TavernException):
    """
    Raised when MQTT operations fail.
    
    Common causes:
    - MQTT broker connection failures
    - Subscription timeout
    - Message publication errors
    - Topic permission issues
    - QoS level conflicts
    """

gRPC Exceptions

Exceptions specific to gRPC protocol testing.

class GRPCRequestException(TavernException):
    """
    Raised when gRPC request execution fails.
    
    Common causes:
    - gRPC service unavailable
    - Method not found
    - Protocol buffer serialization errors
    - Authentication failures
    - Deadline exceeded
    """

System and Configuration Exceptions

Plugin and Loading Errors

Exceptions related to plugin loading and system configuration.

class PluginLoadError(TavernException):
    """
    Raised when plugin loading fails.
    
    Common causes:
    - Plugin not found
    - Plugin import errors
    - Incompatible plugin version
    - Missing plugin dependencies
    - Entry point configuration errors
    """

class InvalidSettingsError(TavernException):
    """
    Raised when configuration is invalid or incorrectly formatted.
    
    Common causes:
    - Invalid global configuration file
    - Malformed YAML configuration
    - Missing configuration files
    - Type errors in configuration values
    - File permission issues
    """

Validation Function Errors

Exceptions from response validation functions.

class JMESError(TavernException):
    """
    Raised when JMESPath query execution fails.
    
    Common causes:
    - Invalid JMESPath syntax
    - JMESPath query returns None
    - Type errors in query result
    - Expected value mismatch
    """

class RegexAccessError(TavernException):
    """
    Raised when regex validation fails.
    
    Common causes:
    - Regex pattern doesn't match content
    - Invalid regex expression
    - JMESPath extraction failure before regex
    - Content type mismatch
    """

class UnexpectedExceptionError(TavernException):
    """
    Raised when exception validation fails.
    
    Common causes:
    - Response doesn't match expected exception format
    - Status code mismatch
    - Exception description doesn't match
    - Exception title/error field mismatch
    """

Exception Usage Examples

Basic Exception Handling

from tavern.core import run
from tavern._core.exceptions import (
    TavernException,
    TestFailError,
    BadSchemaError,
    InvalidSettingsError
)

try:
    exit_code = run("test_api.tavern.yaml")
except InvalidSettingsError as e:
    print(f"Configuration error: {e}")
    print(f"Stage: {e.stage}")
except BadSchemaError as e:
    print(f"Schema validation error: {e}")
except TestFailError as e:
    print(f"Test execution failed: {e}")
except TavernException as e:
    print(f"Tavern error: {e}")
    if e.test_block_config:
        print(f"Test config: {e.test_block_config}")

Validation Function Error Handling

from tavern.helpers import validate_jwt, validate_regex
from tavern._core.exceptions import JMESError, RegexAccessError

try:
    # JWT validation
    jwt_data = validate_jwt(response, "access_token", verify_signature=False)
except Exception as e:
    print(f"JWT validation failed: {e}")

try:
    # Regex validation
    regex_result = validate_regex(response, r"ID: (\d+)")
except RegexAccessError as e:
    print(f"Regex matching failed: {e}")

try:
    # JMESPath validation
    from tavern.helpers import check_jmespath_match
    result = check_jmespath_match(data, "users[0].id")
except JMESError as e:
    print(f"JMESPath query failed: {e}")

Plugin Development Error Handling

from tavern.request import BaseRequest
from tavern._core.exceptions import TavernException

class CustomRequest(BaseRequest):
    def run(self):
        try:
            return self.session.execute_request(self.rspec)
        except ConnectionError as e:
            raise TavernException(
                f"Custom protocol connection failed: {e}",
                stage=self.rspec,
                test_block_config=self.test_block_config
            )
        except TimeoutError as e:
            raise TavernException(
                f"Custom protocol timeout: {e}",
                stage=self.rspec,
                test_block_config=self.test_block_config,
                is_final=True  # Mark as final error
            )

Exception Context in YAML Tests

# Validation functions can raise exceptions that provide context
test_name: Error handling example

stages:
  - name: Test with potential validation errors
    request:
      url: https://api.example.com/data
      method: GET
    response:
      status_code: 200
      validate:
        # This might raise JMESError if path not found
        - function: tavern.helpers:check_jmespath_match
          extra_kwargs:
            query: "data.nonexistent_field"
            expected: "some_value"
        
        # This might raise RegexAccessError if pattern doesn't match
        - function: tavern.helpers:validate_regex
          extra_kwargs:
            expression: "Order #([0-9]+)"
            header: "X-Order-ID"

Custom Exception Classes

# Custom exceptions for specific use cases
class APIRateLimitError(TavernException):
    """Raised when API rate limit is exceeded."""
    pass

class AuthenticationError(TavernException):
    """Raised when authentication fails."""
    pass

class DataValidationError(TavernException):
    """Raised when response data validation fails."""
    pass

# Usage in custom validation functions
def validate_api_response(response, expected_data):
    if response.status_code == 429:
        raise APIRateLimitError("API rate limit exceeded")
    
    if response.status_code == 401:
        raise AuthenticationError("Authentication failed")
    
    if not validate_response_data(response.json(), expected_data):
        raise DataValidationError(
            "Response data validation failed",
            stage={'response': response.json()},
            is_final=True
        )

Error Context and Debugging

Accessing Exception Context

def handle_tavern_exception(e: TavernException):
    """Extract useful debugging information from Tavern exceptions."""
    
    print(f"Error: {e}")
    
    if e.stage:
        print(f"Failed stage: {e.stage}")
    
    if e.test_block_config:
        print(f"Test variables: {e.test_block_config.variables}")
        print(f"Test name: {e.test_block_config.test_name}")
    
    if e.is_final:
        print("This is a final error - test execution should stop")

Exception Hierarchy for Debugging

# Exception hierarchy allows for specific error handling
try:
    run_tavern_tests()
except KeyMismatchError as e:
    # Handle response validation specifically
    log_response_mismatch(e)
except BadSchemaError as e:
    # Handle schema issues specifically
    fix_test_schema(e)
except MQTTError as e:
    # Handle MQTT-specific issues
    check_mqtt_broker(e)
except TavernException as e:
    # Handle any other Tavern errors
    log_general_error(e)
except Exception as e:
    # Handle non-Tavern errors
    log_unexpected_error(e)

Types

from typing import Optional

TestConfig = "tavern._core.pytest.config.TestConfig"

Install with Tessl CLI

npx tessl i tessl/pypi-tavern

docs

core-execution.md

exceptions.md

index.md

plugin-system.md

pytest-integration.md

response-validation.md

tile.json