tessl/pypi-tavern

Simple testing of RESTful APIs

Overview

Eval results

Files

Response Validation

Name: tessl/pypi-tavern
Author: tessl

Comprehensive validation functions for API responses including JWT validation, regex matching, schema validation, and content assertions using JMESPath expressions.

Capabilities

JWT Token Validation

Validates and decodes JWT tokens from API responses using the PyJWT library, returning decoded claims for use in subsequent test stages.

def validate_jwt(
    response: requests.Response, 
    jwt_key: str, 
    **kwargs
) -> Mapping[str, Box]:
    """
    Validate and decode JWT token from response.

    Parameters:
    - response: HTTP response object containing JWT
    - jwt_key: Key name in response JSON containing the JWT token
    - **kwargs: Additional arguments passed to jwt.decode() (e.g., verify_signature=False)

    Returns:
    Mapping[str, Box]: Dictionary with 'jwt' key containing boxed JWT claims

    Usage:
    Can be used both for validation and to extract JWT claims for template variables
    """

Usage Examples:

import requests
from tavern.helpers import validate_jwt

# Basic JWT validation
response = requests.get("https://api.example.com/auth")
jwt_data = validate_jwt(response, "access_token", verify_signature=False)
print(jwt_data["jwt"]["user_id"])  # Access decoded claims

# In YAML test with validation
# response:
#   status_code: 200
#   json:
#     access_token: !anything
#   validate:
#     - function: tavern.helpers:validate_jwt
#       extra_kwargs:
#         jwt_key: access_token
#         verify_signature: false
#   save:
#     json:
#       token_claims: !jwt_claims access_token

Regex Pattern Validation

Validates response content against regular expressions with support for header matching and JMESPath content extraction.

def validate_regex(
    response: requests.Response,
    expression: str,
    *,
    header: Optional[str] = None,
    in_jmespath: Optional[str] = None,
) -> dict[str, Box]:
    """
    Validate response content against regex pattern.

    Parameters:
    - response: HTTP response object
    - expression: Regular expression pattern to match
    - header: Optional header name to match against instead of body
    - in_jmespath: Optional JMESPath to extract content before regex matching

    Returns:
    dict[str, Box]: Dictionary with 'regex' key containing boxed capture groups

    Raises:
    BadSchemaError: If both header and in_jmespath are specified
    RegexAccessError: If regex doesn't match or JMESPath extraction fails
    """

Usage Examples:

from tavern.helpers import validate_regex

# Match against response body
result = validate_regex(response, r"User ID: (\d+)")
user_id = result["regex"]["1"]  # First capture group

# Match against specific header
result = validate_regex(
    response, 
    r"Bearer ([a-zA-Z0-9]+)", 
    header="Authorization"
)
token = result["regex"]["1"]

# Match against JMESPath extracted content
result = validate_regex(
    response,
    r"Error: (.+)",
    in_jmespath="error.message"
)
error_msg = result["regex"]["1"]

# In YAML test
# response:
#   status_code: 200
#   validate:
#     - function: tavern.helpers:validate_regex
#       extra_kwargs:
#         expression: "Order #([0-9]+) created"
#   save:
#     regex:
#       order_id: "1"  # First capture group

Content Validation with JMESPath

Validates response content using JMESPath expressions with various comparison operators for complex assertions.

def validate_content(
    response: requests.Response, 
    comparisons: Iterable[dict]
) -> None:
    """
    Validate response content using JMESPath expressions and operators.

    Parameters:
    - response: HTTP response object with JSON content
    - comparisons: List of comparison dictionaries with keys:
      - jmespath: JMESPath expression to extract data
      - operator: Comparison operator (eq, ne, gt, lt, gte, lte, in, contains)
      - expected: Expected value to compare against

    Raises:
    JMESError: If validation fails or JMESPath expression is invalid
    """

Usage Examples:

from tavern.helpers import validate_content

# Single comparison
comparisons = [{
    "jmespath": "users.length(@)",
    "operator": "gt",
    "expected": 0
}]
validate_content(response, comparisons)

# Multiple comparisons
comparisons = [
    {
        "jmespath": "status",
        "operator": "eq", 
        "expected": "success"
    },
    {
        "jmespath": "data.users[0].age",
        "operator": "gte",
        "expected": 18
    },
    {
        "jmespath": "data.users[].name",
        "operator": "contains",
        "expected": "John"
    }
]
validate_content(response, comparisons)

JMESPath Query Matching

Executes JMESPath queries against parsed response data with optional value validation.

def check_jmespath_match(
    parsed_response, 
    query: str, 
    expected: Optional[str] = None
):
    """
    Check JMESPath query against response data.

    Parameters:
    - parsed_response: Parsed response data (dict or list)
    - query: JMESPath query expression
    - expected: Optional expected value to match against query result

    Returns:
    Query result value

    Raises:
    JMESError: If query path not found or doesn't match expected value
    """

Usage Examples:

from tavern.helpers import check_jmespath_match

# Check if path exists and return value
data = {"users": [{"name": "John", "age": 30}]}
result = check_jmespath_match(data, "users[0].name")
print(result)  # "John"

# Check path exists with expected value
check_jmespath_match(data, "users[0].age", 30)  # Passes
check_jmespath_match(data, "users.length(@)", 1)  # Passes

# Check complex queries
check_jmespath_match(
    data, 
    "users[?age > `25`].name",
    ["John"]
)

Schema Validation

Validates response JSON against Pykwalify schemas for structural and type validation.

def validate_pykwalify(
    response: requests.Response, 
    schema: dict
) -> None:
    """
    Validate response JSON against Pykwalify schema.

    Parameters:
    - response: HTTP response object with JSON content
    - schema: Pykwalify schema dictionary

    Raises:
    BadSchemaError: If response is not JSON or doesn't match schema
    """

Usage Examples:

from tavern.helpers import validate_pykwalify

# Define schema
schema = {
    "type": "map",
    "mapping": {
        "status": {"type": "str", "required": True},
        "data": {
            "type": "map",
            "mapping": {
                "users": {
                    "type": "seq",
                    "sequence": [{
                        "type": "map",
                        "mapping": {
                            "id": {"type": "int"},
                            "name": {"type": "str"},
                            "email": {"type": "str", "pattern": r".+@.+\..+"}
                        }
                    }]
                }
            }
        }
    }
}

# Validate response
validate_pykwalify(response, schema)

Exception Validation

Validates that API error responses match expected exception formats and status codes.

def check_exception_raised(
    response: requests.Response, 
    exception_location: str
) -> None:
    """
    Validate response matches expected exception format.

    Parameters:
    - response: HTTP response object containing error
    - exception_location: Entry point style location of exception class 
                         (e.g., "myapp.exceptions:ValidationError")

    Raises:
    UnexpectedExceptionError: If response doesn't match expected exception format
    """

Usage Examples:

from tavern.helpers import check_exception_raised

# Validate error response format
try:
    response = requests.post("https://api.example.com/invalid")
    check_exception_raised(response, "myapp.exceptions:ValidationError")
except UnexpectedExceptionError:
    print("Error response format doesn't match expected exception")

# In YAML test
# response:
#   status_code: 400
#   validate:
#     - function: tavern.helpers:check_exception_raised
#       extra_kwargs:
#         exception_location: "myapp.exceptions:ValidationError"

YAML Integration Examples

Using Validation Functions in Tests

test_name: Comprehensive validation example

stages:
  - name: Test with multiple validations
    request:
      url: https://api.example.com/data
      method: GET
    response:
      status_code: 200
      validate:
        # JWT validation
        - function: tavern.helpers:validate_jwt
          extra_kwargs:
            jwt_key: access_token
            verify_signature: false
        
        # Regex validation
        - function: tavern.helpers:validate_regex
          extra_kwargs:
            expression: "Request ID: ([A-Z0-9-]+)"
            header: "X-Request-ID"
        
        # Content validation
        - function: tavern.helpers:validate_content
          extra_kwargs:
            comparisons:
              - jmespath: "data.items.length(@)"
                operator: "gt"
                expected: 0
              - jmespath: "status"
                operator: "eq"
                expected: "success"
        
        # Schema validation  
        - function: tavern.helpers:validate_pykwalify
          extra_kwargs:
            schema:
              type: map
              mapping:
                status: {type: str}
                data: {type: map}
      
      save:
        jwt:
          user_id: user_id
        regex:
          request_id: "1"

Types

from typing import Optional, Mapping, Iterable
import requests
from box.box import Box

BadSchemaError = "tavern._core.exceptions.BadSchemaError"
RegexAccessError = "tavern._core.exceptions.RegexAccessError"  
JMESError = "tavern._core.exceptions.JMESError"
UnexpectedExceptionError = "tavern._core.exceptions.UnexpectedExceptionError"

Install with Tessl CLI