or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

async-operations.mdauthentication.mdcore-client.mddocument-modeling.mddsl-queries.mdhelper-functions.mdindex.mdnamespaced-apis.mdplugin-apis.md
tile.json

tessl/pypi-opensearch-py

Python client for OpenSearch providing comprehensive search, indexing, and cluster management capabilities

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/opensearch-py@3.0.x

To install, run

npx @tessl/cli install tessl/pypi-opensearch-py@3.0.0

index.mddocs/

OpenSearch Python Client

A comprehensive Python client library for OpenSearch, an open-source search and analytics engine. The client provides both synchronous and asynchronous APIs for performing search operations, document indexing, cluster management, and administrative tasks with advanced features including SSL/TLS configuration, authentication mechanisms, connection pooling, and extensive plugin support.

Package Information

  • Package Name: opensearch-py
  • Language: Python
  • Installation: pip install opensearch-py
  • Version: 3.0.0

Core Imports

Basic client import:

from opensearchpy import OpenSearch

Async client import:

from opensearchpy import AsyncOpenSearch

DSL imports for search and document modeling:

from opensearchpy import Search, Q, A, Document, Mapping

Helper function imports:

from opensearchpy.helpers import bulk, scan, reindex

Analysis function imports:

from opensearchpy import analyzer, tokenizer, token_filter, char_filter, normalizer

Authentication imports:

from opensearchpy import RequestsAWSV4SignerAuth

Basic Usage

Basic Client Setup

from opensearchpy import OpenSearch

# Basic client with host and port
client = OpenSearch(
    hosts=[{'host': 'localhost', 'port': 9200}],
    http_compress=True,
    use_ssl=True,
    verify_certs=True,
    ssl_assert_hostname=False,
    ssl_show_warn=False,
)

# Test connection
response = client.ping()
print(response)  # True if successful

# Get cluster info
info = client.info()
print(info)

Basic Search Operations

# Index a document
doc = {
    'title': 'OpenSearch Python Client',
    'content': 'This is a sample document for indexing',
    'timestamp': '2024-01-01T00:00:00'
}

response = client.index(
    index='sample-index',
    id='1',
    body=doc
)

# Search for documents
search_body = {
    'query': {
        'match': {
            'title': 'OpenSearch'
        }
    }
}

results = client.search(
    index='sample-index',
    body=search_body
)

print(results['hits']['hits'])

Basic DSL Usage

from opensearchpy import Search, Q

# Create a search object
s = Search(using=client, index='sample-index')

# Add a query using Q object
s = s.query(Q('match', title='OpenSearch'))

# Execute the search
response = s.execute()

for hit in response:
    print(hit.title, hit.content)

Architecture

The OpenSearch Python client provides a multi-layered architecture:

  • Client Layer: OpenSearch and AsyncOpenSearch classes provide the main interface
  • Namespaced APIs: Specialized clients for different OpenSearch APIs (cat, cluster, indices, etc.)
  • Plugin Support: Dedicated clients for OpenSearch plugins (security, alerting, ML, etc.)
  • Transport Layer: HTTP transport handling with connection pooling and retry logic
  • DSL Layer: Domain-specific language for query building and document modeling
  • Helper Functions: High-level utilities for common operations like bulk indexing

Capabilities

Core Client Operations

Essential OpenSearch operations including document indexing, searching, updating, and deletion. Provides both low-level API access and high-level convenience methods for interacting with OpenSearch clusters.

class OpenSearch:
    def ping(self, **kwargs): ...
    def info(self, **kwargs): ...
    def search(self, index=None, body=None, **kwargs): ...
    def index(self, index, body, id=None, **kwargs): ...
    def get(self, index, id, **kwargs): ...
    def update(self, index, id, body, **kwargs): ...
    def delete(self, index, id, **kwargs): ...
    def bulk(self, body, index=None, **kwargs): ...
    def count(self, index=None, body=None, **kwargs): ...
    def exists(self, index, id, **kwargs): ...

Core Client Operations

Namespaced APIs

Specialized clients for different aspects of OpenSearch cluster management including indices, cluster operations, node management, and administrative functions.

class OpenSearch:
    @property
    def indices(self) -> IndicesClient: ...
    @property 
    def cluster(self) -> ClusterClient: ...
    @property
    def cat(self) -> CatClient: ...
    @property
    def nodes(self) -> NodesClient: ...
    @property
    def ingest(self) -> IngestClient: ...
    @property
    def snapshot(self) -> SnapshotClient: ...
    @property
    def tasks(self) -> TasksClient: ...

Namespaced APIs

Plugin APIs

Dedicated clients for OpenSearch plugins providing advanced functionality like security, machine learning, alerting, and specialized search capabilities.

class OpenSearch:
    @property
    def security(self) -> SecurityClient: ...
    @property
    def ml(self) -> MlClient: ...
    @property
    def alerting(self) -> AlertingClient: ...
    @property
    def sql(self) -> SqlClient: ...
    @property
    def knn(self) -> KnnClient: ...
    @property
    def neural(self) -> NeuralClient: ...
    @property
    def geospatial(self) -> GeospatialClient: ...
    @property
    def observability(self) -> ObservabilityClient: ...
    @property
    def replication(self) -> ReplicationClient: ...
    @property
    def search_relevance(self) -> SearchRelevanceClient: ...
    @property
    def ltr(self) -> LtrClient: ...
    @property
    def query(self) -> QueryClient: ...

Plugin APIs

DSL and Query Building

Domain-specific language for building complex search queries, aggregations, and document models with a Pythonic API that abstracts OpenSearch's JSON-based query syntax.

class Search:
    def __init__(self, using=None, index=None): ...
    def query(self, query): ...
    def filter(self, filter): ...
    def aggs(self, aggs): ...
    def sort(self, *keys): ...
    def execute(self): ...

class Q:
    @classmethod
    def match(cls, **kwargs): ...
    @classmethod
    def term(cls, **kwargs): ...
    @classmethod
    def range(cls, **kwargs): ...
    @classmethod
    def bool(cls, **kwargs): ...

DSL and Query Building

Document Modeling

Object-relational mapping (ORM) style document modeling with field definitions, automatic mapping generation, and validation for structured data handling in OpenSearch.

class Document:
    def __init__(self, **kwargs): ...
    def save(self, **kwargs): ...
    def delete(self, **kwargs): ...
    @classmethod
    def get(cls, id, **kwargs): ...
    @classmethod
    def search(cls): ...

class Mapping:
    def __init__(self): ...
    def field(self, name, field_type, **kwargs): ...
    def save(self, index, **kwargs): ...

Document Modeling

Helper Functions

High-level utility functions for common operations like bulk indexing, scanning large result sets, and data reindexing with built-in error handling and performance optimizations.

def bulk(client, actions, **kwargs):
    """
    Perform bulk indexing operations.
    
    Parameters:
    - client: OpenSearch client instance
    - actions: iterable of action dictionaries
    - chunk_size: number of docs per chunk (default: 500)
    - max_chunk_bytes: max size per chunk in bytes
    - thread_count: number of parallel threads
    
    Returns:
    Tuple of (success_count, failed_actions)
    """

def scan(client, query=None, scroll='5m', **kwargs):
    """
    Scan search results for large datasets.
    
    Parameters:
    - client: OpenSearch client instance  
    - query: search query body
    - scroll: scroll timeout
    - index: index name(s)
    
    Yields:
    Individual document hits
    """

Helper Functions

Authentication and Security

Authentication mechanisms including AWS IAM integration, SSL/TLS configuration, and security features for secure connections to OpenSearch clusters.

class RequestsAWSV4SignerAuth:
    def __init__(self, credentials, region, service='es'): ...

class OpenSearch:
    def __init__(
        self,
        hosts=None,
        http_auth=None,
        use_ssl=False,
        verify_certs=True,
        ssl_context=None,
        **kwargs
    ): ...

Authentication and Security

Async Operations

Asynchronous client operations using Python's asyncio for high-performance applications requiring concurrent OpenSearch operations.

class AsyncOpenSearch:
    async def ping(self, **kwargs): ...
    async def info(self, **kwargs): ...
    async def search(self, index=None, body=None, **kwargs): ...
    async def index(self, index, body, id=None, **kwargs): ...
    async def bulk(self, body, index=None, **kwargs): ...

Async Operations

Exception Handling

The client provides a comprehensive exception hierarchy for handling different types of errors:

class OpenSearchException(Exception): ...
class TransportError(OpenSearchException): ...
class ConnectionError(TransportError): ...
class RequestError(TransportError): ...  # HTTP 400
class AuthenticationException(RequestError): ...  # HTTP 401
class AuthorizationException(RequestError): ...  # HTTP 403
class NotFoundError(RequestError): ...  # HTTP 404
class ConflictError(RequestError): ...  # HTTP 409

Version Information

__version__: tuple  # Version tuple (major, minor, patch)
__versionstr__: str  # Version string
VERSION: tuple  # Version tuple alias

Type Definitions

Core Types

from typing import Any, Dict, List, Optional, Union

# Common type aliases used throughout the API
JsonDict = Dict[str, Any]
JsonList = List[Any]
JsonValue = Union[str, int, float, bool, None, JsonDict, JsonList]

# Client configuration types
HostConfig = Union[str, Dict[str, Any]]
HostsList = List[HostConfig]

# Authentication types  
AuthType = Union[tuple, callable, Any]