tessl/pypi-opensearch-py

Python client for OpenSearch providing comprehensive search, indexing, and cluster management capabilities

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

OpenSearch Python Client

Name: tessl/pypi-opensearch-py
Author: tessl

A comprehensive Python client library for OpenSearch, an open-source search and analytics engine. The client provides both synchronous and asynchronous APIs for performing search operations, document indexing, cluster management, and administrative tasks with advanced features including SSL/TLS configuration, authentication mechanisms, connection pooling, and extensive plugin support.

Package Information

Package Name: opensearch-py
Language: Python
Installation: pip install opensearch-py
Version: 3.0.0

Core Imports

Basic client import:

from opensearchpy import OpenSearch

Async client import:

from opensearchpy import AsyncOpenSearch

DSL imports for search and document modeling:

from opensearchpy import Search, Q, A, Document, Mapping

Helper function imports:

from opensearchpy.helpers import bulk, scan, reindex

Analysis function imports:

from opensearchpy import analyzer, tokenizer, token_filter, char_filter, normalizer

Authentication imports:

from opensearchpy import RequestsAWSV4SignerAuth

Basic Usage

Basic Client Setup

from opensearchpy import OpenSearch

# Basic client with host and port
client = OpenSearch(
    hosts=[{'host': 'localhost', 'port': 9200}],
    http_compress=True,
    use_ssl=True,
    verify_certs=True,
    ssl_assert_hostname=False,
    ssl_show_warn=False,
)

# Test connection
response = client.ping()
print(response)  # True if successful

# Get cluster info
info = client.info()
print(info)

Basic Search Operations

# Index a document
doc = {
    'title': 'OpenSearch Python Client',
    'content': 'This is a sample document for indexing',
    'timestamp': '2024-01-01T00:00:00'
}

response = client.index(
    index='sample-index',
    id='1',
    body=doc
)

# Search for documents
search_body = {
    'query': {
        'match': {
            'title': 'OpenSearch'
        }
    }
}

results = client.search(
    index='sample-index',
    body=search_body
)

print(results['hits']['hits'])

Basic DSL Usage

from opensearchpy import Search, Q

# Create a search object
s = Search(using=client, index='sample-index')

# Add a query using Q object
s = s.query(Q('match', title='OpenSearch'))

# Execute the search
response = s.execute()

for hit in response:
    print(hit.title, hit.content)

Architecture

The OpenSearch Python client provides a multi-layered architecture:

Client Layer: OpenSearch and AsyncOpenSearch classes provide the main interface
Namespaced APIs: Specialized clients for different OpenSearch APIs (cat, cluster, indices, etc.)
Plugin Support: Dedicated clients for OpenSearch plugins (security, alerting, ML, etc.)
Transport Layer: HTTP transport handling with connection pooling and retry logic
DSL Layer: Domain-specific language for query building and document modeling
Helper Functions: High-level utilities for common operations like bulk indexing

Capabilities

Core Client Operations

Essential OpenSearch operations including document indexing, searching, updating, and deletion. Provides both low-level API access and high-level convenience methods for interacting with OpenSearch clusters.

class OpenSearch:
    def ping(self, **kwargs): ...
    def info(self, **kwargs): ...
    def search(self, index=None, body=None, **kwargs): ...
    def index(self, index, body, id=None, **kwargs): ...
    def get(self, index, id, **kwargs): ...
    def update(self, index, id, body, **kwargs): ...
    def delete(self, index, id, **kwargs): ...
    def bulk(self, body, index=None, **kwargs): ...
    def count(self, index=None, body=None, **kwargs): ...
    def exists(self, index, id, **kwargs): ...

Core Client Operations

Namespaced APIs

Specialized clients for different aspects of OpenSearch cluster management including indices, cluster operations, node management, and administrative functions.

class OpenSearch:
    @property
    def indices(self) -> IndicesClient: ...
    @property 
    def cluster(self) -> ClusterClient: ...
    @property
    def cat(self) -> CatClient: ...
    @property
    def nodes(self) -> NodesClient: ...
    @property
    def ingest(self) -> IngestClient: ...
    @property
    def snapshot(self) -> SnapshotClient: ...
    @property
    def tasks(self) -> TasksClient: ...

Namespaced APIs

Plugin APIs

Dedicated clients for OpenSearch plugins providing advanced functionality like security, machine learning, alerting, and specialized search capabilities.

class OpenSearch:
    @property
    def security(self) -> SecurityClient: ...
    @property
    def ml(self) -> MlClient: ...
    @property
    def alerting(self) -> AlertingClient: ...
    @property
    def sql(self) -> SqlClient: ...
    @property
    def knn(self) -> KnnClient: ...
    @property
    def neural(self) -> NeuralClient: ...
    @property
    def geospatial(self) -> GeospatialClient: ...
    @property
    def observability(self) -> ObservabilityClient: ...
    @property
    def replication(self) -> ReplicationClient: ...
    @property
    def search_relevance(self) -> SearchRelevanceClient: ...
    @property
    def ltr(self) -> LtrClient: ...
    @property
    def query(self) -> QueryClient: ...

Plugin APIs

DSL and Query Building

Domain-specific language for building complex search queries, aggregations, and document models with a Pythonic API that abstracts OpenSearch's JSON-based query syntax.

class Search:
    def __init__(self, using=None, index=None): ...
    def query(self, query): ...
    def filter(self, filter): ...
    def aggs(self, aggs): ...
    def sort(self, *keys): ...
    def execute(self): ...

class Q:
    @classmethod
    def match(cls, **kwargs): ...
    @classmethod
    def term(cls, **kwargs): ...
    @classmethod
    def range(cls, **kwargs): ...
    @classmethod
    def bool(cls, **kwargs): ...

DSL and Query Building

Document Modeling

Object-relational mapping (ORM) style document modeling with field definitions, automatic mapping generation, and validation for structured data handling in OpenSearch.

class Document:
    def __init__(self, **kwargs): ...
    def save(self, **kwargs): ...
    def delete(self, **kwargs): ...
    @classmethod
    def get(cls, id, **kwargs): ...
    @classmethod
    def search(cls): ...

class Mapping:
    def __init__(self): ...
    def field(self, name, field_type, **kwargs): ...
    def save(self, index, **kwargs): ...

Document Modeling

Helper Functions

High-level utility functions for common operations like bulk indexing, scanning large result sets, and data reindexing with built-in error handling and performance optimizations.

def bulk(client, actions, **kwargs):
    """
    Perform bulk indexing operations.
    
    Parameters:
    - client: OpenSearch client instance
    - actions: iterable of action dictionaries
    - chunk_size: number of docs per chunk (default: 500)
    - max_chunk_bytes: max size per chunk in bytes
    - thread_count: number of parallel threads
    
    Returns:
    Tuple of (success_count, failed_actions)
    """

def scan(client, query=None, scroll='5m', **kwargs):
    """
    Scan search results for large datasets.
    
    Parameters:
    - client: OpenSearch client instance  
    - query: search query body
    - scroll: scroll timeout
    - index: index name(s)
    
    Yields:
    Individual document hits
    """

Helper Functions

Authentication and Security

Authentication mechanisms including AWS IAM integration, SSL/TLS configuration, and security features for secure connections to OpenSearch clusters.

class RequestsAWSV4SignerAuth:
    def __init__(self, credentials, region, service='es'): ...

class OpenSearch:
    def __init__(
        self,
        hosts=None,
        http_auth=None,
        use_ssl=False,
        verify_certs=True,
        ssl_context=None,
        **kwargs
    ): ...

Authentication and Security

Async Operations

Asynchronous client operations using Python's asyncio for high-performance applications requiring concurrent OpenSearch operations.

class AsyncOpenSearch:
    async def ping(self, **kwargs): ...
    async def info(self, **kwargs): ...
    async def search(self, index=None, body=None, **kwargs): ...
    async def index(self, index, body, id=None, **kwargs): ...
    async def bulk(self, body, index=None, **kwargs): ...

Async Operations

Exception Handling

The client provides a comprehensive exception hierarchy for handling different types of errors:

class OpenSearchException(Exception): ...
class TransportError(OpenSearchException): ...
class ConnectionError(TransportError): ...
class RequestError(TransportError): ...  # HTTP 400
class AuthenticationException(RequestError): ...  # HTTP 401
class AuthorizationException(RequestError): ...  # HTTP 403
class NotFoundError(RequestError): ...  # HTTP 404
class ConflictError(RequestError): ...  # HTTP 409

Version Information

__version__: tuple  # Version tuple (major, minor, patch)
__versionstr__: str  # Version string
VERSION: tuple  # Version tuple alias

Type Definitions

Core Types

from typing import Any, Dict, List, Optional, Union

# Common type aliases used throughout the API
JsonDict = Dict[str, Any]
JsonList = List[Any]
JsonValue = Union[str, int, float, bool, None, JsonDict, JsonList]

# Client configuration types
HostConfig = Union[str, Dict[str, Any]]
HostsList = List[HostConfig]

# Authentication types  
AuthType = Union[tuple, callable, Any]

Workspace: tessl
Visibility: Public
Created: 7 months ago
Last updated: 3 months ago
Describes: pkg:pypi/opensearch-py@3.0.x
Publish Source: CLI
Badge