CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-google-cloud-vision

Google Cloud Vision API client library for image analysis, OCR, face detection, and machine learning-based visual content recognition.

Pending
Overview
Eval results
Files

types-and-data.mddocs/

Data Types and Response Objects

Comprehensive type system for Google Cloud Vision API, including request/response objects, annotation results, geometry types, configuration parameters, and enums. This provides strong typing for all API interactions with detailed structured data for analysis results.

Request and Response Types

Core Annotation Types

Primary request and response types for image annotation operations.

class AnnotateImageRequest:
    """Request for annotating a single image."""
    image: Image
    features: List[Feature]
    image_context: Optional[ImageContext]

class AnnotateImageResponse:
    """Response from annotating a single image."""
    face_annotations: List[FaceAnnotation]
    landmark_annotations: List[EntityAnnotation]  
    logo_annotations: List[EntityAnnotation]
    label_annotations: List[EntityAnnotation]
    text_annotations: List[EntityAnnotation]
    full_text_annotation: Optional[TextAnnotation]
    safe_search_annotation: Optional[SafeSearchAnnotation]
    image_properties_annotation: Optional[ImageProperties]
    crop_hints_annotation: Optional[CropHintsAnnotation]
    web_detection: Optional[WebDetection]
    product_search_results: Optional[ProductSearchResults]
    localized_object_annotations: List[LocalizedObjectAnnotation]
    error: Optional[Status]

class BatchAnnotateImagesRequest:
    """Request for batch annotation of multiple images."""
    requests: List[AnnotateImageRequest]
    parent: Optional[str]

class BatchAnnotateImagesResponse:
    """Response from batch annotation of multiple images."""
    responses: List[AnnotateImageResponse]

class BatchAnnotateFilesRequest:
    """Request for batch annotation of files (PDF/TIFF)."""
    requests: List[AnnotateFileRequest]
    parent: Optional[str]

class BatchAnnotateFilesResponse:
    """Response from batch annotation of files."""
    responses: List[AnnotateFileResponse]

class AnnotateFileRequest:
    """Request for annotating a file."""
    input_config: InputConfig
    features: List[Feature]
    image_context: Optional[ImageContext]
    pages: Optional[List[int]]

class AnnotateFileResponse:
    """Response from annotating a file."""
    input_config: InputConfig
    responses: List[AnnotateImageResponse]
    total_pages: int
    error: Optional[Status]

Async Operation Types

Types for long-running asynchronous operations.

class AsyncAnnotateFileRequest:
    """Request for asynchronous file annotation."""
    input_config: InputConfig
    features: List[Feature]
    image_context: Optional[ImageContext]
    output_config: OutputConfig

class AsyncAnnotateFileResponse:
    """Response from asynchronous file annotation."""
    output_config: OutputConfig

class AsyncBatchAnnotateImagesRequest:
    """Request for asynchronous batch image annotation."""
    requests: List[AnnotateImageRequest]
    output_config: OutputConfig
    parent: Optional[str]

class AsyncBatchAnnotateImagesResponse:
    """Response from asynchronous batch image annotation."""
    output_config: OutputConfig

class AsyncBatchAnnotateFilesRequest:
    """Request for asynchronous batch file annotation."""
    requests: List[AsyncAnnotateFileRequest]
    parent: Optional[str]

class AsyncBatchAnnotateFilesResponse:
    """Response from asynchronous batch file annotation."""
    responses: List[AsyncAnnotateFileResponse]

Detection Result Types

Face Detection

Detailed face detection results with landmarks and attributes.

class FaceAnnotation:
    """Face detection result with detailed attributes."""
    bounding_poly: BoundingPoly
    fd_bounding_poly: BoundingPoly
    landmarks: List[Landmark]
    roll_angle: float
    pan_angle: float
    tilt_angle: float
    detection_confidence: float
    landmarking_confidence: float
    joy_likelihood: Likelihood
    sorrow_likelihood: Likelihood
    anger_likelihood: Likelihood
    surprise_likelihood: Likelihood
    under_exposed_likelihood: Likelihood
    blurred_likelihood: Likelihood
    headwear_likelihood: Likelihood

class Landmark:
    """Individual facial landmark."""
    type_: LandmarkType
    position: Position

Entity Detection

General entity detection results for landmarks, logos, and labels.

class EntityAnnotation:
    """Generic entity detected in image."""
    mid: str
    locale: str
    description: str
    score: float
    confidence: float
    topicality: float
    bounding_poly: Optional[BoundingPoly]
    locations: List[LocationInfo]
    properties: List[Property]

Object Localization

Localized object detection results.

class LocalizedObjectAnnotation:
    """Localized object detection result."""
    mid: str
    language_code: str
    name: str
    score: float
    bounding_poly: BoundingPoly

Text Detection

Comprehensive text detection and OCR results.

class TextAnnotation:
    """Full text annotation result."""
    pages: List[Page]
    text: str

class Page:
    """Page of text with structured hierarchy."""
    property: Optional[TextProperty]
    width: int
    height: int
    blocks: List[Block]
    confidence: float

class Block:
    """Block of text (e.g., paragraph)."""
    property: Optional[TextProperty]
    bounding_box: BoundingPoly
    paragraphs: List[Paragraph]
    block_type: BlockType
    confidence: float

class Paragraph:
    """Paragraph within a block."""
    property: Optional[TextProperty]
    bounding_box: BoundingPoly
    words: List[Word]
    confidence: float

class Word:
    """Word within a paragraph."""
    property: Optional[TextProperty]
    bounding_box: BoundingPoly
    symbols: List[Symbol]
    confidence: float

class Symbol:
    """Individual symbol/character."""
    property: Optional[TextProperty]
    bounding_box: BoundingPoly
    text: str
    confidence: float

Safe Search Detection

Content safety classification results.

class SafeSearchAnnotation:
    """Safe search detection result."""
    adult: Likelihood
    spoof: Likelihood
    medical: Likelihood
    violence: Likelihood
    racy: Likelihood

Image Properties

Image color and visual properties analysis.

class ImageProperties:
    """Properties of an image."""
    dominant_colors: DominantColorsAnnotation

class DominantColorsAnnotation:
    """Dominant color information."""
    colors: List[ColorInfo]

class ColorInfo:
    """Color information for detected objects."""
    color: Color
    score: float
    pixel_fraction: float

class Color:
    """RGB color representation."""
    red: float
    green: float
    blue: float
    alpha: Optional[float]

Crop Hints

Suggested image cropping information.

class CropHintsAnnotation:
    """Collection of crop hints."""
    crop_hints: List[CropHint]

class CropHint:
    """Suggested crop for an image."""
    bounding_poly: BoundingPoly
    confidence: float
    importance_fraction: float

Web Detection

Web entity and similar image detection results.

class WebDetection:
    """Web detection results."""
    web_entities: List[WebEntity]
    full_matching_images: List[WebImage]
    partial_matching_images: List[WebImage]
    pages_with_matching_images: List[WebPage]
    visually_similar_images: List[WebImage]
    best_guess_labels: List[WebLabel]

class WebEntity:
    """Entity from the web."""
    entity_id: str
    score: float
    description: str

class WebImage:
    """Metadata for online image."""
    url: str
    score: float

class WebPage:
    """Metadata for web page."""
    url: str
    score: float
    page_title: str
    full_matching_images: List[WebImage]
    partial_matching_images: List[WebImage]

class WebLabel:
    """Label from web detection."""
    label: str
    language_code: str

Geometry Types

Spatial positioning and boundary information.

class BoundingPoly:
    """Bounding polygon for detected objects."""
    vertices: List[Vertex]
    normalized_vertices: List[NormalizedVertex]

class Vertex:
    """2D vertex coordinates."""
    x: int
    y: int

class NormalizedVertex:
    """Vertex coordinates normalized to [0,1]."""
    x: float
    y: float

class Position:
    """3D position coordinates."""
    x: float
    y: float
    z: float

class LatLongRect:
    """Latitude/longitude rectangle."""
    min_lat_lng: LatLng
    max_lat_lng: LatLng

class LatLng:
    """Latitude/longitude pair."""
    latitude: float
    longitude: float

Configuration Types

Feature Configuration

Configuration for detection features and processing options.

class Feature:
    """Detection feature configuration."""
    type_: FeatureType
    max_results: int
    model: str
    
    class Type:
        """Feature type enumeration."""
        TYPE_UNSPECIFIED = 0
        FACE_DETECTION = 1
        LANDMARK_DETECTION = 2
        LOGO_DETECTION = 3
        LABEL_DETECTION = 4
        TEXT_DETECTION = 5
        DOCUMENT_TEXT_DETECTION = 11
        SAFE_SEARCH_DETECTION = 6
        IMAGE_PROPERTIES = 7
        CROP_HINTS = 9
        WEB_DETECTION = 10
        PRODUCT_SEARCH = 12
        OBJECT_LOCALIZATION = 19

class ImageContext:
    """Context and region hints for image analysis."""
    lat_long_rect: Optional[LatLongRect]
    language_hints: List[str]
    crop_hints_params: Optional[CropHintsParams]
    product_search_params: Optional[ProductSearchParams]
    web_detection_params: Optional[WebDetectionParams]
    text_detection_params: Optional[TextDetectionParams]

class CropHintsParams:
    """Parameters for crop hint detection."""
    aspect_ratios: List[float]

class ProductSearchParams:
    """Parameters for product search."""
    bounding_poly: Optional[BoundingPoly]
    product_set: str
    product_categories: List[str]
    filter: str

class WebDetectionParams:
    """Parameters for web detection."""
    include_geo_results: bool

class TextDetectionParams:
    """Parameters for text detection."""
    enable_text_detection_confidence_score: bool
    advanced_ocr_options: List[str]

Image Input Types

Types for specifying image input sources.

class Image:
    """Image data for analysis."""
    content: Optional[bytes]
    source: Optional[ImageSource]

class ImageSource:
    """Source location of image."""
    gcs_image_uri: Optional[str]
    image_uri: Optional[str]

class ImageAnnotationContext:
    """Context for image annotation."""
    uri: str
    page_number: int

Product Search Types

Product Catalog Types

Types for managing product catalogs and search functionality.

class Product:
    """Product definition."""
    name: str
    display_name: str
    description: str
    product_category: str
    product_labels: List[KeyValue]

class ProductSet:
    """Product set definition."""
    name: str
    display_name: str
    index_time: Timestamp
    index_error: Optional[Status]

class ReferenceImage:
    """Reference image for products."""
    name: str
    uri: str
    bounding_polys: List[BoundingPoly]

class ProductSearchResults:
    """Results from product search."""
    index_time: Timestamp
    results: List[Result]
    product_grouped_results: List[GroupedResult]
    
class Result:
    """Individual search result."""
    product: Product
    score: float
    image: str

class GroupedResult:
    """Grouped search results."""
    bounding_poly: BoundingPoly
    results: List[Result]
    object_annotations: List[ObjectAnnotation]

Product Management Request Types

Request types for product catalog management operations.

class CreateProductSetRequest:
    """Request to create product set."""
    parent: str
    product_set: ProductSet
    product_set_id: str

class ListProductSetsRequest:
    """Request to list product sets."""
    parent: str
    page_size: int
    page_token: str

class ListProductSetsResponse:
    """Response from listing product sets."""
    product_sets: List[ProductSet]
    next_page_token: str

class GetProductSetRequest:
    """Request to get product set."""
    name: str

class UpdateProductSetRequest:
    """Request to update product set."""
    product_set: ProductSet
    update_mask: FieldMask

class DeleteProductSetRequest:
    """Request to delete product set."""
    name: str

class CreateProductRequest:
    """Request to create product."""
    parent: str
    product: Product
    product_id: str

class ListProductsRequest:
    """Request to list products."""
    parent: str
    page_size: int
    page_token: str

class ListProductsResponse:
    """Response from listing products."""
    products: List[Product]
    next_page_token: str

class GetProductRequest:
    """Request to get product."""
    name: str

class UpdateProductRequest:
    """Request to update product."""
    product: Product
    update_mask: FieldMask

class DeleteProductRequest:
    """Request to delete product."""
    name: str

class CreateReferenceImageRequest:
    """Request to create reference image."""
    parent: str
    reference_image: ReferenceImage
    reference_image_id: str

class ListReferenceImagesRequest:
    """Request to list reference images."""
    parent: str
    page_size: int
    page_token: str

class ListReferenceImagesResponse:
    """Response from listing reference images."""
    reference_images: List[ReferenceImage]
    page_size: int
    next_page_token: str

class GetReferenceImageRequest:
    """Request to get reference image."""
    name: str

class DeleteReferenceImageRequest:
    """Request to delete reference image."""
    name: str

class AddProductToProductSetRequest:
    """Request to add product to set."""
    name: str
    product: str

class RemoveProductFromProductSetRequest:
    """Request to remove product from set."""
    name: str
    product: str

class ListProductsInProductSetRequest:
    """Request to list products in set."""
    name: str
    page_size: int
    page_token: str

class ListProductsInProductSetResponse:
    """Response from listing products in set."""
    products: List[Product]
    next_page_token: str

Import and Batch Operations

Types for bulk operations and data import.

class ImportProductSetsRequest:
    """Request to import product sets."""
    parent: str
    input_config: ImportProductSetsInputConfig

class ImportProductSetsResponse:
    """Response from importing product sets."""
    reference_images: List[ReferenceImage]
    statuses: List[Status]

class ImportProductSetsInputConfig:
    """Input configuration for importing."""
    gcs_source: ImportProductSetsGcsSource

class ImportProductSetsGcsSource:
    """GCS source for importing product sets."""
    csv_file_uri: str

class PurgeProductsRequest:
    """Request to purge products."""
    parent: str
    product_set_purge_config: Optional[ProductSetPurgeConfig]
    delete_orphan_products: bool
    force: bool

class ProductSetPurgeConfig:
    """Configuration for purging product sets."""
    product_set_id: str

class BatchOperationMetadata:
    """Metadata for batch operations."""
    state: State
    submit_time: Timestamp
    end_time: Timestamp

File and Storage Types

Types for file processing and cloud storage integration.

class InputConfig:
    """Input configuration for batch operations."""
    gcs_source: Optional[GcsSource]
    content: Optional[bytes]
    mime_type: str

class OutputConfig:
    """Output configuration for batch operations."""
    gcs_destination: GcsDestination
    batch_size: int

class GcsSource:
    """Google Cloud Storage source."""
    uri: str

class GcsDestination:
    """Google Cloud Storage destination."""
    uri: str

class OperationMetadata:
    """Metadata for long-running operations."""
    state: OperationState
    create_time: Timestamp
    update_time: Timestamp

Enums and Constants

Likelihood Enumeration

Standard likelihood levels for various detections.

class Likelihood:
    """Likelihood levels for detection confidence."""
    UNKNOWN = 0          # Unknown likelihood
    VERY_UNLIKELY = 1    # Very unlikely
    UNLIKELY = 2         # Unlikely
    POSSIBLE = 3         # Possible
    LIKELY = 4           # Likely
    VERY_LIKELY = 5      # Very likely

Utility Types

Common utility types used throughout the API.

class Property:
    """Generic name/value property pair."""
    name: str
    value: str
    uint64_value: int

class KeyValue:
    """Key-value pair."""
    key: str
    value: str

class LocationInfo:
    """Location information."""
    lat_lng: LatLng

class Status:
    """Error status information."""
    code: int
    message: str
    details: List[Any]

class Timestamp:
    """Timestamp representation."""
    seconds: int
    nanos: int

class FieldMask:
    """Field mask for partial updates."""
    paths: List[str]

Usage Examples

Working with Detection Results

from google.cloud.vision import ImageAnnotatorClient, Image, Likelihood

client = ImageAnnotatorClient()
image = Image(source={'image_uri': 'https://example.com/photo.jpg'})
response = client.annotate_image({'image': image})

# Face detection results
for face in response.face_annotations:
    print(f"Face bounds: {face.bounding_poly.vertices}")
    print(f"Joy: {face.joy_likelihood}")
    print(f"Confidence: {face.detection_confidence}")
    
    # Check likelihood levels
    if face.joy_likelihood >= Likelihood.LIKELY:
        print("Happy face detected!")

# Text detection results  
if response.text_annotations:
    full_text = response.text_annotations[0].description
    print(f"Detected text: {full_text}")
    
    # Structured text from full_text_annotation
    if response.full_text_annotation:
        for page in response.full_text_annotation.pages:
            for block in page.blocks:
                for paragraph in block.paragraphs:
                    para_text = ''.join([
                        symbol.text for word in paragraph.words 
                        for symbol in word.symbols
                    ])
                    print(f"Paragraph: {para_text}")

# Label detection results
for label in response.label_annotations:
    print(f"Label: {label.description}")
    print(f"Score: {label.score}")
    print(f"Topicality: {label.topicality}")

# Object localization results
for obj in response.localized_object_annotations:
    print(f"Object: {obj.name}")
    print(f"Confidence: {obj.score}")
    vertices = obj.bounding_poly.normalized_vertices
    print(f"Bounds: ({vertices[0].x}, {vertices[0].y}) to ({vertices[2].x}, {vertices[2].y})")

Creating Structured Requests

from google.cloud.vision import (
    AnnotateImageRequest, Feature, Image, ImageContext, 
    TextDetectionParams, ProductSearchParams
)

# Create structured request
image = Image(source={'image_uri': 'https://example.com/document.jpg'})

# Configure text detection with confidence scores
text_params = TextDetectionParams(
    enable_text_detection_confidence_score=True
)

image_context = ImageContext(
    text_detection_params=text_params,
    language_hints=['en']
)

features = [
    Feature(type_=Feature.Type.DOCUMENT_TEXT_DETECTION, max_results=1),
    Feature(type_=Feature.Type.LABEL_DETECTION, max_results=10)
]

request = AnnotateImageRequest(
    image=image,
    features=features,
    image_context=image_context
)

response = client.annotate_image(request)

# Access confidence scores in text detection
if response.full_text_annotation:
    for page in response.full_text_annotation.pages:
        print(f"Page confidence: {page.confidence}")
        for block in page.blocks:
            print(f"Block confidence: {block.confidence}")

Batch Processing with Type Safety

from google.cloud.vision import BatchAnnotateImagesRequest

# Create multiple requests
requests = []
image_urls = [
    'https://example.com/image1.jpg',
    'https://example.com/image2.jpg', 
    'https://example.com/image3.jpg'
]

for url in image_urls:
    image = Image(source={'image_uri': url})
    request = AnnotateImageRequest(
        image=image,
        features=[Feature(type_=Feature.Type.LABEL_DETECTION, max_results=5)]
    )
    requests.append(request)

# Create batch request
batch_request = BatchAnnotateImagesRequest(requests=requests)
batch_response = client.batch_annotate_images(batch_request)

# Process results with type safety
for i, response in enumerate(batch_response.responses):
    print(f"Image {i+1} results:")
    if response.error:
        print(f"Error: {response.error.message}")
        continue
        
    for label in response.label_annotations:
        print(f"  {label.description}: {label.score}")

Install with Tessl CLI

npx tessl i tessl/pypi-google-cloud-vision

docs

image-annotation.md

index.md

product-search.md

types-and-data.md

tile.json