or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

client-operations.mdgoogle-directory-apis.mdindex.mdmain-connector.md
tile.json

tessl/pypi-airbyte-source-google-directory

Airbyte source connector for Google Directory (Google Workspace Admin Directory API) that extracts users, groups, and group memberships.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/airbyte-source-google-directory@0.2.x

To install, run

npx @tessl/cli install tessl/pypi-airbyte-source-google-directory@0.2.0

index.mddocs/

Airbyte Source Google Directory

An Airbyte source connector that extracts data from Google Directory (Google Workspace Admin Directory API). This connector provides access to users, groups, and group memberships within a Google Workspace organization through structured data extraction with OAuth 2.0 and Service Account authentication support.

Package Information

  • Package Name: source-google-directory
  • Package Type: pypi
  • Language: Python
  • Installation: This package is part of the Airbyte ecosystem and is typically used within Airbyte platform. For development: poetry install --with dev
  • Dependencies: Requires airbyte-cdk, google-auth-oauthlib, google-auth-httplib2, google-api-python-client, backoff

Core Imports

from source_google_directory import SourceGoogleDirectory

For running the connector:

from source_google_directory.run import run

For accessing individual components:

from source_google_directory.client import Client
from source_google_directory.api import API, UsersAPI, GroupsAPI, GroupMembersAPI
from source_google_directory.utils import rate_limit_handling

Basic Usage

from source_google_directory import SourceGoogleDirectory

# Initialize the source connector
source = SourceGoogleDirectory()

# For command-line usage
from source_google_directory.run import run
run()  # Uses sys.argv for command-line arguments

# For programmatic access to individual components
from source_google_directory.client import Client

# OAuth credentials configuration
oauth_credentials = {
    "client_id": "your-client-id",
    "client_secret": "your-client-secret", 
    "refresh_token": "your-refresh-token"
}

# Service account credentials configuration  
service_credentials = {
    "credentials_json": '{"type": "service_account", ...}',
    "email": "admin@yourdomain.com"
}

# Create client with credentials
client = Client(credentials=oauth_credentials)

# Perform health check
is_healthy, error_msg = client.health_check()

Architecture

The connector follows Airbyte's CDK architecture with these key components:

  • SourceGoogleDirectory: Main connector class extending BaseSource from Airbyte CDK
  • Client: Primary client interface extending BaseClient, orchestrates API interactions
  • API: Core Google Directory API wrapper handling authentication and requests
  • StreamAPI: Abstract base class for specific data stream implementations (UsersAPI, GroupsAPI, GroupMembersAPI)
  • Utilities: Helper functions for rate limiting and error handling

The authentication system supports both OAuth 2.0 (web application flow) and Service Account scenarios, automatically detecting the credential type and establishing appropriate authentication flows.

Configuration

The connector accepts configuration following this schema:

# OAuth 2.0 Web Server Application Configuration
OAuthConfig = {
    "credentials": {
        "credentials_title": "Web server app",  # Authentication scenario identifier
        "client_id": str,                       # OAuth client ID (secret)
        "client_secret": str,                   # OAuth client secret (secret)
        "refresh_token": str                    # OAuth refresh token (secret)
    }
}

# Service Account Configuration  
ServiceAccountConfig = {
    "credentials": {
        "credentials_title": "Service accounts",  # Authentication scenario identifier
        "credentials_json": str,                  # JSON service account key (secret)
        "email": str                             # Admin email for delegation
    }
}

# The connector configuration uses oneOf schema validation
CredentialsConfig = Union[OAuthConfig, ServiceAccountConfig]

Configuration Fields

OAuth 2.0 Web Server Application:

  • client_id: The Client ID of the developer application (required, secret)
  • client_secret: The Client Secret of the developer application (required, secret)
  • refresh_token: The Token for obtaining a new access token (required, secret)

Service Account:

  • credentials_json: Contents of the JSON service account key file (required, secret)
  • email: Email of the user with Google Workspace Admin API permissions (required)

Both authentication methods require appropriate Google Cloud Console setup and Directory API scopes.

Capabilities

Main Connector Interface

The primary interface for the Airbyte source connector, providing the main entry point and connector lifecycle management.

class SourceGoogleDirectory(BaseSource):
    client_class = Client
def run():
    """Main entry point function that launches the connector."""

Main Connector

Client Operations

High-level client interface for managing Google Directory API interactions, including authentication, health checks, and stream enumeration.

class Client(BaseClient):
    def __init__(self, credentials: Mapping[str, Any] = None, credentials_json: str = None, email: str = None): ...
    def health_check(self) -> Tuple[bool, str]: ...
    def streams(self) -> Generator[AirbyteStream, None, None]: ...

Client Operations

Google Directory APIs

Direct access to Google Directory API resources including users, groups, and group membership data with pagination and error handling.

class API:
    def __init__(self, credentials: Mapping[str, Any]): ...
    def get(self, name: str, params: Dict = None) -> Dict: ...

class UsersAPI(StreamAPI):
    def list(self, fields: Sequence[str] = None) -> Iterator[dict]: ...

class GroupsAPI(StreamAPI):
    def list(self, fields: Sequence[str] = None) -> Iterator[dict]: ...

class GroupMembersAPI(StreamAPI):
    def list(self, fields: Sequence[str] = None) -> Iterator[dict]: ...

Google Directory APIs

Types

# Core Python types used in API signatures
from typing import Any, Callable, Dict, Generator, Iterator, Mapping, Sequence, Tuple, Union
from abc import ABC, abstractmethod
from functools import partial
import json
import sys

# Google OAuth2 and API types
from google.oauth2.credentials import Credentials
from google.oauth2 import service_account
from google.auth.transport.requests import Request
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError as GoogleApiHttpError

# Airbyte CDK types
from airbyte_cdk.models import AirbyteStream
from airbyte_cdk.sources.deprecated.base_source import BaseSource
from airbyte_cdk.sources.deprecated.client import BaseClient
from airbyte_cdk.entrypoint import launch

# Backoff library for retry logic
import backoff

# Configuration types for credentials
OAuthCredentials = Dict[str, str]  # Contains client_id, client_secret, refresh_token
ServiceAccountCredentials = Dict[str, str]  # Contains credentials_json, email
CredentialsConfig = Dict[str, Any]  # Union of above credential types

Constants

# OAuth scopes required for Google Directory API access
SCOPES = [
    "https://www.googleapis.com/auth/admin.directory.user.readonly",
    "https://www.googleapis.com/auth/admin.directory.group.readonly"
]