or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

app-management.mddevice-management.mdimage-processing.mdindex.mdscreen-input.mdui-interaction.mdwatchers-automation.mdxpath-selection.md
tile.json

tessl/pypi-uiautomator2

A simple, easy-to-use, and stable Android automation library

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/uiautomator2@3.2.x

To install, run

npx @tessl/cli install tessl/pypi-uiautomator2@3.2.0

index.mddocs/

UIAutomator2

A simple, easy-to-use, and stable Android automation library that enables developers to automate Android device interactions through a Python interface. It consists of a device-side HTTP service based on UiAutomator and a Python client that communicates via HTTP protocol.

Package Information

  • Package Name: uiautomator2
  • Package Type: pypi
  • Language: Python
  • Installation: pip install uiautomator2

Core Imports

import uiautomator2 as u2

Common patterns:

# Connect to device
d = u2.connect()  # or u2.connect('device_serial')

# Create session for app monitoring
session = d.session('com.example.app')

Basic Usage

import uiautomator2 as u2

# Connect to device
d = u2.connect('Q5S5T19611004599')  # Use device serial or omit for default

# Get device information
print(d.info)

# App management
d.app_start('tv.danmaku.bili', stop=True)  # Start Bilibili app
d.wait_activity('.MainActivityV2')

# UI interaction via selectors
d(text="My", className="android.widget.TextView").click()

# UI interaction via XPath
d.xpath('//*[@text="My"]').click()

# Screen interaction
d.click(100, 200)  # Click at coordinates
d.swipe(100, 100, 500, 500)  # Swipe gesture

# Screenshot
screenshot = d.screenshot()
screenshot.save("screenshot.png")

# Text input
d.send_keys("Hello World")
d.press('enter')

Architecture

UIAutomator2 consists of two main components:

  • Device Side: HTTP service based on Android UiAutomator framework running on the device
  • Python Client: Communicates with device via HTTP protocol, providing Pythonic API

Key architectural patterns:

  • Device Class: Main automation interface with app management, UI interaction, and system control
  • Selector System: Flexible UI element selection using attributes, XPath, or coordinate-based targeting
  • Plugin System: Extensible architecture with watchers, image processing, and screen recording
  • Session Management: App lifecycle monitoring and automatic recovery

Capabilities

Device Connection and Management

Core functionality for connecting to Android devices and managing device state, including device discovery, connection establishment, and device information retrieval.

def connect(serial: Union[str, adbutils.AdbDevice] = None) -> Device: ...
def connect_usb(serial: Optional[str] = None) -> Device: ...
def enable_pretty_logging(level=logging.DEBUG): ...

Device Management

UI Element Selection and Interaction

Comprehensive UI element selection using UiAutomator selectors, XPath expressions, and coordinate-based targeting. Supports clicking, text input, gestures, and element property inspection.

class Device:
    def __call__(self, **kwargs) -> UiObject: ...
    def exists(self, **kwargs): ...
    def click(self, x: Union[float, int], y: Union[float, int]): ...
    def long_click(self, x, y, duration: float = .5): ...
    def double_click(self, x, y, duration=0.1): ...
    def swipe(self, fx, fy, tx, ty, duration: Optional[float] = None, steps: Optional[int] = None): ...
    def drag(self, sx, sy, ex, ey, duration=0.5): ...

UI Interaction

XPath-based Element Selection

Advanced XPath-based element selection providing powerful querying capabilities for complex UI hierarchies and dynamic content.

class Device:
    @cached_property
    def xpath(self) -> xpath.XPathEntry: ...

class XPathEntry:
    def __call__(self, xpath_expression: str) -> XPathSelector: ...

XPath Selection

Application Management

Complete application lifecycle management including installation, launching, stopping, monitoring, and permission handling for Android applications.

class Device:
    def app_start(self, package_name: str, activity: Optional[str] = None, wait: bool = False, stop: bool = False, use_monkey: bool = False): ...
    def app_stop(self, package_name: str): ...
    def app_install(self, data: str): ...
    def app_uninstall(self, package_name: str) -> bool: ...
    def app_info(self, package_name: str) -> Dict[str, Any]: ...
    def session(self, package_name: str, attach: bool = False) -> Session: ...

Application Management

Screen and Input Operations

Screen capture, device orientation control, hardware key simulation, and text input functionality for comprehensive device interaction.

class Device:
    def screenshot(self, filename: Optional[str] = None, format="pillow", display_id: Optional[int] = None): ...
    def dump_hierarchy(self, compressed=False, pretty=False, max_depth: Optional[int] = None) -> str: ...
    def press(self, key: Union[int, str], meta=None): ...
    def send_keys(self, text: str, clear: bool = False): ...
    def orientation(self) -> str: ...

Screen and Input

Watchers and Automation

Automated response system for handling popups, dialogs, and recurring UI events through configurable watchers and monitoring contexts.

class Device:
    @cached_property
    def watcher(self) -> Watcher: ...
    def watch_context(self, autostart: bool = True, builtin: bool = False) -> WatchContext: ...

class WatchContext:
    def start(self): ...
    def stop(self): ...

Watchers and Automation

Image Processing and Template Matching

Computer vision capabilities for image-based UI automation, template matching, and visual element detection.

class Device:
    @cached_property
    def image(self): ...

class ImageX:
    def click(self, template, **kwargs): ...
    def match(self, template, **kwargs): ...

Advanced Touch Control

Low-level touch gesture builder for complex multi-touch interactions.

class Device:
    @property
    def touch(self):
        """Advanced touch gesture builder for multi-touch sequences"""
        
class TouchBuilder:
    def down(self, x: float, y: float): """Touch down at coordinates"""
    def move(self, x: float, y: float): """Move touch to coordinates"""
    def up(self, x: float, y: float): """Touch up at coordinates"""
    def sleep(self, seconds: float): """Pause during gesture sequence"""

Advanced Touch Control

System Integration

System-level interactions including toast messages, clipboard operations, and shell commands.

class Device:
    def open_notification(self): """Open notification panel"""
    def open_quick_settings(self): """Open quick settings panel"""
    def open_url(self, url: str): """Open URL in default browser"""
    def keyevent(self, key: str): """Send system key event"""
    def clear_toast(self): """Clear last toast message"""
    def set_clipboard(self, text: str, label: Optional[str] = None): """Set clipboard with optional label"""
    @property
    def last_toast(self) -> Optional[str]: """Get last toast message"""

Input Method Management

Control input methods and keyboard functionality for text input automation.

class Device:
    def set_input_ime(self, enable: bool = True): """Enable/disable custom input method"""
    def is_input_ime_installed(self) -> bool: """Check if custom IME is installed"""
    def current_ime(self) -> str: """Get current input method name"""
    def send_action(self, code: Union[str, int]): """Send editor action (done, next, etc.)"""
    def hide_keyboard(self): """Hide soft keyboard"""

Image Processing

Constants and Enums

from enum import Enum

# Direction constants for swipe and scroll operations
class Direction(Enum):
    LEFT = "left"
    RIGHT = "right"
    UP = "up"
    DOWN = "down"
    FORWARD = "up"
    BACKWARD = "down"
    HORIZ_FORWARD = "left"
    HORIZ_BACKWARD = "right"

# Configuration constants
SCROLL_STEPS: int = 55  # Default scroll steps
HTTP_TIMEOUT: int = 300  # HTTP timeout in seconds
WAIT_FOR_DEVICE_TIMEOUT: int = 20  # Default device wait timeout

Exception Hierarchy

# Base exceptions
class BaseException(Exception): ...
class DeviceError(BaseException): ...
class RPCError(BaseException): ...

# Device connection and communication errors
class ConnectError(DeviceError): ...
class HTTPError(DeviceError): ...
class HTTPTimeoutError(HTTPError): ...
class AdbShellError(DeviceError): ...
class AdbBroadcastError(DeviceError): ...

# UI automation errors
class UiAutomationError(DeviceError): ...
class UiAutomationNotConnectedError(UiAutomationError): ...
class InjectPermissionError(UiAutomationError): ...
class APKSignatureError(UiAutomationError): ...
class LaunchUiAutomationError(UiAutomationError): ...
class AccessibilityServiceAlreadyRegisteredError(UiAutomationError): ...
class InputIMEError(UiAutomationError): ...

# RPC communication errors
class RPCUnknownError(RPCError): ...
class RPCInvalidError(RPCError): ...
class HierarchyEmptyError(RPCError): ...
class RPCStackOverflowError(RPCError): ...

# Operation errors
class NormalError(BaseException): ...
class XPathElementNotFoundError(NormalError): ...
class SessionBrokenError(NormalError): ...
class UiObjectNotFoundError(NormalError): ...
class AppNotFoundError(NormalError): ...

Types

from typing import Any, Dict, List, Optional, Tuple, Union
import adbutils
from PIL import Image

# Core device class
class Device:
    def __init__(self, serial: Union[str, adbutils.AdbDevice] = None): ...

# UI element selector and interaction
class Selector(dict):
    def __init__(self, **kwargs): ...
    def clone(self): ...
    def child(self, **kwargs): ...
    def sibling(self, **kwargs): ...

class UiObject:
    def __init__(self, session, selector: Selector): ...
    @property
    def exists(self): ...
    @property
    def info(self): ...

# Session management
class Session(Device):
    def __init__(self, dev: adbutils.AdbDevice, package_name: str): ...
    def running(self) -> bool: ...
    def restart(self): ...
    def close(self): ...

# Settings configuration
class Settings:
    def __init__(self, d): ...
    def __getitem__(self, key): ...
    def __setitem__(self, key, value): ...

# XPath support classes
class XMLElement:
    @property
    def tag(self) -> str: """Element tag name"""
    @property
    def text(self) -> str: """Element text content"""
    @property
    def attrib(self) -> Dict[str, str]: """Element attributes"""
    @property
    def bounds(self) -> Tuple[int, int, int, int]: """Element bounds"""
    @property
    def center(self) -> Tuple[int, int]: """Element center point"""
    def get(self, key: str, default: Any = None) -> Any: """Get attribute value"""
    def click(self): """Click element"""
    def screenshot(self) -> Image.Image: """Take element screenshot"""

class XPathSelector:
    @property
    def exists(self) -> bool: """Check if element exists"""
    def get(self, timeout: float = 10) -> XMLElement: """Get single element"""
    def get_last_match(self) -> XMLElement: """Get last matched element"""
    def get_text(self) -> str: """Get element text"""
    def set_text(self, text: str): """Set element text"""
    def click(self): """Click element"""
    def all(self) -> List[XMLElement]: """Get all matching elements"""
    def wait(self, timeout: float = 10) -> XMLElement: """Wait for element"""

class XPathEntry:
    def __call__(self, xpath: str, timeout: float = 10) -> XPathSelector: """Select elements by XPath"""
    def exists(self, xpath: str) -> bool: """Check if XPath element exists"""
    def get(self, xpath: str, timeout: float = 10) -> XMLElement: """Get single XPath element"""
    def all(self, xpath: str) -> List[XMLElement]: """Get all XPath elements"""
    def wait(self, xpath: str, timeout: float = 10) -> XMLElement: """Wait for XPath element"""

# Watcher and monitoring classes
class Watcher:
    def when(self, selector: Dict[str, Any]) -> 'Watcher': """Define watch condition"""
    def call(self, callback: callable): """Set callback function"""
    def click(self): """Auto-click action"""
    def start(self, interval: float = 2.0): """Start monitoring"""
    def stop(self): """Stop monitoring"""
    def remove_all(self): """Remove all watchers"""

class WatchContext:
    def when(self, xpath: str) -> 'WatchContext': """Chain watch conditions"""
    def call(self, fn: callable): """Set callback function"""
    def click(self): """Auto-click action"""
    def start(self): """Start watch context"""
    def stop(self): """Stop watch context"""
    def wait_stable(self, seconds: float = 5.0, timeout: float = 60.0): """Wait for UI stability"""

# Image processing classes
class ImageX:
    def click(self, template: Union[str, Image.Image], **kwargs): """Click template match"""
    def match(self, template: Union[str, Image.Image], **kwargs): """Find template matches"""

# Advanced swipe extension
class SwipeExt:
    def __call__(self, direction: str, scale: float = 0.9, box: Optional[Tuple] = None, **kwargs): """Extended swipe with bounds"""

# Touch gesture builder  
class TouchBuilder:
    def down(self, x: float, y: float) -> 'TouchBuilder': """Touch down at coordinates"""
    def move(self, x: float, y: float) -> 'TouchBuilder': """Move touch to coordinates"""
    def up(self, x: float, y: float) -> 'TouchBuilder': """Touch up at coordinates"""
    def sleep(self, seconds: float) -> 'TouchBuilder': """Pause during gesture"""