0
# UIAutomator2
1

2
A simple, easy-to-use, and stable Android automation library that enables developers to automate Android device interactions through a Python interface. It consists of a device-side HTTP service based on UiAutomator and a Python client that communicates via HTTP protocol.
3

4
## Package Information
5

6
- **Package Name**: uiautomator2
7
- **Package Type**: pypi
8
- **Language**: Python
9
- **Installation**: `pip install uiautomator2`
10

11
## Core Imports
12

13
```python
14
import uiautomator2 as u2
15
```
16

17
Common patterns:
18

19
```python
20
# Connect to device
21
d = u2.connect()  # or u2.connect('device_serial')
22

23
# Create session for app monitoring
24
session = d.session('com.example.app')
25
```
26

27
## Basic Usage
28

29
```python
30
import uiautomator2 as u2
31

32
# Connect to device
33
d = u2.connect('Q5S5T19611004599')  # Use device serial or omit for default
34

35
# Get device information
36
print(d.info)
37

38
# App management
39
d.app_start('tv.danmaku.bili', stop=True)  # Start Bilibili app
40
d.wait_activity('.MainActivityV2')
41

42
# UI interaction via selectors
43
d(text="My", className="android.widget.TextView").click()
44

45
# UI interaction via XPath
46
d.xpath('//*[@text="My"]').click()
47

48
# Screen interaction
49
d.click(100, 200)  # Click at coordinates
50
d.swipe(100, 100, 500, 500)  # Swipe gesture
51

52
# Screenshot
53
screenshot = d.screenshot()
54
screenshot.save("screenshot.png")
55

56
# Text input
57
d.send_keys("Hello World")
58
d.press('enter')
59
```
60

61
## Architecture
62

63
UIAutomator2 consists of two main components:
64

65
- **Device Side**: HTTP service based on Android UiAutomator framework running on the device
66
- **Python Client**: Communicates with device via HTTP protocol, providing Pythonic API
67

68
Key architectural patterns:
69

70
- **Device Class**: Main automation interface with app management, UI interaction, and system control
71
- **Selector System**: Flexible UI element selection using attributes, XPath, or coordinate-based targeting  
72
- **Plugin System**: Extensible architecture with watchers, image processing, and screen recording
73
- **Session Management**: App lifecycle monitoring and automatic recovery
74

75
## Capabilities
76

77
### Device Connection and Management
78

79
Core functionality for connecting to Android devices and managing device state, including device discovery, connection establishment, and device information retrieval.
80

81
```python { .api }
82
def connect(serial: Union[str, adbutils.AdbDevice] = None) -> Device: ...
83
def connect_usb(serial: Optional[str] = None) -> Device: ...
84
def enable_pretty_logging(level=logging.DEBUG): ...
85
```
86

87
[Device Management](./device-management.md)
88

89
### UI Element Selection and Interaction
90

91
Comprehensive UI element selection using UiAutomator selectors, XPath expressions, and coordinate-based targeting. Supports clicking, text input, gestures, and element property inspection.
92

93
```python { .api }
94
class Device:
95
    def __call__(self, **kwargs) -> UiObject: ...
96
    def exists(self, **kwargs): ...
97
    def click(self, x: Union[float, int], y: Union[float, int]): ...
98
    def long_click(self, x, y, duration: float = .5): ...
99
    def double_click(self, x, y, duration=0.1): ...
100
    def swipe(self, fx, fy, tx, ty, duration: Optional[float] = None, steps: Optional[int] = None): ...
101
    def drag(self, sx, sy, ex, ey, duration=0.5): ...
102
```
103

104
[UI Interaction](./ui-interaction.md)
105

106
### XPath-based Element Selection
107

108
Advanced XPath-based element selection providing powerful querying capabilities for complex UI hierarchies and dynamic content.
109

110
```python { .api }
111
class Device:
112
    @cached_property
113
    def xpath(self) -> xpath.XPathEntry: ...
114

115
class XPathEntry:
116
    def __call__(self, xpath_expression: str) -> XPathSelector: ...
117
```
118

119
[XPath Selection](./xpath-selection.md)
120

121
### Application Management
122

123
Complete application lifecycle management including installation, launching, stopping, monitoring, and permission handling for Android applications.
124

125
```python { .api }
126
class Device:
127
    def app_start(self, package_name: str, activity: Optional[str] = None, wait: bool = False, stop: bool = False, use_monkey: bool = False): ...
128
    def app_stop(self, package_name: str): ...
129
    def app_install(self, data: str): ...
130
    def app_uninstall(self, package_name: str) -> bool: ...
131
    def app_info(self, package_name: str) -> Dict[str, Any]: ...
132
    def session(self, package_name: str, attach: bool = False) -> Session: ...
133
```
134

135
[Application Management](./app-management.md)
136

137
### Screen and Input Operations
138

139
Screen capture, device orientation control, hardware key simulation, and text input functionality for comprehensive device interaction.
140

141
```python { .api }
142
class Device:
143
    def screenshot(self, filename: Optional[str] = None, format="pillow", display_id: Optional[int] = None): ...
144
    def dump_hierarchy(self, compressed=False, pretty=False, max_depth: Optional[int] = None) -> str: ...
145
    def press(self, key: Union[int, str], meta=None): ...
146
    def send_keys(self, text: str, clear: bool = False): ...
147
    def orientation(self) -> str: ...
148
```
149

150
[Screen and Input](./screen-input.md)
151

152
### Watchers and Automation
153

154
Automated response system for handling popups, dialogs, and recurring UI events through configurable watchers and monitoring contexts.
155

156
```python { .api }
157
class Device:
158
    @cached_property
159
    def watcher(self) -> Watcher: ...
160
    def watch_context(self, autostart: bool = True, builtin: bool = False) -> WatchContext: ...
161

162
class WatchContext:
163
    def start(self): ...
164
    def stop(self): ...
165
```
166

167
[Watchers and Automation](./watchers-automation.md)
168

169
### Image Processing and Template Matching
170

171
Computer vision capabilities for image-based UI automation, template matching, and visual element detection.
172

173
```python { .api }
174
class Device:
175
    @cached_property
176
    def image(self): ...
177

178
class ImageX:
179
    def click(self, template, **kwargs): ...
180
    def match(self, template, **kwargs): ...
181
```
182

183
### Advanced Touch Control
184

185
Low-level touch gesture builder for complex multi-touch interactions.
186

187
```python { .api }
188
class Device:
189
    @property
190
    def touch(self):
191
        """Advanced touch gesture builder for multi-touch sequences"""
192
        
193
class TouchBuilder:
194
    def down(self, x: float, y: float): """Touch down at coordinates"""
195
    def move(self, x: float, y: float): """Move touch to coordinates"""
196
    def up(self, x: float, y: float): """Touch up at coordinates"""
197
    def sleep(self, seconds: float): """Pause during gesture sequence"""
198
```
199

200
[Advanced Touch Control](./advanced-touch.md)
201

202
### System Integration
203

204
System-level interactions including toast messages, clipboard operations, and shell commands.
205

206
```python { .api }
207
class Device:
208
    def open_notification(self): """Open notification panel"""
209
    def open_quick_settings(self): """Open quick settings panel"""
210
    def open_url(self, url: str): """Open URL in default browser"""
211
    def keyevent(self, key: str): """Send system key event"""
212
    def clear_toast(self): """Clear last toast message"""
213
    def set_clipboard(self, text: str, label: Optional[str] = None): """Set clipboard with optional label"""
214
    @property
215
    def last_toast(self) -> Optional[str]: """Get last toast message"""
216
```
217

218
### Input Method Management
219

220
Control input methods and keyboard functionality for text input automation.
221

222
```python { .api }
223
class Device:
224
    def set_input_ime(self, enable: bool = True): """Enable/disable custom input method"""
225
    def is_input_ime_installed(self) -> bool: """Check if custom IME is installed"""
226
    def current_ime(self) -> str: """Get current input method name"""
227
    def send_action(self, code: Union[str, int]): """Send editor action (done, next, etc.)"""
228
    def hide_keyboard(self): """Hide soft keyboard"""
229
```
230

231
[Image Processing](./image-processing.md)
232

233
## Constants and Enums
234

235
```python { .api }
236
from enum import Enum
237

238
# Direction constants for swipe and scroll operations
239
class Direction(Enum):
240
    LEFT = "left"
241
    RIGHT = "right"
242
    UP = "up"
243
    DOWN = "down"
244
    FORWARD = "up"
245
    BACKWARD = "down"
246
    HORIZ_FORWARD = "left"
247
    HORIZ_BACKWARD = "right"
248

249
# Configuration constants
250
SCROLL_STEPS: int = 55  # Default scroll steps
251
HTTP_TIMEOUT: int = 300  # HTTP timeout in seconds
252
WAIT_FOR_DEVICE_TIMEOUT: int = 20  # Default device wait timeout
253
```
254

255
## Exception Hierarchy
256

257
```python { .api }
258
# Base exceptions
259
class BaseException(Exception): ...
260
class DeviceError(BaseException): ...
261
class RPCError(BaseException): ...
262

263
# Device connection and communication errors
264
class ConnectError(DeviceError): ...
265
class HTTPError(DeviceError): ...
266
class HTTPTimeoutError(HTTPError): ...
267
class AdbShellError(DeviceError): ...
268
class AdbBroadcastError(DeviceError): ...
269

270
# UI automation errors
271
class UiAutomationError(DeviceError): ...
272
class UiAutomationNotConnectedError(UiAutomationError): ...
273
class InjectPermissionError(UiAutomationError): ...
274
class APKSignatureError(UiAutomationError): ...
275
class LaunchUiAutomationError(UiAutomationError): ...
276
class AccessibilityServiceAlreadyRegisteredError(UiAutomationError): ...
277
class InputIMEError(UiAutomationError): ...
278

279
# RPC communication errors
280
class RPCUnknownError(RPCError): ...
281
class RPCInvalidError(RPCError): ...
282
class HierarchyEmptyError(RPCError): ...
283
class RPCStackOverflowError(RPCError): ...
284

285
# Operation errors
286
class NormalError(BaseException): ...
287
class XPathElementNotFoundError(NormalError): ...
288
class SessionBrokenError(NormalError): ...
289
class UiObjectNotFoundError(NormalError): ...
290
class AppNotFoundError(NormalError): ...
291
```
292

293
## Types
294

295
```python { .api }
296
from typing import Any, Dict, List, Optional, Tuple, Union
297
import adbutils
298
from PIL import Image
299

300
# Core device class
301
class Device:
302
    def __init__(self, serial: Union[str, adbutils.AdbDevice] = None): ...
303

304
# UI element selector and interaction
305
class Selector(dict):
306
    def __init__(self, **kwargs): ...
307
    def clone(self): ...
308
    def child(self, **kwargs): ...
309
    def sibling(self, **kwargs): ...
310

311
class UiObject:
312
    def __init__(self, session, selector: Selector): ...
313
    @property
314
    def exists(self): ...
315
    @property
316
    def info(self): ...
317

318
# Session management
319
class Session(Device):
320
    def __init__(self, dev: adbutils.AdbDevice, package_name: str): ...
321
    def running(self) -> bool: ...
322
    def restart(self): ...
323
    def close(self): ...
324

325
# Settings configuration
326
class Settings:
327
    def __init__(self, d): ...
328
    def __getitem__(self, key): ...
329
    def __setitem__(self, key, value): ...
330

331
# XPath support classes
332
class XMLElement:
333
    @property
334
    def tag(self) -> str: """Element tag name"""
335
    @property
336
    def text(self) -> str: """Element text content"""
337
    @property
338
    def attrib(self) -> Dict[str, str]: """Element attributes"""
339
    @property
340
    def bounds(self) -> Tuple[int, int, int, int]: """Element bounds"""
341
    @property
342
    def center(self) -> Tuple[int, int]: """Element center point"""
343
    def get(self, key: str, default: Any = None) -> Any: """Get attribute value"""
344
    def click(self): """Click element"""
345
    def screenshot(self) -> Image.Image: """Take element screenshot"""
346

347
class XPathSelector:
348
    @property
349
    def exists(self) -> bool: """Check if element exists"""
350
    def get(self, timeout: float = 10) -> XMLElement: """Get single element"""
351
    def get_last_match(self) -> XMLElement: """Get last matched element"""
352
    def get_text(self) -> str: """Get element text"""
353
    def set_text(self, text: str): """Set element text"""
354
    def click(self): """Click element"""
355
    def all(self) -> List[XMLElement]: """Get all matching elements"""
356
    def wait(self, timeout: float = 10) -> XMLElement: """Wait for element"""
357

358
class XPathEntry:
359
    def __call__(self, xpath: str, timeout: float = 10) -> XPathSelector: """Select elements by XPath"""
360
    def exists(self, xpath: str) -> bool: """Check if XPath element exists"""
361
    def get(self, xpath: str, timeout: float = 10) -> XMLElement: """Get single XPath element"""
362
    def all(self, xpath: str) -> List[XMLElement]: """Get all XPath elements"""
363
    def wait(self, xpath: str, timeout: float = 10) -> XMLElement: """Wait for XPath element"""
364

365
# Watcher and monitoring classes
366
class Watcher:
367
    def when(self, selector: Dict[str, Any]) -> 'Watcher': """Define watch condition"""
368
    def call(self, callback: callable): """Set callback function"""
369
    def click(self): """Auto-click action"""
370
    def start(self, interval: float = 2.0): """Start monitoring"""
371
    def stop(self): """Stop monitoring"""
372
    def remove_all(self): """Remove all watchers"""
373

374
class WatchContext:
375
    def when(self, xpath: str) -> 'WatchContext': """Chain watch conditions"""
376
    def call(self, fn: callable): """Set callback function"""
377
    def click(self): """Auto-click action"""
378
    def start(self): """Start watch context"""
379
    def stop(self): """Stop watch context"""
380
    def wait_stable(self, seconds: float = 5.0, timeout: float = 60.0): """Wait for UI stability"""
381

382
# Image processing classes
383
class ImageX:
384
    def click(self, template: Union[str, Image.Image], **kwargs): """Click template match"""
385
    def match(self, template: Union[str, Image.Image], **kwargs): """Find template matches"""
386

387
# Advanced swipe extension
388
class SwipeExt:
389
    def __call__(self, direction: str, scale: float = 0.9, box: Optional[Tuple] = None, **kwargs): """Extended swipe with bounds"""
390

391
# Touch gesture builder  
392
class TouchBuilder:
393
    def down(self, x: float, y: float) -> 'TouchBuilder': """Touch down at coordinates"""
394
    def move(self, x: float, y: float) -> 'TouchBuilder': """Move touch to coordinates"""
395
    def up(self, x: float, y: float) -> 'TouchBuilder': """Touch up at coordinates"""
396
    def sleep(self, seconds: float) -> 'TouchBuilder': """Pause during gesture"""
397
```