0
# UIAutomation
1

2
Python UIAutomation for Windows provides comprehensive automation capabilities for Windows applications using Microsoft's UIAutomation framework. It enables Python developers to interact with and control Windows applications built with various UI frameworks including MFC, Windows Forms, WPF, Modern UI, Qt (partial support), Firefox, and Chrome.
3

4
## Package Information
5

6
- **Package Name**: uiautomation
7
- **Language**: Python
8
- **Installation**: `pip install uiautomation`
9
- **Supported Platforms**: Windows XP SP3+, Windows Vista SP2+, Windows 7+, Windows 8/8.1/10
10
- **Python Versions**: Python 2.7, Python 3.4+
11
- **License**: Apache 2.0
12

13
## Core Imports
14

15
```python
16
import uiautomation
17
```
18

19
All public API is available through the main module import:
20

21
```python
22
# Control classes are available directly
23
window = uiautomation.WindowControl()
24
button = uiautomation.ButtonControl()
25

26
# Global functions are available
27
root = uiautomation.GetRootControl()
28
focused = uiautomation.GetFocusedControl()
29
```
30

31
## Basic Usage
32

33
```python
34
import uiautomation
35

36
# Find and interact with a window
37
calculator = uiautomation.WindowControl(searchDepth=1, Name='Calculator')
38

39
# Find a button within the window and click it
40
button7 = calculator.ButtonControl(Name='Seven')
41
button7.Click()
42

43
# Type into an edit control
44
edit = calculator.EditControl(ClassName='Edit')
45
edit.SendKeys('123')
46

47
# Get control properties
48
print(f"Button name: {button7.Name}")
49
print(f"Button bounds: {button7.BoundingRectangle}")
50
print(f"Is enabled: {button7.IsEnabled}")
51

52
# Wait for controls to appear or disappear
53
uiautomation.WaitForExist(button7, timeout=5)
54
```
55

56
## Architecture
57

58
UIAutomation is built around the Windows UI Automation framework and provides a Python interface with several key components:
59

60
- **Control Hierarchy**: Tree-based structure representing UI elements from desktop root to individual controls
61
- **Control Types**: 40+ specialized control classes for different UI element types (Button, Edit, List, etc.)
62
- **Patterns**: UI Automation patterns for specialized operations (Invoke, Value, Selection, etc.)
63
- **Search System**: Flexible control location using property matching with depth control for performance
64
- **Input Simulation**: Mouse and keyboard input simulation through Windows APIs
65
- **Screen Capture**: Bitmap operations for visual verification and image-based automation
66

67
The library follows a consistent API pattern where controls are located using named parameters matching their Windows properties, then manipulated through methods or patterns.
68

69
## Capabilities
70

71
### Control Location and Management
72

73
Core functionality for finding, accessing, and managing UI controls within the Windows control tree. Provides the foundation for all automation operations.
74

75
```python { .api }
76
def GetRootControl() -> Control: ...
77
def GetFocusedControl() -> Control: ...
78
def GetForegroundControl() -> Control: ...
79
def ControlFromPoint(x: int, y: int) -> Control: ...
80
def ControlFromHandle(handle: int) -> Control: ...
81
def FindControl(root: Control, **kwargs) -> Control: ...
82
```
83

84
[Control Management](./control-management.md)
85

86
### Control Classes and Types
87

88
Comprehensive set of 40+ control classes representing all Windows UI element types, from basic buttons and text fields to complex data grids and tree views.
89

90
```python { .api }
91
class Control:
92
    def Click(self) -> None: ...
93
    def SendKeys(self, keys: str) -> None: ...
94
    def SetFocus(self) -> None: ...
95
    
96
class WindowControl(Control): ...
97
class ButtonControl(Control): ...
98
class EditControl(Control): ...
99
class ListControl(Control): ...
100
```
101

102
[Control Types](./control-types.md)
103

104
### Input Simulation
105

106
Mouse and keyboard input simulation for automating user interactions with applications. Supports clicking, typing, dragging, and special key combinations.
107

108
```python { .api }
109
def Click(x: int, y: int) -> None: ...
110
def SendKey(key: int, modifiers: int = 0) -> None: ...
111
def SendKeys(keys: str) -> None: ...
112
def DragDrop(x1: int, y1: int, x2: int, y2: int) -> None: ...
113
```
114

115
[Input Simulation](./input-simulation.md)
116

117
### UI Automation Patterns
118

119
Implementation of Windows UI Automation patterns for specialized control operations like invoking buttons, setting values, handling selections, and managing scrollable content.
120

121
```python { .api }
122
class InvokePattern:
123
    def Invoke(self) -> None: ...
124

125
class ValuePattern:
126
    def SetValue(self, value: str) -> None: ...
127
    def GetValue(self) -> str: ...
128

129
class SelectionPattern:
130
    def GetSelection(self) -> list: ...
131
```
132

133
[Automation Patterns](./automation-patterns.md)
134

135
### Screen Capture and Imaging
136

137
Bitmap operations for capturing screenshots of controls or screen areas, pixel color analysis, and image-based verification in automation scripts.
138

139
```python { .api }
140
class Bitmap:
141
    @staticmethod
142
    def FromControl(control: Control) -> Bitmap: ...
143
    def ToFile(self, filename: str) -> None: ...
144
    def GetPixelColor(self, x: int, y: int) -> int: ...
145
```
146

147
[Screen Capture](./screen-capture.md)
148

149
### Windows API Integration
150

151
Comprehensive Windows API wrapper providing low-level system operations for window management, process control, and system utilities.
152

153
```python { .api }
154
class Win32API:
155
    @staticmethod
156
    def SetForegroundWindow(handle: int) -> bool: ...
157
    @staticmethod
158
    def ShowWindow(handle: int, state: int) -> bool: ...
159
    @staticmethod
160
    def GetWindowText(handle: int) -> str: ...
161
```
162

163
[Windows API](./windows-api.md)
164

165
### Logging and Debugging
166

167
Comprehensive logging system with color support and control tree enumeration for debugging automation scripts and understanding application structure.
168

169
```python { .api }
170
class Logger:
171
    def WriteLine(self, text: str) -> None: ...
172
    def ColorfulWriteLine(self, text: str, color: int) -> None: ...
173

174
def LogControl(control: Control) -> None: ...
175
def EnumAndLogControl(control: Control, depth: int) -> None: ...
176
```
177

178
[Logging and Debugging](./logging-debugging.md)
179

180
## Error Handling
181

182
Common exceptions that may be raised:
183

184
- `RuntimeError`: "Can not get an instance of IUIAutomation" - requires Windows update KB971513 for Windows XP
185
- Control not found errors when search criteria don't match existing controls  
186
- Access denied errors when running without administrator privileges
187
- Timeout errors when controls don't appear within specified time limits
188

189
## Requirements and Setup
190

191
- **Administrator privileges recommended** for complete access to all applications
192
- **Windows update KB971513** required for Windows XP users
193
- **Chrome automation** requires `--force-renderer-accessibility` command line flag
194
- **Firefox automation** supports versions ≤56 or ≥60
195
- **Metro app automation** on Windows 8/8.1 requires apps to be in foreground
196

197
## Types
198

199
```python { .api }
200
# Version Information
201
VERSION: str  # Library version string (e.g., "1.1.15")
202

203
# Constants
204
OPERATION_WAIT_TIME: float  # Default wait time after operations (0.5 seconds)
205

206
class Point:
207
    def __init__(self, x: int, y: int): ...
208
    x: int
209
    y: int
210

211
class Rect:
212
    def __init__(self, left: int, top: int, right: int, bottom: int): ...
213
    left: int
214
    top: int
215
    right: int
216
    bottom: int
217

218
class Control:
219
    Name: str
220
    ControlType: int
221
    ClassName: str
222
    AutomationId: str
223
    ProcessId: int
224
    IsEnabled: bool
225
    HasKeyboardFocus: bool
226
    IsKeyboardFocusable: bool
227
    IsOffScreen: bool
228
    BoundingRectangle: Rect
229
    Handle: int
230
```