CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-uiautomation

Python UIAutomation for Windows - comprehensive library for automating Windows applications using Microsoft's UIAutomation framework

Overview
Eval results
Files

automation-patterns.mddocs/

Automation Patterns

Implementation of Windows UI Automation patterns for specialized control operations. Patterns provide standardized interfaces for common control behaviors like invoking buttons, setting values, handling selections, and managing scrollable content.

Capabilities

Invoke Pattern

The Invoke pattern enables triggering of controls that perform a single action when activated.

class InvokePattern:
    """Pattern for controls that can be invoked (clicked/activated)."""
    
    def Invoke(self) -> None:
        """
        Invoke the control (equivalent to clicking).
        Typically used for buttons, menu items, and hyperlinks.
        """

Value Pattern

The Value pattern provides access to controls that have a string value that can be read or set.

class ValuePattern:
    """Pattern for controls with settable string values."""
    
    def SetValue(self, value: str) -> None:
        """
        Set the value of the control.
        
        Args:
            value: String value to set
        """
    
    def GetValue(self) -> str:
        """
        Get the current value of the control.
        
        Returns:
            str: Current string value of the control
        """
    
    @property
    def IsReadOnly(self) -> bool:
        """Whether the control's value is read-only."""

Toggle Pattern

The Toggle pattern supports controls that can cycle through a set of states.

class TogglePattern:
    """Pattern for controls with toggle states (checkboxes, toggle buttons)."""
    
    def Toggle(self) -> None:
        """Toggle the control to its next state."""
    
    @property
    def ToggleState(self) -> int:
        """
        Current toggle state.
        
        Returns:
            int: ToggleState constant (Off=0, On=1, Indeterminate=2)
        """

class ToggleState:
    """Constants for toggle states."""
    Off: int = 0
    On: int = 1  
    Indeterminate: int = 2

Range Value Pattern

The Range Value pattern provides access to controls that represent a value within a range.

class RangeValuePattern:
    """Pattern for controls with numeric ranges (sliders, progress bars)."""
    
    def SetValue(self, value: float) -> None:
        """
        Set the numeric value.
        
        Args:
            value: Numeric value within the valid range
        """
    
    @property
    def Value(self) -> float:
        """Current numeric value."""
    
    @property
    def Minimum(self) -> float:
        """Minimum allowed value."""
    
    @property
    def Maximum(self) -> float:
        """Maximum allowed value."""
    
    @property
    def SmallChange(self) -> float:
        """Small increment/decrement amount."""
    
    @property
    def LargeChange(self) -> float:
        """Large increment/decrement amount."""
    
    @property
    def IsReadOnly(self) -> bool:
        """Whether the value is read-only."""

Selection Pattern

The Selection pattern supports controls that act as containers for selectable child items.

class SelectionPattern:
    """Pattern for selection containers (lists, combo boxes)."""
    
    def GetSelection(self) -> list:
        """
        Get currently selected items.
        
        Returns:
            list: List of selected control elements
        """
    
    @property
    def CanSelectMultiple(self) -> bool:
        """Whether multiple items can be selected."""
    
    @property
    def IsSelectionRequired(self) -> bool:
        """Whether at least one item must be selected."""

Selection Item Pattern

The Selection Item pattern supports individual items that can be selected within a selection container.

class SelectionItemPattern:
    """Pattern for selectable items within selection containers."""
    
    def Select(self) -> None:
        """Select this item (deselecting others if single-select)."""
    
    def AddToSelection(self) -> None:
        """Add this item to the selection (multi-select containers)."""
    
    def RemoveFromSelection(self) -> None:
        """Remove this item from the selection."""
    
    @property
    def IsSelected(self) -> bool:
        """Whether this item is currently selected."""
    
    @property
    def SelectionContainer(self):
        """The container control that manages selection."""

Scroll Pattern

The Scroll pattern provides scrolling functionality for controls with scrollable content.

class ScrollPattern:
    """Pattern for scrollable controls."""
    
    def Scroll(self, horizontalAmount: int, verticalAmount: int) -> None:
        """
        Scroll by specified amounts.
        
        Args:
            horizontalAmount: Horizontal scroll amount
            verticalAmount: Vertical scroll amount
        """
    
    def SetScrollPercent(self, horizontalPercent: float, verticalPercent: float) -> None:
        """
        Set scroll position by percentage.
        
        Args:
            horizontalPercent: Horizontal position (0-100)
            verticalPercent: Vertical position (0-100)
        """
    
    @property
    def HorizontalScrollPercent(self) -> float:
        """Current horizontal scroll percentage."""
    
    @property
    def VerticalScrollPercent(self) -> float:
        """Current vertical scroll percentage."""
    
    @property
    def HorizontalViewSize(self) -> float:
        """Horizontal view size as percentage of total."""
    
    @property
    def VerticalViewSize(self) -> float:
        """Vertical view size as percentage of total."""
    
    @property
    def HorizontallyScrollable(self) -> bool:
        """Whether horizontal scrolling is available."""
    
    @property
    def VerticallyScrollable(self) -> bool:
        """Whether vertical scrolling is available."""

Scroll Item Pattern

The Scroll Item pattern enables individual items to scroll themselves into view.

class ScrollItemPattern:
    """Pattern for items that can scroll themselves into view."""
    
    def ScrollIntoView(self) -> None:
        """Scroll this item into the visible area of its container."""

Expand Collapse Pattern

The Expand Collapse pattern supports controls that can expand to show more content or collapse to hide content.

class ExpandCollapsePattern:
    """Pattern for expandable/collapsible controls (tree nodes, menus)."""
    
    def Expand(self) -> None:
        """Expand the control to show child items."""
    
    def Collapse(self) -> None:
        """Collapse the control to hide child items."""
    
    @property
    def ExpandCollapseState(self) -> int:
        """
        Current expand/collapse state.
        
        Returns:
            int: ExpandCollapseState constant
        """

class ExpandCollapseState:
    """Constants for expand/collapse states."""
    Collapsed: int = 0
    Expanded: int = 1
    PartiallyExpanded: int = 2
    LeafNode: int = 3

Grid Pattern

The Grid pattern provides access to controls that act as containers for a collection of child elements organized in a grid.

class GridPattern:
    """Pattern for grid controls (data grids, tables)."""
    
    def GetItem(self, row: int, column: int):
        """
        Get the control at the specified grid coordinates.
        
        Args:
            row: Zero-based row index
            column: Zero-based column index
            
        Returns:
            Control: The control at the specified position
        """
    
    @property
    def RowCount(self) -> int:
        """Number of rows in the grid."""
    
    @property
    def ColumnCount(self) -> int:
        """Number of columns in the grid."""

Grid Item Pattern

The Grid Item pattern provides information about individual items within a grid.

class GridItemPattern:
    """Pattern for individual items within grid controls."""
    
    @property
    def Row(self) -> int:
        """Zero-based row index of this item."""
    
    @property
    def Column(self) -> int:
        """Zero-based column index of this item."""
    
    @property
    def RowSpan(self) -> int:
        """Number of rows spanned by this item."""
    
    @property
    def ColumnSpan(self) -> int:
        """Number of columns spanned by this item."""
    
    @property
    def ContainingGrid(self):
        """The grid control containing this item."""

Window Pattern

The Window pattern provides access to window-specific functionality.

class WindowPattern:
    """Pattern for window controls."""
    
    def Close(self) -> None:
        """Close the window."""
    
    def SetWindowVisualState(self, state: int) -> None:
        """
        Set the window's visual state.
        
        Args:
            state: WindowVisualState constant
        """
    
    def WaitForInputIdle(self, timeout: int) -> bool:
        """
        Wait for the window to be ready for user input.
        
        Args:
            timeout: Timeout in milliseconds
            
        Returns:
            bool: True if window became ready, False if timeout
        """
    
    @property
    def WindowVisualState(self) -> int:
        """Current window visual state."""
    
    @property
    def WindowInteractionState(self) -> int:
        """Current window interaction state."""
    
    @property
    def IsModal(self) -> bool:
        """Whether the window is modal."""
    
    @property
    def IsTopmost(self) -> bool:
        """Whether the window is topmost."""
    
    @property
    def Maximizable(self) -> bool:
        """Whether the window can be maximized."""
    
    @property
    def Minimizable(self) -> bool:
        """Whether the window can be minimized."""

class WindowVisualState:
    """Constants for window visual states."""
    Normal: int = 0
    Maximized: int = 1
    Minimized: int = 2

Text Pattern

The Text pattern provides access to text content and formatting information.

class TextPattern:
    """Pattern for controls containing text content."""
    
    def GetText(self, maxLength: int = -1) -> str:
        """
        Get text content.
        
        Args:
            maxLength: Maximum length to retrieve (-1 for all)
            
        Returns:
            str: Text content
        """
    
    def GetSelection(self) -> list:
        """
        Get selected text ranges.
        
        Returns:
            list: List of selected text range objects
        """
    
    def GetVisibleRanges(self) -> list:
        """
        Get visible text ranges.
        
        Returns:
            list: List of visible text range objects
        """

Usage Examples

Using Invoke Pattern

import uiautomation

# Get button and invoke it using pattern
button = uiautomation.ButtonControl(Name='Submit')
invoke_pattern = button.GetInvokePattern()
if invoke_pattern:
    invoke_pattern.Invoke()

Using Value Pattern

# Set value in a text box
text_box = uiautomation.EditControl(Name='Username')
value_pattern = text_box.GetValuePattern()
if value_pattern and not value_pattern.IsReadOnly:
    value_pattern.SetValue('myusername')

# Get current value
current_value = value_pattern.GetValue()
print(f"Current value: {current_value}")

Using Selection Patterns

# Work with a list box
list_box = uiautomation.ListControl(Name='Items')
selection_pattern = list_box.GetSelectionPattern()

# Get current selection
selected_items = selection_pattern.GetSelection()
for item in selected_items:
    print(f"Selected: {item.Name}")

# Select an item
list_item = list_box.ListItemControl(Name='Item 3')
selection_item_pattern = list_item.GetSelectionItemPattern()
if selection_item_pattern:
    selection_item_pattern.Select()

Using Toggle Pattern

# Toggle a checkbox
checkbox = uiautomation.CheckBoxControl(Name='Enable notifications')
toggle_pattern = checkbox.GetTogglePattern()
if toggle_pattern:
    current_state = toggle_pattern.ToggleState
    if current_state == uiautomation.ToggleState.Off:
        toggle_pattern.Toggle()  # Turn on

Using Range Value Pattern

# Set slider value
slider = uiautomation.SliderControl(Name='Volume')
range_pattern = slider.GetRangeValuePattern()
if range_pattern and not range_pattern.IsReadOnly:
    # Set to 75% of the range
    range_size = range_pattern.Maximum - range_pattern.Minimum
    target_value = range_pattern.Minimum + (range_size * 0.75)
    range_pattern.SetValue(target_value)

Using Scroll Pattern

# Scroll in a scrollable area
text_area = uiautomation.EditControl(Name='Content')
scroll_pattern = text_area.GetScrollPattern()
if scroll_pattern and scroll_pattern.VerticallyScrollable:
    # Scroll to bottom
    scroll_pattern.SetScrollPercent(-1, 100)  # -1 means no change for horizontal

Using Expand Collapse Pattern

# Expand a tree node
tree = uiautomation.TreeControl(Name='Directory')
folder_node = tree.TreeItemControl(Name='Documents')
expand_pattern = folder_node.GetExpandCollapsePattern()
if expand_pattern:
    if expand_pattern.ExpandCollapseState == uiautomation.ExpandCollapseState.Collapsed:
        expand_pattern.Expand()

Using Grid Pattern

# Access grid data
data_grid = uiautomation.DataGridControl(Name='Results')
grid_pattern = data_grid.GetGridPattern()
if grid_pattern:
    # Get cell at row 2, column 1
    cell = grid_pattern.GetItem(2, 1)
    if cell:
        print(f"Cell value: {cell.Name}")
    
    print(f"Grid size: {grid_pattern.RowCount} x {grid_pattern.ColumnCount}")

Using Window Pattern

# Manage window state
window = uiautomation.WindowControl(Name='My Application')
window_pattern = window.GetWindowPattern()
if window_pattern:
    # Maximize the window
    window_pattern.SetWindowVisualState(uiautomation.WindowVisualState.Maximized)
    
    # Wait for window to be ready
    ready = window_pattern.WaitForInputIdle(5000)  # 5 second timeout
    if ready:
        print("Window is ready for input")

Pattern Availability Checking

def safe_invoke_button(button_name):
    """Safely invoke a button using pattern if available."""
    button = uiautomation.ButtonControl(Name=button_name)
    if not button.Exists():
        print(f"Button '{button_name}' not found")
        return False
    
    # Try using invoke pattern first
    invoke_pattern = button.GetInvokePattern()
    if invoke_pattern:
        invoke_pattern.Invoke()
        return True
    
    # Fall back to regular click
    button.Click()
    return True

Install with Tessl CLI

npx tessl i tessl/pypi-uiautomation

docs

automation-patterns.md

control-management.md

control-types.md

index.md

input-simulation.md

logging-debugging.md

screen-capture.md

windows-api.md

tile.json