testland/desktop-test-strategy-reference
Pure-reference catalog of desktop GUI test strategies across Windows, macOS, and Linux. Defines the three accessibility-tree backends (Microsoft UI Automation on Windows, Apple Accessibility / XCTest on macOS, AT-SPI on Linux), the wrapper-tools that drive each backend (WinAppDriver, Appium-Windows, XCUIApplication, AT-SPI clients), the cross-toolkit Electron + Qt paths, and a per-OS decision matrix with accessibility-first locator strategy. Deep operational detail (per-OS asynchronous-wait hierarchies, parallel-test policy, foreground-lock / UAC / TCC / AT-SPI elevation hazards, and the high-DPI / per-monitor test matrix) lives in references/. Use as the strategic reference before picking a desktop test stack, ahead of the per-tool implementation skills.
74
93%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Passed
No findings from the security scan
SKILL.md
- name:
- desktop-test-strategy-reference
- description:
- Pure-reference catalog of desktop GUI test strategies across Windows, macOS, and Linux. Defines the three accessibility-tree backends (Microsoft UI Automation on Windows, Apple Accessibility / XCTest on macOS, AT-SPI on Linux), the wrapper-tools that drive each backend (WinAppDriver, Appium-Windows, XCUIApplication, AT-SPI clients), the cross-toolkit Electron + Qt paths, and a per-OS decision matrix with accessibility-first locator strategy. Deep operational detail (per-OS asynchronous-wait hierarchies, parallel-test policy, foreground-lock / UAC / TCC / AT-SPI elevation hazards, and the high-DPI / per-monitor test matrix) lives in references/. Use when choosing how to test or automate a desktop GUI application (desktop app testing, GUI automation, automate desktop UI) on Windows, macOS, or Linux - the strategic reference before picking a desktop test stack, ahead of the per-tool implementation skills.
- metadata:
- {"keywords":"desktop, ui-automation, xctest, at-spi, electron, qt"}
desktop-test-strategy-reference
Overview
Desktop GUI testing is fragmented along two axes: operating system (Windows / macOS / Linux) and UI toolkit (native, Electron, Qt, GTK, Cocoa, WPF, WinUI, Win32). Unlike the web - where a single DOM-driving tool (Playwright / Selenium) covers every browser - each desktop OS exposes its own accessibility tree, and that tree is the substrate every reliable desktop test driver uses.
Per Microsoft Learn - UI Automation Win32 overview:
"Microsoft UI Automation is an accessibility framework that enables Windows applications to provide and consume programmatic information about user interfaces (UIs). … UI Automation also allows automated test scripts to interact with the UI."
Per Apple - Testing with Xcode, UI Testing chapter:
"UI Testing rests upon two core technologies: the XCTest framework and Accessibility."
Per the at-spi2-core project:
"Base DBus XML interfaces for accessibility, the accessibility registry daemon, and atspi library."
This skill is a pure reference consumed by the per-tool skills below and by anyone choosing a desktop test stack.
How to use this reference
- Pick OS + toolkit from the per-OS / per-toolkit decision matrix.
- Choose stable locators for that backend from the locator-strategy table.
- Apply the matching async-wait primitive for the OS (XCTest waiters / FlaUI Retry / AT-SPI polling) - see references/async-waits-and-concurrency.md.
- Pre-handle the OS elevation hazard (UAC / TCC / AT-SPI) before it blocks the run - see references/platform-hazards-and-dpi.md.
When to use
- Starting a new desktop test program - which OS + toolkit needs which driver?
- Auditing an existing desktop test suite - is the right backend in use, or is the suite working around a wrong-tool fit?
- Cross-platform desktop app (Electron, Qt, .NET MAUI) - which test tools cover which platforms?
- Onboarding a tester from web E2E - what's the mental-model difference between DOM-driving and accessibility-tree-driving?
The three OS backends
Every reliable desktop driver routes through one of three OS-native accessibility trees. Direct pixel / coordinate scripting (SikuliX, AutoIt screen scraping) is in the anti-pattern column below.
Windows - Microsoft UI Automation (UIA)
UI Automation (UIA) is Microsoft's accessibility framework for
Windows desktop applications, replacing the older Microsoft Active
Accessibility (MSAA) interface (msuia). UIA exposes an
AutomationElement tree with control patterns (InvokePattern,
ValuePattern, SelectionPattern, etc.) that describe what each
element does, not just how it looks.
Per msuia:
"UI Automation is designed for experienced C/C++ developers. In general, developers need a moderate level of understanding about Component Object Model (COM) objects and interfaces, Unicode, and Windows API programming."
Higher-level language bindings (C#, PowerShell, Python via pywinauto)
sit on top. For test-automation purposes, the WinAppDriver and
Appium-Windows projects expose UIA as a W3C WebDriver endpoint -
which is what most QA toolchains actually drive.
macOS - Apple Accessibility + XCTest
macOS GUI testing uses the same XCTest framework that ships with
Xcode for unit and integration tests. Per appleuit:
"UI testing rests upon two core technologies: the XCTest framework and Accessibility."
The three foundational classes (appleuit) are:
| Class | Role |
|---|---|
XCUIApplication | The application under test |
XCUIElement | A single UI element in the accessibility tree |
XCUIElementQuery | A query that resolves to zero or more XCUIElement |
The general UI-test pattern per appleuit:
"Use an XCUIElementQuery to find an XCUIElement. Synthesize an event and send it to the XCUIElement. Use an assertion to compare the state of the XCUIElement against an expected reference state."
Xcode's Accessibility Inspector (bundled with Xcode → Xcode → Open Developer Tool → Accessibility Inspector) is the read-side companion: it walks the same tree the tests see, lets the engineer verify accessibility identifiers exist before writing the query, and flags WCAG-style accessibility gaps in the same pass.
Linux - AT-SPI
The Linux desktop accessibility stack is AT-SPI (Assistive
Technology Service Provider Interface). Per atspi2core,
the canonical implementation at-spi2-core provides "Base DBus XML
interfaces for accessibility, the accessibility registry daemon,
and atspi library." The registry daemon (at-spi2-registryd)
exposes a system-wide D-Bus service that assistive tools - and test
clients - connect to.
GTK applications expose AT-SPI automatically via the atk bridge
(at-spi2-atk); Qt applications expose AT-SPI via the
QAccessible infrastructure (see qt-test-framework SKILL for
the Qt-specific path); Electron and Chromium apps on Linux expose
their own AT-SPI surface.
Python clients (dogtail, pyatspi) and the GNOME
inspector tool Accerciser walk the registry to drive tests.
The toolkit overlays
Two cross-toolkit families need explicit treatment because they ride on top of (or around) the OS backends:
Electron
Electron apps are Chromium + Node.js wrapped in a native window. They expose two parallel surfaces:
- Renderer process - a normal Chromium DOM. Page-level Playwright / Selenium / Puppeteer drive this just like a browser tab.
- Main process - the Node.js process that owns native windows, menus, IPC, file dialogs, lifecycle.
A packaged Electron app cannot be driven by browser-only
Playwright - the entry point is the packaged binary, not a URL. The
Playwright _electron API (electron-playwright SKILL in this
plugin) launches the packaged binary, exposes the main process as a
typed ElectronApplication handle, and returns Chromium-window
pages for the renderer. Legacy alternative (deprecated 2021):
Spectron - covered as a legacy reference in electron-spectron.
Qt
Qt has its own first-party test framework - QtTest - that lives
in the application's process and emits events directly into the
QObject event queue. It does not go through the OS
accessibility tree by default. For UI tests of Qt apps that need
to be driven from outside the process, the OS-native drivers
(WinAppDriver on Windows, XCTest on macOS, AT-SPI on Linux via
Qt's QAccessible) are the path. See qt-test-framework SKILL.
Per-OS / per-toolkit decision matrix
| App type | Windows driver | macOS driver | Linux driver |
|---|---|---|---|
| Win32 / WinForms / WPF | WinAppDriver, Appium-Windows | n/a | n/a |
| WinUI 3 / UWP | WinAppDriver, Appium-Windows | n/a | n/a |
| Cocoa / SwiftUI / AppKit | n/a | XCTest UI (XCUIApplication) | n/a |
| GTK | n/a | n/a | AT-SPI (dogtail / pyatspi) |
| Qt - in-process | QtTest (QTEST_MAIN) | QtTest | QtTest |
| Qt - out-of-process | WinAppDriver (via UIA) | XCTest UI | AT-SPI (via QAccessible) |
| Electron - renderer only | Playwright _electron page handle | Playwright _electron page handle | Playwright _electron page handle |
| Electron - main + IPC | Playwright _electron | Playwright _electron | Playwright _electron |
Cross-references to per-tool SKILLs:
winappdriver- WinAppDriver against UIAappium-windows-driver- Appium-Windows driver (newer WinAppDriver wrapping)xctest-mac-desktop- XCTest UI for macOSat-spi-linux- AT-SPI via dogtail / pyatspiqt-test-framework- QtTest in-processelectron-playwright- Playwright_electronAPIelectron-spectron- legacy reference for migrations
Locator strategy across backends
The portable lesson from web E2E (per
playwright-testing in the qa-web-e2e plugin) - accessibility-first locators - carries directly over: every
desktop backend resolves elements through the same accessibility
tree assistive technology uses.
| Backend | Stable locator | Brittle locator |
|---|---|---|
| UIA (WinAppDriver, Appium-Windows) | AutomationId (preferred - locale-independent), ControlType + property combo (disambiguation when no AutomationId) | Name (last resort - Name is the localised label, fails across language packs); absolute screen coordinates; XPath (officially supported by WinAppDriver but the community-canonical guidance treats it as fragile and slow) |
| XCTest (macOS) | accessibilityIdentifier, role-based queries (buttons["Submit"]) | Label-based queries when the label is localised; image matching; hard-coded coordinates |
| AT-SPI (Linux) | Object name field set explicitly on the widget (GTK widget.set_property('name', ...), Qt QObject::setObjectName) | Visible label or role-only queries (collapse under localisation and theme changes); pixel coordinates |
| QtTest in-process | Qt object name (setObjectName("…")) | child-index navigation through QWidget tree |
| Electron renderer (Playwright) | getByRole / getByLabel on the renderer DOM | CSS class chains; Electron debug-build internal IDs |
The single biggest portability win across desktop platforms is: every app under test sets a stable accessibility identifier on every interactive widget. Without it, the tree's only resolution key is the human-visible label, which collapses under localisation.
Localisation rule of thumb: if the test passes on a US-English build
but the same app ships in 20 locales, every Name / label-based locator
is a latent failure. AccessibilityId / AutomationId / object name are
locale-independent by design - make the developer set them at the source.
Coverage scope per layer
| Layer | What it can cover | What it cannot cover |
|---|---|---|
| Unit test (XCTest / QtTest / .NET MSTest) | Pure logic, view-model bindings, QObject signals | OS-level dialog interactions, IME input, multi-window focus |
In-process UI test (XCUIApplication-in-app, QtTest with QTest::mouseClick) | App-internal widget events | Cross-app drag-drop, system shortcuts, OS file pickers |
Out-of-process UI driver (WinAppDriver / AT-SPI / Playwright _electron) | Full end-to-end including file dialogs, menus, system tray | Pre-login OS UI, GPU-accelerated rendered surfaces (canvas, WebGL) that don't expose the accessibility tree |
| Visual snapshot | Pixel-level regressions, font rendering | Logic, state, async behaviour |
Cross-references for the upstream + downstream slots:
- Unit testing - see the per-language
qa-unit-tests-*plugins. - Visual regression - see
qa-visual-regressionfor desktop screenshot comparison patterns. - CI integration - desktop runners cost more than web runners;
see the
winappdriverandxctest-mac-desktopSKILLs for GitHub-hosted vs self-hosted considerations.
Operating the tests - deep references
Once the stack is chosen, the operational depth lives in two companion references:
- Asynchronous waits + concurrency - per-OS retry primitives
(XCTest waiters, FlaUI
Retry, AT-SPI polling) and the per-OS parallel-test policy: references/async-waits-and-concurrency.md. - Platform hazards + high-DPI - foreground-lock, UAC secure desktop, macOS TCC prompts, AT-SPI session enablement, and the per-monitor DPI test matrix: references/platform-hazards-and-dpi.md.
Anti-patterns
| Anti-pattern | Why it fails | Fix |
|---|---|---|
| Image-recognition / pixel-matching as the primary driver (SikuliX-only, AutoIt screen scraping) | Brittle to font / theme / DPI / OS-chrome changes; opaque to localisation | Use the OS accessibility tree; reserve image matching for canvas-rendered content only |
| Hard-coded screen coordinates | Multi-monitor / DPI / window-state dependent | Resolve via accessibility identifier; click resolves to the element's hit-test centre |
Win32 message sending (SendMessage / PostMessage) for modern apps | Doesn't reach UIA-only WinUI / WPF controls; bypasses event ordering | Route through UIA (WinAppDriver) - the documented control-pattern API |
| Using only XCTest on a packaged macOS app shipped to the App Store via TestFlight | Loses cross-platform consistency with Windows + Linux | Pair with a higher-level cross-platform layer (Appium Mac2 driver) only when the same test source must run on multiple OSes; otherwise XCTest is the right tool on macOS |
| Spectron for new Electron projects (archived 2022) | No maintenance, no support for modern Electron | Use Playwright _electron (electron-playwright SKILL) |
| Driving Qt apps with Win32-only tools when accessibility isn't wired in | Qt's QAccessible interface must be enabled; without it UIA sees no children | Verify with Accessibility Insights for Windows before writing tests; enable QAccessible in the Qt build |
| Running desktop UI tests on GitHub-hosted Linux runners with no virtual display | at-spi2-registryd requires a session bus | Self-hosted runners with a real (or Xvfb / dbus-launch synthetic) session, or use Windows / macOS hosted runners for those OS-specific suites |
Limitations
- Apple developer documentation is behind a Cloudflare-style
challenge on most "https://developer.apple.com/documentation/…"
pages - automated fetches return the SPA shell only. This
reference cites Apple's stable-archive
testing_with_xcodechapter (appleuit) for prose; per-API surface (XCUIApplication, XCUIElement) is treated as a stable identifier in thexctest-mac-desktopSKILL. - No single cross-platform driver. Tools that claim to be one (Appium across Windows + macOS + Linux desktop) are thin facades over the per-OS backends - coverage gaps follow the underlying backend's gaps.
- GPU-accelerated content (canvas, WebGL, custom-painted Qt widgets, Win32 DirectComposition surfaces) does not expose itself to the accessibility tree by default; any driver routed through the tree sees them as opaque rectangles.
- Cross-process focus + window-Z order is OS-policy-dependent
(Windows foreground-lock, macOS Stage Manager); tests that assume
a window is foreground often need explicit
app.activate()calls. - Localisation collapses label-based locators. The per-tool
SKILLs all push toward
AutomationId/accessibilityIdentifier/ object name for this reason.
References
- Microsoft UI Automation overview - msuia.
- Apple Testing with Xcode - UI Testing chapter (appleuit).
- at-spi2-core - atspi2core.
- dogtail (Python AT-SPI client) - dogtail.
- Operational depth (with their own citations): async waits + concurrency in references/async-waits-and-concurrency.md; foreground / elevation hazards + high-DPI matrix in references/platform-hazards-and-dpi.md.
- Per-tool implementation SKILLs:
winappdriver,appium-windows-driver,xctest-mac-desktop,at-spi-linux,qt-test-framework,electron-playwright,electron-spectron. - Web-side neighbour:
playwright-testing- DOM-driving for browser apps (the contrast surface for desktop drivers). - Architecture-pattern sibling:
object-model-patterns(in the qa-test-review plugin) - Pattern 7 (Screen Object) is the desktop analog of POM and cites this skill for the backend substrate.