testland/unreal-automation-system

Author and run Unreal Engine's Automation Test Framework - Epic's C++ test framework for UE 4.x / 5.x, documented at dev.epicgames.com/documentation/en-us/unreal-engine. Covers the five test categories Epic defines (Unit / Feature / Smoke / Content Stress / Screenshot Comparison), the IMPLEMENT_SIMPLE_AUTOMATION_TEST and IMPLEMENT_COMPLEX_AUTOMATION_TEST macros, the BDD-style Automation Spec API (DEFINE_SPEC / BEGIN_DEFINE_SPEC / Describe / It / BeforeEach / LatentIt / xIt), latent commands (ADD_LATENT_AUTOMATION_COMMAND), the Automation Driver for UI input simulation (IAutomationDriverModule::Get().CreateDriver(), By::Id / By::Path locators), running via Session Frontend (Window > Test Automation) and command line (-ExecCmds="Automation RunTests …"), and CI integration. Use when the unit under test is C++ Unreal code that needs the UE runtime, editor, or UMG UI surface.

Quality

93%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

name:: unreal-automation-system
description:: Author and run Unreal Engine's Automation Test Framework - Epic's C++ test framework for UE 4.x / 5.x, documented at dev.epicgames.com/documentation/en-us/unreal-engine. Covers the five test categories Epic defines (Unit / Feature / Smoke / Content Stress / Screenshot Comparison), the IMPLEMENT_SIMPLE_AUTOMATION_TEST and IMPLEMENT_COMPLEX_AUTOMATION_TEST macros, the BDD-style Automation Spec API (DEFINE_SPEC / BEGIN_DEFINE_SPEC / Describe / It / BeforeEach / LatentIt / xIt), latent commands (ADD_LATENT_AUTOMATION_COMMAND), the Automation Driver for UI input simulation (IAutomationDriverModule::Get().CreateDriver(), By::Id / By::Path locators), running via Session Frontend (Window > Test Automation) and command line (-ExecCmds="Automation RunTests …"), and CI integration. Use when the unit under test is C++ Unreal code that needs the UE runtime, editor, or UMG UI surface.
metadata:: {"keywords":"unreal, ue4, ue5, automation, automation-spec, automation-driver, session-frontend, gauntlet, cpp, game-engine"}

unreal-automation-system

Name: testland/unreal-automation-system
Rating: 74.4 (1 reviews)
Author: testland

Overview

This skill wraps the C++ Automation Test Framework (UE 4.x / 5.x) plus the two most commonly composed sub-systems. Per Epic's Automation Test Framework documentation, the framework spans five test categories (Unit, Feature, Smoke, Content Stress, Screenshot Comparison) and supports multiple authoring styles (traditional, BDD Spec, UI Driver, Functional, Python / Blueprint).

Automation Spec - BDD-style Describe / It / BeforeEach per Automation Spec documentation.
Automation Driver - UI input simulation per Automation Driver documentation.

The full authoring code (macros, Spec, Driver) lives in references/authoring-macros-and-apis.md; running, report parsing, and CI wiring live in references/running-and-reporting.md. This file is the decision surface.

Composes with:

game-test-categories-reference for the canonical six categories Unreal's five categories map to.
platform-cert-overview-reference for cert-gated requirements automation tests should cover.
multiplayer-state-machine-coverage for replication / dedicated-server state coverage authored as automation tests.

When to use

Unit under test is C++ code in an Unreal Engine project (UE 4.x or UE 5.x) that needs the engine runtime, editor APIs, or UMG UI surface.
You want CI-runnable tests via Unreal's command-line Automation entry point.
You need BDD-style readable specs (Automation Spec) or scripted UI input simulation (Automation Driver) on top of plain assertion tests.

For Python / Blueprint editor tests outside C++, see Editor Automation in Unreal Engine. For end-to-end pipeline / build-farm orchestration, use Gauntlet on top of the framework this skill covers.

How to use

Pick the category from the table below (Unit / Feature / Smoke / Content Stress / Screenshot Comparison) and the matching EAutomationTestFlags filter + application-context mask.
Pick the authoring style - plain RunTest macro, data-driven complex macro, BDD Spec, or Automation Driver for UI - and lift the pattern from references/authoring-macros-and-apis.md.
Name the test path as dot-separated segments (MyGame.Health.Damage_…); the tree keys the Session Frontend.
Run in-editor via Session Frontend to iterate, then run headless via UnrealEditor-Cmd.exe … -ExecCmds="Automation RunTests …" for CI.
Parse the -ReportOutputPath JSON - treat any "state": "Fail" as a failed build; full schema in references/running-and-reporting.md.
Split the CI matrix - SmokeFilter on PRs (sub-second), ProductFilter | StressFilter plus screenshot comparison nightly.

Test categories and flags

Per the Automation Test Framework page, Epic's five categories are:

Category	Purpose
Unit	"API level verification tests."
Feature	"System-level tests that verify such things as PIE, in-game stats, and changing resolution."
Smoke	Tests that "complete within 1 second" and run automatically.
Content Stress	"More thorough testing of a particular system to avoid crashes."
Screenshot Comparison	For comparing renders "between versions or builds".

Each test declares flags from EAutomationTestFlags that mix:

Filter - SmokeFilter, EngineFilter, ProductFilter, PerfFilter, StressFilter, NegativeFilter.
Application context - EditorContext, ClientContext, ServerContext, CommandletContext, plus the convenience mask ApplicationContextMask.

Typical combo for a product-level test runnable in editor / client / server contexts: EAutomationTestFlags::ProductFilter | EAutomationTestFlags::ApplicationContextMask.

Authoring styles

Pick the style that matches the unit under test, then lift the full pattern from references/authoring-macros-and-apis.md:

Style	Macro / API	Use when
Simple test	`IMPLEMENT_SIMPLE_AUTOMATION_TEST` + `RunTest`	One assertion body over plain C++
Data-driven	`IMPLEMENT_COMPLEX_AUTOMATION_TEST` + `GetTests`	One sub-test per enumerated row (assets, configs)
Multi-frame	`ADD_LATENT_AUTOMATION_COMMAND`	Test must yield to the tick loop across frames
BDD Spec	`DEFINE_SPEC` / `BEGIN_DEFINE_SPEC` + `Describe` / `It` / `LatentIt`	Readable specs; `.spec.cpp` files; async via `FDoneDelegate`
UI Driver	`IAutomationDriverModule::Get().CreateDriver()` + `By::Id`	Simulate cursor / click / type on UMG; runs off the GameThread

Running

Iterate in-editor via Window → Test Automation (Session Frontend), then run headless for CI:

UnrealEditor-Cmd.exe MyGame.uproject \
    -ExecCmds="Automation RunTests MyGame.Inventory; Quit" \
    -unattended -nopause -testexit="Automation Test Queue Empty" \
    -ReportOutputPath="artifacts/automation" \
    -log

Command variants, the -ReportOutputPath JSON schema, Gauntlet, and a full GitHub Actions job are in references/running-and-reporting.md.

Worked example

Goal: cover an inventory system's stacking rule as a CI-gating BDD spec, plus a menu-close UI check.

Category + flags. Stacking logic is pure C++ product logic, so ProductFilter | ApplicationContextMask; it runs in well under a second, so a second SmokeFilter copy joins the PR job.
Authoring. Write FInventorySpec with DEFINE_SPEC and a Describe("AddItem") block holding two It("should …") cases - one asserting count increases by the stack amount, one asserting an over-cap add is rejected (pattern in references/authoring-macros-and-apis.md). Save it as InventorySpec.spec.cpp.
UI check. Add FMenuDriverSpec with the Driver enabled in BeforeEach on TaskGraphMainThread, the It running on EAsyncExecution::ThreadPool, By::Id("SubmitButton") clicked, and Disable() in AfterEach.
Run headless. UnrealEditor-Cmd.exe MyGame.uproject -ExecCmds="Automation RunTests MyGame.Inventory; Quit" -unattended -nopause -ReportOutputPath="artifacts/automation" -log.
Gate. CI reads artifacts/automation/index.json; the over-cap It shows "state": "Fail" when the rule regresses, so the job fails the build. Nightly re-runs the same specs under ProductFilter | StressFilter.

Anti-patterns

Anti-pattern	Why it fails	Fix
All tests as `SmokeFilter`	Smoke tests "complete within 1 second" per framework docs - long tests break the smoke contract	Use `ProductFilter` for tests > 1 s
Running Automation Driver on GameThread	API "cannot run on the GameThread" per Driver docs	Use `EAsyncExecution::ThreadPool` on the driver-using `It`
Spec without `It` descriptions starting with "should"	Runner output reads poorly	Per Spec docs, start descriptions with "should"
Trusting `By::Path` locators	"Brittle" per Driver docs	Prefer `By::Id` with tagged metadata
`Sleep`-style waits in latent commands	Flaky under CI load	Use `FDoneDelegate` (LatentIt) or custom `IAutomationLatentCommand::Update()` polling
Spec test file without `.spec.cpp` extension	Build system may not pick it up	Per Spec docs, use `.spec.cpp` suffix and no "Test" in filename
Cloning non-thread-safe shared pointers in async `It`	Crash under threadpool	Per Driver docs, cache them on the test class
No `BeforeEach` / `AfterEach` cleanup of `IAutomationDriverModule`	Driver state leaks between specs	Pair `Enable()` / `Disable()` calls in `BeforeEach` / `AfterEach` per Driver docs

Limitations

C++ build required. Tests live in C++ modules; pure-content Blueprint projects need a code module added to use this framework. (Pure Blueprint projects can use Blueprint Functional Tests in-level - see Epic's docs on Functional Testing.)
No common exit code definition comparable to Unity's caveat - parse the -ReportOutputPath JSON or scrape the log for LogAutomationController Fail lines.
Spec parameterised tests are loop-generated per Spec docs; there is no [TestCase] analogue from NUnit. The framework is NUnit-inspired but not NUnit-derived (unlike Unity's UTF).
Screenshot comparison baseline storage + tolerance configuration is engine-version-specific - consult per-version Epic docs.
Editor-context tests cannot run in dedicated-server only builds - declare EditorContext on those tests; client / server tests need their own flag set.
Documentation source. The dev.epicgames.com docs are the public mirror; deeper detail (full macro implementations, precise JSON report schema) lives in the engine source under Engine/Source/Runtime/AutomationController/ and Engine/Source/Developer/AutomationMessages/ - partners with engine source access should consult those for authoritative details.

References

Automation Test Framework - dev.epicgames.com.
Automation Spec (BDD) - dev.epicgames.com.
Automation Driver (UI input) - dev.epicgames.com.
Full authoring code (macros, Spec, Driver): references/authoring-macros-and-apis.md.
Running, report parsing, and CI wiring: references/running-and-reporting.md.
Category mapping: game-test-categories-reference.
Cert-gated requirements: platform-cert-overview-reference.
Replication / dedicated-server coverage: multiplayer-state-machine-coverage.

.tessl-plugin

references

SKILL.md

tile.json