Run Helix Tests Skill

Submit and monitor .NET MAUI unit tests on Helix infrastructure from your local machine.

Tools Required

This skill uses pwsh (PowerShell 7+) to run the scripts. The scripts call the local .dotnet/dotnet SDK.

When to Use

• User asks to "run tests on Helix"
• User wants to validate test changes on Helix infrastructure
• User asks to "test on multiple platforms" (Windows + macOS)
• User wants to debug Helix-specific test failures locally
• User asks about "Helix unit tests" (not device tests - those use helix_xharness.proj)

Prerequisites

Before running Helix tests, ensure:

1. Local SDK provisioned: The .dotnet/ folder must exist with the SDK

   # If missing, run:
   dotnet cake --target=dotnet

2. Build tasks compiled (for first run):

   # Windows
   .\build.cmd -restore -build -configuration Release -projects .\Microsoft.Maui.BuildTasks.slnf
   
   # macOS/Linux  
   ./build.sh -restore -build -configuration Release -projects ./Microsoft.Maui.BuildTasks.slnf

Scripts

All scripts are in .github/skills/run-helix-tests/scripts/

1. Submit Unit Tests to Helix

# Submit all unit tests (XAML, Core, Essentials, Resizetizer, etc.)
pwsh .github/skills/run-helix-tests/scripts/Submit-HelixTests.ps1

# Submit with specific configuration
pwsh .github/skills/run-helix-tests/scripts/Submit-HelixTests.ps1 -Configuration Debug

# Submit to specific queue only
pwsh .github/skills/run-helix-tests/scripts/Submit-HelixTests.ps1 -Queue "Windows.10.Amd64.Open"

2. Monitor Helix Job Status

# Monitor a submitted job (use job ID from submission output)
pwsh .github/skills/run-helix-tests/scripts/Get-HelixJobStatus.ps1 -JobId <JOB_ID>

# Wait for completion with polling
pwsh .github/skills/run-helix-tests/scripts/Get-HelixJobStatus.ps1 -JobId <JOB_ID> -Wait

3. Get Helix Work Item Console Logs

# Get console output for a specific work item
pwsh .github/skills/run-helix-tests/scripts/Get-HelixWorkItemLog.ps1 -JobId <JOB_ID> -WorkItem <WORK_ITEM_NAME>

# Get failed work items only
pwsh .github/skills/run-helix-tests/scripts/Get-HelixWorkItemLog.ps1 -JobId <JOB_ID> -FailedOnly

Quick Start (Manual)

If you prefer to run directly without scripts:

# Windows
.\helix.cmd

# macOS/Linux
./helix.sh

These wrapper scripts set the required environment variables and submit to Helix.

Test Projects Submitted

The following test projects are submitted to Helix (defined in eng/helix.proj):

Helix Queues

Tests run on multiple queues in parallel:

Workflow

Submit and Monitor Tests

1. Submit tests:

   $result = pwsh .github/skills/run-helix-tests/scripts/Submit-HelixTests.ps1
   # Note the job IDs from output

2. Monitor progress:

   pwsh .github/skills/run-helix-tests/scripts/Get-HelixJobStatus.ps1 -JobId $jobId -Wait

3. Check failures (if any):

   pwsh .github/skills/run-helix-tests/scripts/Get-HelixWorkItemLog.ps1 -JobId $jobId -FailedOnly

Debug a Specific Test Failure

1. Get the job ID from the Helix submission output
2. Find the failed work item name
3. Get the console log:

   pwsh .github/skills/run-helix-tests/scripts/Get-HelixWorkItemLog.ps1 -JobId <JOB_ID> -WorkItem "Microsoft.Maui.Controls.Xaml.UnitTests.dll"

Understanding Results

Helix test results are reported to the console and can be viewed at:

• Helix Portal: https://helix.dot.net/api/2019-06-17/jobs/{jobId}
• Work Item Console: https://helix.dot.net/api/2019-06-17/jobs/{jobId}/workitems/{workItemName}/console

Common Issues

Difference from Device Tests

This skill runs unit tests on Helix (eng/helix.proj).

For device tests (iOS, Android, MacCatalyst emulators/simulators), use:

# Device tests use helix_xharness.proj instead
./eng/common/msbuild.sh ./eng/helix_xharness.proj /t:Test /p:TargetOS=ios

See .github/instructions/helix-device-tests.instructions.md for device test guidance.