run-integration-tests

Build, pack, and run .NET MAUI integration tests locally. Validates templates, samples, and end-to-end scenarios using the local workload.

Quality

77%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.github/skills/run-integration-tests/SKILL.md

Run Integration Tests Skill

Build the MAUI product, install local workloads, and run integration tests.

When to Use

User asks to "run integration tests"
User asks to "test templates locally"
User asks to "validate MAUI build with templates"
User wants to verify changes don't break template scenarios
User asks to run specific test categories (WindowsTemplates, Samples, Build, Blazor, etc.)

Available Test Categories

Category	Platform	Description
`Build`	All	Basic template build tests
`WindowsTemplates`	Windows	Windows-specific template scenarios
`macOSTemplates`	macOS	macOS-specific scenarios
`Blazor`	All	Blazor hybrid templates
`MultiProject`	All	Multi-project templates
`Samples`	All	Sample project builds
`AOT`	macOS	Native AOT compilation
`RunOnAndroid`	macOS	Build, install, run on Android emulator
`RunOniOS`	macOS	iOS simulator tests

Scripts

All scripts are in .github/skills/run-integration-tests/scripts/

Run Integration Tests (Full Workflow)

# Run with specific category
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "WindowsTemplates"

# Run with Release configuration
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Samples" -Configuration "Release"

# Run with custom test filter
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~BuildSample"

# Skip build step (if already built)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -SkipBuild

# macOS: Skip Xcode version check (for version mismatches)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "macOSTemplates" -SkipBuild -SkipInstall -SkipXcodeVersionCheck

# Auto-provision SDK if not found (first-time setup)
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -AutoProvision

Parameters

Parameter	Required	Default	Description
`-Category`	No	-	Test category to run (WindowsTemplates, Samples, Build, etc.)
`-TestFilter`	No	-	Custom NUnit test filter expression
`-Configuration`	No	Debug	Build configuration (Debug/Release)
`-SkipBuild`	No	false	Skip build/pack step if already done
`-SkipInstall`	No	false	Skip workload installation if already done
`-SkipXcodeVersionCheck`	No	false	Skip Xcode version validation (macOS)
`-AutoProvision`	No	false	Automatically provision local SDK if not found
`-ResultsDirectory`	No	artifacts/integration-tests	Directory for test results

Workflow Steps

The script performs these steps:

Build & Pack: .\build.cmd -restore -pack -configuration $Configuration
Install Workloads: .dotnet\dotnet build .\src\DotNet\DotNet.csproj -t:Install -c $Configuration
Extract Version: Reads MAUI_PACKAGE_VERSION from installed packs
Run Tests: .dotnet\dotnet test ... -filter "Category=$Category"

Example Usage

# Run WindowsTemplates tests
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "WindowsTemplates"

# Run Samples tests
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Samples"

# Run multiple categories
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "Category=Build|Category=Blazor"

Prerequisites

Windows for WindowsTemplates, macOS for macOSTemplates/RunOniOS/RunOnAndroid
.NET SDK (version from global.json)
Sufficient disk space for build artifacts
Local SDK and workloads must be provisioned first

Provisioning the Local SDK (Required First Time)

Before running integration tests, you must provision the local .NET SDK and MAUI workloads:

# Step 1: Restore dotnet tools
dotnet tool restore

# Step 2: Provision local SDK and install workloads (~5 minutes)
dotnet cake --target=dotnet

# Step 3: Install MAUI local workloads
dotnet cake --target=dotnet-local-workloads

Verification:

# Check SDK exists
ls .dotnet/dotnet

# Check MAUI SDK version
ls .dotnet/packs/Microsoft.Maui.Sdk

Note: The old ./build.sh --target=dotnet syntax no longer works. Use dotnet cake directly.

Output

Test results in TRX format at <ResultsDirectory>/
Build logs in artifacts/ directory
Console output with test pass/fail summary

Troubleshooting

Issue	Solution
"MAUI_PACKAGE_VERSION was not set"	Ensure build step completed successfully
"Local .dotnet SDK not found"	Run `dotnet tool restore && dotnet cake --target=dotnet && dotnet cake --target=dotnet-local-workloads`
Template not found	Workload installation may have failed
Build failures	Check `artifacts/log/` for detailed build logs
"Cannot proceed with locked .dotnet folder"	Kill processes using `.dotnet`: `Get-Process \| Where-Object { $_.Path -like "\.dotnet\" } \| ForEach-Object { Stop-Process -Id $_.Id -Force }`
Session times out / becomes invalid	Integration tests are long-running (15-60+ min). Run manually in a terminal window instead of via Copilot CLI
Tests take too long	Start with `Build` category (fastest), then run others. Use `-SkipBuild -SkipInstall` if workloads are already installed
iOS tests fail with "mlaunch exited with 1"	Simulator state issue. Run individual tests instead of the whole category (see below)
iOS simulator state errors (code 137/149)	Reset simulator: `xcrun simctl shutdown all && xcrun simctl erase all` or run tests individually

Running Manually (Recommended for Long-Running Tests)

Integration tests can take 15-60+ minutes depending on the category. For best results, run directly in a terminal:

cd D:\repos\dotnet\maui

# Option 1: Use the skill script
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category "Build" -SkipBuild -SkipInstall

# Option 2: Run dotnet test directly (if workloads already installed)
$env:MAUI_PACKAGE_VERSION = (Get-ChildItem .dotnet\packs\Microsoft.Maui.Sdk -Directory | Sort-Object Name -Descending | Select-Object -First 1).Name
.dotnet\dotnet test src\TestUtils\src\Microsoft.Maui.IntegrationTests --filter "Category=Build"

Running All Categories Sequentially

# Windows categories (run on Windows)
$categories = @("Build", "WindowsTemplates", "Blazor", "MultiProject", "Samples")
foreach ($cat in $categories) {
    Write-Host "Running $cat..." -ForegroundColor Cyan
    pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -Category $cat -SkipBuild -SkipInstall
}

Running Individual iOS Tests (Recommended)

Running all iOS tests together (-Category "RunOniOS") can cause simulator state issues. For better reliability, run tests individually:

# Available iOS tests
$iosTests = @(
    "RunOniOS_MauiDebug",
    "RunOniOS_MauiRelease",
    "RunOniOS_MauiReleaseTrimFull",
    "RunOniOS_BlazorDebug",
    "RunOniOS_BlazorRelease",
    "RunOniOS_MauiNativeAOT"
)

# Run a specific iOS test
pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~RunOniOS_MauiDebug" -SkipBuild -SkipInstall -SkipXcodeVersionCheck

# Run all iOS tests individually (more reliable than running category)
foreach ($test in $iosTests) {
    Write-Host "Running $test..." -ForegroundColor Cyan
    pwsh .github/skills/run-integration-tests/scripts/Run-IntegrationTests.ps1 -TestFilter "FullyQualifiedName~$test" -SkipBuild -SkipInstall -SkipXcodeVersionCheck
}

Repository: dotnet/maui
Path: .github/skills/run-integration-tests/SKILL.md
Commit: 6a24ec5

Last updated: 4 days ago
First committed: 6 months ago

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.