run-device-tests

Run Device Tests Skill

Safety Notice

This listing is imported from skills.sh public index metadata. Review upstream SKILL.md and repository scripts before running.

Copy this and send it to your AI assistant to learn

Install skill "run-device-tests" with this command: npx skills add dotnet/maui/dotnet-maui-run-device-tests

Run Device Tests Skill

Build and run .NET MAUI device tests locally on iOS simulators, MacCatalyst, Android emulators, or Windows.

Platform Support

Host OS Supported Platforms

macOS ios, maccatalyst, android

Windows android, windows

Tools Required

This skill uses bash together with pwsh (PowerShell 7+) to run the PowerShell scripts. Requires:

  • xharness

  • Global dotnet tool for running tests on iOS/MacCatalyst/Android

  • dotnet

  • .NET SDK with platform workloads installed

  • iOS/MacCatalyst: Xcode with iOS simulators

  • Android: Android SDK with emulator

  • Windows: Windows SDK

Dependencies

This skill uses shared infrastructure scripts:

  • .github/scripts/shared/Start-Emulator.ps1

  • Detects and boots iOS simulators / Android emulators

  • .github/scripts/shared/shared-utils.ps1

  • Common utility functions

These are automatically loaded by the Run-DeviceTests.ps1 script.

When to Use

  • User wants to run device tests locally

  • User wants to verify iOS/MacCatalyst/Android/Windows compatibility

  • User wants to test on a specific iOS version (e.g., iOS 26)

  • User asks "run device tests for Controls/Core/Essentials/Graphics"

  • User asks "test on iOS simulator" or "test on Android emulator"

  • User asks "run device tests on MacCatalyst"

  • User wants to run only specific test categories (e.g., "run Button tests")

Available Test Projects

Project Path

Controls src/Controls/tests/DeviceTests/Controls.DeviceTests.csproj

Core src/Core/tests/DeviceTests/Core.DeviceTests.csproj

Essentials src/Essentials/test/DeviceTests/Essentials.DeviceTests.csproj

Graphics src/Graphics/tests/DeviceTests/Graphics.DeviceTests.csproj

BlazorWebView src/BlazorWebView/tests/DeviceTests/MauiBlazorWebView.DeviceTests.csproj

AI src/AI/tests/Essentials.AI.DeviceTests/Essentials.AI.DeviceTests.csproj

Scripts

All scripts are in .github/skills/run-device-tests/scripts/

Run Device Tests (Full Workflow)

Run Controls device tests on iOS simulator (default on macOS)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

Run Core device tests on MacCatalyst

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform maccatalyst

Run Controls device tests on Android emulator

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

Run Controls device tests on Windows (default on Windows)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

Run on specific iOS version

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform ios -iOSVersion 26

Run with test filter

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

Run other test projects

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Essentials -Platform android pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Graphics -Platform maccatalyst pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project BlazorWebView -Platform ios

Build Only (No Test Run)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -BuildOnly

List Available Simulators/Emulators

iOS simulators

xcrun simctl list devices available

Android emulators

emulator -list-avds

Workflow

  • Run tests: scripts/Run-DeviceTests.ps1 -Project <name> -Platform <platform> automatically detects/boots device, builds, and runs tests

  • Check results: Look at the console output or artifacts/log/ directory for detailed test results

Output

  • Build artifacts: artifacts/bin/<Project>.DeviceTests/<Configuration>/<tfm>/<rid>/

  • Test logs: artifacts/log/

  • Test results summary is printed to console

Prerequisites

  • xharness global tool: dotnet tool install --global Microsoft.DotNet.XHarness.CLI

  • .NET SDK with platform workloads

  • iOS/MacCatalyst: Xcode with simulators installed

  • Android: Android SDK with emulator configured

  • Windows: Windows SDK

  • For iOS 26: macOS Tahoe (26) with Xcode 26

Examples

Quick test run for Controls on iOS (default on macOS)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios

Test on MacCatalyst

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform maccatalyst

Test on Android emulator (works on both macOS and Windows)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android

Test on Windows (default on Windows)

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform windows

Test on iOS 26 specifically

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -iOSVersion 26

Run only Button category tests on iOS

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

Build for Android without running tests

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Core -Platform android -BuildOnly

Notes

  • The script automatically detects and boots an iOS simulator / Android emulator using the shared Start-Emulator.ps1 infrastructure

  • Default iOS simulator is iPhone Xs with iOS 18.5 (same as UI tests)

  • Default Android emulator priority: API 30 Nexus > API 30 > Nexus > First available

  • MacCatalyst runs directly on the Mac (no simulator needed)

  • Windows tests run directly on the local machine

  • Simulator/emulator selection and boot logic is handled by .github/scripts/shared/Start-Emulator.ps1

  • xharness manages test execution and reporting for iOS/MacCatalyst/Android

  • Windows uses vstest for test execution

Test Filtering

The -TestFilter parameter allows running specific test categories instead of all tests. This is useful for quick iteration during development.

Filter Syntax

Format Description Example

Category=X

Run only category X Category=Button

SkipCategories=X,Y,Z

Skip categories X, Y, Z SkipCategories=Shell,CollectionView

Examples

Run only Button tests on iOS

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "Category=Button"

Run only Button tests on Android

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android -TestFilter "Category=Button"

Skip heavy test categories

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -TestFilter "SkipCategories=Shell,CollectionView,HybridWebView"

How Test Filtering Works

Test filtering is implemented in src/Core/tests/DeviceTests.Shared/DeviceTestSharedHelpers.cs :

Platform How Filter is Passed How Filter is Read

iOS/MacCatalyst --set-env=TestFilter=...

NSProcessInfo.ProcessInfo.Environment["TestFilter"]

Android --arg TestFilter=...

MauiTestInstrumentation.Current.Arguments.GetString("TestFilter")

Windows --filter "Category=..."

Native vstest filter

Available Test Categories

Common categories in Controls.DeviceTests:

  • Button , Label , Entry , Editor

  • Individual control tests

  • CollectionView , ListView , CarouselView

  • Collection controls (heavy)

  • Shell , Navigation , TabbedPage

  • Navigation tests (heavy)

  • Layout , FlexLayout

  • Layout tests

  • Memory

  • Memory leak tests

  • Accessibility

  • Accessibility tests

  • Gesture

  • Gesture recognition tests

To see all categories, check src/Controls/tests/DeviceTests/TestCategory.cs .

Troubleshooting

Xcode Version Mismatch

If you see errors about Xcode version mismatch (e.g., "Xcode 26.2 installed but 26.0 required"):

Use -SkipXcodeVersionCheck to bypass validation

pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform ios -SkipXcodeVersionCheck

MacCatalyst App Bundle Not Found

MacCatalyst apps use display names (e.g., "Controls Tests.app") not assembly names. The script handles this automatically by searching for .app bundles in the output directory.

Android Package Name Issues

Android package names must be lowercase (e.g., com.microsoft.maui.controls.devicetests ). The script has a mapping of project names to correct package names.

Test Filter Not Working

Ensure full rebuild: The test filter code must be compiled into the app. Delete artifacts and rebuild:

rm -rf artifacts/bin/<Project>.DeviceTests pwsh .github/skills/run-device-tests/scripts/Run-DeviceTests.ps1 -Project Controls -Platform android -TestFilter "Category=Button"

Check the console output: Look for TestFilter: Category=Button in the test output to confirm filter was applied.

Verify category exists: Ensure the category name matches exactly (case-sensitive).

XHarness Device Detection

The script automatically handles XHarness device targeting for iOS and Android:

iOS

  • Simulator Boot: Start-Emulator.ps1 detects and boots appropriate iOS simulator

  • UDID Extraction: Script extracts simulator UDID from Start-Emulator.ps1 output

  • iOS Version Detection: Script queries xcrun simctl to get iOS version from booted simulator

  • XHarness Targeting: Passes both --target ios-simulator-64_VERSION and --device UDID to xharness for explicit targeting

MacCatalyst

  • Runs directly on the Mac, no device targeting needed

  • XHarness uses --target maccatalyst

Android

  • Emulator Boot: Start-Emulator.ps1 detects running device or boots emulator

  • Device ID Extraction: Script gets device ID (e.g., emulator-5554 )

  • XHarness Targeting: Passes --device-id to xharness android command

Windows

  • No device/emulator needed

  • Uses vstest (dotnet test ) for test execution

Why both --target and --device for iOS?

  • XHarness requires --target ios-simulator-64 (or ios-simulator-64_VERSION ) to specify platform type

  • Adding --device UDID explicitly tells xharness which simulator to use

  • This combination ensures reliable device selection even on ARM64 Macs where automatic detection can fail

Example xharness invocations:

iOS with test filter

dotnet xharness apple test
--app path/to/app
--target ios-simulator-64_18.5
--device 56AE278D-60F7-4892-9DE0-6341357CA068
-o artifacts/log
--timeout 01:00:00
-v
--set-env=TestFilter=Category=Button

MacCatalyst with test filter

dotnet xharness apple test
--app path/to/app
--target maccatalyst
-o artifacts/log
--timeout 01:00:00
-v
--set-env=TestFilter=Category=Button

Android with test filter

dotnet xharness android test
--app path/to/app.apk
--package-name com.microsoft.maui.controls.devicetests
--device-id emulator-5554
-o artifacts/log
--timeout 01:00:00
-v
--arg TestFilter=Category=Button

This ensures tests run on the correct device with proper version targeting and filtering.

Source Transparency

This detail page is rendered from real SKILL.md content. Trust labels are metadata-based hints, not a safety guarantee.

Open in GitHubOpen in ClawHub

Related Skills

Related by shared tags or category signals.

General

try-fix

No summary provided by upstream source.

Repository SourceNeeds Review
-28
dotnet
General

pr-finalize

No summary provided by upstream source.

Repository SourceNeeds Review
-18
dotnet
General

learn-from-pr

No summary provided by upstream source.

Repository SourceNeeds Review
-18
dotnet