Qt Pilot Usage

Qt Pilot is a bundled MCP server that lets Claude launch, interact with, and visually test PySide6 Qt applications — without a physical display. It uses Xvfb (X Virtual Framebuffer) for headless rendering and communicates with a test harness over a Unix socket.

Use /qt:visual to start a visual testing session.

Architecture

Claude (MCP tools) ↓ Qt Pilot MCP server (mcp/qt-pilot/main.py) ↓ (Unix socket) Qt Pilot Harness (runs inside Xvfb) ↓ Target Qt/PySide6 Application

The harness launches inside the virtual display, imports the application, and exposes widget interactions back to the MCP server.

Prerequisites

• Xvfb installed (Xvfb binary on PATH). Run scripts/check-prerequisites.sh to verify.

• Application widgets must have object names set with setObjectName() to be targetable by name.

• Application must use QApplication (or QGuiApplication ) — not just a bare Qt import.

Setting Object Names (Required for Most Tools)

Without object names, only coordinate-based interaction (click_at ) and list_all_widgets work. Add names to all interactive elements:

Python/PySide6

self.calculate_btn = QPushButton("Calculate") self.calculate_btn.setObjectName("calculate_btn")

self.result_label = QLabel("") self.result_label.setObjectName("result_label")

self.input_field = QLineEdit() self.input_field.setObjectName("input_field")

// C++ equivalent QPushButton *btn = new QPushButton("Calculate"); btn->setObjectName("calculate_btn");

Available MCP Tools (15 total)

Full argument types, return schemas, and error handling — see references/mcp-tools-reference.md.

Quick reference by category:

• App lifecycle: launch_app , get_app_status , wait_for_idle , close_app

• Discovery: find_widgets , list_all_widgets , get_widget_info , list_actions

• Named interaction (requires setObjectName ): click_widget , hover_widget , type_text , press_key , trigger_action

• Coordinate interaction: click_at

• Visual capture: capture_screenshot

Standard Workflow

1. Launch the App

launch_app(script_path="/path/to/project/main.py")

or for module mode:

launch_app(module="myapp.main", working_dir="/path/to/project")

Wait for success: true before proceeding. If success: false , check get_app_status for stderr.

2. Discover Widgets

find_widgets("*") → lists all named widgets list_all_widgets() → lists everything including unnamed, with coordinates list_actions() → lists all menu/toolbar actions

Use discovery before writing the test scenario — it reveals what's actually available.

3. Interact

click_widget("calculate_btn") wait_for_idle() → let Qt process the click event type_text("42", widget_name="input_field") press_key("Enter")

Always call wait_for_idle() after actions that trigger async processing or animations.

4. Verify State

get_widget_info("result_label") → check text, visibility, enabled capture_screenshot() → visual confirmation

5. Close

close_app()

Typical Visual Test Session

1. launch_app(script_path="main.py")
2. find_widgets("*") → discover widget names
3. click_widget("open_file_btn")
4. wait_for_idle()
5. capture_screenshot() → save to tests/reports/
6. get_widget_info("file_path_label") → verify file loaded
7. type_text("42", widget_name="amount_input")
8. click_widget("calculate_btn")
9. wait_for_idle()
10. get_widget_info("result_display") → assert result
11. capture_screenshot() → document final state
12. close_app()

Common Failure Modes

Symptom Likely Cause Fix

launch_app returns success: false

Import error in app, missing dependency Check stderr in get_app_status

Widget not found by name setObjectName() not called Add names to widgets; use list_all_widgets for coords

Connection refused App crashed after launch Call get_app_status to see exit code + stderr

Click has no effect Event not processed yet Add wait_for_idle() after click

Screenshot is black Xvfb not running / display not set Check prerequisites with scripts/check-prerequisites.sh

Writing a Markdown Test Report

The gui-tester agent saves reports to tests/reports/gui-YYYY-MM-DD-HH-MM.md .

Full report format and template — see references/test-report-template.md.

Additional Resources

• references/mcp-tools-reference.md — Full argument types, return schemas, and error handling for all 15 MCP tools

Examples

• examples/visual_test_session.py — Annotated walkthrough of a complete Qt Pilot session: launch → discover → interact → verify → screenshot → close → report