Windows Automation Skill

Automate Windows desktop interactions using PyAutoGUI

Overview

This skill provides comprehensive Windows desktop automation capabilities. Use it when you need to:

• Simulate mouse clicks and keyboard input
• Automate GUI interactions in non-web applications
• Capture screenshots
• Manage clipboard content
• Launch applications
• Run system commands

Setup

The required packages are already installed:

• pyautogui - Core automation engine
• pywin32 - Windows API access
• pyperclip - Clipboard operations

If needed, install manually:

pip install pyautogui pywin32 pyperclip

Tools

🖱️ Mouse Control

windows_click

Click at specified screen coordinates.

windows_click(x=100, y=200, button="left", clicks=1)

Parameters:

windows_double_click

Double click at position.

windows_double_click(x=500, y=300)

windows_right_click

Right click at position.

windows_right_click(x=100, y=200)

windows_move

Move mouse to position.

windows_move(x=800, y=600, duration=0.5)

Parameters:

⌨️ Keyboard Control

windows_type

Type text using keyboard.

windows_type(text="Hello World!", interval=0.05)

Parameters:

windows_press

Press a single key.

windows_press(key="enter")

Supported keys:

• Function keys: f1 - f24
• Special keys: enter, esc, tab, space, backspace, delete
• Arrow keys: up, down, left, right
• Modifiers: ctrl, alt, shift, win

windows_hotkey

Press keyboard shortcut combination.

windows_hotkey("ctrl", "c")   # Copy
windows_hotkey("ctrl", "v")   # Paste
windows_hotkey("alt", "f4")   # Close window
windows_hotkey("win", "r")    # Open Run dialog

📸 Screen Operations

windows_screenshot

Take a screenshot.

windows_screenshot(filename="screenshot.png")

Parameters:

ℹ️ Information

windows_cursor_position

Get current cursor position.

windows_cursor_position()
# Returns: {"x": 500, "y": 300}

windows_screen_size

Get screen resolution.

windows_screen_size()
# Returns: {"width": 1920, "height": 1080}

windows_system_info

Get basic system information.

windows_system_info()
# Returns: {
#   "platform": "Windows",
#   "version": "10.0.22631",
#   "screen_size": {"width": 1920, "height": 1080},
#   "cursor_position": {"x": 500, "y": 300}
# }

🪟 Applications

windows_launch_app

Launch a Windows application.

windows_launch_app(name="notepad")
windows_launch_app(name="calc")
windows_launch_app(name="excel")
windows_launch_app(name="C:\\Program Files\\App\\app.exe")

💻 System

windows_command

Run a PowerShell or cmd command.

windows_command(command="ipconfig")
windows_command(command="Get-Process | Select-Object -First 5")
windows_command(command="dir C:\\")

📋 Clipboard

windows_clipboard_read

Read clipboard content.

windows_clipboard_read()
# Returns: "clipboard content"

windows_clipboard_write

Write text to clipboard.

windows_clipboard_write(text="Hello from OpenClaw!")

Usage Examples

Example 1: Open Notepad and Type Text

User: Open notepad and type hello
→ windows_launch_app with name="notepad"
→ windows_type with text="Hello World!"

Example 2: Fill a Form

User: Click on the name field and enter John
→ windows_click x=300 y=200
→ windows_type text="John"
→ windows_press key="tab"
→ windows_type text="john@example.com"

Example 3: Screenshot and Save

User: Take a screenshot
→ windows_screenshot filename="screen_20260101.png"

Example 4: Copy File Path

User: Copy C:\Documents\file.txt to clipboard
→ windows_clipboard_write text="C:\\Documents\\file.txt"

Example 5: Get System Info

User: What's my screen resolution?
→ windows_screen_size

Safety Notes

• ⚠️ The mouse failsafe is disabled for smoother operation
• 📍 Screen coordinates start at (0, 0) in top-left corner
• 🖥️ Multi-monitor setups may have different coordinate systems
• 🔒 Some applications may block simulated input (games, secure apps)

Dependencies

Built for OpenClaw 🦞