Cancel Async Tasks

Overview

This skill provides guidance for implementing robust asyncio task cancellation in Python, particularly when dealing with signal handling (SIGINT/KeyboardInterrupt), semaphore-based concurrency limiting, and ensuring proper cleanup of all tasks including those waiting in queues.

Key Concepts

Signal Propagation in Asyncio

Understanding how signals interact with asyncio is critical:

KeyboardInterrupt vs CancelledError: When SIGINT is received during asyncio.run() , the behavior differs from catching exceptions inside async code. The event loop typically converts the interrupt to CancelledError that propagates through tasks.

Signal handler context: Signal handlers run in the main thread, but asyncio tasks may be in various states (running, waiting on semaphore, waiting on I/O).

Event loop state: The event loop's handling of SIGINT depends on whether it's running asyncio.run() vs manual loop management.

Task Lifecycle States

When cancellation occurs, tasks can be in different states:

• Running tasks: Currently executing code

• Awaiting tasks: Blocked on I/O or other coroutines

• Semaphore-waiting tasks: Waiting to acquire a semaphore for concurrency limiting

• Not-yet-started tasks: Created but not yet scheduled

Each state requires different handling for proper cleanup.

Potential Approaches

Approach 1: Task Group with Exception Handling

Use asyncio.TaskGroup (Python 3.11+) for automatic cancellation propagation:

• TaskGroup automatically cancels remaining tasks when one fails

• Provides structured concurrency guarantees

• Consider whether this matches the cleanup requirements

Approach 2: Manual Task Tracking with Shield

Track all task objects explicitly and handle cancellation:

• Maintain a list of all created task objects

• Use asyncio.shield() for cleanup operations that must complete

• Implement explicit cancellation loop for all tracked tasks

Approach 3: Signal Handler Registration

Register explicit signal handlers for SIGINT/SIGTERM:

• Use loop.add_signal_handler() to register custom handlers

• Set a cancellation flag or event that tasks check

• Coordinate shutdown through the event loop

Approach 4: Context Manager Pattern

Wrap task execution in a context manager that handles cleanup:

• aenter sets up tasks and tracking

• aexit ensures all tasks are cancelled and awaited

• Handles exceptions uniformly

Verification Strategies

Testing with Real Signals

Critical: Test with actual signals, not timeouts:

Correct approach: Use subprocess with actual SIGINT

import subprocess import signal import time

proc = subprocess.Popen(['python', 'script.py']) time.sleep(1) # Let tasks start proc.send_signal(signal.SIGINT) stdout, stderr = proc.communicate(timeout=5)

Verify cleanup messages in output

Incorrect approach (gives false confidence):

• Using asyncio.wait_for() with timeout does not replicate SIGINT behavior

• Using asyncio.CancelledError directly differs from signal-triggered cancellation

Verification Checklist

• Running task cleanup: Verify tasks actively executing receive cancellation

• Waiting task cleanup: Verify tasks blocked on I/O are cancelled

• Semaphore queue cleanup: Verify tasks waiting on semaphore acquisition are cancelled

• Cleanup code execution: Verify finally blocks and cleanup handlers run

• No resource leaks: Verify file handles, connections, etc. are closed

• Exit code verification: Verify process exits with expected code after interrupt

Test Scenarios to Cover

• Interrupt when all slots are filled (max_concurrent tasks running)

• Interrupt when tasks are queued waiting for semaphore

• Interrupt during cleanup phase itself

• Rapid repeated interrupts

• Interrupt before any task starts

Common Pitfalls

Pitfall 1: Catching KeyboardInterrupt Inside Async Functions

Problem: KeyboardInterrupt doesn't propagate normally through asyncio - it's typically converted to CancelledError by the event loop.

Symptom: Exception handlers for KeyboardInterrupt inside async functions never trigger during actual Ctrl+C.

Solution: Handle CancelledError instead, or register explicit signal handlers at the event loop level.

Pitfall 2: asyncio.gather Doesn't Cancel Queued Tasks

Problem: When using asyncio.gather with more tasks than can run concurrently (via semaphore), cancelling gather doesn't automatically cancel tasks waiting to acquire the semaphore.

Symptom: Tasks that haven't started don't have their cleanup code run.

Solution: Explicitly track all task objects and cancel them individually, not just rely on gather's cancellation.

Pitfall 3: Testing with Timeouts Instead of Signals

Problem: Using asyncio.wait_for() timeout to simulate interruption doesn't replicate actual signal handling behavior.

Symptom: Tests pass but actual Ctrl+C behavior differs.

Solution: Use subprocess with signal.SIGINT to test actual signal handling behavior.

Pitfall 4: Cleanup During Cancellation

Problem: Cleanup code itself may be cancelled if not protected.

Symptom: Partial cleanup, resources not released.

Solution: Use asyncio.shield() for critical cleanup operations, or handle CancelledError and re-raise after cleanup.

Pitfall 5: Duplicate Exception Handling Code

Problem: Identical cleanup code in multiple exception handlers (CancelledError , KeyboardInterrupt , etc.).

Symptom: Code duplication, maintenance burden.

Solution: Use a single handler with except (asyncio.CancelledError, KeyboardInterrupt) or abstract cleanup into a helper function.

Pitfall 6: Not Awaiting Cancelled Tasks

Problem: Cancelling a task and not awaiting it leaves the task in a partially-cleaned-up state.

Symptom: Resource leaks, warnings about pending tasks.

Solution: Always await asyncio.gather(*cancelled_tasks, return_exceptions=True) after cancelling.

Decision Framework

When implementing async task cancellation, consider:

• Python version: TaskGroup (3.11+) vs manual management

• Concurrency model: Fixed pool, semaphore-limited, or unlimited

• Cleanup requirements: What must happen before exit?

• Signal handling needs: Just SIGINT, or also SIGTERM, SIGHUP?

• Testing environment: Can tests send real signals?

Debugging Tips

• Add logging at task entry, exit, and cancellation points

• Log the task state when cancellation is received

• Use asyncio.current_task() to identify which task is executing

• Check task.cancelled() vs task.done() states

• Enable asyncio debug mode: asyncio.run(main(), debug=True)