The Three-Layer Agent Stack

Overview

A framework for building effective AI agents by synchronizing innovation across three distinct layers: Model, API, and Harness. Success requires tight integration—not treating the model as a black box.

Core principle: Features like "compaction" (long-running tasks) require simultaneous changes across all three layers.

The Stack

┌─────────────────────────────────────────────────────────────────┐ │ LAYER 3: HARNESS / PRODUCT LAYER │ │ ───────────────────────────────────────────────────────────── │ │ The environment that executes actions and provides context │ │ • VS Code / IDE integration │ │ • Terminal / Shell access │ │ • Sandbox / Secure execution environment │ ├─────────────────────────────────────────────────────────────────┤ │ LAYER 2: API LAYER │ │ ───────────────────────────────────────────────────────────── │ │ Interface handling state, context windows, and orchestration │ │ • Context management / Compaction │ │ • State handoff between sessions │ │ • Tool routing and formatting │ ├─────────────────────────────────────────────────────────────────┤ │ LAYER 1: MODEL LAYER │ │ ───────────────────────────────────────────────────────────── │ │ Foundation model providing reasoning and intelligence │ │ • Code generation / Reasoning │ │ • Summarization for compaction │ │ • Environment-specific training │ └─────────────────────────────────────────────────────────────────┘

Key Principles

Principle Description

Full-Stack Iteration Changes often need Model + API + Harness together

Harness Specificity Models perform best when trained for specific environments

Feedback Loops Product usage (Harness) must inform model training

Safety Sandboxing Harness provides secure environment for code execution

Common Mistakes

• Model-only optimization: Changing model without adapting harness

• Generic API assumptions: Assuming generic API supports agentic behaviors

• No feedback loop: Harness doesn't feed back to model training

Real-World Example

Implementing "Compaction" to allow Codex to run 24 hours:

• Model: Must understand summarization

• API: Must handle the context handoff

• Harness: Must prepare and format the payload

Source: Alexander Embiricos (OpenAI Codex) via Lenny's Podcast