Learn From PR

Extracts lessons learned from a completed PR to improve repository documentation and agent capabilities.

Inputs

Input Required Source

PR number or Issue number Yes User provides (e.g., "PR #33352" or "issue 33352")

Outputs

Learning Analysis - Structured markdown with:

• What happened (problem, attempts, solution)

• Fix location analysis (attempted vs actual)

• Failure modes identified

• Prioritized recommendations

Actionable Recommendations - Each with:

• Category, Priority, Location, Specific Change, Why It Helps

Completion Criteria

The skill is complete when you have:

• Gathered PR diff and metadata

• Analyzed fix location (attempted vs actual)

• Identified failure modes

• Generated at least one concrete recommendation

• Presented findings to user

When to Use

• After agent failed to find the right fix

• After agent succeeded but took many attempts

• After agent succeeded quickly (to understand what worked)

• When asked "what can we learn from PR #XXXXX?"

When NOT to Use

• Before PR is finalized (use pr-finalize first)

• For trivial PRs (typo fixes, simple changes)

• When no agent was involved (nothing to analyze)

Workflow

Step 1: Gather Data

Required: Get PR info

gh pr view XXXXX --json title,body,files gh pr diff XXXXX

Analyze the PR to extract learning:

• PR discussion - Comments reveal what was tried

• Commit history - Multiple commits may show iteration

• Code complexity - Non-obvious fixes suggest learning opportunities

• Similar past issues - Search for related bugs

Focus on: "What would have helped an agent find this fix faster?"

Step 2: Fix Location Analysis

Critical question: Did agent attempts target the same files as the final fix?

Where did final fix go?

gh pr view XXXXX --json files --jq '.files[].path' | grep -v test

Scenario Implication

Same files Agent found right location

Different files Major learning opportunity - document why

If different files: Answer these questions:

• Why did agent think that was the right file?

• What search would have found the correct file?

Step 3: Analyze Outcome

Determine which scenario applies and look for the relevant patterns:

Scenario A: Agent Failed

Pattern Indicator

Wrong file entirely All attempts in File A, fix in File B

Tunnel vision Only looked at file mentioned in error

Trusted issue title Issue said "crash in X" so only looked at X

Pattern not generalized Fixed one instance, missed others

Didn't search codebase Never found similar code patterns

Missing platform knowledge Didn't know iOS/Android/Windows specifics

Wrong abstraction layer Fixed handler when problem was in core

Misread error message Error pointed to symptom, not cause

Incomplete context Didn't read enough surrounding code

Over-engineered Complex fix when simple one existed

Scenario B: Agent Succeeded Slowly (many attempts)

Pattern Indicator

Correct file, wrong approach Found right file but tried wrong fixes first

Needed multiple iterations Each attempt got closer but wasn't quite right

Discovery was slow Eventually found it but search was inefficient

Missing domain knowledge Had to learn something that could be documented

Key question: What would have gotten agent to the solution faster?

Scenario C: Agent Succeeded Quickly

Pattern Indicator

Good search strategy Found right file immediately

Understood the pattern Recognized similar issues from past

Documentation helped Existing docs pointed to solution

Simple, minimal fix Didn't over-engineer

Key question: What made this work? Should we reinforce this pattern?

Step 4: Find Improvement Locations

Discover where agent guidance lives

find .github/instructions -name "*.instructions.md" 2>/dev/null find .github/skills -name "SKILL.md" 2>/dev/null ls docs/design/ 2>/dev/null ls .github/copilot-instructions.md 2>/dev/null

Location When to Add Here

.github/instructions/*.instructions.md

Domain-specific AI guidance (testing patterns, platform rules)

.github/skills/*/SKILL.md

Skill needs new step, checklist, or improved workflow

/docs/design/*.md

Detailed architectural documentation

.github/copilot-instructions.md

General AI workflow guidance

Code comments Non-obvious code behavior

Step 5: Generate Recommendations

For each recommendation, provide:

• Category: Instruction file / Skill / Architecture doc / Inline comment / Linting issue

• Priority: High (prevents class of bugs) / Medium (helps discovery) / Low (nice to have)

• Location: Exact file path

• Specific Change: Exact text to add

• Why It Helps: Which failure mode it prevents

Prioritization factors:

• How common is this pattern?

• Would future agents definitely hit this again?

• How hard is it to implement?

Pattern-to-Improvement Mapping (Failures/Slow Success):

Pattern Likely Improvement

Wrong file entirely Check /docs/design/ for component relationships

Tunnel vision Instruction file: "Always search for pattern across codebase"

Missing platform knowledge Platform-specific instruction file

Wrong abstraction layer Reference /docs/design/HandlerResolution.md

Misread error message Code comment explaining the real cause

Over-engineered Skill enhancement: "Try simplest fix first"

Pattern-to-Improvement Mapping (Quick Success - reinforce):

Pattern Improvement

Good search strategy Document the search pattern that worked in skills

Documentation helped Note which docs were valuable, ensure they stay updated

Recognized pattern Add to instruction files as known pattern

Step 6: Present Findings

Present your analysis covering:

• What happened and what made it hard

• Where agent looked vs actual fix location

• Which patterns applied and evidence

• Prioritized recommendations with full details (category, priority, location, exact change, why it helps)

Error Handling

Situation Action

PR not found Ask user to verify PR number

No agent involvement evident Ask user if they still want analysis

Can't determine failure mode State "insufficient data" and what's missing

Constraints

• Analysis only - Don't apply changes (use learn-from-pr agent for that)

• Actionable recommendations - Every recommendation must have specific file path and text

• Don't duplicate - Check existing docs before recommending new ones

• Focus on high-value learning - Skip trivial observations

• Respect PR scope - Don't recommend improvements unrelated to the PR's learnings

Examples

Example: Wrong File Entirely

PR #33352 - TraitCollectionDidChange crash on MacCatalyst

What happened:

• Issue title: "ObjectDisposedException in ShellSectionRootRenderer"

• Agent made 11 attempts, ALL in ShellSectionRootRenderer.cs

• Actual fix was in PageViewController.cs

Failure Mode: Trusted issue title instead of searching for pattern.

Recommendation:

• Category: Instruction File

• Location: .github/instructions/ios-debugging.instructions.md

• Change: "When fixing iOS crashes, search for the PATTERN across all files, not just the file named in the error"

• Why: Prevents tunnel vision on named file

Example: Slow Success

PR #34567 - CollectionView scroll position not preserved

What happened:

• Agent took 5 attempts to find fix

• First 3 attempts were in wrong layer (handler vs core)

• Eventually found it after reading more context

• Final fix was simple once the right layer was identified

Pattern: Wrong abstraction layer - fixed handler when problem was in core.

Recommendation:

• Category: Architecture Doc

• Location: .github/architecture/handler-vs-core.md

• Change: Document layer responsibilities - handlers map properties, core handles behavior

• Why: Helps agent identify correct layer faster

Example: Quick Success

PR #35678 - Button disabled state not updating

What happened:

• Agent found fix in 1 attempt

• Searched for "IsEnabled" pattern across codebase immediately

• Found similar past fix in another control and applied same approach

• Simple, minimal change

Pattern: Good search strategy - recognized pattern from similar code.

Recommendation:

• Category: Skill Enhancement

• Location: .github/skills/try-fix/SKILL.md

• Change: Add to search strategy: "Search for same property pattern in other controls"

• Why: Reinforces successful discovery technique

Integration

• pr-finalize → Use first to verify PR is ready

• learn-from-pr skill → Analysis only (this skill)

• learn-from-pr agent → Analysis + apply changes

For automated application of recommendations, use the learn-from-pr agent instead.