Learn From PR

Extracts lessons learned from a completed PR to improve repository documentation and agent capabilities.

Inputs

Outputs

1. Learning Analysis - Structured markdown with:

   • What happened (problem, attempts, solution)
   • Fix location analysis (attempted vs actual)
   • Failure modes identified
   • Prioritized recommendations

2. Actionable Recommendations - Each with:

   • Category, Priority, Location, Specific Change, Why It Helps

Completion Criteria

The skill is complete when you have:

• Gathered PR diff and metadata
• Analyzed fix location (attempted vs actual)
• Identified failure modes
• Generated at least one concrete recommendation
• Presented findings to user

When to Use

• After agent failed to find the right fix
• After agent succeeded but took many attempts
• After agent succeeded quickly (to understand what worked)
• When asked "what can we learn from PR #XXXXX?"

When NOT to Use

• Before PR is finalized (use pr-finalize first)
• For trivial PRs (typo fixes, simple changes)
• When no agent was involved (nothing to analyze)

Workflow

Step 1: Gather Data

# Required: Get PR info
gh pr view XXXXX --json title,body,files
gh pr diff XXXXX

Analyze the PR to extract learning:

1. PR discussion - Comments reveal what was tried
2. Commit history - Multiple commits may show iteration
3. Code complexity - Non-obvious fixes suggest learning opportunities
4. Similar past issues - Search for related bugs

Focus on: "What would have helped an agent find this fix faster?"

Step 2: Fix Location Analysis

Critical question: Did agent attempts target the same files as the final fix?

# Where did final fix go?
gh pr view XXXXX --json files --jq '.files[].path' | grep -v test

If different files: Answer these questions:

• Why did agent think that was the right file?
• What search would have found the correct file?

Step 3: Analyze Outcome

Determine which scenario applies and look for the relevant patterns:

Scenario A: Agent Failed

Scenario B: Agent Succeeded Slowly (many attempts)

Key question: What would have gotten agent to the solution faster?

Scenario C: Agent Succeeded Quickly

Key question: What made this work? Should we reinforce this pattern?

Step 4: Find Improvement Locations

# Discover where agent guidance lives
find .github/instructions -name "*.instructions.md" 2>/dev/null
find .github/skills -name "SKILL.md" 2>/dev/null
ls docs/design/ 2>/dev/null
ls .github/copilot-instructions.md 2>/dev/null

Step 5: Generate Recommendations

For each recommendation, provide:

1. Category: Instruction file / Skill / Architecture doc / Inline comment / Linting issue
2. Priority: High (prevents class of bugs) / Medium (helps discovery) / Low (nice to have)
3. Location: Exact file path
4. Specific Change: Exact text to add
5. Why It Helps: Which failure mode it prevents

Prioritization factors:

• How common is this pattern?
• Would future agents definitely hit this again?
• How hard is it to implement?

Pattern-to-Improvement Mapping (Failures/Slow Success):

Pattern-to-Improvement Mapping (Quick Success - reinforce):

Step 6: Present Findings

Present your analysis covering:

• What happened and what made it hard
• Where agent looked vs actual fix location
• Which patterns applied and evidence
• Prioritized recommendations with full details (category, priority, location, exact change, why it helps)

Error Handling

Constraints

• Analysis only - Don't apply changes (use learn-from-pr agent for that)
• Actionable recommendations - Every recommendation must have specific file path and text
• Don't duplicate - Check existing docs before recommending new ones
• Focus on high-value learning - Skip trivial observations
• Respect PR scope - Don't recommend improvements unrelated to the PR's learnings

Examples

Example: Wrong File Entirely

PR #33352 - TraitCollectionDidChange crash on MacCatalyst

What happened:

• Issue title: "ObjectDisposedException in ShellSectionRootRenderer"
• Agent made 11 attempts, ALL in ShellSectionRootRenderer.cs
• Actual fix was in PageViewController.cs

Failure Mode: Trusted issue title instead of searching for pattern.

Recommendation:

• Category: Instruction File
• Location: .github/instructions/ios-debugging.instructions.md
• Change: "When fixing iOS crashes, search for the PATTERN across all files, not just the file named in the error"
• Why: Prevents tunnel vision on named file

Example: Slow Success

PR #34567 - CollectionView scroll position not preserved

What happened:

• Agent took 5 attempts to find fix
• First 3 attempts were in wrong layer (handler vs core)
• Eventually found it after reading more context
• Final fix was simple once the right layer was identified

Pattern: Wrong abstraction layer - fixed handler when problem was in core.

Recommendation:

• Category: Architecture Doc
• Location: .github/architecture/handler-vs-core.md
• Change: Document layer responsibilities - handlers map properties, core handles behavior
• Why: Helps agent identify correct layer faster

Example: Quick Success

PR #35678 - Button disabled state not updating

What happened:

• Agent found fix in 1 attempt
• Searched for "IsEnabled" pattern across codebase immediately
• Found similar past fix in another control and applied same approach
• Simple, minimal change

Pattern: Good search strategy - recognized pattern from similar code.

Recommendation:

• Category: Skill Enhancement
• Location: .github/skills/try-fix/SKILL.md
• Change: Add to search strategy: "Search for same property pattern in other controls"
• Why: Reinforces successful discovery technique

Integration

• pr-finalize → Use first to verify PR is ready
• learn-from-pr skill → Analysis only (this skill)
• learn-from-pr agent → Analysis + apply changes

For automated application of recommendations, use the learn-from-pr agent instead.