skill-improvement-from-observability

Complete feedback loop from observability insights to skill updates. Use when analyzing enhanced telemetry patterns and automatically improving skills.

Quality

63%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./.claude/skills/skill-improvement-from-observability/SKILL.md

The canonical home for this skill is skill-improvement-from-observability in fernandezbaptiste/Skrillz

Skill Improvement from Observability

Name: skill-improvement-from-observability
Rating: 56.00000000000001 (1 reviews)
Author: fernandezbaptiste

The Self-Improvement Loop: Enhanced Telemetry → Pattern Analysis → Skill Updates → Better Performance

Data Source

Primary: {job="claude_code_enhanced"} in Loki (from enhanced-telemetry hooks)

Workflow

1. Collect Observability Insights

Use observability-analyzer with enhanced telemetry:

# Session analytics
{job="claude_code_enhanced", event_type="session_end"} | json

# Error patterns
{job="claude_code_enhanced", event_type="tool_result", status="error"} | json

# Tool sequences
{job="claude_code_enhanced", event_type="tool_call"} | json

# Prompt patterns
{job="claude_code_enhanced", event_type="user_prompt"} | json

2. Run Pattern Detection

Use observability-pattern-detector operations:

detect-failures → Error patterns by tool
detect-tool-sequences → Inefficient tool chains
detect-conversation-patterns → User behavior insights
detect-context-issues → Context management problems
detect-waste → Redundant operations

3. Extract Actionable Patterns

Filter high-impact issues from enhanced telemetry:

Error Analysis:

sum by (tool, error_type) (count_over_time({job="claude_code_enhanced", event_type="tool_result", status="error"} | json [7d]))

Tool Inefficiency:

# Repeated Read→Read patterns (waste)
{job="claude_code_enhanced", event_type="tool_call"} | json | previous_tool="Read" and tool_name="Read"

Context Issues:

# Auto compaction frequency
count_over_time({job="claude_code_enhanced", event_type="context_compact", trigger="auto"} [7d])

4. Map Patterns to Skills

Pattern	Likely Skill	Action
Bash command errors	bash-related skills	Add existence checks
File not found	file operation skills	Add path validation
Repeated Glob→Read	search skills	Optimize file discovery
High context usage	context-heavy skills	Add chunking
Many debugging prompts	core skills	Improve error messages

5. Generate Improvement Recommendations

Based on enhanced telemetry patterns:

{
  "improvement": {
    "pattern": "File not found errors",
    "occurrences": 45,
    "source_query": "{job=\"claude_code_enhanced\", event_type=\"tool_result\", status=\"error\"} | json | error_type=~\".*not found.*\"",
    "affected_skills": ["file-operations"],
    "recommendation": "Add file existence check before Read/Edit operations",
    "implementation": "Add pathlib.Path(file).exists() check",
    "priority": "high",
    "expected_impact": "Reduce errors by 80%"
  }
}

6. Track Effectiveness

After improvements deployed, measure:

# Before vs After error rates
sum(count_over_time({job="claude_code_enhanced", event_type="tool_result", status="error"} | json [7d]))

# Tool success rate improvement
sum(count_over_time({job="claude_code_enhanced", event_type="tool_result", status="success"} | json [7d])) /
sum(count_over_time({job="claude_code_enhanced", event_type="tool_result"} | json [7d]))

Example Improvement Flows

Flow 1: Error Reduction

Telemetry: "npm not found" × 45 in tool_result errors
    ↓
Pattern: Bash tool failures with npm commands
    ↓
Recommendation: Add npm availability check
    ↓
skill-updater applies changes
    ↓
Telemetry tracks: npm errors = 0 after deployment
    ↓
Result: ✅ 100% reduction

Flow 2: Context Optimization

Telemetry: Auto-compaction triggered 12 times in 7 days
    ↓
Pattern: Large file reads accumulating tokens
    ↓
Recommendation: Add file chunking for large reads
    ↓
skill-updater applies changes
    ↓
Telemetry tracks: Auto-compactions = 2 after deployment
    ↓
Result: ✅ 83% reduction

Flow 3: Tool Sequence Optimization

Telemetry: Glob→Read→Glob→Read pattern 89 times
    ↓
Pattern: Redundant file discovery
    ↓
Recommendation: Cache glob results within session
    ↓
skill-updater applies changes
    ↓
Telemetry tracks: Redundant glob reduced by 70%
    ↓
Result: ✅ Faster file operations

Key Queries for Improvement Analysis

High-Impact Errors

topk(10, sum by (tool, error_type) (count_over_time({job="claude_code_enhanced", event_type="tool_result", status="error"} | json [7d])))

Session Quality Issues

# High error sessions
{job="claude_code_enhanced", event_type="session_end"} | json | error_count > 5

# Low productivity sessions (high turns, few tool calls)
{job="claude_code_enhanced", event_type="session_end"} | json | turn_count > 20 and tools_used < 5

Tool Efficiency

# Tool usage distribution
sum by (tool) (count_over_time({job="claude_code_enhanced", event_type="tool_call"} | json [7d]))

# Error rate by tool
sum by (tool) (count_over_time({job="claude_code_enhanced", event_type="tool_result", status="error"} | json [7d])) /
sum by (tool) (count_over_time({job="claude_code_enhanced", event_type="tool_result"} | json [7d]))

User Behavior Insights

# Prompt pattern trends
sum by (pattern) (count_over_time({job="claude_code_enhanced", event_type="user_prompt"} | json [7d]))

# Debugging frequency (indicates pain points)
count_over_time({job="claude_code_enhanced", event_type="user_prompt", pattern="debugging"} [7d])

Integration with Ecosystem

Uses existing skills:

observability-analyzer: Query enhanced telemetry data
observability-pattern-detector: Detect improvement patterns
skill-updater: Apply safe improvements
review-multi: Validate changes
skill-tester: Regression testing
enhanced-telemetry: Source of all observability data

Safety Classification

Auto-Apply Safe:

✅ Adding existence checks
✅ Adding error handling
✅ Improving error messages
✅ Updating documentation
✅ Adding validation

Require Review:

❌ Changing core logic
❌ Modifying APIs
❌ Removing functionality
❌ Changing data structures

Effectiveness Metrics

Track improvement success:

# Calculate error reduction percentage
(before_errors - after_errors) / before_errors * 100

# Track pattern elimination
count_over_time({job="claude_code_enhanced"} | json | <pattern_filter> [7d])

Report format:

{
  "improvement_id": "file-existence-check",
  "deployed": "2025-11-27",
  "before_errors": 45,
  "after_errors": 2,
  "reduction_percent": 95.6,
  "status": "successful"
}

Repository: fernandezbaptiste/Skrillz
Path: .claude/skills/skill-improvement-from-observability/SKILL.md
Commit: 93ed392

Last updated: 1 day ago
First committed: 8 months ago

Canonical home

fernandezbaptiste/Skrillz

In sync

since Sep 12, 2026

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.