predictor-hand-skill

Expert knowledge for AI forecasting — superforecasting principles, signal taxonomy, confidence calibration, reasoning chains, and accuracy tracking

Quality

67%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Low

Low-risk findings worth noting

Fix and improve this skill with Tessl

tessl review fix ./crates/openfang-hands/bundled/predictor/SKILL.md

Quality

Content

77%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is a well-structured, actionable forecasting reference with concrete templates, named sources, and a clear reasoning workflow including a scoring feedback loop. Its main weaknesses are mild verbosity around concepts Claude already knows and a monolithic single-file structure that exceeds the 'simple skill' threshold without external references.

Suggestions

Trim explanations of concepts Claude already knows (e.g., the Tetlock/GJP attribution and basic bias definitions) to keep the token budget lean.

Split the large domain source-guide tables and prediction-tracking templates into reference files (e.g., SOURCES.md, TEMPLATES.md) and link to them from SKILL.md to improve progressive disclosure.

Add one fully worked example of the reasoning-chain template (with real values filled in) to complement the placeholder version.

Dimension	Reasoning	Score
Conciseness	Largely dense high-signal reference material (tables, templates, terse checklists), but includes some concepts Claude already knows such as the Tetlock attribution and basic bias definitions like 'Anchoring: Am I fixated on the first number'.	2 / 3
Actionability	Provides concrete, copy-paste-ready artifacts — a fill-in reasoning-chain template, a prediction-ledger JSON schema, an accuracy-report template, named data sources per domain (FRED, SEC, IPCC), and numeric calibration rules ('±5-15% per strong signal').	3 / 3
Workflow Clarity	The reasoning chain lays out a clear 5-step sequence (reference class → specific evidence → synthesis → key assumptions → resolution) and the prediction lifecycle includes an explicit feedback loop via Brier scoring and the accuracy dashboard.	3 / 3
Progressive Disclosure	The body is well-organized into clear sections, but at ~265 lines it is monolithic with no bundle files and no one-level-deep references; content such as domain source guides and templates remains inline rather than split into navigable references.	2 / 3
	Total	10 / 12 Passed

Description

57%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description clearly establishes a specialized forecasting niche and enumerates its knowledge components in third person, but it states knowledge categories rather than concrete actions and lacks any 'Use when...' trigger guidance. Adding explicit when-to-use triggers and plain-language verbs would lift the completeness and trigger-term scores.

Suggestions

Add an explicit 'Use when...' clause naming natural trigger phrases (e.g., 'Use when the user asks you to forecast, estimate the odds, or calibrate confidence on a prediction').

Replace knowledge-category phrasing with concrete actions (e.g., 'make calibrated probability forecasts, decompose questions, and track prediction accuracy').

Include plainer trigger terms a user would actually say ('predict', 'forecast', 'likelihood', 'how confident') alongside the technical ones.

Dimension	Reasoning	Score
Specificity	Names the AI forecasting domain and enumerates concrete knowledge areas ('superforecasting principles, signal taxonomy, confidence calibration, reasoning chains, and accuracy tracking'), but describes knowledge categories rather than explicit actions like 'make predictions' or 'calibrate probabilities'.	2 / 3
Completeness	Clearly answers 'what' via the enumerated knowledge areas, but entirely omits a 'when'/'Use when...' trigger clause, capping completeness at 2 per the rubric guideline.	2 / 3
Trigger Term Quality	Includes genuine relevant terms ('AI forecasting', 'superforecasting') but leans technical ('signal taxonomy', 'confidence calibration') and misses common natural variations a user would say such as 'predict', 'estimate the odds', or 'how confident should I be'.	2 / 3
Distinctiveness Conflict Risk	The AI forecasting/superforecasting niche is specialized and distinct, making it unlikely to trigger for unrelated document, code, or data skills.	3 / 3
	Total	9 / 12 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: RightNow-AI/openfang
Path: crates/openfang-hands/bundled/predictor/SKILL.md
Commit: acf2587

Reviewed: about 8 hours ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.