Name: scale-ai
Rating: 56.8 (1 reviews)
Author: membranedev

Blog Docs Log in Get started

scale-ai

Scale AI integration. Manage Organizations, Users, Goals. Use when the user wants to interact with Scale AI data.

Quality

65%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./skills/scale-ai/SKILL.md

Quality

Content

67%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

The content is highly actionable with executable CLI commands and a well-structured, state-driven connection workflow. Its main weakness is conciseness — background explanation and repeated guidance add tokens Claude does not need.

Suggestions

Trim the opening background paragraph ("Scale AI is a data annotation platform...") and the motivational auth-plumbing sentence; assume Claude knows the platform.

De-duplicate the `membrane action list` discovery command, which appears nearly identically in both "Searching for actions" and "Popular actions".

Resolve the dangling "Step 2" reference by adding a matching header or rewording the connection flow so forward references point to labeled sections.

Dimension	Reasoning	Score
Conciseness	The body opens by explaining what Scale AI is and who uses it, restates the `action list` discovery command in two places, and includes motivational phrasing ("so you can focus on the integration logic rather than auth plumbing") that Claude does not need.	3 / 5
Actionability	It provides concrete, copy-paste CLI commands for install, login, connection, action search/run, and proxying, plus a flags table; minor gaps are placeholder values and the "Popular actions" section pointing to discovery rather than listing concrete Scale AI actions.	4 / 5
Workflow Clarity	The connection setup is sequenced with explicit state checkpoints (READY / BUILDING / CLIENT_ACTION_REQUIRED / CONFIGURATION_ERROR) and a re-poll feedback loop, but the referenced "Step 2" has no matching header and there is no validation after running actions.	4 / 5
Progressive Disclosure	No bundle files exist, and the single SKILL.md is organized into clear sections covering the full integration flow; it stays a 4 rather than 5 because the proxy flag table and detailed clientAction states are inlined where a separate reference could keep the overview leaner.	4 / 5
	Total	15 / 20 Passed

Description

62%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description names a distinct platform and includes an explicit Use-when trigger, satisfying the what/when requirement. Its weakness is generic action verbs and thin trigger-term coverage that limit specificity and trigger quality.

Suggestions

Replace the generic verb "Manage" with concrete actions, e.g., "Create and list Organizations, Users, and Goals; query Datasets and Model Runs."

Broaden the Use-when clause with concrete trigger phrases users would say, e.g., "Use when the user wants to query Scale AI datasets, annotations, model runs, organizations, users, or goals."

Add the file/record-level terms users actually reference (datasets, rows, annotations, model runs) to improve natural keyword coverage.

Dimension	Reasoning	Score
Specificity	"Manage Organizations, Users, Goals" names the domain and three entities but applies a single generic verb ("Manage") rather than concrete actions like create/list/update, so it matches the anchor naming domain + actions without comprehensive coverage.	3 / 5
Completeness	It states a clear "what" ("Manage Organizations, Users, Goals") and an explicit "Use when..." clause, but the "when" is generic ("interact with Scale AI data") rather than tied to concrete trigger phrases.	4 / 5
Trigger Term Quality	The trigger relies on "Scale AI" and "Scale AI data" — relevant natural terms, but it omits common task-level variations users would actually say (e.g., datasets, annotations, model runs, labeling).	3 / 5
Distinctiveness Conflict Risk	Naming the specific "Scale AI" platform gives it a clear niche with minimal conflict risk, but "Manage Organizations, Users, Goals" is generic enough to lightly overlap with other SaaS-admin skills.	4 / 5
	Total	14 / 20 Passed

Validation

93%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

Criteria	Description	Result
frontmatter_unknown_keys	Unknown frontmatter key(s) found; consider removing or moving to metadata	Warning

	Total	15 / 16 Passed

Repository: membranedev/application-skills
Path: skills/scale-ai/SKILL.md
Commit: f484c82

Reviewed: 4 days ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.