Uncapped, sitemap-complete crawl of a Shopify storefront using the merchant's Crawler Access signature from a project-local .shopify-env, feeding the standard seo-audit pipeline. Use when the user says 'Shopify audit', 'crawl the whole store', 'Crawler access', 'signature-agent', 'web-bot-auth', 'rate limited while crawling', or audits a host listed in .shopify-env.
90%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
Low
Low-risk findings worth noting
Shopify rate-limits storefront crawling, and seo-audit caps its link-following
crawl at 500 pages. A merchant can mint a Crawler Access signature in their
admin (Online Store > Preferences > Crawler access). A crawler that replays the
three resulting headers is treated as a crawler the merchant authorized; Shopify's
help page says rate-limit errors on a signed crawl mean the signature is invalid.
Shopify also publishes a complete nested sitemap for every store. Together that
makes an uncapped, sitemap-complete crawl possible without guessing at a cap.
This skill adds that crawl. It replaces step 3 only of seo-audit (the crawl).
Business-type detection, subagent delegation, scoring and reports stay as
seo-audit defines them.
seo-audit.shop.example, www.shop.example and
shop.myshopify.com are different authorities; crawl the host the signature was
issued for. The crawler never sends it to any other host.| Command | Effect |
|---|---|
/seo shopify <url> | Precheck, signed uncapped crawl, then the seo-audit pipeline on the artifacts |
/seo shopify crawl <url> | Crawl only, artifacts in ./crawl |
/seo shopify check <url> | Report whether .shopify-env holds a valid signature for the host |
"${CLAUDE_PLUGIN_ROOT}/scripts/claude-seo" run shopify_env.py precheck <url>Always exits 0 and always prints JSON. Read mode:
"signed": continue with step 2."fallback": run the plain seo-audit skill, completely unchanged, cap and
link-following crawl included. Tell the user in one sentence why (the detail
field says it) and get on with the audit. A capped audit is a normal result.Fallback reasons: no_env_file, no_signature, signature_invalid,
signature_expired, insecure_scheme, env_unreadable. Before accepting
no_signature, remember the signature is bound to one host: if the store serves on
the other of apex/www, re-run the precheck against that host.
signature_invalid means the block is in the file but cannot be used: its
Signature-Input lacks the web-bot-auth tag, @authority, a key id or an
expiry, or a header value is malformed. The block is never sent and gives no
higher limits. Tell the user to copy the three values again from the Shopify admin.
"${CLAUDE_PLUGIN_ROOT}/scripts/claude-seo" run shopify_crawl.py <url> --out ./crawlNo --max-pages. A signed crawl covers the complete sitemap; a cap applies only when
.shopify-env or the command line sets one (an unsigned crawl defaults to 500
pages), and the summary then carries
"truncated": true. Report the real number of pages crawled, never a rounded cap.
Exit codes:
0: crawl complete, continue.5: the storefront rate limited the crawl. Read summary.json:
"aborted": null: the crawl finished, but a signed request was rate limited.
Shopify treats that as an invalid signature. Continue the audit on the
artifacts and flag in the report that the signature should be reissued."aborted": "rate_limited": the crawl stopped early after 8 rate-limit
responses in a row (429, 430 or 503, each already followed by a longer
backoff). The artifacts are partial. Do not retry in a loop: tell the user,
resume later with --resume, or audit the partial crawl and say it is partial.2: bad arguments, or --include/--exclude matched no discovered URL. Fix the
pattern; this is not a reason to fall back.1: nothing crawled (no sitemap reachable, password-protected store, wrong
host, robots.txt unreadable or disallowing everything, or the URL failed
url_safety). Fall back to plain seo-audit and say why. When the message says
the sitemap redirects to another host (a store that sends apex to www), that
host is the one to audit: redirects are not followed and the signature is bound
to one host, so re-run the precheck against the host named in the message.Large stores: the crawl is resumable (--resume). If it was interrupted, resume it
instead of auditing a partial crawl. Failed and rate-limited URLs are fetched again.
Defaults: no page limit, concurrency 6 and a 0.2 s delay when signed; 500 pages,
concurrency 2 and 1 s unsigned (concurrency is bounded to 1-16, delay to 0-60 s);
never signed over plain http://; redirects recorded, not followed; bodies read up
to 10 MiB per page and the sitemap walk bounded at 5 levels, 500 sitemaps and 250,000
URLs (discovery_capped in summary.json names the cap that was hit). Precedence is
command line > domain block > global keys > these defaults, with one exception: for
an unsigned crawl .shopify-env can only lower the page cap or concurrency or
lengthen the delay. Lifting the unsigned limits takes an explicit command-line flag.
Useful flags: --include/--exclude (regex), --save-html,
--sample-per-template N, --no-signature to measure the unsigned baseline.
robots.txt is respected by default, signed or not: disallowed URLs are skipped, a
longer Crawl-delay is applied, and a robots.txt that answers with a server error
stops the crawl. A Crawler Access signature authorizes the crawler; it does not
override robots.txt. The only way to change that is the --ignore-robots
command-line flag, which crawls disallowed URLs too and skips Crawl-delay.
.shopify-env cannot set it. Never add --ignore-robots on your own: use it only
when the user, who administers the store, asks for it, and say so in the report.
Follow seo-audit from its step 4 onwards. The crawl is done; nobody crawls again.
Business type is settled: a Shopify storefront is E-commerce, so seo-ecommerce is
in scope.
What each subagent gets:
crawl/summary.json: aggregate counts across all crawled pages. The evidence
base for anything quantitative: status distribution, per-template counts,
missing and duplicate titles, meta description lengths, missing canonicals,
noindex, h1 problems, thin pages, images without alt, schema coverage, latency
percentiles.crawl/sample.json: a stratified sample, N pages per template type, for the
agents that need real page content (seo-content, seo-schema, seo-sxo,
seo-ecommerce).crawl/pages.jsonl: never passed to a subagent. It is the raw record set and
will not fit in a context window on any store worth crawling uncapped. Query it
with a script when a specific number is missing from summary.json.Say in each subagent prompt that the pages were already fetched, so they analyze
the artifacts instead of re-fetching the site. Agents that need live requests
(seo-performance, seo-visual, seo-google) make their handful of unsigned
requests as usual.
Unchanged from seo-audit: same weights, same FULL-AUDIT-REPORT.md and
ACTION-PLAN.md. Two additions to the executive summary, because they change how
the numbers read: the pages crawled and that the crawl was signed and complete,
and any truncation with what was left out.
See extensions/shopify/docs/SHOPIFY-SETUP.md for issuing the signature and the
.shopify-env format. .shopify-env holds a client credential: keep it out of git.
shopify_env.py and
shopify_crawl.py never output it, and the crawl artifacts never contain it.http:// and never to another host. Do not work around that by rewriting URLs,
following redirects by hand, or copying the headers into another tool..shopify-env is a file in the project folder, so it is not proof of anyone's
consent: it cannot switch robots.txt handling off and cannot raise the limits of
an unsigned crawl.url_safety (validated root, pinned DNS, same origin
only, redirects recorded and not followed). The sitemap walk and every response
body are bounded, and Concurrency and Delay are clamped, so a hostile sitemap
or env file cannot turn the crawl into an unbounded fetch..shopify-env must stay out of git. If the crawl warns that it is not
git-ignored, tell the user before doing anything else.AgriciDaniel/claude-seo
468ed5b
Also appears in
since Nov 15, 2026
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.