Obsidian Linked Research

Purpose

Take a URL → fetch its content → generate a structured summary → check for thesis drift against existing notes → write a standalone research note into the active Research/Library/ taxonomy in the Obsidian vault.

The library is no longer flat. Notes should be routed into the existing numbered subfolders and tagged using the vocabulary established by the master Research Library MOC and the vault tag index.

Tweet/X URLs get rich content via xAI's x_search tool (engagement stats, thread context, media descriptions). All other URLs get plain-text extraction via HTTP fetch. The VS Code model (you, the agent) does all summarization — no external LLM API call needed for the intelligence step.

Vault writes go through the obsidian skill (composable CLI wrapper).

Always-Capture Sources

The following sources bypass quality filters and are always captured as research notes when encountered — in email, X, or any other ingestion pipeline. These are high-signal sources that publish in digest/roundup format but contain original analysis worth preserving.

When a calling skill (e.g., gmail-daily-briefing) encounters content from an always-capture source, it should invoke this skill regardless of whether the content looks like a "newsletter digest" or "roundup." The quality filter exclusion for digests does NOT apply to always-capture sources.

To add more sources, append rows to the table above.

When to Use

Prerequisites

• Obsidian should be running with CLI enabled. If it is not, the skill still runs in degraded mode — see below. Do not abort the run.
• XAI_API_KEY required for tweet URLs (via keyring automation/api, env var, or ~/.config/last30days/.env)
• Web URLs work without any API key

Run on the Windows host — not a Linux sandbox

Every command in this workflow must run on the machine where Obsidian is installed. In environments that expose both a POSIX sandbox and the Windows host (Cowork, and any agent with a Linux bash tool alongside Desktop Commander), the sandbox is a different filesystem with no Obsidian install, and Obsidian() fails at construction:

FileNotFoundError: Obsidian CLI not found. Ensure Obsidian 1.12+ is installed...

Note that this is a raise, not the empty-result degradation described below — degraded mode covers "app closed", not "wrong machine". Do not interpret it as DEGRADED and do not try to work around it with filesystem fallbacks against a mounted vault path; switch shells and re-run. Use PowerShell on the Windows host (Desktop_Commander__start_process, or the native shell) for all wrapper calls, fetch.py, image downloads, and the connection detector.

Degraded mode — Obsidian not running

The obsidian wrapper is half disk-backed, half CLI-backed. When the Obsidian app is closed, the disk-backed half keeps working and the CLI-backed half returns err(1) with empty stdout — it does not raise. A run that ignores this will silently build a note on empty taxonomy data.

Preflight probe (run this first, every time):

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; ob=Obsidian(); r=ob.vault_info(); print('LIVE' if r.ok else 'DEGRADED'); print('vault:', ob._vault_path)"

ob._vault_path resolves from obsidian.json on disk and is populated even when the app is closed — use it anywhere the workflow calls for a vault path.

Rules for degraded mode:

• Announce it once, up front: Obsidian not running — degraded mode, using filesystem fallbacks.
• Degraded mode changes how Step 0 and Step 2.5a gather data, never whether they run. The URL-dedup gate is still mandatory.
• Only Step 6 may be skipped outright. Record it as SKIPPED, not FAIL.

Workflow

Step 0 — Inspect Taxonomy First

First, pin RUN_DATE — once, here — and use it for every date in the run.

from datetime import date
RUN_DATE = date.today().isoformat()   # capture ONCE; never call date.today() again

Every date_found, date_saved, Recently Added inline date tag, and the Step 5 prune cutoff must come from this single value. Runs cross midnight routinely — a long enrichment pass or a multi-note batch can span days (observed 2026-09-17 → 09-19, and 2026-09-22 → 09-24). Recomputing the date mid-run produces two defects at once: notes in the same batch get inconsistent date_saved values, and the second Step 5 pass prunes Recently Added entries that the first pass had just deliberately kept inside the 7-day window.

Then inspect the live research taxonomy so you do not invent a second folder or tag scheme.

Read the master library MOC first:

python .github/skills/obsidian/scripts/obsidian.py read --path "Research/Library/00 MOC/🗺️ MOC - Research Library.md"

SILENT-FAILURE TRAP — the master MOC filename contains a hidden character. 🗺️ is two code points: U+1F5FA (map) + U+FE0F (variation selector). The "obvious" spelling with a bare U+1F5FA does not match the file on disk. A mistyped filename must not be allowed to look like an empty MOC, or the run proceeds to invent its own taxonomy.

ob.read() signals a missing path in TWO different ways depending on mode — an emptiness check alone is not a valid guard. Measured 2026-09-22:

Mode	ob.read() on a missing path returns
LIVE (app running)	'Error: File "<path>" not found.' — a non-empty string
DEGRADED (app closed)	'' — empty, via CLI err(1) with empty stdout

So assert moc.strip() passes on garbage in LIVE mode. Use this helper for every read whose content drives a decision (Step 0 MOC, Step 2.5b, Step 4 dedup, Step 5) and never hand-roll either check:

def note_text(ob, path):
    """Return note content, or None if the path does not exist.
    Handles both the LIVE 'Error: ...' string and the DEGRADED empty string."""
    r = ob.read(path=path)
    if not r or not r.strip() or r.startswith('Error'):
        return None
    return r

MOC_PATH = 'Research/Library/00 MOC/' + chr(0x1F5FA) + chr(0xFE0F) + ' MOC - Research Library.md'
moc = note_text(ob, MOC_PATH)
assert moc, 'Master MOC read failed — wrong filename or Obsidian down. Stop and resolve.'

If it comes back None, resolve the real filename from disk instead of guessing: next(Path(ob._vault_path, 'Research/Library/00 MOC').glob('*MOC - Research Library.md')).

Corollary for existence checks (Step 4's slug guard): "exists" is note_text(...) is not None. Testing truthiness or .strip() alone reports a non-existent note as EXISTS in LIVE mode and will abort a legitimate write.

Read any relevant topic MOC only after the master MOC, and only for extra domain context. Topic MOCs are secondary maps, not the source of truth for canonical tags.

Read the current vault tag index:

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; print(Obsidian().tags().text)"

List the live research-library note paths so you can see the active buckets:

python -c "import sys; sys.stdout.reconfigure(encoding='utf-8'); sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; print(Obsidian().files(folder='Research/Library', ext='md').text)"

Windows UTF-8 note: On Windows, subprocess pipes default to the system code page (cp1252), which cannot encode emoji or non-ASCII vault paths. Always add sys.stdout.reconfigure(encoding='utf-8') before any print() call in one-liner subprocess commands, or prefer calling the Obsidian Python API directly.

\u escapes are decoded before Python ever sees them. Tool input is JSON, so writing '️' inside a python -c "..." string does not reach Python as a 6-character escape sequence — the JSON layer decodes it first and sends the literal character, which then hits a cp1252 decode error on stdin. Do not try to escape your way out of it. Two fixes:

1. Build the character in code: chr(0x1F5FA) + chr(0xFE0F) — pure ASCII source, no escape survives to be mangled.
2. Better: put the code in a one-shot .py file via the Write tool and run python _tmp_x.py.

Use one-shot script files, not an interactive Python REPL. Interactive REPL sessions in this workflow have died mid-run more than once (observed during image download and during wrapper init), losing all in-memory state. One-shot scripts are reliable and re-runnable. This is the same rule as the PowerShell $ gotcha in Step 3b, generalized: any payload containing emoji, $, backticks, or nested quotes goes in a temp script file.

URL-based dedup (mandatory before fetching): Search for an existing note whose url: frontmatter matches the input URL. This catches duplicates even when the slug or folder differ. Use the Obsidian search API:

python -c "import sys; sys.stdout.reconfigure(encoding='utf-8'); sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; r=Obsidian().search(query='<INPUT_URL>'); print(r.text if hasattr(r,'text') else r)"

Note: Obsidian's local search does not support url: field-operator syntax — url: <value> throws "Operator not recognized". Use a plain-text query with the URL string or a distinctive fragment of it (e.g., the tweet ID or domain path).

Degraded-mode dedup (Obsidian not running): search() returns err(1) with empty output — which is indistinguishable from "no duplicate found", so an unchecked run will happily create a duplicate note. An empty search() result is only trustworthy when the preflight probe said LIVE. When degraded, do the scan on disk instead — this is a required substitute, not an optional extra:

from pathlib import Path
lib = Path(ob._vault_path) / 'Research/Library'
frag = '<distinctive URL fragment>'
hits = [p for p in lib.rglob('*.md') if frag in p.read_text(encoding='utf-8', errors='ignore')]

Same rule applies to the Step 2.5a related-note search.

If a match is found, stop — tell the user the note already exists, give them the path and an Obsidian link, and ask whether they want to update the existing note instead. Do NOT proceed to fetch or create a new note.

Use this routing table unless the live vault has changed again:

This table drifts. It listed only 01–08 while the live vault had 01–13 (caught 2026-09-16, which would have mis-routed any health, crypto, UI, economics or alignment note). Step 0 already lists the live buckets — trust the Step 0 output over this table whenever the two disagree, and update the table when they do.

Tagging rules before you write anything:

• Reuse tags already present in the master MOC's Canonical Tag Guidance first: agents, ai-agents, agent-harnesses, skills, claude-code, copilot, mcp, evals, rag, sdlc, workflow-design, research
• Prefer an existing vault tag over a new synonym if the meaning is the same
• Normalize to lowercase kebab-case; do not create variants like AIagents, MachineLearning, or mixed singular/plural duplicates if a canonical form already exists
• Only mint a new canonical tag if both the master MOC and the vault tag index are missing a genuinely useful concept
• Choose one primary library folder per note; do not duplicate the same note across folders

Step 1 — Fetch Content

Run the fetch script to retrieve URL content:

python .github/skills/obsidian-linked-research/scripts/fetch.py "<url>"

The script outputs JSON to stdout:

For tweets:

{
  "type": "tweet",
  "url": "https://x.com/...",
  "text": "Full tweet/thread text",
  "author_handle": "username",
  "author_name": "Display Name",
  "date": "2026-03-01",
  "engagement": {"likes": 150, "reposts": 30, "replies": 12, "quotes": 5},
  "thread_context": "...",
  "is_thread": false,
  "media_descriptions": ["..."],
  "image_urls": ["https://pbs.twimg.com/media/..."],
  "article_content": "Full article text if X Article detected",
  "article_title": "Article title if X Article"
}

For web pages:

{
  "type": "web",
  "url": "https://...",
  "title": "Page Title",
  "content": "Plain text content (up to 8000 chars)",
  "image_urls": ["https://example.com/hero.jpg", "https://example.com/diagram.png"]
}

The image_urls array includes OG images and meaningful <img> tags found in the page (tracking pixels, favicons, and tiny icons are filtered out). Use Step 3b to download them into the vault.

If the result contains an "error" key, report it to the user and stop.

Transient retry: If the error is "No JSON in response" for a tweet URL, retry once — the xAI Responses API occasionally returns a top-level array [{...}] instead of a plain {...} object; a second call usually returns the plain object. If it fails twice, report to the user.

X/Twitter fallback dead-ends: Do NOT try Playwright for x.com URLs — it returns a login wall for unauthenticated sessions. Do NOT try Firecrawl — it explicitly does not support x.com. The only supported path for tweet content is fetch.py via the xAI x_search tool.

Enrichment pass for long-form sources — expect this, don't treat it as an exception.

fetch.py hard-caps web content at 8,000 chars. For any long-form source — essays, podcast transcripts, academic papers, substantial blog posts — that cap is hit essentially every time (a 106k-char podcast transcript returns 7.5% of itself). Treat the enrichment pass as the default second half of Step 1 for long-form, not as an error path.

Check the cap explicitly rather than eyeballing the output:

truncated = len(result.get('content', '')) >= 8000   # at the cap == assume clipped

Content at or near 8,000 chars is clipped, full stop. Content well under it is complete and needs no enrichment. Then:

• The calling skill already has the full text → use it, skip the second fetch. When obsidian-linked-research is invoked from gmail-daily-briefing on a newsletter, the email body is the publisher's own full article and is usually more complete than a re-fetch (no paywall interstitial, no nav chrome). Keep fetch.py's metadata and image_urls; take the prose from the email. Confirm the email body actually reaches the article's conclusion before trusting it — some senders truncate at a paywall.

Resolving a Substack URL does not require reading the email at all. Newsletter sends are large — a single Latent Space podcast email measured 738 KB, and get_message PLAIN_TEXT on these routinely exceeds the tool-result cap (the harness then spills it to a file). If all you need is the canonical URL, hit the publication's archive API instead:

https://<publication>/api/v1/archive?sort=new&limit=25

It returns JSON with canonical_url, title, and post_date per post — match on title and you have the URL for a few KB. Verified 2026-09-22 on www.latent.space (resolved /p/jev without touching the 738 KB body).

When you do need the prose and the email is oversized, parse the spilled JSON's plaintextBody on the host, strip the [ https://substack.com/redirect/... ] noise, and read it in ~26k-char slices. Note that paid Substack issues truncate at the paywall even in the delivered email, so a free-tier body may stop mid-issue — check before trusting it as complete.

• Article / essay / blog post → WebFetch (Claude Code tool) or fetch_webpage on the same URL to recover the article structure.
• Very long or many-sectioned source (academic paper, full podcast transcript, multi-part essay) → go straight to a general-purpose sub-agent with WebFetch and ask it to extract all major sections. Do not burn a round-trip on a plain WebFetch first; it will return an overview, not the sections.
• Prompt phrasing matters: never ask WebFetch for "verbatim" or "word for word" text — it refuses on copyright grounds. Ask for "a structured outline with headings, key facts, short attributed quotes, and any comparison tables". That yields the structure the note actually needs.
• Do not skip fetch.py; it remains the canonical fetch step and the source of image_urls and metadata.
• Keep the original fetch.py metadata (title, image_urls, url) as the source of truth unless the secondary fetch clearly corrects it.
• If parts of the source were provably not retrieved (e.g. sub-pages behind nav links), flag that in the note as an explicit scope boundary rather than inventing content.

If "needs_browser": true: The tweet contains an X Article that requires JavaScript rendering. Do NOT use fetch_webpage — it cannot render X Article pages. Instead, use the Playwright browser tools:

1. Navigate to the tweet URL (not the article URL — X Article URLs redirect to a "not supported" page):

   browser_navigate(url=tweet_url)

2. Wait for content to load (X Articles take a few seconds to render):

   browser_wait_for(time=5000)

3. The browser snapshot (returned by wait_for) contains the full article text in the accessibility tree. Extract the article content from the snapshot YAML.

Use the returned content as the article body for summarization. Combine it with the tweet metadata (author, engagement) already in the fetch result.

Extracting images from browser content: Use browser_evaluate to extract pbs.twimg.com image URLs from the DOM (the snapshot YAML won't include src attributes):

browser_evaluate(function="() => { return JSON.stringify(Array.from(document.querySelectorAll('img')).filter(i => i.src && i.src.includes('pbs.twimg.com') && !i.src.includes('_bigger')).map(i => i.src)); }")

Then download the images using:

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian-linked-research/scripts'); from fetch import download_images; import json; r=download_images(['url1','url2'], '<vault_path>/Research/Library/attachments', '{slug}'); print(json.dumps(r))"

Close the browser when done: browser_close()

Step 2 — Summarize (You, the Agent)

Using the fetched content, generate a deep, structured analysis — not a shallow summary. Do NOT call an external API — use your own reasoning.

Quality bar: Study the existing notes in the target Research/Library/ subfolder for reference. Good notes are section-by-section breakdowns with tables, code examples, specific details, and "Relevance to This Repo" sections. They read like comprehensive technical references, not tweet-length summaries.

Think through the content and produce this structure internally:

{
  "title": "Clear, descriptive title for the note",
  "slug": "kebab-case-filename-slug (3-6 words, no special chars)",
  "library_bucket": "one of the live buckets listed by Step 0 — do not pick from a hardcoded list here",
  "library_path": "Research/Library/<bucket>/<slug>.md",
  "author": "@handle or Author Name",
  "source": "x|reddit|blog|article|github",
  "core_thesis": "1-2 sentences capturing the central argument or claim",
  "sections": [
    {
      "heading": "Section heading from the article",
      "content": "Detailed breakdown — preserve key arguments, lists, code, tables, quotes"
    }
  ],
  "key_takeaways_table": [
    {"lesson": "Short label", "detail": "Specific explanation"}
  ],
  "relevance_to_repo": "2-4 sentences on how this connects to Eric Cartman / the user's workflow",
  "related_notes": ["[[Note Title 1]]", "[[Note Title 2]]"],
  "master_moc_section": "The existing section in the master Research Library MOC this note belongs under",
  "topic_moc": "Optional topic MOC to update secondarily, if one clearly applies",
  "tag_rationale": "Short note on which tags were reused from the master MOC/tag index and whether any new canonical tag is truly needed",
  "tags": ["tag1", "tag2", "tag3"]
}

Summarization rules:

• slug: lowercase, hyphens only, 3-6 words, no special chars
• library_bucket: choose exactly one existing library folder based on the dominant theme
• library_path: always point at Research/Library/<bucket>/<slug>.md
• tags: lowercase, 4-7 tags, topically relevant, hyphens in multi-word, with existing vault tags preferred over new inventions
• sections: Preserve the article's own structure. Include code blocks, tables, numbered lists, and blockquotes from the original. DO NOT flatten rich content into bullet points.
• key_takeaways_table: 4-8 rows. Each "lesson" is a short label; "detail" is the specific, non-obvious insight. Avoid generic platitudes.
• relevance_to_repo: Connect the content to this repo's architecture, skills, or workflow patterns. Be specific.
• related_notes: Use [[wiki-link]] format to connect to other Library notes.
• master_moc_section: prefer an existing folder section in 🗺️ MOC - Research Library; also use Recently Added
• topic_moc: optional; only set this if the note clearly belongs to a subordinate topic MOC such as 🤖 MOC - AI Agent Development
• tag_rationale: explicitly check the master MOC and Obsidian().tags().text output before deciding to introduce any new canonical tag
• source: detect from URL (x.com→x, reddit.com→reddit, github.com→github, else→article)
• author: extract from content or URL. "Unknown" if not identifiable
• For tweets: incorporate engagement stats and thread context

Step 2.5 — Thesis Drift Check

After generating the summary (Step 2), check whether the new note's core thesis conflicts with, supersedes, or complements existing vault notes. This step uses your own reasoning (the VS Code agent model) — no external API call.

2.5a — Find related existing notes

Search the vault for existing notes that share tags or are in the same library bucket as the new note:

# Search by primary tags (run for each of the top 2-3 tags)
python -c "import sys; sys.stdout.reconfigure(encoding='utf-8'); sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; ob=Obsidian(); r=ob.search(query='<primary_tag>', path='Research/Library/<bucket>'); print(r.text if hasattr(r,'text') else r)"

Degraded mode: if the preflight probe reported DEGRADED, search() returns empty — use the disk-walk fallback from Step 0 (filter Research/Library/**/*.md on tags: frontmatter) instead of concluding there are no related notes.

Also check the related_notes identified in Step 2 — read any that exist via Step 0's note_text() (a missing path is 'Error: ...' in LIVE mode and '' in DEGRADED mode, so neither truthiness nor emptiness alone is a valid check).

Same article, different URL. The URL-dedup gate misses a source captured earlier from another channel. The usual case is a claude.dev blog post whose X thread is already in the vault. That is not a duplicate stop. Classify it as Supersedes when the new URL is the canonical, more complete version, and check the older note's facts against it (on 2026-10-02 the thread note had "20 of 50" where the blog says 35).

From the search results plus related notes, select the top 3-5 most related existing notes by this priority:

1. Notes in the same library bucket with 2+ shared tags
2. Notes in any bucket with 3+ shared tags
3. Notes explicitly listed in related_notes from Step 2

2.5b — Read and compare theses

For each of the top 3-5 related notes:

python -c "import sys; sys.stdout.reconfigure(encoding='utf-8'); sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; ob=Obsidian(); content=ob.read(path='<existing_note_path>'); print(content[:2000])"

Extract the ## Core Thesis section (or the first substantive paragraph if no Core Thesis heading exists). Compare it against the new note's core_thesis from Step 2.

2.5c — Classify the relationship

For each comparison, make a judgment call. This is semantic analysis, not keyword matching. Apply these definitions:

Judgment guidance:

• A 2025 study updating a 2024 study's numbers = Supersedes (if same methodology and claims)
• "LLMs can't reason" vs "LLMs exhibit emergent reasoning" = Conflicts
• "Here's how to implement RAG" vs "Here's how to evaluate RAG" = Complements
• A broader survey covering the same narrow topic = Supersedes only if it renders the narrow one redundant; otherwise Complements
• Two articles saying similar things with different examples = Complements (not supersedes — both add value)

2.5d — Handle conflicts and supersessions

If Supersedes:

1. Add to the new note's frontmatter (will be written in Step 4):

   supersedes: "[[old-note-slug]]"

2. After the new note is written (end of Step 4), update the old note:

   python -c "
   import sys
   sys.path.insert(0, '.github/skills/obsidian/scripts')
   from obsidian import Obsidian
   ob = Obsidian()
   content = ob.read(path='<old_note_path>')
   # Add superseded status to frontmatter
   content = content.replace('status: unread', 'status: superseded', 1)
   # Insert superseded_by before the CLOSING frontmatter delimiter.
   # Do not use re.sub(r'^---', ..., count=1): that matches the OPENING
   # delimiter and puts the field above the frontmatter (caught 2026-10-02).
   import re
   if 'superseded_by:' not in content:
       m = re.match(r'^---\n.*?\n---\n', content, flags=re.S)
       end = m.end() - len('---\n')
       content = content[:end] + 'superseded_by: \"[[<new-note-slug>]]\"\n' + content[end:]
   ob.create(path='<old_note_path>', content=content, overwrite=True)
   print('Updated old note with superseded status')
   "

3. Report to user: "Note X supersedes [[old-note]]: [one-line explanation]"

If Conflicts (genuinely important disagreement):

1. Flag in Step 7 output: "THESIS CONFLICT: [[new-note]] vs [[old-note]]: [explanation]"

2. Create a synthesis note that captures both sides of the disagreement:

   ---
   type: research-note
   source: synthesis
   date_saved: {today YYYY-MM-DD}
   tags: [synthesis, {shared-tags}]
   status: unread
   resolves:
     - "[[note-slug-1]]"
     - "[[note-slug-2]]"
   ---
   
   # Conflict: {Topic} — {Slug1} vs {Slug2}
   
   ## The Disagreement
   
   **[[{slug1}]]** argues: {thesis 1}
   
   **[[{slug2}]]** argues: {thesis 2}
   
   ## Why It Matters
   
   {2-3 sentences on why this tension is meaningful for the research library}
   
   ## Possible Resolution
   
   {Your analysis of which position has stronger evidence, or whether they're
   actually addressing different aspects of the same problem}
   
   ## Related
   
   - [[{slug1}]] · [[{slug2}]]

   Write this synthesis note to: Research/Library/<bucket>/conflict-<slug1>-vs-<slug2>.md

3. Add the synthesis note to both original notes' ## Related sections.

If Complements:

No special action. Ensure the existing note appears in related_notes (Step 2 output) so the ## Related section in the new note will include it.

2.5e — Report drift check results

Before proceeding to Step 3, briefly report what the drift check found:

• Number of existing notes compared
• Any supersessions or conflicts detected
• Any synthesis notes that will be created
• If no conflicts found: "Thesis drift check: clean — no conflicts with N existing notes"

Step 3 — Compose Note

Assemble the markdown. The note should be comprehensive enough to replace reading the original — a full technical reference, not a summary card.

If Step 2.5 found a supersession, include supersedes: "[[old-note-slug]]" in the frontmatter. If a conflict was found, the synthesis note will be created after this note is written (in Step 4).

Use this template as a starting point, but adapt sections to match the content:

---
type: research-note
source: {source}
author: "{author}"
url: {url}
date_found: {today YYYY-MM-DD}
date_saved: {today YYYY-MM-DD}
tags: [{comma-separated tags}]
status: unread
supersedes: "[[old-note-slug]]"  # only if Step 2.5 found a supersession
---

**Author**: {author}
**Published**: {date}
**Views**: {views if known}
**Source**: {url}
**Engagement**: ❤️ {likes} 🔁 {reposts} 💬 {replies} 📝 {quotes}

---

## Core Thesis

{1-2 sentences: the central argument or finding}

---

## {Section 1 Heading from Article}

{Detailed content — preserve tables, code blocks, lists, quotes from original.
Include specific numbers, tool names, people. Don't flatten into generic bullets.}

## {Section 2 Heading}

{Continue for each major section of the source material...}

---

## Key Takeaways

| Lesson | Detail |
|--------|--------|
| {short label} | {specific, non-obvious insight} |
| ... | ... |

---

## Relevance to This Repo

{2-4 sentences connecting this to Eric Cartman's architecture, skills, or
the user's workflow. Be specific about what to keep, change, or investigate.}

---

## Related

- [[Note Title 1]] · [[Note Title 2]]
- External source: {url}

## My Notes

Formatting rules:

• Use --- horizontal rules between major sections for visual scanability
• Preserve code blocks with language hints (bash, json, etc.)
• Use tables for structured comparisons (don't convert tables to bullet lists)
• Use blockquotes (>) for direct quotes from the source
• Include **bold** for key terms and emphasis as in the original
• ## Related should use [[wiki-links]] to connect to other Library notes. Link by slug, not by title: library notes have no # H1 — the filename slug is the only resolvable link target, so [[Note Title 1]] from the template above creates a broken link. Use [[slug|Display Text]], which is also the form the master MOC uses.
• Keep the frontmatter compatible with existing research notes in the chosen folder

And if it's a thread, add a Thread Context section before Summary:

## Thread Context

{thread_context}

If images are present, embed them inline within the section they belong to — right after the paragraph or heading they illustrate. Do NOT collect images into a separate ## Images section at the end. Place each image where a reader would naturally expect to see it, using this format:

![[{slug}-1.jpg]]
*{media_description_1}*

Use ![[filename]] (Obsidian wiki-link embed) for each image. If media_descriptions are available, add them as italic captions below each image. If an image doesn't clearly belong to a specific section (e.g., a generic hero image or author avatar), place it just below the metadata header block.

Web sources give you URLs with no captions — do not guess placement. For tweets, media_descriptions pairs descriptions to images. For web pages, image_urls is a bare, unordered-ish list with no alt text, no captions, and no position anchors, so on an image-heavy article (Substack posts routinely return 8–14 URLs, mixed with avatars, logos and embedded-tweet thumbnails) there is no way to tell which figure goes where. Placing them by guesswork produces confidently wrong captions, which is worse than omitting them.

Rule: embed only images you can actually identify — normally the hero (the OG image, usually the first entry and the one that repeats at two widths).

Filter list — drop any URL containing w_32, w_36, w_40, h_40, w_96, w_144, h_72, w_150, profile_images, or default-light. The last few matter more than they look: h_72 is the publication wordmark, and w_150 is Substack's sidebar thumbnail for other episodes — embedding one puts a different article's artwork in this note. Measured 2026-09-22: a Latent Space podcast post returned 14 image_urls, only 2 of them usable.

Podcast posts: the OG image is a substack-video.s3.amazonaws.com/.../transcoded-*.png player thumbnail, not a content figure. It is fine as the header image; do not caption it as a diagram.

Prefer inspect-then-caption over omit-by-default. Download the full-width candidates, then open each one with the Read tool — you can see images natively, and identification is usually exact (a comparison table, a docs page, a roster screenshot). That converts "omit, cannot caption" into accurate captions plus transcribable table content. Fall back to omission only for images you still cannot place after looking. Captions from the fetched body text (they appear inline in content, e.g. "Grok 4 Fast is the least likely to betray you…") may be used only when the pairing is unambiguous. State any remaining omission in a ## Scope Boundary section rather than silently dropping the figures.

Step 3b — Download Images

If the fetch result contains image_urls, download them to the vault's attachment folder. First, discover the vault path:

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; print(Obsidian().vault_info().text)"

If Obsidian is not running, vault_info() returns empty. Use Obsidian()._vault_path instead — it is resolved from obsidian.json on disk at construction time and works with the app closed.

ob._vault_path is a pathlib.Path, not a str. The '<vault_path>/Research/Library/attachments' concatenation shown below raises TypeError: unsupported operand type(s) for +: 'WindowsPath' and 'str' if you build it with +. Join with / and cast at the boundary: str(Path(ob._vault_path) / 'Research/Library/attachments').

Parse the vault path from the output, then download images:

python .github/skills/obsidian-linked-research/scripts/fetch.py "<url>" --download-images "<vault_path>/Research/Library/attachments" --slug "{slug}"

Or if you already have the fetch result and just need to download, call Python directly:

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian-linked-research/scripts'); from fetch import download_images; download_images({image_urls_list}, '<vault_path>/Research/Library/attachments', '{slug}')"

Images will be saved as {slug}-1.jpg, {slug}-2.png, etc. Use ![[{slug}-1.jpg]] in the note to embed them.

PowerShell $ gotcha (Substack/Cloudinary CDN URLs): substackcdn.com image URLs contain a signature segment like $s_!5LAd!,w_1456,.... PowerShell interpolates $s_ as a variable and mangles the URL, producing a 404 even though the URL is valid. Do not pass these URLs inside a python -c "..." one-liner from PowerShell. Two fixes, in order of preference:

1. Use the underlying origin URL instead — strip the https://substackcdn.com/image/fetch/<sig>/ prefix and URL-decode the remainder to get the plain https://substack-post-media.s3.amazonaws.com/public/images/<uuid>_<WxH>.png URL. No $, no resizing params, full resolution.
2. If you must keep the CDN URL, write the URL list into a temp .py file with the Write tool and run python _tmp_dl.py — never inline it in a shell argument.

Same rule applies to any one-liner: prefer a temp script over python -c whenever the payload contains $, backticks, or nested quotes.

PowerShell > redirection writes UTF-16LE with a BOM. Redirecting fetch.py stdout to a file (python fetch.py "<url>" > out.json) produces a file that json.load rejects with UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0. Never redirect fetch.py with >. Run it from a Python wrapper and capture bytes directly: subprocess.run([sys.executable, FETCH, url], capture_output=True).stdout.decode('utf-8').

Step 4 — Write to Vault

Primary pattern (works in all environments, including Claude Code on Windows):

Use the Write tool to write the note content to a temp file, then call the Obsidian Python API directly:

# 1. Write content to temp file using the Write tool:
#    file_path: Z:\Projects\Eric-Cartman\_tmp_note.md
#    content: {full markdown content}

# 2. Call the Obsidian Python API (in a Bash tool):
python -c "
import sys
sys.path.insert(0, '.github/skills/obsidian/scripts')
from obsidian import Obsidian

with open('_tmp_note.md', 'r', encoding='utf-8') as f:
    content = f.read()

ob = Obsidian()
result = ob.create(path='Research/Library/{library_bucket}/{slug}.md', content=content)
print(result)
"

# 3. Delete the temp file:
rm _tmp_note.md

IMPORTANT — keyword arguments required: ob.create() does not accept positional arguments. Always use path= and content= as keyword arguments. ob.create(path, content) will fail.

WARNING — do not use heredoc pipes from bash: The @'...'@ heredoc pipe pattern only works in a native PowerShell shell. When Claude Code runs bash on Windows, attempting bash -c "powershell -Command \"@'...'@\"" fails due to irreparably lossy quote escaping. Any note containing apostrophes, contractions, or code will break the heredoc approach. Use the Write-tool + Python API pattern above instead.

Verify the write:

python .github/skills/obsidian/scripts/obsidian.py read --path "Research/Library/{library_bucket}/{slug}.md" 2>&1 | head -6

If the first few lines match your frontmatter, the write succeeded.

Check for existing notes before creating (deduplicate):

The primary dedup gate is the URL-based search in Step 0. This slug check is a secondary guard for the rare case where two different URLs produce the same slug:

python -c "
import sys
sys.path.insert(0, '.github/skills/obsidian/scripts')
from obsidian import Obsidian
ob = Obsidian()
r = ob.read(path='Research/Library/{library_bucket}/{slug}.md')
exists = bool(r) and bool(r.strip()) and not r.startswith('Error')
print('EXISTS' if exists else 'NOT_FOUND')
"

Do not simplify this to if result: or if result.strip():. In LIVE mode a missing path returns 'Error: File "..." not found.', which is truthy — that check reports EXISTS for a note that does not exist and aborts a valid write. This is the same hazard as the Step 0 trap box; note_text() from Step 0 is the canonical form.

If the note already exists, append -2, -3, etc. to the slug before calling ob.create().

Post-write drift actions (from Step 2.5):

After the new note is successfully written, apply any pending drift actions:

• If a supersession was detected: update the old note's frontmatter (see Step 2.5d)
• If a conflict was detected: create the synthesis note (see Step 2.5d)
• If synthesis notes were created: add them to both original notes' ## Related sections

Step 5 — Update the Master MOC

After the note exists, refresh the master Research Library MOC as part of the same run unless doing so would require a major restructure.

At minimum:

• add the note to the relevant folder section if it is useful for navigation
• append or refresh an entry under Recently Added
• keep the one-line description current and specific
• prune Recently Added entries older than 7 days (see pruning logic below)

If the master MOC would need a larger restructure, stop after creating the note and suggest a separate obsidian-vault-linker refresh.

If you need to insert the note into an existing section (for example the correct library bucket list plus Recently Added), prefer a clean overwrite instead of stacking append-only blocks at the end of the file:

1. Read the current MOC
2. Prune stale Recently Added entries (older than 7 days)
3. Add the new note to the relevant folder section and Recently Added
4. Rewrite the full note through the obsidian wrapper using create --overwrite

Example pattern (use the Python API directly — do not use heredoc pipes from bash):

python -c "
import sys
sys.path.insert(0, '.github/skills/obsidian/scripts')
from obsidian import Obsidian

ob = Obsidian()

# Read current MOC
content = ob.read(path='Research/Library/00 MOC/🗺️ MOC - Research Library.md')

# Make in-memory edits (string replacement)
updated = content.replace(old_text, new_text)

# Overwrite — always use keyword args
ob.create(path='Research/Library/00 MOC/🗺️ MOC - Research Library.md', content=updated, overwrite=True)
print('MOC updated')
"

Windows UTF-8: ob.read() and ob.create() handle encoding correctly. Do not go through subprocess pipes for MOC reads/writes — encoding errors will occur with emoji paths.

Recently Added entry format

When writing a new Recently Added entry, always include an inline date tag so the pruning step can identify stale entries without reading every linked note:

- [[{slug}|{title}]] — {one-line why this note matters} `{YYYY-MM-DD}`

Example:

- [[five-ai-paradigm-shifts|Five AI Paradigm Shifts]] — Miessler's framework for agent-era mental models `2026-03-30`

Pruning stale Recently Added entries

Whenever Step 5 runs (i.e., on every successful note save), prune Recently Added entries whose inline date is more than 7 days before today. Entries without an inline date are left untouched (they predate this feature).

Use this Python snippet as part of the in-memory MOC edit before rewriting:

import re
from datetime import date, timedelta

cutoff = date.today() - timedelta(days=7)

def prune_recently_added(moc_content: str) -> tuple[str, int]:
    """Remove Recently Added list items whose `YYYY-MM-DD` date is older than cutoff.
    Returns (updated_content, number_pruned).
    Items without a date tag are kept unchanged.
    """
    # Match list items that end with a backtick-wrapped date: `2026-03-22`
    date_pattern = re.compile(r'`(\d{4}-\d{2}-\d{2})`')
    pruned = 0
    output_lines = []
    in_recently_added = False

    for line in moc_content.splitlines(keepends=True):
        # Detect entry into / exit from the Recently Added section
        if re.match(r'^##\s+Recently Added', line):
            in_recently_added = True
            output_lines.append(line)
            continue
        if in_recently_added and re.match(r'^##\s+', line):
            in_recently_added = False

        if in_recently_added and line.startswith('- '):
            m = date_pattern.search(line)
            if m:
                entry_date = date.fromisoformat(m.group(1))
                if entry_date < cutoff:
                    pruned += 1
                    continue  # drop this line
        output_lines.append(line)

    return ''.join(output_lines), pruned

Apply it before inserting the new entry:

updated_content, n_pruned = prune_recently_added(moc_content)
if n_pruned:
    print(f"Pruned {n_pruned} stale Recently Added entries")
# then insert new entry and rewrite

Use the lighter append approach only when a small footer-style refresh is good enough and the file does not need an in-place insertion. Even in that case, still run prune_recently_added on the current content first, rewrite with pruning applied, then append the new entry.

If a topic MOC clearly applies, update it secondarily after the master MOC is current. The master MOC remains authoritative for canonical tags and library-wide freshness.

Step 6 — Open in Obsidian (optional)

python -c "import sys; sys.path.insert(0,'.github/skills/obsidian/scripts'); from obsidian import Obsidian; ob=Obsidian(); ob.open('Research/Library/{library_bucket}/{slug}')"

Cosmetic step — never a run failure. open() is CLI-backed and fails with err(1) when the Obsidian app is closed. The note is already written and verified by this point. If the preflight probe reported DEGRADED, skip this step entirely, record it as SKIPPED, and rely on the obsidian:// link in Step 7. Do not retry it, and do not report the run as failed because of it.

Step 6.5 — Detect Connections

After writing the note and updating MOCs, run the connection detector to classify relationships between the new note and existing vault content:

python .github/skills/obsidian-connection-detector/scripts/detect.py --note "Research/Library/{library_bucket}/{slug}.md" --no-section

Run it inline, ONE note per tool call. Measured wall time is ~45–105 seconds per note (63.1s / 62.5s / 48.5s / ~60s on 2026-09-19; 101s on 2026-09-22) — not the "5+ minutes" this step claimed through 2026-09-22. The old advice cost more than it saved: it told the agent to background and re-check something that returns inline. A single note will not time out.

What does time out is chaining several notes into one command (confirmed 2026-09-22 with three notes). Invoke the detector once per note, in its own tool call.

If you do background it, redirect to a log file — never pipe a long-running process through Select-String, and never drive it via PowerShell Start-Process -ArgumentList (the array splits bucket paths on spaces, and every library bucket name contains a space). Use a Python subprocess.run([...]) list instead.

Should a call genuinely time out, it is still NOT a failure — the work completes in the background. Confirm via connections.json rather than re-running:

from pathlib import Path; import time
p = Path(ob._vault_path) / 'Research/connections.json'
print(p.stat().st_size, time.ctime(p.stat().st_mtime))

A recent mtime means it is working — record Step 6.5 as PASS and move to Step 7.

This is best-effort — if it genuinely errors or is unavailable, continue to Step 7. The detector will classify relationships (supports, contradicts, extends, bridges) and write to Research/connections.json. The thesis tracker will pick up new connections on its next run and flag emerging theses.

Step 7 — Confirm

Report to the user:

• Note title and path
• Brief summary (1-2 sentences)
• Tag list
• Whether the master MOC was updated, and whether any topic MOC was also updated
• Thesis drift results: how many notes were compared, any supersessions or conflicts
• If a supersession occurred: which old note was marked superseded
• If a conflict was found: path to the synthesis note
• Connections found: how many and what types (supports/contradicts/extends/bridges)
• Obsidian link: obsidian://open?vault=Obsidian%20Vault&file=Research%2FLibrary%2F{library_bucket_urlencoded}%2F{slug}

Step 8 — Reflection (composable)

Invoke the skill-reflection skill with the following context:

• Calling skill: obsidian-linked-research
• SKILL.md path: .github/skills/obsidian-linked-research/SKILL.md
• Steps completed: list each step with pass/fail/skipped (including Step 2.5)
• Friction notes: any workarounds, retries, unexpected errors, or manual interventions

The reflection skill will analyze the run and produce improvement recommendations.

Output Format

The output is a Research/Library/<bucket>/{slug}.md file in the Obsidian vault, following the enriched note template above. The format matches the existing library pages and respects the live MOC/tag taxonomy already in the vault.

If thesis drift was detected, additional outputs may include:

• Updated frontmatter on superseded notes (status: superseded, superseded_by:)
• Synthesis notes at Research/Library/<bucket>/conflict-<slug1>-vs-<slug2>.md

Rules

1. Never skip the fetch step — always run fetch.py even if you think you know the content
2. Never call an external LLM API for summarization — use your own model (VS Code)
3. Always use the Obsidian Python API for vault writes — use ob.create(path=, content=) with keyword args; never use positional arguments and never write vault files directly via filesystem tools
4. Write-tool + Python API is the primary write pattern — write content to a temp file with the Write tool, read it in Python, call ob.create(path=..., content=...), then delete the temp file; do not use @'...'@ heredocs from bash on Windows (they break on apostrophes and single quotes in note content)
5. Never use bash -c "powershell -Command \"...\"" for heredocs — quote escaping through that shell boundary is irreparably lossy; the Write-tool + Python API pattern avoids this entirely
6. Windows UTF-8 — on Windows, subprocess pipes default to cp1252 and cannot encode emoji or non-ASCII vault paths; add sys.stdout.reconfigure(encoding='utf-8') to one-liner subprocess commands, or call the Obsidian Python API directly (preferred)
7. Handle fetch errors gracefully — if fetch returns an error, tell the user and suggest alternatives
8. Deduplicate by URL first, slug second — search all Research/Library/ frontmatter for the input URL in Step 0 before fetching; check slug in Step 4 as a secondary guard
9. Zero pip deps — fetch.py uses only Python stdlib
10. Read the master MOC first — tag and folder decisions must be based on the live Research/Library/00 MOC/🗺️ MOC - Research Library.md
11. Prefer existing tags — reuse the master MOC's canonical lowercase kebab-case tags before introducing a new one
12. Keep the master MOC fresh — update it with every successful research-note addition unless the change truly requires larger curation work
13. Use topic MOCs secondarily — they are optional supporting maps, not the authoritative taxonomy source
14. Thesis drift check is semantic, not syntactic — use your own judgment to compare core theses; do not reduce this to keyword overlap or string matching
15. Supersession updates are post-write — only modify old notes after the new note is successfully written to the vault
16. Synthesis notes are for genuine conflicts only — do not create synthesis notes for mere differences in emphasis, scope, or audience; reserve them for real disagreements on factual claims or framework incompatibilities
17. Probe before you trust an empty result — run the Prerequisites preflight first; when Obsidian is closed, every CLI-backed call (search, tags, files, vault_info, open) returns err(1) with empty stdout, which is indistinguishable from a legitimate "no results". Treat empty as unknown, not as absence, and switch to the documented filesystem fallback
18. Degraded mode is not an abort — disk-backed read/create/append keep working with the app closed, so the run continues; only Step 6 (open) may be skipped, and it is recorded SKIPPED, never FAIL
19. ob.read() signals a missing path differently per mode — emptiness alone is not a valid guard — LIVE returns the truthy string 'Error: File "..." not found.', DEGRADED returns ''. Use Step 0's note_text() helper (None on either signal) for every read whose content drives a decision, especially the master MOC (🗺️ = U+1F5FA + U+FE0F) and the Step 4 slug check. assert content.strip() passes on garbage in LIVE mode, and a bare truthiness test reports a non-existent note as EXISTS
20. Assume long-form sources are truncated — fetch.py caps at 8,000 chars; check len(content) >= 8000 and run the enrichment pass as a matter of course, going straight to a sub-agent for transcripts and papers
21. One-shot scripts over interactive REPLs and \u escapes — interactive Python sessions die mid-run and lose state; JSON tool input decodes \uXXXX before Python sees it, so build non-ASCII with chr(0x...) or write a temp .py file
22. Run Step 6.5 inline, one note per tool call — the connection detector takes ~45–105s per note, not 5+ minutes; a single note returns inline. Chaining several notes into one command is what times out. If a call does time out it is still a PASS — verify via connections.json mtime rather than re-running it
23. Pin RUN_DATE once at Step 0 — runs cross midnight and can span days; recomputing date.today() mid-run gives one batch inconsistent date_saved values and makes the second Step 5 pass prune Recently Added entries the first pass deliberately kept

Related Skills

Architecture

User: "research this: <url>"
          │
          ▼
┌─────────────────────────┐
│  Read master MOC        │  ← Determine folder + canonical tags
│  + tag index + library  │
└──────────┬──────────────┘
           │
           ▼
┌─────────────────────────┐
│  fetch.py <url>         │  ← Python stdlib only
│  ├─ Tweet? → xAI API   │     (x_search tool, grok-4-1-fast)
│  └─ Web?  → urllib      │     (HTML strip, plain text)
└──────────┬──────────────┘
           │ JSON stdout
           ▼
┌─────────────────────────┐
│  Agent (VS Code model)  │  ← Summarize, structure, compose
│  ├─ Generate summary    │
│  ├─ Thesis drift check  │     ← NEW: compare against existing notes
│  │  ├─ Supersedes?      │        Update old note frontmatter
│  │  ├─ Conflicts?       │        Create synthesis note
│  │  └─ Complements?     │        Add to Related
│  ├─ Compose markdown    │
│  └─ Write → Python API  │
└──────────┬──────────────┘
           │ Write tool → _tmp_note.md → ob.create(path=, content=)
           ▼
┌─────────────────────────┐
│  obsidian.py create     │  ← CLI wrapper (composable)
│  └─ Research/Library/   │
│     <bucket>/{slug}.md  │
└──────────┬──────────────┘
           │
           ▼
┌─────────────────────────┐
│  Post-write drift ops   │  ← Update superseded notes, create synthesis
└──────────┬──────────────┘
           │
           ▼
┌─────────────────────────┐
│  Update master MOC      │  ← Keep library map and tags current
│  (+ topic MOC if needed)│
└─────────────────────────┘