tessl/npm-astro

Astro is a modern site builder with web best practices, performance, and DX front-of-mind.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Content Loaders

Name: tessl/npm-astro
Author: tessl

Content Loaders are the foundation of Astro's Content Layer API, providing flexible ways to load data from various sources into Content Collections. The built-in loaders support loading data from local files, glob patterns, and structured data formats.

Capabilities

Glob Loader

Loads multiple entries using glob patterns to match files.

import { glob } from 'astro/loaders';

/**
 * Loads multiple entries using a glob pattern to match files
 */
function glob(options: GlobOptions): Loader;

interface GlobOptions {
  pattern: string | Array<string>;
  base?: string | URL;
  generateId?: (options: GenerateIdOptions) => string;
}

interface GenerateIdOptions {
  entry: string;
  base: URL;
  data: Record<string, unknown>;
}

interface Loader {
  name: string;
  load: (context: LoaderContext) => Promise<void>;
  schema?: ZodSchema | Promise<ZodSchema> | (() => ZodSchema | Promise<ZodSchema>);
}

import { defineCollection, glob } from 'astro:content';

// Load all markdown files from a directory
const blog = defineCollection({
  loader: glob({ pattern: '*.md', base: './src/data/blog' }),
  schema: z.object({
    title: z.string(),
    date: z.date(),
  }),
});

// Load multiple file types with an array pattern
const docs = defineCollection({
  loader: glob({
    pattern: ['*.md', '*.mdx'],
    base: './src/data/docs'
  }),
});

// Custom ID generation from frontmatter
const posts = defineCollection({
  loader: glob({
    pattern: '**/*.md',
    base: './content',
    generateId: ({ entry, data }) => {
      return data.customId as string || entry.replace(/\.md$/, '');
    }
  }),
});

Important Notes:

Glob patterns cannot start with ../ or /. Use the base option to specify parent directories.
The default generateId function generates a slug from the entry path, or uses a slug field from frontmatter if present.
The loader automatically detects content types (markdown, MDX, etc.) based on file extensions.

File Loader

Loads entries from a single data file (JSON, YAML, or TOML).

import { file } from 'astro/loaders';

/**
 * Loads entries from a JSON, YAML, or TOML file
 */
function file(fileName: string, options?: FileOptions): Loader;

interface FileOptions {
  /**
   * @default JSON.parse or yaml.load, depending on the extension
   */
  parser?: (text: string) => Record<string, Record<string, unknown>> | Array<Record<string, unknown>>;
}

import { defineCollection, file } from 'astro:content';
import { z } from 'astro/zod';

// Load from JSON array
const products = defineCollection({
  loader: file('products.json'),
  schema: z.object({
    name: z.string(),
    price: z.number(),
    description: z.string(),
  }),
});

// Load from YAML file
const team = defineCollection({
  loader: file('team.yml'),
  schema: z.object({
    name: z.string(),
    role: z.string(),
    bio: z.string(),
  }),
});

// Custom parser for specialized formats
const custom = defineCollection({
  loader: file('data.txt', {
    parser: (text) => {
      const lines = text.split('\n');
      return lines.map((line, index) => ({
        id: `${index}`,
        content: line,
      }));
    },
  }),
});

Data Format Requirements:

The loaded file must contain either:

Array format: An array of objects, each with a unique id or slug field:

[
  { "id": "1", "title": "First Post", "content": "..." },
  { "id": "2", "title": "Second Post", "content": "..." }
]

Object format: An object where keys serve as IDs:

{
  "first-post": { "title": "First Post", "content": "..." },
  "second-post": { "title": "Second Post", "content": "..." }
}

Supported File Formats:

JSON (.json): Uses JSON.parse
YAML (.yml, .yaml): Uses yaml.load
TOML (.toml): Uses TOML parser
Custom: Provide a custom parser function for other formats

Important Notes:

Glob patterns are not supported. To load multiple files, use the glob loader instead.
Each entry must have a unique id or slug field (for arrays), or unique object keys (for objects).
The file loader automatically detects the format based on file extension unless a custom parser is provided.

Loader Context

Both loaders receive a context object with utilities for data processing:

interface LoaderContext {
  collection: string;
  store: DataStore;
  meta: MetaStore;
  logger: AstroIntegrationLogger;
  config: AstroConfig;
  parseData<TData extends Record<string, unknown>>(props: ParseDataOptions<TData>): Promise<TData>;
  renderMarkdown(content: string): Promise<RenderedContent>;
  generateDigest(data: Record<string, unknown> | string): string;
  watcher?: FSWatcher;
  refreshContextData?: Record<string, unknown>;
}

interface ParseDataOptions<TData extends Record<string, unknown>> {
  id: string;
  data: TData;
  filePath?: string;
}

Creating Custom Loaders

You can create custom loaders by implementing the Loader interface:

import type { Loader } from 'astro/loaders';

export function customLoader(): Loader {
  return {
    name: 'custom-loader',
    async load({ store, logger, parseData }) {
      const data = await fetchDataFromAPI();

      for (const item of data) {
        const parsed = await parseData({
          id: item.id,
          data: item,
        });
        store.set({ id: item.id, data: parsed });
      }

      logger.info('Custom loader completed');
    },
  };
}

Custom loaders can load data from any source: APIs, databases, CMSs, or custom file formats.

Live Loaders

Live loaders provide dynamic data loading for collections that can update at runtime.

/**
 * Live loader for dynamic data collections
 */
interface LiveLoader<TData, TEntryFilter, TCollectionFilter, TError> {
  loadEntry(
    context: LoadEntryContext<TEntryFilter>
  ): Promise<LiveDataEntry<TData> | undefined | { error: TError }>;

  loadCollection(
    context: LoadCollectionContext<TCollectionFilter>
  ): Promise<LiveDataCollection<TData> | { error: TError }>;
}

interface LiveDataEntry<TData> {
  id: string;
  data: TData;
}

interface LiveDataCollection<TData> {
  entries: Array<LiveDataEntry<TData>>;
}

interface LoadEntryContext<TEntryFilter> {
  collection: string;
  filter: TEntryFilter;
  logger: AstroIntegrationLogger;
  config: AstroConfig;
}

interface LoadCollectionContext<TCollectionFilter> {
  collection: string;
  filter: TCollectionFilter;
  logger: AstroIntegrationLogger;
  config: AstroConfig;
}

import type { LiveLoader, LiveDataEntry } from 'astro/loaders';

// Example: Live loader for a REST API
function apiLiveLoader<T>(baseUrl: string): LiveLoader<T, { id: string }, { limit?: number }, Error> {
  return {
    async loadEntry({ filter, logger }) {
      try {
        const response = await fetch(`${baseUrl}/items/${filter.id}`);
        if (!response.ok) return undefined;
        const data = await response.json();
        return {
          id: filter.id,
          data,
        };
      } catch (error) {
        return { error: error as Error };
      }
    },

    async loadCollection({ filter, logger }) {
      try {
        const url = `${baseUrl}/items${filter.limit ? `?limit=${filter.limit}` : ''}`;
        const response = await fetch(url);
        const items = await response.json();

        return {
          entries: items.map((item: any) => ({
            id: item.id,
            data: item,
          })),
        };
      } catch (error) {
        return { error: error as Error };
      }
    },
  };
}

// Use in collection definition
const liveData = defineCollection({
  loader: apiLiveLoader('https://api.example.com'),
  schema: z.object({
    title: z.string(),
    content: z.string(),
  }),
});

Key Features:

Dynamic Loading: Data is fetched at runtime, not build time
Entry-level Access: Load individual entries on demand
Collection-level Access: Load entire collections with optional filtering
Error Handling: Explicit error return type for handling failures
Type Safety: Full TypeScript support with generic type parameters

Common Issues

File paths: Relative paths in loaders are resolved from the project root, not from the config file location.
Glob patterns: Patterns like ../ or / are not allowed. Use the base option instead.
Data format: File loader requires either an array with id fields or an object with keys as IDs.
Custom loaders: Remember to call parseData() to validate entries against your schema.