or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

cli.mdcomment-tags.mdconfiguration.mdcore-generation.mdindex.mdoutput-generation.mdparsing.mdplugins.md
tile.json

parsing.mddocs/

Internal Architecture and Parsing System

This document describes the internal architecture and parsing system of ApiDoc. These are implementation details not exposed through the public API, but understanding them can be helpful for plugin development and advanced usage.

Overview

ApiDoc's parsing system is built around several internal components that work together to extract API documentation from source code:

  • Core Parser: Extracts API documentation blocks from source files
  • Language Handlers: Process different programming language comment patterns
  • Tag Parsers: Parse individual @api tags within documentation blocks
  • Workers: Post-process and validate parsed data
  • Filters: Clean and structure the final output

Internal API Structure

The internal parsing functionality is located in lib/core/index.js but is not exposed through the main module exports. These functions are used internally by the createDoc function.

Internal Core Functions (Not Public API)

// Internal functions - NOT part of public API
function parse(options): boolean | ApiDocResult;
function parseSource(source: string, options): ParsedElement[];
function getSpecificationVersion(): string;
function setGeneratorInfos(generator: GeneratorInfo): void;
function setLogger(logger: Logger): void; 
function setMarkdownParser(markdownParser: MarkdownParser): void;
function setPackageInfos(packageInfos: PackageInfo): void;

Important: These functions are implementation details and should not be called directly. Use the public createDoc function instead.

Language Support

ApiDoc automatically detects and processes multiple programming languages based on file extensions:

Supported Languages and Comment Patterns

interface LanguageSupport {
  '.js': 'default';           // /** ... */
  '.jsx': 'default';          // /** ... */
  '.ts': 'default';           // /** ... */
  '.tsx': 'default';          // /** ... */
  '.java': 'default';         // /** ... */
  '.php': 'default';          // /** ... */
  '.go': 'default';           // /** ... */
  '.cs': 'default';           // /** ... */
  '.cpp': 'default';          // /** ... */
  '.py': 'py';                // """ ... """
  '.rb': 'rb';                // =begin ... =end
  '.clj': 'clj';              // ;;;; ... ;;;;
  '.coffee': 'coffee';        // ### ... ###
  '.erl': 'erl';              // %{ ... %}
  '.ex': 'ex';                // #{ ... #}
  '.exs': 'ex';               // #{ ... #}
  '.lua': 'lua';              // --[[ ... ]]
  '.pl': 'pm';                // =pod ... =cut
  '.pm': 'pm';                // =pod ... =cut
  default: 'default';         // /** ... */
}

Language-Specific Examples

JavaScript/TypeScript/Java/PHP/Go/C#

/**
 * @api {get} /users/:id Get User
 * @apiName GetUser
 * @apiGroup User
 * @apiParam {Number} id User unique ID
 * @apiSuccess {String} name User name
 */
function getUser(id) { }

Python

"""
@api {get} /users/:id Get User  
@apiName GetUser
@apiGroup User
@apiParam {Number} id User unique ID
@apiSuccess {String} name User name
"""
def get_user(id):
    pass

Ruby

=begin
@api {get} /users/:id Get User
@apiName GetUser  
@apiGroup User
@apiParam {Number} id User unique ID
@apiSuccess {String} name User name
=end
def get_user(id)
end

Parsing Process

The internal parsing process follows these steps:

  1. File Discovery: Scan input directories for matching files
  2. Language Detection: Determine parser based on file extension
  3. Comment Extraction: Extract documentation blocks from source
  4. Tag Parsing: Parse individual @api tags within blocks
  5. Data Validation: Validate and structure parsed data
  6. Post-Processing: Apply workers and filters
  7. Output Generation: Generate final documentation

API Tag Structure

Core Tags

interface ApiTags {
  // Required core tags
  '@api': '{method} path [title]';
  '@apiName': 'name';
  '@apiGroup': 'group';
  
  // Optional descriptive tags
  '@apiVersion': 'version';
  '@apiDescription': 'text';
  '@apiDeprecated': '[text]';
  '@apiPrivate': '';
  
  // Parameter tags
  '@apiParam': '[(group)] [{type}] [field=defaultValue] [description]';
  '@apiQuery': '[(group)] [{type}] [field=defaultValue] [description]';
  '@apiBody': '[(group)] [{type}] [field=defaultValue] [description]';
  '@apiHeader': '[(group)] [{type}] [field=defaultValue] [description]';
  
  // Response tags
  '@apiSuccess': '[(group)] [{type}] field [description]';
  '@apiError': '[(group)] [{type}] field [description]';
  
  // Example tags
  '@apiExample': '{type} title';
  '@apiParamExample': '{type} title';
  '@apiHeaderExample': '{type} title';
  '@apiSuccessExample': '{type} title';
  '@apiErrorExample': '{type} title';
  
  // Control tags
  '@apiPermission': 'name';
  '@apiSampleRequest': 'url|off';
  '@apiUse': 'name';
  '@apiDefine': 'name [title]';
}

Internal Data Structures

ParsedElement

interface ParsedElement {
  type: string;                     // Element type (e.g., 'api', 'apidefine')
  name: string;                     // API endpoint name
  group: string;                    // API group/category
  version: string;                  // API version
  filename: string;                 // Source file path
  title?: string;                   // API title/description
  description?: string;             // Detailed description
  url?: string;                     // API endpoint URL
  parameter?: ParameterElement[];   // Request parameters
  query?: ParameterElement[];       // Query parameters
  body?: ParameterElement[];        // Body parameters
  header?: ParameterElement[];      // Header parameters
  success?: SuccessElement[];       // Success responses
  error?: ErrorElement[];           // Error responses
  examples?: ExampleElement[];      // Usage examples
  permission?: PermissionElement[]; // Required permissions
  sampleRequest?: string | boolean; // Sample request URL
  deprecated?: DeprecatedElement;   // Deprecation info
}

Parameter Types

interface ParameterElement {
  group?: string;           // Parameter group
  type?: string;            // Data type
  size?: string;            // Size constraints
  allowedValues?: string[]; // Allowed values
  optional?: boolean;       // Whether optional
  field: string;            // Parameter name
  defaultValue?: any;       // Default value
  description?: string;     // Description
}

interface SuccessElement {
  group?: string;      // Response group
  type?: string;       // Data type
  size?: string;       // Size constraints
  field: string;       // Field name
  optional?: boolean;  // Whether optional
  description?: string; // Description
}

interface ErrorElement {
  group?: string;      // Error group
  field: string;       // Error code/field
  description?: string; // Error description
}

interface ExampleElement {
  type: string;        // Example type (json, curl, etc.)
  title: string;       // Example title
  content: string;     // Example content
}

Plugin System Integration

The internal parsing system supports plugin extensions:

Plugin Types

interface PluginTypes {
  filters: Record<string, string>;   // Custom output filters
  languages: Record<string, string>; // Custom language parsers
  parsers: Record<string, string>;   // Custom tag parsers
  workers: Record<string, string>;   // Custom post-processors
}

Built-in Parsers

ApiDoc includes built-in parsers for all standard tags:

  • api: Main API definition parser
  • apiname: API name parser
  • apigroup: API group parser
  • apiparam: Parameter parser
  • apisuccess: Success response parser
  • apierror: Error response parser
  • apiexample: Example parser
  • And many more...

Built-in Workers

Workers handle post-processing tasks:

  • apistructure: Validates API structure
  • apigroup: Processes group information
  • apiname: Validates API names
  • apipermission: Handles permission validation
  • And others...

Error Handling

The parsing system includes comprehensive error handling:

Error Types

interface ParsingErrors {
  FileError: 'File access or reading errors';
  ParserError: 'Comment or tag parsing errors';  
  WorkerError: 'Post-processing validation errors';
}

Error Context

Errors include detailed context information:

interface ErrorContext {
  file?: string;        // Source file path
  block?: string;       // Documentation block
  element?: string;     // Specific tag element  
  source?: string;      // Source code context
  extra?: Array<Record<string, any>>; // Additional context
}

Performance Considerations

  • File Filtering: Use include/exclude filters to limit processed files
  • Parallel Processing: Large projects benefit from multiple input directories
  • Memory Usage: Very large codebases may require chunked processing
  • Caching: Template and configuration data is cached during processing

Debugging Internal Parsing

When debugging parsing issues, use these techniques:

# Enable debug logging
apidoc -i src/ -o docs/ --debug

# Dry run to see parsing without output
apidoc -i src/ -o docs/ --dry-run --verbose

# Generate JSON data for inspection
apidoc -i src/ -o docs/ --write-json

The generated api_data.json file contains the raw parsed data structure, which can help understand how your API comments are being interpreted by the parsing system.