or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

configuration.md index.md language-extensions.md line-features.md shell-features.md syntax-highlighting.md

tile.json

Language Extensions

System for adding custom language definitions or extending existing PrismJS languages with custom tokens and syntax rules.

Capabilities

Language Extension Interface

Defines the structure for custom language extensions that can be added to PrismJS.

/**
 * Language extension configuration
 */
interface LanguageExtension {
  /** Name of the new language (required if not extending existing language) */
  language?: string;
  /** Language to extend (required if not creating new language) */
  extend?: string;
  /** Language definition tokens (RegExp or string patterns) */
  definition?: Record<string, RegExp | string>;
  /** Insert tokens before existing ones in grammar */
  insertBefore?: Record<string, Record<string, RegExp | string>>;
}

Requirements:

One of language or extend must be provided
One of definition or insertBefore must be provided
When using insertBefore, either extend or definition must also be provided
String patterns are automatically converted to RegExp objects

Load Language Extension Function

Loads custom language extensions into the PrismJS system.

/**
 * Loads custom language extensions into PrismJS
 * @param languageExtensions - Array of language extension configurations
 * @throws Error if extension configuration is invalid
 */
function loadLanguageExtension(languageExtensions: LanguageExtension[]): void;

Usage Examples:

// Create a new language from scratch
const customLanguage = {
  language: "mylang",
  definition: {
    keyword: /\b(?:func|var|if|else)\b/,
    string: /"(?:[^"\\]|\\.)*"/,
    number: /\b\d+\b/
  }
};

// Extend an existing language
const extendedJS = {
  language: "superjs",
  extend: "javascript", 
  definition: {
    superscript_types: /(SuperType)/,
  },
  insertBefore: {
    function: {
      superscript_keywords: /(superif|superelse)/,
    },
  },
};

// Load extensions
loadLanguageExtension([customLanguage, extendedJS]);

Extension Patterns

Different patterns for creating and extending languages.

Create New Language:

{
  language: "newlang",
  definition: {
    comment: /\/\/.*$/m,
    keyword: /\b(?:def|class|if|else)\b/,
    string: /"(?:[^"\\]|\\.)*"/,
    operator: /[+\-*\/=]/
  }
}

Extend Existing Language:

{
  extend: "javascript",
  definition: {
    custom_keyword: /\b(?:async|await)\b/
  }
}

Create Language Based on Another:

{
  language: "enhanced-js",
  extend: "javascript",
  definition: {
    enhanced_feature: /\$\{[^}]+\}/
  }
}

Insert Before Existing Tokens:

{
  extend: "javascript",
  insertBefore: {
    function: {
      arrow_function: /=>\s*\{/
    }
  }
}

String to RegExp Conversion

The system automatically converts string patterns to RegExp objects for compatibility.

/**
 * Recursively converts string patterns to RegExp objects
 * @param object - Object containing string patterns to convert
 * @returns Object with RegExp patterns
 */
function replaceStringWithRegex(object: Record<string, any>): Record<string, any>;

Usage Example:

// Input with string patterns
const extension = {
  language: "test",
  definition: {
    keyword: "\\b(?:func|var)\\b",  // String pattern
    number: "\\d+"                  // String pattern
  }
};

// Automatically converted to:
// {
//   language: "test", 
//   definition: {
//     keyword: /\b(?:func|var)\b/,  // RegExp object
//     number: /\d+/                 // RegExp object
//   }
// }

Token Insertion Points

When using insertBefore, specify where in the existing grammar to insert new tokens.

Common Insertion Points:

keyword: Before keyword tokens
function: Before function-related tokens
string: Before string literals
operator: Before operators
punctuation: Before punctuation marks
comment: Before comments

Example with Multiple Insertion Points:

{
  extend: "javascript",
  insertBefore: {
    keyword: {
      special_keyword: /\b(?:magic|spell)\b/
    },
    function: {
      lambda: /=>/
    },
    string: {
      template: /`[^`]*`/
    }
  }
}

CSS Integration

When creating custom tokens, you need corresponding CSS for styling.

Generated HTML Structure:

<span class="token custom_keyword">magic</span>
<span class="token superscript_types">SuperType</span>

Required CSS:

.token.custom_keyword {
  color: #d73a49;
  font-weight: bold;
}

.token.superscript_types {
  color: #6f42c1;
  font-style: italic;
}

Error Handling

The extension system provides detailed error messages for invalid configurations.

Configuration Errors:

Extension must be an object, not array or primitive
Must contain required properties (language/extend and definition/insertBefore)
insertBefore cannot be used without extend or definition

RegExp Conversion Errors:

Invalid regex patterns log warnings but continue processing
Malformed patterns are skipped with console warnings

Usage Example with Error Handling:

// Invalid extension - missing required properties
try {
  loadLanguageExtension([{
    // Missing language/extend and definition/insertBefore
  }]);
} catch (error) {
  console.error(error.message);
  // "A languageExtension needs to contain 'language' and 'extend' or both and a 'definition'"
}

// Valid extension
loadLanguageExtension([{
  language: "example",
  definition: {
    keyword: /\bexample\b/
  }
}]);

Version

Tile

Files