or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.mdparsing.mdrequests.mdservices.md
tile.json

parsing.mddocs/

Notebook Parsing

Utilities for analyzing Jupyter notebook content and extracting metadata like environment variables used in notebook cells. The NotebookParser class provides static methods for parsing notebook JSON and code analysis.

import { NotebookParser, IDictionary } from "@elyra/application";

Capabilities

Notebook Parser Class

Utilities class for parsing notebook files and extracting information.

/**
 * A utilities class for parsing notebook files.
 */
class NotebookParser {
  /**
   * Takes in a notebook and finds all env vars accessed in it.
   * @param notebookStr - Raw notebook JSON in string format
   * @returns A string array of the env vars accessed in the given notebook
   */
  static getEnvVars(notebookStr: string): string[];
  
  /**
   * Splits code string on new lines and matches each line on the given regex.
   * @param code - Multi-line string to match on
   * @param regex - Match regex to run on each line of code
   * @returns A 2d string array containing an array of the matched array results
   */
  static findInCode(code: string, regex: RegExp): string[][];
}

Environment Variable Extraction

Extract environment variables accessed in notebook cells.

/**
 * Takes in a notebook and finds all env vars accessed in it.
 * @param notebookStr - Raw notebook JSON in string format
 * @returns A string array of the env vars accessed in the given notebook
 */
static getEnvVars(notebookStr: string): string[];

Usage Examples:

import { NotebookParser } from "@elyra/application";

// Parse notebook JSON string
const notebookJson = `{
  "cells": [
    {
      "cell_type": "code",
      "source": [
        "import os\\n",
        "print(os.environ['HOME'])\\n",
        "api_key = os.getenv('API_KEY')\\n"
      ]
    }
  ]
}`;

const envVars = NotebookParser.getEnvVars(notebookJson);
console.log(envVars); // ['HOME', 'API_KEY']

The method recognizes various Python patterns for accessing environment variables:

  • os.environ['VAR_NAME']
  • os.environ["VAR_NAME"]
  • os.environb['VAR_NAME']
  • os.environb["VAR_NAME"]
  • os.getenv('VAR_NAME')
  • os.getenv("VAR_NAME")
  • os.getenvb('VAR_NAME')
  • os.getenvb("VAR_NAME")

Code Pattern Matching

General-purpose utility for finding patterns in code using regular expressions.

/**
 * Splits code string on new lines and matches each line on the given regex.
 * @param code - Multi-line string to match on
 * @param regex - Match regex to run on each line of code
 * @returns A 2d string array containing an array of the matched array results
 */
static findInCode(code: string, regex: RegExp): string[][];

Usage Examples:

import { NotebookParser } from "@elyra/application";

// Find import statements
const code = `
import pandas as pd
import numpy as np
from sklearn import datasets
`;

const importRegex = /^(?:import|from)\s+(\w+)/;
const matches = NotebookParser.findInCode(code, importRegex);

// matches will contain arrays of regex match results
matches.forEach(match => {
  console.log('Found import:', match[1]); // pandas, numpy, sklearn
});

Custom Environment Variable Patterns:

import { NotebookParser } from "@elyra/application";

// Custom regex for environment variable patterns
const customEnvRegex = /process\.env\.(\w+)/g;
const jsCode = `
const apiKey = process.env.API_KEY;
const dbUrl = process.env.DATABASE_URL;
`;

const envMatches = NotebookParser.findInCode(jsCode, customEnvRegex);
console.log('JavaScript env vars found:', envMatches);

Type Definitions

/**
 * An interface for typing json dictionaries in typescript
 */
interface IDictionary<T> {
  [key: string]: T;
}

The IDictionary<T> interface provides a type-safe way to define objects with string keys and typed values, commonly used throughout the Elyra ecosystem for configuration objects and metadata structures.

Usage Examples:

import { IDictionary } from "@elyra/application";

// Type-safe metadata object
const metadata: IDictionary<string> = {
  name: 'my-snippet',
  language: 'Python',
  description: 'A useful code snippet'
};

// Configuration object with mixed types
const config: IDictionary<any> = {
  timeout: 5000,
  retries: 3,
  debug: true,
  endpoints: ['api1', 'api2']
};

Implementation Details

The NotebookParser uses regex patterns to analyze notebook content:

  • Environment Variable Regex: Matches various Python os.environ and os.getenv patterns
  • Line-by-Line Processing: Code is split on newlines for efficient pattern matching
  • Deduplication: Environment variables are deduplicated using Set operations
  • JSON Parsing: Notebook strings are parsed as JSON to access cell structure

The parsing is designed to work with standard Jupyter notebook JSON format and focuses on Python code cells, though the findInCode method can be used with any programming language by providing appropriate regex patterns.