tessl/npm-llamaindex

Data framework for your LLM application

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Securityby

Pending

The risk profile of this skill

Overview

Eval results

Files

Settings and Configuration

Name: tessl/npm-llamaindex
Author: tessl

Global configuration system for managing LLMs, embedding models, and processing parameters across the LlamaIndex.TS framework.

Import

import { Settings } from "llamaindex";

Overview

The Settings object provides centralized configuration management for all LlamaIndex.TS components. It uses a singleton pattern to ensure consistent settings across your application and provides temporary context switching capabilities.

Settings Object

interface GlobalSettings {
  llm: LLM;
  embedModel: BaseEmbedding;
  nodeParser: NodeParser;
  promptHelper: PromptHelper;
  callbackManager: CallbackManager;
  chunkSize: number | undefined;
  chunkOverlap: number | undefined;
  prompt: PromptConfig;
  debug: boolean;
  
  withLLM<Result>(llm: LLM, fn: () => Result): Result;
  withEmbedModel<Result>(embedModel: BaseEmbedding, fn: () => Result): Result;
  withNodeParser<Result>(nodeParser: NodeParser, fn: () => Result): Result;
  withPromptHelper<Result>(promptHelper: PromptHelper, fn: () => Result): Result;
  withCallbackManager<Result>(callbackManager: CallbackManager, fn: () => Result): Result;
  withChunkSize<Result>(chunkSize: number, fn: () => Result): Result;
  withChunkOverlap<Result>(chunkOverlap: number, fn: () => Result): Result;
  withPrompt<Result>(prompt: PromptConfig, fn: () => Result): Result;
}

Configuration Types

interface Config {
  prompt: PromptConfig;
  promptHelper: PromptHelper | null;
  embedModel: BaseEmbedding | null;
  nodeParser: NodeParser | null;
  callbackManager: CallbackManager | null;
  chunkSize: number | undefined;
  chunkOverlap: number | undefined;
}

interface PromptConfig {
  llm?: string;
  lang?: string;
}

Basic Usage

Global Configuration

import { Settings, OpenAI, OpenAIEmbedding } from "llamaindex";

// Set global LLM
Settings.llm = new OpenAI({
  model: "gpt-4",
  temperature: 0.1,
});

// Set global embedding model
Settings.embedModel = new OpenAIEmbedding({
  model: "text-embedding-ada-002",
});

// Set chunk size for text splitting
Settings.chunkSize = 1024;
Settings.chunkOverlap = 20;

// Enable debug mode
Settings.debug = true;

Temporary Context Switching

Use the with* methods to temporarily override settings for specific operations:

import { Settings, OpenAI, Document, VectorStoreIndex } from "llamaindex";

const documents = [new Document({ text: "Example content" })];

// Use different LLM for this specific operation
const result = Settings.withLLM(
  new OpenAI({ model: "gpt-3.5-turbo" }),
  () => {
    return VectorStoreIndex.fromDocuments(documents);
  }
);

// Settings.llm reverts to original value after the function completes

Multiple Settings Override

// Override multiple settings temporarily
Settings.withChunkSize(512, () => {
  Settings.withChunkOverlap(50, () => {
    // Process with different chunking parameters
    const index = VectorStoreIndex.fromDocuments(documents);
    return index;
  });
});

Properties

Core Components

llm: The language model used for text generation and completion
embedModel: The embedding model used for generating vector representations
nodeParser: The text splitter used for chunking documents
promptHelper: Helper for managing prompt templates and formatting
callbackManager: Manager for handling events and callbacks

Processing Parameters

chunkSize: Default size for text chunks (undefined uses model default)
chunkOverlap: Overlap between text chunks for better continuity
prompt: Prompt configuration including language and LLM preferences
debug: Enable debug logging and verbose output

Context Methods

All with* methods follow the same pattern - they temporarily override a setting for the duration of the provided function:

withLLM(llm, fn)

Temporarily use a different language model.

Parameters:

llm: LLM - The LLM to use temporarily
fn: () => Result - Function to execute with the temporary LLM

Returns: The result of executing fn

withEmbedModel(embedModel, fn)

Temporarily use a different embedding model.

Parameters:

embedModel: BaseEmbedding - The embedding model to use temporarily
fn: () => Result - Function to execute with the temporary embedding model

Returns: The result of executing fn

withChunkSize(chunkSize, fn)

Temporarily use a different chunk size.

Parameters:

chunkSize: number - The chunk size to use temporarily
fn: () => Result - Function to execute with the temporary chunk size

Returns: The result of executing fn

Best Practices

Initialize Early

Set up your global settings at the start of your application:

// At app initialization
Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
Settings.embedModel = new OpenAIEmbedding();
Settings.chunkSize = 1024;
Settings.chunkOverlap = 20;

Use Context Methods for Variations

Use temporary context switching when you need different settings for specific operations:

// Different chunk size for processing large documents
const processLargeDoc = (doc: Document) => {
  return Settings.withChunkSize(2048, () => {
    return VectorStoreIndex.fromDocuments([doc]);
  });
};

// Different LLM for specific queries
const getCheapAnswer = (query: string, index: VectorStoreIndex) => {
  return Settings.withLLM(new OpenAI({ model: "gpt-3.5-turbo" }), () => {
    const queryEngine = index.asQueryEngine();
    return queryEngine.query(query);
  });
};

Environment-Specific Configuration

Configure settings based on your environment:

if (process.env.NODE_ENV === "development") {
  Settings.debug = true;
  Settings.llm = new OpenAI({ model: "gpt-3.5-turbo" }); // Cheaper for dev
} else {
  Settings.debug = false;
  Settings.llm = new OpenAI({ model: "gpt-4" }); // Better for prod
}