tessl/npm-langfuse--core

Core functions and utilities for Langfuse packages including API client, logging, media handling, and OpenTelemetry tracing attributes

Overview

Eval results

Files

Media Handling

Name: tessl/npm-langfuse--core
Author: tessl

Rich media support with content-addressable storage, SHA-256 hashing, and media reference tags for embedding images, audio, video, and documents in Langfuse traces.

Capabilities

LangfuseMedia Class

A class for wrapping media objects for upload to Langfuse with automatic content hashing and reference tag generation.

class LangfuseMedia {
  constructor(params: LangfuseMediaParams);

  async getId(): Promise<string | null>;
  async getSha256Hash(): Promise<string | undefined>;
  async getTag(): Promise<string | null>;

  get contentLength(): number | undefined;
  get base64DataUri(): string | null;

  toJSON(): string | null;
}

type LangfuseMediaParams =
  | {
      source: "base64_data_uri";
      base64DataUri: string;
    }
  | {
      source: "bytes";
      contentBytes: Uint8Array;
      contentType: MediaContentType;
    };

type MediaContentType =
  // Images
  | "image/png" | "image/jpeg" | "image/jpg" | "image/webp"
  | "image/gif" | "image/svg+xml" | "image/tiff" | "image/bmp"
  // Audio
  | "audio/mpeg" | "audio/mp3" | "audio/wav" | "audio/ogg"
  | "audio/oga" | "audio/aac" | "audio/mp4" | "audio/flac"
  // Video
  | "video/mp4" | "video/webm"
  // Text
  | "text/plain" | "text/html" | "text/css" | "text/csv"
  // Documents
  | "application/pdf" | "application/msword" | "application/vnd.ms-excel"
  // Archives
  | "application/zip"
  // Data
  | "application/json" | "application/xml" | "application/octet-stream";

Import:

import { LangfuseMedia, type LangfuseMediaParams, type MediaContentType } from '@langfuse/core';

Constructor

Creates a new LangfuseMedia instance from either a base64 data URI or raw bytes.

constructor(params: LangfuseMediaParams)

Parameters:

Option 1: Base64 Data URI

params.source: "base64_data_uri" - Indicates media is from a data URI
params.base64DataUri: string - Complete data URI (e.g., "data:image/png;base64,...")

Option 2: Raw Bytes

params.source: "bytes" - Indicates media is from raw bytes
params.contentBytes: Uint8Array - The raw content bytes
params.contentType: MediaContentType - MIME type of the content

Usage Examples:

import { LangfuseMedia } from '@langfuse/core';

// From base64 data URI (common with web APIs)
const mediaFromUri = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAUA..."
});

// From raw bytes (common with file system operations)
const imageBytes = new Uint8Array([137, 80, 78, 71, 13, 10, 26, 10, ...]);
const mediaFromBytes = new LangfuseMedia({
  source: "bytes",
  contentBytes: imageBytes,
  contentType: "image/png"
});

// From file in Node.js
import { readFile } from 'fs/promises';

const fileBuffer = await readFile('screenshot.png');
const mediaFromFile = new LangfuseMedia({
  source: "bytes",
  contentBytes: new Uint8Array(fileBuffer),
  contentType: "image/png"
});

// From canvas in browser
const canvas = document.getElementById('myCanvas') as HTMLCanvasElement;
const dataUri = canvas.toDataURL('image/png');
const mediaFromCanvas = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: dataUri
});

getId()

Gets a unique content-based identifier for the media.

/**
 * Gets a unique identifier derived from the SHA-256 hash of the content
 * @returns The media ID (22 characters) or null if hash generation failed
 */
async getId(): Promise<string | null>;

Returns: A 22-character URL-safe base64-encoded string derived from the SHA-256 hash, or null if unavailable.

Description: The ID is deterministic - identical content always produces the same ID, enabling content-addressable storage and deduplication.

Usage Example:

const media = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: "data:image/png;base64,iVBORw0..."
});

const mediaId = await media.getId();
console.log(mediaId);  // "A1B2C3D4E5F6G7H8I9J0K1"

// Same content = same ID (deduplication)
const media2 = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: "data:image/png;base64,iVBORw0..."
});
const mediaId2 = await media2.getId();
console.log(mediaId === mediaId2);  // true

getSha256Hash()

Gets the SHA-256 hash of the media content.

/**
 * Gets the SHA-256 hash of the content as a base64-encoded string
 * @returns The hash or undefined if unavailable
 */
async getSha256Hash(): Promise<string | undefined>;

Returns: Base64-encoded SHA-256 hash or undefined if content is unavailable.

Usage Example:

const media = new LangfuseMedia({
  source: "bytes",
  contentBytes: new Uint8Array([72, 101, 108, 108, 111]),
  contentType: "text/plain"
});

const hash = await media.getSha256Hash();
console.log(hash);  // Base64-encoded SHA-256 hash

// Use for integrity verification
const verifyHash = await media.getSha256Hash();
console.log(hash === verifyHash);  // true

getTag()

Gets the media reference tag for embedding in trace data.

/**
 * Gets the media reference tag for embedding in trace attributes
 * @returns The media tag or null if required data is missing
 */
async getTag(): Promise<string | null>;

Returns: A special tag string in the format @@@langfuseMedia:type=<contentType>|id=<mediaId>|source=<source>@@@, or null if required data is missing.

Description: This tag can be embedded in trace input/output or observation data. When viewed in the Langfuse UI, the tag is replaced with the actual media content (image viewer, video player, etc.).

Usage Example:

const media = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: "data:image/png;base64,iVBORw0KGgoAAAA..."
});

const tag = await media.getTag();
console.log(tag);
// "@@@langfuseMedia:type=image/png|id=A1B2C3D4E5F6G7H8I9J0K1|source=base64_data_uri@@@"

// Embed in trace data
import { LangfuseAPIClient } from '@langfuse/core';

const client = new LangfuseAPIClient({ /* ... */ });

await client.ingestion.batch({
  batch: [{
    type: 'trace-create',
    id: 'event-1',
    timestamp: new Date().toISOString(),
    body: {
      id: 'trace-1',
      name: 'Image Processing',
      input: {
        description: 'Processing this image:',
        image: tag  // Embedded media tag
      },
      output: {
        result: 'Image processed successfully'
      }
    }
  }]
});

contentLength (getter)

Gets the length of the media content in bytes.

/**
 * Gets the content length in bytes
 */
get contentLength(): number | undefined;

Returns: The content size in bytes, or undefined if no content is available.

Usage Example:

const media = new LangfuseMedia({
  source: "bytes",
  contentBytes: new Uint8Array(1024 * 100),  // 100 KB
  contentType: "image/jpeg"
});

console.log(`Size: ${media.contentLength} bytes`);  // "Size: 102400 bytes"

// Check size before upload
if (media.contentLength && media.contentLength > 10 * 1024 * 1024) {
  console.warn('Media exceeds 10MB, consider compression');
}

base64DataUri (getter)

Gets the media content as a base64 data URI.

/**
 * Gets the complete base64 data URI
 */
get base64DataUri(): string | null;

Returns: The complete data URI string (e.g., "data:image/png;base64,..."), or null if no content.

Usage Example:

const media = new LangfuseMedia({
  source: "bytes",
  contentBytes: new Uint8Array([72, 101, 108, 108, 111]),
  contentType: "text/plain"
});

const dataUri = media.base64DataUri;
console.log(dataUri);  // "data:text/plain;base64,SGVsbG8="

// Use in HTML img tag
const img = document.createElement('img');
img.src = dataUri;
document.body.appendChild(img);

toJSON()

Serializes the media to JSON (returns the base64 data URI).

/**
 * Serializes the media to JSON
 * @returns The base64 data URI or null
 */
toJSON(): string | null;

Returns: The base64 data URI, or null if no content is available.

Usage Example:

const media = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: "data:image/png;base64,iVBORw0..."
});

const json = JSON.stringify({ screenshot: media });
// Automatically calls toJSON()
console.log(json);
// '{"screenshot":"data:image/png;base64,iVBORw0..."}'

// Parse back
const parsed = JSON.parse(json);
const restoredMedia = new LangfuseMedia({
  source: "base64_data_uri",
  base64DataUri: parsed.screenshot
});

ParsedMediaReference Type

Represents a parsed media reference from trace data.

interface ParsedMediaReference {
  mediaId: string;
  source: string;
  contentType: MediaContentType;
}

Import:

import { type ParsedMediaReference } from '@langfuse/core';

Properties:

mediaId: string - The unique media identifier (22 characters)
source: string - The source type ("base64_data_uri" or "bytes")
contentType: MediaContentType - The MIME type of the media

Usage Patterns

Basic Image Upload

import { LangfuseMedia, LangfuseAPIClient } from '@langfuse/core';

async function uploadScreenshot(imageDataUri: string) {
  // Create media object
  const media = new LangfuseMedia({
    source: "base64_data_uri",
    base64DataUri: imageDataUri
  });

  // Get media tag for embedding
  const mediaTag = await media.getTag();

  // Create trace with embedded media
  const client = new LangfuseAPIClient({ /* ... */ });

  await client.ingestion.batch({
    batch: [{
      type: 'trace-create',
      id: 'event-1',
      timestamp: new Date().toISOString(),
      body: {
        id: 'trace-1',
        name: 'UI Screenshot',
        input: {
          action: 'screenshot_captured',
          image: mediaTag
        }
      }
    }]
  });
}

Multiple Media Types

import { LangfuseMedia } from '@langfuse/core';

async function processMultimodalInput(
  image: Uint8Array,
  audio: Uint8Array,
  text: string
) {
  // Create media objects
  const imageMedia = new LangfuseMedia({
    source: "bytes",
    contentBytes: image,
    contentType: "image/jpeg"
  });

  const audioMedia = new LangfuseMedia({
    source: "bytes",
    contentBytes: audio,
    contentType: "audio/mp3"
  });

  // Get tags
  const imageTag = await imageMedia.getTag();
  const audioTag = await audioMedia.getTag();

  // Create trace with multiple media
  await client.ingestion.batch({
    batch: [{
      type: 'trace-create',
      id: 'event-1',
      timestamp: new Date().toISOString(),
      body: {
        id: 'trace-1',
        name: 'Multimodal Processing',
        input: {
          text: text,
          image: imageTag,
          audio: audioTag
        }
      }
    }]
  });
}

File System Integration (Node.js)

import { LangfuseMedia } from '@langfuse/core';
import { readFile } from 'fs/promises';
import { extname } from 'path';

const mimeTypes: Record<string, MediaContentType> = {
  '.png': 'image/png',
  '.jpg': 'image/jpeg',
  '.jpeg': 'image/jpeg',
  '.gif': 'image/gif',
  '.pdf': 'application/pdf',
  '.mp3': 'audio/mp3',
  '.mp4': 'video/mp4'
};

async function createMediaFromFile(filePath: string) {
  // Read file
  const buffer = await readFile(filePath);
  const ext = extname(filePath).toLowerCase();
  const contentType = mimeTypes[ext];

  if (!contentType) {
    throw new Error(`Unsupported file type: ${ext}`);
  }

  // Create media
  return new LangfuseMedia({
    source: "bytes",
    contentBytes: new Uint8Array(buffer),
    contentType: contentType
  });
}

// Usage
const media = await createMediaFromFile('./screenshot.png');
const tag = await media.getTag();

Canvas Capture (Browser)

import { LangfuseMedia } from '@langfuse/core';

function captureCanvas(canvas: HTMLCanvasElement) {
  // Convert canvas to data URI
  const dataUri = canvas.toDataURL('image/png');

  // Create media
  return new LangfuseMedia({
    source: "base64_data_uri",
    base64DataUri: dataUri
  });
}

// Usage
const canvas = document.getElementById('myCanvas') as HTMLCanvasElement;
const media = captureCanvas(canvas);
const tag = await media.getTag();

Media Deduplication

import { LangfuseMedia } from '@langfuse/core';

async function trackMediaUsage(dataUri: string) {
  const media = new LangfuseMedia({
    source: "base64_data_uri",
    base64DataUri: dataUri
  });

  const mediaId = await media.getId();

  // Check if already uploaded
  const mediaCache = new Map<string, boolean>();

  if (mediaCache.has(mediaId!)) {
    console.log('Media already uploaded, reusing reference');
  } else {
    console.log('New media, uploading');
    mediaCache.set(mediaId!, true);
  }

  return media.getTag();
}

Size Validation

import { LangfuseMedia } from '@langfuse/core';

async function createMediaWithValidation(
  contentBytes: Uint8Array,
  contentType: MediaContentType,
  maxSize: number = 10 * 1024 * 1024  // 10 MB
) {
  if (contentBytes.length > maxSize) {
    throw new Error(
      `Media size ${contentBytes.length} exceeds maximum ${maxSize} bytes`
    );
  }

  const media = new LangfuseMedia({
    source: "bytes",
    contentBytes,
    contentType
  });

  console.log(`Created media: ${media.contentLength} bytes, ${contentType}`);
  return media;
}

Compression Before Upload

import { LangfuseMedia } from '@langfuse/core';

async function compressAndCreateMedia(
  imageDataUri: string,
  maxSizeKb: number = 500
) {
  // Load image
  const img = new Image();
  img.src = imageDataUri;
  await new Promise(resolve => img.onload = resolve);

  // Create canvas for compression
  const canvas = document.createElement('canvas');
  const ctx = canvas.getContext('2d')!;

  // Calculate new dimensions
  let width = img.width;
  let height = img.height;
  const maxDimension = 1920;

  if (width > maxDimension || height > maxDimension) {
    if (width > height) {
      height = Math.round((height * maxDimension) / width);
      width = maxDimension;
    } else {
      width = Math.round((width * maxDimension) / height);
      height = maxDimension;
    }
  }

  canvas.width = width;
  canvas.height = height;
  ctx.drawImage(img, 0, 0, width, height);

  // Try different quality levels
  let quality = 0.9;
  let compressedDataUri: string;

  do {
    compressedDataUri = canvas.toDataURL('image/jpeg', quality);
    quality -= 0.1;
  } while (
    compressedDataUri.length > maxSizeKb * 1024 * 1.37 &&
    quality > 0.1
  );

  // Create media from compressed image
  return new LangfuseMedia({
    source: "base64_data_uri",
    base64DataUri: compressedDataUri
  });
}

Type Definitions

class LangfuseMedia {
  constructor(params: LangfuseMediaParams);
  async getId(): Promise<string | null>;
  async getSha256Hash(): Promise<string | undefined>;
  async getTag(): Promise<string | null>;
  get contentLength(): number | undefined;
  get base64DataUri(): string | null;
  toJSON(): string | null;
}

type LangfuseMediaParams =
  | {
      source: "base64_data_uri";
      base64DataUri: string;
    }
  | {
      source: "bytes";
      contentBytes: Uint8Array;
      contentType: MediaContentType;
    };

type MediaContentType =
  | "image/png" | "image/jpeg" | "image/jpg" | "image/webp"
  | "image/gif" | "image/svg+xml" | "image/tiff" | "image/bmp"
  | "audio/mpeg" | "audio/mp3" | "audio/wav" | "audio/ogg"
  | "audio/oga" | "audio/aac" | "audio/mp4" | "audio/flac"
  | "video/mp4" | "video/webm"
  | "text/plain" | "text/html" | "text/css" | "text/csv"
  | "application/pdf" | "application/msword" | "application/vnd.ms-excel"
  | "application/zip"
  | "application/json" | "application/xml" | "application/octet-stream";

interface ParsedMediaReference {
  mediaId: string;
  source: string;
  contentType: MediaContentType;
}

Best Practices

Content Type Accuracy: Always specify the correct MIME type for reliable media handling
Size Limits: Consider compressing large media before upload to reduce storage costs
Deduplication: Leverage content-based IDs to avoid uploading duplicate media
Error Handling: Always check for null returns from async methods (getId, getTag)
Browser Compatibility: Use appropriate polyfills for crypto APIs in older browsers
Memory Management: Process large files in streams rather than loading entirely in memory
Format Selection: Prefer web-optimized formats (JPEG for photos, PNG for graphics, WebP when supported)
Metadata: Include descriptive context around media in trace input/output, not just the tag
Security: Never include sensitive information in media that might be stored long-term
Testing: Test media handling with various file sizes and types during development