CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-arizeai--phoenix-client

A TypeScript client for the Phoenix API providing AI observability, prompt management, datasets, experiments, and tracing capabilities.

Overview
Eval results
Files

datasets.mddocs/

Dataset Management

Dataset creation, management, and versioning system for AI evaluation and experimentation workflows. Supports structured examples with input/output pairs and metadata for comprehensive evaluation datasets.

Capabilities

Dataset Creation

Create new datasets with structured examples for AI model evaluation and experimentation.

/**
 * Create a new dataset with examples
 * @param params - Dataset creation parameters
 * @returns Promise resolving to dataset creation response
 */
function createDataset(params: {
  client?: PhoenixClient;
  name: string;
  description: string;
  examples: Example[];
}): Promise<CreateDatasetResponse>;

interface Example {
  input: Record<string, unknown>;
  output?: Record<string, unknown> | null;
  metadata?: Record<string, unknown> | null;
}

interface CreateDatasetResponse {
  datasetId: string;
}

Usage Example:

import { createDataset } from "@arizeai/phoenix-client/datasets";

const response = await createDataset({
  name: "customer-support-qa",
  description: "Customer support Q&A dataset for evaluation",
  examples: [
    {
      input: { question: "How do I reset my password?" },
      output: { answer: "Click 'Forgot Password' on the login page and follow the instructions." },
      metadata: { category: "authentication", difficulty: "easy" }
    },
    {
      input: { question: "What are your business hours?" },
      output: { answer: "We are open Monday-Friday 9AM-6PM EST." },
      metadata: { category: "general", difficulty: "easy" }
    }
  ]
});

console.log(`Created dataset with ID: ${response.datasetId}`);

Dataset Retrieval

Retrieve complete datasets by ID or name, with optional version specification.

/**
 * Get a dataset by selector with optional version
 * @param params - Dataset retrieval parameters
 * @returns Promise resolving to complete dataset
 */
function getDataset(params: {
  client?: PhoenixClient;
  dataset: DatasetSelector;
  versionId?: string;
}): Promise<Dataset>;

type DatasetSelector =
  | { datasetId: string }
  | { datasetName: string };

interface Dataset extends DatasetInfo {
  examples: ExampleWithId[];
  versionId: string;
}

interface DatasetInfo {
  id: string;
  name: string;
  description?: string | null;
  metadata?: Record<string, unknown>;
}

interface ExampleWithId extends Example {
  id: string;
  updatedAt: Date;
}

Usage Examples:

import { getDataset } from "@arizeai/phoenix-client/datasets";

// Get dataset by ID
const dataset = await getDataset({
  dataset: { datasetId: "dataset_123" }
});

// Get dataset by name
const dataset = await getDataset({
  dataset: { datasetName: "customer-support-qa" }
});

// Get specific version
const dataset = await getDataset({
  dataset: { datasetName: "customer-support-qa" },
  versionId: "version_456"
});

console.log(`Dataset ${dataset.name} has ${dataset.examples.length} examples`);

Dataset Examples Retrieval

Retrieve only the examples from a dataset without full dataset metadata.

/**
 * Get examples from a dataset
 * @param params - Dataset examples retrieval parameters
 * @returns Promise resolving to dataset examples
 */
function getDatasetExamples(params: {
  client?: PhoenixClient;
  dataset: DatasetSelector;
  versionId?: string;
}): Promise<DatasetExamples>;

interface DatasetExamples {
  examples: ExampleWithId[];
  versionId: string;
}

Usage Example:

import { getDatasetExamples } from "@arizeai/phoenix-client/datasets";

const { examples, versionId } = await getDatasetExamples({
  dataset: { datasetName: "eval-dataset" }
});

// Process examples for evaluation
for (const example of examples) {
  console.log(`Input: ${JSON.stringify(example.input)}`);
  if (example.output) {
    console.log(`Expected output: ${JSON.stringify(example.output)}`);
  }
}

Dataset Examples Appending

Add new examples to an existing dataset, creating a new version.

/**
 * Append examples to an existing dataset
 * @param params - Dataset append parameters
 * @returns Promise resolving to append operation response
 */
function appendDatasetExamples(params: {
  client?: PhoenixClient;
  dataset: DatasetSelector;
  examples: Example[];
}): Promise<void>;

Usage Example:

import { appendDatasetExamples } from "@arizeai/phoenix-client/datasets";

await appendDatasetExamples({
  dataset: { datasetName: "customer-support-qa" },
  examples: [
    {
      input: { question: "How do I cancel my subscription?" },
      output: { answer: "Visit your account settings and click 'Cancel Subscription'." },
      metadata: { category: "billing", difficulty: "medium" }
    }
  ]
});

Dataset Information

Get dataset metadata without examples for lightweight operations.

/**
 * Get dataset metadata without examples
 * @param params - Dataset info parameters
 * @returns Promise resolving to dataset information
 */
function getDatasetInfo(params: {
  client?: PhoenixClient;
  dataset: DatasetSelector;
}): Promise<DatasetInfo>;

Usage Example:

import { getDatasetInfo } from "@arizeai/phoenix-client/datasets";

const info = await getDatasetInfo({
  dataset: { datasetName: "large-evaluation-set" }
});

console.log(`Dataset: ${info.name}`);
console.log(`Description: ${info.description}`);
console.log(`Metadata:`, info.metadata);

Dataset Information by Name

Get dataset metadata by name without the full dataset selector interface.

/**
 * Get dataset information by name only
 * @param params - Dataset info by name parameters
 * @returns Promise resolving to dataset information
 */
function getDatasetInfoByName(params: {
  client?: PhoenixClient;
  datasetName: string;
}): Promise<DatasetInfo>;

Usage Example:

import { getDatasetInfoByName } from "@arizeai/phoenix-client/datasets";

const info = await getDatasetInfoByName({
  datasetName: "customer-support-qa"
});

console.log(`Found dataset: ${info.name} (ID: ${info.id})`);

List All Datasets

Retrieve information about all available datasets with creation and update timestamps.

/**
 * List all datasets available to the client
 * @param params - List datasets parameters
 * @returns Promise resolving to array of dataset information with timestamps
 */
function listDatasets(params: {
  client?: PhoenixClient;
}): Promise<FullDatasetInfo[]>;

interface FullDatasetInfo extends DatasetInfo {
  startDate: Date;
  endDate: Date;
}

Usage Example:

import { listDatasets } from "@arizeai/phoenix-client/datasets";

const datasets = await listDatasets({});

console.log(`Found ${datasets.length} datasets:`);
for (const dataset of datasets) {
  console.log(`- ${dataset.name}: ${dataset.description || 'No description'}`);
  console.log(`  Created: ${dataset.startDate.toISOString()}`);
  console.log(`  Updated: ${dataset.endDate.toISOString()}`);
}

Dataset Versioning

Datasets support versioning to track changes over time and maintain reproducible experiments.

/**
 * Dataset version selector for retrieving specific versions
 */
interface DatasetVersionSelector {
  dataset: DatasetSelector;
  versionId?: string;
}

/**
 * Information about a specific dataset version
 */
interface DatasetVersionInfo {
  id: string;
  description?: string | null;
  metadata?: Record<string, unknown>;
  createdAt: Date;
}

/**
 * Dataset with version information
 */
interface DatasetWithVersion extends Dataset {
  versionInfo: DatasetVersionInfo;
}

Example Structure

Dataset examples follow a flexible structure supporting various AI use cases.

Example Components:

  • input: Required input data for the AI task (any JSON structure)
  • output: Optional expected output or ground truth (any JSON structure)
  • metadata: Optional additional information for categorization and analysis

Common Example Patterns:

// Question-answering
{
  input: { question: "What is the capital of France?" },
  output: { answer: "Paris" },
  metadata: { category: "geography", difficulty: "easy" }
}

// Text classification
{
  input: { text: "This product is amazing!" },
  output: { sentiment: "positive", confidence: 0.95 },
  metadata: { source: "reviews", language: "en" }
}

// Function calling
{
  input: {
    query: "What's the weather in New York?",
    tools: ["get_weather", "get_location"]
  },
  output: {
    function_call: "get_weather",
    arguments: { location: "New York, NY" }
  },
  metadata: { use_case: "function_calling" }
}

// Multi-modal
{
  input: {
    image_url: "https://example.com/image.jpg",
    question: "What objects are in this image?"
  },
  output: { objects: ["car", "tree", "building"] },
  metadata: { modality: "vision", source: "coco" }
}

Best Practices

  • Structured Input: Use consistent input structure within a dataset
  • Meaningful Metadata: Include categorization and source information
  • Version Management: Create new versions when adding significant examples
  • Validation: Ensure output format matches expected evaluation criteria
  • Documentation: Provide clear dataset descriptions for team collaboration

Install with Tessl CLI

npx tessl i tessl/npm-arizeai--phoenix-client

docs

client.md

datasets.md

experiments.md

index.md

prompts.md

sdk-integration.md

spans.md

tile.json