or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

advanced.md client-class.md file-handling.md index.md standalone-functions.md types.md

tile.json

Advanced Topics

Advanced configuration, protocol management, environment-specific optimizations, and production deployment considerations for the @gradio/client package.

Capabilities

Protocol Management

The @gradio/client supports multiple communication protocols with automatic detection and fallback mechanisms.

interface ProtocolOptions {
  /** Preferred protocol version */
  protocol_version?: "sse_v1" | "sse_v2" | "sse_v2.1" | "sse_v3";
  /** Enable WebSocket fallback */
  websocket_fallback?: boolean;
  /** Connection timeout in milliseconds */
  timeout?: number;
  /** Retry attempts for failed connections */
  max_retries?: number;
}

Protocol Features:

SSE (Server-Sent Events): Primary streaming protocol
WebSocket: Fallback for environments where SSE is blocked
HTTP: Basic request/response for simple predictions
Automatic Protocol Detection: Client negotiates best available protocol

Usage Examples:

import { Client } from "@gradio/client";

// Force specific protocol version
const client = await Client.connect("https://app-url", {
  headers: {
    "Accept": "text/event-stream",
    "Sec-Protocol-Version": "sse_v3"
  }
});

// Monitor protocol in use
const job = client.submit("/stream", ["input"]);
for await (const message of job) {
  if (message.type === "status" && message.status?.protocol) {
    console.log("Using protocol:", message.status.protocol);
  }
}

Connection Lifecycle Management

Advanced connection handling for production applications.

interface ConnectionState {
  /** Current connection status */
  status: "connected" | "connecting" | "disconnected" | "error";
  /** Protocol information */
  protocol?: {
    version: string;
    transport: "sse" | "websocket" | "http";
  };
  /** Connection quality metrics */
  metrics?: {
    latency: number;
    reconnect_count: number;
    last_activity: Date;
  };
}

Connection Monitoring:

import { Client } from "@gradio/client";

class RobustClient {
  private client: Client | null = null;
  private reconnectAttempts = 0;
  private maxReconnectAttempts = 5;

  async connect(appUrl: string) {
    try {
      this.client = await Client.connect(appUrl, {
        headers: {
          "Connection-Timeout": "30000",
          "Keep-Alive": "timeout=30"
        }
      });
      
      this.reconnectAttempts = 0;
      console.log("Connected successfully");
    } catch (error) {
      await this.handleConnectionError(error, appUrl);
    }
  }

  private async handleConnectionError(error: Error, appUrl: string) {
    if (this.reconnectAttempts < this.maxReconnectAttempts) {
      this.reconnectAttempts++;
      const delay = Math.pow(2, this.reconnectAttempts) * 1000; // Exponential backoff
      
      console.log(`Reconnection attempt ${this.reconnectAttempts} in ${delay}ms`);
      setTimeout(() => this.connect(appUrl), delay);
    } else {
      console.error("Max reconnection attempts reached");
      throw error;
    }
  }

  async healthCheck(): Promise<boolean> {
    if (!this.client) return false;
    
    try {
      const status = await this.client.reconnect();
      return status === "connected";
    } catch {
      return false;
    }
  }
}

Environment-Specific Configuration

Optimizations and configurations for different deployment environments.

Browser Environment

// Browser-specific optimizations
const client = await Client.connect("https://app-url", {
  headers: {
    "User-Agent": navigator.userAgent,
    "Accept-Language": navigator.language,
    "DNT": "1" // Do Not Track
  },
  // Enable service worker caching
  normalise_unicode: true
});

// Handle browser-specific events
window.addEventListener("beforeunload", () => {
  client.close();
});

window.addEventListener("online", async () => {
  const status = await client.reconnect();
  console.log("Reconnection status:", status);
});

// Progressive Web App considerations
if ("serviceWorker" in navigator) {
  navigator.serviceWorker.ready.then(registration => {
    // Use service worker for offline caching
    console.log("Service worker ready for caching");
  });
}

Node.js Environment

import { Client } from "@gradio/client";
import { Agent } from "https";

// Node.js-specific configuration
const httpsAgent = new Agent({
  keepAlive: true,
  maxSockets: 50,
  timeout: 60000
});

const client = await Client.connect("https://app-url", {
  headers: {
    "User-Agent": "Node.js/@gradio/client",
    "Connection": "keep-alive"
  }
});

// Process lifecycle management
process.on("SIGINT", () => {
  console.log("Gracefully shutting down...");
  client.close();
  process.exit(0);
});

process.on("SIGTERM", () => {
  client.close();
  process.exit(0);
});

// Memory usage monitoring
setInterval(() => {
  const usage = process.memoryUsage();
  console.log("Memory usage:", {
    rss: Math.round(usage.rss / 1024 / 1024) + "MB",
    heapUsed: Math.round(usage.heapUsed / 1024 / 1024) + "MB"
  });
}, 30000);

Performance Optimization

Advanced patterns for high-performance applications.

Connection Pooling

class ClientPool {
  private clients: Client[] = [];
  private currentIndex = 0;
  private maxClients = 5;

  async initialize(appUrl: string) {
    const connectionPromises = Array(this.maxClients).fill(null).map(() =>
      Client.connect(appUrl)
    );
    
    this.clients = await Promise.all(connectionPromises);
  }

  getClient(): Client {
    const client = this.clients[this.currentIndex];
    this.currentIndex = (this.currentIndex + 1) % this.clients.length;
    return client;
  }

  async predict(endpoint: string, data: unknown[]) {
    const client = this.getClient();
    return await client.predict(endpoint, data);
  }

  close() {
    this.clients.forEach(client => client.close());
  }
}

// Usage
const pool = new ClientPool();
await pool.initialize("https://app-url");

// Distribute load across connections
const results = await Promise.all([
  pool.predict("/endpoint1", ["data1"]),
  pool.predict("/endpoint2", ["data2"]),
  pool.predict("/endpoint3", ["data3"])
]);

Batch Processing

class BatchProcessor {
  private queue: Array<{
    endpoint: string;
    data: unknown[];
    resolve: (value: any) => void;
    reject: (error: any) => void;
  }> = [];
  
  private processing = false;
  private batchSize = 10;
  private batchDelay = 100; // ms

  constructor(private client: Client) {
    this.startProcessing();
  }

  async predict(endpoint: string, data: unknown[]): Promise<any> {
    return new Promise((resolve, reject) => {
      this.queue.push({ endpoint, data, resolve, reject });
    });
  }

  private async startProcessing() {
    if (this.processing) return;
    this.processing = true;

    while (this.queue.length > 0) {
      const batch = this.queue.splice(0, this.batchSize);
      
      try {
        const results = await Promise.all(
          batch.map(item => this.client.predict(item.endpoint, item.data))
        );
        
        batch.forEach((item, index) => {
          item.resolve(results[index]);
        });
      } catch (error) {
        batch.forEach(item => item.reject(error));
      }

      if (this.queue.length > 0) {
        await new Promise(resolve => setTimeout(resolve, this.batchDelay));
      }
    }

    this.processing = false;
  }
}

Authentication Strategies

Advanced authentication patterns for different deployment scenarios.

Token Refresh

class AuthenticatedClient {
  private client: Client | null = null;
  private token: string | null = null;
  private tokenExpiry: Date | null = null;

  async connect(appUrl: string, initialToken: string) {
    this.token = initialToken;
    this.client = await Client.connect(appUrl, {
      hf_token: this.token
    });
  }

  private async refreshToken(): Promise<string> {
    // Implement your token refresh logic
    const response = await fetch("/api/refresh-token", {
      method: "POST",
      headers: {
        "Authorization": `Bearer ${this.token}`
      }
    });
    
    const { token, expires_at } = await response.json();
    this.tokenExpiry = new Date(expires_at);
    return token;
  }

  private async ensureValidToken() {
    if (!this.tokenExpiry || new Date() > this.tokenExpiry) {
      this.token = await this.refreshToken();
      
      // Reconnect with new token
      if (this.client) {
        this.client.close();
        this.client = await Client.connect(this.client.app_reference, {
          hf_token: this.token
        });
      }
    }
  }

  async predict(endpoint: string, data: unknown[]) {
    await this.ensureValidToken();
    return this.client!.predict(endpoint, data);
  }
}

Multi-Provider Authentication

interface AuthProvider {
  name: string;
  authenticate(): Promise<string>;
  refresh(token: string): Promise<string>;
}

class HuggingFaceAuth implements AuthProvider {
  name = "huggingface";
  
  async authenticate(): Promise<string> {
    // HuggingFace OAuth flow
    return "hf_token";
  }
  
  async refresh(token: string): Promise<string> {
    // Refresh HF token
    return "new_hf_token";
  }
}

class CustomAuth implements AuthProvider {
  name = "custom";
  
  async authenticate(): Promise<string> {
    // Custom authentication
    return "custom_token";
  }
  
  async refresh(token: string): Promise<string> {
    // Custom token refresh
    return "new_custom_token";
  }
}

class MultiAuthClient {
  private providers = new Map<string, AuthProvider>();
  
  registerProvider(provider: AuthProvider) {
    this.providers.set(provider.name, provider);
  }
  
  async connect(appUrl: string, providerName: string) {
    const provider = this.providers.get(providerName);
    if (!provider) throw new Error(`Unknown provider: ${providerName}`);
    
    const token = await provider.authenticate();
    return Client.connect(appUrl, { hf_token: token });
  }
}

Error Recovery Strategies

Comprehensive error handling and recovery mechanisms.

Circuit Breaker Pattern

class CircuitBreaker {
  private failures = 0;
  private lastFailureTime = 0;
  private state: "closed" | "open" | "half-open" = "closed";
  
  constructor(
    private threshold = 5,
    private timeout = 60000,
    private client: Client
  ) {}

  async predict(endpoint: string, data: unknown[]) {
    if (this.state === "open") {
      if (Date.now() - this.lastFailureTime > this.timeout) {
        this.state = "half-open";
      } else {
        throw new Error("Circuit breaker is open");
      }
    }

    try {
      const result = await this.client.predict(endpoint, data);
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  private onSuccess() {
    this.failures = 0;
    this.state = "closed";
  }

  private onFailure() {
    this.failures++;
    this.lastFailureTime = Date.now();
    
    if (this.failures >= this.threshold) {
      this.state = "open";
    }
  }
}

Retry with Exponential Backoff

class RetryClient {
  constructor(
    private client: Client,
    private maxRetries = 3,
    private baseDelay = 1000
  ) {}

  async predict(endpoint: string, data: unknown[]) {
    let lastError: Error;
    
    for (let attempt = 0; attempt <= this.maxRetries; attempt++) {
      try {
        return await this.client.predict(endpoint, data);
      } catch (error) {
        lastError = error as Error;
        
        if (attempt === this.maxRetries) break;
        if (!this.isRetryableError(error)) break;
        
        const delay = this.baseDelay * Math.pow(2, attempt);
        await new Promise(resolve => setTimeout(resolve, delay));
        
        // Attempt reconnection
        await this.client.reconnect();
      }
    }
    
    throw lastError!;
  }

  private isRetryableError(error: any): boolean {
    return (
      error.message.includes("network") ||
      error.message.includes("timeout") ||
      error.message.includes("queue full")
    );
  }
}

Monitoring and Observability

Production monitoring and debugging capabilities.

Metrics Collection

interface Metrics {
  requests_total: number;
  requests_failed: number;
  request_duration_ms: number[];
  connection_events: Array<{
    type: "connect" | "disconnect" | "error";
    timestamp: Date;
    details?: any;
  }>;
}

class MonitoredClient {
  private metrics: Metrics = {
    requests_total: 0,
    requests_failed: 0,
    request_duration_ms: [],
    connection_events: []
  };

  constructor(private client: Client) {
    this.setupMonitoring();
  }

  private setupMonitoring() {
    // Log connection events
    this.logConnectionEvent("connect", { app: this.client.app_reference });
  }

  async predict(endpoint: string, data: unknown[]) {
    const startTime = Date.now();
    this.metrics.requests_total++;

    try {
      const result = await this.client.predict(endpoint, data);
      const duration = Date.now() - startTime;
      this.metrics.request_duration_ms.push(duration);
      return result;
    } catch (error) {
      this.metrics.requests_failed++;
      throw error;
    }
  }

  private logConnectionEvent(type: Metrics["connection_events"][0]["type"], details?: any) {
    this.metrics.connection_events.push({
      type,
      timestamp: new Date(),
      details
    });
  }

  getMetrics(): Metrics {
    return { ...this.metrics };
  }

  getHealthStatus() {
    const failureRate = this.metrics.requests_failed / this.metrics.requests_total;
    const avgDuration = this.metrics.request_duration_ms.reduce((a, b) => a + b, 0) / 
                       this.metrics.request_duration_ms.length;

    return {
      healthy: failureRate < 0.1 && avgDuration < 5000,
      failure_rate: failureRate,
      avg_duration_ms: avgDuration,
      total_requests: this.metrics.requests_total
    };
  }
}

Debug Logging

class DebugClient {
  private debug = process.env.NODE_ENV === "development";

  constructor(private client: Client) {}

  async predict(endpoint: string, data: unknown[]) {
    if (this.debug) {
      console.log(`[DEBUG] Prediction request:`, {
        endpoint,
        data_length: Array.isArray(data) ? data.length : 0,
        timestamp: new Date().toISOString()
      });
    }

    try {
      const result = await this.client.predict(endpoint, data);
      
      if (this.debug) {
        console.log(`[DEBUG] Prediction response:`, {
          endpoint,
          result_length: Array.isArray(result.data) ? result.data.length : 0,
          duration: result.duration,
          success: true
        });
      }

      return result;
    } catch (error) {
      if (this.debug) {
        console.error(`[DEBUG] Prediction error:`, {
          endpoint,
          error: error.message,
          stack: error.stack
        });
      }
      throw error;
    }
  }

  submit(endpoint: string, data: unknown[]) {
    const job = this.client.submit(endpoint, data);
    
    if (this.debug) {
      console.log(`[DEBUG] Stream started:`, { endpoint });
      
      // Wrap the async iterator to add debug logging
      return this.debugStream(job, endpoint);
    }
    
    return job;
  }

  private async* debugStream(job: any, endpoint: string) {
    let messageCount = 0;
    
    try {
      for await (const message of job) {
        messageCount++;
        
        if (this.debug) {
          console.log(`[DEBUG] Stream message ${messageCount}:`, {
            endpoint,
            type: message.type,
            has_data: !!message.data
          });
        }
        
        yield message;
      }
    } finally {
      if (this.debug) {
        console.log(`[DEBUG] Stream ended:`, { 
          endpoint, 
          total_messages: messageCount 
        });
      }
    }
  }
}

Version

Tile

Files

advanced.mddocs/

Advanced Topics

Capabilities

Protocol Management

Connection Lifecycle Management

Environment-Specific Configuration

Browser Environment

Node.js Environment

Performance Optimization

Connection Pooling

Batch Processing

Authentication Strategies

Token Refresh

Multi-Provider Authentication

Error Recovery Strategies

Circuit Breaker Pattern

Retry with Exponential Backoff

Monitoring and Observability

Metrics Collection

Debug Logging

See Also

Version

Tile

Files

advanced.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Advanced Topics

Capabilities

Protocol Management

Connection Lifecycle Management

Environment-Specific Configuration

Browser Environment

Node.js Environment

Performance Optimization

Connection Pooling

Batch Processing

Authentication Strategies

Token Refresh

Multi-Provider Authentication

Error Recovery Strategies

Circuit Breaker Pattern

Retry with Exponential Backoff

Monitoring and Observability

Metrics Collection

Debug Logging

See Also

advanced.mddocs/