CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-make-fetch-happen

Opinionated, caching, retrying fetch client for Node.js with enterprise-grade HTTP features

74

1.68x
Overview
Eval results
Files

network.mddocs/

Proxy and Network Configuration

Comprehensive proxy support with automatic proxy detection, connection pooling, and network configuration options through @npmcli/agent.

Capabilities

Proxy Configuration

Configure HTTP, HTTPS, and SOCKS proxies with automatic environment variable detection.

/**
 * Proxy configuration options
 */
interface ProxyOptions {
  proxy?: string | URL;        // Proxy URL (http://, https://, socks://)
  noProxy?: string | string[]; // Domains to bypass proxy for
}

/**
 * Supported proxy protocols:
 * - http://proxy.example.com:8080
 * - https://proxy.example.com:8443  
 * - socks://proxy.example.com:1080
 * - socks4://proxy.example.com:1080
 * - socks5://proxy.example.com:1080
 */

Usage Examples:

const fetch = require('make-fetch-happen');

// HTTP proxy
const response1 = await fetch('https://api.example.com/data', {
  proxy: 'http://proxy.company.com:8080'
});

// HTTPS proxy with authentication
const response2 = await fetch('https://api.example.com/data', {
  proxy: 'https://username:password@proxy.company.com:8443'
});

// SOCKS5 proxy
const response3 = await fetch('https://api.example.com/data', {
  proxy: 'socks5://proxy.company.com:1080'
});

// Proxy with URL object
const proxyUrl = new URL('http://proxy.company.com:8080');
const response4 = await fetch('https://api.example.com/data', {
  proxy: proxyUrl
});

Environment Variable Support

Automatic proxy detection from standard environment variables:

/**
 * Environment variables (checked in order of precedence):
 * - HTTP_PROXY: Proxy for HTTP requests
 * - HTTPS_PROXY: Proxy for HTTPS requests  
 * - PROXY: Fallback proxy for all requests
 * - NO_PROXY: Comma-separated list of domains to bypass proxy
 * 
 * Variables are case-insensitive and support both upper and lowercase
 */

Usage Examples:

# Set environment variables
export HTTP_PROXY=http://proxy.company.com:8080
export HTTPS_PROXY=https://proxy.company.com:8443
export NO_PROXY=localhost,127.0.0.1,.company.com

# Or in package.json scripts
{
  "scripts": {
    "start": "HTTP_PROXY=http://proxy:8080 node app.js"
  }
}
// Proxy will be automatically detected from environment
const response = await fetch('https://api.example.com/data');

// Override environment with explicit proxy
const response2 = await fetch('https://api.example.com/data', {
  proxy: 'http://different-proxy:8080'
});

// Disable proxy for this request
const response3 = await fetch('https://api.example.com/data', {
  proxy: null
});

No Proxy Configuration

Bypass proxy for specific domains or IP addresses:

/**
 * No proxy patterns support:
 * - Exact domain matches: example.com
 * - Subdomain matches: .example.com (matches any.sub.example.com)
 * - IP addresses: 192.168.1.1
 * - IP ranges: 192.168.1.0/24
 * - Port specifications: example.com:8080
 * - Wildcards: *.example.com
 */

Usage Examples:

// Bypass proxy for specific domains
const response = await fetch('https://internal-api.company.com/data', {
  proxy: 'http://proxy.company.com:8080',
  noProxy: [
    'localhost',
    '127.0.0.1', 
    '.company.com',
    '192.168.1.0/24'
  ]
});

// Single domain as string
const response2 = await fetch('https://localhost:3000/api', {
  proxy: 'http://proxy.company.com:8080',
  noProxy: 'localhost'
});

// Environment variable format (comma-separated)
const response3 = await fetch('https://api.example.com/data', {
  proxy: 'http://proxy.company.com:8080',
  noProxy: 'localhost,127.0.0.1,.company.com,*.internal.com'
});

Connection Pooling

Configure connection pooling and socket management:

/**
 * Connection pooling options
 */
interface ConnectionOptions {
  maxSockets?: number;     // Maximum concurrent connections per host (default: 15)
  localAddress?: string;   // Local address to bind connections to
  agent?: object;         // Custom HTTP agent configuration
}

Usage Examples:

// Limit concurrent connections
const response = await fetch('https://api.example.com/data', {
  maxSockets: 10
});

// Bind to specific local interface
const response2 = await fetch('https://api.example.com/data', {
  localAddress: '192.168.1.100'
});

// Global connection pool configuration
const apiFetch = fetch.defaults({
  maxSockets: 25,
  proxy: 'http://proxy.company.com:8080'
});

Advanced Agent Configuration

Configure the underlying HTTP agent with detailed options:

/**
 * Agent configuration passed to @npmcli/agent
 */
interface AgentOptions {
  agent?: {
    http?: object;          // HTTP agent options
    https?: object;         // HTTPS agent options  
    proxy?: object;         // Proxy agent options
    dns?: DNSOptions;       // DNS configuration
    timeout?: {             // Timeout configuration
      connection?: number;  // Connection timeout
      idle?: number;        // Idle timeout
      response?: number;    // Response timeout
      transfer?: number;    // Transfer timeout
    };
  };
}

interface DNSOptions {
  ttl?: number;            // DNS cache TTL in milliseconds (default: 300000)
  lookup?: Function;       // Custom DNS lookup function
}

Usage Examples:

// Custom agent configuration
const response = await fetch('https://api.example.com/data', {
  agent: {
    timeout: {
      connection: 10000,  // 10 second connection timeout
      response: 30000,    // 30 second response timeout
      idle: 60000,        // 60 second idle timeout
      transfer: 120000    // 2 minute transfer timeout
    },
    dns: {
      ttl: 60000,         // 1 minute DNS cache
      lookup: require('dns').lookup
    }
  }
});

// Custom HTTP agent options
const response2 = await fetch('https://api.example.com/data', {
  agent: {
    http: {
      keepAlive: true,
      keepAliveMsecs: 30000,
      maxSockets: 20,
      maxFreeSockets: 10
    },
    https: {
      keepAlive: true,
      rejectUnauthorized: true
    }
  }
});

DNS Configuration

Configure DNS caching and lookup behavior:

/**
 * DNS configuration options
 */
interface DNSOptions {
  ttl?: number;            // DNS cache TTL in milliseconds
  lookup?: Function;       // Custom DNS lookup function
}

Usage Examples:

// Custom DNS configuration
const response = await fetch('https://api.example.com/data', {
  dns: {
    ttl: 300000,  // 5 minute DNS cache
    lookup: (hostname, options, callback) => {
      // Custom DNS resolution logic
      console.log(`Resolving ${hostname}`);
      require('dns').lookup(hostname, options, callback);
    }
  }
});

// Disable DNS caching
const response2 = await fetch('https://api.example.com/data', {
  dns: {
    ttl: 0  // No DNS caching
  }
});

Network Error Handling

Handle network-specific errors and timeouts:

/**
 * Network error codes from @npmcli/agent:
 * - ECONNECTIONTIMEOUT: Connection timeout
 * - EIDLETIMEOUT: Idle timeout
 * - ERESPONSETIMEOUT: Response timeout  
 * - ETRANSFERTIMEOUT: Transfer timeout
 * - EINVALIDPROXY: Invalid proxy configuration
 * - EINVALIDRESPONSE: Invalid response from server
 */

Usage Examples:

try {
  const response = await fetch('https://slow-api.example.com/data', {
    timeout: 30000,
    agent: {
      timeout: {
        connection: 5000,
        response: 10000
      }
    }
  });
} catch (error) {
  switch (error.code) {
    case 'ECONNECTIONTIMEOUT':
      console.log('Failed to establish connection within timeout');
      break;
    case 'ERESPONSETIMEOUT':
      console.log('Server took too long to respond');
      break;
    case 'ETRANSFERTIMEOUT':
      console.log('Data transfer took too long');
      break;
    case 'EINVALIDPROXY':
      console.log('Proxy configuration is invalid');
      break;
    default:
      console.log('Network error:', error.message);
  }
}

Corporate Network Patterns

Common patterns for corporate environments:

// Corporate proxy configuration
const createCorporateFetch = () => {
  return fetch.defaults({
    proxy: process.env.CORPORATE_PROXY || 'http://proxy.company.com:8080',
    noProxy: [
      'localhost',
      '127.0.0.1',
      '.company.com',
      '.internal',
      '10.0.0.0/8',
      '172.16.0.0/12',
      '192.168.0.0/16'
    ],
    maxSockets: 10,
    timeout: 30000
  });
};

// Development vs production proxy
const createEnvironmentFetch = (env) => {
  const config = {
    maxSockets: env === 'production' ? 25 : 10
  };
  
  if (env === 'development') {
    config.proxy = 'http://localhost:8888'; // Development proxy
  } else if (env === 'staging') {
    config.proxy = 'http://staging-proxy.company.com:8080';
  } else if (env === 'production') {
    config.proxy = 'http://prod-proxy.company.com:8080';
  }
  
  return fetch.defaults(config);
};

// Proxy authentication with token refresh
const createAuthenticatedProxyFetch = () => {
  let proxyAuth = null;
  
  const refreshProxyAuth = async () => {
    // Refresh proxy authentication token
    const authResponse = await fetch('https://auth.company.com/proxy-token');
    const { token } = await authResponse.json();
    proxyAuth = `Bearer ${token}`;
  };
  
  return async (url, options = {}) => {
    if (!proxyAuth) {
      await refreshProxyAuth();
    }
    
    return fetch(url, {
      ...options,
      proxy: `http://${proxyAuth}@proxy.company.com:8080`
    });
  };
};

Install with Tessl CLI

npx tessl i tessl/npm-make-fetch-happen

docs

caching.md

configuration.md

core-fetch.md

index.md

network.md

retry.md

security.md

tile.json