CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-node-fetch

A light-weight module that brings Fetch API to Node.js

—

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

body-processing.mddocs/

Body Processing

Unified interface for reading and processing request/response bodies with support for multiple formats including JSON, text, binary data, form data, and streams.

Capabilities

Body Interface

Shared interface implemented by both Request and Response classes for consistent body handling.

/**
 * Body interface for reading request/response content
 */
interface Body {
  readonly body: ReadableStream | null;
  readonly bodyUsed: boolean;
  readonly size: number;
  
  arrayBuffer(): Promise<ArrayBuffer>;
  blob(): Promise<Blob>;
  formData(): Promise<FormData>;
  json(): Promise<any>;
  text(): Promise<string>;
  
  /** @deprecated Use arrayBuffer() instead */
  buffer(): Promise<Buffer>;
}

Usage Examples:

import fetch from 'node-fetch';

const response = await fetch('https://api.github.com/users/octocat');

// Check if body has been consumed
console.log(response.bodyUsed); // false

// Read as JSON
const userData = await response.json();
console.log(response.bodyUsed); // true

// Cannot read again after consumption
try {
  await response.text(); // This will throw an error
} catch (error) {
  console.error('Body already used:', error.message);
}

JSON Processing

Parse response bodies as JSON with automatic error handling for malformed JSON.

/**
 * Parse body as JSON
 * @returns Promise resolving to parsed JSON data
 * @throws SyntaxError if JSON is malformed
 */
json(): Promise<any>;

Usage Examples:

// API response as JSON
const response = await fetch('https://api.github.com/users/octocat');
const userData = await response.json();

console.log(userData.login);    // 'octocat'
console.log(userData.id);       // 583231
console.log(userData.name);     // 'The Octocat'

// Handle JSON parsing errors
try {
  const response = await fetch('https://example.com/malformed-json');
  const data = await response.json();
} catch (error) {
  if (error instanceof SyntaxError) {
    console.error('Invalid JSON response:', error.message);
  }
}

// POST request with JSON response
const postResponse = await fetch('https://httpbin.org/post', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ name: 'Alice', age: 30 })
});
const result = await postResponse.json();
console.log(result.json); // { name: 'Alice', age: 30 }

Text Processing

Read response bodies as plain text with automatic UTF-8 decoding.

/**
 * Read body as text string
 * @returns Promise resolving to text content
 */
text(): Promise<string>;

Usage Examples:

// HTML content
const response = await fetch('https://example.com');
const htmlContent = await response.text();
console.log(htmlContent); // '<!DOCTYPE html><html>...'

// Plain text API
const textResponse = await fetch('https://httpbin.org/uuid');
const uuid = await textResponse.text();
console.log(uuid.trim()); // 'f47ac10b-58cc-4372-a567-0e02b2c3d479'

// CSV data
const csvResponse = await fetch('https://example.com/data.csv');
const csvContent = await csvResponse.text();
const rows = csvContent.split('\n');
console.log('Number of rows:', rows.length);

// Large text processing
const largeTextResponse = await fetch('https://example.com/large-file.txt');
const text = await largeTextResponse.text();
console.log('File size:', text.length, 'characters');

Binary Data Processing

Handle binary data as ArrayBuffer for images, files, and other binary content.

/**
 * Read body as ArrayBuffer
 * @returns Promise resolving to ArrayBuffer
 */
arrayBuffer(): Promise<ArrayBuffer>;

Usage Examples:

// Download image as binary data
const imageResponse = await fetch('https://github.com/images/error/octocat_happy.gif');
const imageBuffer = await imageResponse.arrayBuffer();

// Save to file (Node.js)
import { writeFileSync } from 'fs';
writeFileSync('octocat.gif', Buffer.from(imageBuffer));

// Process binary data
const uint8Array = new Uint8Array(imageBuffer);
console.log('File size:', uint8Array.length, 'bytes');
console.log('First 4 bytes:', Array.from(uint8Array.slice(0, 4)).map(b => b.toString(16)));

// Download ZIP file
const zipResponse = await fetch('https://example.com/archive.zip');
const zipBuffer = await zipResponse.arrayBuffer();
console.log('ZIP file size:', zipBuffer.byteLength, 'bytes');

// Handle different binary formats
const pdfResponse = await fetch('https://example.com/document.pdf');
const pdfBuffer = await pdfResponse.arrayBuffer();
// Process PDF data...

Buffer Processing (Deprecated)

⚠️ DEPRECATED: The buffer() method is deprecated in favor of arrayBuffer(). Use arrayBuffer() for new code.

/**
 * Read body as Buffer (Node.js specific)
 * @deprecated Use response.arrayBuffer() instead
 * @returns Promise resolving to Buffer
 */
buffer(): Promise<Buffer>;

Migration Example:

// OLD - Deprecated approach
const response = await fetch('https://example.com/file.pdf');
const buffer = await response.buffer(); // ⚠️ Deprecated

// NEW - Recommended approach  
const response = await fetch('https://example.com/file.pdf');
const arrayBuffer = await response.arrayBuffer();
const buffer = Buffer.from(arrayBuffer);

// Or for Node.js compatibility
const response = await fetch('https://example.com/file.pdf');
const buffer = Buffer.from(await response.arrayBuffer());

Why migrate from buffer() to arrayBuffer()?

  • Web Standards Compliance: arrayBuffer() is part of the standard Fetch API
  • Cross-platform Compatibility: Works in both Node.js and browsers
  • Future-proof: The buffer() method will be removed in future versions
  • Better Type Safety: ArrayBuffer provides clearer type semantics

Blob Processing

Convert response bodies to Blob objects for file-like operations.

/**
 * Read body as Blob
 * @returns Promise resolving to Blob object
 */
blob(): Promise<Blob>;

Usage Examples:

// Create blob from response
const imageResponse = await fetch('https://example.com/image.jpg');
const imageBlob = await imageResponse.blob();

console.log('Blob size:', imageBlob.size);
console.log('Blob type:', imageBlob.type); // 'image/jpeg'

// Convert blob to other formats
const arrayBuffer = await imageBlob.arrayBuffer();
const text = await imageBlob.text();

// Create object URL (browser environment)
if (typeof URL !== 'undefined') {
  const objectUrl = URL.createObjectURL(imageBlob);
  console.log('Object URL:', objectUrl);
  
  // Don't forget to revoke when done
  URL.revokeObjectURL(objectUrl);
}

// Stream blob data
const stream = imageBlob.stream();
const reader = stream.getReader();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  console.log('Chunk size:', value.length);
}

Form Data Processing

Parse multipart/form-data and application/x-www-form-urlencoded content.

/**
 * Parse body as FormData
 * @returns Promise resolving to FormData object
 * @throws Error if content is not form data
 */
formData(): Promise<FormData>;

Usage Examples:

// Parse multipart form data
const formResponse = await fetch('https://httpbin.org/post', {
  method: 'POST',
  body: new FormData() // This would be received as form data
});

const formData = await formResponse.formData();

// Iterate over form fields
for (const [name, value] of formData.entries()) {
  console.log(`${name}: ${value}`);
}

// Get specific field
const username = formData.get('username');
const avatar = formData.get('avatar'); // Could be a File object

// Parse URL-encoded form data
const urlEncodedResponse = await fetch('https://httpbin.org/post', {
  method: 'POST',
  headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
  body: 'name=Alice&email=alice@example.com'
});

const parsedForm = await urlEncodedResponse.formData();
console.log(parsedForm.get('name'));  // 'Alice'
console.log(parsedForm.get('email')); // 'alice@example.com'

Stream Processing

Access the underlying ReadableStream for streaming data processing.

/**
 * Access to the underlying ReadableStream
 */
interface StreamAccess {
  readonly body: ReadableStream | null;
}

Usage Examples:

import { pipeline } from 'stream/promises';
import { createWriteStream } from 'fs';

// Stream large file download
const response = await fetch('https://example.com/large-file.zip');

if (!response.ok) {
  throw new Error(`HTTP error! status: ${response.status}`);
}

// Pipe response stream to file
await pipeline(response.body, createWriteStream('./large-file.zip'));
console.log('Download completed');

// Process streaming JSON data
const streamResponse = await fetch('https://httpbin.org/stream/10');

// Read stream chunk by chunk
const reader = streamResponse.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  
  if (done) break;
  
  const chunk = decoder.decode(value, { stream: true });
  console.log('Received chunk:', chunk);
}

// Stream with async iteration (Node.js 10+)
const asyncStreamResponse = await fetch('https://httpbin.org/stream/5');

try {
  for await (const chunk of asyncStreamResponse.body) {
    const text = chunk.toString();
    console.log('Processed chunk:', text);
  }
} catch (error) {
  console.error('Stream error:', error);
}

Body Size and Limits

Control and monitor body size limits for security and performance.

interface BodyLimits {
  readonly size: number;  // Size limit in bytes (0 = unlimited)
}

Usage Examples:

// Set response size limit
const response = await fetch('https://example.com/large-file', {
  size: 1024 * 1024 // Limit to 1MB
});

try {
  const data = await response.arrayBuffer();
  console.log('Downloaded:', data.byteLength, 'bytes');
} catch (error) {
  if (error.type === 'max-size') {
    console.error('Response too large:', error.message);
  }
}

// Check body size before processing
if (response.headers.has('content-length')) {
  const contentLength = parseInt(response.headers.get('content-length'));
  if (contentLength > 10 * 1024 * 1024) { // 10MB
    console.warn('Large response detected:', contentLength, 'bytes');
  }
}

Body Usage Patterns

Common patterns for handling different types of response bodies.

interface BodyUsagePattern {
  // Pattern for API responses
  handleApiResponse<T>(): Promise<T>;
  
  // Pattern for file downloads
  handleFileDownload(): Promise<Buffer>;
  
  // Pattern for streaming data
  handleStreamingData(): AsyncIterableIterator<Uint8Array>;
}

Usage Examples:

// API response handler with error checking
async function handleApiResponse(url, options = {}) {
  const response = await fetch(url, options);
  
  if (!response.ok) {
    const errorText = await response.text();
    throw new Error(`API Error ${response.status}: ${errorText}`);
  }
  
  const contentType = response.headers.get('content-type');
  
  if (contentType?.includes('application/json')) {
    return await response.json();
  } else if (contentType?.includes('text/')) {
    return await response.text();
  } else {
    return await response.arrayBuffer();
  }
}

// File download with progress tracking
async function downloadFile(url, onProgress) {
  const response = await fetch(url);
  const contentLength = response.headers.get('content-length');
  const total = contentLength ? parseInt(contentLength) : 0;
  
  let loaded = 0;
  const chunks = [];
  
  const reader = response.body.getReader();
  
  while (true) {
    const { done, value } = await reader.read();
    
    if (done) break;
    
    chunks.push(value);
    loaded += value.length;
    
    if (onProgress && total > 0) {
      onProgress({ loaded, total, percentage: (loaded / total) * 100 });
    }
  }
  
  return Buffer.concat(chunks);
}

// Usage
const fileData = await downloadFile('https://example.com/file.zip', (progress) => {
  console.log(`Download progress: ${progress.percentage.toFixed(1)}%`);
});

docs

body-processing.md

error-handling.md

file-blob.md

headers.md

http-client.md

index.md

request-response.md

utilities.md

tile.json