CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-arweave

JavaScript/TypeScript client library for the Arweave decentralized permanent data storage network

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

data-upload-chunking.mddocs/

Data Upload and Chunking

Advanced data upload capabilities with chunked uploading for large files, progress monitoring, and resumable uploads. Handles large data sets efficiently through Arweave's chunked data protocol.

Capabilities

Get Transaction Uploader

Create a resumable uploader for large transactions with progress tracking.

/**
 * Get a resumable uploader for large transactions
 * @param upload - Transaction, serialized uploader, or transaction ID
 * @param data - Optional data buffer for new uploads
 * @returns Promise resolving to TransactionUploader instance
 */
getUploader(
  upload: Transaction | SerializedUploader | string,
  data?: Uint8Array | ArrayBuffer
): Promise<TransactionUploader>;

Usage Example:

import Arweave from "arweave";

const arweave = Arweave.init();
const key = await arweave.wallets.generate();

// Large data upload
const largeData = new Uint8Array(1024 * 1024); // 1MB of data
const transaction = await arweave.createTransaction({ data: largeData }, key);
await arweave.transactions.sign(transaction, key);

const uploader = await arweave.transactions.getUploader(transaction);

// Upload with progress monitoring
while (!uploader.isComplete) {
  await uploader.uploadChunk();
  console.log(`Progress: ${uploader.pctComplete}%`);
}

Upload with Progress Generator

Upload data using an async generator that yields progress updates.

/**
 * Upload data with async generator for progress tracking
 * @param upload - Transaction, serialized uploader, or transaction ID  
 * @param data - Data to upload
 * @returns Async generator yielding TransactionUploader instances
 */
upload(
  upload: Transaction | SerializedUploader | string, 
  data: Uint8Array
): AsyncGenerator<TransactionUploader>;

Usage Example:

const largeData = new Uint8Array(5 * 1024 * 1024); // 5MB
const transaction = await arweave.createTransaction({ data: largeData }, key);
await arweave.transactions.sign(transaction, key);

// Upload with generator
for await (const uploader of arweave.transactions.upload(transaction, largeData)) {
  console.log(`Uploaded ${uploader.uploadedChunks}/${uploader.totalChunks} chunks`);
  console.log(`Progress: ${uploader.pctComplete}%`);
}

Get Transaction Offset

Get offset information for a transaction in the Arweave weave.

/**
 * Get transaction offset information
 * @param id - Transaction ID
 * @returns Promise resolving to offset response
 */
getTransactionOffset(id: string): Promise<TransactionOffsetResponse>;

Usage Example:

const offsetInfo = await arweave.chunks.getTransactionOffset("transaction-id");
console.log(`Transaction size: ${offsetInfo.size}`);
console.log(`Transaction offset: ${offsetInfo.offset}`);

Get Chunk

Retrieve a specific chunk from the Arweave weave by offset.

/**
 * Get chunk at specific offset
 * @param offset - Chunk offset as string, number, or BigInt
 * @returns Promise resolving to chunk response
 */
getChunk(offset: string | number | BigInt): Promise<TransactionChunkResponse>;

Usage Example:

const chunk = await arweave.chunks.getChunk(12345);
console.log("Chunk data:", chunk.chunk);
console.log("Data path:", chunk.data_path);

Get Chunk Data

Get the raw data from a chunk as a Uint8Array.

/**
 * Get chunk data as Uint8Array
 * @param offset - Chunk offset
 * @returns Promise resolving to chunk data
 */
getChunkData(offset: string | number | BigInt): Promise<Uint8Array>;

Usage Example:

const chunkData = await arweave.chunks.getChunkData(12345);
console.log("Raw chunk data:", chunkData);

Calculate First Chunk Offset

Calculate the first chunk offset from transaction offset response.

/**
 * Calculate first chunk offset
 * @param offsetResponse - Transaction offset response
 * @returns First chunk offset as number
 */
firstChunkOffset(offsetResponse: TransactionOffsetResponse): number;

Download Chunked Data

Download all chunks for a transaction and combine them.

/**
 * Download all chunks for a transaction
 * @param id - Transaction ID
 * @returns Promise resolving to complete data
 */
downloadChunkedData(id: string): Promise<Uint8Array>;

Usage Example:

// Download complete data for a large transaction
const fullData = await arweave.chunks.downloadChunkedData("transaction-id");
console.log("Downloaded data size:", fullData.length);

// Convert to string if it's text data
const textData = new TextDecoder().decode(fullData);
console.log("Text content:", textData);

TransactionUploader Class

Properties

class TransactionUploader {
  /** Transaction data being uploaded */
  data: Uint8Array;
  /** Last HTTP response status code */
  lastResponseStatus: number;
  /** Last error message if any */
  lastResponseError: string;
  /** Whether upload is complete */
  readonly isComplete: boolean;
  /** Total number of chunks */
  readonly totalChunks: number;
  /** Number of uploaded chunks */
  readonly uploadedChunks: number;
  /** Upload percentage complete (0-100) */
  readonly pctComplete: number;
}

Methods

/**
 * Upload the next chunk or a specific chunk
 * @param chunkIndex - Optional specific chunk index to upload
 */
uploadChunk(chunkIndex?: number): Promise<void>;

/**
 * Serialize uploader state for resuming later
 * @returns Serialized uploader state
 */
toJSON(): SerializedUploader;

/**
 * Restore uploader from serialized state
 * @param api - API instance
 * @param serialized - Serialized uploader state
 * @param data - Transaction data
 * @returns Promise resolving to restored uploader
 */
static fromSerialized(
  api: Api, 
  serialized: SerializedUploader, 
  data: Uint8Array
): Promise<TransactionUploader>;

/**
 * Create uploader from transaction ID
 * @param api - API instance
 * @param id - Transaction ID
 * @returns Promise resolving to serialized uploader
 */
static fromTransactionId(api: Api, id: string): Promise<SerializedUploader>;

Usage Example:

// Resume an upload
const serializedState = localStorage.getItem("upload-state");
if (serializedState) {
  const parsed = JSON.parse(serializedState);
  const uploader = await TransactionUploader.fromSerialized(
    arweave.api,
    parsed,
    originalData
  );
  
  // Continue upload
  while (!uploader.isComplete) {
    await uploader.uploadChunk();
    // Save state for resuming
    localStorage.setItem("upload-state", JSON.stringify(uploader.toJSON()));
  }
}

Types

TransactionOffsetResponse

interface TransactionOffsetResponse {
  /** Transaction data size in bytes */
  size: string;
  /** Transaction offset in the weave */
  offset: string;
}

TransactionChunkResponse

interface TransactionChunkResponse {
  /** Base64url encoded chunk data */
  chunk: string;
  /** Merkle path proof for the chunk */
  data_path: string;
  /** Transaction path proof */
  tx_path: string;
}

SerializedUploader

interface SerializedUploader {
  /** Current chunk index being processed */
  chunkIndex: number;
  /** Whether transaction has been posted to network */
  txPosted: boolean;
  /** Transaction data object */
  transaction: object;
  /** Timestamp of last request completion */
  lastRequestTimeEnd: number;
  /** Last HTTP response status */
  lastResponseStatus: number;
  /** Last error message */
  lastResponseError: string;
}

Api

interface Api {
  /** API configuration */
  config: ApiConfig;
  /** HTTP GET method constant */
  METHOD_GET: string;
  /** HTTP POST method constant */
  METHOD_POST: string;
  /** Make HTTP GET request */
  get(endpoint: string, config?: object): Promise<any>;
  /** Make HTTP POST request */
  post(endpoint: string, body?: any, config?: object): Promise<any>;
}

Constants

Chunk Size Limits

/** Maximum chunk size in bytes (256 KB) */
const MAX_CHUNK_SIZE: number = 262144;

/** Minimum chunk size in bytes (32 KB) */
const MIN_CHUNK_SIZE: number = 32768;

docs

cryptographic-operations.md

currency-utilities.md

data-upload-chunking.md

data-utilities.md

encrypted-storage-silo.md

index.md

network-blockchain.md

transaction-management.md

wallet-operations.md

tile.json