CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/npm-uint8arrays

Utility functions to make dealing with Uint8Arrays easier

Pending
Overview
Eval results
Files

string-encoding.mddocs/

String Encoding

Comprehensive string encoding and decoding functions supporting UTF-8, hex, base64, and all multibase formats. Provides seamless conversion between strings and Uint8Arrays with extensive encoding support.

Capabilities

String to Uint8Array Conversion

Converts strings to Uint8Arrays using specified encoding. Supports UTF-8 and all multibase encoding formats.

/**
 * Create a Uint8Array from the passed string
 * Supports utf8, utf-8, hex, and any encoding supported by the multiformats module.
 * Also ascii which is similar to node's 'binary' encoding.
 * @param string - String to convert
 * @param encoding - Encoding format (default: 'utf8')
 * @returns Uint8Array representation of the string
 * @throws Error if encoding is unsupported
 */
function fromString(string: string, encoding?: SupportedEncodings): Uint8Array;

Usage Examples:

import { fromString } from "uint8arrays/from-string";

// Default UTF-8 encoding
const utf8Bytes = fromString("Hello World");
console.log(utf8Bytes); // Uint8Array(11) [72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100]

// Hex encoding
const hexBytes = fromString("48656c6c6f", "hex");
console.log(hexBytes); // Uint8Array(5) [72, 101, 108, 108, 111] ("Hello")

// Base64 encoding
const base64Bytes = fromString("SGVsbG8gV29ybGQ=", "base64");
console.log(base64Bytes); // "Hello World" as bytes

// Base16 (hex with multibase)
const base16Bytes = fromString("48656c6c6f", "base16");

// ASCII encoding (binary-like)
const asciiBytes = fromString("ABC", "ascii");
console.log(asciiBytes); // Uint8Array(3) [65, 66, 67]

// Unicode strings
const unicodeBytes = fromString("🚀 Hello", "utf8");
console.log(unicodeBytes); // Properly encoded UTF-8 bytes

// Base58 encoding
const base58Bytes = fromString("StV1DL6CwTryKyV", "base58btc");

Uint8Array to String Conversion

Converts Uint8Arrays to strings using specified encoding. Supports the same comprehensive set of encodings as fromString.

/**
 * Turns a Uint8Array into a string.
 * Supports utf8, utf-8 and any encoding supported by the multibase module.
 * Also ascii which is similar to node's 'binary' encoding.
 * @param array - Uint8Array to convert
 * @param encoding - Encoding format (default: 'utf8')
 * @returns String representation of the array
 * @throws Error if encoding is unsupported
 */
function toString(array: Uint8Array, encoding?: SupportedEncodings): string;

Usage Examples:

import { toString } from "uint8arrays/to-string";

// UTF-8 decoding (default)
const bytes = new Uint8Array([72, 101, 108, 108, 111]);
const text = toString(bytes);
console.log(text); // "Hello"

// Hex encoding
const hexString = toString(bytes, "hex");
console.log(hexString); // "48656c6c6f"

// Base64 encoding  
const base64String = toString(bytes, "base64");
console.log(base64String); // "SGVsbG8="

// ASCII decoding
const asciiBytes = new Uint8Array([65, 66, 67]);
const asciiText = toString(asciiBytes, "ascii");
console.log(asciiText); // "ABC"

// Unicode handling
const unicodeBytes = new Uint8Array([240, 159, 154, 128, 32, 72, 101, 108, 108, 111]);
const unicodeText = toString(unicodeBytes, "utf8");
console.log(unicodeText); // "🚀 Hello"

// Different base encodings
const base16String = toString(bytes, "base16");
const base32String = toString(bytes, "base32");
const base58String = toString(bytes, "base58btc");

Supported Encodings

The SupportedEncodings type includes all available encoding formats:

type SupportedEncodings = 
  | 'utf8'           // UTF-8 text encoding (default)
  | 'utf-8'          // Alias for utf8
  | 'hex'            // Hexadecimal encoding
  | 'latin1'         // Latin-1 encoding
  | 'ascii'          // ASCII encoding (binary-like)
  | 'binary'         // Alias for ascii
  | 'base2'          // Binary base encoding
  | 'base8'          // Octal base encoding  
  | 'base10'         // Decimal base encoding
  | 'base16'         // Hexadecimal (lowercase)
  | 'base16upper'    // Hexadecimal (uppercase)
  | 'base32'         // Base32 encoding
  | 'base32upper'    // Base32 (uppercase)
  | 'base32pad'      // Base32 with padding
  | 'base32padupper' // Base32 with padding (uppercase)
  | 'base32hex'      // Base32 hex alphabet
  | 'base32hexupper' // Base32 hex alphabet (uppercase)
  | 'base32hexpad'   // Base32 hex with padding
  | 'base32hexpadupper' // Base32 hex with padding (uppercase)
  | 'base32z'        // Base32 z-base encoding
  | 'base36'         // Base36 encoding
  | 'base36upper'    // Base36 (uppercase)
  | 'base58btc'      // Base58 Bitcoin alphabet
  | 'base58flickr'   // Base58 Flickr alphabet
  | 'base64'         // Base64 encoding
  | 'base64pad'      // Base64 with padding
  | 'base64url'      // Base64 URL-safe encoding
  | 'base64urlpad';  // Base64 URL-safe with padding

Common Use Cases

Data Serialization

import { fromString, toString } from "uint8arrays";

// JSON data to bytes and back
const jsonData = JSON.stringify({ name: "Alice", age: 30 });
const jsonBytes = fromString(jsonData, "utf8");
const recovered = toString(jsonBytes, "utf8");
const parsed = JSON.parse(recovered);

Cryptographic Applications

// Hex-encoded hash digests
const hashHex = "a665a45920422f9d417e4867efdc4fb8a04a1f3fff1fa07e998e86f7f7a27ae3";
const hashBytes = fromString(hashHex, "hex");

// Base64-encoded keys
const keyBase64 = "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA...";
const keyBytes = fromString(keyBase64, "base64");

Network Protocols

// URL-safe base64 for tokens
const token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9";
const tokenBytes = fromString(token, "base64url");

// Multibase for content addressing  
const cid = "QmYjtig7VJQ6XsnUjqqJvj7QaMcCAwtrgNdahSiFofrE7o";
const cidBytes = fromString(cid, "base58btc");

Platform Optimizations

  • Node.js: Uses optimized Buffer encoding methods when available
  • Browser: Uses TextEncoder/TextDecoder for UTF-8, falls back to polyfills
  • Multiformats: Leverages multiformats library for advanced base encodings
  • Performance: Automatic selection of fastest encoding method per platform

Error Handling

Both functions provide clear error handling:

  • Unsupported Encoding: Throws Error with descriptive message
  • Invalid Input: Handles malformed strings gracefully where possible
  • Unicode: Proper handling of invalid UTF-8 sequences
  • Empty Input: Handles empty strings and arrays correctly
// Error examples
try {
  fromString("hello", "unsupported" as any);
} catch (error) {
  console.log(error.message); // 'Unsupported encoding "unsupported"'
}

Install with Tessl CLI

npx tessl i tessl/npm-uint8arrays

docs

array-operations.md

index.md

memory-allocation.md

string-encoding.md

xor-operations.md

tile.json