tessl/maven-co-cask-cdap--cdap-api-common

Core API classes and utilities for CDAP application development, providing common data schema definitions, data format abstractions, stream event handling, and byte manipulation utilities

—

Pending

Overview

Eval results

Files

Byte Array Utilities

Name: tessl/maven-co-cask-cdap--cdap-api-common
Author: tessl

Comprehensive byte manipulation utilities optimized for big data processing, including conversions between primitive types and byte arrays, array operations, comparisons, and hash computations. The Bytes class provides essential functionality for data serialization, network protocols, and storage systems in distributed computing environments.

Capabilities

Type Conversions

Convert between primitive Java types and their byte array representations using big-endian encoding.

// String conversions
/**
 * Convert string to UTF-8 byte array
 * @param s String to convert
 * @return UTF-8 encoded byte array
 */
public static byte[] toBytes(String s);

/**
 * Convert UTF-8 byte array to string
 * @param b UTF-8 encoded byte array
 * @return String or null if conversion fails
 */
public static String toString(byte[] b);

/**
 * Convert UTF-8 byte array segment to string
 * @param b Byte array
 * @param off Offset in array
 * @param len Length to convert
 * @return String or null if conversion fails
 */
public static String toString(byte[] b, int off, int len);

// Numeric type conversions
/**
 * Convert boolean to byte array (true=-1, false=0)
 * @param b Boolean value
 * @return Byte array representation
 */
public static byte[] toBytes(boolean b);

/**
 * Convert long to 8-byte array (big-endian)
 * @param val Long value
 * @return 8-byte array
 */
public static byte[] toBytes(long val);

/**
 * Convert int to 4-byte array (big-endian)  
 * @param val Int value
 * @return 4-byte array
 */
public static byte[] toBytes(int val);

/**
 * Convert short to 2-byte array (big-endian)
 * @param val Short value
 * @return 2-byte array
 */
public static byte[] toBytes(short val);

/**
 * Convert float to 4-byte array (IEEE 754)
 * @param f Float value
 * @return 4-byte array
 */
public static byte[] toBytes(float f);

/**
 * Convert double to 8-byte array (IEEE 754)
 * @param d Double value
 * @return 8-byte array
 */
public static byte[] toBytes(double d);

// Reverse conversions
public static boolean toBoolean(byte[] b);
public static long toLong(byte[] bytes);
public static int toInt(byte[] bytes);
public static short toShort(byte[] bytes);
public static float toFloat(byte[] bytes);
public static double toDouble(byte[] bytes);

Usage Examples:

// String conversions
String text = "Hello, World!";
byte[] textBytes = Bytes.toBytes(text);
String recovered = Bytes.toString(textBytes); // "Hello, World!"

// Numeric conversions
long timestamp = System.currentTimeMillis();
byte[] timestampBytes = Bytes.toBytes(timestamp);
long recoveredTimestamp = Bytes.toLong(timestampBytes);

int count = 42;
byte[] countBytes = Bytes.toBytes(count);
int recoveredCount = Bytes.toInt(countBytes);

// Boolean conversion
boolean flag = true;
byte[] flagBytes = Bytes.toBytes(flag); // [-1]
boolean recoveredFlag = Bytes.toBoolean(flagBytes); // true

Complex Type Conversions

Convert complex types including BigDecimal, UUID, and ByteBuffer to byte arrays.

/**
 * Convert BigDecimal to byte array
 * @param val BigDecimal value
 * @return Byte array with scale and unscaled value
 */
public static byte[] toBytes(BigDecimal val);

/**
 * Convert byte array to BigDecimal
 * @param bytes Byte array
 * @return BigDecimal value
 */
public static BigDecimal toBigDecimal(byte[] bytes);

/**
 * Convert UUID to 16-byte array
 * @param uuid UUID value
 * @return 16-byte array
 */
public static byte[] toBytes(UUID uuid);

/**
 * Convert 16-byte array to UUID
 * @param bytes 16-byte array
 * @return UUID value
 */
public static UUID toUUID(byte[] bytes);

/**
 * Convert ByteBuffer to byte array
 * @param bb ByteBuffer
 * @return Byte array copy of buffer contents
 */
public static byte[] toBytes(ByteBuffer bb);

Usage Examples:

import java.math.BigDecimal;
import java.util.UUID;
import java.nio.ByteBuffer;

// BigDecimal conversion
BigDecimal price = new BigDecimal("123.45");
byte[] priceBytes = Bytes.toBytes(price);
BigDecimal recoveredPrice = Bytes.toBigDecimal(priceBytes); // 123.45

// UUID conversion
UUID id = UUID.randomUUID();
byte[] idBytes = Bytes.toBytes(id);
UUID recoveredId = Bytes.toUUID(idBytes);

// ByteBuffer conversion
ByteBuffer buffer = ByteBuffer.allocate(10);
buffer.put("test".getBytes());
buffer.flip();
byte[] bufferBytes = Bytes.toBytes(buffer);

Array Operations

Manipulate byte arrays with concatenation, slicing, and padding operations.

/**
 * Concatenate two byte arrays
 * @param a First array
 * @param b Second array
 * @return Combined array
 */
public static byte[] add(byte[] a, byte[] b);

/**
 * Concatenate three byte arrays
 * @param a First array
 * @param b Second array
 * @param c Third array
 * @return Combined array
 */
public static byte[] add(byte[] a, byte[] b, byte[] c);

/**
 * Concatenate multiple byte arrays
 * @param arrays Variable number of arrays
 * @return Combined array
 */
public static byte[] concat(byte[]... arrays);

/**
 * Get first N bytes from array
 * @param a Source array
 * @param length Number of bytes
 * @return First N bytes or null if array too short
 */
public static byte[] head(byte[] a, int length);

/**
 * Get last N bytes from array
 * @param a Source array  
 * @param length Number of bytes
 * @return Last N bytes or null if array too short
 */
public static byte[] tail(byte[] a, int length);

/**
 * Prepend zero bytes to array
 * @param a Source array
 * @param length Number of zeros to prepend
 * @return Padded array
 */
public static byte[] padHead(byte[] a, int length);

/**
 * Append zero bytes to array
 * @param a Source array
 * @param length Number of zeros to append
 * @return Padded array
 */
public static byte[] padTail(byte[] a, int length);

Usage Examples:

byte[] part1 = Bytes.toBytes("Hello");
byte[] part2 = Bytes.toBytes(" ");
byte[] part3 = Bytes.toBytes("World");

// Concatenation
byte[] greeting = Bytes.add(part1, part2, part3);
String result = Bytes.toString(greeting); // "Hello World"

// Multiple array concatenation
byte[] combined = Bytes.concat(part1, part2, part3);

// Array slicing
byte[] data = Bytes.toBytes("abcdefghij");
byte[] firstFive = Bytes.head(data, 5);  // "abcde"
byte[] lastFive = Bytes.tail(data, 5);   // "fghij"

// Padding
byte[] original = Bytes.toBytes("test");
byte[] headPadded = Bytes.padHead(original, 3);  // [0,0,0,t,e,s,t]
byte[] tailPadded = Bytes.padTail(original, 3);  // [t,e,s,t,0,0,0]

Byte Array Writing and Reading

Write values at specific positions in byte arrays and read from ByteBuffers.

/**
 * Write bytes at specified position
 * @param tgtBytes Target array
 * @param tgtOffset Position in target
 * @param srcBytes Source array
 * @param srcOffset Source position
 * @param srcLength Length to copy
 * @return New offset after write
 */
public static int putBytes(byte[] tgtBytes, int tgtOffset, byte[] srcBytes, int srcOffset, int srcLength);

/**
 * Write long at specified position
 * @param bytes Target array
 * @param offset Position to write
 * @param val Long value
 * @return New offset after write
 * @throws IllegalArgumentException if insufficient space
 */
public static int putLong(byte[] bytes, int offset, long val);

/**
 * Write int at specified position
 * @param bytes Target array
 * @param offset Position to write
 * @param val Int value
 * @return New offset after write
 */
public static int putInt(byte[] bytes, int offset, int val);

// Similar methods for putShort, putFloat, putDouble, putByte

/**
 * Extract bytes from ByteBuffer without changing position
 * @param buf ByteBuffer
 * @return Byte array copy
 */
public static byte[] getBytes(ByteBuffer buf);

Usage Examples:

// Create buffer for structured data
byte[] buffer = new byte[16];
int offset = 0;

// Write multiple values sequentially
offset = Bytes.putInt(buffer, offset, 12345);     // Write int at position 0
offset = Bytes.putLong(buffer, offset, 67890L);   // Write long at position 4
offset = Bytes.putFloat(buffer, offset, 3.14f);   // Write float at position 12

// Read values back
int intValue = Bytes.toInt(buffer, 0);      // 12345
long longValue = Bytes.toLong(buffer, 4);   // 67890L
float floatValue = Bytes.toFloat(buffer, 12); // 3.14f

// ByteBuffer extraction
ByteBuffer bb = ByteBuffer.wrap("example data".getBytes());
byte[] extracted = Bytes.getBytes(bb); // Preserves buffer position

Comparison Operations

Compare byte arrays lexicographically with optimized implementations.

/**
 * Lexicographically compare byte arrays
 * @param left Left operand
 * @param right Right operand
 * @return 0 if equal, <0 if left < right, >0 if left > right
 */
public static int compareTo(byte[] left, byte[] right);

/**
 * Compare byte array segments
 * @param buffer1 First array
 * @param offset1 First array start position
 * @param length1 First array length
 * @param buffer2 Second array
 * @param offset2 Second array start position
 * @param length2 Second array length
 * @return Comparison result
 */
public static int compareTo(byte[] buffer1, int offset1, int length1,
                           byte[] buffer2, int offset2, int length2);

/**
 * Check byte array equality
 * @param left Left array
 * @param right Right array
 * @return true if arrays are equal
 */
public static boolean equals(byte[] left, byte[] right);

/**
 * Check if array starts with prefix
 * @param bytes Array to check
 * @param prefix Prefix to match
 * @return true if bytes starts with prefix
 */
public static boolean startsWith(byte[] bytes, byte[] prefix);

/**
 * Byte array comparator for use with TreeMap, TreeSet, etc.
 */
public static final Comparator<byte[]> BYTES_COMPARATOR;

Usage Examples:

byte[] array1 = Bytes.toBytes("apple");
byte[] array2 = Bytes.toBytes("banana");
byte[] array3 = Bytes.toBytes("apple");

// Comparison
int result1 = Bytes.compareTo(array1, array2); // < 0 (apple < banana)
int result2 = Bytes.compareTo(array1, array3); // 0 (equal)

// Equality
boolean equal = Bytes.equals(array1, array3); // true

// Prefix checking
byte[] data = Bytes.toBytes("Hello World");
byte[] prefix = Bytes.toBytes("Hello");
boolean hasPrefix = Bytes.startsWith(data, prefix); // true

// Use with sorted collections
TreeMap<byte[], String> sortedMap = new TreeMap<>(Bytes.BYTES_COMPARATOR);
sortedMap.put(Bytes.toBytes("key1"), "value1");
sortedMap.put(Bytes.toBytes("key2"), "value2");

Hash Computation

Generate hash codes for byte arrays optimized for use as map keys.

/**
 * Compute hash code for byte array
 * @param b Byte array
 * @return Hash code
 */
public static int hashCode(byte[] b);

/**
 * Compute hash code for byte array segment
 * @param bytes Byte array
 * @param offset Start position
 * @param length Length to hash
 * @return Hash code
 */
public static int hashCode(byte[] bytes, int offset, int length);

/**
 * Generate Integer hash suitable for HashMap keys
 * @param b Byte array
 * @return Integer hash code
 */
public static Integer mapKey(byte[] b);

/**
 * Generate Integer hash for array segment
 * @param b Byte array
 * @param length Length to hash
 * @return Integer hash code
 */
public static Integer mapKey(byte[] b, int length);

Usage Examples:

byte[] key1 = Bytes.toBytes("user123");
byte[] key2 = Bytes.toBytes("user456");

// Generate hash codes
int hash1 = Bytes.hashCode(key1);
int hash2 = Bytes.hashCode(key2);

// Use as HashMap keys
Map<Integer, String> dataMap = new HashMap<>();
dataMap.put(Bytes.mapKey(key1), "User data for 123");
dataMap.put(Bytes.mapKey(key2), "User data for 456");

// Hash array segments
byte[] largeArray = Bytes.toBytes("This is a long string with data");
int segmentHash = Bytes.hashCode(largeArray, 5, 10); // Hash bytes 5-14

String Representation

Convert byte arrays to readable string formats including hex and binary representations.

/**
 * Convert byte array to hex string
 * @param bytes Byte array
 * @return Lowercase hex string
 */
public static String toHexString(byte[] bytes);

/**
 * Convert hex string to byte array
 * @param str Hex string (must have even length)
 * @return Byte array
 * @throws IllegalArgumentException if odd length
 */
public static byte[] fromHexString(String str);

/**
 * Convert byte array to printable binary representation
 * @param b Byte array
 * @return String with printable chars and \xHH escapes
 */
public static String toStringBinary(byte[] b);

/**
 * Parse binary-escaped string to byte array
 * @param in String with \xHH escape sequences
 * @return Parsed byte array
 */
public static byte[] toBytesBinary(String in);

Usage Examples:

byte[] data = {0x48, 0x65, 0x6c, 0x6c, 0x6f}; // "Hello"

// Hex representation
String hex = Bytes.toHexString(data); // "48656c6c6f"
byte[] fromHex = Bytes.fromHexString(hex); // Recovers original data

// Binary representation (printable + escapes)
byte[] mixedData = {0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x00, 0x01, 0x02};
String binary = Bytes.toStringBinary(mixedData); // "Hello\x00\x01\x02"
byte[] fromBinary = Bytes.toBytesBinary(binary); // Recovers original

// Regular string representation
String readable = Bytes.toString(data); // "Hello"

Advanced Operations

Specialized operations for range splitting, incrementation, and prefix operations.

/**
 * Split byte range for MapReduce-style processing
 * @param a Start of range
 * @param b End of range  
 * @param num Number of splits
 * @return Array of split points
 */
public static byte[][] split(byte[] a, byte[] b, int num);

/**
 * Increment byte array by specified amount
 * @param value Byte array to increment (≤8 bytes)
 * @param amount Increment amount
 * @return Incremented byte array
 */
public static byte[] incrementBytes(byte[] value, long amount);

/**
 * Generate stop key for prefix scanning
 * @param prefix Prefix bytes
 * @return Stop key for range scanning, or null if prefix cannot be incremented
 */
public static byte[] stopKeyForPrefix(byte[] prefix);

/**
 * Write fixed-size string with zero padding
 * @param out DataOutput stream
 * @param s String to write
 * @param size Fixed field size
 * @throws IOException if string too long or write fails
 */
public static void writeStringFixedSize(DataOutput out, String s, int size) throws IOException;

Usage Examples:

// Range splitting for distributed processing
byte[] startKey = Bytes.toBytes("user000");
byte[] endKey = Bytes.toBytes("user999");
byte[][] splits = Bytes.split(startKey, endKey, 4); // Create 4 ranges

// Byte array incrementation
byte[] counter = Bytes.toBytes(100L);
byte[] incremented = Bytes.incrementBytes(counter, 50); // Now contains 150L

// Prefix scanning
byte[] prefix = Bytes.toBytes("user");
byte[] stopKey = Bytes.stopKeyForPrefix(prefix); // For scanning user* keys

// Fixed-size string writing
ByteArrayOutputStream baos = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(baos);
Bytes.writeStringFixedSize(dos, "test", 10); // Writes "test\0\0\0\0\0\0"

Constants and Utilities

Size Constants

public static final int SIZEOF_BOOLEAN = 1;
public static final int SIZEOF_BYTE = 1;
public static final int SIZEOF_CHAR = 2;
public static final int SIZEOF_SHORT = 2;
public static final int SIZEOF_INT = 4;
public static final int SIZEOF_FLOAT = 4;
public static final int SIZEOF_LONG = 8;
public static final int SIZEOF_DOUBLE = 8;

Encoding Constants

public static final String UTF8_ENCODING = "UTF-8";
public static final byte[] EMPTY_BYTE_ARRAY = new byte[0];

Comparator

/**
 * Optimized byte array comparator for collections
 */
public static final Comparator<byte[]> BYTES_COMPARATOR;

Performance Considerations

Optimization Features

Uses optimized comparison implementations (Unsafe when available, pure Java fallback)
Efficient big-endian encoding/decoding for numeric types
Minimal object allocation in conversion methods
Reusable comparator instances for sorted collections
Lazy string generation for hex/binary representations

Memory Efficiency

All conversion methods create new byte arrays (no shared state)
String conversions handle UTF-8 encoding efficiently
ByteBuffer operations preserve original buffer state
Hash computations use consistent algorithms suitable for distributed systems

Big Data Optimizations

Lexicographical comparison optimized for adjacent sorted data
Range splitting supports MapReduce-style distributed processing
Incrementation operations handle both positive and negative values
Prefix operations enable efficient range scanning in key-value stores

Install with Tessl CLI

npx tessl i tessl/maven-co-cask-cdap--cdap-api-common

docs

byte-utilities.md

data-format-system.md

index.md

schema-system.md

stream-processing.md

structured-records.md

tile.json

tessl/maven-co-cask-cdap--cdap-api-common

byte-utilities.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Byte Array Utilities

Capabilities

Type Conversions

Complex Type Conversions

Array Operations

Byte Array Writing and Reading

Comparison Operations

Hash Computation

String Representation

Advanced Operations

Constants and Utilities

Size Constants

Encoding Constants

Comparator

Performance Considerations

Optimization Features

Memory Efficiency

Big Data Optimizations

byte-utilities.mddocs/