or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

array-operations.mddata-types-utilities.mdhash-bitset-operations.mdindex.mdmemory-management.mdplatform-operations.mdutf8-string-processing.md
tile.json

platform-operations.mddocs/

Platform Operations

Direct memory access and platform-specific operations using sun.misc.Unsafe for maximum performance in big data processing scenarios. This module provides the foundational low-level operations that enable Spark's high-performance data processing capabilities.

Capabilities

Direct Memory Access

Methods for reading and writing primitive values directly to/from memory addresses, bypassing Java's normal memory access restrictions for maximum performance.

public static int getInt(Object object, long offset);
public static void putInt(Object object, long offset, int value);
public static boolean getBoolean(Object object, long offset);
public static void putBoolean(Object object, long offset, boolean value);
public static byte getByte(Object object, long offset);
public static void putByte(Object object, long offset, byte value);
public static short getShort(Object object, long offset);
public static void putShort(Object object, long offset, short value);
public static long getLong(Object object, long offset);
public static void putLong(Object object, long offset, long value);
public static float getFloat(Object object, long offset);
public static void putFloat(Object object, long offset, float value);
public static double getDouble(Object object, long offset);
public static void putDouble(Object object, long offset, double value);

Volatile Memory Operations

Memory operations with volatile semantics for thread-safe object reference access.

public static Object getObjectVolatile(Object object, long offset);
public static void putObjectVolatile(Object object, long offset, Object value);

Off-Heap Memory Management

Low-level memory allocation and deallocation operations for managing off-heap memory directly.

public static long allocateMemory(long size);
public static void freeMemory(long address);
public static long reallocateMemory(long address, long oldSize, long newSize);
public static ByteBuffer allocateDirectBuffer(int size);

Memory Manipulation

Bulk memory operations for efficient data movement and initialization.

public static void setMemory(Object object, long offset, long size, byte value);
public static void setMemory(long address, byte value, long size);
public static void copyMemory(Object src, long srcOffset, Object dst, long dstOffset, long length);

Platform Information

Methods to query platform capabilities and characteristics.

public static boolean cleanerCreateMethodIsDefined();
public static boolean unaligned();

Exception Handling

Utility for throwing exceptions that bypass compiler checks.

public static void throwException(Throwable t);

Array Base Offsets

Constants providing the base memory offsets for different array types, used for direct array access.

public static final int BOOLEAN_ARRAY_OFFSET;
public static final int BYTE_ARRAY_OFFSET;
public static final int SHORT_ARRAY_OFFSET;
public static final int INT_ARRAY_OFFSET;
public static final int LONG_ARRAY_OFFSET;
public static final int FLOAT_ARRAY_OFFSET;
public static final int DOUBLE_ARRAY_OFFSET;

Unsafe Aligned Offset Operations

Utility operations for managing record length offsets across different platforms with proper alignment handling provided by the UnsafeAlignedOffset class.

UAO Size Management

Methods for managing the Unsafe Aligned Offset size, which varies between 4 and 8 bytes depending on the platform.

public class UnsafeAlignedOffset {
    public static void setUaoSize(int size);
    public static int getUaoSize();
    public static int getSize(Object object, long offset);
    public static void putSize(Object object, long offset, int value);
}

Usage Examples

Basic Memory Operations

import org.apache.spark.unsafe.Platform;

// Allocate 1KB of off-heap memory
long address = Platform.allocateMemory(1024);

// Write and read values
Platform.putLong(null, address, 123456789L);
long value = Platform.getLong(null, address);

// Fill memory with specific byte value
Platform.setMemory(null, address, 1024, (byte) 0xFF);

// Free the memory
Platform.freeMemory(address);

Array Access

import org.apache.spark.unsafe.Platform;

byte[] array = new byte[100];
long offset = Platform.BYTE_ARRAY_OFFSET;

// Direct array access
Platform.putByte(array, offset + 10, (byte) 42);
byte value = Platform.getByte(array, offset + 10);

Memory Copying

import org.apache.spark.unsafe.Platform;

byte[] source = "Hello World".getBytes();
byte[] dest = new byte[source.length];

// High-performance memory copy
Platform.copyMemory(
    source, Platform.BYTE_ARRAY_OFFSET,
    dest, Platform.BYTE_ARRAY_OFFSET,
    source.length
);