or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.spark/spark-unsafe_2.13@3.5.x

docs

array-operations.mdbitset-operations.mdbyte-array-utilities.mddata-types-utilities.mdhash-functions.mdindex.mdkv-iterator.mdmemory-management.mdplatform-operations.mdutf8-string-processing.md
tile.json

tessl/maven-org-apache-spark--spark-unsafe_2-13

tessl install tessl/maven-org-apache-spark--spark-unsafe_2-13@3.5.0

Low-level unsafe operations and optimized data structures for Apache Spark's internal memory management and performance-critical operations.

bitset-operations.mddocs/

BitSet Operations

High-performance bitset manipulation utilities for working with fixed-size uncompressed bitsets in memory. Designed for columnar data processing and efficient bit-level operations with word-aligned data structures.

Capabilities

Set Bit

Sets the bit at the specified index to true.

/**
 * Sets the bit at the specified index to true
 * @param baseObject the base object containing the bitset (or null for absolute addresses)
 * @param baseOffset the offset from the base object where the bitset starts
 * @param index the bit index to set (0-based)
 */
public static void set(Object baseObject, long baseOffset, int index);

Usage Example:

import org.apache.spark.unsafe.bitset.BitSetMethods;
import org.apache.spark.unsafe.Platform;

// Allocate memory for a bitset (8 words = 512 bits)
long address = Platform.allocateMemory(64);

// Set bit at index 42
BitSetMethods.set(null, address, 42);

// Clean up
Platform.freeMemory(address);

Unset Bit

Sets the bit at the specified index to false.

/**
 * Sets the bit at the specified index to false
 * @param baseObject the base object containing the bitset (or null for absolute addresses)
 * @param baseOffset the offset from the base object where the bitset starts
 * @param index the bit index to unset (0-based)
 */
public static void unset(Object baseObject, long baseOffset, int index);

Check Bit Status

Returns true if the bit is set at the specified index.

/**
 * Returns true if the bit is set at the specified index
 * @param baseObject the base object containing the bitset (or null for absolute addresses)
 * @param baseOffset the offset from the base object where the bitset starts
 * @param index the bit index to check (0-based)
 * @return true if the bit is set, false otherwise
 */
public static boolean isSet(Object baseObject, long baseOffset, int index);

Check Any Bits Set

Returns true if any bit is set in the entire bitset.

/**
 * Returns true if any bit is set in the bitset
 * @param baseObject the base object containing the bitset (or null for absolute addresses)
 * @param baseOffset the offset from the base object where the bitset starts
 * @param bitSetWidthInWords the size of the bitset in 8-byte words
 * @return true if any bit is set, false if all bits are unset
 */
public static boolean anySet(Object baseObject, long baseOffset, long bitSetWidthInWords);

Find Next Set Bit

Returns the index of the first bit that is set to true on or after the specified starting index. Returns -1 if no such bit exists.

/**
 * Returns the index of the next set bit starting from the given index
 * @param baseObject the base object containing the bitset (or null for absolute addresses)
 * @param baseOffset the offset from the base object where the bitset starts
 * @param fromIndex the index to start checking from (inclusive, 0-based)
 * @param bitsetSizeInWords the size of the bitset in 8-byte words
 * @return the index of the next set bit, or -1 if no such bit exists
 */
public static int nextSetBit(
    Object baseObject,
    long baseOffset,
    int fromIndex,
    int bitsetSizeInWords
);

Usage Example - Iterating Over Set Bits:

import org.apache.spark.unsafe.bitset.BitSetMethods;
import org.apache.spark.unsafe.Platform;

// Allocate and initialize a bitset
long address = Platform.allocateMemory(64); // 8 words = 512 bits
int sizeInWords = 8;

// Set some bits
BitSetMethods.set(null, address, 10);
BitSetMethods.set(null, address, 25);
BitSetMethods.set(null, address, 100);

// Iterate over all set bits
for (int i = BitSetMethods.nextSetBit(null, address, 0, sizeInWords); 
     i >= 0; 
     i = BitSetMethods.nextSetBit(null, address, i + 1, sizeInWords)) {
    System.out.println("Bit " + i + " is set");
}
// Output: Bit 10 is set, Bit 25 is set, Bit 100 is set

Platform.freeMemory(address);

Architecture Notes

  • Word-Aligned: Assumes bitset data is word-aligned (multiple of 8 bytes in length)
  • Fixed-Size: Designed for fixed-size uncompressed bitsets
  • Platform Operations: Uses Platform class for direct memory access
  • Performance: Each bit occupies exactly one bit of storage with efficient bit manipulation
  • Memory Layout: Bits are organized in 64-bit words with standard bit ordering

Import Requirements

import org.apache.spark.unsafe.bitset.BitSetMethods;
import org.apache.spark.unsafe.Platform; // Required for memory operations