or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.apache.spark/spark-unsafe_2.13@3.5.x

docs

array-operations.mdbitset-operations.mdbyte-array-utilities.mddata-types-utilities.mdhash-functions.mdindex.mdkv-iterator.mdmemory-management.mdplatform-operations.mdutf8-string-processing.md
tile.json

tessl/maven-org-apache-spark--spark-unsafe_2-13

tessl install tessl/maven-org-apache-spark--spark-unsafe_2-13@3.5.0

Low-level unsafe operations and optimized data structures for Apache Spark's internal memory management and performance-critical operations.

kv-iterator.mddocs/

Key-Value Iterator

Abstract iterator interface for traversing key-value pairs in Spark's unsafe operations. Provides a generic foundation for iterating over data structures containing key-value relationships with proper resource management.

Capabilities

KVIterator Abstract Class

Generic abstract class for key-value iteration with type safety and resource management.

/**
 * Abstract iterator for key-value pairs with generic type parameters
 * @param <K> the type of keys returned by this iterator
 * @param <V> the type of values returned by this iterator
 */
public abstract class KVIterator<K, V> {
    /** 
     * Advances to the next key-value pair
     * @return true if a next pair exists, false if iteration is complete
     * @throws IOException if an I/O error occurs during iteration
     */
    public abstract boolean next() throws IOException;
    
    /**
     * Returns the current key
     * @return the key of the current key-value pair
     */
    public abstract K getKey();
    
    /**
     * Returns the current value
     * @return the value of the current key-value pair
     */
    public abstract V getValue();
    
    /**
     * Closes the iterator and releases any associated resources
     */
    public abstract void close();
}

Usage Patterns

Basic Iterator Implementation

import org.apache.spark.unsafe.KVIterator;
import java.io.IOException;

// Example concrete implementation
public class MyKVIterator extends KVIterator<String, Integer> {
    private int currentIndex = -1;
    private String[] keys = {"key1", "key2", "key3"};
    private Integer[] values = {100, 200, 300};
    
    @Override
    public boolean next() throws IOException {
        currentIndex++;
        return currentIndex < keys.length;
    }
    
    @Override
    public String getKey() {
        return keys[currentIndex];
    }
    
    @Override
    public Integer getValue() {
        return values[currentIndex];
    }
    
    @Override
    public void close() {
        // Clean up resources if needed
    }
}

Safe Iterator Usage

KVIterator<String, Integer> iterator = new MyKVIterator();
try {
    while (iterator.next()) {
        String key = iterator.getKey();
        Integer value = iterator.getValue();
        System.out.println(key + " -> " + value);
    }
} finally {
    iterator.close(); // Always close to prevent resource leaks
}

Try-with-Resources Pattern

For implementations that implement AutoCloseable:

// If your implementation also implements AutoCloseable
public class AutoCloseableKVIterator<K, V> extends KVIterator<K, V> 
    implements AutoCloseable {
    // ... implementation
}

// Usage with automatic resource management
try (AutoCloseableKVIterator<String, Integer> iterator = new AutoCloseableKVIterator<>()) {
    while (iterator.next()) {
        processKeyValue(iterator.getKey(), iterator.getValue());
    }
}

Architecture Notes

  • Generic Type Safety: Supports parameterized types for both keys and values
  • Exception Handling: next() method can throw IOException for I/O errors
  • Resource Management: close() method enables proper cleanup of resources
  • Abstract Design: Provides flexibility for various underlying data structures
  • Iterator Protocol: Follows standard iterator pattern with next() advancement

Error Handling

The next() method can throw IOException, so proper exception handling is required:

try {
    while (iterator.next()) {
        // Process current key-value pair
        processData(iterator.getKey(), iterator.getValue());
    }
} catch (IOException e) {
    // Handle I/O errors during iteration
    System.err.println("Error during iteration: " + e.getMessage());
} finally {
    iterator.close();
}

Import Requirements

import org.apache.spark.unsafe.KVIterator;
import java.io.IOException; // Required for exception handling