A local key/value store abstraction library for Apache Spark applications that provides automatic serialization, indexing, and key management features.
—
Configurable iteration and querying system with index-based filtering, sorting, and pagination. The view system provides flexible ways to retrieve and iterate over stored data without loading everything into memory.
Create a configurable view for iterating over entities of a specific type.
/**
* Returns a configurable view for iterating over entities of the given type.
* @param type - The class type to create a view for
* @return A KVStoreView instance for configuring iteration
* @throws Exception If type introspection or store access fails
*/
<T> KVStoreView<T> view(Class<T> type) throws Exception;Usage Examples:
// Create a basic view for all persons
KVStoreView<Person> allPersons = store.view(Person.class);
// Iterate over all persons
for (Person person : allPersons) {
System.out.println(person.name);
}Configure views to iterate based on specific indices for efficient data access.
/**
* Iterates according to the given index.
* @param name - The index name to use for iteration
* @return This view instance for method chaining
*/
public KVStoreView<T> index(String name);Usage Examples:
// Query persons by age index
KVStoreView<Person> byAge = store.view(Person.class).index("age");
// Query persons by name index
KVStoreView<Person> byName = store.view(Person.class).index("name");
// Iterate in natural key order (default)
KVStoreView<Person> byNaturalKey = store.view(Person.class).index(KVIndex.NATURAL_INDEX_NAME);Set start and end boundaries for iteration over indexed values.
/**
* Iterates starting at the given value of the chosen index (inclusive).
* @param value - The starting value for iteration
* @return This view instance for method chaining
*/
public KVStoreView<T> first(Object value);
/**
* Stops iteration at the given value of the chosen index (inclusive).
* @param value - The ending value for iteration
* @return This view instance for method chaining
*/
public KVStoreView<T> last(Object value);Usage Examples:
// Get persons aged 25 to 65
KVStoreView<Person> workingAge = store.view(Person.class)
.index("age")
.first(25)
.last(65);
// Get persons with names from "A" to "M"
KVStoreView<Person> firstHalf = store.view(Person.class)
.index("name")
.first("A")
.last("M");
// Get all persons aged 30 exactly
KVStoreView<Person> thirtyYearOlds = store.view(Person.class)
.index("age")
.first(30)
.last(30);Control the sort order of iteration results.
/**
* Reverses the order of iteration. By default, iterates in ascending order.
* @return This view instance for method chaining
*/
public KVStoreView<T> reverse();Usage Examples:
// Get persons in descending age order
KVStoreView<Person> oldestFirst = store.view(Person.class)
.index("age")
.reverse();
// Get persons in reverse alphabetical order
KVStoreView<Person> reverseAlpha = store.view(Person.class)
.index("name")
.reverse();Control how many results to retrieve and skip for pagination scenarios.
/**
* Stops iteration after a number of elements has been retrieved.
* @param max - Maximum number of elements to retrieve (must be positive)
* @return This view instance for method chaining
*/
public KVStoreView<T> max(long max);
/**
* Skips a number of elements at the start of iteration. Skipped elements are not accounted
* when using max(long).
* @param n - Number of elements to skip
* @return This view instance for method chaining
*/
public KVStoreView<T> skip(long n);Usage Examples:
// Get first 10 persons
KVStoreView<Person> firstTen = store.view(Person.class).max(10);
// Get persons 11-20 (pagination)
KVStoreView<Person> secondPage = store.view(Person.class)
.skip(10)
.max(10);
// Get top 5 oldest persons
KVStoreView<Person> oldestFive = store.view(Person.class)
.index("age")
.reverse()
.max(5);Query data using parent-child index relationships for hierarchical data structures.
/**
* Defines the value of the parent index when iterating over a child index. Only elements that
* match the parent index's value will be included in the iteration.
*
* Required for iterating over child indices, will generate an error if iterating over a
* parent-less index.
* @param value - The parent index value to filter by
* @return This view instance for method chaining
*/
public KVStoreView<T> parent(Object value);Usage Examples:
// Assuming Task has a parent index "projectId" and child index "priority"
// Get all high-priority tasks for project "proj123"
KVStoreView<Task> highPriorityTasks = store.view(Task.class)
.index("priority")
.parent("proj123") // Filter by parent project ID
.first("high")
.last("high");
// Get all tasks for a specific project, ordered by status
KVStoreView<Task> projectTasks = store.view(Task.class)
.index("status")
.parent("proj123");Get closeable iterators for fine-grained control over resource management and batch processing.
/**
* Returns an iterator for the current configuration.
* @return A closeable iterator that should be explicitly closed
* @throws Exception If iteration setup fails
*/
public KVStoreIterator<T> closeableIterator() throws Exception;
public interface KVStoreIterator<T> extends Iterator<T>, Closeable {
/**
* Retrieve multiple elements from the store.
* @param max - Maximum number of elements to retrieve
* @return List containing up to max elements
*/
List<T> next(int max);
/**
* Skip in the iterator.
* @param n - Number of elements to skip
* @return Whether there are items left after skipping
*/
boolean skip(long n);
}Usage Examples:
// Manual iterator management with try-with-resources
try (KVStoreIterator<Person> iterator = store.view(Person.class)
.index("age")
.first(25)
.closeableIterator()) {
// Process in batches of 100
List<Person> batch;
while (!(batch = iterator.next(100)).isEmpty()) {
processBatch(batch);
}
}
// Skip and retrieve pattern
try (KVStoreIterator<Person> iterator = store.view(Person.class)
.index("name")
.closeableIterator()) {
// Skip first 50 entries
iterator.skip(50);
// Get next 20 entries
List<Person> next20 = iterator.next(20);
// Check if more entries exist
boolean hasMore = iterator.hasNext();
}Combining multiple view configuration methods for sophisticated queries.
Usage Examples:
// Get the 10 oldest active users, starting from age 21
KVStoreView<User> query1 = store.view(User.class)
.index("age") // Order by age
.first(21) // Start from age 21
.reverse() // Oldest first
.max(10); // Limit to 10 results
// Get users with names between "John" and "Mary", skip first 5
KVStoreView<User> query2 = store.view(User.class)
.index("name") // Order by name
.first("John") // Start from "John"
.last("Mary") // End at "Mary"
.skip(5) // Skip first 5 matches
.max(20); // Limit to 20 results
// Get tasks for specific project, ordered by priority, highest first
KVStoreView<Task> query3 = store.view(Task.class)
.index("priority") // Order by priority
.parent("project123") // Only tasks from project123
.reverse(); // Highest priority first
// Pagination: get page 3 (entries 21-30) of users ordered by registration date
int page = 3;
int pageSize = 10;
KVStoreView<User> page3 = store.view(User.class)
.index("registrationDate")
.skip((page - 1) * pageSize) // Skip 20 entries
.max(pageSize); // Get 10 entriesInstall with Tessl CLI
npx tessl i tessl/maven-org-apache-spark--spark-kvstore