A local key/value store abstraction library for Apache Spark applications that provides automatic serialization, indexing, and key management features.
—
Basic CRUD operations for storing and retrieving objects with automatic key management and serialization. These operations form the foundation of the KVStore API and handle all data persistence operations.
Retrieves a specific object instance by its natural key from the store.
/**
* Read a specific instance of an object.
* @param klass - The object's class type
* @param naturalKey - The object's "natural key", which uniquely identifies it. Null keys are not allowed.
* @return The retrieved object instance
* @throws java.util.NoSuchElementException If an element with the given key does not exist
* @throws Exception If serialization or store access fails
*/
<T> T read(Class<T> klass, Object naturalKey) throws Exception;Usage Examples:
// Read a person by ID
Person person = store.read(Person.class, "person123");
// Read configuration by key
Config config = store.read(Config.class, "app.settings");Writes objects to the store with automatic indexing and serialization. Creates or updates entries based on the natural key.
/**
* Writes the given object to the store, including indexed fields. Indices are updated based
* on the annotated fields of the object's class.
*
* Writes may be slower when the object already exists in the store, since it will involve
* updating existing indices.
*
* @param value - The object to write. Cannot be null.
* @throws Exception If serialization or store access fails
*/
void write(Object value) throws Exception;Usage Examples:
// Write a new person
Person newPerson = new Person("p123", "John Doe", 25);
store.write(newPerson);
// Update existing person (same natural key)
Person updated = new Person("p123", "John Smith", 26);
store.write(updated); // Overwrites previous entryRemoves objects and all associated data including index entries from the store.
/**
* Removes an object and all data related to it, like index entries, from the store.
* @param type - The object's type
* @param naturalKey - The object's "natural key", which uniquely identifies it. Null keys are not allowed.
* @throws java.util.NoSuchElementException If an element with the given key does not exist
* @throws Exception If store access fails
*/
void delete(Class<?> type, Object naturalKey) throws Exception;Usage Examples:
// Delete a person by ID
store.delete(Person.class, "person123");
// Delete configuration entry
store.delete(Config.class, "app.settings");Returns the total number of objects of a given type or matching specific index values.
/**
* Returns the number of items of the given type currently in the store.
* @param type - The object type to count
* @return Total count of objects of the specified type
* @throws Exception If store access fails
*/
long count(Class<?> type) throws Exception;
/**
* Returns the number of items of the given type which match the given indexed value.
* @param type - The object type to count
* @param index - The index name to filter by
* @param indexedValue - The value to match in the index
* @return Count of objects matching the indexed value
* @throws Exception If store access fails
*/
long count(Class<?> type, String index, Object indexedValue) throws Exception;Usage Examples:
// Count all persons in the store
long totalPersons = store.count(Person.class);
// Count persons with specific age
long adultsCount = store.count(Person.class, "age", 30);
// Count persons with specific status
long activeUsers = store.count(Person.class, "status", "active");Efficiently removes multiple items from the store based on index values.
/**
* A cheaper way to remove multiple items from the KVStore
* @param klass - The object type to remove from
* @param index - The index name to filter by
* @param indexValues - Collection of index values to match for removal
* @return true if any items were removed, false otherwise
* @throws Exception If store access fails
*/
<T> boolean removeAllByIndexValues(Class<T> klass, String index, Collection<?> indexValues) throws Exception;Usage Examples:
// Remove all persons with specific IDs
List<String> idsToRemove = Arrays.asList("p1", "p2", "p3");
boolean removed = store.removeAllByIndexValues(Person.class, KVIndex.NATURAL_INDEX_NAME, idsToRemove);
// Remove all persons with specific ages
List<Integer> agesToRemove = Arrays.asList(25, 30, 35);
store.removeAllByIndexValues(Person.class, "age", agesToRemove);Store and retrieve application-specific metadata that persists across store sessions.
/**
* Returns app-specific metadata from the store, or null if it's not currently set.
* The metadata type is application-specific. This is a convenience method so that applications
* don't need to define their own keys for this information.
* @param klass - The metadata class type
* @return The metadata object or null if not set
* @throws Exception If deserialization or store access fails
*/
<T> T getMetadata(Class<T> klass) throws Exception;
/**
* Writes the given value in the store metadata key.
* @param value - The metadata object to store, or null to remove
* @throws Exception If serialization or store access fails
*/
void setMetadata(Object value) throws Exception;Usage Examples:
// Store application configuration as metadata
AppConfig config = new AppConfig("1.0", "production");
store.setMetadata(config);
// Retrieve application configuration
AppConfig retrievedConfig = store.getMetadata(AppConfig.class);
if (retrievedConfig != null) {
System.out.println("Version: " + retrievedConfig.version);
}
// Remove metadata
store.setMetadata(null);Properly close the store to release resources and ensure data persistence.
/**
* Closes the store and releases all associated resources.
* For persistent stores like LevelDB, ensures all data is written to disk.
* @throws IOException If there are issues closing the underlying storage
*/
void close() throws IOException;Usage Examples:
KVStore store = new LevelDB(new File("./mystore"));
try {
// Use the store
store.write(someObject);
// ... other operations
} finally {
// Always close the store
store.close();
}
// Or use try-with-resources
try (KVStore store = new InMemoryStore()) {
store.write(someObject);
// Store automatically closed
}Install with Tessl CLI
npx tessl i tessl/maven-org-apache-spark--spark-kvstore