CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-apache-spark--spark-kvstore

A local key/value store abstraction library for Apache Spark applications that provides automatic serialization, indexing, and key management features.

Pending
Overview
Eval results
Files

core-operations.mddocs/

Core Store Operations

Basic CRUD operations for storing and retrieving objects with automatic key management and serialization. These operations form the foundation of the KVStore API and handle all data persistence operations.

Capabilities

Reading Objects

Retrieves a specific object instance by its natural key from the store.

/**
 * Read a specific instance of an object.
 * @param klass - The object's class type
 * @param naturalKey - The object's "natural key", which uniquely identifies it. Null keys are not allowed.
 * @return The retrieved object instance
 * @throws java.util.NoSuchElementException If an element with the given key does not exist
 * @throws Exception If serialization or store access fails
 */
<T> T read(Class<T> klass, Object naturalKey) throws Exception;

Usage Examples:

// Read a person by ID
Person person = store.read(Person.class, "person123");

// Read configuration by key
Config config = store.read(Config.class, "app.settings");

Writing Objects

Writes objects to the store with automatic indexing and serialization. Creates or updates entries based on the natural key.

/**
 * Writes the given object to the store, including indexed fields. Indices are updated based
 * on the annotated fields of the object's class.
 * 
 * Writes may be slower when the object already exists in the store, since it will involve
 * updating existing indices.
 * 
 * @param value - The object to write. Cannot be null.
 * @throws Exception If serialization or store access fails
 */
void write(Object value) throws Exception;

Usage Examples:

// Write a new person
Person newPerson = new Person("p123", "John Doe", 25);
store.write(newPerson);

// Update existing person (same natural key)
Person updated = new Person("p123", "John Smith", 26);
store.write(updated); // Overwrites previous entry

Deleting Objects

Removes objects and all associated data including index entries from the store.

/**
 * Removes an object and all data related to it, like index entries, from the store.
 * @param type - The object's type
 * @param naturalKey - The object's "natural key", which uniquely identifies it. Null keys are not allowed.
 * @throws java.util.NoSuchElementException If an element with the given key does not exist
 * @throws Exception If store access fails
 */
void delete(Class<?> type, Object naturalKey) throws Exception;

Usage Examples:

// Delete a person by ID
store.delete(Person.class, "person123");

// Delete configuration entry
store.delete(Config.class, "app.settings");

Counting Objects

Returns the total number of objects of a given type or matching specific index values.

/**
 * Returns the number of items of the given type currently in the store.
 * @param type - The object type to count
 * @return Total count of objects of the specified type
 * @throws Exception If store access fails
 */
long count(Class<?> type) throws Exception;

/**
 * Returns the number of items of the given type which match the given indexed value.
 * @param type - The object type to count
 * @param index - The index name to filter by
 * @param indexedValue - The value to match in the index
 * @return Count of objects matching the indexed value
 * @throws Exception If store access fails
 */
long count(Class<?> type, String index, Object indexedValue) throws Exception;

Usage Examples:

// Count all persons in the store
long totalPersons = store.count(Person.class);

// Count persons with specific age
long adultsCount = store.count(Person.class, "age", 30);

// Count persons with specific status
long activeUsers = store.count(Person.class, "status", "active");

Bulk Removal Operations

Efficiently removes multiple items from the store based on index values.

/**
 * A cheaper way to remove multiple items from the KVStore
 * @param klass - The object type to remove from
 * @param index - The index name to filter by
 * @param indexValues - Collection of index values to match for removal
 * @return true if any items were removed, false otherwise
 * @throws Exception If store access fails
 */
<T> boolean removeAllByIndexValues(Class<T> klass, String index, Collection<?> indexValues) throws Exception;

Usage Examples:

// Remove all persons with specific IDs
List<String> idsToRemove = Arrays.asList("p1", "p2", "p3");
boolean removed = store.removeAllByIndexValues(Person.class, KVIndex.NATURAL_INDEX_NAME, idsToRemove);

// Remove all persons with specific ages
List<Integer> agesToRemove = Arrays.asList(25, 30, 35);
store.removeAllByIndexValues(Person.class, "age", agesToRemove);

Metadata Operations

Store and retrieve application-specific metadata that persists across store sessions.

/**
 * Returns app-specific metadata from the store, or null if it's not currently set.
 * The metadata type is application-specific. This is a convenience method so that applications
 * don't need to define their own keys for this information.
 * @param klass - The metadata class type
 * @return The metadata object or null if not set
 * @throws Exception If deserialization or store access fails
 */
<T> T getMetadata(Class<T> klass) throws Exception;

/**
 * Writes the given value in the store metadata key.
 * @param value - The metadata object to store, or null to remove
 * @throws Exception If serialization or store access fails
 */
void setMetadata(Object value) throws Exception;

Usage Examples:

// Store application configuration as metadata
AppConfig config = new AppConfig("1.0", "production");
store.setMetadata(config);

// Retrieve application configuration
AppConfig retrievedConfig = store.getMetadata(AppConfig.class);
if (retrievedConfig != null) {
    System.out.println("Version: " + retrievedConfig.version);
}

// Remove metadata
store.setMetadata(null);

Resource Management

Properly close the store to release resources and ensure data persistence.

/**
 * Closes the store and releases all associated resources.
 * For persistent stores like LevelDB, ensures all data is written to disk.
 * @throws IOException If there are issues closing the underlying storage
 */
void close() throws IOException;

Usage Examples:

KVStore store = new LevelDB(new File("./mystore"));
try {
    // Use the store
    store.write(someObject);
    // ... other operations
} finally {
    // Always close the store
    store.close();
}

// Or use try-with-resources
try (KVStore store = new InMemoryStore()) {
    store.write(someObject);
    // Store automatically closed
}

Install with Tessl CLI

npx tessl i tessl/maven-org-apache-spark--spark-kvstore

docs

core-operations.md

index.md

indexing.md

querying-views.md

serialization.md

storage-implementations.md

type-information.md

tile.json