0
# Storage Implementations
1

2
In-memory and persistent storage backends with different performance and durability characteristics. Choose the appropriate implementation based on your application's performance, persistence, and memory requirements.
3

4
## Capabilities
5

6
### InMemoryStore
7

8
Fast in-memory storage implementation that keeps all data deserialized in memory for maximum performance.
9

10
```java { .api }
11
/**
12
 * Implementation of KVStore that keeps data deserialized in memory. This store does not index
13
 * data; instead, whenever iterating over an indexed field, the stored data is copied and sorted
14
 * according to the index. This saves memory but makes iteration more expensive.
15
 */
16
public class InMemoryStore implements KVStore {
17
    /**
18
     * Creates a new in-memory store.
19
     */
20
    public InMemoryStore();
21
}
22
```
23

24
**Characteristics:**
25
- **Performance**: Fastest for read/write operations
26
- **Memory Usage**: Keeps all data in memory
27
- **Persistence**: Data lost when application stops
28
- **Indexing**: Sorts data on-demand during iteration
29
- **Thread Safety**: Thread-safe for concurrent access
30

31
**Usage Examples:**
32

33
```java
34
// Create an in-memory store
35
KVStore store = new InMemoryStore();
36

37
// Fast writes - data kept in memory
38
Person person = new Person("p1", "Alice", 30);
39
store.write(person);  // Very fast
40

41
// Fast reads - no disk I/O
42
Person retrieved = store.read(Person.class, "p1");  // Very fast
43

44
// Iteration sorts data on-demand
45
for (Person p : store.view(Person.class).index("age")) {
46
    System.out.println(p.name + " is " + p.age);  // Sorts during iteration
47
}
48

49
// Data is lost when store is closed or application exits
50
store.close();  // All data lost
51
```
52

53
**Best Use Cases:**
54
- Caching and temporary data storage
55
- High-performance scenarios where persistence isn't required
56
- Development and testing environments
57
- Applications with limited data sets that fit in memory
58

59
### LevelDB
60

61
Persistent storage implementation using LevelDB for durability and efficient indexed access.
62

63
```java { .api }
64
/**
65
 * Implementation of KVStore that uses LevelDB as the underlying data store.
66
 */
67
public class LevelDB implements KVStore {
68
    /**
69
     * Creates a LevelDB store at the specified path with default serializer.
70
     * @param path - Directory path where LevelDB files will be stored
71
     * @throws Exception If store creation or initialization fails
72
     */
73
    public LevelDB(File path) throws Exception;
74
    
75
    /**
76
     * Creates a LevelDB store at the specified path with custom serializer.
77
     * @param path - Directory path where LevelDB files will be stored  
78
     * @param serializer - Custom serializer for object persistence
79
     * @throws Exception If store creation or initialization fails
80
     */
81
    public LevelDB(File path, KVStoreSerializer serializer) throws Exception;
82
}
83
```
84

85
**Characteristics:**
86
- **Performance**: Slower than in-memory but optimized for disk operations
87
- **Memory Usage**: Uses disk storage with configurable caching
88
- **Persistence**: Data survives application restarts
89
- **Indexing**: Maintains persistent indices on disk for efficient queries
90
- **Thread Safety**: Thread-safe for concurrent access
91
- **Compression**: Automatic data compression to save disk space
92

93
**Usage Examples:**
94

95
```java
96
// Create a persistent LevelDB store
97
KVStore store = new LevelDB(new File("./my-data-store"));
98

99
// Or with custom serializer
100
KVStoreSerializer customSerializer = new MyCustomSerializer();
101
KVStore storeWithCustomSerializer = new LevelDB(new File("./custom-store"), customSerializer);
102

103
// Writes are persisted to disk
104
Person person = new Person("p1", "Alice", 30);
105
store.write(person);  // Data written to disk
106

107
// Data survives application restart
108
store.close();
109

110
// Reopen the same store later
111
KVStore reopenedStore = new LevelDB(new File("./my-data-store"));
112
Person retrieved = reopenedStore.read(Person.class, "p1");  // Data still there!
113

114
// Efficient indexed queries using disk-based indices
115
for (Person p : reopenedStore.view(Person.class).index("age").first(25).last(35)) {
116
    System.out.println(p.name + " is " + p.age);  // Uses persistent index
117
}
118

119
reopenedStore.close();
120
```
121

122
**Best Use Cases:**
123
- Production applications requiring data persistence
124
- Large datasets that don't fit in memory
125
- Applications that need to preserve data across restarts
126
- Scenarios where query performance is important
127

128
### Version Compatibility and Internal Management
129

130
LevelDB stores include version checking and use internal keys for metadata and type alias management.
131

132
```java { .api }
133
/**
134
 * Exception thrown when the store implementation is not compatible with the underlying data.
135
 */
136
public class UnsupportedStoreVersionException extends IOException {
137
}
138

139
/**
140
 * Internal constants used by LevelDB implementation for store management
141
 */
142
static final long STORE_VERSION = 1L;
143
static final byte[] STORE_VERSION_KEY = "__version__".getBytes(UTF_8);
144
static final byte[] METADATA_KEY = "__meta__".getBytes(UTF_8);
145
static final byte[] TYPE_ALIASES_KEY = "__types__".getBytes(UTF_8);
146
```
147

148
**Usage Examples:**
149

150
```java
151
try {
152
    // Attempt to open an existing LevelDB store
153
    KVStore store = new LevelDB(new File("./legacy-store"));
154
    // Store opened successfully
155
} catch (UnsupportedStoreVersionException e) {
156
    // The store was created with an incompatible version
157
    System.err.println("Store version incompatible - data migration required");
158
    // Handle version incompatibility (backup, migrate, etc.)
159
}
160
```
161

162
### Performance Comparison
163

164
**Write Performance:**
165
```java
166
// Benchmark: Writing 10,000 objects
167

168
// InMemoryStore - Fastest
169
KVStore memStore = new InMemoryStore();
170
long startTime = System.currentTimeMillis();
171
for (int i = 0; i < 10000; i++) {
172
    memStore.write(new Person("p" + i, "Name" + i, 20 + (i % 50)));
173
}
174
long memTime = System.currentTimeMillis() - startTime;
175
System.out.println("InMemoryStore: " + memTime + "ms");
176

177
// LevelDB - Slower but persistent
178
KVStore levelStore = new LevelDB(new File("./benchmark-store"));
179
startTime = System.currentTimeMillis();
180
for (int i = 0; i < 10000; i++) {
181
    levelStore.write(new Person("p" + i, "Name" + i, 20 + (i % 50)));
182
}
183
long levelTime = System.currentTimeMillis() - startTime;
184
System.out.println("LevelDB: " + levelTime + "ms");
185
```
186

187
**Query Performance:**
188
```java
189
// Indexed queries
190

191
// InMemoryStore - Sorts on each iteration
192
for (Person p : memStore.view(Person.class).index("age").first(30).last(40)) {
193
    // Data sorted during iteration
194
}
195

196
// LevelDB - Uses pre-built disk indices  
197
for (Person p : levelStore.view(Person.class).index("age").first(30).last(40)) {
198
    // Uses persistent index for efficient access
199
}
200
```
201

202
### Memory Management
203

204
**InMemoryStore Memory Usage:**
205
```java
206
KVStore memStore = new InMemoryStore();
207

208
// All objects kept in memory
209
for (int i = 0; i < 100000; i++) {
210
    memStore.write(new LargeObject("obj" + i, generateLargeData()));
211
    // Memory usage grows with each write
212
}
213

214
// Memory usage = number of objects × average object size
215
// Monitor with: Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()
216
```
217

218
**LevelDB Memory Usage:**
219
```java
220
KVStore levelStore = new LevelDB(new File("./large-store"));
221

222
// Objects stored on disk, minimal memory usage
223
for (int i = 0; i < 100000; i++) {
224
    levelStore.write(new LargeObject("obj" + i, generateLargeData()));
225
    // Memory usage remains relatively constant
226
}
227

228
// Memory usage = cache size + working set (much smaller than data size)
229
```
230

231
### Store Selection Guidelines
232

233
**Choose InMemoryStore when:**
234
- Maximum performance is required
235
- Data size fits comfortably in available memory  
236
- Data doesn't need to persist across application restarts
237
- Caching temporary or computed data
238
- Development and testing scenarios
239

240
**Choose LevelDB when:**
241
- Data must persist across application restarts
242
- Working with large datasets that may not fit in memory
243
- Query performance on indexed fields is important
244
- Production environments requiring reliability
245
- Need efficient range queries and sorting
246

247
**Hybrid Approach:**
248
```java
249
// Use both stores for different purposes
250
KVStore cache = new InMemoryStore();      // For frequently accessed data
251
KVStore persistent = new LevelDB(new File("./data"));  // For permanent storage
252

253
// Cache frequently accessed users in memory
254
User frequentUser = persistent.read(User.class, "frequent_user_123");
255
cache.write(frequentUser);  // Cache for fast access
256

257
// Read from cache first, fall back to persistent store
258
User user;
259
try {
260
    user = cache.read(User.class, userId);  // Try cache first
261
} catch (NoSuchElementException e) {
262
    user = persistent.read(User.class, userId);  // Fall back to persistent
263
    cache.write(user);  // Cache for next time
264
}
265
```