0
# Data Query and Inspection
1

2
Methods for querying stored data, retrieving keys and values, and inspecting storage contents with filtering capabilities.
3

4
## Capabilities
5

6
### Get All Keys
7

8
Returns an array of all stored keys, optionally filtered.
9

10
```javascript { .api }
11
/**
12
 * Returns an array of all stored keys, optionally filtered
13
 * @param filter - Optional filter function to select specific items
14
 * @returns Promise resolving to array of key strings
15
 */
16
async function keys(filter?: (datum: any) => boolean): Promise<string[]>;
17
```
18

19
**Usage Examples:**
20

21
```javascript
22
const storage = require('node-persist');
23
await storage.init();
24

25
// Store some data
26
await storage.setItem('user1', { name: 'Alice', active: true });
27
await storage.setItem('user2', { name: 'Bob', active: false });
28
await storage.setItem('config', { theme: 'dark' });
29

30
// Get all keys
31
const allKeys = await storage.keys();
32
console.log(allKeys); // ['user1', 'user2', 'config']
33

34
// Filter keys (filter receives full datum object with key, value, ttl)
35
const activeUserKeys = await storage.keys(datum => 
36
  datum.key.startsWith('user') && datum.value.active
37
);
38
console.log(activeUserKeys); // ['user1']
39
```
40

41
### Get All Values
42

43
Returns an array of all stored values, optionally filtered.
44

45
```javascript { .api }
46
/**
47
 * Returns an array of all stored values, optionally filtered
48
 * @param filter - Optional filter function to select specific items
49
 * @returns Promise resolving to array of stored values
50
 */
51
async function values(filter?: (datum: any) => boolean): Promise<any[]>;
52
```
53

54
**Usage Examples:**
55

56
```javascript
57
// Get all values
58
const allValues = await storage.values();
59
console.log(allValues); 
60
// [{ name: 'Alice', active: true }, { name: 'Bob', active: false }, { theme: 'dark' }]
61

62
// Filter values
63
const activeUsers = await storage.values(datum => 
64
  datum.key.startsWith('user') && datum.value.active
65
);
66
console.log(activeUsers); // [{ name: 'Alice', active: true }]
67

68
// Filter by value properties
69
const darkThemeConfigs = await storage.values(datum => 
70
  datum.value.theme === 'dark'
71
);
72
```
73

74
### Get Storage Length
75

76
Returns the count of stored items, optionally filtered.
77

78
```javascript { .api }
79
/**
80
 * Returns the count of stored items, optionally filtered
81
 * @param filter - Optional filter function to select specific items
82
 * @returns Promise resolving to number of items
83
 */
84
async function length(filter?: (datum: any) => boolean): Promise<number>;
85
```
86

87
**Usage Examples:**
88

89
```javascript
90
// Get total count
91
const totalItems = await storage.length();
92
console.log(totalItems); // 3
93

94
// Count filtered items
95
const userCount = await storage.length(datum => datum.key.startsWith('user'));
96
console.log(userCount); // 2
97

98
const activeUserCount = await storage.length(datum => 
99
  datum.key.startsWith('user') && datum.value.active
100
);
101
console.log(activeUserCount); // 1
102
```
103

104
### Values with Key Match
105

106
Returns values for keys that match a string or regular expression pattern.
107

108
```javascript { .api }
109
/**
110
 * Returns values for keys matching a string or RegExp pattern
111
 * @param match - String or RegExp to match against keys
112
 * @returns Promise resolving to array of matching values
113
 */
114
function valuesWithKeyMatch(match: string | RegExp): Promise<any[]>;
115
```
116

117
**Usage Examples:**
118

119
```javascript
120
await storage.setItem('user:alice', { name: 'Alice' });
121
await storage.setItem('user:bob', { name: 'Bob' });
122
await storage.setItem('admin:charlie', { name: 'Charlie' });
123
await storage.setItem('config:theme', 'dark');
124

125
// String matching (substring search)
126
const userValues = await storage.valuesWithKeyMatch('user');
127
console.log(userValues); // [{ name: 'Alice' }, { name: 'Bob' }]
128

129
// RegExp matching
130
const colonUsers = await storage.valuesWithKeyMatch(/^user:/);
131
console.log(colonUsers); // [{ name: 'Alice' }, { name: 'Bob' }]
132

133
// Case-insensitive RegExp
134
const adminValues = await storage.valuesWithKeyMatch(/admin/i);
135
console.log(adminValues); // [{ name: 'Charlie' }]
136
```
137

138
### Iterate Over All Items
139

140
Asynchronously iterates over all stored key-value pairs.
141

142
```javascript { .api }
143
/**
144
 * Asynchronously iterates over all stored key-value pairs
145
 * @param callback - Async function called for each item
146
 * @returns Promise resolving when iteration completes
147
 */
148
async function forEach(callback: (datum: { key: string, value: any }) => Promise<void>): Promise<void>;
149
```
150

151
**Usage Examples:**
152

153
```javascript
154
// Simple iteration
155
await storage.forEach(async (datum) => {
156
  console.log(`${datum.key}: ${JSON.stringify(datum.value)}`);
157
});
158

159
// Process each item
160
await storage.forEach(async (datum) => {
161
  if (datum.key.startsWith('user:')) {
162
    // Perform async operation on user data
163
    await processUser(datum.value);
164
  }
165
});
166

167
// Update items during iteration
168
await storage.forEach(async (datum) => {
169
  if (datum.value.needsUpdate) {
170
    await storage.setItem(datum.key, { 
171
      ...datum.value, 
172
      updated: new Date().toISOString() 
173
    });
174
  }
175
});
176
```
177

178
### Get All Data
179

180
Returns the complete raw data array with all datum objects containing key, value, and TTL information.
181

182
```javascript { .api }
183
/**
184
 * Returns all stored data as array of datum objects
185
 * @returns Promise resolving to array of datum objects
186
 */
187
function data(): Promise<Array<{ key: string, value: any, ttl?: number }>>;
188

189
/**
190
 * Gets the raw JSON string content of a storage file
191
 * @param key - The key to get raw data for
192
 * @returns Promise resolving to raw JSON string
193
 */
194
function getRawDatum(key: string): Promise<string>;
195
```
196

197
**Usage Examples:**
198

199
```javascript
200
// Get complete data structure
201
const allData = await storage.data();
202
console.log(allData);
203
// [
204
//   { key: 'user1', value: { name: 'Alice' }, ttl: 1640995200000 },
205
//   { key: 'config', value: { theme: 'dark' } }
206
// ]
207

208
// Inspect TTL information
209
allData.forEach(datum => {
210
  console.log(`Key: ${datum.key}`);
211
  if (datum.ttl) {
212
    const expiresAt = new Date(datum.ttl);
213
    console.log(`Expires at: ${expiresAt.toISOString()}`);
214
  } else {
215
    console.log('No expiration');
216
  }
217
});
218

219
// Find items expiring soon
220
const expiringSoon = allData.filter(datum => {
221
  if (!datum.ttl) return false;
222
  const now = Date.now();
223
  const timeUntilExpiry = datum.ttl - now;
224
  return timeUntilExpiry > 0 && timeUntilExpiry < 60000; // expires within 1 minute
225
});
226
```
227

228
### Get Raw Datum
229

230
Returns the raw JSON string content of a storage file for debugging or inspection purposes.
231

232
**Usage Examples:**
233

234
```javascript
235
await storage.setItem('user', { name: 'Alice', age: 30 }, { ttl: 60000 });
236

237
// Get raw JSON string
238
const rawData = await storage.getRawDatum('user');
239
console.log(rawData);
240
// '{"key":"user","value":{"name":"Alice","age":30},"ttl":1640995200000}'
241

242
// Parse it manually if needed
243
const parsedData = JSON.parse(rawData);
244
console.log(parsedData); // { key: 'user', value: { name: 'Alice', age: 30 }, ttl: 1640995200000 }
245

246
// Non-existent key returns empty JSON object string
247
const missing = await storage.getRawDatum('nonexistent');
248
console.log(missing); // '{}'
249
```
250

251
## Advanced Query Patterns
252

253
### Complex Filtering
254

255
The filter functions receive the complete datum object with `key`, `value`, and `ttl` properties:
256

257
```javascript
258
// Filter by multiple criteria
259
const complexFilter = (datum) => {
260
  return datum.key.startsWith('user:') && 
261
         datum.value.active === true &&
262
         datum.value.role === 'admin';
263
};
264

265
const adminUsers = await storage.values(complexFilter);
266
const adminUserKeys = await storage.keys(complexFilter);
267
const adminUserCount = await storage.length(complexFilter);
268
```
269

270
### Data Analysis and Reporting
271

272
```javascript
273
// Analyze storage usage
274
const allData = await storage.data();
275

276
const report = {
277
  totalItems: allData.length,
278
  itemsWithTTL: allData.filter(d => d.ttl).length,
279
  itemsWithoutTTL: allData.filter(d => !d.ttl).length,
280
  expiredItems: allData.filter(d => d.ttl && d.ttl < Date.now()).length,
281
  keyPrefixes: {}
282
};
283

284
// Count items by key prefix
285
allData.forEach(datum => {
286
  const prefix = datum.key.split(':')[0];
287
  report.keyPrefixes[prefix] = (report.keyPrefixes[prefix] || 0) + 1;
288
});
289

290
console.log(report);
291
```
292

293
### Working with Large Datasets
294

295
```javascript
296
// Process large datasets in batches
297
const batchSize = 100;
298
let processed = 0;
299

300
await storage.forEach(async (datum) => {
301
  await processItem(datum);
302
  processed++;
303
  
304
  if (processed % batchSize === 0) {
305
    console.log(`Processed ${processed} items`);
306
    // Optional: Add delay to prevent overwhelming the system
307
    await new Promise(resolve => setTimeout(resolve, 10));
308
  }
309
});
310
```
311

312
## Performance Considerations
313

314
- **Filter functions**: Execute on all items in memory after loading from disk
315
- **Large datasets**: Consider using specific key patterns and `valuesWithKeyMatch()` for better performance
316
- **Frequent queries**: Cache results when appropriate, as file system access is required
317
- **Memory usage**: `data()`, `keys()`, and `values()` load all items into memory simultaneously