0
# Data Management
1

2
File handling, caching, and configuration options for managing SRTM data files including custom cache directories, batch processing modes, and network settings.
3

4
## Capabilities
5

6
### File Handler
7

8
Custom file handling system for managing SRTM data file storage, caching, and retrieval with configurable cache directories.
9

10
```python { .api }
11
class FileHandler:
12
    def __init__(self, local_cache_dir: Optional[str] = None): ...
13
    def exists(self, file_name: str) -> bool: ...
14
    def write(self, file_name: str, contents: bytes) -> None: ...
15
    def read(self, file_name: str) -> bytes: ...
16
```
17

18
**Constructor Parameters:**
19
- `local_cache_dir` (Optional[str]): Custom directory for caching SRTM files. Default: None (uses ~/.cache/srtm/)
20

21
**Methods:**
22

23
**exists()**: Check if a file exists in the cache
24
- **Parameters:** `file_name` (str) - Name of the SRTM file to check
25
- **Returns:** bool - True if file exists, False otherwise
26

27
**write()**: Write file contents to the cache
28
- **Parameters:** `file_name` (str), `contents` (bytes)
29
- **Returns:** None
30

31
**read()**: Read file contents from the cache  
32
- **Parameters:** `file_name` (str)
33
- **Returns:** bytes - File contents
34

35
**Usage Example:**
36
```python
37
import srtm
38
from srtm.utils import FileHandler
39

40
# Create custom file handler with specific cache directory
41
custom_handler = FileHandler(local_cache_dir="/data/srtm_cache")
42

43
# Use with elevation data
44
elevation_data = srtm.get_data(file_handler=custom_handler)
45

46
# Check if specific file is cached
47
if custom_handler.exists("N45E007.hgt"):
48
    print("File is already cached locally")
49
else:
50
    print("File will be downloaded on first use")
51

52
# Manual file operations (advanced usage)
53
if custom_handler.exists("N45E007.hgt"):
54
    file_data = custom_handler.read("N45E007.hgt")
55
    print(f"File size: {len(file_data)} bytes")
56
```
57

58
### Cache Directory Configuration
59

60
Configure custom cache directories for different use cases and deployment scenarios.
61

62
**Usage Example:**
63
```python
64
import srtm
65
import os
66

67
# Project-specific cache directory
68
project_cache = os.path.join(os.getcwd(), "elevation_cache")
69
elevation_data = srtm.get_data(local_cache_dir=project_cache)
70

71
# Shared cache directory for multiple applications
72
shared_cache = "/opt/shared/srtm_data"
73
os.makedirs(shared_cache, exist_ok=True)
74
elevation_data = srtm.get_data(local_cache_dir=shared_cache)
75

76
# Temporary cache directory
77
import tempfile
78
temp_cache = tempfile.mkdtemp(prefix="srtm_")
79
elevation_data = srtm.get_data(local_cache_dir=temp_cache)
80
print(f"Using temporary cache: {temp_cache}")
81
```
82

83
### Batch Processing Mode
84

85
Memory-efficient processing mode that keeps only the most recently accessed SRTM file in memory, ideal for processing large datasets or many geographic locations.
86

87
**Usage Example:**
88
```python
89
import srtm
90

91
# Enable batch mode for memory efficiency
92
elevation_data = srtm.get_data(batch_mode=True)
93

94
# Process many coordinates efficiently
95
coordinates = [
96
    (45.0, 7.0), (45.1, 7.1), (45.2, 7.2),
97
    (46.0, 8.0), (46.1, 8.1), (46.2, 8.2),
98
    (47.0, 9.0), (47.1, 9.1), (47.2, 9.2)
99
]
100

101
elevations = []
102
for lat, lon in coordinates:
103
    elevation = elevation_data.get_elevation(lat, lon)
104
    elevations.append(elevation)
105
    print(f"({lat}, {lon}): {elevation}m")
106

107
print(f"Processed {len(coordinates)} coordinates in batch mode")
108
print(f"Memory usage optimized by keeping only current file loaded")
109
```
110

111
### File Compression Options
112

113
Configure whether downloaded SRTM files should be stored in compressed or uncompressed format.
114

115
**Usage Example:**
116
```python
117
import srtm
118

119
# Store files as compressed ZIP files (saves disk space)
120
elevation_data = srtm.get_data(leave_zipped=True)
121

122
# Store files uncompressed (faster access, more disk space)
123
elevation_data = srtm.get_data(leave_zipped=False)
124

125
# Compare storage requirements
126
import os
127
from srtm.utils import FileHandler
128

129
handler = FileHandler()
130
if handler.exists("N45E007.hgt.zip"):
131
    zip_size = len(handler.read("N45E007.hgt.zip"))
132
    print(f"Compressed file size: {zip_size / 1024 / 1024:.1f} MB")
133

134
if handler.exists("N45E007.hgt"):
135
    hgt_size = len(handler.read("N45E007.hgt"))
136
    print(f"Uncompressed file size: {hgt_size / 1024 / 1024:.1f} MB")
137
    print(f"Compression ratio: {zip_size / hgt_size:.2f}")
138
```
139

140
### Network Configuration
141

142
Configure network timeouts and URL sources for downloading SRTM data files.
143

144
**Usage Example:**
145
```python
146
import srtm
147

148
# Configure custom timeout for slow networks
149
elevation_data = srtm.get_data(timeout=60)  # 60 second timeout
150

151
# Disable pre-cached URL list (force fresh URL discovery)
152
elevation_data = srtm.get_data(use_included_urls=False)
153

154
# Combine custom network settings
155
elevation_data = srtm.get_data(
156
    timeout=45,                    # 45 second timeout
157
    use_included_urls=True,        # Use cached URLs for speed
158
    local_cache_dir="./srtm",      # Custom cache location
159
    batch_mode=True                # Memory efficient mode
160
)
161
```
162

163
### Cache Management Utilities
164

165
Utilities for managing and monitoring the SRTM file cache.
166

167
**Usage Example:**
168
```python
169
import srtm
170
import os
171
from srtm.utils import FileHandler
172

173
def analyze_cache(cache_dir=None):
174
    """Analyze SRTM cache directory contents."""
175
    handler = FileHandler(local_cache_dir=cache_dir)
176
    
177
    # Get cache directory path
178
    if cache_dir:
179
        cache_path = cache_dir
180
    else:
181
        cache_path = os.path.expanduser("~/.cache/srtm/")
182
    
183
    if not os.path.exists(cache_path):
184
        print(f"Cache directory does not exist: {cache_path}")
185
        return
186
    
187
    # Count files and calculate sizes
188
    hgt_files = [f for f in os.listdir(cache_path) if f.endswith('.hgt')]
189
    zip_files = [f for f in os.listdir(cache_path) if f.endswith('.hgt.zip')]
190
    
191
    total_size = 0
192
    for filename in hgt_files + zip_files:
193
        filepath = os.path.join(cache_path, filename)
194
        total_size += os.path.getsize(filepath)
195
    
196
    print(f"Cache directory: {cache_path}")
197
    print(f"HGT files: {len(hgt_files)}")
198
    print(f"ZIP files: {len(zip_files)}")
199
    print(f"Total size: {total_size / 1024 / 1024:.1f} MB")
200
    
201
    return {
202
        'cache_path': cache_path,
203
        'hgt_count': len(hgt_files),
204
        'zip_count': len(zip_files),
205
        'total_size_mb': total_size / 1024 / 1024
206
    }
207

208
# Analyze default cache
209
cache_info = analyze_cache()
210

211
# Analyze custom cache
212
custom_info = analyze_cache("/data/custom_srtm")
213
```
214

215
### Advanced File Operations
216

217
Advanced file handling operations for specialized use cases.
218

219
**Usage Example:**
220
```python
221
import srtm
222
from srtm.utils import FileHandler
223
import shutil
224
import os
225

226
def backup_cache(source_cache=None, backup_dir="srtm_backup"):
227
    """Backup SRTM cache directory."""
228
    handler = FileHandler(local_cache_dir=source_cache)
229
    
230
    # Determine source cache directory
231
    if source_cache:
232
        cache_path = source_cache
233
    else:
234
        cache_path = os.path.expanduser("~/.cache/srtm/")
235
    
236
    if not os.path.exists(cache_path):
237
        print(f"Source cache directory does not exist: {cache_path}")
238
        return False
239
    
240
    # Create backup
241
    if os.path.exists(backup_dir):
242
        shutil.rmtree(backup_dir)
243
    
244
    shutil.copytree(cache_path, backup_dir)
245
    print(f"Cache backed up to: {backup_dir}")
246
    return True
247

248
def restore_cache(backup_dir="srtm_backup", target_cache=None):
249
    """Restore SRTM cache from backup."""
250
    if not os.path.exists(backup_dir):
251
        print(f"Backup directory does not exist: {backup_dir}")
252
        return False
253
    
254
    # Determine target cache directory
255
    if target_cache:
256
        cache_path = target_cache
257
    else:
258
        cache_path = os.path.expanduser("~/.cache/srtm/")
259
    
260
    # Remove existing cache and restore from backup
261
    if os.path.exists(cache_path):
262
        shutil.rmtree(cache_path)
263
    
264
    shutil.copytree(backup_dir, cache_path)
265
    print(f"Cache restored from: {backup_dir}")
266
    return True
267

268
# Backup current cache
269
backup_cache()
270

271
# Restore cache (example)
272
# restore_cache("srtm_backup")
273
```
274

275
## Configuration Constants
276

277
Key configuration constants for data management:
278

279
```python { .api }
280
# Default cache location
281
DEFAULT_CACHE_DIR = "~/.cache/srtm/"
282

283
# Network settings
284
DEFAULT_TIMEOUT = 15  # seconds
285
SRTM1_URL = "https://srtm.kurviger.de/SRTM1/"
286
SRTM3_URL = "https://srtm.kurviger.de/SRTM3/"
287

288
# File settings
289
DEFAULT_LIST_JSON = "list.json"  # Pre-cached file URLs
290
```
291

292
## Error Handling
293

294
Common error conditions in data management operations:
295

296
- **Permission Errors**: Unable to write to cache directory
297
- **Disk Space**: Insufficient space for downloading SRTM files  
298
- **Network Errors**: Connection failures, timeouts, or server errors
299
- **File Corruption**: Downloaded files that are invalid or corrupted
300
- **Path Errors**: Invalid cache directory paths or missing parent directories
301
- **Concurrent Access**: Multiple processes accessing the same cache simultaneously
302

303
**Best Practices:**
304
- Always check disk space before bulk downloads
305
- Use appropriate timeouts for your network conditions
306
- Handle permission errors gracefully
307
- Implement retry logic for network operations
308
- Validate downloaded files before caching