0
# Archive Management
1

2
Archive creation and extraction functionality with support for compressed tar.gz format and automatic timestamping. The archive management system packages backup files into portable archives for storage and distribution.
3

4
## Capabilities
5

6
### Archive Creation
7

8
Creates compressed tar.gz archives from backup directory structures with automatic timestamping and optional cleanup.
9

10
```python { .api }
11
def main(args, settings):
12
    """
13
    Create compressed archive from backup directory
14
    
15
    Module: grafana_backup.archive
16
    Args:
17
        args (dict): Command line arguments including --no-archive flag
18
        settings (dict): Configuration settings with backup directory and timestamp
19
        
20
    Features: Gzip compression, timestamped filenames, automatic cleanup
21
    Output: backup_{timestamp}.tar.gz in backup directory
22
    """
23
```
24

25
## Archive Creation Process
26

27
### Automatic Archive Creation
28

29
Archive creation is automatically triggered after successful backup completion unless disabled:
30

31
```python
32
# Archive creation workflow
33
from grafana_backup.save import main as save_backup
34

35
# Normal backup process includes automatic archiving
36
save_args = {
37
    'save': True,
38
    '--components': None,
39
    '--no-archive': False,  # Archive will be created
40
    '--config': None
41
}
42
save_backup(save_args, settings)
43

44
# Skip archive creation for troubleshooting
45
save_args['--no-archive'] = True  # Archive creation skipped
46
save_backup(save_args, settings)
47
```
48

49
### Archive File Structure
50

51
Archives contain the complete backup directory structure:
52

53
```
54
backup_{timestamp}.tar.gz
55
├── dashboards/
56
│   └── {timestamp}/
57
│       ├── dashboard1.json
58
│       ├── dashboard2.json
59
│       └── ...
60
├── datasources/
61
│   └── {timestamp}/
62
│       ├── datasource1.json
63
│       ├── datasource2.json
64
│       └── ...
65
├── folders/
66
│   └── {timestamp}/
67
│       ├── folder1.json
68
│       └── ...
69
└── [other components]/
70
    └── {timestamp}/
71
        └── ...
72
```
73

74
### Archive Naming Convention
75

76
Archive files use consistent timestamped naming:
77

78
```python { .api }
79
# Archive filename format
80
filename_pattern = "backup_{timestamp}.tar.gz"
81

82
# Example with default timestamp format (%Y%m%d%H%M)
83
example_filename = "backup_202501011200.tar.gz"
84

85
# Customizable via BACKUP_FILE_FORMAT setting
86
custom_format = "%Y-%m-%d_%H-%M-%S"
87
custom_filename = "backup_2025-01-01_12-00-00.tar.gz"
88
```
89

90
## Archive Extraction
91

92
### Automatic Extraction During Restore
93

94
Archive extraction is automatically handled during restore operations:
95

96
```python
97
# Restore process automatically handles extraction
98
from grafana_backup.restore import main as restore_backup
99

100
restore_args = {
101
    'restore': True,
102
    '<archive_file>': 'backup_202501011200.tar.gz',
103
    '--components': None,
104
    '--config': None
105
}
106

107
# Extraction process:
108
# 1. Validates archive format and integrity
109
# 2. Creates temporary directory for extraction
110
# 3. Extracts archive contents
111
# 4. Processes component files
112
# 5. Cleans up temporary directory
113
restore_backup(restore_args, settings)
114
```
115

116
### Manual Archive Extraction
117

118
Archives can be manually extracted using standard tools:
119

120
```bash
121
# Extract archive manually
122
tar -xzf backup_202501011200.tar.gz
123

124
# List archive contents
125
tar -tzf backup_202501011200.tar.gz
126

127
# Extract specific components
128
tar -xzf backup_202501011200.tar.gz dashboards/
129
```
130

131
## Configuration Options
132

133
### Archive Control Settings
134

135
```python { .api }
136
# Configuration settings affecting archive operations
137
BACKUP_DIR: str          # Directory for backup files and archives
138
BACKUP_FILE_FORMAT: str  # Timestamp format for archive naming
139
TIMESTAMP: str           # Current timestamp using BACKUP_FILE_FORMAT
140
```
141

142
### Archive Creation Behavior
143

144
Archive creation can be controlled through command-line arguments:
145

146
```python
147
# Control archive creation
148
args = {
149
    '--no-archive': False  # Create archive (default)
150
}
151

152
args = {
153
    '--no-archive': True   # Skip archive creation
154
}
155
```
156

157
## Usage Examples
158

159
### Standard Archive Creation
160

161
```python
162
from grafana_backup.archive import main as create_archive
163
from grafana_backup.grafanaSettings import main as load_config
164

165
# Load configuration
166
settings = load_config('/path/to/grafanaSettings.json')
167

168
# Create archive from existing backup
169
args = {
170
    '--no-archive': False,
171
    '--config': None
172
}
173

174
create_archive(args, settings)
175
```
176

177
### Integrated Backup with Archive
178

179
```python
180
from grafana_backup.save import main as save_backup
181

182
# Backup with automatic archive creation
183
save_args = {
184
    'save': True,
185
    '--components': 'dashboards,datasources',
186
    '--no-archive': False,  # Archive will be created
187
    '--config': None
188
}
189

190
save_backup(save_args, settings)
191
# Result: Individual JSON files + backup_{timestamp}.tar.gz
192
```
193

194
### Backup Without Archive (Troubleshooting)
195

196
```python
197
# Backup without archive for troubleshooting
198
save_args = {
199
    'save': True,
200
    '--components': 'dashboards',
201
    '--no-archive': True,   # Skip archive creation
202
    '--config': None
203
}
204

205
save_backup(save_args, settings)
206
# Result: Only individual JSON files, no archive
207
```
208

209
## Archive Format Details
210

211
### Compression Algorithm
212

213
Archives use gzip compression for optimal balance of size and compatibility:
214

215
- **Format**: tar.gz (tar archive with gzip compression)
216
- **Compression level**: Default gzip compression level
217
- **Compatibility**: Standard format supported by all major platforms
218

219
### Archive Structure Preservation
220

221
The archive maintains the complete backup directory structure:
222

223
- **Timestamps**: All timestamped directories are preserved
224
- **File hierarchy**: Folder structure exactly matches backup directory
225
- **Metadata**: File modification times and permissions are preserved
226
- **Completeness**: All backup files are included without filtering
227

228
### Archive Integrity
229

230
Archive creation includes integrity validation:
231

232
- **Compression verification**: Ensures archive can be properly decompressed
233
- **Content validation**: Verifies all expected files are included
234
- **Size validation**: Confirms archive size is reasonable for content
235

236
## Error Handling
237

238
### Archive Creation Errors
239

240
Comprehensive error handling for archive creation:
241

242
```python
243
# Common archive creation errors and handling:
244
# - Insufficient disk space: Clear error message with space requirements
245
# - Permission issues: Detailed error about directory access
246
# - Compression failures: Fallback to uncompressed tar if needed
247
# - File access errors: Skip problematic files with warnings
248
```
249

250
### Archive Extraction Errors
251

252
Robust error handling for archive extraction:
253

254
```python
255
# Common extraction errors and handling:
256
# - Corrupted archives: Clear error message with integrity check results
257
# - Insufficient space: Space requirements and available space reporting
258
# - Permission issues: Temporary directory creation and access validation
259
# - Format errors: Validation of archive format before extraction
260
```
261

262
## Performance Considerations
263

264
### Archive Creation Performance
265

266
Archive creation is optimized for performance:
267

268
- **Streaming compression**: Files are compressed during archive creation
269
- **Memory efficiency**: Large files are processed in chunks
270
- **I/O optimization**: Minimizes disk I/O through efficient buffering
271
- **Progress reporting**: Provides feedback for large archive operations
272

273
### Archive Size Optimization
274

275
Archives are optimized for size while maintaining data integrity:
276

277
- **JSON compression**: JSON files compress well with gzip
278
- **Duplicate elimination**: File deduplication where appropriate
279
- **Structure preservation**: Maintains necessary directory structure
280
- **Metadata retention**: Preserves essential file metadata
281

282
## Integration with Cloud Storage
283

284
Archives are designed for cloud storage integration:
285

286
### Upload Integration
287
```python
288
# Archive creation automatically triggers cloud uploads
289
# if cloud storage is configured
290
save_backup(save_args, settings)
291
# 1. Creates individual backup files
292
# 2. Creates compressed archive
293
# 3. Uploads archive to configured cloud storage
294
# 4. Optionally cleans up local files
295
```
296

297
### Download Integration
298
```python
299
# Cloud downloads provide archives ready for extraction
300
restore_backup(restore_args, settings)
301
# 1. Downloads archive from cloud storage
302
# 2. Extracts archive to temporary directory
303
# 3. Processes extracted component files
304
# 4. Cleans up temporary files
305
```
306

307
## Best Practices
308

309
### Archive Management
310

311
- **Regular cleanup**: Implement retention policies for local archives
312
- **Storage monitoring**: Monitor disk space usage for archive directory
313
- **Backup verification**: Periodically test archive integrity
314
- **Documentation**: Maintain records of archive contents and purposes
315

316
### Performance Optimization
317

318
- **Disk space**: Ensure sufficient space for both backup files and archives
319
- **I/O performance**: Use fast storage for backup directory during operations
320
- **Cleanup timing**: Balance between storage usage and troubleshooting needs
321
- **Compression trade-offs**: Consider compression level vs. creation time for large backups
322

323
### Troubleshooting
324

325
- **Use --no-archive**: Disable archiving when troubleshooting backup issues
326
- **Manual extraction**: Use standard tar tools for archive inspection
327
- **Temporary retention**: Keep uncompressed files temporarily for validation
328
- **Error logging**: Review detailed logs for archive operation issues
329

330
The archive management system provides reliable, portable backup packaging with comprehensive error handling and performance optimization.