0
# Management Commands
1

2
Django management commands for command-line import and export operations, supporting various formats and configuration options.
3

4
## Capabilities
5

6
### Export Command
7

8
Command-line tool for exporting data from Django models or resources.
9

10
```python { .api }
11
class Command(BaseCommand):
12
    help = "Export data from a specified resource or model in a chosen format."
13

14
    def add_arguments(self, parser):
15
        """
16
        Add command-line arguments for export command.
17
        
18
        Arguments:
19
        - format: Export format (CSV, XLSX, JSON, etc.)
20
        - resource: Resource class or model class in dotted path format
21
        - --encoding: Character encoding for text formats
22
        """
23

24
    def handle(self, *args, **options):
25
        """
26
        Handle export command execution.
27
        
28
        Parameters:
29
        - *args: Positional arguments
30
        - **options: Command options from arguments
31
        """
32
```
33

34
### Import Command
35

36
Command-line tool for importing data into Django models from files.
37

38
```python { .api }
39
class Command(BaseCommand):
40
    help = "Import data from a file into a specified resource or model."
41

42
    def add_arguments(self, parser):
43
        """
44
        Add command-line arguments for import command.
45
        
46
        Arguments:
47
        - format: Import format (CSV, XLSX, JSON, etc.)
48
        - resource: Resource class or model class in dotted path format
49
        - file: Path to file to import
50
        - --encoding: Character encoding for text formats
51
        - --dry-run: Perform dry run without saving data
52
        """
53

54
    def handle(self, *args, **options):
55
        """
56
        Handle import command execution.
57
        
58
        Parameters:
59
        - *args: Positional arguments
60
        - **options: Command options from arguments
61
        """
62
```
63

64
## Command Usage
65

66
### Export Command Syntax
67

68
```bash
69
python manage.py export <format> <resource> [--encoding=<encoding>]
70
```
71

72
**Parameters:**
73
- `format`: Export format name (CSV, XLSX, JSON, YAML, etc.) or custom format class path
74
- `resource`: Resource class or model class in dotted path format
75
- `--encoding`: Optional character encoding for text formats (default: utf-8)
76

77
### Import Command Syntax
78

79
```bash
80
python manage.py import <format> <resource> <file> [--encoding=<encoding>] [--dry-run]
81
```
82

83
**Parameters:**
84
- `format`: Import format name (CSV, XLSX, JSON, YAML, etc.) or custom format class path
85
- `resource`: Resource class or model class in dotted path format
86
- `file`: Path to file to import
87
- `--encoding`: Optional character encoding for text formats (default: utf-8)
88
- `--dry-run`: Perform validation without saving data
89

90
## Usage Examples
91

92
### Basic Export Examples
93

94
```bash
95
# Export books to CSV format
96
python manage.py export CSV myapp.models.Book
97

98
# Export using custom resource
99
python manage.py export CSV myapp.resources.BookResource
100

101
# Export to Excel format
102
python manage.py export XLSX myapp.models.Book
103

104
# Export to JSON format
105
python manage.py export JSON myapp.models.Book
106

107
# Export with specific encoding
108
python manage.py export CSV myapp.models.Book --encoding=latin-1
109
```
110

111
### Basic Import Examples
112

113
```bash
114
# Import books from CSV file
115
python manage.py import CSV myapp.models.Book books.csv
116

117
# Import using custom resource
118
python manage.py import CSV myapp.resources.BookResource books.csv
119

120
# Import from Excel file
121
python manage.py import XLSX myapp.models.Book books.xlsx
122

123
# Import from JSON file
124
python manage.py import JSON myapp.models.Book books.json
125

126
# Dry run import (validation only)
127
python manage.py import CSV myapp.models.Book books.csv --dry-run
128

129
# Import with specific encoding
130
python manage.py import CSV myapp.models.Book books.csv --encoding=utf-8
131
```
132

133
### Advanced Usage Patterns
134

135
```bash
136
# Export with custom format class
137
python manage.py export myproject.formats.CustomCSV myapp.models.Book
138

139
# Import large dataset
140
python manage.py import CSV myapp.resources.BulkBookResource large_books.csv
141

142
# Import with error handling
143
python manage.py import CSV myapp.models.Book books.csv --dry-run > validation_report.txt
144

145
# Chain commands for processing
146
python manage.py export CSV myapp.models.Book > temp_export.csv
147
python manage.py import CSV myapp.models.Book temp_export.csv --dry-run
148
```
149

150
### Scripted Operations
151

152
```bash
153
#!/bin/bash
154
# Export and backup script
155

156
DATE=$(date +%Y%m%d)
157
BACKUP_DIR="/backups/django_exports/$DATE"
158
mkdir -p "$BACKUP_DIR"
159

160
# Export all models
161
python manage.py export CSV myapp.models.Book > "$BACKUP_DIR/books.csv"
162
python manage.py export CSV myapp.models.Author > "$BACKUP_DIR/authors.csv"
163
python manage.py export CSV myapp.models.Publisher > "$BACKUP_DIR/publishers.csv"
164

165
echo "Export completed to $BACKUP_DIR"
166
```
167

168
```bash
169
#!/bin/bash
170
# Import validation script
171

172
IMPORT_FILE="$1"
173
RESOURCE="$2"
174

175
if [ -z "$IMPORT_FILE" ] || [ -z "$RESOURCE" ]; then
176
    echo "Usage: $0 <import_file> <resource>"
177
    exit 1
178
fi
179

180
echo "Validating import file: $IMPORT_FILE"
181
python manage.py import CSV "$RESOURCE" "$IMPORT_FILE" --dry-run
182

183
if [ $? -eq 0 ]; then
184
    echo "Validation passed. Proceeding with import..."
185
    python manage.py import CSV "$RESOURCE" "$IMPORT_FILE"
186
else
187
    echo "Validation failed. Import aborted."
188
    exit 1
189
fi
190
```
191

192
## Custom Management Commands
193

194
### Extended Export Command
195

196
```python
197
from django.core.management.base import BaseCommand, CommandError
198
from import_export.command_utils import get_resource_class, get_format_class
199
import sys
200
import os
201

202
class Command(BaseCommand):
203
    help = 'Enhanced export command with additional options'
204

205
    def add_arguments(self, parser):
206
        parser.add_argument('format', help='Export format')
207
        parser.add_argument('resource', help='Resource or model class')
208
        parser.add_argument(
209
            '--output', '-o',
210
            help='Output file path (default: stdout)'
211
        )
212
        parser.add_argument(
213
            '--encoding',
214
            default='utf-8',
215
            help='Character encoding (default: utf-8)'
216
        )
217
        parser.add_argument(
218
            '--limit',
219
            type=int,
220
            help='Limit number of records to export'
221
        )
222
        parser.add_argument(
223
            '--filter',
224
            help='Django filter expression (e.g., "published=True")'
225
        )
226
        parser.add_argument(
227
            '--fields',
228
            help='Comma-separated list of fields to export'
229
        )
230

231
    def handle(self, *args, **options):
232
        try:
233
            # Get resource and format
234
            resource_class = get_resource_class(options['resource'])
235
            format_class = get_format_class(options['format'])
236
            
237
            # Initialize resource
238
            resource = resource_class()
239
            
240
            # Get queryset
241
            if hasattr(resource, 'get_queryset'):
242
                queryset = resource.get_queryset()
243
                
244
                # Apply filters
245
                if options['filter']:
246
                    filter_kwargs = {}
247
                    for filter_expr in options['filter'].split(','):
248
                        key, value = filter_expr.split('=')
249
                        filter_kwargs[key.strip()] = value.strip()
250
                    queryset = queryset.filter(**filter_kwargs)
251
                
252
                # Apply limit
253
                if options['limit']:
254
                    queryset = queryset[:options['limit']]
255
            else:
256
                queryset = None
257
            
258
            # Export data
259
            dataset = resource.export(queryset)
260
            
261
            # Filter fields if specified
262
            if options['fields']:
263
                field_names = [f.strip() for f in options['fields'].split(',')]
264
                # Filter dataset to only include specified fields
265
                headers = dataset.headers
266
                field_indices = [headers.index(f) for f in field_names if f in headers]
267
                
268
                filtered_data = []
269
                for row in dataset:
270
                    filtered_row = [row[i] for i in field_indices]
271
                    filtered_data.append(filtered_row)
272
                
273
                dataset.headers = [headers[i] for i in field_indices]
274
                dataset._dataset = filtered_data
275
            
276
            # Export to format
277
            export_data = format_class.export_data(dataset)
278
            
279
            # Output data
280
            if options['output']:
281
                mode = 'wb' if format_class.is_binary() else 'w'
282
                encoding = None if format_class.is_binary() else options['encoding']
283
                
284
                with open(options['output'], mode, encoding=encoding) as f:
285
                    f.write(export_data)
286
                
287
                self.stdout.write(
288
                    self.style.SUCCESS(f'Exported to {options["output"]}')
289
                )
290
            else:
291
                if format_class.is_binary():
292
                    sys.stdout.buffer.write(export_data)
293
                else:
294
                    self.stdout.write(export_data)
295
                    
296
        except Exception as e:
297
            raise CommandError(f'Export failed: {e}')
298
```
299

300
### Enhanced Import Command
301

302
```python
303
from django.core.management.base import BaseCommand, CommandError
304
from import_export.command_utils import get_resource_class, get_format_class
305
from import_export.results import Result
306
import os
307

308
class Command(BaseCommand):
309
    help = 'Enhanced import command with detailed reporting'
310

311
    def add_arguments(self, parser):
312
        parser.add_argument('format', help='Import format')
313
        parser.add_argument('resource', help='Resource or model class')
314
        parser.add_argument('file', help='File to import')
315
        parser.add_argument(
316
            '--encoding',
317
            default='utf-8',
318
            help='Character encoding (default: utf-8)'
319
        )
320
        parser.add_argument(
321
            '--dry-run',
322
            action='store_true',
323
            help='Perform dry run without saving'
324
        )
325
        parser.add_argument(
326
            '--batch-size',
327
            type=int,
328
            default=1000,
329
            help='Batch size for bulk operations'
330
        )
331
        parser.add_argument(
332
            '--skip-errors',
333
            action='store_true',
334
            help='Skip rows with errors and continue'
335
        )
336
        parser.add_argument(
337
            '--report',
338
            help='Path to save detailed import report'
339
        )
340

341
    def handle(self, *args, **options):
342
        try:
343
            # Validate file exists
344
            if not os.path.exists(options['file']):
345
                raise CommandError(f'File not found: {options["file"]}')
346
            
347
            # Get resource and format
348
            resource_class = get_resource_class(options['resource'])
349
            format_class = get_format_class(options['format'])
350
            
351
            # Initialize resource
352
            resource = resource_class()
353
            
354
            # Read and parse file
355
            mode = 'rb' if format_class.is_binary() else 'r'
356
            encoding = None if format_class.is_binary() else options['encoding']
357
            
358
            with open(options['file'], mode, encoding=encoding) as f:
359
                content = f.read()
360
                dataset = format_class.create_dataset(content)
361
            
362
            self.stdout.write(f'Loaded {len(dataset)} rows from {options["file"]}')
363
            
364
            # Perform import
365
            result = resource.import_data(
366
                dataset,
367
                dry_run=options['dry_run'],
368
                raise_errors=not options['skip_errors'],
369
                use_transactions=True,
370
                collect_failed_rows=True,
371
            )
372
            
373
            # Report results
374
            self.report_results(result, options)
375
            
376
            # Save detailed report if requested
377
            if options['report']:
378
                self.save_report(result, options['report'])
379
                
380
        except Exception as e:
381
            raise CommandError(f'Import failed: {e}')
382

383
    def report_results(self, result, options):
384
        """Print import results summary."""
385
        
386
        if options['dry_run']:
387
            self.stdout.write(self.style.WARNING('DRY RUN - No data was saved'))
388
        
389
        self.stdout.write(f'Total rows processed: {result.total_rows}')
390
        self.stdout.write(f'Valid rows: {len(result.valid_rows())}')
391
        
392
        if result.has_errors():
393
            self.stdout.write(
394
                self.style.ERROR(f'Errors: {len(result.base_errors)}')
395
            )
396
            for error in result.base_errors[:5]:  # Show first 5 errors
397
                self.stdout.write(f'  - {error}')
398
            
399
            if len(result.base_errors) > 5:
400
                self.stdout.write(f'  ... and {len(result.base_errors) - 5} more errors')
401
        
402
        if result.has_validation_errors():
403
            self.stdout.write(
404
                self.style.ERROR(f'Validation errors: {len(result.invalid_rows)}')
405
            )
406
        
407
        # Show summary by import type
408
        totals = result.totals
409
        if totals:
410
            self.stdout.write('Import summary:')
411
            for import_type, count in totals.items():
412
                if count > 0:
413
                    self.stdout.write(f'  {import_type}: {count}')
414

415
    def save_report(self, result, report_path):
416
        """Save detailed import report to file."""
417
        
418
        with open(report_path, 'w') as f:
419
            f.write('Django Import Export - Detailed Report\n')
420
            f.write('=' * 50 + '\n\n')
421
            
422
            f.write(f'Total rows: {result.total_rows}\n')
423
            f.write(f'Valid rows: {len(result.valid_rows())}\n')
424
            f.write(f'Error rows: {len(result.base_errors)}\n')
425
            f.write(f'Invalid rows: {len(result.invalid_rows)}\n\n')
426
            
427
            # Detailed errors
428
            if result.base_errors:
429
                f.write('ERRORS:\n')
430
                f.write('-' * 20 + '\n')
431
                for i, error in enumerate(result.base_errors, 1):
432
                    f.write(f'{i}. {error}\n')
433
                f.write('\n')
434
            
435
            # Validation errors
436
            if result.invalid_rows:
437
                f.write('VALIDATION ERRORS:\n')
438
                f.write('-' * 20 + '\n')
439
                for i, invalid_row in enumerate(result.invalid_rows, 1):
440
                    f.write(f'{i}. Row {invalid_row.number}: {invalid_row.error}\n')
441
                f.write('\n')
442
            
443
            # Success summary
444
            if result.totals:
445
                f.write('SUMMARY:\n')
446
                f.write('-' * 20 + '\n')
447
                for import_type, count in result.totals.items():
448
                    f.write(f'{import_type}: {count}\n')
449
        
450
        self.stdout.write(f'Detailed report saved to: {report_path}')
451
```
452

453
### Batch Processing Command
454

455
```python
456
class Command(BaseCommand):
457
    help = 'Batch process multiple import files'
458

459
    def add_arguments(self, parser):
460
        parser.add_argument('format', help='Import format')
461
        parser.add_argument('resource', help='Resource or model class')
462
        parser.add_argument('directory', help='Directory containing files to import')
463
        parser.add_argument(
464
            '--pattern',
465
            default='*',
466
            help='File pattern to match (default: *)'
467
        )
468
        parser.add_argument(
469
            '--dry-run',
470
            action='store_true',
471
            help='Perform dry run without saving'
472
        )
473

474
    def handle(self, *args, **options):
475
        import glob
476
        
477
        # Find files to process
478
        pattern = os.path.join(options['directory'], options['pattern'])
479
        files = glob.glob(pattern)
480
        
481
        if not files:
482
            self.stdout.write(f'No files found matching pattern: {pattern}')
483
            return
484
        
485
        self.stdout.write(f'Found {len(files)} files to process')
486
        
487
        # Process each file
488
        total_processed = 0
489
        total_errors = 0
490
        
491
        for file_path in files:
492
            self.stdout.write(f'\nProcessing: {os.path.basename(file_path)}')
493
            
494
            try:
495
                # Import file using the enhanced import logic
496
                result = self.import_file(file_path, options)
497
                
498
                total_processed += result.total_rows
499
                if result.has_errors():
500
                    total_errors += len(result.base_errors)
501
                    
502
                self.stdout.write(
503
                    f'  Processed {result.total_rows} rows, '
504
                    f'{len(result.base_errors)} errors'
505
                )
506
                
507
            except Exception as e:
508
                self.stdout.write(
509
                    self.style.ERROR(f'  Failed to process file: {e}')
510
                )
511
        
512
        # Final summary
513
        self.stdout.write(f'\nBatch processing complete:')
514
        self.stdout.write(f'  Files processed: {len(files)}')
515
        self.stdout.write(f'  Total rows: {total_processed}')
516
        self.stdout.write(f'  Total errors: {total_errors}')
517

518
    def import_file(self, file_path, options):
519
        """Import a single file and return results."""
520
        
521
        resource_class = get_resource_class(options['resource'])
522
        format_class = get_format_class(options['format'])
523
        
524
        resource = resource_class()
525
        
526
        with open(file_path, 'r') as f:
527
            dataset = format_class.create_dataset(f.read())
528
        
529
        return resource.import_data(
530
            dataset,
531
            dry_run=options['dry_run'],
532
            raise_errors=False,
533
        )
534
```
535

536
## Integration with CI/CD
537

538
### Docker Integration
539

540
```dockerfile
541
# Dockerfile for import/export operations
542
FROM python:3.9
543
WORKDIR /app
544
COPY requirements.txt .
545
RUN pip install -r requirements.txt
546
COPY . .
547

548
# Command for data export
549
CMD ["python", "manage.py", "export", "CSV", "myapp.models.Book"]
550
```
551

552
### GitHub Actions Workflow
553

554
```yaml
555
name: Data Import/Export
556

557
on:
558
  schedule:
559
    - cron: '0 2 * * *'  # Daily at 2 AM
560
  workflow_dispatch:
561

562
jobs:
563
  export-data:
564
    runs-on: ubuntu-latest
565
    steps:
566
      - uses: actions/checkout@v2
567
      
568
      - name: Set up Python
569
        uses: actions/setup-python@v2
570
        with:
571
          python-version: 3.9
572
          
573
      - name: Install dependencies
574
        run: |
575
          pip install -r requirements.txt
576
          
577
      - name: Export data
578
        run: |
579
          python manage.py export CSV myapp.models.Book > books_export.csv
580
          python manage.py export CSV myapp.models.Author > authors_export.csv
581
          
582
      - name: Upload exports
583
        uses: actions/upload-artifact@v2
584
        with:
585
          name: data-exports
586
          path: |
587
            books_export.csv
588
            authors_export.csv
589
```
590

591
### Monitoring and Logging
592

593
```python
594
import logging
595
from django.core.management.base import BaseCommand
596

597
logger = logging.getLogger('import_export.commands')
598

599
class Command(BaseCommand):
600
    def handle(self, *args, **options):
601
        logger.info(f'Starting export: {options}')
602
        
603
        try:
604
            # Export logic here
605
            result = self.perform_export(options)
606
            logger.info(f'Export completed successfully: {result}')
607
            
608
        except Exception as e:
609
            logger.error(f'Export failed: {e}', exc_info=True)
610
            raise
611
```