0
# File Operations
1

2
File transfer capabilities and advanced file management functions for uploading, downloading, template rendering, text processing, and project synchronization. These operations provide comprehensive file handling capabilities for deployment and configuration management workflows.
3

4
## Capabilities
5

6
### Basic File Transfer
7

8
Core functions for uploading and downloading files between local and remote systems with comprehensive options for permissions, ownership, and error handling.
9

10
```python { .api }
11
def put(local_path=None, remote_path=None, use_sudo=False, mirror_local_mode=False, mode=None, use_glob=True, temp_dir=""):
12
    """
13
    Upload files to remote host.
14
    
15
    Args:
16
        local_path (str): Local file/directory path to upload
17
        remote_path (str): Remote destination path
18
        use_sudo (bool): Use sudo for file operations (default: False)
19
        mirror_local_mode (bool): Copy local file permissions (default: False)  
20
        mode (str|int): Set specific file permissions (octal or symbolic)
21
        use_glob (bool): Allow glob patterns in local_path (default: True)
22
        temp_dir (str): Temporary directory for staged uploads
23
        
24
    Returns:
25
        _AttributeList: List of uploaded file paths with .failed/.succeeded attributes
26
    """
27

28
def get(remote_path, local_path=None, use_sudo=False, temp_dir=""):
29
    """
30
    Download files from remote host.
31
    
32
    Args:
33
        remote_path (str): Remote file/directory path to download
34
        local_path (str): Local destination path (default: same as remote filename)
35
        use_sudo (bool): Use sudo for file access (default: False)
36
        temp_dir (str): Temporary directory for staged downloads
37
        
38
    Returns:
39
        _AttributeList: List of downloaded file paths with .failed/.succeeded attributes
40
    """
41
```
42

43
**Usage Examples:**
44

45
```python
46
from fabric.api import put, get, run, cd
47

48
# Basic file upload
49
put('local-config.json', '/etc/myapp/config.json')
50

51
# Upload with sudo and specific permissions
52
put('nginx.conf', '/etc/nginx/sites-available/myapp', 
53
    use_sudo=True, mode='644')
54

55
# Upload directory with glob patterns
56
put('dist/*', '/var/www/myapp/', use_glob=True)
57

58
# Mirror local permissions
59
put('script.sh', '/opt/scripts/deploy.sh', 
60
    mirror_local_mode=True, use_sudo=True)
61

62
# Basic file download
63
get('/var/log/nginx/error.log')
64

65
# Download to specific location
66
get('/etc/nginx/nginx.conf', 'backup/nginx.conf')
67

68
# Download with sudo for protected files
69
get('/var/log/auth.log', use_sudo=True)
70

71
# Check transfer results
72
results = put('app.tar.gz', '/tmp/')
73
if results.succeeded:
74
    print(f"Uploaded {len(results)} files successfully")
75
for failed_file in results.failed:
76
    print(f"Failed to upload: {failed_file}")
77
```
78

79
### Advanced File Management
80

81
Extended file operations from the `fabric.contrib.files` module providing file existence checking, text processing, and template rendering capabilities.
82

83
```python { .api }
84
def exists(path, use_sudo=False, verbose=False):
85
    """
86
    Check if remote path exists.
87
    
88
    Args:
89
        path (str): Remote path to check
90
        use_sudo (bool): Use sudo for access check (default: False)
91
        verbose (bool): Show command output (default: False)
92
        
93
    Returns:
94
        bool: True if path exists
95
    """
96

97
def is_link(path, use_sudo=False, verbose=False):
98
    """
99
    Check if remote path is a symbolic link.
100
    
101
    Args:
102
        path (str): Remote path to check
103
        use_sudo (bool): Use sudo for access (default: False)
104
        verbose (bool): Show command output (default: False)
105
        
106
    Returns:
107
        bool: True if path is a symbolic link
108
    """
109

110
def first(*args, **kwargs):
111
    """
112
    Return first existing path from arguments.
113
    
114
    Args:
115
        *args: Paths to check in order
116
        **kwargs: Keyword arguments passed to exists()
117
        
118
    Returns:
119
        str: First existing path or None
120
    """
121
```
122

123
**Usage Examples:**
124

125
```python
126
from fabric.contrib.files import exists, is_link, first
127
from fabric.api import run, sudo
128

129
# Check if file exists before operations
130
if exists('/etc/nginx/nginx.conf'):
131
    run('nginx -t')  # Test configuration
132
else:
133
    print("Nginx config not found")
134

135
# Check with sudo for protected paths
136
if exists('/var/log/secure', use_sudo=True):
137
    sudo('tail /var/log/secure')
138

139
# Check if path is symbolic link
140
if is_link('/etc/nginx/sites-enabled/default'):
141
    run('readlink /etc/nginx/sites-enabled/default')
142

143
# Find first existing configuration file
144
config_path = first(
145
    '/etc/myapp/config.json',
146
    '/opt/myapp/config.json', 
147
    '/usr/local/etc/myapp.conf'
148
)
149
if config_path:
150
    print(f"Found config at: {config_path}")
151

152
# Conditional operations based on existence
153
if not exists('/var/www/myapp'):
154
    sudo('mkdir -p /var/www/myapp')
155
    sudo('chown www-data:www-data /var/www/myapp')
156
```
157

158
### Template Processing
159

160
Upload and render template files with variable substitution using Jinja2 or simple string formatting.
161

162
```python { .api }
163
def upload_template(filename, destination, context=None, use_jinja=False, template_dir=None, use_sudo=False, backup=True, mirror_local_mode=False, mode=None, pty=None):
164
    """
165
    Upload rendered template file to remote host.
166
    
167
    Args:
168
        filename (str): Template filename (relative to template_dir)
169
        destination (str): Remote destination path
170
        context (dict): Variables for template rendering
171
        use_jinja (bool): Use Jinja2 instead of string formatting (default: False)
172
        template_dir (str): Directory containing templates (default: 'templates/')
173
        use_sudo (bool): Use sudo for upload (default: False)
174
        backup (bool): Create backup of existing file (default: True)
175
        mirror_local_mode (bool): Mirror local file permissions (default: False)
176
        mode (str|int): Set specific file permissions
177
        pty (bool): Request pseudo-terminal for command execution
178
        
179
    Returns:
180
        _AttributeList: Upload results
181
    """
182
```
183

184
**Usage Examples:**
185

186
```python
187
from fabric.contrib.files import upload_template
188
from fabric.api import env
189

190
# Simple template upload with string formatting
191
context = {
192
    'server_name': 'myapp.example.com',
193
    'document_root': '/var/www/myapp',
194
    'port': 80
195
}
196
upload_template('nginx.conf.template', '/etc/nginx/sites-available/myapp',
197
                context=context, use_sudo=True, backup=True)
198

199
# Jinja2 template with complex logic
200
jinja_context = {
201
    'app_name': 'myapp',
202
    'servers': ['web1.example.com', 'web2.example.com'],
203
    'ssl_enabled': True,
204
    'environment': env.environment
205
}
206
upload_template('app.conf.j2', '/etc/supervisor/conf.d/myapp.conf',
207
                context=jinja_context, use_jinja=True, use_sudo=True)
208

209
# Template from custom directory
210
upload_template('database.yml', '/opt/myapp/config/database.yml',
211
                context={'db_host': env.db_host, 'db_pass': env.db_password},
212
                template_dir='deployment/templates/', mode='600')
213

214
# Example template content (nginx.conf.template):
215
# server {
216
#     listen %(port)s;
217
#     server_name %(server_name)s;
218
#     root %(document_root)s;
219
#     
220
#     location / {
221
#         try_files $uri $uri/ =404;
222
#     }
223
# }
224
```
225

226
### Text Processing
227

228
In-place text file editing functions for search and replace, commenting, and content manipulation.
229

230
```python { .api }
231
def sed(filename, before, after, limit='', use_sudo=False, backup='.bak', flags='', shell=False):
232
    """
233
    Search and replace text in files using sed.
234
    
235
    Args:
236
        filename (str): File to modify
237
        before (str): Pattern to search for (regex)
238
        after (str): Replacement text
239
        limit (str): Limit to specific line numbers
240
        use_sudo (bool): Use sudo for file access (default: False)
241
        backup (str): Backup file extension (default: '.bak')
242
        flags (str): Additional sed flags
243
        shell (bool): Use shell for command execution (default: False)
244
    """
245

246
def comment(filename, regex, use_sudo=False, char='#', backup='.bak', shell=False):
247
    """
248
    Comment lines matching regex pattern.
249
    
250
    Args:
251
        filename (str): File to modify
252
        regex (str): Pattern to match for commenting
253
        use_sudo (bool): Use sudo for file access (default: False)
254
        char (str): Comment character (default: '#')
255
        backup (str): Backup file extension (default: '.bak')
256
        shell (bool): Use shell for command execution (default: False)
257
    """
258

259
def uncomment(filename, regex, use_sudo=False, char='#', backup='.bak', shell=False):
260
    """
261
    Uncomment lines matching regex pattern.
262
    
263
    Args:
264
        filename (str): File to modify
265
        regex (str): Pattern to match for uncommenting
266
        use_sudo (bool): Use sudo for file access (default: False)
267
        char (str): Comment character to remove (default: '#')
268
        backup (str): Backup file extension (default: '.bak')
269
        shell (bool): Use shell for command execution (default: False)
270
    """
271
```
272

273
**Usage Examples:**
274

275
```python
276
from fabric.contrib.files import sed, comment, uncomment
277
from fabric.api import sudo
278

279
# Replace configuration values
280
sed('/etc/nginx/nginx.conf', 
281
    'worker_processes auto;', 
282
    'worker_processes 4;', 
283
    use_sudo=True)
284

285
# Update database connection string
286
sed('/opt/myapp/config.py',
287
    r'DATABASE_URL = .*',
288
    r'DATABASE_URL = "postgresql://user:pass@localhost/myapp"',
289
    use_sudo=True, backup='.backup')
290

291
# Comment out debug settings
292
comment('/etc/ssh/sshd_config', 
293
        r'^PasswordAuthentication yes',
294
        use_sudo=True)
295

296
# Uncomment production settings  
297
uncomment('/etc/php/7.4/apache2/php.ini',
298
          r'^;opcache.enable=1',
299
          char=';', use_sudo=True)
300

301
# Multiple replacements with sed flags
302
sed('/var/log/app.log',
303
    r'ERROR',
304
    'WARNING',
305
    flags='g',  # Global replacement
306
    backup='')  # No backup
307
```
308

309
### Content Checking and Modification
310

311
Functions for checking file contents and appending text to files.
312

313
```python { .api }
314
def contains(filename, text, exact=False, use_sudo=False, escape=True, shell=False):
315
    """
316
    Check if file contains specific text.
317
    
318
    Args:
319
        filename (str): File to check
320
        text (str): Text to search for
321
        exact (bool): Exact string match vs regex (default: False)
322
        use_sudo (bool): Use sudo for file access (default: False)
323
        escape (bool): Escape regex special characters (default: True)
324
        shell (bool): Use shell for command execution (default: False)
325
        
326
    Returns:
327
        bool: True if text found in file
328
    """
329

330
def append(filename, text, use_sudo=False, partial=False, escape=True, shell=False):
331
    """
332
    Append text to file if not already present.
333
    
334
    Args:
335
        filename (str): File to modify
336
        text (str): Text to append
337
        use_sudo (bool): Use sudo for file access (default: False)
338
        partial (bool): Allow partial line matches (default: False)
339
        escape (bool): Escape regex special characters (default: True)  
340
        shell (bool): Use shell for command execution (default: False)
341
    """
342
```
343

344
**Usage Examples:**
345

346
```python
347
from fabric.contrib.files import contains, append
348
from fabric.api import sudo
349

350
# Check if configuration exists
351
if contains('/etc/hosts', 'myapp.local'):
352
    print("Host entry already exists")
353
else:
354
    append('/etc/hosts', '127.0.0.1 myapp.local', use_sudo=True)
355

356
# Add SSH key if not present
357
ssh_key = 'ssh-rsa AAAAB3... user@example.com'
358
if not contains('/home/deploy/.ssh/authorized_keys', ssh_key):
359
    append('/home/deploy/.ssh/authorized_keys', ssh_key, use_sudo=True)
360

361
# Add cron job if not already scheduled  
362
cron_line = '0 2 * * * /opt/myapp/backup.sh'
363
if not contains('/var/spool/cron/crontabs/root', cron_line, use_sudo=True):
364
    append('/var/spool/cron/crontabs/root', cron_line, use_sudo=True)
365

366
# Check for exact string match
367
if contains('/proc/version', 'Ubuntu', exact=True):
368
    print("Running on Ubuntu")
369

370
# Append with partial matching (substring check)
371
append('/etc/environment', 
372
       'PATH="/usr/local/bin:$PATH"',
373
       partial=True, use_sudo=True)
374
```
375

376
### Project Synchronization
377

378
High-level functions for synchronizing entire projects and directories using rsync and tar-based transfers.
379

380
```python { .api }
381
def rsync_project(remote_dir, local_dir=None, exclude=(), delete=False, extra_opts='', ssh_opts='', capture=False, upload=True, default_opts='-pthrvz'):
382
    """
383
    Synchronize project files via rsync.
384
    
385
    Args:
386
        remote_dir (str): Remote directory path
387
        local_dir (str): Local directory path (default: current directory)
388
        exclude (tuple): Patterns to exclude from sync
389
        delete (bool): Delete remote files not in local (default: False)
390
        extra_opts (str): Additional rsync options
391
        ssh_opts (str): SSH connection options
392
        capture (bool): Capture output instead of printing (default: False)
393
        upload (bool): Upload to remote (True) or download (False)
394
        default_opts (str): Default rsync options (default: '-pthrvz')
395
        
396
    Returns:
397
        _AttributeString: rsync command output
398
    """
399

400
def upload_project(local_dir=None, remote_dir="", use_sudo=False):
401
    """
402
    Upload project directory via tar/gzip compression.
403
    
404
    Args:
405
        local_dir (str): Local project directory (default: current directory)
406
        remote_dir (str): Remote destination directory
407
        use_sudo (bool): Use sudo for remote operations (default: False)
408
        
409
    Returns:
410
        _AttributeString: Upload operation result
411
    """
412
```
413

414
**Usage Examples:**
415

416
```python
417
from fabric.contrib.project import rsync_project, upload_project
418
from fabric.api import cd, run
419

420
# Basic project sync
421
rsync_project('/var/www/myapp/', 'dist/')
422

423
# Sync with exclusions and delete
424
rsync_project('/var/www/myapp/', 
425
              exclude=('node_modules/', '.git/', '*.log'),
426
              delete=True)
427

428
# Download from remote (backup)
429
rsync_project('/var/www/myapp/', 'backup/', upload=False)
430

431
# Custom rsync options
432
rsync_project('/opt/myapp/', 
433
              extra_opts='--compress-level=9 --progress',
434
              ssh_opts='-o StrictHostKeyChecking=no')
435

436
# Upload entire project as tar.gz
437
upload_project('myapp/', '/tmp/deployments/', use_sudo=True)
438

439
# Typical deployment workflow
440
def deploy():
441
    # Build locally
442
    local('npm run build')
443
    
444
    # Sync to remote
445
    rsync_project('/var/www/myapp/', 'dist/',
446
                  exclude=('*.map', 'tests/'),
447
                  delete=True)
448
    
449
    # Restart services
450
    with cd('/var/www/myapp'):
451
        sudo('systemctl reload nginx')
452

453
# Large project upload with compression
454
def deploy_large_project():
455
    # Create compressed archive locally
456
    local('tar czf myapp.tar.gz --exclude=node_modules .')
457
    
458
    # Upload and extract
459
    put('myapp.tar.gz', '/tmp/', use_sudo=True)
460
    with cd('/var/www'):
461
        sudo('tar xzf /tmp/myapp.tar.gz')
462
        sudo('rm /tmp/myapp.tar.gz')
463
    
464
    local('rm myapp.tar.gz')  # Cleanup local archive
465
```
466

467
## File Transfer Results
468

469
Both `put()` and `get()` return `_AttributeList` objects that provide detailed information about transfer operations:
470

471
```python
472
from fabric.api import put, get
473

474
# Upload multiple files
475
results = put('config/*', '/etc/myapp/', use_glob=True)
476

477
# Check overall success
478
if results.succeeded:
479
    print(f"Successfully uploaded {len(results)} files")
480

481
# Handle partial failures
482
if results.failed:
483
    print("Some uploads failed:")
484
    for failed_file in results.failed:
485
        print(f"  Failed: {failed_file}")
486

487
# Process successful uploads
488
for uploaded_file in results:
489
    if uploaded_file not in results.failed:
490
        print(f"  Success: {uploaded_file}")
491

492
# Download with error handling
493
downloads = get('/var/log/*.log', use_glob=True)
494
if not downloads.succeeded:
495
    print("Download failed completely")
496
elif downloads.failed:
497
    print(f"Partial failure: {len(downloads.failed)} files failed")
498
```
499

500
### User Interaction
501

502
Interactive functions for user confirmation during file operations and deployment workflows.
503

504
```python { .api }
505
def confirm(question, default=True):
506
    """
507
    Ask user a yes/no question and return True or False.
508
    
509
    Args:
510
        question (str): Question to ask the user
511
        default (bool): Default value if user presses enter (default: True)
512
        
513
    Returns:
514
        bool: True if user confirms, False otherwise
515
    """
516
```
517

518
**Usage Examples:**
519

520
```python
521
from fabric.contrib.console import confirm
522
from fabric.api import run, sudo
523

524
# Confirm dangerous operations
525
if confirm('Delete all log files?', default=False):
526
    sudo('rm -rf /var/log/*.log')
527

528
# Confirm deployment
529
if confirm('Deploy to production?'):
530
    run('git pull origin master')
531
    sudo('systemctl restart nginx')
532
else:
533
    print("Deployment cancelled")
534

535
# Use in deployment scripts
536
@task
537
def deploy():
538
    if not confirm('Deploy to production? This cannot be undone.', default=False):
539
        abort('Deployment cancelled by user')
540
    
541
    # Continue with deployment
542
    rsync_project('/var/www/myapp/', 'dist/')
543
    sudo('systemctl reload nginx')
544
```
545

546
## Integration with Context Managers
547

548
File operations work seamlessly with Fabric's context managers:
549

550
```python
551
from fabric.api import *
552
from fabric.contrib.files import *
553

554
# Upload within directory context
555
with cd('/var/www'):
556
    put('app.tar.gz', '.')
557
    run('tar xzf app.tar.gz')
558

559
# Use sudo context for protected operations
560
with settings(use_sudo=True):
561
    upload_template('nginx.conf', '/etc/nginx/sites-available/myapp')
562
    append('/etc/hosts', '127.0.0.1 myapp.local')
563

564
# Combine multiple contexts
565
with cd('/opt/myapp'), settings(use_sudo=True):
566
    if not exists('config.json'):
567
        upload_template('config.json.template', 'config.json',
568
                       context={'debug': False})
569
```