0
# File Transfer Operations
1

2
High-performance file copying between local and remote hosts using both SFTP and SCP protocols. Supports recursive directory operations, per-host file naming, and both parallel and single-host transfers.
3

4
## Capabilities
5

6
### SFTP File Operations
7

8
Secure file transfer protocol operations for reliable file copying with support for directory recursion and custom file naming.
9

10
```python { .api }
11
def copy_file(self, local_file, remote_file, recurse=False, copy_args=None):
12
    """
13
    Copy local file to remote hosts via SFTP.
14

15
    Parameters:
16
    - local_file (str): Path to local file or directory
17
    - remote_file (str): Remote destination path
18
    - recurse (bool, optional): Recursively copy directories (default: False)
19
    - copy_args (list, optional): Per-host copy arguments
20

21
    Returns:
22
    list[gevent.Greenlet]: List of greenlets for copy operations (ParallelSSHClient)
23
    None: For SSHClient (operation completes synchronously)
24
    """
25

26
def copy_remote_file(self, remote_file, local_file, recurse=False,
27
                    suffix_separator='_', copy_args=None, encoding='utf-8'):
28
    """
29
    Copy remote files to local host via SFTP.
30

31
    Parameters:
32
    - remote_file (str): Remote file path
33
    - local_file (str): Local destination path
34
    - recurse (bool, optional): Recursively copy directories (default: False)
35
    - suffix_separator (str, optional): Separator for per-host files (default: '_')
36
    - copy_args (list, optional): Per-host copy arguments
37
    - encoding (str, optional): File encoding (default: 'utf-8')
38

39
    Returns:
40
    list[gevent.Greenlet]: List of greenlets for copy operations (ParallelSSHClient)
41
    None: For SSHClient (operation completes synchronously)
42
    """
43
```
44

45
Usage examples:
46

47
```python
48
from pssh.clients import ParallelSSHClient, SSHClient
49
from gevent import joinall
50

51
# Parallel SFTP upload
52
hosts = ['web1.example.com', 'web2.example.com']
53
client = ParallelSSHClient(hosts)
54

55
# Upload single file to all hosts
56
greenlets = client.copy_file('/local/config.txt', '/etc/app/config.txt')
57
joinall(greenlets, raise_error=True)
58

59
# Upload directory recursively
60
greenlets = client.copy_file('/local/website/', '/var/www/html/', recurse=True)
61
joinall(greenlets, raise_error=True)
62

63
# Download files from all hosts (creates host-specific local files)
64
greenlets = client.copy_remote_file('/var/log/app.log', '/local/logs/app.log')
65
joinall(greenlets)
66
# Creates: /local/logs/app.log_web1.example.com, /local/logs/app.log_web2.example.com
67

68
# Single host SFTP
69
single_client = SSHClient('server.example.com')
70
single_client.copy_file('/local/backup.tar.gz', '/remote/backups/backup.tar.gz')
71
single_client.copy_remote_file('/remote/data.csv', '/local/data.csv')
72
```
73

74
### SCP File Operations  
75

76
Secure copy protocol operations providing the best performance for file transfers, with support for recursive directory copying.
77

78
```python { .api }
79
def scp_send(self, local_file, remote_file, recurse=False, copy_args=None):
80
    """
81
    Send files to remote hosts via SCP.
82

83
    Parameters:
84
    - local_file (str): Path to local file or directory
85
    - remote_file (str): Remote destination path  
86
    - recurse (bool, optional): Recursively copy directories (default: False)
87
    - copy_args (list, optional): Per-host copy arguments
88

89
    Returns:
90
    list[gevent.Greenlet]: List of greenlets for copy operations (ParallelSSHClient)
91
    None: For SSHClient (operation completes synchronously)
92

93
    Note: 
94
    SCP does not overwrite existing remote files and raises SCPError instead.
95
    Recursive copying requires server SFTP support for directory creation.
96
    """
97

98
def scp_recv(self, remote_file, local_file, recurse=False, copy_args=None,
99
            suffix_separator='_'):
100
    """
101
    Receive files from remote hosts via SCP.
102

103
    Parameters:
104
    - remote_file (str): Remote file path
105
    - local_file (str): Local destination path
106
    - recurse (bool, optional): Recursively copy directories (default: False)  
107
    - copy_args (list, optional): Per-host copy arguments
108
    - suffix_separator (str, optional): Separator for per-host files (default: '_')
109

110
    Returns:
111
    list[gevent.Greenlet]: List of greenlets for copy operations (ParallelSSHClient)
112
    None: For SSHClient (operation completes synchronously)
113
    """
114
```
115

116
Usage examples:
117

118
```python
119
# Parallel SCP upload (highest performance)
120
hosts = ['server1.example.com', 'server2.example.com']
121
client = ParallelSSHClient(hosts)
122

123
# Send large file to all hosts
124
greenlets = client.scp_send('/local/large_file.bin', '/remote/large_file.bin')
125
joinall(greenlets, raise_error=True)
126

127
# Send directory recursively
128
greenlets = client.scp_send('/local/app/', '/opt/app/', recurse=True)
129
joinall(greenlets, raise_error=True)
130

131
# Receive files from all hosts
132
greenlets = client.scp_recv('/var/log/system.log', '/local/logs/system.log')
133
joinall(greenlets)
134

135
# Single host SCP with error handling
136
from pssh.exceptions import SCPError
137

138
single_client = SSHClient('server.example.com')
139
try:
140
    single_client.scp_send('/local/file.txt', '/remote/file.txt')
141
    print("File sent successfully")
142
except SCPError as e:
143
    print(f"SCP error: {e}")
144
```
145

146
### Per-Host File Operations
147

148
Customize file operations for individual hosts using copy arguments for different file names, paths, or operations per host.
149

150
```python
151
# Per-host copy arguments for different destinations
152
hosts = ['web1.example.com', 'web2.example.com', 'db.example.com']
153
client = ParallelSSHClient(hosts)
154

155
copy_args = [
156
    {'local_file': '/local/web_config.txt', 'remote_file': '/etc/nginx/site.conf'},  # web1
157
    {'local_file': '/local/web_config.txt', 'remote_file': '/etc/apache2/site.conf'}, # web2
158
    {'local_file': '/local/db_config.txt', 'remote_file': '/etc/mysql/my.cnf'}        # db
159
]
160

161
greenlets = client.copy_file(copy_args=copy_args)
162
joinall(greenlets, raise_error=True)
163

164
# Per-host download with custom local naming
165
copy_args = [
166
    {'remote_file': '/var/log/nginx/access.log', 'local_file': '/logs/web1_access.log'},
167
    {'remote_file': '/var/log/apache2/access.log', 'local_file': '/logs/web2_access.log'}, 
168
    {'remote_file': '/var/log/mysql/slow.log', 'local_file': '/logs/db_slow.log'}
169
]
170

171
greenlets = client.copy_remote_file(copy_args=copy_args)
172
joinall(greenlets)
173
```
174

175
## Error Handling
176

177
File transfer operations provide specific error handling for various failure scenarios:
178

179
```python
180
from pssh.exceptions import SFTPError, SFTPIOError, SCPError
181
from gevent import joinall
182

183
try:
184
    # SFTP operations
185
    greenlets = client.copy_file('/nonexistent/file.txt', '/remote/file.txt')
186
    joinall(greenlets, raise_error=True)
187
except SFTPError as e:
188
    print(f"SFTP initialization error: {e}")
189
except SFTPIOError as e:
190
    print(f"SFTP I/O error: {e}")
191

192
try:
193
    # SCP operations  
194
    greenlets = client.scp_send('/local/file.txt', '/remote/existing_file.txt')
195
    joinall(greenlets, raise_error=True)
196
except SCPError as e:
197
    print(f"SCP error (file may already exist): {e}")
198
```
199

200
## Protocol Comparison
201

202
### SFTP vs SCP
203

204
**SFTP (SSH File Transfer Protocol)**:
205
- More reliable with better error handling
206
- Supports resuming interrupted transfers  
207
- Can overwrite existing files
208
- Slightly more overhead than SCP
209
- Better for general-purpose file transfers
210

211
**SCP (Secure Copy Protocol)**:
212
- Highest performance for large file transfers
213
- Does not overwrite existing files (raises error instead)
214
- Requires SFTP support for recursive directory creation
215
- Best for one-time file deployments
216
- Minimal protocol overhead
217

218
### Usage Recommendations
219

220
```python
221
# Use SFTP for:
222
# - Regular file synchronization  
223
# - When files might already exist
224
# - When you need detailed error information
225
greenlets = client.copy_file('/local/config/', '/etc/app/', recurse=True)
226

227
# Use SCP for:
228
# - Large file transfers requiring maximum performance
229
# - Initial deployment to clean remote directories
230
# - When you want to prevent accidental overwrites
231
greenlets = client.scp_send('/local/release.tar.gz', '/opt/releases/release.tar.gz')
232
```
233

234
## Large File Transfer Best Practices
235

236
```python
237
# For large files or many files, consider:
238

239
# 1. Compress before transfer
240
import tarfile
241
with tarfile.open('/tmp/archive.tar.gz', 'w:gz') as tar:
242
    tar.add('/large/directory', arcname='directory')
243

244
greenlets = client.scp_send('/tmp/archive.tar.gz', '/remote/archive.tar.gz')
245
joinall(greenlets)
246

247
# Extract on remote hosts
248
output = client.run_command('cd /remote && tar -xzf archive.tar.gz')
249
client.join()
250

251
# 2. Use appropriate pool_size for I/O intensive operations
252
client = ParallelSSHClient(hosts, pool_size=50)  # Reduce for file transfers
253

254
# 3. Monitor transfer progress for very large operations
255
from gevent import spawn
256
import time
257

258
def monitor_transfers(greenlets):
259
    while not all(g.ready() for g in greenlets):
260
        completed = sum(1 for g in greenlets if g.ready())
261
        print(f"Transfers completed: {completed}/{len(greenlets)}")
262
        time.sleep(5)
263

264
greenlets = client.scp_send('/large/file.bin', '/remote/file.bin')
265
monitor = spawn(monitor_transfers, greenlets)
266
joinall(greenlets + [monitor])
267
```