0
# FTP Connection Management
1
2
Low-level hook classes for establishing and managing FTP connections with comprehensive file operations, authentication support, and SSL/TLS encryption capabilities.
3
4
## Capabilities
5
6
### FTP Hook
7
8
Primary hook class for standard FTP connections with full file system operations and connection management.
9
10
```python { .api }
11
class FTPHook(BaseHook):
12
"""
13
Interact with FTP servers.
14
15
Parameters:
16
- ftp_conn_id (str): The FTP connection ID reference (default: "ftp_default")
17
"""
18
19
conn_name_attr = "ftp_conn_id"
20
default_conn_name = "ftp_default"
21
conn_type = "ftp"
22
hook_name = "FTP"
23
24
def __init__(self, ftp_conn_id: str = "ftp_default") -> None: ...
25
26
def __enter__(self) -> FTPHook: ...
27
def __exit__(self, exc_type: Any, exc_val: Any, exc_tb: Any) -> None: ...
28
29
def get_conn(self) -> ftplib.FTP:
30
"""
31
Return an FTP connection object.
32
33
Returns:
34
ftplib.FTP: Active FTP connection
35
"""
36
37
def close_conn(self) -> None:
38
"""
39
Close the FTP connection.
40
41
Raises:
42
Exception: If connection was never opened
43
"""
44
45
def test_connection(self) -> tuple[bool, str]:
46
"""
47
Test FTP connection by calling path with directory.
48
49
Returns:
50
tuple[bool, str]: (success_status, message)
51
"""
52
```
53
54
### Directory Operations
55
56
Methods for managing directories on the FTP server.
57
58
```python { .api }
59
def describe_directory(self, path: str) -> dict:
60
"""
61
Return directory listing with file attributes using MLSD.
62
63
Parameters:
64
- path (str): Full path to remote directory
65
66
Returns:
67
dict: Dictionary of {filename: {attributes}}
68
"""
69
70
def list_directory(self, path: str) -> list[str]:
71
"""
72
Return list of files in remote directory using NLST.
73
74
Parameters:
75
- path (str): Full path to remote directory
76
77
Returns:
78
list[str]: List of filenames
79
"""
80
81
def create_directory(self, path: str) -> None:
82
"""
83
Create directory on remote system.
84
85
Parameters:
86
- path (str): Full path to remote directory to create
87
"""
88
89
def delete_directory(self, path: str) -> None:
90
"""
91
Delete directory on remote system.
92
93
Parameters:
94
- path (str): Full path to remote directory to delete
95
"""
96
```
97
98
### File Operations
99
100
Methods for transferring and managing files on the FTP server.
101
102
```python { .api }
103
def retrieve_file(
104
self,
105
remote_full_path: str,
106
local_full_path_or_buffer: Any,
107
callback: Callable | None = None,
108
block_size: int = 8192
109
) -> None:
110
"""
111
Transfer remote file to local location.
112
113
Parameters:
114
- remote_full_path (str): Full path to remote file
115
- local_full_path_or_buffer (str | file-like): Local path or buffer
116
- callback (Callable, optional): Called for each data block
117
- block_size (int): Transfer chunk size (default: 8192)
118
"""
119
120
def store_file(
121
self,
122
remote_full_path: str,
123
local_full_path_or_buffer: Any,
124
block_size: int = 8192
125
) -> None:
126
"""
127
Transfer local file to remote location.
128
129
Parameters:
130
- remote_full_path (str): Full path to remote file
131
- local_full_path_or_buffer (str | file-like): Local path or buffer
132
- block_size (int): Transfer chunk size (default: 8192)
133
"""
134
135
def delete_file(self, path: str) -> None:
136
"""
137
Remove file on FTP server.
138
139
Parameters:
140
- path (str): Full path to remote file
141
"""
142
143
def rename(self, from_name: str, to_name: str) -> str:
144
"""
145
Rename file on FTP server.
146
147
Parameters:
148
- from_name (str): Current filename
149
- to_name (str): New filename
150
151
Returns:
152
str: Server response
153
"""
154
```
155
156
### File Information
157
158
Methods for retrieving file metadata and properties.
159
160
```python { .api }
161
def get_mod_time(self, path: str) -> datetime.datetime:
162
"""
163
Return file last modification time.
164
165
Parameters:
166
- path (str): Remote file path
167
168
Returns:
169
datetime.datetime: Last modification timestamp
170
"""
171
172
def get_size(self, path: str) -> int | None:
173
"""
174
Return file size in bytes.
175
176
Parameters:
177
- path (str): Remote file path
178
179
Returns:
180
int | None: File size or None if unavailable
181
"""
182
```
183
184
### FTPS Hook
185
186
Secure FTP hook supporting SSL/TLS encryption for encrypted file transfers.
187
188
```python { .api }
189
class FTPSHook(FTPHook):
190
"""
191
Interact with FTPS (FTP over SSL/TLS) servers.
192
193
Inherits all FTPHook methods with SSL/TLS encryption support.
194
"""
195
196
def get_conn(self) -> ftplib.FTP:
197
"""
198
Return FTPS connection object with SSL context.
199
200
Returns:
201
ftplib.FTP: Active FTPS connection with SSL/TLS encryption
202
"""
203
```
204
205
## Usage Examples
206
207
### Basic File Operations
208
209
```python
210
from airflow.providers.ftp.hooks.ftp import FTPHook
211
212
# Using context manager for automatic connection cleanup
213
with FTPHook(ftp_conn_id='my_ftp') as hook:
214
# List directory contents
215
files = hook.list_directory('/remote/data')
216
217
# Download file
218
hook.retrieve_file('/remote/data/input.csv', '/local/data/input.csv')
219
220
# Upload file
221
hook.store_file('/remote/data/output.csv', '/local/data/output.csv')
222
223
# Get file information
224
size = hook.get_size('/remote/data/input.csv')
225
mod_time = hook.get_mod_time('/remote/data/input.csv')
226
```
227
228
### Advanced File Transfer with Callback
229
230
```python
231
from airflow.providers.ftp.hooks.ftp import FTPHook
232
233
def progress_callback(data):
234
print(f"Transferred {len(data)} bytes")
235
236
hook = FTPHook(ftp_conn_id='my_ftp')
237
238
# Download with progress tracking
239
hook.retrieve_file(
240
remote_full_path='/remote/large_file.zip',
241
local_full_path_or_buffer='/local/large_file.zip',
242
callback=progress_callback,
243
block_size=16384
244
)
245
246
hook.close_conn()
247
```
248
249
### Secure FTP Operations
250
251
```python
252
from airflow.providers.ftp.hooks.ftp import FTPSHook
253
254
# Using FTPS for encrypted transfers
255
with FTPSHook(ftp_conn_id='my_secure_ftp') as hook:
256
# All operations are encrypted via SSL/TLS
257
hook.store_file('/secure/path/confidential.txt', '/local/confidential.txt')
258
files = hook.list_directory('/secure/path')
259
```
260
261
## Error Handling
262
263
FTP operations may raise various exceptions from the underlying ftplib:
264
265
- **ftplib.error_perm**: Permanent FTP errors (file not found, permission denied)
266
- **ftplib.error_temp**: Temporary FTP errors (connection issues, server busy)
267
- **socket.error**: Network connectivity issues
268
- **Exception**: General connection and operation failures
269
270
Always handle exceptions appropriately and use the `test_connection()` method to verify connectivity before performing operations.