0
# Utility Functions
1

2
Helper functions for temporary URLs, header processing, data formatting, and various Swift-specific operations.
3

4
## Capabilities
5

6
### Temporary URL Generation
7

8
Create time-limited URLs for direct object access without authentication.
9

10
```python { .api }
11
def generate_temp_url(
12
    path,
13
    seconds,
14
    key,
15
    method,
16
    absolute=False,
17
    prefix_based=False,
18
    iso8601=False,
19
    ip_range=None,
20
    digest=None,
21
    auth_url=None
22
):
23
    """
24
    Generate a temporary URL for Swift object access.
25

26
    Parameters:
27
    - path: str, object path (e.g., '/v1/AUTH_account/container/object')
28
    - seconds: int, URL validity duration in seconds from now
29
    - key: str, secret temporary URL key configured on account/container
30
    - method: str, HTTP method ('GET', 'PUT', 'POST', 'DELETE')
31
    - absolute: bool, return absolute timestamp instead of duration (default False)
32
    - prefix_based: bool, allow access to all objects with path prefix (default False)
33
    - iso8601: bool, use ISO 8601 timestamp format (default False)
34
    - ip_range: str, restrict access to IP range (e.g., '192.168.1.0/24')
35
    - digest: str, digest algorithm ('sha1', 'sha256', 'sha512', default 'sha1')
36
    - auth_url: str, complete auth URL for absolute URLs
37

38
    Returns:
39
    str: temporary URL with signature and expiration
40

41
    Usage:
42
    Set temp URL key on account: X-Account-Meta-Temp-Url-Key: secret_key
43
    Or on container: X-Container-Meta-Temp-Url-Key: secret_key
44
    """
45
```
46

47
### Response Processing
48

49
Parse and process Swift API responses.
50

51
```python { .api }
52
def parse_api_response(headers, body):
53
    """
54
    Parse Swift API JSON response.
55

56
    Parameters:
57
    - headers: dict, response headers
58
    - body: bytes, response body
59

60
    Returns:
61
    list or dict: parsed JSON response, or empty list if no body
62
    """
63

64
def get_body(headers, body):
65
    """
66
    Get response body content with proper encoding handling.
67

68
    Parameters:
69
    - headers: dict, response headers
70
    - body: bytes, raw response body
71

72
    Returns:
73
    str: decoded response body
74
    """
75
```
76

77
### Configuration and Formatting
78

79
Utilities for configuration parsing and data formatting.
80

81
```python { .api }
82
def config_true_value(value):
83
    """
84
    Check if value represents boolean true.
85

86
    Parameters:
87
    - value: any, value to check
88

89
    Returns:
90
    bool: True if value is True or string in ('true', '1', 'yes', 'on', 't', 'y')
91
    """
92

93
def prt_bytes(num_bytes, human_flag):
94
    """
95
    Format bytes for human-readable display.
96

97
    Parameters:
98
    - num_bytes: int, number of bytes
99
    - human_flag: bool, use human-readable format (K, M, G suffixes)
100

101
    Returns:
102
    str: formatted byte string (4 chars for human, 12 chars right-justified otherwise)
103
    """
104

105
def parse_timeout(value):
106
    """
107
    Parse timeout strings with suffixes.
108

109
    Parameters:
110
    - value: str, timeout value with optional suffix ('30s', '5m', '2h', '1d')
111

112
    Returns:
113
    float: timeout in seconds
114

115
    Supported suffixes:
116
    - s: seconds
117
    - m, min: minutes
118
    - h, hr: hours
119
    - d: days
120
    """
121

122
def parse_timestamp(seconds, absolute=False):
123
    """
124
    Parse timestamp values from various formats.
125

126
    Parameters:
127
    - seconds: str or float, timestamp value or ISO 8601 string
128
    - absolute: bool, return absolute timestamp (default False for relative)
129

130
    Returns:
131
    float: Unix timestamp
132

133
    Raises:
134
    ValueError: Invalid timestamp format
135
    """
136
```
137

138
### Header Processing
139

140
Utilities for handling HTTP headers and metadata.
141

142
```python { .api }
143
def split_request_headers(options, prefix=''):
144
    """
145
    Split header options into dictionary format.
146

147
    Parameters:
148
    - options: list, header strings in 'name:value' format
149
    - prefix: str, prefix to add to header names
150

151
    Returns:
152
    dict: headers dictionary with properly formatted names and values
153
    """
154

155
def report_traceback():
156
    """
157
    Report exception traceback for debugging.
158

159
    Returns:
160
    str: formatted traceback string
161
    """
162
```
163

164
### Data Streaming and Processing
165

166
Helper classes and functions for data streaming and processing.
167

168
```python { .api }
169
class ReadableToIterable:
170
    def __init__(self, content, chunk_size=65536, md5=False):
171
        """
172
        Convert file-like readable object to iterable.
173

174
        Parameters:
175
        - content: file-like object with read() method
176
        - chunk_size: int, size of chunks to read (default 65536)
177
        - md5: bool, calculate MD5 hash while reading (default False)
178
        """
179

180
    def __iter__(self):
181
        """Iterate over chunks of data."""
182

183
    def get_md5sum(self):
184
        """Get MD5 hash if md5=True was specified."""
185

186
class LengthWrapper:
187
    def __init__(self, readable, length, md5=False):
188
        """
189
        Wrap readable object with length limiting.
190

191
        Parameters:
192
        - readable: file-like object with read() method
193
        - length: int, maximum bytes to read
194
        - md5: bool, calculate MD5 hash while reading (default False)
195
        """
196

197
    def __iter__(self):
198
        """Iterate over chunks up to specified length."""
199

200
    def read(self, amt=None):
201
        """Read up to amt bytes or remaining length."""
202

203
    def get_md5sum(self):
204
        """Get MD5 hash if md5=True was specified."""
205

206
class NoopMD5:
207
    """No-operation MD5 hasher for when MD5 is disabled."""
208

209
    def update(self, data):
210
        """No-op update method."""
211

212
    def hexdigest(self):
213
        """Return empty MD5 hash."""
214
        return ""
215

216
class JSONableIterable(list):
217
    """JSON-serializable iterable that extends list."""
218

219
    def __init__(self, iterable=None):
220
        """Initialize with optional iterable."""
221

222
def iter_wrapper(iterable):
223
    """
224
    Wrap iterable for streaming uploads.
225

226
    Parameters:
227
    - iterable: any iterable object
228

229
    Returns:
230
    generator: wrapped iterable suitable for streaming
231
    """
232

233
def n_at_a_time(seq, n):
234
    """
235
    Split sequence into chunks of size n.
236

237
    Parameters:
238
    - seq: sequence to split
239
    - n: int, chunk size
240

241
    Yields:
242
    list: chunks of the sequence
243
    """
244

245
def n_groups(seq, n):
246
    """
247
    Split sequence into n groups of roughly equal size.
248

249
    Parameters:
250
    - seq: sequence to split
251
    - n: int, number of groups
252

253
    Returns:
254
    list: list of groups
255
    """
256

257
def normalize_manifest_path(path):
258
    """
259
    Normalize manifest paths for consistency.
260

261
    Parameters:
262
    - path: str, manifest path to normalize
263

264
    Returns:
265
    str: normalized path
266
    """
267
```
268

269
### Constants
270

271
```python { .api }
272
TRUE_VALUES = {'true', '1', 'yes', 'on', 't', 'y'}
273
EMPTY_ETAG = 'd41d8cd98f00b204e9800998ecf8427e'
274
EXPIRES_ISO8601_FORMAT = '%Y-%m-%dT%H:%M:%SZ'
275
SHORT_EXPIRES_ISO8601_FORMAT = '%Y-%m-%d'
276
TIME_ERRMSG = 'time must either be a whole number or in specific ISO 8601 format.'
277
```
278

279
## Usage Examples
280

281
### Temporary URL Generation
282

283
```python
284
from swiftclient.utils import generate_temp_url
285
import time
286

287
# Set temporary URL key on account (do this once via swift CLI or API)
288
# swift post -m "Temp-URL-Key:my-secret-key"
289

290
# Generate temporary GET URL valid for 1 hour
291
path = '/v1/AUTH_account/documents/confidential.pdf'
292
temp_url = generate_temp_url(
293
    path=path,
294
    seconds=3600,  # 1 hour
295
    key='my-secret-key',
296
    method='GET'
297
)
298

299
print(f"Temporary URL: https://swift.example.com{temp_url}")
300

301
# Generate temporary PUT URL for uploads
302
upload_url = generate_temp_url(
303
    path='/v1/AUTH_account/uploads/new-file.txt',
304
    seconds=1800,  # 30 minutes
305
    key='my-secret-key',
306
    method='PUT'
307
)
308

309
# Generate prefix-based URL for directory access
310
prefix_url = generate_temp_url(
311
    path='/v1/AUTH_account/documents/',
312
    seconds=7200,  # 2 hours
313
    key='my-secret-key',
314
    method='GET',
315
    prefix_based=True  # Access all objects under documents/
316
)
317

318
# Generate URL with IP restrictions
319
restricted_url = generate_temp_url(
320
    path='/v1/AUTH_account/private/data.csv',
321
    seconds=3600,
322
    key='my-secret-key',
323
    method='GET',
324
    ip_range='192.168.1.0/24'  # Only allow from this subnet
325
)
326

327
# Use absolute timestamp instead of duration
328
absolute_timestamp = int(time.time()) + 3600  # 1 hour from now
329
absolute_url = generate_temp_url(
330
    path=path,
331
    seconds=absolute_timestamp,
332
    key='my-secret-key',
333
    method='GET',
334
    absolute=True
335
)
336
```
337

338
### Data Formatting and Configuration
339

340
```python
341
from swiftclient.utils import prt_bytes, config_true_value, parse_timeout
342

343
# Format bytes for display
344
print(prt_bytes(1024, False))      # '        1024'
345
print(prt_bytes(1024, True))       # '1.0K'
346
print(prt_bytes(1536000, True))    # '1.5M'
347
print(prt_bytes(5368709120, True)) # '5.0G'
348

349
# Check boolean configuration values
350
config_values = ['true', '1', 'yes', 'on', 'false', '0', 'no', 'off']
351
for value in config_values:
352
    print(f"'{value}' is {config_true_value(value)}")
353

354
# Parse timeout strings
355
timeouts = ['30', '30s', '5m', '2h', '1d']
356
for timeout in timeouts:
357
    seconds = parse_timeout(timeout)
358
    print(f"'{timeout}' = {seconds} seconds")
359
```
360

361
### Header Processing
362

363
```python
364
from swiftclient.utils import split_request_headers
365

366
# Parse header options from command line format
367
header_options = [
368
    'Content-Type:application/json',
369
    'X-Object-Meta-Author:John Doe',
370
    'X-Object-Meta-Version:2.0',
371
    'Cache-Control:max-age=3600'
372
]
373

374
headers = split_request_headers(header_options)
375
print(headers)
376
# {
377
#     'content-type': 'application/json',
378
#     'x-object-meta-author': 'John Doe',
379
#     'x-object-meta-version': '2.0',
380
#     'cache-control': 'max-age=3600'
381
# }
382

383
# Parse with prefix
384
container_headers = split_request_headers([
385
    'Meta-Owner:TeamA',
386
    'Read:.r:*',
387
    'Write:.r:*'
388
], prefix='X-Container-')
389

390
print(container_headers)
391
# {
392
#     'x-container-meta-owner': 'TeamA',
393
#     'x-container-read': '.r:*',
394
#     'x-container-write': '.r:*'
395
# }
396
```
397

398
### Data Streaming
399

400
```python
401
from swiftclient.utils import ReadableToIterable, LengthWrapper
402
import io
403

404
# Convert file to iterable for streaming upload
405
with open('large_file.dat', 'rb') as f:
406
    iterable = ReadableToIterable(f, chunk_size=65536, md5=True)
407

408
    # Upload using the iterable (example with low-level function)
409
    etag = put_object(storage_url, token, 'container', 'object', iterable)
410

411
    # Get MD5 hash
412
    md5_hash = iterable.get_md5sum()
413
    print(f"Uploaded with ETag: {etag}, MD5: {md5_hash}")
414

415
# Limit reading to specific length
416
data = io.BytesIO(b'x' * 10000)  # 10KB of data
417
limited = LengthWrapper(data, 5000, md5=True)  # Only read first 5KB
418

419
chunk_count = 0
420
for chunk in limited:
421
    chunk_count += 1
422
    print(f"Chunk {chunk_count}: {len(chunk)} bytes")
423

424
print(f"MD5 of first 5KB: {limited.get_md5sum()}")
425
```
426

427
### Sequence Processing
428

429
```python
430
from swiftclient.utils import n_at_a_time, n_groups
431

432
# Process items in batches
433
items = list(range(23))
434

435
# Split into chunks of 5
436
for i, chunk in enumerate(n_at_a_time(items, 5)):
437
    print(f"Batch {i}: {chunk}")
438
# Batch 0: [0, 1, 2, 3, 4]
439
# Batch 1: [5, 6, 7, 8, 9]
440
# ...
441

442
# Split into 4 roughly equal groups
443
groups = n_groups(items, 4)
444
for i, group in enumerate(groups):
445
    print(f"Group {i}: {group} ({len(group)} items)")
446
```
447

448
### Response Processing
449

450
```python
451
from swiftclient.utils import parse_api_response, get_body
452
import json
453

454
# Parse Swift API JSON response
455
headers = {'content-type': 'application/json; charset=utf-8'}
456
body = json.dumps([
457
    {'name': 'container1', 'count': 42, 'bytes': 1024000},
458
    {'name': 'container2', 'count': 17, 'bytes': 512000}
459
]).encode('utf-8')
460

461
containers = parse_api_response(headers, body)
462
for container in containers:
463
    print(f"Container: {container['name']}, Objects: {container['count']}")
464

465
# Get response body with encoding
466
text_body = get_body(headers, body)
467
print(f"Response body: {text_body}")
468
```
469

470
### Advanced Utility Classes
471

472
Specialized utility classes for stream processing, content handling, and data conversion.
473

474
```python { .api }
475
class ReadableToIterable:
476
    def __init__(self, content, checksum=None, md5=None, chunk_size=65536):
477
        """
478
        Convert file-like objects to iterables suitable for Swift uploads.
479

480
        Parameters:
481
        - content: file-like object or iterable to convert
482
        - checksum: bool, whether to calculate content checksum
483
        - md5: hashlib MD5 object, existing MD5 hasher to update
484
        - chunk_size: int, chunk size for reading (default 65536)
485

486
        Yields:
487
        bytes: content chunks for streaming upload
488
        """
489

490
    def __iter__(self):
491
        """Return iterator for content chunks."""
492

493
    def get_md5sum(self):
494
        """Get MD5 checksum of processed content."""
495

496
class LengthWrapper:
497
    def __init__(self, readable, length):
498
        """
499
        Wrap readable object with known content length.
500

501
        Parameters:
502
        - readable: file-like object or iterable
503
        - length: int, total content length in bytes
504

505
        Used for streaming uploads where content length must be known upfront.
506
        """
507

508
    def __iter__(self):
509
        """Return iterator for content."""
510

511
    def __len__(self):
512
        """Return content length."""
513

514
    def read(self, size=-1):
515
        """Read up to size bytes from content."""
516

517
class JSONableIterable(list):
518
    def __init__(self, iterable):
519
        """
520
        JSON-serializable iterable that preserves iteration behavior.
521

522
        Parameters:
523
        - iterable: any iterable to wrap
524

525
        Allows iterables to be JSON serialized while maintaining
526
        their iteration properties for Swift operations.
527
        """
528

529
class NoopMD5:
530
    """
531
    No-operation MD5 hasher for environments where hashlib MD5 is unavailable.
532

533
    Provides the same interface as hashlib.md5() but performs no actual hashing.
534
    Used as fallback when FIPS mode or other restrictions disable MD5.
535
    """
536

537
    def update(self, data):
538
        """Accept data but perform no hashing."""
539

540
    def digest(self):
541
        """Return empty digest."""
542

543
    def hexdigest(self):
544
        """Return empty hex digest."""
545
```
546

547
### Advanced Utility Usage
548

549
```python
550
from swiftclient.utils import ReadableToIterable, LengthWrapper
551

552
# Convert file to iterable for streaming upload
553
with open('largefile.dat', 'rb') as f:
554
    iterable = ReadableToIterable(f, checksum=True)
555

556
    # Use in Swift upload
557
    conn.put_object('container', 'object', iterable)
558

559
    # Get checksum after upload
560
    md5_hash = iterable.get_md5sum()
561

562
# Wrap content with known length
563
content = b'Hello, Swift!'
564
wrapped = LengthWrapper(iter([content]), len(content))
565

566
# Use for precise content length uploads
567
conn.put_object(
568
    'container',
569
    'object',
570
    wrapped,
571
    content_length=len(wrapped)
572
)
573
```