0
# Iteration & Processing Utilities
1

2
Comprehensive tools for working with iterables including chunking, windowing, flattening, uniqueness operations, and advanced data structure manipulation. Provides both list and iterator versions of operations for memory efficiency and performance optimization.
3

4
## Capabilities
5

6
### Chunking Operations
7

8
Break iterables into chunks of specified sizes for batch processing.
9

10
```python { .api }
11
def chunked(src, size, count=None, **kw):
12
    """
13
    Break iterable into chunks of specified size.
14
    
15
    Parameters:
16
    - src (iterable): Source iterable to chunk
17
    - size (int): Size of each chunk
18
    - count (int, optional): Maximum number of chunks to return
19
    
20
    Returns:
21
    list: List of chunks (each chunk is a list)
22
    """
23

24
def chunked_iter(src, size, **kw):
25
    """
26
    Iterator version of chunked.
27
    
28
    Parameters:
29
    - src (iterable): Source iterable to chunk
30
    - size (int): Size of each chunk
31
    
32
    Yields:
33
    list: Each chunk as a list
34
    """
35

36
def chunk_ranges(input_size, chunk_size, **kwargs):
37
    """
38
    Generate ranges for chunking operations.
39
    
40
    Parameters:
41
    - input_size (int): Total size of input data
42
    - chunk_size (int): Desired chunk size
43
    
44
    Returns:
45
    list: List of (start, end) tuples representing chunk ranges
46
    """
47
```
48

49
### Windowing Operations
50

51
Create sliding windows over iterables for sequential data analysis.
52

53
```python { .api }
54
def windowed(src, size, fill=_UNSET):
55
    """
56
    Create sliding window over iterable.
57
    
58
    Parameters:
59
    - src (iterable): Source iterable
60
    - size (int): Window size
61
    - fill: Fill value for incomplete windows (default: no fill)
62
    
63
    Returns:
64
    list: List of windows (each window is a tuple)
65
    """
66

67
def windowed_iter(src, size, fill=_UNSET):
68
    """
69
    Iterator version of windowed.
70
    
71
    Parameters:
72
    - src (iterable): Source iterable  
73
    - size (int): Window size
74
    - fill: Fill value for incomplete windows
75
    
76
    Yields:
77
    tuple: Each window as a tuple
78
    """
79

80
def pairwise(src, end=_UNSET):
81
    """
82
    Iterate over successive pairs from iterable.
83
    
84
    Parameters:
85
    - src (iterable): Source iterable
86
    - end: Value to pair with last element (default: no end pairing)
87
    
88
    Returns:
89
    list: List of (current, next) tuples
90
    """
91

92
def pairwise_iter(src, end=_UNSET):
93
    """
94
    Iterator version of pairwise.
95
    
96
    Parameters:
97
    - src (iterable): Source iterable
98
    - end: Value to pair with last element
99
    
100
    Yields:
101
    tuple: Each (current, next) pair
102
    """
103
```
104

105
### Sequence Operations
106

107
Split, strip, and manipulate sequences with flexible separators.
108

109
```python { .api }
110
def split(src, sep=None, maxsplit=None):
111
    """
112
    Split iterable by separator.
113
    
114
    Parameters:
115
    - src (iterable): Source iterable to split
116
    - sep: Separator value (None means any falsy value)
117
    - maxsplit (int, optional): Maximum number of splits
118
    
119
    Returns:
120
    list: List of split segments
121
    """
122

123
def split_iter(src, sep=None, maxsplit=None):
124
    """Iterator version of split."""
125

126
def lstrip(iterable, strip_value=None):
127
    """
128
    Strip values from left side of iterable.
129
    
130
    Parameters:
131
    - iterable: Source iterable
132
    - strip_value: Value to strip (None means falsy values)
133
    
134
    Returns:
135
    list: Iterable with left values stripped
136
    """
137

138
def rstrip(iterable, strip_value=None):
139
    """Strip values from right side of iterable."""
140

141
def strip(iterable, strip_value=None):
142
    """Strip values from both sides of iterable."""
143

144
def strip_iter(iterable, strip_value=None):
145
    """Iterator version of strip."""
146
```
147

148
### Uniqueness and Filtering
149

150
Remove duplicates, find redundant elements, and filter iterables.
151

152
```python { .api }
153
def unique(src, key=None):
154
    """
155
    Remove duplicates while preserving order.
156
    
157
    Parameters:
158
    - src (iterable): Source iterable
159
    - key (callable, optional): Function to compute unique key
160
    
161
    Returns:
162
    list: List with duplicates removed
163
    """
164

165
def unique_iter(src, key=None):
166
    """Iterator version of unique."""
167

168
def redundant(src, key=None, **kwargs):
169
    """
170
    Find duplicate elements.
171
    
172
    Parameters:
173
    - src (iterable): Source iterable
174
    - key (callable, optional): Function to compute comparison key
175
    
176
    Returns:
177
    list: List of duplicate elements
178
    """
179

180
def partition(src, key=None, **kwargs):
181
    """
182
    Partition iterable into two lists by predicate.
183
    
184
    Parameters:
185
    - src (iterable): Source iterable
186
    - key (callable): Predicate function
187
    
188
    Returns:
189
    tuple: (truthy_items, falsy_items)
190
    """
191
```
192

193
### Data Structure Manipulation
194

195
Recursively transform and query nested data structures.
196

197
```python { .api }
198
def remap(root_obj, **kwargs):
199
    """
200
    Recursively transform nested data structures.
201
    
202
    Parameters:
203
    - root_obj: Root object to transform
204
    - visit (callable, optional): Function to visit each value
205
    - enter (callable, optional): Function called when entering containers
206
    - exit (callable, optional): Function called when exiting containers
207
    
208
    Returns:
209
    Transformed data structure
210
    """
211

212
def get_path(root_obj, path, default=_UNSET):
213
    """
214
    Get value from nested structure by path.
215
    
216
    Parameters:
217
    - root_obj: Root object to query
218
    - path (iterable): Path to the desired value
219
    - default: Default value if path not found
220
    
221
    Returns:
222
    Value at the specified path
223
    """
224

225
def research(root_obj, **kwargs):
226
    """
227
    Search nested structures with query function.
228
    
229
    Parameters:
230
    - root_obj: Root object to search
231
    - query (callable): Function to test values
232
    
233
    Returns:
234
    list: List of matching values
235
    """
236
```
237

238
### Flattening Operations
239

240
Recursively flatten nested iterables and data structures.
241

242
```python { .api }
243
def flatten(iterable):
244
    """
245
    Flatten nested iterables to list.
246
    
247
    Parameters:
248
    - iterable: Nested iterable to flatten
249
    
250
    Returns:
251
    list: Flattened list
252
    """
253

254
def flatten_iter(iterable):
255
    """
256
    Recursively flatten nested iterables.
257
    
258
    Parameters:
259
    - iterable: Nested iterable to flatten
260
    
261
    Yields:
262
    Individual items from nested structure
263
    """
264
```
265

266
### Utility Functions
267

268
Miscellaneous iteration utilities for common patterns.
269

270
```python { .api }
271
def bucketize(src, key=None, **kwargs):
272
    """
273
    Group items into buckets by key function.
274
    
275
    Parameters:
276
    - src (iterable): Source iterable
277
    - key (callable): Function to compute bucket key
278
    
279
    Returns:
280
    dict: Dictionary mapping keys to lists of items
281
    """
282

283
def one(src, default=_UNSET, **kwargs):
284
    """
285
    Ensure iterable contains exactly one element.
286
    
287
    Parameters:
288
    - src (iterable): Source iterable
289
    - default: Default value if iterable is empty
290
    
291
    Returns:
292
    The single element
293
    
294
    Raises:
295
    ValueError: If iterable has more than one element
296
    """
297

298
def first(iterable, default=_UNSET, key=None):
299
    """
300
    Get first element matching condition.
301
    
302
    Parameters:
303
    - iterable: Source iterable
304
    - default: Default if no match found
305
    - key (callable, optional): Condition function
306
    
307
    Returns:
308
    First matching element or default
309
    """
310

311
def same(iterable, ref=_UNSET):
312
    """
313
    Check if all elements are the same.
314
    
315
    Parameters:
316
    - iterable: Source iterable
317
    - ref: Reference value to compare against
318
    
319
    Returns:
320
    bool: True if all elements are the same
321
    """
322
```
323

324
### Numeric Range Operations
325

326
Extended range functions with float support and exponential backoff.
327

328
```python { .api }
329
def xfrange(stop, start=None, step=1.0):
330
    """
331
    Extended float range generator.
332
    
333
    Parameters:
334
    - stop (float): End value (exclusive)
335
    - start (float, optional): Start value (default: 0)
336
    - step (float): Step size (default: 1.0)
337
    
338
    Yields:
339
    float: Each value in the range
340
    """
341

342
def frange(stop, start=None, step=1.0):
343
    """
344
    Float range as list.
345
    
346
    Parameters:
347
    - stop (float): End value (exclusive)
348
    - start (float, optional): Start value
349
    - step (float): Step size
350
    
351
    Returns:
352
    list: List of float values
353
    """
354

355
def backoff(start=1, **kwargs):
356
    """
357
    Generate exponential backoff delays.
358
    
359
    Parameters:
360
    - start (float): Initial delay value
361
    - factor (float): Multiplier for each step (default: 2)
362
    - max (float): Maximum delay value
363
    - jitter (bool): Add random jitter
364
    
365
    Returns:
366
    list: List of delay values
367
    """
368

369
def backoff_iter(start=1, **kwargs):
370
    """Iterator version of backoff."""
371
```
372

373
### Sorting Operations
374

375
Sort operations with error handling for mixed types.
376

377
```python { .api }
378
def soft_sorted(iterable, **kwargs):
379
    """
380
    Sort with soft error handling for incomparable types.
381
    
382
    Parameters:
383
    - iterable: Source iterable to sort
384
    - key (callable, optional): Function to compute sort key
385
    - reverse (bool): Sort in reverse order
386
    
387
    Returns:
388
    list: Sorted list with error handling
389
    """
390

391
def untyped_sorted(iterable, **kwargs):
392
    """
393
    Sort mixed types by converting to strings.
394
    
395
    Parameters:
396
    - iterable: Source iterable with mixed types
397
    - key (callable, optional): Function to compute sort key
398
    - reverse (bool): Sort in reverse order
399
    
400
    Returns:
401
    list: Sorted list with all types converted to strings
402
    """
403
```
404

405
### Type Checking
406

407
Utilities for checking iterable properties.
408

409
```python { .api }
410
def is_iterable(obj):
411
    """
412
    Check if object is iterable.
413
    
414
    Parameters:
415
    - obj: Object to check
416
    
417
    Returns:
418
    bool: True if object is iterable
419
    """
420

421
def is_scalar(obj):
422
    """
423
    Check if object is scalar (non-iterable).
424
    
425
    Parameters:
426
    - obj: Object to check
427
    
428
    Returns:
429
    bool: True if object is scalar
430
    """
431

432
def is_collection(obj):
433
    """
434
    Check if object is collection (iterable but not string/bytes).
435
    
436
    Parameters:
437
    - obj: Object to check
438
    
439
    Returns:
440
    bool: True if object is a collection
441
    """
442
```
443

444
## Usage Examples
445

446
```python
447
from boltons.iterutils import (
448
    chunked, windowed, flatten, unique, remap,
449
    get_path, backoff, soft_sorted
450
)
451

452
# Process data in chunks
453
data = list(range(100))
454
for chunk in chunked(data, 10):
455
    print(f"Processing {len(chunk)} items")
456

457
# Create sliding windows
458
sequence = [1, 2, 3, 4, 5]
459
windows = windowed(sequence, 3)
460
print(windows)  # [(1, 2, 3), (2, 3, 4), (3, 4, 5)]
461

462
# Flatten nested structures
463
nested = [[1, 2], [3, [4, 5]], 6]
464
flat = flatten(nested)
465
print(flat)  # [1, 2, 3, 4, 5, 6]
466

467
# Remove duplicates preserving order
468
items = [1, 2, 2, 3, 1, 4]
469
unique_items = unique(items)
470
print(unique_items)  # [1, 2, 3, 4]
471

472
# Transform nested data structures
473
data = {'a': [1, 2], 'b': {'c': 3}}
474
doubled = remap(data, visit=lambda path, key, value: 
475
                value * 2 if isinstance(value, int) else value)
476
print(doubled)  # {'a': [2, 4], 'b': {'c': 6}}
477

478
# Extract values by path
479
nested_dict = {'user': {'profile': {'name': 'Alice'}}}
480
name = get_path(nested_dict, ['user', 'profile', 'name'])
481
print(name)  # 'Alice'
482

483
# Generate exponential backoff delays
484
delays = backoff(start=0.1, factor=2, max=10)
485
print(delays)  # [0.1, 0.2, 0.4, 0.8, 1.6, 3.2, 6.4, 10, 10, ...]
486
```
487

488
## Types
489

490
```python { .api }
491
class PathAccessError(Exception):
492
    """Exception for invalid path access in nested structures."""
493
    pass
494

495
class GUIDerator:
496
    """Base iterator with guidance functionality."""
497
    def __init__(self, iterable): ...
498
    def __iter__(self): ...
499
    def __next__(self): ...
500

501
class SequentialGUIDerator(GUIDerator):
502
    """Sequential iterator with reseed capability."""
503
    def reseed(self, new_iterable): ...
504

505
# Sentinel values
506
_UNSET = object()  # Sentinel for unset values
507
```