0
# Utilities
1
2
Helper functions for memory alignment and system detection, plus utility classes for window management and memory region tracking. These components support the core memory mapping functionality with cross-platform compatibility and resource management.
3
4
## Capabilities
5
6
### Memory Alignment
7
8
Function for aligning memory offsets and sizes to system page boundaries, required for efficient memory mapping operations.
9
10
```python { .api }
11
def align_to_mmap(num, round_up):
12
"""
13
Align integer to closest memory page boundary.
14
15
Parameters:
16
- num (int): Number to align to page boundary
17
- round_up (bool): If True, round up to next page boundary.
18
If False, round down to previous page boundary.
19
20
Returns:
21
int: Number aligned to page boundary (typically 4096-byte multiples)
22
23
Example:
24
- align_to_mmap(1, True) returns 4096
25
- align_to_mmap(1, False) returns 0
26
- align_to_mmap(4096, True) returns 4096
27
- align_to_mmap(4097, False) returns 4096
28
"""
29
```
30
31
#### Usage Example
32
33
```python
34
import smmap
35
36
# Align offsets for optimal memory mapping
37
offset = 1234
38
size = 5678
39
40
# Align start offset down to page boundary
41
aligned_offset = smmap.align_to_mmap(offset, False)
42
print(f"Original: {offset}, Aligned: {aligned_offset}") # Original: 1234, Aligned: 0
43
44
# Adjust size to maintain coverage and align to page boundary
45
adjusted_size = size + (offset - aligned_offset)
46
aligned_size = smmap.align_to_mmap(adjusted_size, True)
47
print(f"Adjusted size: {aligned_size}") # Aligned to next page boundary
48
49
# Use aligned values for memory mapping
50
cursor.use_region(offset=aligned_offset, size=aligned_size)
51
```
52
53
### System Architecture Detection
54
55
Function to detect system architecture for optimizing memory management strategies.
56
57
```python { .api }
58
def is_64_bit():
59
"""
60
Detect if system is 64-bit architecture.
61
62
Returns:
63
bool: True if 64-bit system, False if 32-bit system
64
65
Note: Used internally to select appropriate default window sizes
66
and memory limits. 64-bit systems can handle larger memory mappings.
67
"""
68
```
69
70
#### Usage Example
71
72
```python
73
import smmap
74
75
# Configure memory manager based on architecture
76
if smmap.is_64_bit():
77
# 64-bit: Use larger windows and memory limits
78
window_size = 1024 * 1024 * 1024 # 1GB windows
79
max_memory = 8 * 1024 * 1024 * 1024 # 8GB limit
80
else:
81
# 32-bit: Use conservative settings
82
window_size = 64 * 1024 * 1024 # 64MB windows
83
max_memory = 1 * 1024 * 1024 * 1024 # 1GB limit
84
85
manager = smmap.SlidingWindowMapManager(
86
window_size=window_size,
87
max_memory_size=max_memory
88
)
89
```
90
91
### Window Management
92
93
Utility class for calculating and adjusting memory window positions and sizes.
94
95
```python { .api }
96
class MapWindow:
97
def __init__(self, offset, size):
98
"""
99
Initialize memory window specification.
100
101
Parameters:
102
- offset (int): Offset into file in bytes
103
- size (int): Window size in bytes
104
"""
105
106
def ofs_end(self):
107
"""
108
Get end offset of window.
109
110
Returns:
111
int: Offset to byte after window end
112
"""
113
114
def align(self):
115
"""
116
Align window to memory page boundaries.
117
118
Adjusts offset down to page boundary and size up to
119
maintain coverage while ensuring page alignment.
120
"""
121
122
def extend_left_to(self, window, max_size):
123
"""
124
Extend window leftward toward another window.
125
126
Parameters:
127
- window (MapWindow): Left neighboring window
128
- max_size (int): Maximum allowed window size
129
130
Adjusts this window to start where the left window ends,
131
but doesn't exceed max_size. Maintains coverage of original area.
132
"""
133
134
def extend_right_to(self, window, max_size):
135
"""
136
Extend window rightward toward another window.
137
138
Parameters:
139
- window (MapWindow): Right neighboring window
140
- max_size (int): Maximum allowed window size
141
142
Adjusts this window to end where the right window begins,
143
but doesn't exceed max_size.
144
"""
145
146
@classmethod
147
def from_region(cls, region):
148
"""
149
Create window from existing memory region.
150
151
Parameters:
152
- region (MapRegion): Existing mapped region
153
154
Returns:
155
MapWindow: Window specification matching the region
156
"""
157
```
158
159
### Memory Region Management
160
161
Low-level class representing a mapped memory region with automatic resource cleanup.
162
163
```python { .api }
164
class MapRegion:
165
def __init__(self, path_or_fd, ofs, size, flags=0):
166
"""
167
Initialize memory-mapped region.
168
169
Parameters:
170
- path_or_fd (str | int): File path or open file descriptor
171
- ofs (int): Aligned offset into file (must be page-aligned)
172
- size (int): Size to map. If larger than file, maps to end of file.
173
- flags (int): Additional flags for os.open
174
175
Raises:
176
Exception: If memory cannot be allocated or file cannot be opened
177
"""
178
179
def buffer(self):
180
"""
181
Get memory buffer for mapped region.
182
183
Returns:
184
mmap.mmap: Memory map buffer object
185
"""
186
187
def map(self):
188
"""
189
Get memory map object.
190
191
Returns:
192
mmap.mmap: Same as buffer() - the underlying memory map
193
"""
194
195
def ofs_begin(self):
196
"""
197
Get absolute offset to first byte of mapping.
198
199
Returns:
200
int: File offset in bytes
201
"""
202
203
def ofs_end(self):
204
"""
205
Get absolute offset to byte after mapping end.
206
207
Returns:
208
int: File offset in bytes
209
"""
210
211
def size(self):
212
"""
213
Get size of mapped region.
214
215
Returns:
216
int: Size in bytes
217
"""
218
219
def includes_ofs(self, ofs):
220
"""
221
Check if offset is within mapped region.
222
223
Parameters:
224
- ofs (int): Absolute file offset to check
225
226
Returns:
227
bool: True if offset is accessible through this region
228
"""
229
230
def client_count(self):
231
"""
232
Get number of clients using this region.
233
234
Returns:
235
int: Reference count
236
"""
237
238
def increment_client_count(self, ofs=1):
239
"""
240
Adjust client reference count.
241
242
Parameters:
243
- ofs (int): Increment (positive) or decrement (negative)
244
245
Returns:
246
bool: True if region was released (count reached 0)
247
"""
248
249
def release(self):
250
"""
251
Release mapped memory resources.
252
253
Note: Should only be called when client_count() is zero.
254
Called automatically when reference count reaches zero.
255
"""
256
```
257
258
### Region List Management
259
260
Container class for managing multiple memory regions associated with a single file.
261
262
```python { .api }
263
class MapRegionList(list):
264
def __init__(self, path_or_fd):
265
"""
266
Initialize region list for a file.
267
268
Parameters:
269
- path_or_fd (str | int): File path or descriptor
270
"""
271
272
def path_or_fd(self):
273
"""
274
Get file path or descriptor for this region list.
275
276
Returns:
277
str | int: File path or file descriptor
278
"""
279
280
def file_size(self):
281
"""
282
Get total size of associated file.
283
284
Returns:
285
int: File size in bytes
286
287
Note: Size is cached after first access for performance.
288
"""
289
```
290
291
## System Constants
292
293
```python { .api }
294
ALLOCATIONGRANULARITY: int
295
```
296
297
System memory allocation granularity constant imported from Python's mmap module. Represents the page size used for memory alignment (typically 4096 bytes on most systems, 65536 on some Windows systems).
298
299
#### Usage Example
300
301
```python
302
import smmap
303
304
print(f"System page size: {smmap.ALLOCATIONGRANULARITY} bytes")
305
306
# Use for manual alignment calculations
307
offset = 12345
308
aligned_offset = (offset // smmap.ALLOCATIONGRANULARITY) * smmap.ALLOCATIONGRANULARITY
309
print(f"Manual alignment: {offset} -> {aligned_offset}")
310
311
# Or use the convenience function
312
aligned_offset = smmap.align_to_mmap(offset, False)
313
print(f"Function alignment: {offset} -> {aligned_offset}")
314
```
315
316
## Advanced Usage Patterns
317
318
### Custom Window Management
319
320
```python
321
import smmap
322
323
# Create custom window layout for optimal access
324
window1 = smmap.MapWindow(0, 1024*1024) # 1MB at start
325
window2 = smmap.MapWindow(10*1024*1024, 512*1024) # 512KB at 10MB
326
327
# Align windows for efficient mapping
328
window1.align()
329
window2.align()
330
331
# Create gap-filling window
332
gap_window = smmap.MapWindow(window1.ofs_end(),
333
window2.ofs_begin() - window1.ofs_end())
334
gap_window.align()
335
336
print(f"Window 1: {window1.ofs} - {window1.ofs_end()}")
337
print(f"Gap: {gap_window.ofs} - {gap_window.ofs_end()}")
338
print(f"Window 2: {window2.ofs} - {window2.ofs_end()}")
339
```
340
341
### Region Information
342
343
```python
344
# Access low-level region information
345
cursor = manager.make_cursor('/path/to/file.dat')
346
cursor.use_region(offset=0, size=1024*1024)
347
348
if cursor.is_valid():
349
region = cursor.region()
350
print(f"Region covers: {region.ofs_begin()} to {region.ofs_end()}")
351
print(f"Region size: {region.size()} bytes")
352
print(f"Client count: {region.client_count()}")
353
print(f"Includes offset 512000: {region.includes_ofs(512000)}")
354
```
355
356
### Architecture-Aware Configuration
357
358
```python
359
import smmap
360
import sys
361
362
# Configure based on system capabilities
363
def create_optimized_manager():
364
if smmap.is_64_bit():
365
# 64-bit: aggressive settings
366
return smmap.StaticWindowMapManager(
367
max_memory_size=8 * 1024**3, # 8GB
368
max_open_handles=1000
369
)
370
else:
371
# 32-bit: conservative settings
372
return smmap.SlidingWindowMapManager(
373
window_size=64 * 1024**2, # 64MB windows
374
max_memory_size=1 * 1024**3, # 1GB total
375
max_open_handles=100
376
)
377
378
# Platform-specific optimizations
379
manager = create_optimized_manager()
380
print(f"Created {type(manager).__name__} for {sys.platform}")
381
print(f"64-bit system: {smmap.is_64_bit()}")
382
print(f"Page size: {smmap.ALLOCATIONGRANULARITY} bytes")
383
```