0
# Utilities and Configuration
1
2
Utility functions, configuration management, logging, localization, process management, and string processing utilities that support the core framework functionality.
3
4
## Capabilities
5
6
### Configuration and Options
7
8
Command-line and configuration file parsing with type validation and automatic help generation.
9
10
```python { .api }
11
def define(name: str, default=None, type_=None, help: str = None, metavar: str = None, multiple: bool = False, group: str = None, callback=None):
12
"""
13
Define command-line option in global namespace.
14
15
Args:
16
name: Option name
17
default: Default value
18
type_: Option type (str, int, float, bool, datetime)
19
help: Help text
20
metavar: Metavar for help
21
multiple: Whether option can be specified multiple times
22
group: Option group name
23
callback: Callback when option is parsed
24
"""
25
26
def parse_command_line(args=None, final: bool = True):
27
"""
28
Parse global options from command line.
29
30
Args:
31
args: Command line arguments (default: sys.argv)
32
final: Whether this is final parse
33
"""
34
35
def parse_config_file(path: str, final: bool = True):
36
"""
37
Parse global options from config file.
38
39
Args:
40
path: Config file path
41
final: Whether this is final parse
42
"""
43
44
def print_help(file=None):
45
"""
46
Print command line options help.
47
48
Args:
49
file: Output file (default: sys.stdout)
50
"""
51
52
def add_parse_callback(callback):
53
"""
54
Add callback to be called after parsing.
55
56
Args:
57
callback: Callback function
58
"""
59
60
class OptionParser:
61
"""Collection of options with object-like access."""
62
63
def __init__(self):
64
"""Initialize option parser."""
65
66
def define(self, name: str, default=None, type_=None, help: str = None, metavar: str = None, multiple: bool = False, group: str = None, callback=None):
67
"""Define option in this parser."""
68
69
def parse_command_line(self, args=None, final: bool = True):
70
"""Parse command line arguments."""
71
72
def parse_config_file(self, path: str, final: bool = True):
73
"""Parse configuration file."""
74
75
def print_help(self, file=None):
76
"""Print help message."""
77
78
def add_parse_callback(self, callback):
79
"""Add parse callback."""
80
81
def mockable(self):
82
"""Return mockable version for testing."""
83
84
# Global options instance
85
options = OptionParser()
86
```
87
88
### String Escaping and Encoding
89
90
String processing utilities for HTML/XML escaping, URL encoding, JSON handling, and text formatting.
91
92
```python { .api }
93
def xhtml_escape(value) -> str:
94
"""
95
Escape string for HTML/XML output.
96
97
Args:
98
value: Value to escape
99
100
Returns:
101
Escaped string
102
"""
103
104
def xhtml_unescape(value: str) -> str:
105
"""
106
Unescape XML-escaped string.
107
108
Args:
109
value: Escaped string
110
111
Returns:
112
Unescaped string
113
"""
114
115
def url_escape(value: str, plus: bool = True) -> str:
116
"""
117
URL-encode value.
118
119
Args:
120
value: Value to encode
121
plus: Whether to use + for spaces
122
123
Returns:
124
URL-encoded string
125
"""
126
127
def url_unescape(value: str, encoding: str = 'utf-8', plus: bool = True) -> str:
128
"""
129
Decode URL-encoded value.
130
131
Args:
132
value: URL-encoded string
133
encoding: Character encoding
134
plus: Whether + represents spaces
135
136
Returns:
137
Decoded string
138
"""
139
140
def json_encode(value) -> str:
141
"""
142
JSON-encode Python object.
143
144
Args:
145
value: Object to encode
146
147
Returns:
148
JSON string
149
"""
150
151
def json_decode(value: str):
152
"""
153
Decode JSON string to Python object.
154
155
Args:
156
value: JSON string
157
158
Returns:
159
Decoded Python object
160
"""
161
162
def squeeze(value: str) -> str:
163
"""
164
Replace sequences of whitespace with single space.
165
166
Args:
167
value: String to process
168
169
Returns:
170
Squeezed string
171
"""
172
173
def utf8(value) -> bytes:
174
"""
175
Convert string to UTF-8 bytes.
176
177
Args:
178
value: String or bytes
179
180
Returns:
181
UTF-8 encoded bytes
182
"""
183
184
def to_unicode(value) -> str:
185
"""
186
Convert value to unicode string.
187
188
Args:
189
value: Value to convert
190
191
Returns:
192
Unicode string
193
"""
194
195
def recursive_unicode(obj):
196
"""
197
Convert data structure to unicode recursively.
198
199
Args:
200
obj: Object to convert
201
202
Returns:
203
Unicode version of object
204
"""
205
206
def linkify(text: str, shorten: bool = False, extra_params: str = "", require_protocol: bool = False, permitted_protocols: List[str] = None) -> str:
207
"""
208
Convert URLs in text to HTML links.
209
210
Args:
211
text: Text to process
212
shorten: Whether to shorten long URLs
213
extra_params: Extra link parameters
214
require_protocol: Whether to require protocol
215
permitted_protocols: Allowed protocols
216
217
Returns:
218
Text with HTML links
219
"""
220
221
def parse_qs_bytes(qs: bytes, keep_blank_values: bool = False, strict_parsing: bool = False) -> Dict[str, List[bytes]]:
222
"""
223
Parse query string as bytes.
224
225
Args:
226
qs: Query string bytes
227
keep_blank_values: Keep blank values
228
strict_parsing: Strict parsing mode
229
230
Returns:
231
Dict of parameter lists
232
"""
233
```
234
235
### Logging Utilities
236
237
Enhanced logging support with colored output and Tornado-specific formatting.
238
239
```python { .api }
240
def enable_pretty_logging(options=None, logger=None):
241
"""
242
Enable colored console logging.
243
244
Args:
245
options: Options object with logging settings
246
logger: Logger to configure (default: root logger)
247
"""
248
249
def define_logging_options(options=None):
250
"""
251
Define logging command-line options.
252
253
Args:
254
options: OptionParser to add options to
255
"""
256
257
class LogFormatter(logging.Formatter):
258
"""Log formatter with color support."""
259
260
def __init__(self, fmt: str = None, datefmt: str = None, style: str = '%', color: bool = True, colors: Dict[int, int] = None):
261
"""
262
Initialize formatter.
263
264
Args:
265
fmt: Log format string
266
datefmt: Date format string
267
style: Format style
268
color: Enable color output
269
colors: Color mapping for log levels
270
"""
271
272
def format(self, record) -> str:
273
"""Format log record."""
274
```
275
276
### Process Management
277
278
Multi-process utilities for forking processes and managing subprocesses.
279
280
```python { .api }
281
def cpu_count() -> int:
282
"""
283
Get number of processors.
284
285
Returns:
286
Number of CPU cores
287
"""
288
289
def fork_processes(num_processes: int, max_restarts: int = 100) -> int:
290
"""
291
Start multiple worker processes.
292
293
Args:
294
num_processes: Number of processes to fork
295
max_restarts: Maximum restart attempts
296
297
Returns:
298
Process ID (0 for children, None for parent)
299
"""
300
301
def task_id() -> int:
302
"""
303
Get current task ID (process number).
304
305
Returns:
306
Task ID starting from 0
307
"""
308
309
class Subprocess:
310
"""Async subprocess wrapper."""
311
312
def __init__(self, *args, **kwargs):
313
"""
314
Initialize subprocess.
315
316
Args:
317
*args: subprocess.Popen arguments
318
**kwargs: subprocess.Popen keyword arguments
319
"""
320
321
def set_exit_callback(self, callback):
322
"""
323
Set callback for process exit.
324
325
Args:
326
callback: Exit callback function
327
"""
328
329
async def wait_for_exit(self, raise_error: bool = True):
330
"""
331
Wait for process to exit.
332
333
Args:
334
raise_error: Raise exception on non-zero exit
335
336
Returns:
337
Exit code
338
"""
339
340
@classmethod
341
def initialize(cls, io_loop=None):
342
"""Initialize subprocess support."""
343
344
@classmethod
345
def uninitialize(cls):
346
"""Clean up subprocess support."""
347
```
348
349
### Locale and Internationalization
350
351
Internationalization support with CSV and gettext backends for translations and locale-aware formatting.
352
353
```python { .api }
354
def get(code: str):
355
"""
356
Get locale by code.
357
358
Args:
359
code: Locale code
360
361
Returns:
362
Locale object
363
"""
364
365
def set_default_locale(code: str):
366
"""
367
Set default locale.
368
369
Args:
370
code: Locale code
371
"""
372
373
def load_translations(directory: str, encoding: str = None):
374
"""
375
Load translations from CSV files.
376
377
Args:
378
directory: Translations directory
379
encoding: File encoding
380
"""
381
382
def load_gettext_translations(directory: str, domain: str):
383
"""
384
Load gettext translations.
385
386
Args:
387
directory: Translations directory
388
domain: Translation domain
389
"""
390
391
def get_supported_locales(locale_path: str) -> List[str]:
392
"""
393
Get list of supported locales.
394
395
Args:
396
locale_path: Path to locale files
397
398
Returns:
399
List of locale codes
400
"""
401
402
class Locale:
403
"""Locale object for translations and formatting."""
404
405
def __init__(self, code: str, translations: Dict[str, str]):
406
"""
407
Initialize locale.
408
409
Args:
410
code: Locale code
411
translations: Translation dictionary
412
"""
413
414
@classmethod
415
def get_closest(cls, *locale_codes):
416
"""Get closest matching locale."""
417
418
@classmethod
419
def get(cls, code: str):
420
"""Get locale by code."""
421
422
def translate(self, message: str, plural_message: str = None, count: int = None) -> str:
423
"""
424
Translate message.
425
426
Args:
427
message: Message to translate
428
plural_message: Plural form
429
count: Count for plural selection
430
431
Returns:
432
Translated message
433
"""
434
435
def pgettext(self, context: str, message: str, plural_message: str = None, count: int = None) -> str:
436
"""
437
Translate message with context.
438
439
Args:
440
context: Translation context
441
message: Message to translate
442
plural_message: Plural form
443
count: Count for plural selection
444
445
Returns:
446
Translated message
447
"""
448
449
def format_date(self, date, gmt_offset: int = 0, relative: bool = True, shorter: bool = False, full_format: bool = False) -> str:
450
"""Format date for locale."""
451
452
def format_day(self, date, gmt_offset: int = 0, dow: bool = True) -> str:
453
"""Format day for locale."""
454
455
def list(self, parts: List[str]) -> str:
456
"""Format list for locale."""
457
458
def friendly_number(self, value: int) -> str:
459
"""Format number in friendly way."""
460
461
@property
462
def code(self) -> str:
463
"""Get locale code."""
464
465
@property
466
def name(self) -> str:
467
"""Get locale name."""
468
```
469
470
### General Utilities
471
472
Miscellaneous utility functions and classes for object handling, imports, and data structures.
473
474
```python { .api }
475
def import_object(name: str):
476
"""
477
Import object by name.
478
479
Args:
480
name: Dotted object name
481
482
Returns:
483
Imported object
484
"""
485
486
def exec_in(code, glob, loc=None):
487
"""
488
Execute code in namespace.
489
490
Args:
491
code: Code to execute
492
glob: Global namespace
493
loc: Local namespace
494
"""
495
496
def raise_exc_info(exc_info):
497
"""
498
Raise exception from exc_info tuple.
499
500
Args:
501
exc_info: Exception info tuple
502
"""
503
504
def errno_from_exception(e) -> int:
505
"""
506
Get errno from exception.
507
508
Args:
509
e: Exception object
510
511
Returns:
512
Error number or None
513
"""
514
515
def re_unescape(s: str) -> str:
516
"""
517
Unescape regex string.
518
519
Args:
520
s: Escaped regex string
521
522
Returns:
523
Unescaped string
524
"""
525
526
def timedelta_to_seconds(td) -> float:
527
"""
528
Convert timedelta to seconds.
529
530
Args:
531
td: timedelta object
532
533
Returns:
534
Seconds as float
535
"""
536
537
class ObjectDict(dict):
538
"""Dictionary with attribute access."""
539
540
def __getattr__(self, name):
541
try:
542
return self[name]
543
except KeyError:
544
raise AttributeError(name)
545
546
def __setattr__(self, name, value):
547
self[name] = value
548
549
class GzipDecompressor:
550
"""Streaming gzip decompression."""
551
552
def __init__(self):
553
"""Initialize decompressor."""
554
555
def decompress(self, data: bytes, max_length: int = 0) -> bytes:
556
"""
557
Decompress data chunk.
558
559
Args:
560
data: Compressed data
561
max_length: Maximum output length
562
563
Returns:
564
Decompressed data
565
"""
566
567
@property
568
def unconsumed_tail(self) -> bytes:
569
"""Get unconsumed input data."""
570
571
@property
572
def eof(self) -> bool:
573
"""Check if end of stream reached."""
574
575
class Configurable:
576
"""Base class for configurable objects."""
577
578
@classmethod
579
def configurable_base(cls):
580
"""Get configurable base class."""
581
582
@classmethod
583
def configurable_default(cls):
584
"""Get default implementation class."""
585
586
@classmethod
587
def configure(cls, impl, **kwargs):
588
"""
589
Configure implementation class.
590
591
Args:
592
impl: Implementation class name or class
593
**kwargs: Configuration arguments
594
"""
595
596
@classmethod
597
def configured_class(cls):
598
"""Get currently configured class."""
599
600
class ArgReplacer:
601
"""Replace function arguments."""
602
603
def __init__(self, func, name: str):
604
"""
605
Initialize arg replacer.
606
607
Args:
608
func: Function to modify
609
name: Argument name to replace
610
"""
611
612
def get_old_value(self, args, kwargs):
613
"""Get old argument value."""
614
615
def replace(self, new_value, args, kwargs):
616
"""Replace argument with new value."""
617
```
618
619
## Usage Examples
620
621
### Command-line Options
622
623
```python
624
import tornado.options
625
626
# Define options
627
tornado.options.define("port", default=8888, help="run on the given port", type=int)
628
tornado.options.define("debug", default=False, help="run in debug mode", type=bool)
629
tornado.options.define("config", help="config file path")
630
631
# Parse command line
632
tornado.options.parse_command_line()
633
634
# Use options
635
print(f"Starting server on port {tornado.options.options.port}")
636
if tornado.options.options.debug:
637
print("Debug mode enabled")
638
639
# Parse config file if specified
640
if tornado.options.options.config:
641
tornado.options.parse_config_file(tornado.options.options.config)
642
```
643
644
### String Processing
645
646
```python
647
import tornado.escape
648
649
# HTML escaping
650
html = tornado.escape.xhtml_escape('<script>alert("xss")</script>')
651
print(html) # <script>alert("xss")</script>
652
653
# URL encoding
654
url = tornado.escape.url_escape("hello world")
655
print(url) # hello%20world
656
657
# JSON handling
658
data = {"name": "John", "age": 30}
659
json_str = tornado.escape.json_encode(data)
660
parsed = tornado.escape.json_decode(json_str)
661
662
# Linkify URLs
663
text = "Visit https://www.example.com for more info"
664
linked = tornado.escape.linkify(text)
665
print(linked) # Visit <a href="https://www.example.com">https://www.example.com</a> for more info
666
```
667
668
### Process Management
669
670
```python
671
import tornado.process
672
import tornado.web
673
import tornado.ioloop
674
675
def main():
676
# Fork multiple processes
677
tornado.process.fork_processes(0) # 0 = number of CPUs
678
679
app = tornado.web.Application([
680
(r"/", MainHandler),
681
])
682
683
app.listen(8888)
684
tornado.ioloop.IOLoop.current().start()
685
686
if __name__ == "__main__":
687
main()
688
```
689
690
### Internationalization
691
692
```python
693
import tornado.locale
694
695
# Load translations
696
tornado.locale.load_translations("locale")
697
698
# Get locale
699
locale = tornado.locale.get("en_US")
700
701
# Translate messages
702
message = locale.translate("Hello, world!")
703
plural = locale.translate("You have %(count)d messages",
704
"You have %(count)d messages",
705
count=5)
706
707
# Format dates
708
import datetime
709
date_str = locale.format_date(datetime.datetime.now())
710
```
711
712
### Utility Objects
713
714
```python
715
import tornado.util
716
717
# Object dictionary
718
obj = tornado.util.ObjectDict()
719
obj.name = "John"
720
obj.age = 30
721
print(obj.name) # Attribute access
722
print(obj['age']) # Dictionary access
723
724
# Import by name
725
cls = tornado.util.import_object("tornado.web.RequestHandler")
726
727
# Gzip decompression
728
decompressor = tornado.util.GzipDecompressor()
729
chunk1 = decompressor.decompress(compressed_data)
730
```
731
732
## Types
733
734
```python { .api }
735
# Option types
736
OptionType = Union[type, Callable[[str], Any]]
737
OptionValue = Any
738
OptionDict = Dict[str, OptionValue]
739
740
# Locale types
741
LocaleCode = str
742
TranslationDict = Dict[str, str]
743
744
# Process types
745
ProcessId = int
746
ExitCode = int
747
748
# Utility types
749
ImportPath = str
750
ExcInfo = Tuple[type, Exception, Any]
751
```
752
753
## Exceptions
754
755
```python { .api }
756
class Error(Exception):
757
"""Base exception for options module."""
758
759
class ConfigError(Error):
760
"""Exception for configuration errors."""
761
762
class LocaleNotFoundError(Exception):
763
"""Exception when locale is not found."""
764
765
def __init__(self, code: str):
766
"""
767
Initialize locale not found error.
768
769
Args:
770
code: Locale code that was not found
771
"""
772
773
class ProcessExitError(Exception):
774
"""Exception for subprocess exit errors."""
775
776
def __init__(self, exit_code: int):
777
"""
778
Initialize process exit error.
779
780
Args:
781
exit_code: Process exit code
782
"""
783
```