0
# Built-in Collectors
1

2
Pre-built collectors for system metrics including process statistics, garbage collection data, and platform information. These collectors can be automatically registered to provide comprehensive monitoring of application and system health.
3

4
## Capabilities
5

6
### ProcessCollector
7

8
Collects system process metrics including CPU usage, memory consumption, file descriptors, and other process-level statistics. Useful for monitoring application resource usage.
9

10
```python { .api }
11
class ProcessCollector:
12
    def __init__(
13
        self, 
14
        namespace: str = '', 
15
        pid: Callable = lambda: 'self', 
16
        proc: str = '/proc', 
17
        registry: Optional[CollectorRegistry] = REGISTRY
18
    ) -> None:
19
        """
20
        Create a ProcessCollector.
21
        
22
        Parameters:
23
        - namespace: Metric namespace prefix
24
        - pid: Function returning process ID ('self' for current process, or callable returning PID)
25
        - proc: Path to /proc filesystem (for Linux/Unix systems)
26
        - registry: Registry to register with (None to skip auto-registration)
27
        """
28
    
29
    def collect(self) -> Iterable[Metric]:
30
        """
31
        Collect process metrics.
32
        
33
        Returns:
34
        Iterable of Metric objects containing process statistics
35
        """
36

37
PROCESS_COLLECTOR: ProcessCollector = ProcessCollector()
38
```
39

40
**Metrics Provided:**
41
- `process_cpu_seconds_total` - Total user and system CPU time
42
- `process_open_fds` - Number of open file descriptors  
43
- `process_max_fds` - Maximum number of open file descriptors
44
- `process_virtual_memory_bytes` - Virtual memory size in bytes
45
- `process_resident_memory_bytes` - Resident memory size in bytes
46
- `process_start_time_seconds` - Start time of the process since Unix epoch
47

48
**Usage Example:**
49

50
```python
51
from prometheus_client import CollectorRegistry, generate_latest
52
from prometheus_client.process_collector import ProcessCollector
53

54
# Use default process collector (already registered with REGISTRY)
55
from prometheus_client import REGISTRY
56
print(generate_latest(REGISTRY).decode('utf-8'))
57

58
# Create custom process collector
59
custom_registry = CollectorRegistry()
60
process_collector = ProcessCollector(
61
    namespace='myapp',
62
    registry=custom_registry
63
)
64

65
# Metrics will be prefixed with 'myapp_'
66
output = generate_latest(custom_registry)
67
print(output.decode('utf-8'))
68

69
# Monitor specific process
70
import os
71
def get_child_pid():
72
    return str(child_process.pid)
73

74
child_collector = ProcessCollector(
75
    namespace='child',
76
    pid=get_child_pid,
77
    registry=custom_registry
78
)
79

80
# Disable auto-registration
81
unregistered_collector = ProcessCollector(registry=None)
82
# Manually register later if needed
83
custom_registry.register(unregistered_collector)
84
```
85

86
### GCCollector
87

88
Collects Python garbage collection statistics including collection counts and timing information for each generation. Helps monitor memory management performance.
89

90
```python { .api }
91
class GCCollector:
92
    def __init__(self, registry: CollectorRegistry = REGISTRY) -> None:
93
        """
94
        Create a GCCollector.
95
        
96
        Parameters:
97
        - registry: Registry to register with (None to skip auto-registration)
98
        """
99
    
100
    def collect(self) -> Iterable[Metric]:
101
        """
102
        Collect garbage collection metrics.
103
        
104
        Returns:
105
        Iterable of Metric objects containing GC statistics
106
        """
107

108
GC_COLLECTOR: GCCollector = GCCollector()
109
```
110

111
**Metrics Provided:**
112
- `python_gc_objects_collected_total` - Objects collected during GC by generation
113
- `python_gc_objects_uncollectable_total` - Uncollectable objects found by GC by generation
114
- `python_gc_collections_total` - Number of times this generation was collected
115

116
**Usage Example:**
117

118
```python
119
from prometheus_client import CollectorRegistry, generate_latest
120
from prometheus_client.gc_collector import GCCollector
121
import gc
122

123
# Use default GC collector (already registered with REGISTRY)
124
from prometheus_client import REGISTRY
125

126
# Force some garbage collection activity
127
data = []
128
for i in range(10000):
129
    data.append({'id': i, 'data': list(range(100))})
130
del data
131
gc.collect()
132

133
# Check GC metrics
134
output = generate_latest(REGISTRY)
135
print(output.decode('utf-8'))
136

137
# Create custom GC collector
138
custom_registry = CollectorRegistry()
139
gc_collector = GCCollector(registry=custom_registry)
140

141
# Monitor GC with custom namespace
142
class NamespacedGCCollector(GCCollector):
143
    def __init__(self, namespace='', registry=REGISTRY):
144
        self.namespace = namespace
145
        super().__init__(registry=None)  # Don't auto-register
146
        if registry:
147
            registry.register(self)
148
    
149
    def collect(self):
150
        for metric in super().collect():
151
            # Add namespace to metric name
152
            if self.namespace:
153
                metric.name = f"{self.namespace}_{metric.name}"
154
            yield metric
155

156
app_gc_collector = NamespacedGCCollector(namespace='myapp', registry=custom_registry)
157
```
158

159
### PlatformCollector
160

161
Collects Python platform and interpreter information including version, implementation, and build details. Useful for environment monitoring and debugging.
162

163
```python { .api }
164
class PlatformCollector:
165
    def __init__(
166
        self, 
167
        registry: Optional[CollectorRegistry] = REGISTRY, 
168
        platform=None
169
    ) -> None:
170
        """
171
        Create a PlatformCollector.
172
        
173
        Parameters:
174
        - registry: Registry to register with (None to skip auto-registration)
175
        - platform: Platform module to use (defaults to built-in platform module)
176
        """
177
    
178
    def collect(self) -> Iterable[Metric]:
179
        """
180
        Collect platform information metrics.
181
        
182
        Returns:
183
        Iterable of Metric objects containing platform information
184
        """
185

186
PLATFORM_COLLECTOR: PlatformCollector = PlatformCollector()
187
```
188

189
**Metrics Provided:**
190
- `python_info` - Python platform information (version, implementation, etc.)
191

192
**Usage Example:**
193

194
```python
195
from prometheus_client import CollectorRegistry, generate_latest
196
from prometheus_client.platform_collector import PlatformCollector
197

198
# Use default platform collector (already registered with REGISTRY)
199
from prometheus_client import REGISTRY
200
output = generate_latest(REGISTRY)
201
print(output.decode('utf-8'))
202

203
# Example output includes:
204
# python_info{implementation="CPython",major="3",minor="9",patchlevel="7",version="3.9.7"} 1.0
205

206
# Create custom platform collector
207
custom_registry = CollectorRegistry()
208
platform_collector = PlatformCollector(registry=custom_registry)
209

210
# Custom platform information for testing
211
class MockPlatform:
212
    def python_version_tuple(self):
213
        return ('3', '10', '0')
214
    
215
    def python_implementation(self):
216
        return 'PyPy'
217

218
test_collector = PlatformCollector(
219
    registry=custom_registry,
220
    platform=MockPlatform()
221
)
222
```
223

224
### Collector Registration and Management
225

226
Utilities for managing built-in collectors in custom registries.
227

228
**Usage Example:**
229

230
```python
231
from prometheus_client import CollectorRegistry
232
from prometheus_client.process_collector import ProcessCollector, PROCESS_COLLECTOR
233
from prometheus_client.gc_collector import GCCollector, GC_COLLECTOR
234
from prometheus_client.platform_collector import PlatformCollector, PLATFORM_COLLECTOR
235

236
# Create registry without default collectors
237
custom_registry = CollectorRegistry()
238

239
# Add only specific collectors
240
ProcessCollector(registry=custom_registry)
241
GCCollector(registry=custom_registry)
242
# Skip platform collector for this registry
243

244
# Remove collectors from default registry if needed
245
from prometheus_client import REGISTRY
246
REGISTRY.unregister(PROCESS_COLLECTOR)
247
REGISTRY.unregister(GC_COLLECTOR)
248
REGISTRY.unregister(PLATFORM_COLLECTOR)
249

250
# Re-register with custom settings
251
ProcessCollector(namespace='app', registry=REGISTRY)
252
GCCollector(registry=REGISTRY)
253
PlatformCollector(registry=REGISTRY)
254

255
# Check what's registered
256
collectors = []
257
for metric in REGISTRY.collect():
258
    collectors.append(metric.name)
259
print("Registered metrics:", collectors)
260
```
261

262
### Combined Collector Usage
263

264
Example of using all built-in collectors together for comprehensive system monitoring.
265

266
**Usage Example:**
267

268
```python
269
from prometheus_client import (
270
    CollectorRegistry, Counter, Gauge, generate_latest,
271
    start_http_server
272
)
273
from prometheus_client.process_collector import ProcessCollector  
274
from prometheus_client.gc_collector import GCCollector
275
from prometheus_client.platform_collector import PlatformCollector
276
import time
277
import threading
278
import random
279

280
# Create application registry with all system collectors
281
app_registry = CollectorRegistry()
282

283
# Add system collectors with namespace
284
ProcessCollector(namespace='myapp', registry=app_registry)
285
GCCollector(registry=app_registry) 
286
PlatformCollector(registry=app_registry)
287

288
# Add application metrics
289
request_count = Counter(
290
    'myapp_requests_total', 
291
    'Total application requests',
292
    ['endpoint'],
293
    registry=app_registry
294
)
295

296
active_connections = Gauge(
297
    'myapp_active_connections',
298
    'Active client connections', 
299
    registry=app_registry
300
)
301

302
# Start metrics server
303
start_http_server(8000, registry=app_registry)
304

305
# Simulate application activity
306
def app_activity():
307
    endpoints = ['/api/users', '/api/orders', '/health']
308
    
309
    while True:
310
        # Simulate requests
311
        endpoint = random.choice(endpoints)
312
        request_count.labels(endpoint=endpoint).inc()
313
        
314
        # Simulate connection changes
315
        active_connections.set(random.randint(10, 100))
316
        
317
        # Create some garbage to trigger GC
318
        temp_data = [list(range(1000)) for _ in range(10)]
319
        del temp_data
320
        
321
        time.sleep(1)
322

323
# Run application simulation
324
app_thread = threading.Thread(target=app_activity, daemon=True)
325
app_thread.start()
326

327
print("Application metrics server running on http://localhost:8000")
328
print("Metrics include:")
329
print("- Application metrics (requests, connections)")  
330
print("- Process metrics (CPU, memory, file descriptors)")
331
print("- Python GC metrics (collections, objects)")
332
print("- Platform info (Python version, implementation)")
333

334
try:
335
    while True:
336
        time.sleep(10)
337
        print(f"Current metrics count: {len(list(app_registry.collect()))}")
338
except KeyboardInterrupt:
339
    print("Shutting down...")
340
```
341

342
### Custom Collector Integration
343

344
How to create custom collectors that work alongside built-in collectors.
345

346
**Usage Example:**
347

348
```python
349
from prometheus_client import Collector, Metric, CollectorRegistry
350
from prometheus_client.process_collector import ProcessCollector
351
import psutil
352
import time
353

354
class SystemCollector(Collector):
355
    """Custom collector for additional system metrics."""
356
    
357
    def collect(self):
358
        # Disk usage
359
        disk_metric = Metric('system_disk_usage_bytes', 'Disk usage by mount point', 'gauge')
360
        for partition in psutil.disk_partitions():
361
            try:
362
                usage = psutil.disk_usage(partition.mountpoint)
363
                disk_metric.add_sample(
364
                    'system_disk_usage_bytes',
365
                    {'mountpoint': partition.mountpoint, 'type': 'used'},
366
                    usage.used
367
                )
368
                disk_metric.add_sample(
369
                    'system_disk_usage_bytes', 
370
                    {'mountpoint': partition.mountpoint, 'type': 'free'},
371
                    usage.free
372
                )
373
            except PermissionError:
374
                continue
375
        yield disk_metric
376
        
377
        # Network I/O
378
        net_io = psutil.net_io_counters()
379
        net_metric = Metric('system_network_bytes_total', 'Network I/O bytes', 'counter')
380
        net_metric.add_sample('system_network_bytes_total', {'direction': 'sent'}, net_io.bytes_sent)
381
        net_metric.add_sample('system_network_bytes_total', {'direction': 'received'}, net_io.bytes_recv)
382
        yield net_metric
383

384
# Create comprehensive monitoring registry
385
monitoring_registry = CollectorRegistry()
386

387
# Add built-in collectors
388
ProcessCollector(namespace='app', registry=monitoring_registry)
389
GCCollector(registry=monitoring_registry)
390
PlatformCollector(registry=monitoring_registry)
391

392
# Add custom collector
393
system_collector = SystemCollector()
394
monitoring_registry.register(system_collector)
395

396
# Generate comprehensive metrics
397
output = generate_latest(monitoring_registry)
398
print("Comprehensive system metrics:")
399
print(output.decode('utf-8'))
400
```