0
# Plugin System
1

2
Nuitka's plugin system provides an extensible framework for handling third-party packages, data files, and special compilation cases. The system enables custom plugins to integrate with the compilation pipeline through base classes, lifecycle hooks, and configuration management.
3

4
## Capabilities
5

6
### Base Plugin Classes
7

8
Foundation classes for creating Nuitka plugins with standardized interfaces and lifecycle management.
9

10
```python { .api }
11
class NuitkaPluginBase:
12
    """
13
    Base class for all Nuitka plugins.
14
    
15
    Provides the foundation interface and lifecycle hooks for plugins
16
    that need to interact with the compilation process. All custom
17
    plugins should inherit from this class.
18
    
19
    Attributes:
20
        plugin_name (str): Unique identifier for the plugin
21
        plugin_desc (str): Human-readable description
22
        
23
    Methods:
24
        Various lifecycle hooks called during compilation phases
25
    """
26

27
class NuitkaYamlPluginBase:
28
    """
29
    Base class for YAML-configured plugins.
30
    
31
    Extends NuitkaPluginBase to support plugins that are configured
32
    through YAML files rather than Python code. Useful for simpler
33
    plugins that primarily specify data files and dependencies.
34
    
35
    Attributes:
36
        yaml_config (dict): Configuration loaded from YAML file
37
        config_file (str): Path to YAML configuration file
38
    """
39

40
class Plugins:
41
    """
42
    Plugin management and coordination system.
43
    
44
    Central registry and coordinator for all active plugins. Manages
45
    plugin discovery, loading, activation, and lifecycle execution.
46
    
47
    Methods:
48
        activatePlugins: Enable and initialize plugins
49
        getPlugins: Get list of active plugins
50
        callPluginMethod: Invoke plugin lifecycle methods
51
    """
52
```
53

54
**Usage Example:**
55

56
```python
57
from nuitka.plugins.PluginBase import NuitkaPluginBase
58

59
class MyCustomPlugin(NuitkaPluginBase):
60
    """Custom plugin for handling special package requirements."""
61
    
62
    plugin_name = "my-custom-plugin"
63
    plugin_desc = "Handles custom package compilation requirements"
64
    
65
    def onModuleDiscovered(self, module):
66
        """Called when a module is discovered during compilation."""
67
        if module.getFullName() == "my_special_package":
68
            # Add special handling for this package
69
            return True
70
        return False
71
    
72
    def getImplicitImports(self, module):
73
        """Specify additional modules that should be imported."""
74
        if module.getFullName() == "my_package":
75
            return ["hidden_dependency", "runtime_module"]
76
        return []
77
    
78
    def onDataFiles(self, module):
79
        """Specify data files that should be included."""
80
        if module.getFullName() == "my_package":
81
            return [
82
                ("data/config.json", "my_package/data/config.json"),
83
                ("templates/", "my_package/templates/"),
84
            ]
85
        return []
86
```
87

88
### Plugin Lifecycle Hooks
89

90
Methods called at different stages of the compilation process to allow plugin customization.
91

92
```python { .api }
93
def onModuleDiscovered(self, module):
94
    """
95
    Called when a module is discovered during compilation.
96
    
97
    Allows plugins to react to module discovery and provide
98
    special handling for specific modules or packages.
99
    
100
    Args:
101
        module: Module object being discovered
102
        
103
    Returns:
104
        bool: True if plugin handles this module specially
105
    """
106

107
def getImplicitImports(self, module):
108
    """
109
    Specify additional modules that should be imported.
110
    
111
    Called to determine hidden dependencies that are not
112
    detected by static analysis but are required at runtime.
113
    
114
    Args:
115
        module: Module being analyzed
116
        
117
    Returns:
118
        list: Module names to implicitly import
119
    """
120

121
def onDataFiles(self, module):
122
    """
123
    Specify data files that should be included.
124
    
125
    Called to determine which data files, templates, and
126
    resources should be packaged with the compiled output.
127
    
128
    Args:
129
        module: Module being processed
130
        
131
    Returns:
132
        list: Tuples of (source_path, target_path) for data files
133
    """
134

135
def onCodeGeneration(self, module, code_context):
136
    """
137
    Customize generated C code.
138
    
139
    Called during C code generation to allow plugins to
140
    modify or enhance the generated code.
141
    
142
    Args:
143
        module: Module being compiled
144
        code_context: Code generation context
145
        
146
    Returns:
147
        str: Additional C code or modifications
148
    """
149
```
150

151
**Usage Example:**
152

153
```python
154
from nuitka.plugins.PluginBase import NuitkaPluginBase
155

156
class DataHandlingPlugin(NuitkaPluginBase):
157
    """Plugin that handles data file inclusion for packages."""
158
    
159
    plugin_name = "data-handler"
160
    
161
    def onModuleDiscovered(self, module):
162
        """Check if module needs special data file handling."""
163
        special_modules = ["matplotlib", "django", "flask"]
164
        return module.getFullName().split('.')[0] in special_modules
165
    
166
    def getImplicitImports(self, module):
167
        """Add hidden dependencies for special modules."""
168
        module_name = module.getFullName()
169
        
170
        if module_name.startswith("matplotlib"):
171
            return ["matplotlib.backends._backend_agg"]
172
        elif module_name.startswith("django"):
173
            return ["django.template.loaders.filesystem"]
174
        
175
        return []
176
    
177
    def onDataFiles(self, module):
178
        """Include necessary data files."""
179
        module_name = module.getFullName()
180
        data_files = []
181
        
182
        if module_name == "matplotlib":
183
            data_files.extend([
184
                ("mpl-data/", "matplotlib/mpl-data/"),
185
                ("fonts/", "matplotlib/fonts/"),
186
            ])
187
        elif module_name == "django":
188
            data_files.extend([
189
                ("templates/", "django/templates/"),
190
                ("static/", "django/contrib/admin/static/"),
191
            ])
192
        
193
        return data_files
194
```
195

196
### Plugin Configuration
197

198
Configure plugin behavior through YAML files and runtime options.
199

200
```python { .api }
201
class ConfigurablePlugin(NuitkaYamlPluginBase):
202
    """
203
    Plugin configured through YAML files.
204
    
205
    Loads configuration from YAML files to determine behavior
206
    without requiring code changes. Suitable for plugins that
207
    primarily manage data files and dependencies.
208
    
209
    YAML format:
210
        plugin_name: "my-yaml-plugin"
211
        modules:
212
          - name: "package_name"
213
            implicit_imports: ["hidden_dep1", "hidden_dep2"]
214
            data_files:
215
              - ["source/path", "target/path"]
216
    """
217
```
218

219
**YAML Configuration Example:**
220

221
```yaml
222
# my_plugin.yml
223
plugin_name: "custom-package-handler"
224
plugin_desc: "Handles custom package compilation"
225

226
modules:
227
  - name: "scientific_package"
228
    implicit_imports:
229
      - "scipy.sparse.csgraph._validation"
230
      - "numpy.random._pickle"
231
    data_files:
232
      - ["data/coefficients.dat", "scientific_package/data/coefficients.dat"]
233
      - ["config/", "scientific_package/config/"]
234
    
235
  - name: "web_framework"
236
    implicit_imports:
237
      - "jinja2.ext"
238
    data_files:
239
      - ["templates/", "web_framework/templates/"]
240
      - ["static/css/", "web_framework/static/css/"]
241
```
242

243
## Standard Plugins
244

245
### Built-in Plugin Examples
246

247
Nuitka includes many standard plugins for popular packages.
248

249
```python
250
# Examples of standard plugin usage
251
from nuitka.plugins.Plugins import activatePlugins
252

253
# Standard plugins are activated via command line or configuration
254
plugins_to_activate = [
255
    "numpy",          # NumPy scientific computing
256
    "tk-inter",       # Tkinter GUI framework  
257
    "qt-plugins",     # Qt application framework
258
    "matplotlib",     # Matplotlib plotting library
259
    "django",         # Django web framework
260
    "multiprocessing",# Multiprocessing support
261
    "pkg-resources",  # Package resource management
262
]
263

264
# Plugins are typically activated through Options system
265
# Options.enablePlugin("numpy")
266
activatePlugins()
267
```
268

269
### Plugin-Specific Options
270

271
Configure standard plugins with specific options.
272

273
```python
274
# NumPy plugin configuration
275
numpy_options = {
276
    "include_blas": True,
277
    "optimize_math": True,
278
    "include_tests": False,
279
}
280

281
# Qt plugins configuration  
282
qt_options = {
283
    "plugins": ["platforms", "imageformats", "iconengines"],
284
    "qml_directory": "qml/",
285
    "include_translations": True,
286
}
287

288
# Django plugin configuration
289
django_options = {
290
    "settings_module": "myproject.settings",
291
    "include_admin": True,
292
    "static_files": True,
293
}
294
```
295

296
## Custom Plugin Development
297

298
### Creating Plugins
299

300
Develop custom plugins for specific packages or use cases.
301

302
```python
303
from nuitka.plugins.PluginBase import NuitkaPluginBase
304
import os
305
import glob
306

307
class CustomFrameworkPlugin(NuitkaPluginBase):
308
    """Plugin for a custom application framework."""
309
    
310
    plugin_name = "custom-framework"
311
    plugin_desc = "Support for Custom Application Framework"
312
    
313
    @staticmethod
314
    def isAlwaysEnabled():
315
        """Plugin is only enabled when explicitly requested."""
316
        return False
317
    
318
    def onModuleDiscovered(self, module):
319
        """Handle custom framework modules."""
320
        module_name = module.getFullName()
321
        return module_name.startswith("custom_framework")
322
    
323
    def getImplicitImports(self, module):
324
        """Add framework's hidden dependencies."""
325
        module_name = module.getFullName()
326
        imports = []
327
        
328
        if module_name == "custom_framework":
329
            # Core framework dependencies
330
            imports.extend([
331
                "custom_framework.core.registry",
332
                "custom_framework.utils.helpers",
333
                "custom_framework.plugins.loader",
334
            ])
335
        
336
        elif module_name == "custom_framework.web":
337
            # Web component dependencies
338
            imports.extend([
339
                "custom_framework.web.middleware",
340
                "custom_framework.web.auth.backends",
341
            ])
342
        
343
        return imports
344
    
345
    def onDataFiles(self, module):
346
        """Include framework templates and assets."""
347
        data_files = []
348
        module_name = module.getFullName()
349
        
350
        if module_name == "custom_framework":
351
            # Find framework installation directory
352
            framework_dir = os.path.dirname(module.getCompileTimeFilename())
353
            
354
            # Include templates
355
            template_dir = os.path.join(framework_dir, "templates")
356
            if os.path.exists(template_dir):
357
                for template in glob.glob(f"{template_dir}/**/*.html", recursive=True):
358
                    rel_path = os.path.relpath(template, framework_dir)
359
                    data_files.append((template, f"custom_framework/{rel_path}"))
360
            
361
            # Include static assets
362
            static_dir = os.path.join(framework_dir, "static")
363
            if os.path.exists(static_dir):
364
                for asset in glob.glob(f"{static_dir}/**/*", recursive=True):
365
                    if os.path.isfile(asset):
366
                        rel_path = os.path.relpath(asset, framework_dir)
367
                        data_files.append((asset, f"custom_framework/{rel_path}"))
368
        
369
        return data_files
370
    
371
    def onCodeGeneration(self, module, code_context):
372
        """Add custom initialization code."""
373
        if module.getFullName() == "custom_framework":
374
            return '''
375
            // Custom framework initialization
376
            PyObject *framework_init(void) {
377
                // Initialize framework components
378
                return Py_None;
379
            }
380
            '''
381
        return ""
382

383
# Register the plugin
384
plugin_instance = CustomFrameworkPlugin()
385
```
386

387
### Plugin Testing
388

389
Test custom plugins to ensure they work correctly.
390

391
```python
392
import unittest
393
from unittest.mock import Mock, patch
394
from custom_framework_plugin import CustomFrameworkPlugin
395

396
class TestCustomFrameworkPlugin(unittest.TestCase):
397
    """Test cases for custom framework plugin."""
398
    
399
    def setUp(self):
400
        """Set up test environment."""
401
        self.plugin = CustomFrameworkPlugin()
402
        self.mock_module = Mock()
403
    
404
    def test_module_discovery(self):
405
        """Test module discovery functionality."""
406
        # Test framework module recognition
407
        self.mock_module.getFullName.return_value = "custom_framework.core"
408
        self.assertTrue(self.plugin.onModuleDiscovered(self.mock_module))
409
        
410
        # Test non-framework module
411
        self.mock_module.getFullName.return_value = "other_package"
412
        self.assertFalse(self.plugin.onModuleDiscovered(self.mock_module))
413
    
414
    def test_implicit_imports(self):
415
        """Test implicit import detection.""" 
416
        self.mock_module.getFullName.return_value = "custom_framework"
417
        imports = self.plugin.getImplicitImports(self.mock_module)
418
        
419
        self.assertIn("custom_framework.core.registry", imports)
420
        self.assertIn("custom_framework.utils.helpers", imports)
421
    
422
    @patch('os.path.exists')
423
    @patch('glob.glob')
424
    def test_data_files(self, mock_glob, mock_exists):
425
        """Test data file inclusion."""
426
        mock_exists.return_value = True
427
        mock_glob.return_value = ["/path/to/template.html"]
428
        
429
        self.mock_module.getFullName.return_value = "custom_framework"
430
        self.mock_module.getCompileTimeFilename.return_value = "/path/to/custom_framework/__init__.py"
431
        
432
        data_files = self.plugin.onDataFiles(self.mock_module)
433
        self.assertTrue(len(data_files) > 0)
434

435
if __name__ == "__main__":
436
    unittest.main()
437
```
438

439
## Plugin Management
440

441
### Plugin Discovery and Loading
442

443
Manage plugin discovery and loading process.
444

445
```python
446
from nuitka.plugins.Plugins import Plugins
447

448
def load_custom_plugins():
449
    """Load and activate custom plugins."""
450
    # Plugin discovery and registration
451
    custom_plugins = [
452
        "custom_framework_plugin",
453
        "specialized_data_plugin", 
454
        "optimization_plugin",
455
    ]
456
    
457
    for plugin_name in custom_plugins:
458
        try:
459
            # Load plugin module
460
            plugin_module = __import__(f"plugins.{plugin_name}", fromlist=[plugin_name])
461
            
462
            # Register plugin instance
463
            plugin_instance = getattr(plugin_module, "plugin_instance")
464
            Plugins.addPlugin(plugin_instance)
465
            
466
            print(f"Loaded plugin: {plugin_name}")
467
            
468
        except ImportError as e:
469
            print(f"Failed to load plugin {plugin_name}: {e}")
470
        except AttributeError as e:
471
            print(f"Plugin {plugin_name} missing plugin_instance: {e}")
472

473
# Usage
474
load_custom_plugins()
475
```
476

477
### Plugin Debugging
478

479
Debug plugin behavior and troubleshoot issues.
480

481
```python
482
from nuitka.plugins.PluginBase import NuitkaPluginBase
483
from nuitka.Tracing import my_print
484

485
class DebuggingPlugin(NuitkaPluginBase):
486
    """Plugin with comprehensive debugging output."""
487
    
488
    plugin_name = "debug-plugin"
489
    
490
    def onModuleDiscovered(self, module):
491
        """Log module discovery events."""
492
        module_name = module.getFullName()
493
        my_print(f"DEBUG: Module discovered: {module_name}")
494
        
495
        # Check if this is a module we care about
496
        if module_name.startswith("target_package"):
497
            my_print(f"DEBUG: Handling target module: {module_name}")
498
            return True
499
        
500
        return False
501
    
502
    def getImplicitImports(self, module):
503
        """Log and return implicit imports."""
504
        imports = []
505
        module_name = module.getFullName()
506
        
507
        if module_name == "target_package":
508
            imports = ["hidden_dep1", "hidden_dep2"]
509
            my_print(f"DEBUG: Adding implicit imports for {module_name}: {imports}")
510
        
511
        return imports
512
    
513
    def onDataFiles(self, module):
514
        """Log data file inclusion."""
515
        data_files = []
516
        module_name = module.getFullName()
517
        
518
        if module_name == "target_package":
519
            data_files = [("data/file.txt", "target_package/data/file.txt")]
520
            my_print(f"DEBUG: Including data files for {module_name}: {data_files}")
521
        
522
        return data_files
523
```
524

525
## Error Handling
526

527
### Plugin Error Management
528

529
Handle plugin errors gracefully during compilation.
530

531
```python
532
from nuitka.plugins.PluginBase import NuitkaPluginBase
533
from nuitka.Tracing import my_print
534

535
class RobustPlugin(NuitkaPluginBase):
536
    """Plugin with comprehensive error handling."""
537
    
538
    plugin_name = "robust-plugin"
539
    
540
    def onModuleDiscovered(self, module):
541
        """Safely handle module discovery."""
542
        try:
543
            module_name = module.getFullName()
544
            return self._handle_module(module_name)
545
        except Exception as e:
546
            my_print(f"Error in plugin {self.plugin_name} during module discovery: {e}")
547
            return False
548
    
549
    def getImplicitImports(self, module):
550
        """Safely return implicit imports."""
551
        try:
552
            return self._get_imports_for_module(module)
553
        except Exception as e:
554
            my_print(f"Error getting implicit imports: {e}")
555
            return []
556
    
557
    def onDataFiles(self, module):
558
        """Safely handle data file inclusion."""
559
        try:
560
            return self._get_data_files_for_module(module)
561
        except Exception as e:
562
            my_print(f"Error including data files: {e}")  
563
            return []
564
    
565
    def _handle_module(self, module_name):
566
        """Internal module handling with validation."""
567
        if not isinstance(module_name, str):
568
            raise ValueError("Module name must be string")
569
        return module_name.startswith("handled_package")
570
    
571
    def _get_imports_for_module(self, module):
572
        """Internal import resolution with validation."""
573
        # Safe import resolution logic
574
        return []
575
    
576
    def _get_data_files_for_module(self, module):
577
        """Internal data file resolution with validation."""
578
        # Safe data file resolution logic
579
        return []
580
```
581

582
## Types
583

584
```python { .api }
585
# Plugin types
586
PluginInstance = NuitkaPluginBase
587
PluginName = str
588
PluginDescription = str
589

590
# Module types
591
ModuleObject = Any  # Internal Nuitka module representation
592
ModuleName = str
593
ModuleImports = list[str]
594

595
# Data file types
596
DataFileEntry = tuple[str, str]  # (source_path, target_path)
597
DataFileList = list[DataFileEntry]
598

599
# Configuration types
600
YamlConfig = dict[str, Any]
601
PluginOptions = dict[str, Any]
602

603
# Lifecycle types  
604
LifecycleHook = str
605
HookResult = Any
606
```