0
# Build System
1

2
Declarative build system for managing complex compilation workflows with dependencies, actions, and entrypoints. The system provides a high-level interface for orchestrating multi-step compilation processes and integrating with external build tools.
3

4
## Capabilities
5

6
### Entrypoint Decorators
7

8
Decorators for defining build entrypoints and command-line argument processing.
9

10
```python { .api }
11
def entrypoint(f=None, *, description: str | None = None):
12
    """
13
    Function decorator to turn it into a build entrypoint.
14
    
15
    Parameters:
16
    - f: Function to decorate
17
    - description: Optional description for the entrypoint
18
    
19
    Returns:
20
    Entrypoint: Build entrypoint object
21
    """
22

23
def cl_arg(name: str, **kwargs):
24
    """
25
    Decorator for defining command-line arguments.
26
    
27
    Parameters:
28
    - name: Argument name
29
    - **kwargs: Additional argument parser configuration
30
    """
31
```
32

33
### Core Actions
34

35
Base action classes for implementing build system operations.
36

37
```python { .api }
38
class BuildAction(BuildDependency):
39
    """
40
    Base class for build system actions.
41
    
42
    Actions represent individual build steps that can be composed
43
    into complex workflows with dependency management. This class is
44
    designed to be subclassed by concrete actions.
45
    """
46
    
47
    def _invoke(self) -> None:
48
        """
49
        Execute the action in-process (override for in-process actions).
50
        """
51
    
52
    def _remotable_thunk(self):
53
        """
54
        Return a remotable thunk for out-of-process execution.
55
        
56
        Returns:
57
        Tuple of (callable, args) for out-of-process execution
58
        """
59

60
class CompileAction(BuildAction):
61
    """
62
    Action for compiling IREE modules from source.
63
    """
64
    
65
    def __init__(self, inv: CompilerInvocation, **kwargs): ...
66

67
class ImportOnnxAction(BuildAction):
68
    """
69
    Action for importing ONNX models to MLIR.
70
    """
71
    
72
    def __init__(self, input_file: BuildFile, output_file: BuildFile, upgrade: bool, **kwargs): ...
73

74
class FetchHttpAction(BuildAction):
75
    """
76
    Action for fetching resources over HTTP.
77
    """
78
    
79
    def __init__(self, url: str, output_file: BuildFile, **kwargs): ...
80

81
class ExecuteOutOfProcessThunkAction(BuildAction):
82
    """
83
    Executes a callback thunk with arguments out-of-process.
84
    
85
    Both the thunk and args must be pickleable.
86
    """
87
```
88

89
### Entrypoint Management
90

91
Build system entrypoint wrapping callables with argument definitions.
92

93
```python { .api }
94
class Entrypoint:
95
    """
96
    Build system entrypoint wrapping a callable with argument definitions.
97
    """
98
    
99
    def __init__(self, name: str, wrapped: Callable, description: str | None = None): ...
100
    
101
    def __call__(self, *args, **kwargs): ...
102
```
103

104
### File Management
105

106
File abstraction and namespace management for build artifacts.
107

108
```python { .api }
109
class BuildFile:
110
    """
111
    Represents a file in the build system with dependency tracking.
112
    """
113
    
114
    def __init__(self, namespace: FileNamespace, path: str): ...
115

116
class FileNamespace(Enum):
117
    """
118
    File namespace enumeration for organizing build artifacts.
119
    """
120
    GEN = "gen"  # Transient generated files
121
    PARAMS = "params"  # Distributable parameter files
122
    BIN = "bin"  # Distributable, platform-neutral binaries
123
    PLATFORM_BIN = "platform_bin"  # Platform-specific binaries
124

125
class BuildContext:
126
    """
127
    Build execution context managing file namespaces and dependencies.
128
    """
129
    
130
    @staticmethod
131
    def current() -> BuildContext: ...
132
    
133
    def gen_file(self, path: str) -> BuildFile: ...
134
    
135
    def bin_file(self, path: str) -> BuildFile: ...
136
```
137

138
### Action Concurrency
139

140
Concurrency models for action execution.
141

142
```python { .api }
143
class ActionConcurrency(Enum):
144
    """
145
    Action concurrency execution models.
146
    """
147
    THREAD = "thread"  # Thread-based concurrency
148
    PROCESS = "process"  # Process-based concurrency
149
    NONE = "none"  # Sequential execution
150
```
151

152
### Compilation Actions
153

154
High-level compilation functions that integrate with the build system.
155

156
```python { .api }
157
def compile(
158
    input_file: BuildFileLike,
159
    *,
160
    target_backends: Sequence[str],
161
    output_file: BuildFile | None = None,
162
    **kwargs
163
) -> BuildFile:
164
    """
165
    Compile an MLIR source file to VM bytecode using the build system.
166
    
167
    Parameters:
168
    - input_file: Source file to compile
169
    - target_backends: List of target backends
170
    - output_file: Optional output file (auto-generated if not provided)
171
    - **kwargs: Additional compiler options
172
    
173
    Returns:
174
    BuildFile: Output file containing compiled bytecode
175
    """
176
```
177

178
### Argument Processing
179

180
Command-line argument processing utilities.
181

182
```python { .api }
183
def expand_cl_arg_defaults(func: Callable) -> Callable:
184
    """
185
    Expand command-line argument defaults for a function.
186
    
187
    Parameters:
188
    - func: Function to process
189
    
190
    Returns:
191
    Callable: Function with expanded defaults
192
    """
193

194
def extract_cl_arg_defs(func: Callable) -> List:
195
    """
196
    Extract command-line argument definitions from a function.
197
    
198
    Parameters:
199
    - func: Function to analyze
200
    
201
    Returns:
202
    List: Command-line argument definitions
203
    """
204

205
def current_args_namespace():
206
    """
207
    Get the current command-line arguments namespace.
208
    
209
    Returns:
210
    argparse.Namespace: Current arguments
211
    """
212
```
213

214
## Usage Examples
215

216
### Basic Build Workflow
217

218
```python
219
import iree.build
220
from iree.build import entrypoint, cl_arg
221

222
# Define a simple build entrypoint
223
@entrypoint(description="Compile MLIR model")
224
@cl_arg("--input", required=True, help="Input MLIR file")
225
@cl_arg("--output", help="Output VM file")
226
@cl_arg("--target", default="llvm-cpu", help="Target backend")
227
def compile_model(input_file: str, output_file: str = None, target: str = "llvm-cpu"):
228
    """Simple model compilation entrypoint."""
229
    context = iree.build.BuildContext.current()
230
    
231
    # Create input and output files
232
    input_build_file = context.gen_file(input_file)
233
    if output_file is None:
234
        output_file = input_file.replace(".mlir", ".vmfb")
235
    output_build_file = context.bin_file(output_file)
236
    
237
    # Compile using build system
238
    result = iree.build.compile(
239
        input_build_file,
240
        target_backends=[target],
241
        output_file=output_build_file
242
    )
243
    
244
    return result
245
```
246

247
### ONNX Import and Compilation
248

249
```python
250
from iree.build import entrypoint, cl_arg, BuildContext
251
from iree.build import ImportOnnxAction, CompileAction
252

253
@entrypoint(description="Import and compile ONNX model")
254
@cl_arg("--onnx-file", required=True, help="Input ONNX file")
255
@cl_arg("--targets", nargs="+", default=["llvm-cpu"], help="Target backends")
256
@cl_arg("--upgrade", action="store_true", help="Upgrade ONNX model")
257
def compile_onnx_model(onnx_file: str, targets: list, upgrade: bool = False):
258
    """Import ONNX model and compile for multiple targets."""
259
    context = BuildContext.current()
260
    
261
    # Set up files
262
    input_file = context.gen_file(onnx_file)
263
    mlir_file = context.gen_file(onnx_file.replace(".onnx", ".mlir"))
264
    
265
    # Import ONNX to MLIR
266
    import_action = ImportOnnxAction(
267
        input_file=input_file,
268
        output_file=mlir_file,
269
        upgrade=upgrade
270
    )
271
    
272
    # Compile for each target
273
    results = []
274
    for target in targets:
275
        output_file = context.bin_file(f"model_{target}.vmfb")
276
        
277
        compile_result = iree.build.compile(
278
            mlir_file,
279
            target_backends=[target],
280
            output_file=output_file
281
        )
282
        results.append(compile_result)
283
    
284
    return results
285
```
286

287
### HTTP Fetch and Compilation
288

289
```python
290
from iree.build import entrypoint, cl_arg, BuildContext
291
from iree.build import FetchHttpAction, compile
292

293
@entrypoint(description="Fetch model from URL and compile")
294
@cl_arg("--url", required=True, help="Model URL")
295
@cl_arg("--target", default="llvm-cpu", help="Target backend")
296
def fetch_and_compile(url: str, target: str = "llvm-cpu"):
297
    """Fetch a model from URL and compile it."""
298
    context = BuildContext.current()
299
    
300
    # Determine file extension from URL
301
    if url.endswith(".onnx"):
302
        local_file = context.gen_file("downloaded_model.onnx")
303
    else:
304
        local_file = context.gen_file("downloaded_model.mlir")
305
    
306
    # Fetch the file
307
    fetch_action = FetchHttpAction(
308
        url=url,
309
        output_file=local_file
310
    )
311
    
312
    # Compile the downloaded file
313
    output_file = context.bin_file(f"model_{target}.vmfb")
314
    result = compile(
315
        local_file,
316
        target_backends=[target],
317
        output_file=output_file
318
    )
319
    
320
    return result
321
```
322

323
### Advanced Build with Custom Actions
324

325
```python
326
from iree.build import entrypoint, cl_arg, BuildAction, BuildContext
327

328
class CustomPreprocessAction(BuildAction):
329
    """Custom preprocessing action."""
330
    
331
    def __init__(self, input_file, output_file, **kwargs):
332
        super().__init__(**kwargs)
333
        self.input_file = input_file
334
        self.output_file = output_file
335
        self.output_file.deps.add(self)
336
        self.deps.add(self.input_file)
337
    
338
    def _invoke(self):
339
        # Custom preprocessing logic
340
        with open(self.input_file.get_fs_path(), 'r') as f:
341
            content = f.read()
342
        
343
        # Apply preprocessing
344
        processed_content = content.replace("old_pattern", "new_pattern")
345
        
346
        with open(self.output_file.get_fs_path(), 'w') as f:
347
            f.write(processed_content)
348

349
@entrypoint(description="Custom preprocessing and compilation")
350
@cl_arg("--input", required=True, help="Input file")
351
@cl_arg("--target", default="llvm-cpu", help="Target backend")
352
def custom_build(input_file: str, target: str = "llvm-cpu"):
353
    """Custom build with preprocessing step."""
354
    context = BuildContext.current()
355
    
356
    # Input and intermediate files
357
    raw_input = context.gen_file(input_file)
358
    processed_input = context.gen_file("processed_" + input_file)
359
    
360
    # Custom preprocessing
361
    preprocess_action = CustomPreprocessAction(
362
        input_file=raw_input,
363
        output_file=processed_input
364
    )
365
    
366
    # Compile preprocessed file
367
    output_file = context.bin_file(f"custom_model_{target}.vmfb")
368
    result = compile(
369
        processed_input,
370
        target_backends=[target],
371
        output_file=output_file
372
    )
373
    
374
    return result
375
```