0
# Rule Compilation and Loading
1

2
Rule compilation transforms textual YARA rules into optimized Rules objects that can efficiently scan data. The compilation process supports various input sources, external variables, include mechanisms, and namespace organization.
3

4
## Capabilities
5

6
### Basic Rule Compilation
7

8
Compile YARA rules from source strings with the fundamental compile function.
9

10
```python { .api }
11
def compile(source=None, filepath=None, file=None, filepaths=None, sources=None, 
12
           includes=True, externals=None, error_on_warning=False, include_callback=None):
13
    """
14
    Compile YARA rules from various sources.
15
    
16
    Parameters:
17
    - source (str, optional): YARA rules as a string
18
    - filepath (str, optional): Path to a single rules file
19
    - file (file-like, optional): File-like object containing rules
20
    - filepaths (dict, optional): Mapping of namespaces to file paths
21
    - sources (dict, optional): Mapping of namespaces to rule strings
22
    - includes (bool): Enable include directive processing (default: True)
23
    - externals (dict, optional): External variables for rules
24
    - error_on_warning (bool): Treat warnings as errors (default: False)
25
    - include_callback (callable, optional): Callback for handling includes
26
    
27
    Returns:
28
    Rules: Compiled rules object ready for matching
29
    
30
    Raises:
31
    SyntaxError: If rules contain syntax errors
32
    """
33
```
34

35
**Basic compilation example:**
36

37
```python
38
import yara
39

40
# Simple rule compilation
41
rules = yara.compile(source='''
42
rule MalwarePattern {
43
    strings:
44
        $suspicious = "eval("
45
        $encoded = { 4D 5A }  // MZ header
46
    condition:
47
        $suspicious or $encoded
48
}
49
''')
50
```
51

52
### File-Based Compilation
53

54
Compile rules from files on disk or file-like objects.
55

56
**Single file compilation:**
57

58
```python
59
# From file path
60
rules = yara.compile(filepath="/path/to/rules.yar")
61

62
# From file object
63
with open("rules.yar", "r") as f:
64
    rules = yara.compile(file=f)
65

66
# From StringIO/BytesIO (file-like objects)
67
import io
68

69
rule_content = '''
70
rule TestRule {
71
    strings:
72
        $test = "pattern"
73
    condition:
74
        $test
75
}
76
'''
77

78
# Using StringIO
79
rule_stream = io.StringIO(rule_content)
80
rules = yara.compile(file=rule_stream)
81

82
# Using BytesIO (Python 3)
83
rule_bytes = rule_content.encode('utf-8')
84
byte_stream = io.BytesIO(rule_bytes)
85
rules = yara.compile(file=byte_stream)
86
```
87

88
### Multi-Source Compilation with Namespaces
89

90
Organize rules from multiple sources using namespaces for better organization and conflict resolution.
91

92
**Multiple files with namespaces:**
93

94
```python
95
rules = yara.compile(filepaths={
96
    'malware': '/path/to/malware_rules.yar',
97
    'goodware': '/path/to/goodware_rules.yar',
98
    'network': '/path/to/network_rules.yar'
99
})
100
```
101

102
**Multiple sources with namespaces:**
103

104
```python
105
rules = yara.compile(sources={
106
    'basic': 'rule test1 { condition: true }',
107
    'advanced': '''
108
        rule test2 {
109
            strings:
110
                $hex = { 4D 5A 90 00 }
111
            condition:
112
                $hex at 0
113
        }
114
    '''
115
})
116
```
117

118
### External Variables
119

120
Pass external variables to rules for dynamic behavior and parameterization.
121

122
**Supported external variable types:**
123

124
```python
125
# Integer externals
126
rules = yara.compile(
127
    source='rule test { condition: ext_size > 1000 }',
128
    externals={'ext_size': 2048}
129
)
130

131
# Float externals  
132
rules = yara.compile(
133
    source='rule test { condition: ext_ratio > 0.75 }',
134
    externals={'ext_ratio': 0.85}
135
)
136

137
# Boolean externals
138
rules = yara.compile(
139
    source='rule test { condition: is_executable }',
140
    externals={'is_executable': True}
141
)
142

143
# String externals
144
rules = yara.compile(
145
    source='rule test { condition: filename matches ext_pattern }',
146
    externals={'ext_pattern': '*.exe'}
147
)
148
```
149

150
### Include Callback System
151

152
Handle include directives dynamically with custom callback functions for modular rule organization.
153

154
```python { .api }
155
def include_callback(requested_filename, filename, namespace):
156
    """
157
    Handle include directives in YARA rules.
158
    
159
    Parameters:
160
    - requested_filename (str): The filename requested in the include directive
161
    - filename (str): The file being processed that contains the include
162
    - namespace (str): The namespace context
163
    
164
    Returns:
165
    str or None: Rule content to include, or None to skip
166
    """
167
```
168

169
**Include callback example:**
170

171
```python
172
def my_include_callback(requested_filename, filename, namespace):
173
    include_map = {
174
        'common.yar': '''
175
            rule CommonPattern {
176
                strings:
177
                    $common = "suspicious"
178
                condition:
179
                    $common
180
            }
181
        ''',
182
        'network.yar': '''
183
            rule NetworkPattern {
184
                condition:
185
                    uint16(0) == 0x5A4D  // MZ signature
186
            }
187
        '''
188
    }
189
    return include_map.get(requested_filename)
190

191
rules = yara.compile(
192
    source='include "common.yar" include "network.yar" rule main { condition: true }',
193
    include_callback=my_include_callback
194
)
195
```
196

197
### Rule Serialization
198

199
Save compiled rules to disk for performance optimization and reuse across sessions.
200

201
```python { .api }
202
class Rules:
203
    def save(self, filepath=None, file=None):
204
        """
205
        Save compiled rules to file.
206
        
207
        Parameters:
208
        - filepath (str, optional): Path to save compiled rules
209
        - file (file-like, optional): File-like object to write to
210
        """
211
```
212

213
**Save compiled rules:**
214

215
```python
216
# Save to file path
217
rules.save(filepath="compiled_rules.yarc")
218

219
# Save to file object
220
with open("rules.yarc", "wb") as f:
221
    rules.save(file=f)
222

223
# Save to BytesIO (file-like object)
224
import io
225

226
# Save to memory buffer
227
buffer = io.BytesIO()
228
rules.save(file=buffer)
229

230
# Get the compiled rules as bytes
231
compiled_data = buffer.getvalue()
232
print(f"Compiled rules size: {len(compiled_data)} bytes")
233
```
234

235
### Rule Loading
236

237
Load previously compiled and saved rules for immediate use without recompilation.
238

239
```python { .api }
240
def load(filepath=None, file=None):
241
    """
242
    Load previously saved compiled rules.
243
    
244
    Parameters:
245
    - filepath (str, optional): Path to saved rules file
246
    - file (file-like, optional): File-like object to read from
247
    
248
    Returns:
249
    Rules: Loaded rules object ready for matching
250
    """
251
```
252

253
**Load compiled rules:**
254

255
```python
256
# Load from file path
257
rules = yara.load(filepath="compiled_rules.yarc")
258

259
# Load from file object
260
with open("rules.yarc", "rb") as f:
261
    rules = yara.load(file=f)
262

263
# Load from BytesIO (file-like object)
264
import io
265

266
# Assume compiled_data is bytes from previous save operation
267
buffer = io.BytesIO(compiled_data)
268
rules = yara.load(file=buffer)
269

270
# Complete save/load example with memory buffers
271
original_rules = yara.compile(source='rule test { condition: true }')
272

273
# Save to memory
274
save_buffer = io.BytesIO()
275
original_rules.save(file=save_buffer)
276

277
# Load from memory
278
load_buffer = io.BytesIO(save_buffer.getvalue())
279
restored_rules = yara.load(file=load_buffer)
280

281
# Both rules should behave identically
282
matches1 = original_rules.match(data="test")
283
matches2 = restored_rules.match(data="test")
284
```
285

286
### Advanced Compilation Options
287

288
Control compilation behavior with advanced options for specific use cases.
289

290
**Disable includes:**
291

292
```python
293
# This will raise SyntaxError if source contains include directives
294
try:
295
    rules = yara.compile(
296
        source='include "test.yar" rule main { condition: true }',
297
        includes=False
298
    )
299
except yara.SyntaxError as e:
300
    print(f"Include directive not allowed: {e}")
301
```
302

303
**Treat warnings as errors:**
304

305
```python
306
rules = yara.compile(
307
    source='rule test { condition: true }',
308
    error_on_warning=True
309
)
310
```
311

312
## Rule Object Inspection
313

314
Access individual rules within a compiled Rules object through iteration.
315

316
```python { .api }
317
class Rules:
318
    """Compiled YARA rules container supporting iteration."""
319
    def __iter__(self):
320
        """Iterate over individual Rule objects."""
321

322
class Rule:
323
    """Individual YARA rule representation."""
324
    identifier: str  # Rule name/identifier
325
    tags: list      # List of rule tags
326
    meta: dict      # Rule metadata dictionary
327
```
328

329
**Inspect compiled rules:**
330

331
```python
332
# Iterate over all rules in the compiled Rules object
333
for rule in rules:
334
    print(f"Rule: {rule.identifier}")
335
    print(f"Tags: {rule.tags}")
336
    print(f"Metadata: {rule.meta}")
337

338
# Rules object supports standard iteration protocols
339
rule_list = list(rules)          # Convert to list
340
rule_count = len(rule_list)      # Count rules (indirect)
341
first_rule = next(iter(rules))   # Get first rule
342

343
# Enumerate rules
344
for i, rule in enumerate(rules, 1):
345
    print(f"Rule {i}: {rule.identifier}")
346
    if rule.tags:
347
        print(f"  Tags: {', '.join(rule.tags)}")
348
    if rule.meta:
349
        print(f"  Metadata: {rule.meta}")
350
```