0
# Pattern Matching and Expressions
1

2
Comprehensive pattern expression system for STIX indicator patterns including observation expressions, boolean logic, comparison operations, and temporal qualifiers for complex threat detection rules.
3

4
## Capabilities
5

6
### Pattern Expression Classes
7

8
Core classes for building STIX pattern expressions programmatically.
9

10
```python { .api }
11
class ObservationExpression:
12
    """
13
    Base class for STIX observation expressions in patterns.
14
    
15
    Constructor Parameters:
16
    - operand: The operand expression (comparison, boolean, etc.)
17
    """
18

19
class AndObservationExpression:
20
    """
21
    AND operation between two observation expressions.
22
    
23
    Constructor Parameters:
24
    - operands (list): List of observation expressions to AND together
25
    """
26

27
class OrObservationExpression:
28
    """
29
    OR operation between two observation expressions.
30
    
31
    Constructor Parameters:
32
    - operands (list): List of observation expressions to OR together
33
    """
34

35
class FollowedByObservationExpression:
36
    """
37
    Temporal FOLLOWEDBY operation between observation expressions.
38
    
39
    Constructor Parameters:
40
    - operands (list): List of observation expressions in temporal order
41
    """
42

43
class QualifiedObservationExpression:
44
    """
45
    Observation expression with temporal qualifiers.
46
    
47
    Constructor Parameters:
48
    - observation_expression: Base observation expression
49
    - qualifier: Temporal qualifier (WITHIN, REPEATS, etc.)
50
    """
51

52
class ParentheticalExpression:
53
    """
54
    Parenthetical grouping of expressions.
55
    
56
    Constructor Parameters:
57
    - expression: Expression to wrap in parentheses
58
    """
59
```
60

61
### Boolean Expressions
62

63
Boolean logic operations for combining pattern conditions.
64

65
```python { .api }
66
class AndBooleanExpression:
67
    """
68
    Boolean AND operation between expressions.
69
    
70
    Constructor Parameters:
71
    - operands (list): List of expressions to AND together
72
    """
73

74
class OrBooleanExpression:
75
    """
76
    Boolean OR operation between expressions.
77
    
78
    Constructor Parameters:
79
    - operands (list): List of expressions to OR together
80
    """
81
```
82

83
Usage examples:
84

85
```python
86
from stix2 import (
87
    ObjectPath, EqualityComparisonExpression, StringConstant,
88
    AndBooleanExpression, OrBooleanExpression, ObservationExpression
89
)
90

91
# Create basic comparison expressions
92
ip_comparison = EqualityComparisonExpression(
93
    ObjectPath("ipv4-addr", ["value"]),
94
    StringConstant("192.168.1.100")
95
)
96

97
port_comparison = EqualityComparisonExpression(
98
    ObjectPath("network-traffic", ["dst_port"]),
99
    IntegerConstant(80)
100
)
101

102
# Combine with boolean AND
103
and_expression = AndBooleanExpression([ip_comparison, port_comparison])
104

105
# Create observation expression
106
observation = ObservationExpression(and_expression)
107

108
# Pattern string would be: "[ipv4-addr:value = '192.168.1.100' AND network-traffic:dst_port = 80]"
109

110
# Boolean OR example
111
protocol_http = EqualityComparisonExpression(
112
    ObjectPath("network-traffic", ["protocols", 0]),
113
    StringConstant("http")
114
)
115

116
protocol_https = EqualityComparisonExpression(
117
    ObjectPath("network-traffic", ["protocols", 0]),
118
    StringConstant("https")
119
)
120

121
or_expression = OrBooleanExpression([protocol_http, protocol_https])
122
```
123

124
### Comparison Expressions
125

126
Various comparison operations for pattern matching.
127

128
```python { .api }
129
class EqualityComparisonExpression:
130
    """
131
    Equality comparison (=) between object path and constant.
132
    
133
    Constructor Parameters:
134
    - lhs: Left-hand side (typically ObjectPath)
135
    - rhs: Right-hand side (typically constant value)
136
    """
137

138
class GreaterThanComparisonExpression:
139
    """Greater than comparison (>) between object path and constant."""
140

141
class GreaterThanEqualComparisonExpression:
142
    """Greater than or equal comparison (>=) between object path and constant."""
143

144
class LessThanComparisonExpression:
145
    """Less than comparison (<) between object path and constant."""
146

147
class LessThanEqualComparisonExpression:
148
    """Less than or equal comparison (<=) between object path and constant."""
149

150
class InComparisonExpression:
151
    """IN comparison for checking membership in a set."""
152

153
class LikeComparisonExpression:
154
    """LIKE comparison for pattern matching with wildcards."""
155

156
class MatchesComparisonExpression:
157
    """MATCHES comparison for regular expression matching."""
158

159
class IsSubsetComparisonExpression:
160
    """ISSUBSET comparison for subset relationships."""
161

162
class IsSupersetComparisonExpression:
163
    """ISSUPERSET comparison for superset relationships."""
164
```
165

166
Usage examples:
167

168
```python
169
from stix2 import (
170
    ObjectPath, EqualityComparisonExpression, GreaterThanComparisonExpression,
171
    LikeComparisonExpression, MatchesComparisonExpression, InComparisonExpression,
172
    StringConstant, IntegerConstant, ListConstant
173
)
174

175
# Equality comparison
176
file_hash_eq = EqualityComparisonExpression(
177
    ObjectPath("file", ["hashes", "MD5"]),
178
    StringConstant("d41d8cd98f00b204e9800998ecf8427e")
179
)
180

181
# Numeric comparisons
182
file_size_gt = GreaterThanComparisonExpression(
183
    ObjectPath("file", ["size"]),
184
    IntegerConstant(1000000)  # Files larger than 1MB
185
)
186

187
# Pattern matching with LIKE (SQL-style wildcards)
188
domain_like = LikeComparisonExpression(
189
    ObjectPath("domain-name", ["value"]),
190
    StringConstant("*.evil.com")  # Subdomains of evil.com
191
)
192

193
# Regular expression matching
194
email_regex = MatchesComparisonExpression(
195
    ObjectPath("email-addr", ["value"]),
196
    StringConstant(r"^[a-zA-Z0-9._%+-]+@evil\.com$")
197
)
198

199
# IN comparison for membership
200
suspicious_ports = InComparisonExpression(
201
    ObjectPath("network-traffic", ["dst_port"]),
202
    ListConstant([1337, 31337, 4444, 5555])  # Common backdoor ports
203
)
204
```
205

206
### Object Paths
207

208
Specify paths to properties within STIX Cyber Observable Objects.
209

210
```python { .api }
211
class ObjectPath:
212
    """
213
    Path to a property within a STIX Cyber Observable Object.
214
    
215
    Constructor Parameters:
216
    - object_type_name (str): Type of STIX object (e.g., "file", "ipv4-addr")
217
    - property_path (list): List of property names/indices forming the path
218
    """
219

220
class BasicObjectPathComponent:
221
    """
222
    Basic property name component of an object path.
223
    
224
    Constructor Parameters:
225
    - property_name (str): Name of the property
226
    """
227

228
class ListObjectPathComponent:
229
    """
230
    List index component of an object path.
231
    
232
    Constructor Parameters:
233
    - index (int): List index (0-based)
234
    """
235

236
class ReferenceObjectPathComponent:
237
    """
238
    Object reference component of an object path.
239
    
240
    Constructor Parameters:
241
    - reference_property (str): Name of reference property
242
    """
243
```
244

245
Usage examples:
246

247
```python
248
from stix2 import ObjectPath
249

250
# Simple property paths
251
ip_address = ObjectPath("ipv4-addr", ["value"])
252
file_name = ObjectPath("file", ["name"])
253
email_subject = ObjectPath("email-message", ["subject"])
254

255
# Nested property paths
256
file_hash_md5 = ObjectPath("file", ["hashes", "MD5"])
257
file_hash_sha256 = ObjectPath("file", ["hashes", "SHA-256"])
258

259
# Array/list index paths
260
first_protocol = ObjectPath("network-traffic", ["protocols", 0])
261
second_protocol = ObjectPath("network-traffic", ["protocols", 1])
262

263
# Extension property paths
264
http_method = ObjectPath("network-traffic", ["extensions", "http-request-ext", "request_method"])
265
pe_file_type = ObjectPath("file", ["extensions", "windows-pebinary-ext", "pe_type"])
266

267
# Complex nested paths
268
ntfs_ads_name = ObjectPath("file", ["extensions", "ntfs-ext", "alternate_data_streams", 0, "name"])
269
registry_value = ObjectPath("windows-registry-key", ["values", 0, "data"])
270

271
# Object reference paths (following references)
272
parent_dir = ObjectPath("file", ["parent_directory_ref"])
273
creator_user = ObjectPath("process", ["creator_user_ref"])
274

275
# Patterns using these paths
276
patterns = [
277
    f"[{ip_address} = '192.168.1.1']",
278
    f"[{file_hash_md5} = 'abc123def456']", 
279
    f"[{first_protocol} = 'tcp']",
280
    f"[{http_method} = 'POST']"
281
]
282
```
283

284
### Constant Values
285

286
Various types of constant values used in pattern expressions.
287

288
```python { .api }
289
class StringConstant:
290
    """String constant value."""
291
    def __init__(self, value): ...
292

293
class IntegerConstant:
294
    """Integer constant value."""
295
    def __init__(self, value): ...
296

297
class FloatConstant:
298
    """Float constant value."""
299
    def __init__(self, value): ...
300

301
class BooleanConstant:
302
    """Boolean constant value."""
303
    def __init__(self, value): ...
304

305
class BinaryConstant:
306
    """Binary data constant value."""
307
    def __init__(self, value): ...
308

309
class HashConstant:
310
    """Hash value constant."""
311
    def __init__(self, value): ...
312

313
class HexConstant:
314
    """Hexadecimal constant value."""
315
    def __init__(self, value): ...
316

317
class TimestampConstant:
318
    """Timestamp constant value."""
319
    def __init__(self, value): ...
320

321
class ListConstant:
322
    """List of constant values."""
323
    def __init__(self, value): ...
324
```
325

326
Usage examples:
327

328
```python
329
from stix2 import (
330
    StringConstant, IntegerConstant, FloatConstant, BooleanConstant,
331
    HashConstant, HexConstant, TimestampConstant, ListConstant, BinaryConstant
332
)
333

334
# String constants
335
domain_str = StringConstant("malicious.com")
336
user_agent = StringConstant("Malicious-Bot/1.0")
337

338
# Numeric constants
339
file_size = IntegerConstant(1048576)  # 1MB
340
confidence_score = FloatConstant(0.85)
341

342
# Boolean constants
343
is_encrypted = BooleanConstant(True)
344
is_signed = BooleanConstant(False)
345

346
# Hash constants
347
md5_hash = HashConstant("d41d8cd98f00b204e9800998ecf8427e")
348
sha256_hash = HashConstant("e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855")
349

350
# Hex constants
351
magic_bytes = HexConstant("4D5A")  # PE file magic bytes
352
byte_pattern = HexConstant("DEADBEEF")
353

354
# Timestamp constants
355
start_time = TimestampConstant("2021-04-23T10:30:00.000Z")
356
end_time = TimestampConstant("2021-04-23T11:00:00.000Z")
357

358
# List constants
359
port_list = ListConstant([80, 443, 8080, 8443])
360
protocol_list = ListConstant(["tcp", "http"])
361
hash_list = ListConstant([
362
    "abc123def456", 
363
    "def456ghi789",
364
    "ghi789jkl012"
365
])
366

367
# Binary constants
368
binary_data = BinaryConstant(b"\\x4D\\x5A\\x90\\x00")  # PE header bytes
369
```
370

371
### Temporal Qualifiers
372

373
Add temporal constraints to observation expressions.
374

375
```python { .api }
376
class WithinQualifier:
377
    """
378
    WITHIN qualifier for time-bounded observations.
379
    
380
    Constructor Parameters:
381
    - time_interval (int): Time interval in seconds
382
    """
383

384
class RepeatQualifier:
385
    """
386
    REPEATS qualifier for repeated observations.
387
    
388
    Constructor Parameters:
389
    - times (int): Number of repetitions required
390
    """
391

392
class StartStopQualifier:
393
    """
394
    START/STOP qualifier for time-bounded observations.
395
    
396
    Constructor Parameters:
397
    - start_time (timestamp): Start time for observations
398
    - stop_time (timestamp): Stop time for observations
399
    """
400
```
401

402
Usage examples:
403

404
```python
405
from stix2 import (
406
    WithinQualifier, RepeatQualifier, StartStopQualifier,
407
    QualifiedObservationExpression, ObservationExpression
408
)
409

410
# Create base observation
411
base_observation = ObservationExpression(
412
    EqualityComparisonExpression(
413
        ObjectPath("ipv4-addr", ["value"]),
414
        StringConstant("192.168.1.100")
415
    )
416
)
417

418
# WITHIN qualifier - observe within 300 seconds (5 minutes)
419
within_qualifier = WithinQualifier(300)
420
within_observation = QualifiedObservationExpression(
421
    base_observation,
422
    within_qualifier
423
)
424
# Pattern: "[ipv4-addr:value = '192.168.1.100'] WITHIN 300 SECONDS"
425

426
# REPEATS qualifier - must be observed 3 times
427
repeat_qualifier = RepeatQualifier(3)
428
repeat_observation = QualifiedObservationExpression(
429
    base_observation,
430
    repeat_qualifier
431
)
432
# Pattern: "[ipv4-addr:value = '192.168.1.100'] REPEATS 3 TIMES"
433

434
# START/STOP qualifier - observe within specific time window
435
start_stop_qualifier = StartStopQualifier(
436
    TimestampConstant("2021-04-23T10:00:00.000Z"),
437
    TimestampConstant("2021-04-23T18:00:00.000Z")
438
)
439
bounded_observation = QualifiedObservationExpression(
440
    base_observation,
441
    start_stop_qualifier
442
)
443
# Pattern: "[ipv4-addr:value = '192.168.1.100'] START t'2021-04-23T10:00:00.000Z' STOP t'2021-04-23T18:00:00.000Z'"
444

445
# Combining qualifiers with complex observations
446
complex_observation = QualifiedObservationExpression(
447
    AndObservationExpression([
448
        ObservationExpression(EqualityComparisonExpression(
449
            ObjectPath("network-traffic", ["dst_ref.value"]),
450
            StringConstant("192.168.1.100")
451
        )),
452
        ObservationExpression(EqualityComparisonExpression(
453
            ObjectPath("network-traffic", ["dst_port"]),
454
            IntegerConstant(4444)
455
        ))
456
    ]),
457
    RepeatQualifier(5)
458
)
459
# Pattern: "[network-traffic:dst_ref.value = '192.168.1.100' AND network-traffic:dst_port = 4444] REPEATS 5 TIMES"
460
```
461

462
### Complex Pattern Construction
463

464
Build sophisticated detection patterns using combinations of expressions.
465

466
```python
467
from stix2 import Indicator
468

469
# File-based indicator with multiple conditions
470
file_pattern = AndBooleanExpression([
471
    EqualityComparisonExpression(
472
        ObjectPath("file", ["hashes", "MD5"]),
473
        StringConstant("d41d8cd98f00b204e9800998ecf8427e")
474
    ),
475
    EqualityComparisonExpression(
476
        ObjectPath("file", ["name"]),
477
        StringConstant("malware.exe")
478
    ),
479
    GreaterThanComparisonExpression(
480
        ObjectPath("file", ["size"]),
481
        IntegerConstant(50000)
482
    )
483
])
484

485
file_indicator = Indicator(
486
    name="Malicious PE File",
487
    indicator_types=["malicious-activity"],
488
    pattern_type="stix",
489
    pattern=f"[{file_pattern}]"
490
)
491

492
# Network-based indicator with temporal constraints
493
network_base = AndBooleanExpression([
494
    InComparisonExpression(
495
        ObjectPath("network-traffic", ["dst_ref.value"]),
496
        ListConstant(["192.168.1.100", "10.0.0.1", "172.16.0.1"])
497
    ),
498
    InComparisonExpression(
499
        ObjectPath("network-traffic", ["dst_port"]),
500
        ListConstant([4444, 5555, 6666])
501
    ),
502
    EqualityComparisonExpression(
503
        ObjectPath("network-traffic", ["protocols", 0]),
504
        StringConstant("tcp")
505
    )
506
])
507

508
network_observation = QualifiedObservationExpression(
509
    ObservationExpression(network_base),
510
    WithinQualifier(3600)  # Within 1 hour
511
)
512

513
network_indicator = Indicator(
514
    name="C2 Communication Pattern",
515
    indicator_types=["malicious-activity"],
516
    pattern_type="stix",
517
    pattern=f"{network_observation}"
518
)
519

520
# Multi-observable pattern with FOLLOWEDBY
521
initial_access = ObservationExpression(
522
    EqualityComparisonExpression(
523
        ObjectPath("email-message", ["subject"]),
524
        StringConstant("Urgent: Account Verification Required")
525
    )
526
)
527

528
payload_download = ObservationExpression(
529
    LikeComparisonExpression(
530
        ObjectPath("url", ["value"]), 
531
        StringConstant("http://malicious.com/*")
532
    )
533
)
534

535
file_execution = ObservationExpression(
536
    AndBooleanExpression([
537
        LikeComparisonExpression(
538
            ObjectPath("process", ["name"]),
539
            StringConstant("*.exe")
540
        ),
541
        EqualityComparisonExpression(
542
            ObjectPath("process", ["parent_ref.name"]),
543
            StringConstant("outlook.exe")
544
        )
545
    ])
546
)
547

548
attack_chain = FollowedByObservationExpression([
549
    initial_access,
550
    payload_download, 
551
    file_execution
552
])
553

554
chain_indicator = Indicator(
555
    name="Email-Based Attack Chain",
556
    indicator_types=["malicious-activity"],
557
    pattern_type="stix",
558
    pattern=f"{attack_chain}"
559
)
560

561
# Advanced pattern with multiple qualifiers
562
advanced_pattern = QualifiedObservationExpression(
563
    OrObservationExpression([
564
        QualifiedObservationExpression(
565
            ObservationExpression(
566
                EqualityComparisonExpression(
567
                    ObjectPath("process", ["name"]),
568
                    StringConstant("cmd.exe")
569
                )
570
            ),
571
            RepeatQualifier(3)
572
        ),
573
        QualifiedObservationExpression(
574
            ObservationExpression(
575
                EqualityComparisonExpression(
576
                    ObjectPath("process", ["name"]),
577
                    StringConstant("powershell.exe")
578
                )
579
            ),
580
            RepeatQualifier(2)
581
        )
582
    ]),
583
    WithinQualifier(600)  # Within 10 minutes
584
)
585

586
advanced_indicator = Indicator(
587
    name="Suspicious Process Activity",
588
    indicator_types=["malicious-activity"],
589
    pattern_type="stix",
590
    pattern=f"{advanced_pattern}"
591
)
592
```
593

594
### Pattern Validation and Testing
595

596
Validate pattern syntax and test against observed data.
597

598
```python
599
from stix2 import parse_observable
600

601
def validate_pattern_against_data(pattern_str, observed_objects):
602
    """
603
    Validate a STIX pattern against observed data.
604
    Note: This is a simplified example - real pattern matching 
605
    would require a full STIX pattern engine.
606
    """
607
    # Parse observed data
608
    observables = []
609
    for obj_data in observed_objects:
610
        try:
611
            observable = parse_observable(obj_data)
612
            observables.append(observable)
613
        except Exception as e:
614
            print(f"Error parsing observable: {e}")
615
            continue
616
    
617
    # Simple pattern validation (would need full parser in reality)
618
    print(f"Pattern: {pattern_str}")
619
    print(f"Observables to test: {len(observables)}")
620
    
621
    # Example: check if pattern references match observable types
622
    for observable in observables:
623
        print(f"Observable type: {observable.type}")
624
        if observable.type in pattern_str:
625
            print(f"  - Pattern may match {observable.type}")
626

627
# Test data
628
test_observables = [
629
    {
630
        "type": "file",
631
        "hashes": {
632
            "MD5": "d41d8cd98f00b204e9800998ecf8427e"
633
        },
634
        "name": "malware.exe",
635
        "size": 75000
636
    },
637
    {
638
        "type": "ipv4-addr", 
639
        "value": "192.168.1.100"
640
    },
641
    {
642
        "type": "network-traffic",
643
        "protocols": ["tcp"],
644
        "dst_port": 4444
645
    }
646
]
647

648
# Validate patterns
649
validate_pattern_against_data(
650
    "[file:hashes.MD5 = 'd41d8cd98f00b204e9800998ecf8427e' AND file:size > 50000]",
651
    test_observables
652
)
653
```