LLVM 22.x tile for building compilers, language runtimes, and out-of-tree tooling
88
83%
Does it follow best practices?
Impact
96%
1.23xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent uses the correct ORC v2 JIT APIs (LLJIT/LLLazyJIT) instead of deprecated MCJIT, initializes targets in the right order, wraps modules in ThreadSafeModule with per-module contexts, adds DynamicLibrarySearchGenerator, and handles Expected<T>/Error correctly.",
"type": "weighted_checklist",
"checklist": [
{
"name": "ORC v2 API used",
"description": "Code uses LLJIT or LLLazyJIT from llvm/ExecutionEngine/Orc/ — does NOT use MCJIT (ExecutionEngine, EngineBuilder, createJIT, or #include of llvm/ExecutionEngine/MCJIT.h)",
"max_score": 12
},
{
"name": "LLJITBuilder creation",
"description": "JIT instance created via LLJITBuilder().create() (or LLLazyJITBuilder for lazy variant)",
"max_score": 8
},
{
"name": "InitializeNativeTarget before JIT",
"description": "InitializeNativeTarget() is called before the JIT instance is constructed",
"max_score": 10
},
{
"name": "InitializeNativeTargetAsmPrinter",
"description": "InitializeNativeTargetAsmPrinter() is called (without it the JIT cannot emit code)",
"max_score": 8
},
{
"name": "ThreadSafeModule wrapping",
"description": "IR modules are wrapped in ThreadSafeModule with a std::unique_ptr<LLVMContext> (one context per module)",
"max_score": 10
},
{
"name": "DynamicLibrarySearchGenerator",
"description": "DynamicLibrarySearchGenerator::GetForCurrentProcess() is added to the main JITDylib to expose host symbols",
"max_score": 10
},
{
"name": "Error handling with ExitOnError or Expected",
"description": "ORC API calls are wrapped with ExitOnErr() or explicit Expected<T>/Error handling — no unhandled return values",
"max_score": 8
},
{
"name": "addIRModule usage",
"description": "Modules are added to the JIT via JIT->addIRModule(std::move(TSM)) (not the old addModule API)",
"max_score": 8
},
{
"name": "Symbol lookup",
"description": "Functions are looked up via JIT->lookup(\"name\") returning a symbol that is cast to a function pointer via toPtr<>()",
"max_score": 8
},
{
"name": "InitLLVM in main",
"description": "main() initializes with InitLLVM X(argc, argv) before any LLVM operations",
"max_score": 6
},
{
"name": "No shared LLVMContext across modules",
"description": "Does NOT construct multiple ThreadSafeModule instances sharing the same LLVMContext (each TSM gets its own context)",
"max_score": 6
},
{
"name": "CMake llvm_map_components_to_libnames",
"description": "CMakeLists.txt uses llvm_map_components_to_libnames() including OrcJIT component",
"max_score": 6
}
]
}docs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
add-alias-analysis
add-attributes-metadata
add-calling-convention
add-debug-info
add-exception-handling
add-gc-statepoints
add-intrinsic
add-lto
add-sanitizer
add-vectorization-hint
frontend-to-ir
jit-setup
lit-filecheck
lower-struct-types
new-target
out-of-tree-setup
tessl-llvm
version-sync