embedded-coverage-strategy-reference

Overview

This skill is a pure reference consumed by the per-tool skills (googletest-embedded-arm, unity-test-framework-c, ceedling-build-runner, qemu-system-test-runner) and by the HIL reference (hardware-in-loop-reference).

When to use

• Choosing a coverage criterion for a new embedded test suite - what does "enough coverage" mean for this project?
• Wiring gcov or llvm-cov into the cross-compile build.
• Reading a coverage report and translating between gcov, LCOV, and llvm-cov formats.
• Negotiating coverage requirements against a safety standard (MISRA-C, DO-178C, ISO 26262, IEC 62304).
• Deciding whether to instrument the host build, the QEMU build, or the on-target build.

How to use

1. Pick the coverage criterion the project's safety level demands (DAL A / ASIL D map to MC/DC; DAL B to decision; DAL C / ASIL A to statement) from the criteria hierarchy - full standards mapping in references/safety-standards-and-formats.md.
2. Pick the toolchain: gcov for GCC / ARM-GCC / AVR-GCC; LLVM source-based coverage for clang (MC/DC needs clang -fcoverage-mcdc).
3. Instrument the build at -O0 (--coverage for gcc; -fprofile-instr-generate -fcoverage-mapping for clang) - never optimise a coverage build.
4. Pick where to run (host / QEMU / on-target) from the instrumentation trade-off table.
5. Produce lcov.info as the durable artefact, render HTML for the developer, and gate the PR on changed-line coverage - see Coverage gates that work.

The full gcov-to-HTML path for a single module is in Worked example below.

Coverage criteria hierarchy

The standard hierarchy, from weakest to strongest. ISTQB glossary terms (cite by stable term ID; the glossary is a JS SPA and not WebFetchable):

The escalation matters because higher criteria find different defect classes - branch coverage finds an unreached else; MC/DC finds a short-circuit-evaluated condition whose change never alters the decision.

gcov toolchain (GCC, ARM-GCC, AVR-GCC)

Per gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html:

Compilation

Build the program (or test harness) with --coverage - a convenience alias that "tells the compiler to generate additional information needed by gcov (basically a flow graph of the program) and also includes additional code in the object files for generating the extra profiling information" (per the GCC gcov invocation page).

arm-none-eabi-gcc --coverage -O0 -g \
    main.c parser.c tests.c -o test_binary
# Equivalent to: -fprofile-arcs -ftest-coverage -lgcov

Compilation produces a .gcno file (flow-graph) per source file "at compile time". Running the binary produces a .gcda file per source file with the accumulated execution counts (both per the GCC docs).

Running gcov

./test_binary           # writes parser.gcda etc.
gcov -b -c parser.c     # produces parser.c.gcov text report

The full gcov flag table and the .gcov annotation sentinels (##### for unexecuted, - for non-executable) are in references/gcov-flag-reference.md.

LCOV info format

For HTML reports + CI integration, post-process with lcov/genhtml:

lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory coverage-html/

LCOV's .info is the de-facto interchange format consumed by Codecov, Coveralls, SonarQube. (LCOV is not GCC; it is a separate Linux Test Project tool that wraps gcov.)

LLVM source-based coverage (clang, arm-linux-clang)

Per clang.llvm.org/docs/SourceBasedCodeCoverage.html:

Compilation

clang --target=arm-none-eabi \
      -fprofile-instr-generate -fcoverage-mapping \
      -O0 -g main.c parser.c tests.c -o test_binary

The two flags are independent: -fprofile-instr-generate enables "instrumentation based profiling"; -fcoverage-mapping emits the mapping that "describes the mapping between the source ranges and the profiling instrumentation counters" (per the LLVM Coverage Mapping Format docs at llvm.org/docs/CoverageMappingFormat.html).

For MC/DC, add -fcoverage-mcdc per the Clang docs.

Running + profile merge

Running the binary writes default.profraw in the current directory (or to the path in LLVM_PROFILE_FILE, with pattern strings %p for PID, %h for hostname, %Nm for merge-pool - per the Clang docs).

LLVM_PROFILE_FILE="raw/%p.profraw" ./test_binary
llvm-profdata merge -sparse raw/*.profraw -o test.profdata

-sparse "produces smaller indexed profiles" (per Clang docs).

Reports

llvm-cov show   ./test_binary -instr-profile=test.profdata \
                              -format=html -output-dir=cov-html/
llvm-cov report ./test_binary -instr-profile=test.profdata
llvm-cov export ./test_binary -instr-profile=test.profdata \
                              -format=lcov > coverage.info

show emits per-line annotations, report emits the file-level summary table, export -format=lcov produces a file compatible with the gcov-flavoured LCOV .info consumed by Codecov / SonarQube. (Per the same Clang Source-Based Code Coverage page.)

LLVM coverage mapping regions

The LLVM coverage-mapping format (per llvm.org/docs/CoverageMappingFormat.html) distinguishes:

• Code regions - associate source ranges with counters.
• Skipped regions - preprocessor-excluded code (e.g. #ifdef branches not taken at compile time).
• Expansion regions - macro expansions, so a macro that fires from multiple call sites has separate coverage per site.
• Branch regions - true / false condition paths (added with -fcoverage-mapping).

This region taxonomy is why llvm-cov can show separate counts inside macro expansions - gcov cannot.

On-target vs host vs QEMU instrumentation

A practical trade-off for embedded teams:

For most safety-critical projects, the standard recipe is host build for fast loop, QEMU build for arch sanity, on-target build for the certification artefact.

Worked example

gcov branch coverage on one module, host build (the fast-loop default from the instrumentation table).

# 1. instrument + build on the host at -O0 (writes parser.gcno at compile time)
gcc --coverage -O0 -g parser.c parser_test.c -o parser_test

# 2. run - writes parser.gcda next to parser.gcno
./parser_test

# 3. branch counts as a text report
gcov -b -c parser.c          # -> parser.c.gcov (scan for ##### lines)

# 4. interchange artefact + HTML for the team
lcov --capture --directory . --output-file coverage.info
genhtml coverage.info --output-directory coverage-html/

# 5. gate the PR on coverage.info branch totals

The host loop is fastest, but it misses MCU-specific paths - pair it with at least one QEMU or on-target run before claiming the number for the MCU (see the instrumentation table and the "coverage measured on host then claimed for the MCU" anti-pattern below).

Safety-standard coverage expectations

Coverage targets are set by the project's safety standard, not by taste. DAL A / ASIL D demand MC/DC; DAL B demands decision coverage; DAL C / ASIL A demand statement coverage; MISRA-C:2012 and IEC 62304 Class C prescribe no numeric target but expect a structural-coverage justification. The full per-standard table (DO-178C DAL A-D, ISO 26262 ASIL A-D, MISRA-C:2012 §8, IEC 62304) is in references/safety-standards-and-formats.md. Treat every listed level as the floor, not a turnkey recipe.

Coverage report formats

Produce lcov.info as the durable artefact, render HTML for the developer, and gate on the .info totals. gcov also emits text .gcov and .gcov.json.gz; llvm-cov emits .profdata, HTML, and lcov / JSON exports; gcovr --xml produces Cobertura for Jenkins. The full producer / consumer matrix is in references/safety-standards-and-formats.md.

Coverage gates that work

A flat "global ≥85% branch" gate is the failure mode - it penalises the team for unreviewed legacy code and rewards removing tests for hard-to-cover error paths.

Anti-patterns

Limitations

• Statement coverage is the weakest meaningful target - a 100%-statement-covered suite can miss every else branch.
• Branch coverage from gcov has known imprecision on short-circuit && / || - these compile to two branches; the count attributes to the source line, not the individual operand. For per-operand visibility, use clang -fcoverage-mcdc per the Clang docs.
• .gcda files accumulate across runs. Re-run without deleting them and counters keep climbing. Use __gcov_reset() (provided by libgcov) to zero between scenarios - per the gcov source.
• Counter overflow. Default counters are 64-bit on modern GCC but were 32-bit historically - on a long-running on-target run, check gcov-tool overlap for saturated counts.
• No path coverage from either toolchain. Path coverage ("path coverage" per ISTQB) is exponential and neither gcov nor llvm-cov measures it. For path-sensitive testing pair with fuzzing or symbolic execution.

References

Cited inline above. Foundational documents:

• GCC gcov manual - gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html.
• Clang Source-Based Code Coverage - clang.llvm.org/docs/SourceBasedCodeCoverage.html.
• LLVM Coverage Mapping Format - llvm.org/docs/CoverageMappingFormat.html.
• ARM GNU Toolchain docs (for --specs=rdimon.specs semihosting) - developer.arm.com/Tools and Software/GNU Toolchain.
• ISTQB glossary V4.7.1 terms cited by stable term ID (glossary is a JS SPA at glossary.istqb.org - not WebFetchable).
• MISRA-C:2012 §8 Coverage Guidance (gated standard - cite by stable ID).
• DO-178C §6.4.4 Structural Coverage (gated standard - cite by stable ID).
• ISO 26262-6:2018 Table 12 (gated standard - cite by stable ID).
• Deep references (with their own citations): the full gcov flag table in references/gcov-flag-reference.md; the safety-standard mapping and report-format matrix in references/safety-standards-and-formats.md.