or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

apidiff.mdconstraints.mdebnf.mderrors.mdevent.mdgorelease.mdindex.mdio-i2c.mdio-spi.mdjsonrpc2.mdmaps.mdmmap.mdmodgraphviz.mdrand.mdshiny.mdslices.mdslog.mdstats.mdsumdb.mdtrace.mdtxtar.mdtypeparams.mdutf8string.md
tile.json

sumdb.mddocs/

golang.org/x/exp/sumdb - Go Module Sum Database Verification

The sumdb package provides utilities and tools for working with Go's module checksum database system. It includes verification tools for validating go.sum files against a go.sum database server and internal components for handling signed notes, tamper-evident logs, and transparent key-value storage for module verification.

Package Information

  • Package Name: golang.org/x/exp/sumdb
  • Package Type: Go (golang.org/x/exp)
  • Language: Go
  • Installation: go get golang.org/x/exp/sumdb/...
  • Status: Experimental

Core Imports

import (
    "golang.org/x/exp/sumdb/internal/note"
    "golang.org/x/exp/sumdb/internal/sumweb"
    "golang.org/x/exp/sumdb/internal/tkv"
    "golang.org/x/exp/sumdb/internal/tlog"
)

Basic Usage

package main

import (
    "fmt"
    "log"
    "os"
    "os/exec"
)

func main() {
    // Using gosumcheck tool to verify go.sum against database server
    // gosumcheck checks a go.sum file for correctness against the
    // Go checksum database server

    // Command line usage (recommended):
    // gosumcheck -u https://sum.golang.org -k 3+b+DNQhBUhNtjeDVVw/QvT6aE8jQlklKSnoG0q2sL7lMTm92ZvHoDh04I8Fd2yI2H3 go.sum

    // For programmatic access, use the gosumcheck tool directly
    cmd := exec.Command("gosumcheck", "-v", "go.sum")
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr

    if err := cmd.Run(); err != nil {
        log.Fatal(err)
    }
}

Architecture

The sumdb module follows a layered architecture:

  1. gosumcheck - Command-line tool for verifying go.sum files
  2. note - Cryptographic signing and verification of transparent logs
  3. tlog - Tamper-evident append-only log (Merkle tree-based)
  4. tkv - Transparent key-value storage (tile-based)
  5. sumweb - HTTP server and client protocol for serving/accessing the database

Capabilities

Module Sum Database Verification

Verify go.sum files against the official Go module checksum database server using the gosumcheck tool.

// gosumcheck tool: checks a go.sum file against a go.sum database server
//
// Usage:
//    gosumcheck [-h H] [-k key] [-u url] [-v] go.sum
//
// Flags:
//    -h H     Tile height (default 8) - controls the structure of the
//             Merkle tree used for the tamper-evident log
//    -k key   Database server key for signature verification
//    -u url   Override the URL of the server (usually derived from key name)
//    -v       Enable verbose output, showing URLs and elapsed time for requests
//
// Examples:
//    gosumcheck go.sum
//    gosumcheck -v go.sum
//    gosumcheck -u https://sum.golang.org go.sum
//    gosumcheck -h 8 -k <key> go.sum

Description: The gosumcheck tool is a proof-of-concept utility that validates a go.sum file against a go.sum database server. It verifies that all module versions listed in the go.sum file match the checksums stored in the transparent checksum database, ensuring module integrity and detecting tampering.

Important Warnings:

WARNING! WARNING! WARNING!

Gosumcheck is meant as a proof of concept demo and should NOT be used in
production scripts or continuous integration testing because:

1. It does not cache downloaded information from run to run, making it
   expensive to run repeatedly
2. It cannot detect server misbehavior or successful HTTPS man-in-the-middle
   timeline forks
3. It does not set any exit status to report whether problems were found,
   making it unsuitable for automation

For production use, integrate checksum verification directly into your
build pipeline using the Go toolchain's built-in module verification.

Usage Examples:

# Basic verification of go.sum file
gosumcheck go.sum

# Verbose output showing all network requests
gosumcheck -v go.sum

# Verify against specific server URL
gosumcheck -u https://sum.golang.org go.sum

# Custom tile height for Merkle tree structure
gosumcheck -h 8 -k "3+b+DNQhBUhNtjeDVVw/QvT6aE8jQlklKSnoG0q2sL7lMTm92ZvHoDh04I8Fd2yI2H3" go.sum

# Full example with all flags
gosumcheck -h 8 -k <server-key> -u https://sum.golang.org -v go.sum

Note System (Internal)

The note package handles cryptographic signing and verification of transparent logs using a standardized note format.

// Internal package: golang.org/x/exp/sumdb/internal/note
//
// The note package provides:
// - Note format specification following RFC standards
// - Cryptographic signing with Ed25519
// - Signature verification and validation
// - Multi-signature support for signed notes
// - Note encoding and decoding
//
// Not intended for direct public use; used internally by tlog and sumweb

Description: The note system provides a standardized format for cryptographically signed messages used by the transparent log. Each note can contain multiple signatures from different signers, allowing for distributed trust in the checksum database.

Tamper-Evident Log (Internal)

The tlog package implements a Merkle tree-based append-only log for transparent logging of module checksums.

// Internal package: golang.org/x/exp/sumdb/internal/tlog
//
// The tlog package provides:
// - Merkle tree-based append-only log structure
// - Proof generation and verification
// - Tree hash computation
// - Efficient snapshot management
// - Leaf and tree node operations
//
// Not intended for direct public use; used internally by sumweb

Description: The tamper-evident log uses a Merkle tree structure where each leaf represents a module version checksum entry. The structure makes it impossible to modify historical entries without detection, as any change would alter the tree hash. This provides strong guarantees about the immutability of the checksum database.

Transparent Key-Value Storage (Internal)

The tkv package implements tile-based storage for efficient lookup of module checksums using a hierarchical tile structure.

// Internal package: golang.org/x/exp/sumdb/internal/tkv
//
// The tkv package provides:
// - Tile-based key-value storage
// - Merkle tree with tiles for scalability
// - Efficient partial tree downloads
// - Proof generation and verification
// - Tile hash computation and caching
//
// Not intended for direct public use; used internally by sumweb

Description: The tkv (transparent key-value store) uses a tiled Merkle tree structure to make it practical to query and verify checksums for large numbers of modules. Tiles are small Merkle trees that can be independently verified and cached, avoiding the need to download the entire tree for each query.

HTTP Protocol (Internal)

The sumweb package implements the HTTP client and server protocols for accessing and serving the go.sum database.

// Internal package: golang.org/x/exp/sumdb/internal/sumweb
//
// The sumweb package provides:
// - HTTP server for serving sum database tiles
// - HTTP client for querying sum database
// - Tile download and caching
// - Latest log checkpoint retrieval
// - Module checksum lookup
// - Signature verification
//
// Not intended for direct public use; used by tools and libraries

Description: The sumweb protocol defines the HTTP endpoints for communicating with a go.sum database server. Clients use these endpoints to fetch the latest log tree information, retrieve specific tiles, and verify module checksums. The protocol is designed to be stateless and cacheable.

Common Patterns

Working with Module Checksums

The typical workflow for module checksum verification:

// 1. Read a go.sum file
// 2. Connect to the checksum database server
// 3. Fetch the latest tree head (checkpoint)
// 4. For each module in go.sum:
//    - Look up the module's checksum in the database
//    - Compare with go.sum entry
//    - Verify the lookup proof
// 5. Report any mismatches or verification failures

Merkle Tree Verification

The verification process uses Merkle tree proofs:

// All data entries (module checksums) are stored as leaves in a Merkle tree
// Each leaf is hashed, then pairs are combined to form parent nodes
// The root hash (tree hash) represents the entire state of the database
// A proof path from a leaf to the root can verify membership without
// downloading the entire tree

Tile-Based Scalability

The tile structure allows efficient handling of large databases:

// The database is organized into tiles - small Merkle trees
// Each tile contains multiple key-value entries
// Tiles can be independently verified and cached
// Only the minimum necessary tiles are downloaded for verification
// Parent tiles connect child tiles into a complete tree structure

Error Handling

The sumdb tools may encounter several types of errors:

  • Database Server Errors: Connection failures or HTTP errors when contacting the server
  • Checksum Mismatches: Module checksums in go.sum that don't match the database
  • Signature Verification Failures: Invalid signatures on tree heads or tiles
  • Proof Verification Failures: Invalid inclusion proofs for module entries
  • Parsing Errors: Malformed go.sum files or database responses

When using gosumcheck, check the console output for error messages. For programmatic verification, integrate the underlying verification logic into your build system with appropriate error handling.

Related Tools

  • Go toolchain: Built-in module verification using GOSUMDB environment variable
  • Go mod verify: go mod verify command for basic go.sum integrity checks
  • pkg.go.dev: Browser-based interface to search and browse modules

API Status

The sumdb module's main public interface is through the gosumcheck command-line tool. The underlying packages (note, tlog, tkv, sumweb) are marked as internal and subject to change without notice. Applications should use Go's built-in module verification system rather than directly calling these internal APIs.

Additional Resources

  • Official go.sum database: https://sum.golang.org
  • Go modules documentation: https://golang.org/ref/mod
  • Module checksum database design: https://go.googlesource.com/proposal/+/master/design/25530-sumdb.md
  • Transparent logs (RFC 9162): https://tools.ietf.org/html/rfc9162