or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

dirhash.md gosumcheck.md index.md modfile.md module.md note.md semver.md storage.md sumdb.md tlog.md zip.md

tile.json

sumdb - Checksum Database Client and Server

Package sumdb implements the HTTP protocols for serving or accessing a module checksum database.

Import

import "golang.org/x/mod/sumdb"

Overview

The sumdb package provides both client and server implementations for Go's transparent checksum database. The checksum database allows verification that module checksums haven't been tampered with and provides a verifiable log of all module versions.

Key features:

Client: Fetch and verify module checksums from a checksum database
Server: Serve checksum database over HTTP
Transparency: Uses transparent log structure (see tlog)
Security: Cryptographic verification of all data

The protocol is described in Go Checksum Database.

Variables

var ErrGONOSUMDB = errors.New("skipped (listed in GONOSUMDB)")

ErrGONOSUMDB is returned by Client.Lookup for paths that match a pattern listed in the GONOSUMDB list (set by Client.SetGONOSUMDB, usually from the environment variable).

var ErrSecurity = errors.New("security error: misbehaving server")

ErrSecurity is returned by Client operations that invoke Client.SecurityError.

var ErrWriteConflict = errors.New("write conflict")

ErrWriteConflict signals a write conflict during Client.WriteConfig.

var ServerPaths = []string{
    "/lookup/",
    "/latest",
    "/tile/",
}

ServerPaths are the URL paths the Server can (and should) serve. Typically a server will do:

srv := sumdb.NewServer(ops)
for _, path := range sumdb.ServerPaths {
    http.Handle(path, srv)
}

Types

Client

type Client struct {
    // Has unexported fields.
}

A Client is a client connection to a checksum database. All the methods are safe for simultaneous use by multiple goroutines.

Constructor

func NewClient(ops ClientOps) *Client

Returns a new Client using the given ClientOps.

Example:

client := sumdb.NewClient(&myClientOps{})

Methods

func (c *Client) Lookup(path, vers string) (lines []string, err error)

Returns the go.sum lines for the given module path and version. The version may end in a /go.mod suffix, in which case Lookup returns the go.sum lines for the module's go.mod-only hash.

Example:

lines, err := client.Lookup("golang.org/x/text", "v0.3.0")
if err != nil {
    log.Fatal(err)
}
for _, line := range lines {
    fmt.Println(line)
}
// Output:
// golang.org/x/text v0.3.0 h1:...
// golang.org/x/text v0.3.0/go.mod h1:...

func (c *Client) SetGONOSUMDB(list string)

Sets the list of comma-separated GONOSUMDB patterns for the Client. For any module path matching one of the patterns, Client.Lookup will return ErrGONOSUMDB.

SetGONOSUMDB can be called at most once, and if so it must be called before the first call to Lookup.

Example:

client.SetGONOSUMDB("*.corp.example.com,private.io/repo")

func (c *Client) SetTileHeight(height int)

Sets the tile height for the Client. Any call to SetTileHeight must happen before the first call to Client.Lookup. If SetTileHeight is not called, the Client defaults to tile height 8.

SetTileHeight can be called at most once, and if so it must be called before the first call to Lookup.

Example:

client.SetTileHeight(10)

ClientOps

type ClientOps interface {
    // ReadRemote reads and returns the content served at the given path
    // on the remote database server. The path begins with "/lookup" or "/tile/",
    // and there is no need to parse the path in any way.
    // It is the implementation's responsibility to turn that path into a full URL
    // and make the HTTP request. ReadRemote should return an error for
    // any non-200 HTTP response status.
    ReadRemote(path string) ([]byte, error)

    // ReadConfig reads and returns the content of the named configuration file.
    // There are only a fixed set of configuration files.
    //
    // "key" returns a file containing the verifier key for the server.
    //
    // serverName + "/latest" returns a file containing the latest known
    // signed tree from the server.
    // To signal that the client wishes to start with an "empty" signed tree,
    // ReadConfig can return a successful empty result (0 bytes of data).
    ReadConfig(file string) ([]byte, error)

    // WriteConfig updates the content of the named configuration file,
    // changing it from the old []byte to the new []byte.
    // If the old []byte does not match the stored configuration,
    // WriteConfig must return ErrWriteConflict.
    // Otherwise, WriteConfig should atomically replace old with new.
    // The "key" configuration file is never written using WriteConfig.
    WriteConfig(file string, old, new []byte) error

    // ReadCache reads and returns the content of the named cache file.
    // Any returned error will be treated as equivalent to the file not existing.
    // There can be arbitrarily many cache files, such as:
    //    serverName/lookup/pkg@version
    //    serverName/tile/8/1/x123/456
    ReadCache(file string) ([]byte, error)

    // WriteCache writes the named cache file.
    WriteCache(file string, data []byte)

    // Log prints the given log message (such as with log.Print)
    Log(msg string)

    // SecurityError prints the given security error log message.
    // The Client returns ErrSecurity from any operation that invokes SecurityError,
    // but the return value is mainly for testing. In a real program,
    // SecurityError should typically print the message and call log.Fatal or os.Exit.
    SecurityError(msg string)
}

A ClientOps provides the external operations (file caching, HTTP fetches, and so on) needed by the Client. The methods must be safe for concurrent use by multiple goroutines.

Server

type Server struct {
    // Has unexported fields.
}

A Server is the checksum database HTTP server, which implements http.Handler and should be invoked to serve the paths listed in ServerPaths.

Constructor

func NewServer(ops ServerOps) *Server

Returns a new Server using the given operations.

Example:

server := sumdb.NewServer(&myServerOps{})
for _, path := range sumdb.ServerPaths {
    http.Handle(path, server)
}

Methods

func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request)

Implements http.Handler.

ServerOps

type ServerOps interface {
    // Signed returns the signed hash of the latest tree.
    Signed(ctx context.Context) ([]byte, error)

    // ReadRecords returns the content for the n records id through id+n-1.
    ReadRecords(ctx context.Context, id, n int64) ([][]byte, error)

    // Lookup looks up a record for the given module,
    // returning the record ID.
    Lookup(ctx context.Context, m module.Version) (int64, error)

    // ReadTileData reads the content of tile t.
    // It is only invoked for hash tiles (t.L ≥ 0).
    ReadTileData(ctx context.Context, t tlog.Tile) ([]byte, error)
}

A ServerOps provides the external operations (underlying database access and so on) needed by the Server.

TestServer

type TestServer struct {
    // Has unexported fields.
}

A TestServer is an in-memory implementation of ServerOps for testing.

Constructor

func NewTestServer(signer string, gosum func(path, vers string) ([]byte, error)) *TestServer

Constructs a new TestServer that will sign its tree with the given signer key (see note) and fetch new records as needed by calling gosum.

Example:

signer := "test+key+..." // Ed25519 signer key
gosum := func(path, vers string) ([]byte, error) {
    // Return go.sum lines for the given module
    return []byte("golang.org/x/text v0.3.0 h1:...\n"), nil
}
testServer := sumdb.NewTestServer(signer, gosum)

Methods

func (s *TestServer) Signed(ctx context.Context) ([]byte, error)
func (s *TestServer) ReadRecords(ctx context.Context, id, n int64) ([][]byte, error)
func (s *TestServer) Lookup(ctx context.Context, m module.Version) (int64, error)
func (s *TestServer) ReadTileData(ctx context.Context, t tlog.Tile) ([]byte, error)

Usage Examples

Implementing a Simple Client

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
    "path/filepath"

    "golang.org/x/mod/sumdb"
)

// SimpleClientOps implements sumdb.ClientOps
type SimpleClientOps struct {
    serverURL string
    cacheDir  string
}

func (ops *SimpleClientOps) ReadRemote(path string) ([]byte, error) {
    url := ops.serverURL + path
    resp, err := http.Get(url)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode != 200 {
        return nil, fmt.Errorf("HTTP %d", resp.StatusCode)
    }

    return io.ReadAll(resp.Body)
}

func (ops *SimpleClientOps) ReadConfig(file string) ([]byte, error) {
    path := filepath.Join(ops.cacheDir, "config", file)
    data, err := os.ReadFile(path)
    if os.IsNotExist(err) {
        return []byte{}, nil
    }
    return data, err
}

func (ops *SimpleClientOps) WriteConfig(file string, old, new []byte) error {
    path := filepath.Join(ops.cacheDir, "config", file)
    os.MkdirAll(filepath.Dir(path), 0755)

    // Simple implementation - in production, use atomic writes
    return os.WriteFile(path, new, 0644)
}

func (ops *SimpleClientOps) ReadCache(file string) ([]byte, error) {
    path := filepath.Join(ops.cacheDir, "cache", file)
    return os.ReadFile(path)
}

func (ops *SimpleClientOps) WriteCache(file string, data []byte) {
    path := filepath.Join(ops.cacheDir, "cache", file)
    os.MkdirAll(filepath.Dir(path), 0755)
    os.WriteFile(path, data, 0644)
}

func (ops *SimpleClientOps) Log(msg string) {
    log.Println(msg)
}

func (ops *SimpleClientOps) SecurityError(msg string) {
    log.Fatalf("SECURITY ERROR: %s", msg)
}

func main() {
    ops := &SimpleClientOps{
        serverURL: "https://sum.golang.org",
        cacheDir:  "./sumdb-cache",
    }

    client := sumdb.NewClient(ops)

    // Look up a module
    lines, err := client.Lookup("golang.org/x/text", "v0.3.0")
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println("go.sum lines:")
    for _, line := range lines {
        fmt.Println(line)
    }
}

Implementing a Simple Server

package main

import (
    "context"
    "fmt"
    "log"
    "net/http"
    "sync"

    "golang.org/x/mod/module"
    "golang.org/x/mod/sumdb"
    "golang.org/x/mod/sumdb/tlog"
)

// SimpleServerOps implements sumdb.ServerOps
type SimpleServerOps struct {
    mu      sync.Mutex
    records map[string]int64 // module@version -> record ID
    data    [][]byte         // record contents
    tree    []byte           // signed tree
}

func (ops *SimpleServerOps) Signed(ctx context.Context) ([]byte, error) {
    ops.mu.Lock()
    defer ops.mu.Unlock()
    return ops.tree, nil
}

func (ops *SimpleServerOps) ReadRecords(ctx context.Context, id, n int64) ([][]byte, error) {
    ops.mu.Lock()
    defer ops.mu.Unlock()

    if id < 0 || id+n > int64(len(ops.data)) {
        return nil, fmt.Errorf("invalid record range")
    }

    result := make([][]byte, n)
    for i := int64(0); i < n; i++ {
        result[i] = ops.data[id+i]
    }
    return result, nil
}

func (ops *SimpleServerOps) Lookup(ctx context.Context, m module.Version) (int64, error) {
    ops.mu.Lock()
    defer ops.mu.Unlock()

    key := m.Path + "@" + m.Version
    id, ok := ops.records[key]
    if !ok {
        return 0, fmt.Errorf("not found: %s", key)
    }
    return id, nil
}

func (ops *SimpleServerOps) ReadTileData(ctx context.Context, t tlog.Tile) ([]byte, error) {
    // Simplified - real implementation would store and retrieve tiles
    return nil, fmt.Errorf("not implemented")
}

func main() {
    ops := &SimpleServerOps{
        records: make(map[string]int64),
        data:    make([][]byte, 0),
    }

    server := sumdb.NewServer(ops)

    // Register server paths
    for _, path := range sumdb.ServerPaths {
        http.Handle(path, server)
    }

    log.Println("Starting checksum database server on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Using GONOSUMDB Patterns

package main

import (
    "fmt"
    "log"

    "golang.org/x/mod/sumdb"
)

func main() {
    client := sumdb.NewClient(&myClientOps{})

    // Configure private module patterns
    client.SetGONOSUMDB("*.corp.example.com,private.io/*")

    // This lookup will succeed
    lines, err := client.Lookup("golang.org/x/text", "v0.3.0")
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("Public module lookup succeeded")

    // This lookup will return ErrGONOSUMDB
    _, err = client.Lookup("git.corp.example.com/repo", "v1.0.0")
    if err == sumdb.ErrGONOSUMDB {
        fmt.Println("Private module skipped (GONOSUMDB)")
    }
}

Testing with TestServer

package main

import (
    "context"
    "fmt"
    "log"

    "golang.org/x/mod/module"
    "golang.org/x/mod/sumdb"
    "golang.org/x/mod/sumdb/note"
)

func main() {
    // Generate a test key
    signer, verifier, err := note.GenerateKey(nil, "test-server")
    if err != nil {
        log.Fatal(err)
    }

    // Create a gosum function
    gosum := func(path, vers string) ([]byte, error) {
        // Return mock go.sum lines
        line := fmt.Sprintf("%s %s h1:test-hash\n", path, vers)
        return []byte(line), nil
    }

    // Create test server
    testServer := sumdb.NewTestServer(signer, gosum)

    // Test lookup
    m := module.Version{Path: "example.com/test", Version: "v1.0.0"}
    id, err := testServer.Lookup(context.Background(), m)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Module %s@%s has record ID: %d\n", m.Path, m.Version, id)

    // Read records
    records, err := testServer.ReadRecords(context.Background(), id, 1)
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("Record content: %s\n", string(records[0]))

    fmt.Printf("Verifier key: %s\n", verifier)
}

Client with Custom Tile Height

package main

import (
    "log"

    "golang.org/x/mod/sumdb"
)

func main() {
    client := sumdb.NewClient(&myClientOps{})

    // Set custom tile height (default is 8)
    // Larger tiles reduce the number of HTTP requests
    // but increase individual request sizes
    client.SetTileHeight(10)

    lines, err := client.Lookup("golang.org/x/text", "v0.3.0")
    if err != nil {
        log.Fatal(err)
    }

    for _, line := range lines {
        log.Println(line)
    }
}

Handling Security Errors

package main

import (
    "log"
    "os"

    "golang.org/x/mod/sumdb"
)

type SecureClientOps struct {
    // ... other fields ...
}

func (ops *SecureClientOps) SecurityError(msg string) {
    // Log the security error
    log.Printf("SECURITY ERROR: %s\n", msg)

    // In production, this should:
    // 1. Alert administrators
    // 2. Log to security monitoring
    // 3. Exit immediately

    os.Exit(1)
}

// ... implement other ClientOps methods ...

func main() {
    client := sumdb.NewClient(&SecureClientOps{})

    // If the server misbehaves (e.g., provides inconsistent data),
    // SecurityError will be called and the program will exit
    _, err := client.Lookup("golang.org/x/text", "v0.3.0")
    if err == sumdb.ErrSecurity {
        // This will only be reached in tests where SecurityError
        // returns instead of calling os.Exit
        log.Fatal("Security error detected")
    }
}

Caching Strategy

package main

import (
    "crypto/sha256"
    "encoding/hex"
    "os"
    "path/filepath"
    "time"

    "golang.org/x/mod/sumdb"
)

type CachingClientOps struct {
    cacheDir string
    cacheTTL time.Duration
}

func (ops *CachingClientOps) ReadCache(file string) ([]byte, error) {
    path := filepath.Join(ops.cacheDir, file)

    // Check if cache exists and is fresh
    info, err := os.Stat(path)
    if err != nil {
        return nil, err
    }

    if time.Since(info.ModTime()) > ops.cacheTTL {
        // Cache expired
        return nil, os.ErrNotExist
    }

    return os.ReadFile(path)
}

func (ops *CachingClientOps) WriteCache(file string, data []byte) {
    path := filepath.Join(ops.cacheDir, file)
    os.MkdirAll(filepath.Dir(path), 0755)

    // Atomic write using temp file
    tmpPath := path + ".tmp"
    if err := os.WriteFile(tmpPath, data, 0644); err != nil {
        return
    }
    os.Rename(tmpPath, path)
}

func (ops *CachingClientOps) cacheKey(file string) string {
    // Generate consistent cache keys
    hash := sha256.Sum256([]byte(file))
    return hex.EncodeToString(hash[:])
}

// ... implement other ClientOps methods ...

func main() {
    ops := &CachingClientOps{
        cacheDir: "./sumdb-cache",
        cacheTTL: 24 * time.Hour,
    }

    client := sumdb.NewClient(ops)

    // Subsequent lookups will use cache
    lines, err := client.Lookup("golang.org/x/text", "v0.3.0")
    if err != nil {
        panic(err)
    }

    for _, line := range lines {
        println(line)
    }
}

Protocol Details

Client-Server Communication

The checksum database uses a simple HTTP-based protocol:

Lookup: GET /lookup/{module}@{version}
- Returns go.sum lines for the module
Latest: GET /latest
- Returns the latest signed tree
Tile: GET /tile/{height}/{level}/{tile}[.p/{width}]
- Returns tile data for transparent log

Verification Process

Client fetches the latest signed tree
Client verifies the tree signature
Client looks up module checksums
Client verifies inclusion proofs
Client caches verified data

Security Considerations

Key Management

Store verifier keys securely
Never distribute signer keys
Rotate keys according to policy

Error Handling

Always implement SecurityError to exit on misbehavior
Log all security events
Monitor for unusual patterns

Cache Integrity

Validate cached data before use
Use atomic writes to prevent corruption
Consider cache expiration policies

Performance Tips

Tile Height: Larger tiles (10-12) reduce requests but increase transfer size
Caching: Implement persistent caching to avoid repeated lookups
Batch Operations: Process multiple modules together when possible
Connection Reuse: Use HTTP keepalive for multiple requests

Version

Tile

Files

sumdb.mddocs/

sumdb - Checksum Database Client and Server

Import

Overview

Variables

Types

Client

Constructor

Methods

ClientOps

Server

Constructor

Methods

ServerOps

TestServer

Constructor

Methods

Usage Examples

Implementing a Simple Client

Implementing a Simple Server

Using GONOSUMDB Patterns

Testing with TestServer

Client with Custom Tile Height

Handling Security Errors

Caching Strategy

Protocol Details

Client-Server Communication

Verification Process

Security Considerations

Key Management

Error Handling

Cache Integrity

Performance Tips

See Also

Version

Tile

Files

sumdb.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

sumdb - Checksum Database Client and Server

Import

Overview

Variables

Types

Client

Constructor

Methods

ClientOps

Server

Constructor

Methods

ServerOps

TestServer

Constructor

Methods

Usage Examples

Implementing a Simple Client

Implementing a Simple Server

Using GONOSUMDB Patterns

Testing with TestServer

Client with Custom Tile Height

Handling Security Errors

Caching Strategy

Protocol Details

Client-Server Communication

Verification Process

Security Considerations

Key Management

Error Handling

Cache Integrity

Performance Tips

See Also

sumdb.mddocs/