Errgroup - Goroutine Group Management

The errgroup package provides synchronization, error propagation, and context cancellation for groups of goroutines working on subtasks of a common task. It extends sync.WaitGroup with error handling and goroutine limiting capabilities.

Import

import "golang.org/x/sync/errgroup"

Overview

errgroup.Group manages a collection of goroutines working on related subtasks. Key features:

• Automatic error propagation from any goroutine to the caller
• Context cancellation when any goroutine returns an error
• Optional limits on concurrent goroutine count
• Zero value is ready to use

API Reference

Creating a Group

WithContext

func WithContext(ctx context.Context) (*Group, context.Context)

Creates a new Group and an associated Context derived from the input context.

Parameters:

• ctx: Parent context to derive from

Returns:

• *Group: New errgroup.Group instance
• context.Context: Derived context that will be canceled when the first function returns a non-nil error or when Wait returns

Behavior: The derived context is canceled automatically in two scenarios:

1. The first time a function passed to Go returns a non-nil error
2. When Wait returns, whichever occurs first

Example:

g, ctx := errgroup.WithContext(context.Background())
// Use ctx in goroutines to detect when another goroutine fails

Group Type

type Group struct {
    // Has unexported fields
}

A Group is a collection of goroutines working on subtasks that are part of the same overall task.

Important constraints:

• A Group should not be reused for different tasks
• The first call to Go must happen before a call to Wait

Zero value behavior:

• A zero Group is valid and ready to use
• Has no limit on the number of active goroutines
• Does not automatically cancel on error (no associated context)

Launching Goroutines

Go

func (g *Group) Go(f func() error)

Calls the given function in a new goroutine.

Parameters:

• f: Function to execute in a new goroutine. Must return an error or nil.

Behavior:

• Blocks until the new goroutine can be added without exceeding the configured limit (if any)
• The first goroutine that returns a non-nil error will cancel the associated context (if created with WithContext)
• The error will be available via Wait
• First call to Go must happen before calling Wait

Error handling:

• Only the first error is captured and returned by Wait
• Subsequent errors are discarded
• Panics in goroutines are not recovered (they will crash the program)

Example:

g, ctx := errgroup.WithContext(context.Background())

g.Go(func() error {
    // Do work here
    if err := doSomething(); err != nil {
        return err  // This error will be returned by Wait
    }
    return nil
})

if err := g.Wait(); err != nil {
    log.Printf("task failed: %v", err)
}

TryGo

func (g *Group) TryGo(f func() error) bool

Attempts to call the given function in a new goroutine only if the number of active goroutines is currently below the configured limit.

Parameters:

• f: Function to execute in a new goroutine. Must return an error or nil.

Returns:

• bool: true if the goroutine was started, false if the limit was reached

Behavior:

• Does not block if the limit is reached; returns false immediately
• If there is no limit set, always returns true (goroutine is always started)
• Same error handling as Go method

Use cases:

• Preventing deadlocks when goroutines need to spawn other goroutines
• Implementing backpressure in producer-consumer patterns
• Graceful degradation when resource limits are reached

Example:

g := &errgroup.Group{}
g.SetLimit(10)

for i := 0; i < 100; i++ {
    i := i
    if !g.TryGo(func() error {
        return processItem(i)
    }) {
        // Limit reached, handle the item differently
        log.Printf("Skipping item %d, limit reached", i)
    }
}

g.Wait()

Waiting for Completion

Wait

func (g *Group) Wait() error

Blocks until all function calls from the Go method have returned, then returns the first non-nil error (if any) from them.

Returns:

• error: The first non-nil error returned by any goroutine, or nil if all succeeded

Behavior:

• Blocks until all launched goroutines complete
• Returns the first error encountered (subsequent errors are ignored)
• If the Group was created with WithContext, cancels the context after all goroutines return
• Can only be called once per Group

Example:

g, ctx := errgroup.WithContext(context.Background())

// Launch multiple goroutines
for i := 0; i < 10; i++ {
    i := i
    g.Go(func() error {
        return processItem(ctx, i)
    })
}

// Wait for all to complete
if err := g.Wait(); err != nil {
    log.Fatalf("processing failed: %v", err)
}

Configuring Limits

SetLimit

func (g *Group) SetLimit(n int)

Limits the number of active goroutines in this group to at most n.

Parameters:

• n: Maximum number of concurrent goroutines
  • Negative value: no limit (removes any existing limit)
  • Zero: prevents any new goroutines from being added
  • Positive value: maximum concurrent goroutines

Behavior:

• Affects subsequent calls to Go (which will block) and TryGo (which will return false)
• Must be called before any goroutines are active, or after all have completed
• Panics if called while goroutines are active

Important: The limit must not be modified while any goroutines in the group are active. This will cause a panic.

Example:

g := &errgroup.Group{}
g.SetLimit(5)  // Allow at most 5 concurrent goroutines

for i := 0; i < 100; i++ {
    i := i
    g.Go(func() error {
        return processItem(i)
    })  // Will block when 5 goroutines are active
}

g.Wait()

Usage Examples

Basic Parallel Execution

package main

import (
    "fmt"
    "golang.org/x/sync/errgroup"
)

func main() {
    var g errgroup.Group

    urls := []string{
        "https://example.com",
        "https://example.org",
        "https://example.net",
    }

    for _, url := range urls {
        url := url  // Capture range variable
        g.Go(func() error {
            return fetch(url)
        })
    }

    if err := g.Wait(); err != nil {
        fmt.Printf("Failed: %v\n", err)
    }
}

With Context Cancellation

package main

import (
    "context"
    "fmt"
    "golang.org/x/sync/errgroup"
    "time"
)

func main() {
    g, ctx := errgroup.WithContext(context.Background())

    // Task 1: Long-running operation
    g.Go(func() error {
        select {
        case <-time.After(10 * time.Second):
            return nil
        case <-ctx.Done():
            return ctx.Err()  // Canceled by another goroutine's error
        }
    })

    // Task 2: Fails quickly
    g.Go(func() error {
        time.Sleep(100 * time.Millisecond)
        return fmt.Errorf("task 2 failed")
    })

    // When task 2 fails, ctx is canceled and task 1 exits early
    if err := g.Wait(); err != nil {
        fmt.Printf("Error: %v\n", err)
    }
}

With Concurrency Limiting

package main

import (
    "fmt"
    "golang.org/x/sync/errgroup"
)

func main() {
    g := &errgroup.Group{}
    g.SetLimit(3)  // Process at most 3 items concurrently

    items := make([]int, 100)
    for i := range items {
        i := i
        g.Go(func() error {
            return processItem(i)
        })
    }

    if err := g.Wait(); err != nil {
        fmt.Printf("Processing failed: %v\n", err)
    }
}

Pipeline Pattern

package main

import (
    "context"
    "golang.org/x/sync/errgroup"
)

func main() {
    g, ctx := errgroup.WithContext(context.Background())

    in := make(chan int)
    out := make(chan int)

    // Producer
    g.Go(func() error {
        defer close(in)
        for i := 0; i < 100; i++ {
            select {
            case in <- i:
            case <-ctx.Done():
                return ctx.Err()
            }
        }
        return nil
    })

    // Workers
    const numWorkers = 5
    for i := 0; i < numWorkers; i++ {
        g.Go(func() error {
            for {
                select {
                case v, ok := <-in:
                    if !ok {
                        return nil
                    }
                    result, err := process(v)
                    if err != nil {
                        return err
                    }
                    select {
                    case out <- result:
                    case <-ctx.Done():
                        return ctx.Err()
                    }
                case <-ctx.Done():
                    return ctx.Err()
                }
            }
        })
    }

    // Consumer
    g.Go(func() error {
        for v := range out {
            if err := consume(v); err != nil {
                return err
            }
        }
        return nil
    })

    if err := g.Wait(); err != nil {
        log.Printf("Pipeline failed: %v", err)
    }
}

Error Handling

Only First Error Returned

errgroup.Group captures only the first error. If multiple goroutines fail, only the first error (by timing) is returned by Wait:

g := &errgroup.Group{}

g.Go(func() error {
    return fmt.Errorf("error 1")
})

g.Go(func() error {
    return fmt.Errorf("error 2")
})

err := g.Wait()  // Returns either "error 1" or "error 2", depending on timing

Panics Are Not Recovered

errgroup does not recover panics in goroutines. If a function panics, it will crash the entire program:

g := &errgroup.Group{}

g.Go(func() error {
    panic("something went wrong")  // This will crash the program
})

g.Wait()  // Never reached

If you need panic recovery, wrap your function:

g.Go(func() (err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("panic: %v", r)
        }
    }()
    // Your code here
    return nil
})

Best Practices

1. Always capture range variables: When using errgroup.Go in a loop, capture the loop variable:

   for _, item := range items {
       item := item  // Capture for goroutine
       g.Go(func() error {
           return process(item)
       })
   }

2. Use SetLimit before launching goroutines: Calling SetLimit while goroutines are active will panic.

3. Don't reuse Groups: Create a new Group for each logical group of tasks.

4. Pass context to goroutines: When using WithContext, pass the derived context to goroutines so they can detect cancellation:

   g, ctx := errgroup.WithContext(context.Background())
   g.Go(func() error {
       return doWork(ctx)  // Pass ctx
   })

5. Consider TryGo for deadlock prevention: If goroutines might need to spawn other goroutines and you're using SetLimit, use TryGo to prevent deadlocks.