dlt Dashboard Development Guide

Consult this skill whenever working on the dashboard codebase. It tells you where files live, what each module does, and where new code belongs.

Overview

The dashboard is a marimo-based reactive web UI for inspecting dlt pipelines, schemas, data, and run traces. It lives at dlt/_workspace/helpers/dashboard/.

File Layout

dlt/_workspace/helpers/dashboard/
├── dlt_dashboard.py          # Main marimo app (~30 reactive cells)
├── dlt_dashboard_styles.css  # CSS: colors, badges, timeline, tables
├── runner.py                 # Subprocess launcher, CLI integration
├── config.py                 # DashboardConfiguration @configspec dataclass
├── const.py                  # Constants: port (2718), timeouts, colors, defaults
├── typing.py                 # Shared TypedDicts (TTableListItem, TLoadItem, etc.)
├── strings.py                # All user-facing UI text (~350 lines)
└── utils/
    ├── __init__.py           # Package exports
    ├── pipeline.py           # get_pipeline(), pipeline_details(), exception handling
    ├── queries.py            # SQL execution, caching (BoundedDict), row counts, loads
    ├── schema.py             # Table/column lists, schema version fetching
    ├── trace.py              # Trace rendering, sanitization
    ├── visualization.py      # Execution timeline HTML, load status badges
    ├── home.py               # Home page / workspace home rendering
    ├── ui.py                 # dlt_table(), section_marker(), error_callout(), BoundedDict
    ├── formatters.py         # Datetime humanization, string formatting
    └── data_quality.py       # dlthub data quality integration

Architecture

• Framework: marimo — a reactive Python notebook that renders as a web app
• Cell pattern: @app.cell(hide_code=True) decorators define reactive cells in dlt_dashboard.py
• Dependency tracking: Cells automatically re-execute when their parameter dependencies change
• Routing: URL query params (?pipeline=X&profile=Y) via mo.query_params()
• Caching: BoundedDict (LRU, 64/32 entries) in utils/queries.py and utils/schema.py
• Styling: CSS variables for dlt brand colors (--dlt-color-lime, --dlt-color-aqua, --dlt-color-purple, --dlt-color-pink)

Dashboard Sections (cells in dlt_dashboard.py)

1. Profile & Pipeline Discovery — reads workspace profiles, lists local pipelines
2. Pipeline Attachment — dlt.attach(), file watcher for auto-refresh
3. Overview — pipeline properties, destination info, state version
4. Schema — table browser with configurable column display
5. Browse Data — SQL editor with cached results and query history
6. Trace — execution timeline, step details, resolved config
7. Loads — load history with row counts and schema versions
8. State — raw JSON state viewer
9. Data Quality — integration with dlthub.data_quality
10. Ibis Backend — read-only Ibis connection

How the Dashboard is Launched

Three CLI commands all call runner.run_dashboard():

• dlt dashboard — workspace-level (DashboardCommand in dlt/_workspace/cli/commands.py)
• dlt pipeline <name> show — pipeline-specific (dlt/_workspace/cli/_pipeline_command.py)
• dlt workspace show — workspace-level (dlt/_workspace/cli/_workspace_command.py)

Edit mode (--edit) ejects dlt_dashboard.py and CSS to the cwd for user customization.

Dependencies

From pyproject.toml [workspace] extra: marimo>=0.14.5, duckdb>=0.9, ibis-framework>=12.0.0, pyarrow>=16.0.0.

Where to Put New Code

Coding Conventions

• All user-facing text goes in strings.py, never inline in cells
• Cell-local variables must start with _ (underscore prefix). In marimo, variables that are only used within a single cell and should not be shared across cells must be prefixed with _. Only variables intended to be consumed by other cells should be unprefixed.
• UI helpers: Use utils/ui.py functions (dlt_table(), section(), error_callout()) for consistent UI
• CSS classes: section-header, pipeline-execution-timeline, status-badge, dlt_table
• Badge classes: .badge-grey, .badge-yellow, .badge-green, .badge-red
• Data structures: TypedDicts in typing.py for data passed between cells
• Configuration: @configspec dataclass in config.py

Testing

Test locations

tests/workspace/helpers/dashboard/    # Unit and integration tests
├── conftest.py                       # Fixtures: pipeline types and temp directories
├── test_pipeline_workflow.py         # Pipeline data retrieval integration
├── test_all_cells.py                 # Cell execution and dependency testing
├── test_visualization.py             # Timeline and badge rendering
├── test_ui_elements.py              # UI component styling
├── test_table_operations.py         # Table listing/filtering
├── test_trace_operations.py         # Trace parsing/display
├── test_query_operations.py         # SQL query execution
├── test_loads.py                    # Load history retrieval
├── test_data_formatting.py          # String formatting
├── test_pipeline_operations.py      # Pipeline metadata
└── test_dashboard_config.py         # Configuration resolution

tests/e2e/                           # Playwright browser tests

Running tests

make test-workspace-dashboard          # unit/integration tests
make test-e2e-dashboard                # headless chromium e2e
make test-e2e-dashboard-headed         # visible browser e2e
make create-test-pipelines             # create pipelines for manual testing

Test fixtures

Tests use parametrized pipeline fixtures via ALL_PIPELINES and PIPELINES_WITH_LOAD constants in conftest.py. Key fixtures: success_pipeline_duckdb, success_pipeline_filesystem, extract_exception_pipeline, never_ran_pipeline, load_exception_pipeline.

Where to add new tests

• Utils logic: Add to the matching test_*.py file (e.g., new query helper → test_query_operations.py)
• New section/cell: Add cell execution test to test_all_cells.py, plus a dedicated test file if the section has significant logic
• Visual/rendering: test_visualization.py or test_ui_elements.py
• E2E browser tests: tests/e2e/