Dataset Access Logger

A monitoring system that tracks and logs all dataset access operations (loads and saves) during pipeline execution.

Requirements

Build a system that monitors dataset operations by:

1. Recording every time a dataset is loaded, including the dataset name and a timestamp
2. Recording every time a dataset is saved, including the dataset name and a timestamp
3. Providing a method to retrieve the complete access log as a list of dictionaries

Each log entry should be a dictionary containing:

• dataset_name: Name of the dataset accessed
• operation: Either "load" or "save"
• timestamp: ISO 8601 formatted timestamp string

Test Cases

• When a dataset is successfully loaded, a log entry is created with operation "load" @test
• When a dataset is successfully saved, a log entry is created with operation "save" @test
• Multiple dataset operations are logged in chronological order @test
• The get_log method returns all recorded entries @test

Implementation

@generates

API

class DatasetAccessLogger:
    """Monitors and logs dataset access operations during pipeline execution."""

    def get_log(self) -> list[dict]:
        """
        Retrieve the complete access log.

        Returns:
            A list of dictionaries, each containing:
            - dataset_name (str): Name of the dataset
            - operation (str): Either "load" or "save"
            - timestamp (str): ISO 8601 formatted timestamp
        """
        pass

Dependencies { .dependencies }

kedro { .dependency }

Provides the pipeline and dataset lifecycle hook system.