or run

tessl search

Version

Workspace: tessl
Visibility: Public
Created: 18 days ago
Last updated: 2 days ago
Describes: pkg:pypi/kedro@1.1.x

tile.json

tessl/pypi-kedro

tessl install tessl/pypi-kedro@1.1.0

Kedro helps you build production-ready data and analytics pipelines

Agent Success

Agent success rate when using this tile

98%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.32x

Baseline

Agent success rate without this tile

74%

Web Content Fetcher Pipeline

Build a data pipeline that fetches content from multiple web APIs concurrently and processes the results.

Capabilities

Fetch data from multiple URLs

Given a list of 3 URLs, all URLs are fetched and their response status codes are collected into a list @test
Given an empty list of URLs, returns an empty list @test

Process fetched data

Given a list of status codes [200, 200, 404], returns a dictionary with counts: {"success": 2, "errors": 1} @test
Given a list of all successful status codes [200, 200, 200], returns {"success": 3, "errors": 0} @test

Execute pipeline with concurrent fetching

The pipeline executes the fetch and process nodes using thread-based concurrency for I/O-bound operations @test
When executing with 3 worker threads, all fetch operations run concurrently @test

Implementation

@generates

API

def fetch_url(url: str) -> int:
    """
    Fetch a URL and return its HTTP status code.

    Args:
        url: The URL to fetch

    Returns:
        The HTTP status code (e.g., 200, 404, 500)
    """
    pass


def fetch_all_urls(urls: list[str]) -> list[int]:
    """
    Fetch multiple URLs and return their status codes.

    Args:
        urls: List of URLs to fetch

    Returns:
        List of HTTP status codes in the same order as input URLs
    """
    pass


def process_status_codes(status_codes: list[int]) -> dict[str, int]:
    """
    Process a list of status codes and count successes vs errors.

    Args:
        status_codes: List of HTTP status codes

    Returns:
        Dictionary with keys "success" (status 200-299) and "errors" (all others)
    """
    pass


def create_pipeline():
    """
    Create a Kedro pipeline that:
    1. Takes a list of URLs as input
    2. Fetches all URLs (producing status codes)
    3. Processes the status codes into success/error counts

    Returns:
        A Kedro Pipeline object
    """
    pass


def run_pipeline(urls: list[str], max_workers: int = 3) -> dict[str, int]:
    """
    Execute the pipeline with thread-based concurrency.

    Args:
        urls: List of URLs to process
        max_workers: Maximum number of concurrent threads for I/O operations

    Returns:
        Dictionary with success and error counts
    """
    pass

Dependencies { .dependencies }

kedro { .dependency }

Provides pipeline construction and thread-based execution for I/O-bound workloads.

requests { .dependency }

Provides HTTP request functionality for fetching URLs.