tessl/pypi-connectorx

Load data from databases to dataframes, the fastest way.

1.04x

Quality

Pending

Does it follow best practices?

Impact

86%

1.04x

Average score across 10 eval scenarios

Overview

Eval results

Files

Database Performance Benchmark Tool

Name: tessl/pypi-connectorx
Rating: 0.86 (1 reviews)
Author: tessl

Build a database performance benchmark tool that efficiently loads and compares large datasets from a database to measure memory usage and execution time.

Capabilities

Load Large Dataset

Load a dataset with at least 100,000 rows from a database connection and return it as a dataframe @test
Measure and return the peak memory usage during the data loading operation @test

Performance Comparison

Execute the same query twice: once using a standard approach and once using an optimized approach, measuring execution time for both @test
The optimized approach should complete faster than the standard approach for large datasets @test

Implementation

@generates

API

def load_dataset(connection_string: str, query: str) -> tuple:
    """
    Load a dataset from a database and track memory usage.

    Args:
        connection_string: Database connection string
        query: SQL query to execute

    Returns:
        A tuple of (dataframe, peak_memory_mb) where:
        - dataframe: The loaded data as a pandas DataFrame
        - peak_memory_mb: Peak memory usage in megabytes during loading
    """
    pass

def compare_performance(connection_string: str, query: str) -> dict:
    """
    Compare execution time between standard and optimized data loading approaches.

    Args:
        connection_string: Database connection string
        query: SQL query to execute (should return at least 100K rows for meaningful comparison)

    Returns:
        A dictionary containing:
        {
            'standard_time_seconds': float,
            'optimized_time_seconds': float,
            'speedup_factor': float  # standard_time / optimized_time
        }
    """
    pass

Dependencies { .dependencies }

connectorx { .dependency }

Provides high-performance database-to-dataframe loading with memory optimization.

pandas { .dependency }

Provides standard dataframe operations and pandas.read_sql for comparison baseline.

psutil { .dependency }

Provides memory usage tracking capabilities.

Install with Tessl CLI

npx tessl i tessl/pypi-connectorx