Data Import and Export Pipeline

Overview { .overview }

Build a data pipeline that imports customer records from CSV format, imports product records from Parquet format, and provides export functionality to CSV for analysis.

Context { .context }

You're building a data management system that needs to handle multiple data sources. Customer data arrives in CSV format, while product catalog data comes in Parquet format. Your task is to import these datasets and provide export capabilities for downstream analysis.

Requirements { .requirements }

Import Customer Data { .requirement }

Implement a function import_customers(csv_path: str, output_path: str) that:

1. Imports customer data from a CSV file
2. The CSV contains columns: customer_id, name, email, country
3. Creates a dataset at the specified output path
4. Returns the created dataset object

Import Product Data { .requirement }

Implement a function import_products(parquet_path: str, output_path: str) that:

1. Imports product data from a Parquet file
2. The Parquet file contains columns: product_id, product_name, category, price
3. Creates a dataset at the specified output path
4. Returns the created dataset object

Export Dataset { .requirement }

Implement a function export_to_csv(dataset_path: str, output_csv_path: str) that:

1. Opens an existing dataset from the specified path
2. Exports the complete dataset to a CSV file
3. Returns the path to the exported CSV file

Test Cases { .test-cases }

Test Case 1: Import CSV Data { .test-case }

@test

Import customer data from CSV format.

Input:

• CSV file customers.csv with 3 customers

Expected:

• Dataset created with 3 rows
• Dataset has columns: customer_id, name, email, country

Test Case 2: Import Parquet Data { .test-case }

@test

Import product data from Parquet format.

Input:

• Parquet file products.parquet with 5 products

Expected:

• Dataset created with 5 rows
• Dataset has columns: product_id, product_name, category, price

Test Case 3: Export to CSV { .test-case }

@test

Export a dataset to CSV format.

Input:

• Existing dataset at ./test_dataset with 10 rows

Expected:

• CSV file created at specified path
• CSV contains all 10 rows
• CSV is readable with standard CSV parsers

Constraints { .constraints }

• Use appropriate libraries for handling different file formats
• Ensure exported CSV files are properly formatted
• The datasets should be created in a format optimized for further analysis

Dependencies { .dependencies }

deeplake { .dependency }

Database for AI providing unified data management and storage capabilities.