tessl/pypi-import-tracker

A tool for managing dependencies in a modular python project by tracking which dependencies are needed by which sub-modules

—

Pending

Overview

Eval results

Files

Setup Tools Integration

Name: tessl/pypi-import-tracker
Author: tessl

Automatic generation of setuptools configuration from dependency analysis, bridging import tracking with package management by computing install_requires and extras_require.

Capabilities

Requirements Parsing

Parses requirements files or lists and generates setuptools-compatible install_requires and extras_require dictionaries based on actual module dependencies.

def parse_requirements(
    requirements: Union[List[str], str],
    library_name: str,
    extras_modules: Optional[List[str]] = None,
    full_depth: bool = True,
    keep_optional: Union[bool, Dict[str, List[str]]] = False,
    **kwargs,
) -> Tuple[List[str], Dict[str, List[str]]]:
    """
    Parse requirements and generate install_requires and extras_require for setuptools.
    
    Uses dependency tracking to map each module within the project to its specific
    dependencies, then allocates requirements between install_requires (common/required)
    and extras_require (module-specific/optional).
    
    Args:
        requirements: The list of requirements entries, or a file path pointing to a
                     requirements file
        library_name: The top-level name of the library package
        extras_modules: List of module names that should be used to generate
                       extras_require sets
        full_depth: Include transitive dependencies of direct third-party dependencies
        keep_optional: Indicate which optional dependencies should be kept when computing
                      extras sets. If True, all optional dependencies are kept.
                      If False, none are kept. If dict, maps modules to lists of
                      optional dependencies to keep.
        **kwargs: Additional keyword arguments to pass through to track_module
        
    Returns:
        requirements: The list of requirements to pass to setup()
        extras_require: The extras_require dict to pass to setup()
    """

Basic Setup Integration

Simple integration for generating setuptools configuration:

import setuptools
from import_tracker.setup_tools import parse_requirements

# Parse requirements from file
install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package'
)

# Use in setup.py
setuptools.setup(
    name='my_package',
    version='1.0.0',
    install_requires=install_requires,
    extras_require=extras_require,
    packages=setuptools.find_packages(),
)

Requirements File Parsing

Parse dependencies from requirements.txt files:

# requirements.txt content:
# requests>=2.25.0
# numpy>=1.20.0
# matplotlib>=3.0.0  # Optional for plotting
# seaborn>=0.11.0    # Optional for advanced plotting

install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package',
    extras_modules=['my_package.plotting']
)

print(install_requires)
# ['requests>=2.25.0', 'numpy>=1.20.0']

print(extras_require)
# {
#   'my_package.plotting': ['matplotlib>=3.0.0', 'seaborn>=0.11.0'],
#   'all': ['matplotlib>=3.0.0', 'seaborn>=0.11.0']
# }

In-Memory Requirements

Pass requirements as a list instead of a file:

requirements_list = [
    'requests>=2.25.0',
    'numpy>=1.20.0', 
    'pandas>=1.3.0',
    'matplotlib>=3.0.0'
]

install_requires, extras_require = parse_requirements(
    requirements=requirements_list,
    library_name='my_package',
    extras_modules=[
        'my_package.data_processing',
        'my_package.visualization'
    ]
)

Multiple Extras Groups

Generate multiple extras_require sets for different feature groups:

install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package',
    extras_modules=[
        'my_package.web',      # Web scraping features
        'my_package.ml',       # Machine learning features  
        'my_package.viz'       # Visualization features
    ]
)

print(extras_require)
# {
#   'my_package.web': ['requests', 'beautifulsoup4'],
#   'my_package.ml': ['scikit-learn', 'tensorflow'], 
#   'my_package.viz': ['matplotlib', 'plotly'],
#   'all': ['requests', 'beautifulsoup4', 'scikit-learn', 'tensorflow', 'matplotlib', 'plotly']
# }

Optional Dependency Handling

Control which optional dependencies are included in extras:

# Keep all optional dependencies
install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package',
    extras_modules=['my_package.optional_features'],
    keep_optional=True
)

# Keep no optional dependencies (only required ones)
install_requires, extras_require = parse_requirements(
    requirements='requirements.txt', 
    library_name='my_package',
    extras_modules=['my_package.optional_features'],
    keep_optional=False
)

# Keep specific optional dependencies per module
install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package', 
    extras_modules=['my_package.optional_features'],
    keep_optional={
        'my_package.optional_features': ['matplotlib', 'seaborn']
    }
)

Advanced Configuration

Fine-tune dependency analysis behavior:

install_requires, extras_require = parse_requirements(
    requirements='requirements.txt',
    library_name='my_package',
    extras_modules=['my_package.advanced'],
    full_depth=True,           # Include transitive third-party deps
    keep_optional=True,        # Keep optional dependencies
    # Pass additional track_module arguments
    detect_transitive=True,    # Distinguish direct vs transitive
    show_optional=True         # Track optional dependency status
)

Complete Setup.py Example

Comprehensive setup.py using import_tracker for dependency management:

"""A setuptools setup module for my_package"""

import os
import setuptools
from import_tracker.setup_tools import parse_requirements

# Read the README to provide the long description
base_dir = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(base_dir, "README.md"), "r") as handle:
    long_description = handle.read()

# Determine requirements file path
requirements_file = os.path.join(base_dir, "requirements.txt")

# Parse requirement sets using import_tracker
install_requires, extras_require = parse_requirements(
    requirements_file=requirements_file,
    library_name="my_package",
    extras_modules=[
        "my_package.web_scraping",
        "my_package.data_analysis", 
        "my_package.machine_learning",
        "my_package.visualization"
    ],
    full_depth=True,
    keep_optional=True
)

# Perform the standard setup call
setuptools.setup(
    name="my_package",
    version="1.0.0",
    author="Your Name",
    author_email="your.email@example.com",
    description="A package with managed dependencies",
    long_description=long_description,
    long_description_content_type="text/markdown",
    url="https://github.com/yourusername/my_package",
    license="MIT",
    classifiers=[
        "Development Status :: 4 - Beta",
        "Intended Audience :: Developers",
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python :: 3",
        "Programming Language :: Python :: 3.7",
        "Programming Language :: Python :: 3.8",
        "Programming Language :: Python :: 3.9",
        "Programming Language :: Python :: 3.10",
        "Programming Language :: Python :: 3.11",
    ],
    keywords=["dependencies", "packaging", "imports"],
    packages=setuptools.find_packages(),
    python_requires=">=3.7",
    install_requires=install_requires,
    extras_require=extras_require,
    entry_points={
        "console_scripts": [
            "my-package=my_package.cli:main",
        ],
    },
)

Integration with Lazy Import Errors

Combine setup tools integration with lazy import error handling:

# In your package's __init__.py
from import_tracker import lazy_import_errors
from import_tracker.setup_tools import parse_requirements

# Function to get extras modules for error messages
def get_extras_modules():
    """Return the set of modules managed as extras"""
    return {
        'my_package.web_scraping',
        'my_package.data_analysis',
        'my_package.visualization'
    }

# Enable lazy errors with extras integration
lazy_import_errors(get_extras_modules=get_extras_modules)

# Import all submodules - won't crash if dependencies missing
from . import web_scraping
from . import data_analysis  
from . import visualization

When a user tries to use a feature with missing dependencies:

import my_package

# This will show helpful installation message
my_package.visualization.plot_data()
# ModuleNotFoundError: No module named 'matplotlib'. 
# To install the missing dependencies, run `pip install my_package[my_package.visualization]`

Dependency Mapping

The system automatically maps Python import names to package names:

# Import name -> Package name mapping examples
{
    'requests': 'requests',
    'numpy': 'numpy', 
    'pandas': 'pandas',
    'matplotlib': 'matplotlib',
    'PIL': 'pillow',           # Special case mapping
    'cv2': 'opencv-python',    # Special case mapping
    'sklearn': 'scikit-learn'  # Special case mapping
}

Return Values

install_requires Format

List of requirement strings compatible with setuptools:

[
    'requests>=2.25.0',
    'numpy>=1.20.0',
    'click>=8.0.0'
]

extras_require Format

Dictionary mapping extras names to requirement lists:

{
    'my_package.web': ['beautifulsoup4>=4.9.0', 'lxml>=4.6.0'],
    'my_package.data': ['pandas>=1.3.0', 'openpyxl>=3.0.0'],
    'my_package.viz': ['matplotlib>=3.4.0', 'seaborn>=0.11.0'],
    'all': ['beautifulsoup4>=4.9.0', 'lxml>=4.6.0', 'pandas>=1.3.0', 
            'openpyxl>=3.0.0', 'matplotlib>=3.4.0', 'seaborn>=0.11.0']
}

Types

from typing import Dict, List, Optional, Tuple, Union

# Function parameter types
RequirementsInput = Union[List[str], str]
LibraryName = str
ExtrasModules = Optional[List[str]]
KeepOptional = Union[bool, Dict[str, List[str]]]

# Return types
InstallRequires = List[str]
ExtrasRequire = Dict[str, List[str]]
ParseRequirementsReturn = Tuple[InstallRequires, ExtrasRequire]

# Requirements mapping types
RequirementSpec = str  # e.g., 'requests>=2.25.0'
PackageName = str      # e.g., 'requests'
ModuleName = str       # e.g., 'my_package.web'

Install with Tessl CLI