flake8-comprehensions

A flake8 plugin that helps you write better list, set, and dict comprehensions in Python. The plugin implements over 20 specific rules (C400-C420) that detect unnecessary generators, redundant comprehensions, inefficient patterns, and suboptimal code constructs when working with Python's comprehension syntax.

Package Information

• Package Name: flake8-comprehensions
• Package Type: pypi
• Language: Python
• Installation: pip install flake8-comprehensions
• Requirements: Python 3.9+, flake8 >=3 (!=3.2)

Core Imports

This is a flake8 plugin, so it's not typically imported directly in code. Instead, it integrates automatically with flake8:

# The plugin is automatically loaded by flake8
# No direct import needed in your code

For direct usage (advanced/testing):

from flake8_comprehensions import ComprehensionChecker
import ast

Basic Usage

As a flake8 Plugin (Normal Usage)

After installation, the plugin automatically integrates with flake8 and detects comprehension optimization opportunities:

# Install the plugin
pip install flake8-comprehensions

# Run flake8 - C4xx rules are automatically active
flake8 your_code.py

# Example output:
# your_code.py:5:8: C400 Unnecessary generator - rewrite as a list comprehension.
# your_code.py:12:15: C403 Unnecessary list comprehension - rewrite as a set comprehension.

Configure in setup.cfg or pyproject.toml:

[flake8]
select = C4  # Enable only comprehension rules
# or
ignore = C415,C416  # Disable specific rules

Direct API Usage (Advanced)

import ast
from flake8_comprehensions import ComprehensionChecker

# Parse Python code
code = "foo = list(x + 1 for x in range(10))"
tree = ast.parse(code)

# Create checker instance
checker = ComprehensionChecker(tree)

# Run analysis
for violation in checker.run():
    line, col, message, checker_type = violation
    print(f"Line {line}, Col {col}: {message}")

Architecture

The plugin follows flake8's standard plugin architecture and uses Python's AST (Abstract Syntax Tree) for code analysis:

Plugin Integration

• Entry Point Registration: The plugin registers with flake8 via Python's entry point system under flake8.extension with code prefix C4
• Instantiation: For each Python file, flake8 creates a ComprehensionChecker instance, passing the parsed AST
• Analysis: flake8 calls the run() method which yields tuples of (line, column, message, checker_type)
• Reporting: flake8 formats and reports violations according to user configuration

AST Analysis Pattern

• Tree Walking: Uses ast.walk() to traverse all nodes in the Python AST
• Pattern Matching: Identifies specific AST node patterns (e.g., ast.Call nodes with generators)
• Rule Logic: Each C4xx rule corresponds to specific combinations of AST node types and attributes
• Violation Detection: When a problematic pattern is found, yields structured violation data

Error Code Organization

• C400-C402: Generator to comprehension transformations
• C403-C404: List comprehension to other comprehension types
• C405-C420: Various literal, constructor, and optimization patterns

This architecture enables static analysis of Python code without execution, providing fast and reliable detection of comprehension optimization opportunities.

Capabilities

ComprehensionChecker Class

The main plugin class that analyzes Python AST for comprehension optimization opportunities.

class ComprehensionChecker:
    """
    Flake8 plugin to help you write better list/set/dict comprehensions.
    """
    
    name: str  # "flake8-comprehensions"
    version: str  # Plugin version from package metadata
    messages: dict[str, str]  # Error message templates for all C4xx rules
    tree: ast.AST  # Python AST tree to analyze
    
    def __init__(self, tree: ast.AST) -> None:
        """Initialize checker with AST tree."""
    
    def run(self) -> Generator[tuple[int, int, str, type[Any]]]:
        """
        Main analysis method that yields flake8 violations.
        
        Yields:
        tuple[int, int, str, type]: (line_number, column_offset, message, checker_type)
        """

Helper Functions

Utility functions used internally by the checker:

def has_star_args(call_node: ast.Call) -> bool:
    """Check if AST Call node has starred arguments (*args)."""

def has_double_star_args(call_node: ast.Call) -> bool:
    """Check if AST Call node has double-starred arguments (**kwargs)."""

Constants

comp_type: dict[type[ast.AST], str]
# Maps AST comprehension node types to string names
# {ast.DictComp: "dict", ast.ListComp: "list", ast.SetComp: "set"}

Error Rules Reference

The plugin implements 20 specific rules for comprehension optimization:

Generator to Comprehension Rules (C400-C402)

• C400: Unnecessary generator - rewrite as list comprehension
  • list(f(x) for x in foo) → [f(x) for x in foo]
• C401: Unnecessary generator - rewrite as set comprehension
  • set(f(x) for x in foo) → {f(x) for x in foo}
• C402: Unnecessary generator - rewrite as dict comprehension
  • dict((x, f(x)) for x in foo) → {x: f(x) for x in foo}

List to Other Comprehension Rules (C403-C404)

• C403: Unnecessary list comprehension - rewrite as set comprehension
  • set([f(x) for x in foo]) → {f(x) for x in foo}
• C404: Unnecessary list comprehension - rewrite as dict comprehension
  • dict([(x, f(x)) for x in foo]) → {x: f(x) for x in foo}

Literal Optimization Rules (C405-C406)

• C405: Unnecessary list/tuple literal - rewrite as set literal
  • set([1, 2]) → {1, 2}, set((1, 2)) → {1, 2}
• C406: Unnecessary list/tuple literal - rewrite as dict literal
  • dict([(1, 2)]) → {1: 2}, dict(((1, 2),)) → {1: 2}

Empty Constructor Rules (C408)

• C408: Unnecessary call - rewrite as literal
  • dict() → {}, list() → [], tuple() → ()

Redundant Constructor Rules (C409-C411)

• C409: Unnecessary tuple passed to tuple() - remove outer call or rewrite as literal
  • tuple((1, 2)) → (1, 2), tuple([1, 2]) → (1, 2)
• C410: Unnecessary list passed to list() - remove outer call or rewrite as literal
  • list([1, 2]) → [1, 2], list((1, 2)) → [1, 2]
• C411: Unnecessary list call - remove outer call to list()
  • list([f(x) for x in foo]) → [f(x) for x in foo]

Function Call Optimization Rules (C413-C415)

• C413: Unnecessary list/reversed call around sorted()
  • list(sorted([2, 3, 1])) → sorted([2, 3, 1])
  • reversed(sorted([2, 3, 1])) → sorted([2, 3, 1], reverse=True)
• C414: Unnecessary call within other constructor calls
  • list(list(iterable)) → list(iterable)
  • set(sorted(iterable)) → set(iterable)
• C415: Unnecessary subscript reversal within functions
  • set(iterable[::-1]) → set(iterable)
  • reversed(iterable[::-1]) → iterable

Identity Comprehension Rules (C416, C420)

• C416: Unnecessary comprehension - rewrite using constructor
  • [x for x in iterable] → list(iterable)
  • {x for x in iterable} → set(iterable)
  • {a: b for a, b in iterable} → dict(iterable)
• C420: Unnecessary dict comprehension - rewrite using dict.fromkeys()
  • {x: 1 for x in iterable} → dict.fromkeys(iterable, 1)
  • {x: None for x in iterable} → dict.fromkeys(iterable)

Lambda and Map Rules (C417)

• C417: Unnecessary map usage - rewrite using comprehension
  • map(lambda x: x + 1, iterable) → (x + 1 for x in iterable)
  • list(map(lambda x: x * 2, nums)) → [x * 2 for x in nums]

Dict Constructor Rules (C418)

• C418: Unnecessary dict/dict comprehension passed to dict()
  • dict({}) → {}, dict({"a": 1}) → {"a": 1}

Short-circuiting Rules (C419)

• C419: Unnecessary list comprehension in any/all() prevents short-circuiting
  • all([condition(x) for x in iterable]) → all(condition(x) for x in iterable)
  • any([condition(x) for x in iterable]) → any(condition(x) for x in iterable)

Plugin Integration

The plugin integrates with flake8 through Python's entry point system:

# pyproject.toml entry point configuration
[project.entry-points."flake8.extension"]
C4 = "flake8_comprehensions:ComprehensionChecker"

This registers the plugin with flake8 to handle all C4xx rule codes. The plugin follows flake8's standard interface:

1. Instantiation: flake8 creates a ComprehensionChecker instance for each Python file, passing the parsed AST
2. Analysis: flake8 calls the run() method to get a generator of violations
3. Reporting: Each violation is a tuple of (line, column, message, checker_type)

Usage Examples

Common Patterns Detected

# C400: Generator to list comprehension
# Bad
numbers = list(x * 2 for x in range(10))
# Good  
numbers = [x * 2 for x in range(10)]

# C403: List comprehension to set comprehension
# Bad
unique_squares = set([x**2 for x in numbers])
# Good
unique_squares = {x**2 for x in numbers}

# C411: Unnecessary list() around list comprehension
# Bad
processed = list([process(x) for x in items])
# Good
processed = [process(x) for x in items]

# C417: Map with lambda to comprehension
# Bad
doubled = list(map(lambda x: x * 2, numbers))
# Good
doubled = [x * 2 for x in numbers]

# C419: List comprehension in any/all
# Bad - builds entire list before checking
valid = all([is_valid(item) for item in items])
# Good - stops at first False
valid = all(is_valid(item) for item in items)

Integration with Development Workflow

# In pre-commit hooks
repos:
  - repo: https://github.com/PyCQA/flake8
    rev: 6.0.0
    hooks:
      - id: flake8
        additional_dependencies: [flake8-comprehensions]

# In CI/CD pipelines
flake8 --select=C4 src/
# Returns exit code 1 if any C4xx violations found

# With specific rule configuration
flake8 --ignore=C416,C419 src/  # Ignore specific rules
flake8 --select=C400,C401,C402 src/  # Only generator rules