tessl/pypi-cloudscraper

Enhanced Python module to bypass Cloudflare's anti-bot page with support for v1, v2, v3 challenges, Turnstile, proxy rotation, and stealth mode.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview

Eval results

Files

CAPTCHA Solving

Name: tessl/pypi-cloudscraper
Author: tessl

Integration with external CAPTCHA solving services for handling Cloudflare challenges that require human verification. CloudScraper supports multiple providers with automatic submission and result retrieval.

Capabilities

Base CAPTCHA Interface

Abstract base class that all CAPTCHA solver providers implement for consistent challenge solving across different services.

class Captcha:
    def __init__(self, name: str):
        """
        Initialize CAPTCHA solver.
        
        Parameters:
        - name: str, provider identifier
        """

    @classmethod
    def dynamicImport(cls, name: str):
        """
        Dynamically import and initialize CAPTCHA provider.
        
        Parameters:
        - name: str, provider name to load
        
        Returns:
        Captcha instance
        
        Raises:
        - ImportError: If provider cannot be loaded
        """

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA challenge using external service.
        
        Parameters:
        - captchaType: str, type of CAPTCHA (hcaptcha, recaptcha, turnstile)
        - url: str, page URL where CAPTCHA appears
        - siteKey: str, CAPTCHA site key
        - captchaParams: dict, additional CAPTCHA parameters
        
        Returns:
        CAPTCHA solution token
        
        Raises:
        - CaptchaServiceUnavailable: If service cannot be reached
        - CaptchaAPIError: If API returns error
        - CaptchaTimeout: If solving takes too long
        """

    def solveCaptcha(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Public interface for solving CAPTCHAs.
        
        Parameters: Same as getCaptchaAnswer()
        Returns: CAPTCHA solution token
        """

2captcha Provider

Integration with 2captcha.com service for solving various CAPTCHA types including reCAPTCHA, hCaptcha, and Turnstile.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize 2captcha solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using 2captcha service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key  
        - captchaParams: dict, additional parameters including:
          - api_key: str, 2captcha API key
          - no_proxy: bool, disable proxy forwarding
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure 2captcha solver
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_2captcha_api_key'
    }
)

# Solve Turnstile challenges automatically
response = scraper.get('https://turnstile-protected-site.com')

# With proxy support
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_api_key',
        'no_proxy': False  # Forward proxy to 2captcha
    },
    proxies={'http': 'http://proxy.example.com:8080'}
)

# Disable proxy forwarding to 2captcha
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha', 
        'api_key': 'your_api_key',
        'no_proxy': True  # Don't send proxy to 2captcha
    },
    proxies={'http': 'http://proxy.example.com:8080'}
)

AntiCaptcha Provider

Integration with Anti-Captcha service for high-quality CAPTCHA solving with various challenge types.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize AntiCaptcha solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using Anti-Captcha service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key
        - captchaParams: dict, additional parameters including:
          - api_key: str, Anti-Captcha API key
          - no_proxy: bool, disable proxy forwarding
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure AntiCaptcha solver
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': 'anticaptcha',
        'api_key': 'your_anticaptcha_api_key'
    }
)

# Solve various CAPTCHA types
response = scraper.get('https://recaptcha-protected-site.com')
response = scraper.get('https://hcaptcha-protected-site.com')

9kw Provider

Integration with 9kw.eu CAPTCHA solving service with configurable timeout settings.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize 9kw solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using 9kw service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key
        - captchaParams: dict, additional parameters including:
          - api_key: str, 9kw API key
          - maxtimeout: int, maximum solving timeout in seconds
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure 9kw solver with custom timeout  
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '9kw',
        'api_key': 'your_9kw_api_key',
        'maxtimeout': 300  # 5 minute timeout
    }
)

# Default timeout is 180 seconds
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '9kw',
        'api_key': 'your_api_key'
    }
)

CapMonster Cloud Provider

Integration with CapMonster Cloud service for fast and reliable CAPTCHA solving.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize CapMonster solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using CapMonster Cloud service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key
        - captchaParams: dict, additional parameters including:
          - clientKey: str, CapMonster client key
          - no_proxy: bool, disable proxy forwarding
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure CapMonster solver
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': 'capmonster',
        'clientKey': 'your_capmonster_client_key'
    }
)

# With proxy configuration
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': 'capmonster',
        'clientKey': 'your_client_key',
        'no_proxy': False  # Forward proxy to CapMonster
    }
)

Capsolver Provider

Integration with Capsolver service for modern CAPTCHA solving including Turnstile support.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize Capsolver solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using Capsolver service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key
        - captchaParams: dict, additional parameters including:
          - api_key: str, Capsolver API key
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure Capsolver
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': 'capsolver',
        'api_key': 'your_capsolver_api_key'
    }
)

# Good for Turnstile challenges
response = scraper.get('https://turnstile-site.com')

DeathByCaptcha Provider

Integration with DeathByCaptcha service using username/password authentication.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize DeathByCaptcha solver."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Solve CAPTCHA using DeathByCaptcha service.
        
        Parameters:
        - captchaType: str, CAPTCHA type
        - url: str, page URL
        - siteKey: str, site key
        - captchaParams: dict, additional parameters including:
          - username: str, DeathByCaptcha username
          - password: str, DeathByCaptcha password
        
        Returns:
        str: CAPTCHA solution token
        """

Usage Examples

# Configure DeathByCaptcha solver
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': 'deathbycaptcha',
        'username': 'your_username',
        'password': 'your_password'
    }
)

Return Response Provider

Special provider that returns the challenge response without solving, useful for manual handling or debugging.

class captchaSolver(Captcha):
    def __init__(self):
        """Initialize return response provider."""

    def getCaptchaAnswer(self, captchaType: str, url: str, siteKey: str, captchaParams: dict):
        """
        Return challenge response without solving.
        
        Returns:
        The original response for manual processing
        """

Usage Examples

# Get challenge response for manual handling
scraper = cloudscraper.create_scraper(
    captcha={'provider': 'return_response'}
)

# This will return the CAPTCHA challenge page instead of solving it
response = scraper.get('https://captcha-protected-site.com')
print("Manual CAPTCHA solving required")
print(response.text)  # Contains CAPTCHA challenge HTML

CAPTCHA Configuration and Usage

Provider Selection

Choose the best CAPTCHA provider for your needs:

providers = {
    '2captcha': {
        'pros': 'Reliable, supports all types, good API',
        'cons': 'Can be slower',
        'config': {'provider': '2captcha', 'api_key': 'key'}
    },
    'anticaptcha': {
        'pros': 'Fast, high quality',
        'cons': 'More expensive',  
        'config': {'provider': 'anticaptcha', 'api_key': 'key'}
    },
    'capsolver': {
        'pros': 'Modern, good Turnstile support',
        'cons': 'Newer service',
        'config': {'provider': 'capsolver', 'api_key': 'key'}
    }
}

# Select provider based on needs
provider = 'capsolver'  # Good for Turnstile
scraper = cloudscraper.create_scraper(
    captcha=providers[provider]['config']
)

CAPTCHA Type Support

Different providers support different CAPTCHA types:

captcha_support = {
    'recaptcha_v2': ['2captcha', 'anticaptcha', 'capmonster', 'capsolver'],
    'recaptcha_v3': ['2captcha', 'anticaptcha', 'capmonster', 'capsolver'],
    'hcaptcha': ['2captcha', 'anticaptcha', 'capmonster', 'capsolver'],
    'turnstile': ['2captcha', 'anticaptcha', 'capmonster', 'capsolver'],
    'funcaptcha': ['2captcha', 'anticaptcha', 'capmonster'],
    'geetest': ['2captcha', 'anticaptcha', 'capmonster']
}

# For Turnstile specifically
turnstile_providers = ['2captcha', 'anticaptcha', 'capmonster', 'capsolver']
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',  # Any Turnstile-compatible provider
        'api_key': 'your_api_key'
    }
)

Proxy Integration with CAPTCHA

Configure how proxies are handled with CAPTCHA services:

# Forward proxy to CAPTCHA service (recommended for accuracy)
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_api_key',
        'no_proxy': False  # Send proxy info to 2captcha
    },
    proxies={'http': 'http://proxy.example.com:8080'}
)

# Don't forward proxy (faster but may be less accurate)
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_api_key', 
        'no_proxy': True  # Don't send proxy to 2captcha
    },
    proxies={'http': 'http://proxy.example.com:8080'}
)

Error Handling and Troubleshooting

CAPTCHA Exception Handling

Handle various CAPTCHA-related errors:

try:
    scraper = cloudscraper.create_scraper(
        captcha={
            'provider': '2captcha',
            'api_key': 'your_api_key'
        }
    )
    response = scraper.get('https://captcha-protected-site.com')
    
except cloudscraper.CaptchaServiceUnavailable:
    print("CAPTCHA service is unavailable")
except cloudscraper.CaptchaAPIError as e:
    print(f"CAPTCHA API error: {e}")
except cloudscraper.CaptchaTimeout:
    print("CAPTCHA solving timed out")
except cloudscraper.CaptchaAccountError:
    print("CAPTCHA account issue (balance, credentials)")
except cloudscraper.CloudflareCaptchaProvider:
    print("CAPTCHA challenge detected but no solver configured")

Provider Fallback

Implement fallback between different CAPTCHA providers:

def create_scraper_with_fallback():
    """Create scraper with CAPTCHA provider fallback."""
    providers = [
        {'provider': '2captcha', 'api_key': 'key1'},
        {'provider': 'anticaptcha', 'api_key': 'key2'},
        {'provider': 'capsolver', 'api_key': 'key3'}
    ]
    
    for captcha_config in providers:
        try:
            scraper = cloudscraper.create_scraper(captcha=captcha_config)
            # Test with a simple request
            response = scraper.get('https://httpbin.org/get')
            if response.status_code == 200:
                print(f"Using provider: {captcha_config['provider']}")
                return scraper
        except Exception as e:
            print(f"Provider {captcha_config['provider']} failed: {e}")
            continue
    
    print("All CAPTCHA providers failed, using no solver")
    return cloudscraper.create_scraper()

scraper = create_scraper_with_fallback()

CAPTCHA Debug Mode

Enable debug output to monitor CAPTCHA solving:

scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_api_key'
    },
    debug=True
)

response = scraper.get('https://turnstile-protected-site.com')

# Debug output shows:
# "Detected Cloudflare Turnstile challenge"
# "Submitting CAPTCHA to 2captcha service..."
# "CAPTCHA solved successfully"
# "Submitting solution to Cloudflare..."

Cost Management

Monitor and manage CAPTCHA solving costs:

# Track CAPTCHA usage
captcha_count = 0

def count_captcha_hook(scraper, response):
    """Hook to count CAPTCHA solves."""
    global captcha_count
    if 'captcha solved' in str(response.history):
        captcha_count += 1
        print(f"CAPTCHAs solved: {captcha_count}")
    return response

scraper = cloudscraper.create_scraper(
    captcha={'provider': '2captcha', 'api_key': 'key'},
    requestPostHook=count_captcha_hook
)

# Limit CAPTCHA attempts per session
max_captchas = 10
if captcha_count >= max_captchas:
    print("CAPTCHA limit reached")
    scraper = cloudscraper.create_scraper()  # No CAPTCHA solver

Advanced CAPTCHA Configuration

Custom CAPTCHA Parameters

Pass additional parameters to CAPTCHA services:

# Custom 2captcha parameters
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': '2captcha',
        'api_key': 'your_api_key',
        'pingback': 'https://your-domain.com/callback',  # Custom callback
        'soft_id': 123  # Software ID for tracking
    }
)

# Custom timeout for different providers
timeouts = {
    '2captcha': 180,
    'anticaptcha': 120,
    '9kw': 300
}

provider = '9kw'
scraper = cloudscraper.create_scraper(
    captcha={
        'provider': provider,
        'api_key': 'your_api_key',
        'maxtimeout': timeouts.get(provider, 180)
    }
)

Install with Tessl CLI