0
# Status Detection
1
2
PyPI API integration for checking package upgrade availability, supporting both JSON API and simple HTML formats, with custom index URL handling.
3
4
## Capabilities
5
6
### Package Status Detection
7
8
Queries PyPI or custom package indexes to detect available upgrades for packages from requirements files.
9
10
```python { .api }
11
class PackagesStatusDetector:
12
def __init__(self, packages, use_default_index=False):
13
"""
14
Initialize package status detector.
15
16
Args:
17
packages (list): List of package specification strings from requirements files
18
use_default_index (bool): If True, skip custom index URL detection and use PyPI
19
"""
20
21
def detect_available_upgrades(self, options):
22
"""
23
Detect available upgrades for all packages.
24
25
Args:
26
options (dict): Options dictionary containing:
27
--prerelease (bool): Include prerelease versions
28
-p (list): List of explicitly selected packages or ['all']
29
30
Returns:
31
dict: Package status map with package names as keys and status dicts as values
32
"""
33
```
34
35
### Custom Index URL Configuration
36
37
Automatically detects custom PyPI index URLs from pip configuration files and environment variables.
38
39
```python { .api }
40
def _update_index_url_from_configs(self):
41
"""
42
Check for alternative index-url in pip configuration files.
43
44
Priority order:
45
1. PIP_INDEX_URL environment variable
46
2. pip.conf/pip.ini files in various locations:
47
- ~/.pip/pip.conf, ~/.pip/pip.ini
48
- ~/.config/pip/pip.conf, ~/.config/pip/pip.ini
49
- Virtual environment pip.conf, pip.ini
50
- System-wide pip configuration files
51
52
Updates self.PYPI_API_URL and self.PYPI_API_TYPE accordingly.
53
"""
54
55
def _prepare_api_url(self, index_url):
56
"""
57
Prepare API URL based on index URL format.
58
59
Args:
60
index_url (str): Base index URL from configuration
61
62
Returns:
63
str: API URL template for package queries
64
65
Supported formats:
66
- Standard PyPI JSON API: /pypi/{package}/json
67
- Simple HTML API: /simple/{package}/
68
- Custom formats with /pypi/ in path
69
"""
70
```
71
72
### Package Information Fetching
73
74
Fetches package information from PyPI or custom indexes with timeout and error handling.
75
76
```python { .api }
77
def _fetch_index_package_info(self, package_name, current_version):
78
"""
79
Fetch package information from the configured index.
80
81
Args:
82
package_name (str): Name of the package to query
83
current_version (packaging.version.Version): Current version from requirements
84
85
Returns:
86
tuple: (package_status_dict, reason) where:
87
package_status_dict (dict or False): Package status information or False on error
88
reason (str): Success message or error description
89
90
Network configuration:
91
- 15 second timeout for HTTP requests
92
- Handles HTTPError and connection issues
93
- Uses canonical package names for simple HTML API
94
"""
95
```
96
97
### Package Line Parsing
98
99
Parses package specification lines to extract package names and versions.
100
101
```python { .api }
102
def _expand_package(self, package_line):
103
"""
104
Extract package name and version from requirements line.
105
106
Args:
107
package_line (str): Package specification (e.g., "django==3.2.0")
108
109
Returns:
110
tuple: (package_name, version) or (None, None) if not parseable
111
112
Supports:
113
- Exact version pins: package==1.0.0
114
- Package extras: package[extra1,extra2]==1.0.0
115
- Filters out packages without == specifications
116
"""
117
```
118
119
## API Format Support
120
121
### PyPI JSON API
122
123
Default format using PyPI's JSON API endpoint.
124
125
```python { .api }
126
def _parse_pypi_json_package_info(self, package_name, current_version, response):
127
"""
128
Parse PyPI JSON API response for package information.
129
130
Args:
131
package_name (str): Package name
132
current_version (packaging.version.Version): Current version
133
response (requests.Response): HTTP response from PyPI JSON API
134
135
Returns:
136
tuple: (package_status_dict, reason)
137
138
JSON API features:
139
- Full version history in releases dict
140
- Upload timestamps for each release
141
- Package metadata and description
142
- Handles prerelease and postrelease versions
143
"""
144
```
145
146
### Simple HTML API
147
148
Fallback format for custom package indexes that provide simple HTML listing.
149
150
```python { .api }
151
def _parse_simple_html_package_info(self, package_name, current_version, response):
152
"""
153
Parse simple HTML API response for package information.
154
155
Args:
156
package_name (str): Package name
157
current_version (packaging.version.Version): Current version
158
response (requests.Response): HTTP response from simple HTML API
159
160
Returns:
161
tuple: (package_status_dict, reason)
162
163
HTML API features:
164
- Regex parsing of package links
165
- Version extraction from filenames
166
- No upload timestamp information (returns '-')
167
- Handles prerelease versions through version parsing
168
"""
169
```
170
171
## Configuration Sources
172
173
### Environment Variables
174
175
```bash
176
export PIP_INDEX_URL="https://pypi.company.com/simple/"
177
```
178
179
Takes highest priority over configuration files.
180
181
### Pip Configuration Files
182
183
Searched in order of priority:
184
185
```
186
~/.pip/pip.conf # User-specific (Unix)
187
~/.pip/pip.ini # User-specific (Windows)
188
~/.config/pip/pip.conf # XDG config (Unix)
189
~/.config/pip/pip.ini # XDG config (Windows)
190
$VIRTUAL_ENV/pip.conf # Virtual environment
191
$VIRTUAL_ENV/pip.ini # Virtual environment
192
[system locations] # System-wide configurations
193
```
194
195
Configuration format:
196
```ini
197
[global]
198
index-url = https://pypi.company.com/simple/
199
```
200
201
## Usage Examples
202
203
### Basic Status Detection
204
205
```python
206
from pip_upgrader.packages_status_detector import PackagesStatusDetector
207
from packaging import version
208
209
# Initialize with packages from requirements files
210
packages = ['django==3.2.0', 'requests==2.25.1']
211
detector = PackagesStatusDetector(packages)
212
213
# Check for upgrades
214
options = {'--prerelease': False, '-p': []}
215
status_map = detector.detect_available_upgrades(options)
216
217
# Example output
218
print(status_map)
219
{
220
'django': {
221
'name': 'django',
222
'current_version': Version('3.2.0'),
223
'latest_version': Version('4.1.0'),
224
'upgrade_available': True,
225
'upload_time': '2022-08-03 08:15:30'
226
},
227
'requests': {
228
'name': 'requests',
229
'current_version': Version('2.25.1'),
230
'latest_version': Version('2.25.1'),
231
'upgrade_available': False,
232
'upload_time': '2021-01-05 14:22:18'
233
}
234
}
235
```
236
237
### Custom Index Usage
238
239
```python
240
# Use custom index from environment or config
241
detector = PackagesStatusDetector(packages, use_default_index=False)
242
243
# Force default PyPI index
244
detector = PackagesStatusDetector(packages, use_default_index=True)
245
```
246
247
### Prerelease Handling
248
249
```python
250
# Include prerelease versions
251
options = {'--prerelease': True, '-p': []}
252
status_map = detector.detect_available_upgrades(options)
253
254
# This might return version 4.2.0rc1 instead of 4.1.0 for django
255
```
256
257
### Explicit Package Selection
258
259
```python
260
# Only check specific packages
261
options = {'--prerelease': False, '-p': ['django', 'flask']}
262
status_map = detector.detect_available_upgrades(options)
263
264
# Check all packages
265
options = {'--prerelease': False, '-p': ['all']}
266
status_map = detector.detect_available_upgrades(options)
267
```
268
269
## Package Status Dictionary
270
271
Each package in the returned status map contains:
272
273
```python
274
{
275
'name': str, # Package name as specified in requirements
276
'current_version': Version, # Current version from requirements file
277
'latest_version': Version, # Latest available version from index
278
'upgrade_available': bool, # True if latest > current
279
'upload_time': str # Upload timestamp (JSON API) or '-' (HTML API)
280
}
281
```
282
283
## Version Handling
284
285
### Version Comparison
286
287
Uses the `packaging` library for robust version comparison:
288
289
```python
290
from packaging import version
291
292
current = version.parse('1.0.0')
293
latest = version.parse('1.1.0')
294
upgrade_available = current < latest # True
295
```
296
297
### Prerelease Logic
298
299
- **Default behavior**: Excludes prerelease and postrelease versions
300
- **Prerelease flag**: Includes all prerelease/postrelease versions
301
- **Current version logic**: If current version is prerelease/postrelease, includes prereleases even without flag
302
303
### Version Parsing Edge Cases
304
305
- **Non-RFC versions**: Falls back to version from package info when release parsing fails
306
- **Missing releases**: Returns error when no valid versions found
307
- **Complex version schemes**: Handles PEP 440 compliant versions through packaging library
308
309
## Error Handling
310
311
### Network Errors
312
313
- **HTTP timeouts**: 15-second timeout with graceful failure
314
- **HTTP errors**: Catches HTTPError exceptions and returns error message
315
- **Connection issues**: Network connectivity problems are handled with error messages
316
317
### Parsing Errors
318
319
- **Invalid JSON**: JSON parsing errors result in error status
320
- **Missing data**: Missing fields in API responses are handled gracefully
321
- **Version parsing**: Invalid version strings are caught and reported
322
323
### API Compatibility
324
325
- **Index format detection**: Automatically determines JSON vs HTML API format
326
- **Canonical names**: Uses packaging.utils.canonicalize_name for HTML APIs
327
- **Fallback handling**: Graceful degradation when API format is unsupported