0
# Linear Models
1
2
Accelerated linear regression, logistic regression, and regularized models with Intel optimization. These implementations provide significant performance improvements for large datasets through optimized matrix operations and Intel hardware acceleration.
3
4
## Capabilities
5
6
### Linear Regression
7
8
Intel-accelerated ordinary least squares linear regression with optimized matrix decomposition.
9
10
```python { .api }
11
class LinearRegression:
12
"""
13
Ordinary least squares Linear Regression with Intel optimization.
14
15
Provides significant speedups through optimized BLAS operations
16
and Intel Math Kernel Library (MKL) integration.
17
"""
18
19
def __init__(
20
self,
21
fit_intercept=True,
22
copy_X=True,
23
n_jobs=None,
24
positive=False
25
):
26
"""
27
Initialize Linear Regression.
28
29
Parameters:
30
fit_intercept (bool): Whether to calculate intercept
31
copy_X (bool): Whether to copy X or overwrite
32
n_jobs (int): Number of jobs for computation
33
positive (bool): Whether to force positive coefficients
34
"""
35
36
def fit(self, X, y, sample_weight=None):
37
"""
38
Fit linear model.
39
40
Parameters:
41
X (array-like): Training data of shape (n_samples, n_features)
42
y (array-like): Target values
43
sample_weight (array-like): Individual sample weights
44
45
Returns:
46
self: Fitted estimator
47
"""
48
49
def predict(self, X):
50
"""
51
Predict using the linear model.
52
53
Parameters:
54
X (array-like): Input data
55
56
Returns:
57
array: Predicted values
58
"""
59
60
def score(self, X, y, sample_weight=None):
61
"""
62
Return coefficient of determination R² of prediction.
63
64
Parameters:
65
X (array-like): Test samples
66
y (array-like): True values
67
sample_weight (array-like): Sample weights
68
69
Returns:
70
float: R² score
71
"""
72
73
# Attributes available after fitting
74
coef_: ... # Estimated coefficients
75
intercept_: ... # Independent term
76
n_features_in_: ... # Number of features during fit
77
feature_names_in_: ... # Feature names during fit
78
```
79
80
### Logistic Regression
81
82
Intel-optimized logistic regression for classification with accelerated solver algorithms.
83
84
```python { .api }
85
class LogisticRegression:
86
"""
87
Logistic Regression classifier with Intel optimization.
88
89
Uses optimized solvers and Intel MKL for faster convergence
90
and improved performance on large datasets.
91
"""
92
93
def __init__(
94
self,
95
penalty='l2',
96
dual=False,
97
tol=1e-4,
98
C=1.0,
99
fit_intercept=True,
100
intercept_scaling=1,
101
class_weight=None,
102
random_state=None,
103
solver='lbfgs',
104
max_iter=100,
105
multi_class='auto',
106
verbose=0,
107
warm_start=False,
108
n_jobs=None,
109
l1_ratio=None
110
):
111
"""
112
Initialize Logistic Regression.
113
114
Parameters:
115
penalty (str): Regularization penalty ('l1', 'l2', 'elasticnet', 'none')
116
dual (bool): Dual or primal formulation
117
tol (float): Tolerance for stopping criteria
118
C (float): Inverse of regularization strength
119
fit_intercept (bool): Whether to fit intercept
120
intercept_scaling (float): Scaling for intercept
121
class_weight (dict): Weights for classes
122
random_state (int): Random state for reproducibility
123
solver (str): Algorithm for optimization
124
max_iter (int): Maximum iterations
125
multi_class (str): Multi-class strategy
126
verbose (int): Verbosity level
127
warm_start (bool): Whether to reuse previous solution
128
n_jobs (int): Number of parallel jobs
129
l1_ratio (float): ElasticNet mixing parameter
130
"""
131
132
def fit(self, X, y, sample_weight=None):
133
"""
134
Fit the logistic regression model.
135
136
Parameters:
137
X (array-like): Training data
138
y (array-like): Target values
139
sample_weight (array-like): Sample weights
140
141
Returns:
142
self: Fitted estimator
143
"""
144
145
def predict(self, X):
146
"""
147
Predict class labels.
148
149
Parameters:
150
X (array-like): Input data
151
152
Returns:
153
array: Predicted class labels
154
"""
155
156
def predict_proba(self, X):
157
"""
158
Predict class probabilities.
159
160
Parameters:
161
X (array-like): Input data
162
163
Returns:
164
array: Class probabilities
165
"""
166
167
def predict_log_proba(self, X):
168
"""
169
Predict logarithm of class probabilities.
170
171
Parameters:
172
X (array-like): Input data
173
174
Returns:
175
array: Log probabilities
176
"""
177
178
def decision_function(self, X):
179
"""
180
Predict confidence scores.
181
182
Parameters:
183
X (array-like): Input data
184
185
Returns:
186
array: Confidence scores
187
"""
188
189
# Attributes available after fitting
190
coef_: ... # Coefficients
191
intercept_: ... # Intercept
192
classes_: ... # Class labels
193
n_iter_: ... # Number of iterations
194
```
195
196
### Ridge Regression
197
198
L2-regularized linear regression with Intel-optimized solvers.
199
200
```python { .api }
201
class Ridge:
202
"""
203
Ridge regression with Intel optimization.
204
205
Linear least squares with L2 regularization, using optimized
206
solvers for improved performance on large datasets.
207
"""
208
209
def __init__(
210
self,
211
alpha=1.0,
212
fit_intercept=True,
213
copy_X=True,
214
max_iter=None,
215
tol=1e-4,
216
solver='auto',
217
positive=False,
218
random_state=None
219
):
220
"""
221
Initialize Ridge regression.
222
223
Parameters:
224
alpha (float): Regularization strength
225
fit_intercept (bool): Whether to fit intercept
226
copy_X (bool): Whether to copy X
227
max_iter (int): Maximum iterations
228
tol (float): Tolerance for convergence
229
solver (str): Solver algorithm
230
positive (bool): Force positive coefficients
231
random_state (int): Random state
232
"""
233
234
def fit(self, X, y, sample_weight=None):
235
"""Fit Ridge regression model."""
236
237
def predict(self, X):
238
"""Predict using Ridge regression."""
239
240
def score(self, X, y, sample_weight=None):
241
"""Return R² score."""
242
243
# Attributes
244
coef_: ...
245
intercept_: ...
246
```
247
248
### Lasso Regression
249
250
L1-regularized linear regression with Intel-optimized coordinate descent.
251
252
```python { .api }
253
class Lasso:
254
"""
255
Lasso regression with Intel optimization.
256
257
Linear regression with L1 regularization using optimized
258
coordinate descent algorithm.
259
"""
260
261
def __init__(
262
self,
263
alpha=1.0,
264
fit_intercept=True,
265
precompute=False,
266
copy_X=True,
267
max_iter=1000,
268
tol=1e-4,
269
warm_start=False,
270
positive=False,
271
random_state=None,
272
selection='cyclic'
273
):
274
"""
275
Initialize Lasso regression.
276
277
Parameters:
278
alpha (float): Regularization strength
279
fit_intercept (bool): Whether to fit intercept
280
precompute (bool): Whether to use precomputed Gram matrix
281
copy_X (bool): Whether to copy X
282
max_iter (int): Maximum iterations
283
tol (float): Tolerance for convergence
284
warm_start (bool): Reuse previous solution
285
positive (bool): Force positive coefficients
286
random_state (int): Random state
287
selection (str): Feature selection strategy
288
"""
289
290
def fit(self, X, y, sample_weight=None, check_input=True):
291
"""Fit Lasso regression model."""
292
293
def predict(self, X):
294
"""Predict using Lasso regression."""
295
296
# Attributes
297
coef_: ...
298
intercept_: ...
299
n_iter_: ...
300
```
301
302
### Elastic Net Regression
303
304
Combined L1 and L2 regularization with Intel optimization.
305
306
```python { .api }
307
class ElasticNet:
308
"""
309
Elastic Net regression with Intel optimization.
310
311
Linear regression with combined L1 and L2 regularization,
312
using optimized coordinate descent solver.
313
"""
314
315
def __init__(
316
self,
317
alpha=1.0,
318
l1_ratio=0.5,
319
fit_intercept=True,
320
precompute=False,
321
max_iter=1000,
322
copy_X=True,
323
tol=1e-4,
324
warm_start=False,
325
positive=False,
326
random_state=None,
327
selection='cyclic'
328
):
329
"""
330
Initialize Elastic Net regression.
331
332
Parameters:
333
alpha (float): Regularization strength
334
l1_ratio (float): Mix ratio of L1 and L2 penalties
335
fit_intercept (bool): Whether to fit intercept
336
precompute (bool): Whether to use precomputed Gram matrix
337
max_iter (int): Maximum iterations
338
copy_X (bool): Whether to copy X
339
tol (float): Tolerance for convergence
340
warm_start (bool): Reuse previous solution
341
positive (bool): Force positive coefficients
342
random_state (int): Random state
343
selection (str): Feature selection strategy
344
"""
345
346
def fit(self, X, y, sample_weight=None, check_input=True):
347
"""Fit Elastic Net regression model."""
348
349
def predict(self, X):
350
"""Predict using Elastic Net regression."""
351
352
# Attributes
353
coef_: ...
354
intercept_: ...
355
n_iter_: ...
356
```
357
358
### Incremental Linear Regression
359
360
Memory-efficient linear regression for streaming data with Intel optimization.
361
362
```python { .api }
363
class IncrementalLinearRegression:
364
"""
365
Incremental Linear Regression with Intel optimization.
366
367
Allows fitting linear regression incrementally on mini-batches
368
of data, useful for large datasets that don't fit in memory.
369
"""
370
371
def __init__(self, fit_intercept=True, copy_X=True):
372
"""
373
Initialize Incremental Linear Regression.
374
375
Parameters:
376
fit_intercept (bool): Whether to fit intercept
377
copy_X (bool): Whether to copy input data
378
"""
379
380
def partial_fit(self, X, y, sample_weight=None):
381
"""
382
Incrementally fit linear regression.
383
384
Parameters:
385
X (array-like): Training data batch
386
y (array-like): Target values batch
387
sample_weight (array-like): Sample weights
388
389
Returns:
390
self: Fitted estimator
391
"""
392
393
def fit(self, X, y, sample_weight=None):
394
"""
395
Fit linear regression (clears previous fits).
396
397
Parameters:
398
X (array-like): Training data
399
y (array-like): Target values
400
sample_weight (array-like): Sample weights
401
402
Returns:
403
self: Fitted estimator
404
"""
405
406
def predict(self, X):
407
"""Predict using the linear model."""
408
409
def score(self, X, y, sample_weight=None):
410
"""Return R² score."""
411
412
# Attributes
413
coef_: ...
414
intercept_: ...
415
```
416
417
## Usage Examples
418
419
### Basic Linear Regression
420
421
```python
422
import numpy as np
423
from sklearnex.linear_model import LinearRegression
424
from sklearn.datasets import make_regression
425
from sklearn.model_selection import train_test_split
426
427
# Generate sample data
428
X, y = make_regression(n_samples=1000, n_features=10, noise=0.1, random_state=42)
429
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
430
431
# Create and fit model
432
lr = LinearRegression()
433
lr.fit(X_train, y_train)
434
435
# Make predictions
436
y_pred = lr.predict(X_test)
437
r2_score = lr.score(X_test, y_test)
438
439
print(f"R² Score: {r2_score:.3f}")
440
print(f"Coefficients shape: {lr.coef_.shape}")
441
print(f"Intercept: {lr.intercept_:.3f}")
442
```
443
444
### Logistic Regression Classification
445
446
```python
447
import numpy as np
448
from sklearnex.linear_model import LogisticRegression
449
from sklearn.datasets import make_classification
450
from sklearn.model_selection import train_test_split
451
452
# Generate classification data
453
X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
454
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
455
456
# Create and fit logistic regression
457
lr = LogisticRegression(random_state=42, max_iter=1000)
458
lr.fit(X_train, y_train)
459
460
# Predictions and probabilities
461
y_pred = lr.predict(X_test)
462
y_proba = lr.predict_proba(X_test)
463
accuracy = lr.score(X_test, y_test)
464
465
print(f"Accuracy: {accuracy:.3f}")
466
print(f"Classes: {lr.classes_}")
467
print(f"Coefficients shape: {lr.coef_.shape}")
468
print(f"Probabilities shape: {y_proba.shape}")
469
```
470
471
### Regularized Regression Comparison
472
473
```python
474
import numpy as np
475
from sklearnex.linear_model import Ridge, Lasso, ElasticNet
476
from sklearn.datasets import make_regression
477
from sklearn.model_selection import train_test_split
478
479
# Generate data with some noise
480
X, y = make_regression(n_samples=1000, n_features=50, noise=10, random_state=42)
481
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
482
483
# Compare different regularization methods
484
models = {
485
'Ridge': Ridge(alpha=1.0),
486
'Lasso': Lasso(alpha=1.0, max_iter=2000),
487
'ElasticNet': ElasticNet(alpha=1.0, l1_ratio=0.5, max_iter=2000)
488
}
489
490
for name, model in models.items():
491
model.fit(X_train, y_train)
492
score = model.score(X_test, y_test)
493
n_nonzero = np.sum(np.abs(model.coef_) > 1e-6)
494
495
print(f"{name:12s} - R²: {score:.3f}, Non-zero coefs: {n_nonzero}")
496
```
497
498
### Incremental Learning
499
500
```python
501
import numpy as np
502
from sklearnex.linear_model import IncrementalLinearRegression
503
from sklearn.datasets import make_regression
504
505
# Generate large dataset
506
X, y = make_regression(n_samples=10000, n_features=20, noise=0.1, random_state=42)
507
508
# Create incremental model
509
inc_lr = IncrementalLinearRegression()
510
511
# Fit in batches
512
batch_size = 1000
513
n_batches = len(X) // batch_size
514
515
for i in range(n_batches):
516
start_idx = i * batch_size
517
end_idx = (i + 1) * batch_size
518
519
X_batch = X[start_idx:end_idx]
520
y_batch = y[start_idx:end_idx]
521
522
inc_lr.partial_fit(X_batch, y_batch)
523
524
# Evaluate on current batch
525
score = inc_lr.score(X_batch, y_batch)
526
print(f"Batch {i+1}/{n_batches} - R²: {score:.3f}")
527
528
# Final evaluation on full dataset
529
final_score = inc_lr.score(X, y)
530
print(f"Final R² score: {final_score:.3f}")
531
```
532
533
## Performance Notes
534
535
- Linear regression shows significant speedups on datasets with >1000 samples
536
- Logistic regression benefits most from Intel optimization with large feature spaces
537
- Regularized models (Ridge, Lasso, ElasticNet) have optimized coordinate descent
538
- Incremental learning maintains performance while reducing memory usage
539
- All models maintain numerical stability equivalent to scikit-learn