0
# Classification Algorithms
1

2
Advanced classification methods including ensemble voting, stacking, neural networks, and classic algorithms. All classifiers follow scikit-learn's fit/predict API and are compatible with scikit-learn pipelines and model selection tools.
3

4
## Capabilities
5

6
### Ensemble Vote Classifier
7

8
Combines multiple classifiers using majority voting (hard) or weighted average of predicted probabilities (soft voting).
9

10
```python { .api }
11
class EnsembleVoteClassifier:
12
    def __init__(self, clfs, voting='hard', weights=None, verbose=0, 
13
                 use_clones=True, fit_base_estimators=True):
14
        """
15
        Ensemble classifier combining multiple base classifiers.
16
        
17
        Parameters:
18
        - clfs: list of sklearn-compatible classifiers
19
        - voting: str, 'hard' or 'soft' voting
20
        - weights: list, voting weights for classifiers
21
        - verbose: int, verbosity level
22
        - use_clones: bool, whether to clone base classifiers
23
        - fit_base_estimators: bool, whether to fit base estimators
24
        """
25
    
26
    def fit(self, X, y):
27
        """Fit ensemble classifier"""
28
        
29
    def predict(self, X):
30
        """Make predictions using ensemble"""
31
        
32
    def predict_proba(self, X):
33
        """Predict class probabilities (soft voting only)"""
34
        
35
    def transform(self, X):
36
        """Transform features using fitted classifiers"""
37
        
38
    def get_params(self, deep=True):
39
        """Get parameters for this estimator"""
40
        
41
    def set_params(self, **params):
42
        """Set parameters for this estimator"""
43
```
44

45
### Stacking Classifier
46

47
Meta-learning ensemble that trains a meta-classifier on the predictions of base classifiers.
48

49
```python { .api }
50
class StackingClassifier:
51
    def __init__(self, classifiers, meta_classifier, use_probas=False, 
52
                 average_probas=False, verbose=0, use_features_in_secondary=False):
53
        """
54
        Stacking classifier for meta-learning.
55
        
56
        Parameters:
57
        - classifiers: list of base classifiers
58
        - meta_classifier: classifier for meta-learning
59
        - use_probas: bool, use predicted probabilities as meta-features
60
        - average_probas: bool, average probabilities if use_probas=True
61
        - verbose: int, verbosity level
62
        - use_features_in_secondary: bool, include original features in meta-learning
63
        """
64
    
65
    def fit(self, X, y):
66
        """Fit stacking classifier"""
67
        
68
    def predict(self, X):
69
        """Make predictions using meta-classifier"""
70
        
71
    def predict_meta_features(self, X):
72
        """Generate meta-features from base classifiers"""
73
        
74
    def get_params(self, deep=True):
75
        """Get parameters for this estimator"""
76
        
77
    def set_params(self, **params):
78
        """Set parameters for this estimator"""
79
```
80

81
### Stacking CV Classifier
82

83
Cross-validation stacking classifier that uses cross-validation to generate meta-features and avoid overfitting.
84

85
```python { .api }
86
class StackingCVClassifier:
87
    def __init__(self, classifiers, meta_classifier, cv=2, shuffle=True, 
88
                 stratify=True, random_state=0, verbose=0, use_probas=False,
89
                 use_features_in_secondary=False, store_train_meta_features=False,
90
                 use_clones=True, n_jobs=1):
91
        """
92
        Cross-validation stacking classifier.
93
        
94
        Parameters:
95
        - classifiers: list of base classifiers
96
        - meta_classifier: classifier for meta-learning
97
        - cv: int, number of cross-validation folds
98
        - shuffle: bool, shuffle data before splitting
99
        - stratify: bool, stratified cross-validation
100
        - random_state: int, random state for reproducibility
101
        - verbose: int, verbosity level
102
        - use_probas: bool, use predicted probabilities as meta-features
103
        - use_features_in_secondary: bool, include original features
104
        - store_train_meta_features: bool, store training meta-features
105
        - use_clones: bool, clone base classifiers
106
        - n_jobs: int, number of parallel jobs
107
        """
108
    
109
    def fit(self, X, y, groups=None):
110
        """Fit CV stacking classifier"""
111
        
112
    def predict(self, X):
113
        """Make predictions using meta-classifier"""
114
        
115
    def predict_meta_features(self, X):
116
        """Generate meta-features using cross-validation"""
117
        
118
    def predict_proba(self, X):
119
        """Predict class probabilities if meta-classifier supports it"""
120
```
121

122
### Multi-Layer Perceptron
123

124
Neural network with configurable hidden layers for classification tasks.
125

126
```python { .api }
127
class MultiLayerPerceptron:
128
    def __init__(self, eta=0.5, epochs=50, hidden_layers=[50], n_classes=None,
129
                 momentum=0.0, l1=0.0, l2=0.0, dropout=1.0, minibatches=1,
130
                 random_seed=None, print_progress=0):
131
        """
132
        Multi-layer perceptron classifier.
133
        
134
        Parameters:
135
        - eta: float, learning rate
136
        - epochs: int, number of training epochs
137
        - hidden_layers: list, number of neurons in each hidden layer
138
        - n_classes: int, number of output classes
139
        - momentum: float, momentum parameter
140
        - l1: float, L1 regularization parameter
141
        - l2: float, L2 regularization parameter
142
        - dropout: float, dropout rate
143
        - minibatches: int, number of minibatches
144
        - random_seed: int, random seed
145
        - print_progress: int, print progress every n epochs
146
        """
147
    
148
    def fit(self, X, y):
149
        """Train the multi-layer perceptron"""
150
        
151
    def predict(self, X):
152
        """Make predictions"""
153
        
154
    def predict_proba(self, X):
155
        """Predict class probabilities"""
156
```
157

158
### Logistic Regression
159

160
Binary logistic regression with L2 regularization and gradient descent optimization.
161

162
```python { .api }
163
class LogisticRegression:
164
    def __init__(self, eta=0.01, epochs=50, l2_lambda=0.0, minibatches=1,
165
                 random_seed=None, print_progress=0):
166
        """
167
        Logistic regression for binary classification.
168
        
169
        Parameters:
170
        - eta: float, learning rate
171
        - epochs: int, number of training epochs
172
        - l2_lambda: float, L2 regularization parameter
173
        - minibatches: int, number of minibatches
174
        - random_seed: int, random seed
175
        - print_progress: int, print progress every n epochs
176
        """
177
    
178
    def fit(self, X, y):
179
        """Fit logistic regression model"""
180
        
181
    def predict(self, X):
182
        """Make binary predictions"""
183
        
184
    def predict_proba(self, X):
185
        """Predict class probabilities"""
186
```
187

188
### Softmax Regression
189

190
Multinomial logistic regression for multi-class classification.
191

192
```python { .api }
193
class SoftmaxRegression:
194
    def __init__(self, eta=0.01, epochs=50, l2=0.0, minibatches=1,
195
                 n_classes=None, random_seed=None, print_progress=0):
196
        """
197
        Softmax regression for multi-class classification.
198
        
199
        Parameters:
200
        - eta: float, learning rate
201
        - epochs: int, number of training epochs
202
        - l2: float, L2 regularization parameter
203
        - minibatches: int, number of minibatches
204
        - n_classes: int, number of classes
205
        - random_seed: int, random seed
206
        - print_progress: int, print progress every n epochs
207
        """
208
    
209
    def fit(self, X, y):
210
        """Fit softmax regression model"""
211
        
212
    def predict(self, X):
213
        """Make multi-class predictions"""
214
        
215
    def predict_proba(self, X):
216
        """Predict class probabilities"""
217
```
218

219
### Perceptron
220

221
Classic perceptron algorithm for binary linear classification.
222

223
```python { .api }
224
class Perceptron:
225
    def __init__(self, eta=0.01, epochs=50, random_seed=None, print_progress=0):
226
        """
227
        Perceptron classifier.
228
        
229
        Parameters:
230
        - eta: float, learning rate
231
        - epochs: int, number of training epochs
232
        - random_seed: int, random seed
233
        - print_progress: int, print progress every n epochs
234
        """
235
    
236
    def fit(self, X, y):
237
        """Fit perceptron model"""
238
        
239
    def predict(self, X):
240
        """Make binary predictions"""
241
```
242

243
### Adaline
244

245
Adaptive Linear Neuron with gradient descent learning.
246

247
```python { .api }
248
class Adaline:
249
    def __init__(self, eta=0.01, epochs=50, minibatches=None, random_seed=None,
250
                 print_progress=0):
251
        """
252
        Adaline (ADAptive LInear NEuron) classifier.
253
        
254
        Parameters:
255
        - eta: float, learning rate
256
        - epochs: int, number of training epochs
257
        - minibatches: int, number of minibatches for SGD
258
        - random_seed: int, random seed
259
        - print_progress: int, print progress every n epochs
260
        """
261
    
262
    def fit(self, X, y):
263
        """Fit Adaline model"""
264
        
265
    def predict(self, X):
266
        """Make binary predictions"""
267
```
268

269
### OneR Classifier
270

271
Simple rule-based classifier that creates one rule for each predictor and selects the rule with the smallest total error.
272

273
```python { .api }
274
class OneRClassifier:
275
    def __init__(self, resolve_ties='first'):
276
        """
277
        OneR (One Rule) classifier.
278
        
279
        Parameters:
280
        - resolve_ties: str, method to resolve ties ('first' or 'chi-squared')
281
        """
282
    
283
    def fit(self, X, y):
284
        """Fit OneR classifier"""
285
        
286
    def predict(self, X):
287
        """Make predictions using the single best rule"""
288
```
289

290
## Usage Examples
291

292
### Ensemble Voting Example
293

294
```python
295
from mlxtend.classifier import EnsembleVoteClassifier
296
from sklearn.ensemble import RandomForestClassifier
297
from sklearn.svm import SVC
298
from sklearn.linear_model import LogisticRegression
299
from sklearn.datasets import make_classification
300

301
# Create dataset
302
X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
303

304
# Create base classifiers
305
clf1 = RandomForestClassifier(random_state=42)
306
clf2 = SVC(probability=True, random_state=42)
307
clf3 = LogisticRegression(random_state=42)
308

309
# Create ensemble with soft voting
310
ensemble = EnsembleVoteClassifier(clfs=[clf1, clf2, clf3], voting='soft')
311
ensemble.fit(X, y)
312

313
# Make predictions
314
predictions = ensemble.predict(X)
315
probabilities = ensemble.predict_proba(X)
316
```
317

318
### Stacking Example
319

320
```python
321
from mlxtend.classifier import StackingCVClassifier
322
from sklearn.ensemble import RandomForestClassifier
323
from sklearn.svm import SVC
324
from sklearn.linear_model import LogisticRegression
325
from sklearn.datasets import make_classification
326

327
# Create dataset
328
X, y = make_classification(n_samples=1000, n_features=20, n_classes=2, random_state=42)
329

330
# Create base and meta classifiers
331
clf1 = RandomForestClassifier(random_state=42)
332
clf2 = SVC(probability=True, random_state=42)
333
meta_clf = LogisticRegression(random_state=42)
334

335
# Create stacking classifier with cross-validation
336
stacking = StackingCVClassifier(classifiers=[clf1, clf2], 
337
                               meta_classifier=meta_clf,
338
                               use_probas=True, cv=5)
339
stacking.fit(X, y)
340

341
# Make predictions
342
predictions = stacking.predict(X)
343
```