0
# Lightning
1
2
A unified deep learning framework that integrates PyTorch Lightning, Lightning Fabric, and Lightning Apps to provide a complete solution for training, deploying, and shipping AI products. Lightning abstracts away boilerplate code while maintaining flexibility, enabling everything from simple research experiments to complex multi-cloud production systems.
3
4
## Package Information
5
6
- **Package Name**: pytorch-lightning (also available as unified `lightning`)
7
- **Language**: Python
8
- **Installation**: `pip install pytorch-lightning` or `pip install lightning` (full suite)
9
10
## Core Imports
11
12
**Option 1: Unified Lightning (requires full lightning package):**
13
```python
14
import lightning as L
15
```
16
17
**Option 2: PyTorch Lightning specific:**
18
```python
19
import pytorch_lightning as pl
20
```
21
22
**Option 3: Individual component imports:**
23
```python
24
# Training components (from pytorch-lightning)
25
from pytorch_lightning import Trainer, LightningModule, LightningDataModule, Callback
26
27
# Fabric (from lightning-fabric)
28
from lightning_fabric import Fabric, seed_everything
29
30
# Apps (from lightning-app)
31
from lightning_app import LightningApp, LightningFlow, LightningWork
32
from lightning_app import CloudCompute, BuildConfig
33
```
34
35
**Option 4: Sub-package imports:**
36
```python
37
# If lightning is installed
38
from lightning import Trainer, LightningModule, LightningDataModule, Callback
39
from lightning import Fabric, seed_everything
40
from lightning import LightningApp, LightningFlow, LightningWork
41
from lightning import CloudCompute, BuildConfig
42
```
43
44
## Basic Usage
45
46
```python
47
import lightning as L
48
import torch
49
import torch.nn as nn
50
import torch.nn.functional as F
51
from torch.utils.data import DataLoader, TensorDataset
52
53
# Define a simple model
54
class SimpleModel(L.LightningModule):
55
def __init__(self):
56
super().__init__()
57
self.layer = nn.Linear(10, 1)
58
59
def forward(self, x):
60
return self.layer(x)
61
62
def training_step(self, batch, batch_idx):
63
x, y = batch
64
y_hat = self(x)
65
loss = F.mse_loss(y_hat, y)
66
return loss
67
68
def configure_optimizers(self):
69
return torch.optim.Adam(self.parameters())
70
71
# Create sample data
72
X = torch.randn(1000, 10)
73
y = torch.randn(1000, 1)
74
dataset = TensorDataset(X, y)
75
dataloader = DataLoader(dataset, batch_size=32)
76
77
# Train the model
78
model = SimpleModel()
79
trainer = L.Trainer(max_epochs=5)
80
trainer.fit(model, dataloader)
81
```
82
83
## Architecture
84
85
Lightning is built on three core pillars that work together:
86
87
- **PyTorch Lightning**: Organizes PyTorch training code with automatic multi-device scaling, standardized project structure, and extensive callback system
88
- **Lightning Fabric**: Provides low-level control over training loops while handling device management and distributed training setup
89
- **Lightning Apps**: Enables building end-to-end ML systems with components for data processing, model serving, and cloud deployment
90
91
The main `lightning` package serves as an umbrella that exposes key functionality from all three components, providing a unified interface for the complete ML lifecycle from research to production.
92
93
## Capabilities
94
95
### Training and Model Organization
96
97
Core PyTorch Lightning components for organizing training code, managing experiments, and scaling across devices. Includes the main Trainer class, base classes for models and data modules, and the callback system.
98
99
```python { .api }
100
class Trainer:
101
def __init__(self, max_epochs: int = None, logger: bool = True, **kwargs): ...
102
def fit(self, model: LightningModule, train_dataloaders=None, **kwargs): ...
103
def validate(self, model: LightningModule = None, dataloaders=None, **kwargs): ...
104
def test(self, model: LightningModule = None, dataloaders=None, **kwargs): ...
105
def predict(self, model: LightningModule = None, dataloaders=None, **kwargs): ...
106
107
class LightningModule:
108
def training_step(self, batch, batch_idx): ...
109
def validation_step(self, batch, batch_idx): ...
110
def test_step(self, batch, batch_idx): ...
111
def configure_optimizers(self): ...
112
113
class LightningDataModule:
114
def setup(self, stage: str = None): ...
115
def train_dataloader(self): ...
116
def val_dataloader(self): ...
117
def test_dataloader(self): ...
118
119
class Callback:
120
def on_train_start(self, trainer, pl_module): ...
121
def on_train_end(self, trainer, pl_module): ...
122
def on_epoch_start(self, trainer, pl_module): ...
123
def on_epoch_end(self, trainer, pl_module): ...
124
```
125
126
[Training and Model Organization](./training.md)
127
128
### Low-Level Training Control
129
130
Lightning Fabric provides fine-grained control over training loops while handling device management, distributed training setup, and gradient synchronization automatically.
131
132
```python { .api }
133
class Fabric:
134
def __init__(self, accelerator: str = "auto", devices: int = "auto", **kwargs): ...
135
def setup(self, model, *optimizers): ...
136
def backward(self, loss): ...
137
def step(self, optimizer): ...
138
def load(self, path: str): ...
139
def save(self, path: str, state: dict): ...
140
141
def seed_everything(seed: int, workers: bool = False) -> int: ...
142
```
143
144
[Low-Level Training Control](./fabric.md)
145
146
### Application Development
147
148
Lightning Apps framework for building end-to-end ML systems with components for workflow orchestration, computational work distribution, and cloud deployment.
149
150
```python { .api }
151
class LightningApp:
152
def __init__(self, root: LightningFlow, info: dict = None): ...
153
154
class LightningFlow:
155
def run(self): ...
156
def configure_layout(self): ...
157
158
class LightningWork:
159
def run(self, *args, **kwargs): ...
160
161
class CloudCompute:
162
def __init__(self, name: str, disk_size: int = 0, idle_timeout: int = None): ...
163
164
class BuildConfig:
165
def __init__(self, image: str = None, requirements: list = None, dockerfile: str = None): ...
166
```
167
168
[Application Development](./apps.md)
169
170
### Storage and Utilities
171
172
Storage abstractions and utility modules for Lightning Apps, including file system operations, cloud storage integrations, and debugging tools.
173
174
```python { .api }
175
# Storage module functions and classes
176
# Debugging utilities
177
```
178
179
[Storage and Utilities](./utilities.md)
180
181
## Types
182
183
```python { .api }
184
# Common type aliases used across the framework
185
from typing import Union, Optional, List, Dict, Any, Callable, Iterable, Sequence, Tuple
186
from pathlib import Path
187
from datetime import timedelta
188
from torch import Tensor
189
from torch.nn import Module
190
from torch.optim import Optimizer
191
from torch.utils.data import DataLoader
192
from lightning_fabric.utilities.types import _PATH
193
from pytorch_lightning.loggers import Logger
194
from pytorch_lightning.callbacks import Callback
195
from pytorch_lightning.plugins import PrecisionPlugin
196
from pytorch_lightning.profilers import Profiler
197
198
# Lightning-specific type aliases
199
_PATH = Union[str, Path]
200
```