0
# Celery
1

2
Celery is a distributed task queue for Python that enables asynchronous execution of jobs across multiple workers and machines. It uses message brokers like RabbitMQ or Redis to coordinate between clients and workers, providing high availability, horizontal scaling, and robust task execution with retries, routing, and monitoring capabilities.
3

4
## Package Information
5

6
- **Package Name**: celery
7
- **Language**: Python
8
- **Installation**: `pip install celery`
9
- **Bundles**: `pip install "celery[redis]"` (with Redis support)
10

11
## Core Imports
12

13
```python
14
from celery import Celery
15
```
16

17
Common task creation imports:
18

19
```python
20
from celery import shared_task
21
```
22

23
Application and task state imports:
24

25
```python  
26
from celery import current_app, current_task
27
```
28

29
Canvas workflow imports:
30

31
```python
32
from celery import signature, chain, group, chord, chunks, xmap
33
```
34

35
## Basic Usage
36

37
```python
38
from celery import Celery
39

40
# Create Celery app
41
app = Celery('myapp', broker='redis://localhost:6379/0')
42

43
# Define a task
44
@app.task
45
def add(x, y):
46
    return x + y
47

48
# Execute task asynchronously
49
result = add.delay(4, 4)
50
print(result.get())  # Wait for result
51

52
# Use shared task for library code
53
from celery import shared_task
54

55
@shared_task
56
def multiply(x, y):
57
    return x * y
58

59
# Chain tasks together
60
from celery import chain
61
result = chain(add.s(2, 2), multiply.s(8))()
62
print(result.get())  # Result: 32 (4 * 8)
63
```
64

65
## Architecture
66

67
Celery's architecture consists of several key components working together:
68

69
- **Application (Celery)**: Central application instance managing configuration, task registry, and coordination
70
- **Tasks**: Units of work executed asynchronously by workers, created via decorators or classes
71
- **Workers**: Processes that consume tasks from queues and execute them, supporting multiple concurrency models
72
- **Brokers**: Message transport systems (Redis, RabbitMQ, etc.) that queue and route tasks between clients and workers
73
- **Result Backends**: Storage systems for task results and metadata, enabling result retrieval and monitoring
74
- **Beat Scheduler**: Periodic task scheduler for cron-like functionality and recurring jobs
75
- **Canvas**: Workflow primitives (signature, chain, group, chord) for complex task composition and orchestration
76

77
This distributed architecture enables Celery to scale horizontally, handle failures gracefully, and provide flexible task routing and execution patterns.
78

79
## Capabilities
80

81
### Core Application
82

83
Main Celery application class, task creation with decorators, shared tasks for reusable components, and application lifecycle management.
84

85
```python { .api }
86
class Celery:
87
    def __init__(self, main=None, broker=None, backend=None, **kwargs): ...
88
    def task(self, *args, **opts): ...
89
    def send_task(self, name, args=None, kwargs=None, **options): ...
90

91
def shared_task(*args, **kwargs): ...
92
```
93

94
[Core Application](./core-application.md)
95

96
### Workflow Primitives (Canvas)
97

98
Task composition primitives for building complex workflows including sequential chains, parallel groups, callbacks, and functional programming patterns.
99

100
```python { .api }
101
def signature(task, args=None, kwargs=None, **options): ...
102
def chain(*tasks, **kwargs): ...
103
def group(*tasks, **kwargs): ...
104
def chord(header, body, **kwargs): ...
105
def chunks(it, n, task): ...
106
```
107

108
[Workflow Primitives](./workflow-primitives.md)
109

110
### Results and State Management
111

112
Task result handling, state monitoring, result retrieval with timeouts, and task lifecycle management.
113

114
```python { .api }
115
class AsyncResult:
116
    def get(self, timeout=None, propagate=True, interval=0.5): ...
117
    def ready(self) -> bool: ...
118
    def successful(self) -> bool: ...
119
    def revoke(self, terminate=False): ...
120

121
class GroupResult:
122
    def get(self, timeout=None, propagate=True): ...
123
```
124

125
[Results and State](./results-state.md)
126

127
### Scheduling and Beat
128

129
Periodic task scheduling with cron-like syntax, interval-based schedules, solar event scheduling, and beat scheduler management.
130

131
```python { .api }
132
def crontab(minute='*', hour='*', day_of_week='*', day_of_month='*', month_of_year='*'): ...
133
def schedule(run_every, relative=False): ...
134
def solar(event, lat, lon): ...
135
```
136

137
[Scheduling and Beat](./scheduling-beat.md)
138

139
### Signals and Events
140

141
Task lifecycle signals, worker events, monitoring hooks, and custom signal handlers for debugging and monitoring integration.
142

143
```python { .api }
144
# Task signals
145
task_prerun = Signal()
146
task_postrun = Signal()
147
task_success = Signal()
148
task_failure = Signal()
149

150
# Worker signals  
151
worker_ready = Signal()
152
worker_shutdown = Signal()
153
```
154

155
[Signals and Events](./signals-events.md)
156

157
### Configuration
158

159
Application configuration management, broker and backend setup, task routing, serialization options, and environment-based configuration.
160

161
```python { .api }
162
class Celery:
163
    def config_from_object(self, obj, silent=False, force=False): ...
164
    def config_from_envvar(self, variable_name, silent=False): ...
165
    
166
    @property
167
    def conf(self): ...  # Configuration namespace
168
```
169

170
[Configuration](./configuration.md)
171

172
### Exception Handling
173

174
Complete exception hierarchy for task errors, retry mechanisms, timeout handling, backend errors, and worker-related exceptions.
175

176
```python { .api }
177
class CeleryError(Exception): ...
178
class Retry(CeleryError): ...
179
class MaxRetriesExceededError(CeleryError): ...
180
class TaskRevokedError(CeleryError): ...
181
class SoftTimeLimitExceeded(Exception): ...
182
```
183

184
[Exception Handling](./exceptions.md)