0
# Signal Handling
1
2
Signal handling utilities for interactive job cancellation and REPL interrupt management.
3
4
## Capabilities
5
6
### Signaling Object
7
8
Provides signal handling functionality for graceful job cancellation in the REPL environment.
9
10
```scala { .api }
11
/**
12
* Signal handling utilities for REPL interrupt management
13
* Provides SIGINT handling to cancel running Spark jobs
14
*/
15
private[repl] object Signaling extends Logging {
16
/**
17
* Register a SIGINT handler that terminates all active Spark jobs
18
* or terminates when no jobs are currently running
19
* Makes it possible to interrupt a running shell job by pressing Ctrl+C
20
*/
21
def cancelOnInterrupt(): Unit
22
}
23
```
24
25
**Usage Examples:**
26
27
```scala
28
import org.apache.spark.repl.Signaling
29
30
// Register interrupt handler (typically called during REPL startup)
31
Signaling.cancelOnInterrupt()
32
33
// Now Ctrl+C will:
34
// 1. Cancel active Spark jobs if any are running
35
// 2. Exit REPL if no jobs are active
36
```
37
38
## Signal Handling Behavior
39
40
### SIGINT (Ctrl+C) Handling
41
42
The interrupt handler provides intelligent behavior based on Spark job status:
43
44
**When Spark Jobs Are Active:**
45
1. **First Ctrl+C**: Cancels all active Spark jobs
46
2. Shows warning message: "Cancelling all active jobs, this can take a while. Press Ctrl+C again to exit now."
47
3. **Second Ctrl+C**: Exits immediately
48
49
**When No Jobs Are Active:**
50
1. **First Ctrl+C**: Exits REPL normally
51
52
### Implementation Details
53
54
The signal handler integrates with Spark's job tracking system:
55
56
```scala
57
def cancelOnInterrupt(): Unit = SignalUtils.register("INT") {
58
SparkContext.getActive.map { ctx =>
59
if (!ctx.statusTracker.getActiveJobIds().isEmpty) {
60
logWarning("Cancelling all active jobs, this can take a while. " +
61
"Press Ctrl+C again to exit now.")
62
ctx.cancelAllJobs()
63
true // Handled - don't exit yet
64
} else {
65
false // Not handled - allow normal exit
66
}
67
}.getOrElse(false) // No active context - allow normal exit
68
}
69
```
70
71
## Integration with Spark
72
73
### SparkContext Integration
74
75
- Uses `SparkContext.getActive` to find current Spark context
76
- Leverages `StatusTracker.getActiveJobIds()` to check for running jobs
77
- Calls `SparkContext.cancelAllJobs()` for graceful job termination
78
79
### Job Cancellation Process
80
81
When jobs are cancelled via Ctrl+C:
82
83
1. **Signal Detection**: SIGINT signal captured by handler
84
2. **Job Status Check**: Verify active jobs exist
85
3. **Cancellation Request**: Send cancellation to all active jobs
86
4. **User Notification**: Display cancellation progress message
87
5. **Graceful Shutdown**: Allow jobs to complete cancellation before exit
88
89
### Thread Safety
90
91
The signal handler operates safely with concurrent job execution:
92
93
- Uses atomic job status checking
94
- Handles race conditions between job completion and cancellation
95
- Provides consistent behavior across different job types
96
97
## Error Handling
98
99
### Missing SparkContext
100
101
When no SparkContext is available:
102
103
```scala
104
SparkContext.getActive.map { ctx =>
105
// Handle interrupts with context
106
}.getOrElse(false) // No context - allow normal exit
107
```
108
109
### Exception Handling
110
111
Signal handling includes proper exception management:
112
113
- Logging of cancellation operations
114
- Graceful handling of job cancellation failures
115
- Safe fallback to normal exit behavior
116
117
## REPL Integration
118
119
### Startup Registration
120
121
Signal handling is automatically registered during REPL startup:
122
123
```scala
124
// In Main.scala
125
object Main extends Logging {
126
initializeLogIfNecessary(true)
127
Signaling.cancelOnInterrupt() // Register on startup
128
// ... rest of initialization
129
}
130
```
131
132
### Interactive Experience
133
134
Provides smooth interactive experience:
135
136
- **Long-running operations**: Can be interrupted safely
137
- **Data exploration**: Jobs can be cancelled without losing REPL session
138
- **Script execution**: Runaway computations can be stopped
139
- **Development workflow**: Quick iteration with safe job termination
140
141
### User Feedback
142
143
Clear user communication during interruption:
144
145
```
146
Cancelling all active jobs, this can take a while. Press Ctrl+C again to exit now.
147
```
148
149
## Platform Integration
150
151
### Unix Signal Support
152
153
Uses Spark's `SignalUtils.register()` for cross-platform signal handling:
154
155
- **Linux/macOS**: Native SIGINT handling
156
- **Windows**: Equivalent interrupt handling
157
- **Containers**: Works in Docker/Kubernetes environments
158
159
### JVM Integration
160
161
Integrates properly with JVM signal handling:
162
163
- Registers at JVM level for reliable delivery
164
- Handles JVM shutdown hooks appropriately
165
- Works with Scala/Java threading models
166
167
## Advanced Usage
168
169
### Custom Signal Handling
170
171
While the object is `private[repl]`, it demonstrates patterns for custom signal handling:
172
173
```scala
174
// Example pattern for custom signal handling
175
SignalUtils.register("INT") { /* custom handler */ }
176
SignalUtils.register("TERM") { /* termination handler */ }
177
```
178
179
### Job Management Integration
180
181
Can be extended for more sophisticated job management:
182
183
- **Job Prioritization**: Cancel lower-priority jobs first
184
- **Selective Cancellation**: Cancel specific job types
185
- **Resource Management**: Clean up resources during cancellation
186
- **State Preservation**: Save computation state before cancellation
187
188
## Best Practices
189
190
### REPL Usage
191
192
1. **Long Operations**: Always allow for interruption in long-running operations
193
2. **Resource Cleanup**: Ensure proper cleanup during job cancellation
194
3. **User Communication**: Provide clear feedback during cancellation
195
4. **Recovery**: Design computations to be resumable after interruption
196
197
### Development Patterns
198
199
1. **Checkpointing**: Save intermediate results for large computations
200
2. **Monitoring**: Track job progress for better interruption timing
201
3. **Graceful Degradation**: Handle partial results from cancelled jobs
202
4. **Testing**: Test interrupt behavior in development workflows
203
204
## Logging Integration
205
206
### Warning Messages
207
208
Cancellation events are logged with appropriate levels:
209
210
```scala
211
logWarning("Cancelling all active jobs, this can take a while. Press Ctrl+C again to exit now.")
212
```
213
214
### Debug Information
215
216
Debug-level logging for signal handling events:
217
218
- Signal registration success/failure
219
- Job cancellation initiation
220
- Context availability status
221
- Handler execution timing