0
# Security and Authentication
1
2
The security components provide SASL-based authentication for secure communication between shuffle clients and external shuffle services. This ensures that only authorized clients can access shuffle data.
3
4
## Secret Management
5
6
### ShuffleSecretManager
7
8
```java { .api }
9
public class ShuffleSecretManager implements SecretKeyHolder {
10
public ShuffleSecretManager();
11
12
public void registerApp(String appId, String shuffleSecret);
13
public void unregisterApp(String appId);
14
public String getSecretKey(String appId);
15
public String getSaslUser(String appId);
16
}
17
```
18
19
Manages SASL secrets used by the external shuffle service for authenticating client connections.
20
21
**Key Methods:**
22
23
#### registerApp
24
25
Registers a Spark application with its associated secret key for SASL authentication.
26
27
**Parameters:**
28
- `appId` (String): Spark application identifier
29
- `shuffleSecret` (String): Secret key for SASL authentication
30
31
#### unregisterApp
32
33
Removes an application's secret when the application completes or is no longer active.
34
35
**Parameters:**
36
- `appId` (String): Application identifier to unregister
37
38
#### getSecretKey
39
40
Retrieves the secret key for a given application. Used during SASL authentication.
41
42
**Parameters:**
43
- `appId` (String): Application identifier
44
45
**Returns:**
46
- `String`: Secret key for the application, or null if not registered
47
48
#### getSaslUser
49
50
Gets the SASL username for an application. Returns the standard Spark SASL user.
51
52
**Parameters:**
53
- `appId` (String): Application identifier
54
55
**Returns:**
56
- `String`: SASL username (typically "sparkSaslUser")
57
58
## SASL Configuration
59
60
### Client-Side SASL Setup
61
62
```java
63
import org.apache.spark.network.sasl.SecretKeyHolder;
64
import org.apache.spark.network.shuffle.ExternalShuffleClient;
65
66
// Create secret key holder
67
SecretKeyHolder secretHolder = new SecretKeyHolder() {
68
@Override
69
public String getSaslUser(String appId) {
70
return "sparkSaslUser";
71
}
72
73
@Override
74
public String getSecretKey(String appId) {
75
return "my-app-secret-key";
76
}
77
};
78
79
// Create client with SASL enabled
80
ExternalShuffleClient client = new ExternalShuffleClient(
81
conf,
82
secretHolder,
83
true, // Enable SASL
84
true // Enable SASL encryption
85
);
86
```
87
88
### Server-Side SASL Setup
89
90
The shuffle service uses `ShuffleSecretManager` to validate client authentication:
91
92
```java
93
// Create secret manager
94
ShuffleSecretManager secretManager = new ShuffleSecretManager();
95
96
// Register application secrets (typically done by Spark driver)
97
secretManager.registerApp("app-123", "shared-secret-key");
98
99
// Use with SASL-enabled transport context
100
// (Integration with TransportContext and SaslServerBootstrap)
101
```
102
103
## Authentication Flow
104
105
**SASL Authentication Process:**
106
1. Client initiates connection to shuffle service
107
2. SASL handshake begins using configured mechanism (typically DIGEST-MD5)
108
3. Client provides application ID and proves knowledge of secret key
109
4. Server validates secret using `ShuffleSecretManager`
110
5. On success, encrypted channel is established (if encryption enabled)
111
6. Shuffle requests proceed over authenticated/encrypted connection
112
113
## Security Features
114
115
### Authentication
116
117
- **SASL-based authentication**: Uses industry-standard SASL mechanisms
118
- **Per-application secrets**: Each Spark application has its own secret key
119
- **Mutual authentication**: Both client and server validate each other
120
121
### Encryption
122
123
- **SASL encryption**: Optional encryption of data in transit
124
- **Block data protection**: Shuffle block data is encrypted during transfer
125
- **Metadata protection**: Protocol messages are encrypted
126
127
### Access Control
128
129
- **Application isolation**: Applications can only access their own shuffle data
130
- **Executor validation**: Blocks can only be fetched by registered executors
131
- **Secret rotation**: Applications can update secrets during runtime
132
133
## Usage Examples
134
135
### Basic Authentication Setup
136
137
```java
138
import org.apache.spark.network.sasl.ShuffleSecretManager;
139
140
// Server-side: Create and configure secret manager
141
ShuffleSecretManager secretManager = new ShuffleSecretManager();
142
143
// Register applications (typically done by Spark driver)
144
secretManager.registerApp("spark-app-1", "secret-1");
145
secretManager.registerApp("spark-app-2", "secret-2");
146
147
// Later, validate authentication
148
String secret = secretManager.getSecretKey("spark-app-1");
149
if (secret != null) {
150
System.out.println("Application is registered");
151
} else {
152
System.out.println("Unknown application");
153
}
154
```
155
156
### Client Authentication
157
158
```java
159
// Client-side: Implement SecretKeyHolder
160
class AppSecretHolder implements SecretKeyHolder {
161
private final String appSecret;
162
163
public AppSecretHolder(String secret) {
164
this.appSecret = secret;
165
}
166
167
@Override
168
public String getSaslUser(String appId) {
169
return "sparkSaslUser"; // Standard Spark SASL user
170
}
171
172
@Override
173
public String getSecretKey(String appId) {
174
return appSecret;
175
}
176
}
177
178
// Create authenticated client
179
AppSecretHolder secretHolder = new AppSecretHolder("my-app-secret");
180
ExternalShuffleClient client = new ExternalShuffleClient(
181
conf,
182
secretHolder,
183
true, // SASL enabled
184
false // SASL encryption disabled for performance
185
);
186
187
client.init("spark-app-1");
188
```
189
190
### Full Security Setup
191
192
```java
193
// Maximum security configuration
194
ExternalShuffleClient secureClient = new ExternalShuffleClient(
195
conf,
196
secretHolder,
197
true, // Enable SASL authentication
198
true // Enable SASL encryption
199
);
200
201
// This client will:
202
// 1. Authenticate using SASL DIGEST-MD5
203
// 2. Encrypt all data in transit
204
// 3. Validate server identity
205
```
206
207
### Application Lifecycle Management
208
209
```java
210
// Application startup
211
secretManager.registerApp("new-app", generateSecret());
212
213
try {
214
// Application runs...
215
// Shuffle operations proceed with authentication
216
217
} finally {
218
// Application cleanup
219
secretManager.unregisterApp("new-app");
220
}
221
```
222
223
## Security Best Practices
224
225
### Secret Management
226
227
- **Generate strong secrets**: Use cryptographically secure random generators
228
- **Rotate secrets regularly**: Update secrets periodically for enhanced security
229
- **Secure secret distribution**: Use secure channels to distribute secrets to clients
230
- **Clean up secrets**: Remove secrets when applications complete
231
232
```java
233
import java.security.SecureRandom;
234
import java.util.Base64;
235
236
// Generate secure random secret
237
SecureRandom random = new SecureRandom();
238
byte[] secretBytes = new byte[32];
239
random.nextBytes(secretBytes);
240
String secret = Base64.getEncoder().encodeToString(secretBytes);
241
242
secretManager.registerApp("app-id", secret);
243
```
244
245
### Network Security
246
247
- **Use encryption in production**: Enable SASL encryption for sensitive data
248
- **Validate certificates**: In TLS deployments, validate server certificates
249
- **Monitor authentication failures**: Log and alert on authentication failures
250
251
### Access Control
252
253
- **Principle of least privilege**: Only grant access to necessary shuffle data
254
- **Application isolation**: Ensure applications cannot access each other's data
255
- **Audit access**: Log shuffle data access for security auditing
256
257
## Error Handling
258
259
Common security-related errors:
260
261
```java
262
// Authentication failure handling
263
try {
264
client.fetchBlocks(host, port, execId, blockIds, listener);
265
} catch (SecurityException e) {
266
System.err.println("Authentication failed: " + e.getMessage());
267
// Handle authentication failure:
268
// - Check secret configuration
269
// - Verify application registration
270
// - Retry with updated credentials
271
}
272
273
// SASL configuration errors
274
try {
275
client.init("app-id");
276
} catch (IllegalArgumentException e) {
277
if (e.getMessage().contains("SASL")) {
278
System.err.println("SASL configuration error: " + e.getMessage());
279
// Check SASL settings and secret holder configuration
280
}
281
}
282
```
283
284
## Troubleshooting
285
286
### Common Issues
287
288
1. **Authentication Failures**
289
- Verify secret registration on server
290
- Check client secret configuration
291
- Ensure application IDs match
292
293
2. **SASL Configuration Errors**
294
- Verify SASL mechanism support
295
- Check encryption compatibility
296
- Validate secret holder implementation
297
298
3. **Network Connectivity**
299
- Test without SASL first
300
- Check firewall rules for SASL ports
301
- Verify server SASL configuration
302
303
### Debug Configuration
304
305
```java
306
// Enable SASL debugging (add to JVM arguments)
307
// -Djava.security.debug=gssloginconfig,configfile,configparser,logincontext
308
309
// Enable Spark network debugging
310
// spark.network.crypto.enabled=true (for additional encryption)
311
// spark.authenticate=true (for Spark internal authentication)
312
```
313
314
## Integration with Spark Security
315
316
The shuffle service security integrates with Spark's overall security framework:
317
318
- **Shared secrets**: Uses same secret distribution mechanism as Spark
319
- **Authentication coordination**: Coordinates with Spark's SecurityManager
320
- **Encryption compatibility**: Compatible with Spark's network encryption
321
- **Access control**: Respects Spark's application isolation requirements