0
# TCP Transport Layer
1

2
The TCP transport layer provides high-performance, non-blocking I/O for node-to-node communication in Elasticsearch clusters. Built on Netty 4, it handles cluster coordination, data replication, search requests, and all internal messaging between cluster nodes.
3

4
## Capabilities
5

6
### Netty4Transport Main Implementation
7

8
Core TCP transport implementation extending Elasticsearch's TcpTransport base class with Netty4-specific networking.
9

10
```java { .api }
11
/**
12
 * Netty4 implementation of TCP transport for node-to-node communication
13
 * Handles cluster messaging, data replication, and search coordination
14
 */
15
public class Netty4Transport extends TcpTransport {
16
    // Configuration settings for performance tuning
17
    public static final Setting<Integer> WORKER_COUNT;
18
    public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_SIZE;
19
    public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_MIN;
20
    public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_MAX;
21
    public static final Setting<Integer> NETTY_BOSS_COUNT;
22
}
23
```
24

25
### Transport Configuration Settings
26

27
Performance and resource configuration settings for optimal cluster communication.
28

29
```java { .api }
30
/**
31
 * Number of worker threads for handling network I/O operations
32
 * Default: number of available processors
33
 */
34
public static final Setting<Integer> WORKER_COUNT = new Setting<>(
35
    "transport.netty.worker_count",
36
    (s) -> Integer.toString(EsExecutors.allocatedProcessors(s)),
37
    (s) -> Setting.parseInt(s, 1, "transport.netty.worker_count"),
38
    Property.NodeScope
39
);
40

41
/**
42
 * Initial buffer size for receive buffer prediction
43
 * Netty adaptively adjusts buffer sizes based on actual data patterns
44
 */
45
public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_SIZE = Setting.byteSizeSetting(
46
    "transport.netty.receive_predictor_size",
47
    new ByteSizeValue(64, ByteSizeUnit.KB),
48
    Property.NodeScope
49
);
50

51
/**
52
 * Minimum receive buffer size for adaptive allocation
53
 */
54
public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_MIN = byteSizeSetting(
55
    "transport.netty.receive_predictor_min",
56
    NETTY_RECEIVE_PREDICTOR_SIZE,
57
    Property.NodeScope
58
);
59

60
/**
61
 * Maximum receive buffer size for adaptive allocation  
62
 */
63
public static final Setting<ByteSizeValue> NETTY_RECEIVE_PREDICTOR_MAX = byteSizeSetting(
64
    "transport.netty.receive_predictor_max", 
65
    NETTY_RECEIVE_PREDICTOR_SIZE,
66
    Property.NodeScope
67
);
68

69
/**
70
 * Number of boss threads for accepting connections
71
 * Usually 1 is sufficient for most deployments
72
 */
73
public static final Setting<Integer> NETTY_BOSS_COUNT = intSetting(
74
    "transport.netty.boss_count", 
75
    1, 
76
    1, 
77
    Property.NodeScope
78
);
79
```
80

81
**Configuration Example:**
82

83
```java
84
Settings transportSettings = Settings.builder()
85
    .put("transport.netty.worker_count", 4)
86
    .put("transport.netty.receive_predictor_size", "128kb")
87
    .put("transport.netty.receive_predictor_min", "64kb")
88
    .put("transport.netty.receive_predictor_max", "256kb")
89
    .put("transport.netty.boss_count", 1)
90
    .build();
91
```
92

93
### TCP Channel Implementation
94

95
Client-side TCP channel implementation for outbound connections to other cluster nodes.
96

97
```java { .api }
98
/**
99
 * Netty4 implementation of TCP channel for node-to-node communication
100
 * Handles connection lifecycle and message transmission
101
 */
102
public class Netty4TcpChannel implements TcpChannel {
103
    /**
104
     * Adds listener to be notified when channel closes
105
     * @param listener Callback invoked on channel close
106
     */
107
    void addCloseListener(ActionListener<Void> listener);
108
    
109
    /**
110
     * Checks if channel is currently open and active
111
     * @return true if channel is open, false otherwise
112
     */
113
    boolean isOpen();
114
    
115
    /**
116
     * Gets local socket address for this channel
117
     * @return Local InetSocketAddress
118
     */
119
    InetSocketAddress getLocalAddress();
120
    
121
    /**
122
     * Gets remote socket address for this channel
123
     * @return Remote InetSocketAddress of connected peer
124
     */
125
    InetSocketAddress getRemoteAddress();
126
    
127
    /**
128
     * Sends message bytes to remote peer asynchronously
129
     * @param reference Message bytes to send
130
     * @param listener Callback for send completion or failure
131
     */
132
    void sendMessage(BytesReference reference, ActionListener<Void> listener);
133
    
134
    /**
135
     * Closes the channel and releases resources
136
     */
137
    void close();
138
}
139
```
140

141
**Usage Example:**
142

143
```java
144
// Channel created by transport layer
145
Netty4TcpChannel channel = // obtained from transport
146

147
// Send message to remote node
148
BytesReference messageData = // serialized message
149
channel.sendMessage(messageData, new ActionListener<Void>() {
150
    @Override
151
    public void onResponse(Void response) {
152
        // Message sent successfully
153
    }
154
    
155
    @Override
156
    public void onFailure(Exception e) {
157
        // Handle send failure
158
    }
159
});
160

161
// Monitor channel lifecycle
162
channel.addCloseListener(ActionListener.wrap(
163
    () -> logger.info("Channel closed"),
164
    e -> logger.error("Channel close error", e)
165
));
166
```
167

168
### TCP Server Channel Implementation
169

170
Server-side TCP channel implementation for accepting inbound connections from other cluster nodes.
171

172
```java { .api }
173
/**
174
 * Netty4 server channel implementation for accepting TCP connections
175
 * Binds to local address and accepts connections from cluster peers
176
 */
177
public class Netty4TcpServerChannel implements TcpServerChannel {
178
    /**
179
     * Adds listener to be notified when server channel closes
180
     * @param listener Callback invoked on channel close
181
     */
182
    void addCloseListener(ActionListener<Void> listener);
183
    
184
    /**
185
     * Checks if server channel is currently open and accepting connections
186
     * @return true if channel is open, false otherwise
187
     */
188
    boolean isOpen();
189
    
190
    /**
191
     * Gets local socket address this server is bound to
192
     * @return Local InetSocketAddress for bound server socket
193
     */
194
    InetSocketAddress getLocalAddress();
195
    
196
    /**
197
     * Closes the server channel and stops accepting new connections
198
     */
199
    void close();
200
}
201
```
202

203
**Usage Example:**
204

205
```java
206
// Server channel created by transport layer
207
Netty4TcpServerChannel serverChannel = // obtained from transport
208

209
// Monitor server channel lifecycle
210
serverChannel.addCloseListener(ActionListener.wrap(
211
    () -> logger.info("Server channel closed"),
212
    e -> logger.error("Server channel close error", e)
213
));
214

215
// Check if server is accepting connections
216
if (serverChannel.isOpen()) {
217
    InetSocketAddress boundAddress = serverChannel.getLocalAddress();
218
    logger.info("Transport server listening on {}", boundAddress);
219
}
220
```
221

222
### Message Channel Handler
223

224
Netty pipeline handler for processing transport messages and managing channel lifecycle.
225

226
```java { .api }
227
/**
228
 * Netty channel handler for processing transport protocol messages
229
 * Handles message framing, deserialization, and dispatch to transport layer
230
 */
231
public class Netty4MessageChannelHandler extends // Netty handler
232
```
233

234
### Transport Protocol Features
235

236
The TCP transport layer provides several key features:
237

238
**Message Framing**: Transport messages are framed with length headers for reliable message boundaries
239

240
**Compression**: Optional message compression to reduce network bandwidth usage
241

242
**Connection Pooling**: Reuses connections between nodes to minimize connection overhead
243

244
**Adaptive Buffering**: Netty's adaptive receive buffer allocation optimizes memory usage based on traffic patterns
245

246
**Flow Control**: Back-pressure mechanisms prevent overwhelming slower nodes
247

248
**Connection Management**: Automatic connection retry and failure detection for resilient cluster communication
249

250
## Transport Layer Integration
251

252
The TCP transport integrates with Elasticsearch through several mechanisms:
253

254
1. **Service Discovery**: Integrates with Elasticsearch's discovery system to find cluster nodes
255

256
2. **Message Serialization**: Uses Elasticsearch's StreamInput/StreamOutput for efficient message serialization
257

258
3. **Thread Pool Integration**: Leverages Elasticsearch's thread pools for message processing
259

260
4. **Circuit Breaker Integration**: Respects circuit breakers to prevent resource exhaustion
261

262
5. **Metrics and Monitoring**: Provides transport statistics and performance metrics
263

264
6. **Security Integration**: Supports TLS encryption and authentication when configured
265

266
The transport layer is the foundation for all cluster communication including:
267
- Cluster state propagation
268
- Index and search operations
269
- Data replication
270
- Node discovery and fault detection
271
- Cross-cluster search and replication