0
# Apache Avro IPC
1

2
Apache Avro IPC provides a comprehensive RPC (Remote Procedure Call) framework for Java applications using Avro's data serialization format. It enables type-safe inter-process communication with support for multiple transport mechanisms, protocol implementations, and authentication methods.
3

4
## Package Information
5

6
- **Package Name**: avro-ipc
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: Add to Maven dependencies: `org.apache.avro:avro-ipc:1.12.0`
10
- **Artifact Coordinates**: `org.apache.avro:avro-ipc`
11
- **License**: Apache License 2.0
12

13
## Core Imports
14

15
```java
16
import org.apache.avro.ipc.*; // Core IPC classes
17
import org.apache.avro.ipc.generic.*; // Generic protocol support
18
import org.apache.avro.ipc.specific.*; // Generated interface support
19
import org.apache.avro.ipc.reflect.*; // Reflection-based protocol support
20
```
21

22
Common transport imports:
23
```java
24
import org.apache.avro.ipc.HttpTransceiver;
25
import org.apache.avro.ipc.SaslSocketTransceiver;
26
import org.apache.avro.ipc.ResponderServlet;
27
import org.apache.avro.ipc.SaslSocketServer;
28
```
29

30
## Basic Usage
31

32
### Simple HTTP Client/Server
33

34
```java
35
import org.apache.avro.Protocol;
36
import org.apache.avro.ipc.HttpTransceiver;
37
import org.apache.avro.ipc.ResponderServlet;
38
import org.apache.avro.ipc.specific.SpecificRequestor;
39
import org.apache.avro.ipc.specific.SpecificResponder;
40
import java.net.URL;
41

42
// Client side - using generated interface
43
MyService client = SpecificRequestor.getClient(MyService.class, 
44
    new HttpTransceiver(new URL("http://localhost:8080/rpc")));
45

46
// Server side - servlet deployment
47
MyServiceImpl implementation = new MyServiceImpl();
48
SpecificResponder responder = new SpecificResponder(MyService.class, implementation);
49
ResponderServlet servlet = new ResponderServlet(responder);
50
// Deploy servlet to web container
51
```
52

53
### Socket Server with Authentication
54

55
```java
56
import org.apache.avro.ipc.SaslSocketServer;
57
import org.apache.avro.ipc.SaslSocketTransceiver;
58
import java.net.InetSocketAddress;
59

60
// Server with SASL authentication
61
SpecificResponder responder = new SpecificResponder(MyService.class, implementation);
62
SaslSocketServer server = new SaslSocketServer(responder, 
63
    new InetSocketAddress(65001));
64
server.start();
65

66
// Client with SASL authentication
67
SaslSocketTransceiver transceiver = new SaslSocketTransceiver(
68
    new InetSocketAddress("localhost", 65001));
69
MyService client = SpecificRequestor.getClient(MyService.class, transceiver);
70
```
71

72
## Architecture
73

74
The Avro IPC framework follows a layered architecture:
75

76
- **Transport Layer**: Handles network communication (HTTP, Socket, Datagram, Local)
77
- **Protocol Layer**: Manages serialization and protocol negotiation (Generic, Specific, Reflect)
78
- **Service Layer**: Implements RPC semantics with client/server abstractions
79
- **Plugin System**: Extensible instrumentation and metadata management
80

81
Key components:
82
- **Requestor**: Client-side RPC proxy that converts method calls to network requests
83
- **Responder**: Server-side dispatcher that routes requests to implementation methods
84
- **Transceiver**: Transport abstraction handling network I/O
85
- **Server**: Server lifecycle management interface
86

87
## Capabilities
88

89
### Core RPC Framework
90

91
Foundation classes for client-server RPC communication including base requestor/responder abstractions, transport interfaces, and server lifecycle management.
92

93
```java { .api }
94
// Client-side RPC base class
95
public abstract class Requestor {
96
    protected Requestor(Protocol local, Transceiver transceiver) throws IOException;
97
    public Protocol getLocal();
98
    public Transceiver getTransceiver();
99
    public Protocol getRemote() throws IOException;
100
    public void addRPCPlugin(RPCPlugin plugin);
101
    public Object request(String messageName, Object request) throws Exception;
102
    public <T> void request(String messageName, Object request, Callback<T> callback) throws Exception;
103
    // Abstract methods for protocol-specific implementations
104
    public abstract void writeRequest(Schema schema, Object request, Encoder out) throws IOException;
105
    public abstract Object readResponse(Schema writer, Schema reader, Decoder in) throws IOException;
106
    public abstract Exception readError(Schema writer, Schema reader, Decoder in) throws IOException;
107
}
108

109
// Server-side RPC base class  
110
public abstract class Responder {
111
    protected Responder(Protocol local);
112
    public static Protocol getRemote();
113
    public Protocol getLocal();
114
    public void addRPCPlugin(RPCPlugin plugin);
115
    public List<ByteBuffer> respond(List<ByteBuffer> buffers) throws IOException;
116
    public List<ByteBuffer> respond(List<ByteBuffer> buffers, Transceiver connection) throws IOException;
117
    public abstract Object respond(Message message, Object request) throws Exception;
118
    // Abstract methods for protocol-specific implementations
119
    public abstract Object readRequest(Schema actual, Schema expected, Decoder in) throws IOException;
120
    public abstract void writeResponse(Schema schema, Object response, Encoder out) throws IOException;
121
    public abstract void writeError(Schema schema, Object error, Encoder out) throws IOException;
122
}
123

124
// Transport abstraction
125
public abstract class Transceiver implements Closeable {
126
    public abstract String getRemoteName() throws IOException;
127
    public boolean isConnected();
128
    public void setRemote(Protocol protocol);
129
    public Protocol getRemote();
130
    public void lockChannel();
131
    public void unlockChannel();
132
    public List<ByteBuffer> transceive(List<ByteBuffer> request) throws IOException;
133
    public void transceive(List<ByteBuffer> request, Callback<List<ByteBuffer>> callback) throws IOException;
134
    // Abstract I/O methods
135
    public abstract List<ByteBuffer> readBuffers() throws IOException;
136
    public abstract void writeBuffers(List<ByteBuffer> buffers) throws IOException;
137
    public void close() throws IOException;
138
}
139

140
// Server interface
141
public interface Server {
142
    int getPort();
143
    void start();
144
    void close();
145
    void join() throws InterruptedException;
146
}
147
```
148

149
[Core RPC Framework](./core-rpc.md)
150

151
### Transport Mechanisms
152

153
Multiple transport implementations for different networking requirements including HTTP, socket-based, datagram, and in-process communication options.
154

155
```java { .api }
156
// HTTP transport
157
public class HttpTransceiver extends Transceiver {
158
    public HttpTransceiver(URL url) throws IOException;
159
    public HttpTransceiver(URL url, Proxy proxy) throws IOException;
160
    public void setTimeout(int timeout);
161
}
162

163
// SASL-authenticated socket transport
164
public class SaslSocketTransceiver extends Transceiver {
165
    public SaslSocketTransceiver(SocketAddress address) throws IOException;
166
    public SaslSocketTransceiver(SocketAddress address, SaslClient saslClient) throws IOException;
167
    public SaslSocketTransceiver(SocketChannel channel, SaslServer saslServer) throws IOException;
168
}
169

170
// Socket servers
171
public class SaslSocketServer extends SocketServer {
172
    public SaslSocketServer(Responder responder, SocketAddress addr) throws IOException;
173
    public SaslSocketServer(Responder responder, SocketAddress addr, String mechanism, 
174
                           String protocol, String serverName, Map<String, ?> props, 
175
                           CallbackHandler cbh) throws IOException;
176
}
177

178
// HTTP servlet
179
public class ResponderServlet extends HttpServlet {
180
    public ResponderServlet(Responder responder);
181
}
182
```
183

184
[Transport Mechanisms](./transports.md)
185

186
### Protocol Implementations
187

188
Three protocol implementations supporting different Java object models: generic (no code generation), specific (generated classes), and reflect (existing interfaces).
189

190
```java { .api }
191
// Generic protocol - no code generation required
192
public class GenericRequestor extends Requestor {
193
    public GenericRequestor(Protocol protocol, Transceiver transceiver) throws IOException;
194
    public GenericRequestor(Protocol protocol, Transceiver transceiver, GenericData data) throws IOException;
195
    public GenericData getGenericData();
196
}
197

198
public abstract class GenericResponder extends Responder {
199
    public GenericResponder(Protocol local);
200
    public GenericResponder(Protocol local, GenericData data);
201
    public GenericData getGenericData();
202
}
203

204
// Specific protocol - generated Java classes
205
public class SpecificRequestor extends Requestor implements InvocationHandler {
206
    public SpecificRequestor(Class<?> iface, Transceiver transceiver) throws IOException;
207
    public SpecificRequestor(Class<?> iface, Transceiver transceiver, SpecificData data) throws IOException;
208
    public SpecificRequestor(Protocol protocol, Transceiver transceiver, SpecificData data) throws IOException;
209
    public static <T> T getClient(Class<T> iface, Transceiver transceiver) throws IOException;
210
    public static <T> T getClient(Class<T> iface, Transceiver transceiver, SpecificData data) throws IOException;
211
    public static <T> T getClient(Class<T> iface, SpecificRequestor requestor) throws IOException;
212
    public SpecificData getSpecificData();
213
}
214

215
public class SpecificResponder extends GenericResponder {
216
    public SpecificResponder(Class iface, Object impl);
217
    public SpecificData getSpecificData();
218
}
219

220
// Reflect protocol - existing Java interfaces
221
public class ReflectRequestor extends SpecificRequestor {
222
    public ReflectRequestor(Class<?> iface, Transceiver transceiver) throws IOException;
223
    public static <T> T getClient(Class<T> iface, Transceiver transceiver) throws IOException;
224
    public ReflectData getReflectData();
225
}
226

227
public class ReflectResponder extends SpecificResponder {
228
    public ReflectResponder(Class iface, Object impl);
229
    public ReflectData getReflectData();
230
}
231
```
232

233
[Protocol Implementations](./protocols.md)
234

235
### Asynchronous Operations
236

237
Callback-based and Future-based patterns for non-blocking RPC operations with comprehensive error handling support.
238

239
```java { .api }
240
// Callback interface
241
public interface Callback<T> {
242
    void handleResult(T result);
243
    void handleError(Throwable error);
244
}
245

246
// Future implementation
247
public class CallFuture<T> implements Future<T>, Callback<T> {
248
    public CallFuture();
249
    public CallFuture(Callback<T> chainedCallback);
250
    public T getResult() throws Exception;
251
    public Throwable getError();
252
    public void await() throws InterruptedException;
253
    public void await(long timeout, TimeUnit unit) throws InterruptedException;
254
}
255
```
256

257
[Asynchronous Operations](./async.md)
258

259
### Plugin System and Context
260

261
Extensible instrumentation framework for RPC metadata manipulation, performance monitoring, and custom cross-cutting concerns.
262

263
```java { .api }
264
// Plugin base class
265
public class RPCPlugin {
266
    public void clientStartConnect(RPCContext context);
267
    public void serverConnecting(RPCContext context);
268
    public void clientFinishConnect(RPCContext context);
269
    public void clientSendRequest(RPCContext context);
270
    public void serverReceiveRequest(RPCContext context);
271
    public void serverSendResponse(RPCContext context);
272
    public void clientReceiveResponse(RPCContext context);
273
}
274

275
// RPC context
276
public class RPCContext {
277
    public Map<String, ByteBuffer> requestHandshakeMeta();
278
    public Map<String, ByteBuffer> responseHandshakeMeta();
279
    public Map<String, ByteBuffer> requestCallMeta();
280
    public Map<String, ByteBuffer> responseCallMeta();
281
    public Message getMessage();
282
    public Object response();
283
    public Exception error();
284
    public boolean isError();
285
}
286
```
287

288
[Plugin System](./plugins.md)
289

290
### Statistics and Monitoring
291

292
Built-in performance monitoring with histograms, latency tracking, payload size analysis, and web-based statistics viewer.
293

294
```java { .api }
295
// Statistics collection plugin
296
public class StatsPlugin extends RPCPlugin {
297
    public StatsPlugin();
298
    public Date startupTime;
299
    public static final Segmenter<String, Float> LATENCY_SEGMENTER;
300
    public static final Segmenter<String, Integer> PAYLOAD_SEGMENTER;
301
    public static float nanosToMillis(long elapsedNanos);
302
}
303

304
// Web-based statistics viewer
305
public class StatsServlet extends HttpServlet {
306
    public StatsServlet(StatsPlugin statsPlugin);
307
    public void writeStats(Writer w) throws IOException;
308
}
309

310
// Histogram utilities
311
public class Histogram<B,T> {
312
    public Histogram(Segmenter<B,T> segmenter);
313
    public void add(T value);
314
    public int[] getHistogram();
315
    public int getCount();
316
}
317
```
318

319
[Statistics and Monitoring](./stats.md)
320

321
## Factory Utilities
322

323
```java { .api }
324
// Factory methods for URI-based creation
325
public class Ipc {
326
    public static Transceiver createTransceiver(URI uri) throws IOException;
327
    public static Server createServer(Responder responder, URI uri) throws IOException;
328
}
329
```
330

331
## Types
332

333
All types are provided by the core Avro library (`org.apache.avro:avro`) and are automatically available when using the IPC library.
334

335
```java { .api }
336
// Core Avro types used throughout IPC
337
import org.apache.avro.Protocol;
338
import org.apache.avro.Protocol.Message;
339
import org.apache.avro.Schema;
340
import org.apache.avro.generic.GenericData;
341
import org.apache.avro.specific.SpecificData;
342
import org.apache.avro.reflect.ReflectData;
343

344
// Standard Java types
345
import java.nio.ByteBuffer;
346
import java.util.List;
347
import java.util.Map;
348
import java.util.concurrent.Future;
349
import java.net.URI;
350
import java.net.URL;
351
import java.net.SocketAddress;
352
```