0
# JavaCPP FFmpeg Bindings
1

2
JavaCPP FFmpeg bindings provide comprehensive Java access to FFmpeg 7.1.1, the complete multimedia framework for audio/video encoding, decoding, transcoding, filtering, and streaming. This library enables Java developers to harness FFmpeg's powerful C/C++ libraries through a type-safe Java API.
3

4
## Package Information
5

6
- **Package Name**: org.bytedeco:ffmpeg
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: 
10
  ```xml
11
  <dependency>
12
    <groupId>org.bytedeco</groupId>
13
    <artifactId>ffmpeg-platform</artifactId>
14
    <version>7.1.1-1.5.12</version>
15
  </dependency>
16
  ```
17

18
## Core Imports
19

20
Core FFmpeg functionality access:
21

22
```java
23
import org.bytedeco.ffmpeg.avcodec.*;
24
import org.bytedeco.ffmpeg.avformat.*;
25
import org.bytedeco.ffmpeg.avutil.*;
26
import org.bytedeco.ffmpeg.avfilter.*;
27
import org.bytedeco.ffmpeg.swscale.*;
28
import org.bytedeco.ffmpeg.swresample.*;
29

30
import static org.bytedeco.ffmpeg.global.avcodec.*;
31
import static org.bytedeco.ffmpeg.global.avformat.*;
32
import static org.bytedeco.ffmpeg.global.avutil.*;
33
import static org.bytedeco.ffmpeg.global.swscale.*;
34
```
35

36
Command-line tool access:
37

38
```java
39
import org.bytedeco.ffmpeg.ffmpeg;
40
import org.bytedeco.ffmpeg.ffprobe;
41
import org.bytedeco.javacpp.Loader;
42
```
43

44
## Basic Usage
45

46
```java
47
import org.bytedeco.ffmpeg.avformat.*;
48
import org.bytedeco.ffmpeg.avcodec.*;
49
import org.bytedeco.ffmpeg.avutil.*;
50
import static org.bytedeco.ffmpeg.global.avformat.*;
51
import static org.bytedeco.ffmpeg.global.avcodec.*;
52
import static org.bytedeco.ffmpeg.global.avutil.*;
53

54
// Open video file and read frames
55
AVFormatContext formatContext = new AVFormatContext(null);
56
AVPacket packet = new AVPacket();
57

58
// Open input file
59
int result = avformat_open_input(formatContext, "input.mp4", null, null);
60
if (result < 0) {
61
    throw new RuntimeException("Cannot open file");
62
}
63

64
// Find stream information
65
avformat_find_stream_info(formatContext, (PointerPointer)null);
66

67
// Find video stream
68
int videoStreamIndex = -1;
69
for (int i = 0; i < formatContext.nb_streams(); i++) {
70
    if (formatContext.streams(i).codecpar().codec_type() == AVMEDIA_TYPE_VIDEO) {
71
        videoStreamIndex = i;
72
        break;
73
    }
74
}
75

76
// Set up decoder
77
AVCodecContext codecContext = avcodec_alloc_context3(null);
78
avcodec_parameters_to_context(codecContext, formatContext.streams(videoStreamIndex).codecpar());
79
AVCodec codec = avcodec_find_decoder(codecContext.codec_id());
80
avcodec_open2(codecContext, codec, (PointerPointer)null);
81

82
// Read frames
83
AVFrame frame = av_frame_alloc();
84
while (av_read_frame(formatContext, packet) >= 0) {
85
    if (packet.stream_index() == videoStreamIndex) {
86
        avcodec_send_packet(codecContext, packet);
87
        while (avcodec_receive_frame(codecContext, frame) >= 0) {
88
            // Process frame data
89
            System.out.println("Frame: " + frame.width() + "x" + frame.height());
90
        }
91
    }
92
    av_packet_unref(packet);
93
}
94

95
// Cleanup
96
av_frame_free(frame);
97
avcodec_free_context(codecContext);
98
avformat_close_input(formatContext);
99
```
100

101
## Architecture
102

103
JavaCPP FFmpeg bindings are organized into several key modules:
104

105
- **Global Modules**: Static native method wrappers (`org.bytedeco.ffmpeg.global.*`) providing direct access to FFmpeg C functions
106
- **Data Structure Classes**: Java representations of FFmpeg structs (`org.bytedeco.ffmpeg.avcodec.*`, etc.) with memory management
107
- **Preset Classes**: Compilation configuration for native library integration (`org.bytedeco.ffmpeg.presets.*`)
108
- **Executable Wrappers**: Access to command-line tools (`ffmpeg`, `ffprobe`) for external process execution
109
- **Memory Management**: Reference-counted buffers via AVBuffer system with automatic garbage collection integration
110

111
## Capabilities
112

113
### Format Handling (Muxing/Demuxing)
114

115
Container format operations for reading and writing media files. Supports all major formats including MP4, AVI, MKV, MOV, WebM, and streaming protocols.
116

117
```java { .api }
118
int avformat_open_input(AVFormatContext ctx, String filename, AVInputFormat fmt, PointerPointer options);
119
int avformat_find_stream_info(AVFormatContext ctx, PointerPointer options);
120
int av_read_frame(AVFormatContext ctx, AVPacket pkt);
121
int av_write_frame(AVFormatContext ctx, AVPacket pkt);
122
```
123

124
[Format Handling](./format-handling.md)
125

126
### Codec Operations (Encoding/Decoding)
127

128
Audio and video encoding/decoding with support for all major codecs including H.264, H.265, VP9, AV1, AAC, MP3, and many others.
129

130
```java { .api }
131
AVCodec avcodec_find_encoder(int id);
132
AVCodec avcodec_find_decoder(int id);
133
AVCodecContext avcodec_alloc_context3(AVCodec codec);
134
int avcodec_open2(AVCodecContext avctx, AVCodec codec, PointerPointer options);
135
int avcodec_send_packet(AVCodecContext avctx, AVPacket avpkt);
136
int avcodec_receive_frame(AVCodecContext avctx, AVFrame frame);
137
```
138

139
[Codec Operations](./codec-operations.md)
140

141
### Media Filtering
142

143
Comprehensive audio and video filtering pipeline for effects, transformations, format conversions, and complex processing graphs.
144

145
```java { .api }
146
AVFilterGraph avfilter_graph_alloc();
147
int avfilter_graph_create_filter(AVFilterContext filt_ctx, AVFilter filt, 
148
    String name, String args, Pointer opaque, AVFilterGraph graph_ctx);
149
int avfilter_graph_config(AVFilterGraph graphctx, Pointer log_ctx);
150
```
151

152
[Media Filtering](./media-filtering.md)
153

154
### Scaling and Format Conversion
155

156
Video scaling, pixel format conversion, and color space transformations with optimized implementations.
157

158
```java { .api }
159
SwsContext sws_getContext(int srcW, int srcH, int srcFormat, 
160
    int dstW, int dstH, int dstFormat, int flags,
161
    SwsFilter srcFilter, SwsFilter dstFilter, DoublePointer param);
162
int sws_scale(SwsContext c, PointerPointer srcSlice, IntPointer srcStride,
163
    int srcSliceY, int srcSliceH, PointerPointer dst, IntPointer dstStride);
164
```
165

166
[Scaling and Format Conversion](./scaling-conversion.md)
167

168
### Audio Processing
169

170
Audio resampling, channel layout conversion, and sample format transformations for comprehensive audio processing.
171

172
```java { .api }
173
SwrContext swr_alloc();
174
int swr_init(SwrContext s);
175
int swr_convert(SwrContext s, PointerPointer out, int out_count,
176
    PointerPointer in, int in_count);
177
```
178

179
[Audio Processing](./audio-processing.md)
180

181
### Device I/O
182

183
Device enumeration and access for cameras, microphones, and other multimedia input/output devices across platforms.
184

185
```java { .api }
186
int avdevice_list_devices(AVFormatContext s, AVDeviceInfoList device_list);
187
int avdevice_list_input_sources(AVInputFormat device, String device_name,
188
    AVDictionary device_options, AVDeviceInfoList device_list);
189
```
190

191
[Device I/O](./device-io.md)
192

193
### Command-Line Tools
194

195
Direct access to ffmpeg and ffprobe executables for complex operations and external processing workflows.
196

197
```java { .api }
198
// Access via Loader.load() for process execution
199
String ffmpegPath = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
200
String ffprobePath = Loader.load(org.bytedeco.ffmpeg.ffprobe.class);
201
```
202

203
[Command-Line Tools](./command-line-tools.md)
204

205
### Video Post-Processing
206

207
Video enhancement and quality improvement through deblocking, deringing, and color correction filters for decoded video content.
208

209
```java { .api }
210
pp_context pp_get_context(int width, int height, int flags);
211
pp_mode pp_get_mode_by_name_and_quality(String name, int quality);
212
void pp_postprocess(PointerPointer src, IntPointer srcStride,
213
                   PointerPointer dst, IntPointer dstStride,
214
                   int horizontalSize, int verticalSize,
215
                   BytePointer QP_store, int QP_stride,
216
                   pp_mode mode, pp_context ppContext, int pict_type);
217
```
218

219
[Video Post-Processing](./postproc.md)
220

221
### Hardware Acceleration
222

223
GPU and specialized hardware acceleration support for high-performance encoding, decoding, and processing with CUDA, VAAPI, DXVA2, VideoToolbox, and other acceleration APIs.
224

225
```java { .api }
226
int av_hwdevice_find_type_by_name(String name);
227
AVBufferRef av_hwdevice_ctx_alloc(int type);
228
int av_hwdevice_ctx_create(@ByPtrPtr AVBufferRef device_ctx, int type,
229
                          String device, AVDictionary opts, int flags);
230
int av_hwframe_transfer_data(AVFrame dst, AVFrame src, int flags);
231
AVCodecHWConfig avcodec_get_hw_config(AVCodec codec, int index);
232
```
233

234
[Hardware Acceleration](./hardware-acceleration.md)
235

236
### Cryptographic and Security Features
237

238
Cryptographic functions and security features for multimedia data protection, including AES encryption, hash functions, and media content encryption support.
239

240
```java { .api }
241
AVAES av_aes_alloc();
242
int av_aes_init(AVAES a, byte[] key, int key_bits, int decrypt);
243
void av_aes_crypt(AVAES a, byte[] dst, byte[] src, int count, byte[] iv, int decrypt);
244
int av_hash_alloc(@ByPtrPtr AVHashContext ctx, String name);
245
void av_hash_update(AVHashContext ctx, BytePointer src, int len);
246
void av_hash_final(AVHashContext ctx, BytePointer dst);
247
```
248

249
[Cryptographic and Security Features](./cryptographic-security.md)
250

251
### Constants and Enumerations
252

253
Comprehensive reference of constants, enumerations, and compile-time values for configuration, error handling, and API parameters.
254

255
```java { .api }
256
// Media types, pixel formats, sample formats
257
int AVMEDIA_TYPE_VIDEO = 0;
258
int AV_PIX_FMT_YUV420P = 0;
259
int AV_SAMPLE_FMT_FLTP = 8;
260

261
// Codec IDs
262
int AV_CODEC_ID_H264 = 27;
263
int AV_CODEC_ID_AAC = 86018;
264

265
// Error codes
266
int AVERROR_EOF();
267
int AVERROR_EAGAIN();
268
```
269

270
[Constants and Enumerations](./constants-enums.md)
271

272
## Core Data Structures
273

274
### AVFrame
275
The fundamental structure for raw audio/video data.
276

277
```java { .api }
278
class AVFrame extends Pointer {
279
    AVFrame av_frame_alloc();
280
    void av_frame_free(AVFrame frame);
281
    int av_frame_ref(AVFrame dst, AVFrame src);
282
    void av_frame_unref(AVFrame frame);
283
    
284
    // Data access
285
    PointerPointer data();
286
    IntPointer linesize();
287
    int width();
288
    int height();
289
    int format();
290
    long pts();
291
}
292
```
293

294
### AVPacket
295
Container for compressed/encoded media data.
296

297
```java { .api }
298
class AVPacket extends Pointer {
299
    AVPacket av_packet_alloc();
300
    void av_packet_free(AVPacket pkt);
301
    int av_packet_ref(AVPacket dst, AVPacket src);
302
    void av_packet_unref(AVPacket pkt);
303
    
304
    // Packet properties
305
    BytePointer data();
306
    int size();
307
    long pts();
308
    long dts();
309
    int stream_index();
310
    int flags();
311
}
312
```
313

314
### AVFormatContext
315
Container format context for input/output operations.
316

317
```java { .api }
318
class AVFormatContext extends Pointer {
319
    AVFormatContext avformat_alloc_context();
320
    void avformat_free_context(AVFormatContext s);
321
    
322
    // Stream access
323
    int nb_streams();
324
    PointerPointer streams();
325
    AVInputFormat iformat();
326
    AVOutputFormat oformat();
327
    
328
    // Metadata
329
    AVDictionary metadata();
330
    long duration();
331
    long bit_rate();
332
}
333
```
334

335
## Error Handling
336

337
FFmpeg functions typically return negative values for errors. Use `AVERROR` macros for error checking:
338

339
```java
340
int result = avformat_open_input(formatContext, filename, null, null);
341
if (result < 0) {
342
    if (result == AVERROR_ENOENT()) {
343
        throw new FileNotFoundException("File not found: " + filename);
344
    } else {
345
        throw new RuntimeException("Failed to open file, error code: " + result);
346
    }
347
}
348
```
349

350
## Memory Management
351

352
JavaCPP handles Java-to-native memory mapping automatically, but explicit cleanup is required for native contexts:
353

354
```java
355
// Always pair allocation with deallocation
356
AVFrame frame = av_frame_alloc();
357
try {
358
    // Use frame
359
} finally {
360
    av_frame_free(frame);
361
}
362

363
// Use reference counting for shared resources
364
AVPacket packet = av_packet_alloc();
365
AVPacket copy = av_packet_alloc();
366
av_packet_ref(copy, packet); // Increment reference count
367
av_packet_unref(copy); // Decrement reference count
368
av_packet_free(packet); // Final cleanup
369
```