0
# High-Performance I/O
1

2
Fast, repositionable stream implementations with measurement capabilities for efficient binary and text file operations. These classes provide high-performance alternatives to standard Java I/O classes, optimized for speed and functionality.
3

4
## Overview
5

6
FastUtil's I/O package provides:
7
- **Unsynchronized Operations**: Maximum performance by removing synchronization overhead
8
- **Repositioning Support**: Efficient seeking within streams
9
- **Measurement Capabilities**: Access to stream length and current position
10
- **Enhanced Functionality**: Features missing from standard Java streams
11
- **Memory Mapping**: Utilities for memory-mapped file operations
12

13
All stream classes are designed for single-threaded access to maximize performance.
14

15
## Capabilities
16

17
### Stream Interfaces
18

19
Foundation interfaces that extend standard Java I/O with measurement and repositioning capabilities.
20

21
```java { .api }
22
/**
23
 * Interface for streams that provide access to length and current position
24
 */
25
public interface MeasurableStream {
26
    /**
27
     * Get the overall length of the stream (optional operation)
28
     * @return the stream length, or -1 if unknown
29
     * @throws IOException if an I/O error occurs
30
     */
31
    long length() throws IOException;
32
    
33
    /**
34
     * Get the current position in the stream (optional operation)
35
     * @return the current position, or -1 if unknown
36
     * @throws IOException if an I/O error occurs
37
     */
38
    long position() throws IOException;
39
}
40

41
/**
42
 * Interface for streams supporting repositioning operations
43
 */
44
public interface RepositionableStream {
45
    /**
46
     * Set the stream position
47
     * @param newPosition the new position in the stream
48
     * @throws IOException if an I/O error occurs or position is invalid
49
     */
50
    void position(long newPosition) throws IOException;
51
    
52
    /**
53
     * Get the current stream position
54
     * @return the current position in the stream
55
     * @throws IOException if an I/O error occurs
56
     */
57
    long position() throws IOException;
58
}
59
```
60

61
### Abstract Stream Base Classes
62

63
Base classes providing common functionality for measurable streams.
64

65
```java { .api }
66
/**
67
 * Abstract base class for measurable input streams
68
 */
69
public abstract class MeasurableInputStream extends InputStream implements MeasurableStream {
70
    /**
71
     * Default implementation returns -1 (unknown length)
72
     * @return the stream length, or -1 if unknown
73
     */
74
    public long length() throws IOException {
75
        return -1;
76
    }
77
    
78
    /**
79
     * Default implementation returns -1 (unknown position)
80
     * @return the current position, or -1 if unknown
81
     */
82
    public long position() throws IOException {
83
        return -1;
84
    }
85
}
86

87
/**
88
 * Abstract base class for measurable output streams
89
 */
90
public abstract class MeasurableOutputStream extends OutputStream implements MeasurableStream {
91
    /**
92
     * Default implementation returns -1 (unknown length)
93
     * @return the stream length, or -1 if unknown
94
     */
95
    public long length() throws IOException {
96
        return -1;
97
    }
98
    
99
    /**
100
     * Default implementation returns -1 (unknown position)
101
     * @return the current position, or -1 if unknown
102
     */
103
    public long position() throws IOException {
104
        return -1;
105
    }
106
}
107
```
108

109
### Fast Buffered Streams
110

111
High-performance, unsynchronized buffered streams with repositioning support.
112

113
```java { .api }
114
/**
115
 * High-performance, unsynchronized buffered input stream with repositioning
116
 */
117
public class FastBufferedInputStream extends MeasurableInputStream implements RepositionableStream {
118
    /** Default buffer size */
119
    public static final int DEFAULT_BUFFER_SIZE = 8192;
120
    
121
    /**
122
     * Create buffered input stream with default buffer size
123
     * @param is the underlying input stream
124
     */
125
    public FastBufferedInputStream(InputStream is);
126
    
127
    /**
128
     * Create buffered input stream with specified buffer size
129
     * @param is the underlying input stream
130
     * @param bufSize the buffer size
131
     */
132
    public FastBufferedInputStream(InputStream is, int bufSize);
133
    
134
    /**
135
     * Create buffered input stream from file
136
     * @param file the file to read
137
     * @throws FileNotFoundException if file doesn't exist
138
     */
139
    public FastBufferedInputStream(File file) throws FileNotFoundException;
140
    
141
    /**
142
     * Create buffered input stream from file with specified buffer size
143
     * @param file the file to read
144
     * @param bufSize the buffer size
145
     * @throws FileNotFoundException if file doesn't exist
146
     */
147
    public FastBufferedInputStream(File file, int bufSize) throws FileNotFoundException;
148
    
149
    /**
150
     * Read a single byte
151
     * @return the byte value (0-255), or -1 if end of stream
152
     */
153
    public int read() throws IOException;
154
    
155
    /**
156
     * Read bytes into array
157
     * @param b the byte array
158
     * @param offset starting offset in array
159
     * @param length maximum bytes to read
160
     * @return number of bytes read, or -1 if end of stream
161
     */
162
    public int read(byte[] b, int offset, int length) throws IOException;
163
    
164
    /**
165
     * Skip bytes efficiently (true skipping, not just reading)
166
     * @param n number of bytes to skip
167
     * @return number of bytes actually skipped
168
     */
169
    public long skip(long n) throws IOException;
170
    
171
    /**
172
     * Read a line of text (bytes until newline)
173
     * @param a byte array to store line (will be reallocated if needed)
174
     * @return number of bytes in line, or -1 if end of stream
175
     */
176
    public int readLine(byte[] a) throws IOException;
177
    
178
    /**
179
     * Read a line of text as byte array
180
     * @return byte array containing line, or null if end of stream
181
     */
182
    public byte[] readLine() throws IOException;
183
    
184
    /**
185
     * Set stream position (if underlying stream supports it)
186
     * @param newPosition the new position
187
     */
188
    public void position(long newPosition) throws IOException;
189
    
190
    /**
191
     * Get current stream position (if underlying stream supports it)
192
     * @return the current position
193
     */
194
    public long position() throws IOException;
195
    
196
    /**
197
     * Get stream length (if underlying stream supports it)
198
     * @return the stream length
199
     */
200
    public long length() throws IOException;
201
}
202

203
/**
204
 * High-performance, unsynchronized buffered output stream with repositioning
205
 */
206
public class FastBufferedOutputStream extends MeasurableOutputStream implements RepositionableStream {
207
    /** Default buffer size */
208
    public static final int DEFAULT_BUFFER_SIZE = 8192;
209
    
210
    /**
211
     * Create buffered output stream with default buffer size
212
     * @param os the underlying output stream
213
     */
214
    public FastBufferedOutputStream(OutputStream os);
215
    
216
    /**
217
     * Create buffered output stream with specified buffer size
218
     * @param os the underlying output stream
219
     * @param bufSize the buffer size
220
     */
221
    public FastBufferedOutputStream(OutputStream os, int bufSize);
222
    
223
    /**
224
     * Create buffered output stream to file
225
     * @param file the file to write
226
     * @throws FileNotFoundException if file cannot be created
227
     */
228
    public FastBufferedOutputStream(File file) throws FileNotFoundException;
229
    
230
    /**
231
     * Write a single byte
232
     * @param b the byte value to write
233
     */
234
    public void write(int b) throws IOException;
235
    
236
    /**
237
     * Write bytes from array
238
     * @param b the byte array
239
     * @param offset starting offset in array
240
     * @param length number of bytes to write
241
     */
242
    public void write(byte[] b, int offset, int length) throws IOException;
243
    
244
    /**
245
     * Flush buffered data to underlying stream
246
     */
247
    public void flush() throws IOException;
248
    
249
    /**
250
     * Set stream position (if underlying stream supports it)
251
     * @param newPosition the new position
252
     */
253
    public void position(long newPosition) throws IOException;
254
    
255
    /**
256
     * Get current stream position (if underlying stream supports it)
257
     * @return the current position
258
     */
259
    public long position() throws IOException;
260
    
261
    /**
262
     * Get stream length (if underlying stream supports it)
263
     * @return the stream length
264
     */
265
    public long length() throws IOException;
266
}
267
```
268

269
**Usage Examples:**
270

271
```java
272
import it.unimi.dsi.fastutil.io.*;
273
import java.io.*;
274
import java.nio.charset.StandardCharsets;
275

276
// Fast file reading with repositioning
277
try (FastBufferedInputStream input = new FastBufferedInputStream(new File("data.txt"))) {
278
    // Check file length
279
    long fileSize = input.length();
280
    System.out.println("File size: " + fileSize + " bytes");
281
    
282
    // Read some data
283
    byte[] buffer = new byte[1024];
284
    int bytesRead = input.read(buffer);
285
    
286
    // Jump to middle of file
287
    input.position(fileSize / 2);
288
    long currentPos = input.position();
289
    System.out.println("Current position: " + currentPos);
290
    
291
    // Read line by line
292
    byte[] line;
293
    while ((line = input.readLine()) != null) {
294
        String text = new String(line, StandardCharsets.UTF_8);
295
        System.out.println("Line: " + text);
296
    }
297
}
298

299
// Fast file writing with buffering
300
try (FastBufferedOutputStream output = new FastBufferedOutputStream(new File("output.txt"))) {
301
    // Write data efficiently
302
    String data = "Hello, FastUtil I/O!";
303
    output.write(data.getBytes(StandardCharsets.UTF_8));
304
    
305
    // Position and overwrite
306
    output.position(0);
307
    output.write("Hi!!!".getBytes(StandardCharsets.UTF_8));
308
    
309
    // Ensure data is written
310
    output.flush();
311
}
312
```
313

314
### Fast Byte Array Streams
315

316
Memory-based streams with direct array access and correct repositioning semantics.
317

318
```java { .api }
319
/**
320
 * Fast, repositionable byte array input stream with correct semantics
321
 * (fixes issues with java.io.ByteArrayInputStream)
322
 */
323
public class FastByteArrayInputStream extends MeasurableInputStream implements RepositionableStream {
324
    /** Direct access to the underlying byte array */
325
    public byte[] array;
326
    
327
    /** Starting offset in the array */
328
    public int offset;
329
    
330
    /** Number of valid bytes in the array */
331
    public int length;
332
    
333
    /**
334
     * Create from byte array
335
     * @param a the byte array
336
     */
337
    public FastByteArrayInputStream(byte[] a);
338
    
339
    /**
340
     * Create from portion of byte array
341
     * @param a the byte array
342
     * @param offset starting offset
343
     * @param length number of bytes
344
     */
345
    public FastByteArrayInputStream(byte[] a, int offset, int length);
346
    
347
    /**
348
     * Read a single byte
349
     * @return the byte value (0-255), or -1 if end of stream
350
     */
351
    public int read();
352
    
353
    /**
354
     * Read bytes into array
355
     * @param b destination array
356
     * @param offset starting offset in destination
357
     * @param length maximum bytes to read
358
     * @return number of bytes read, or -1 if end of stream
359
     */
360
    public int read(byte[] b, int offset, int length);
361
    
362
    /**
363
     * Skip bytes (efficiently, without reading)
364
     * @param n number of bytes to skip
365
     * @return number of bytes actually skipped
366
     */
367
    public long skip(long n);
368
    
369
    /**
370
     * Get number of available bytes
371
     * @return number of bytes available for reading
372
     */
373
    public int available();
374
    
375
    /**
376
     * Set stream position
377
     * @param newPosition the new position
378
     */
379
    public void position(long newPosition);
380
    
381
    /**
382
     * Get current stream position
383
     * @return the current position
384
     */
385
    public long position();
386
    
387
    /**
388
     * Get stream length
389
     * @return the stream length
390
     */
391
    public long length();
392
    
393
    /**
394
     * Reset stream to beginning
395
     */
396
    public void reset();
397
    
398
    /**
399
     * Mark current position (supports repositioning)
400
     * @param readlimit ignored (always supports repositioning)
401
     */
402
    public void mark(int readlimit);
403
    
404
    /**
405
     * Test if mark/reset is supported
406
     * @return always true
407
     */
408
    public boolean markSupported();
409
}
410

411
/**
412
 * Fast byte array output stream with direct array access
413
 */
414
public class FastByteArrayOutputStream extends OutputStream {
415
    /** Direct access to the underlying byte array */
416
    public byte[] array;
417
    
418
    /** Current length of valid data */
419
    public int length;
420
    
421
    /**
422
     * Create with default initial capacity
423
     */
424
    public FastByteArrayOutputStream();
425
    
426
    /**
427
     * Create with specified initial capacity
428
     * @param initialCapacity the initial capacity
429
     */
430
    public FastByteArrayOutputStream(int initialCapacity);
431
    
432
    /**
433
     * Write a single byte
434
     * @param b the byte value to write
435
     */
436
    public void write(int b);
437
    
438
    /**
439
     * Write bytes from array
440
     * @param b source array
441
     * @param offset starting offset in source
442
     * @param len number of bytes to write
443
     */
444
    public void write(byte[] b, int offset, int len);
445
    
446
    /**
447
     * Reset the stream (length becomes 0, array is reused)
448
     */
449
    public void reset();
450
    
451
    /**
452
     * Get copy of written data as byte array
453
     * @return copy of the data
454
     */
455
    public byte[] toByteArray();
456
    
457
    /**
458
     * Get current length of data
459
     * @return number of bytes written
460
     */
461
    public int size();
462
    
463
    /**
464
     * Convert data to string using specified charset
465
     * @param charset the character set
466
     * @return string representation of data
467
     */
468
    public String toString(Charset charset);
469
    
470
    /**
471
     * Trim array to exact size (frees unused capacity)
472
     */
473
    public void trim();
474
}
475
```
476

477
**Usage Examples:**
478

479
```java
480
import it.unimi.dsi.fastutil.io.*;
481

482
// Working with byte array input stream
483
byte[] data = "Hello, World! This is test data.".getBytes();
484
FastByteArrayInputStream input = new FastByteArrayInputStream(data);
485

486
// Direct array access
487
System.out.println("Array length: " + input.array.length);
488
System.out.println("Data length: " + input.length);
489

490
// Positioning and reading
491
input.position(7); // Move to "World!"
492
byte[] buffer = new byte[6];
493
int read = input.read(buffer);
494
System.out.println("Read: " + new String(buffer, 0, read)); // "World!"
495

496
// Mark and reset functionality
497
input.mark(0);
498
input.read(); // Read one byte
499
input.reset(); // Back to marked position
500

501
// Working with byte array output stream
502
FastByteArrayOutputStream output = new FastByteArrayOutputStream();
503
output.write("Hello".getBytes());
504
output.write(", ".getBytes());
505
output.write("World!".getBytes());
506

507
// Direct access to written data
508
System.out.println("Written " + output.length + " bytes");
509
System.out.println("Data: " + new String(output.array, 0, output.length));
510

511
// Get copy of data
512
byte[] result = output.toByteArray();
513

514
// Reset and reuse
515
output.reset();
516
output.write("New data".getBytes());
517
```
518

519
### Multi-Array and Cached Streams
520

521
Specialized streams for handling very large data and file caching scenarios.
522

523
```java { .api }
524
/**
525
 * Input stream backed by multiple byte arrays for very large data
526
 */
527
public class FastMultiByteArrayInputStream extends MeasurableInputStream implements RepositionableStream {
528
    /**
529
     * Create from array of byte arrays
530
     * @param a array of byte arrays
531
     */
532
    public FastMultiByteArrayInputStream(byte[][] a);
533
    
534
    /**
535
     * Create from array of byte arrays with specified offset and length
536
     * @param a array of byte arrays
537
     * @param offset starting offset
538
     * @param length total number of bytes
539
     */
540
    public FastMultiByteArrayInputStream(byte[][] a, long offset, long length);
541
    
542
    /**
543
     * Read a single byte
544
     * @return the byte value (0-255), or -1 if end of stream
545
     */
546
    public int read() throws IOException;
547
    
548
    /**
549
     * Read bytes into array
550
     * @param b destination array
551
     * @param offset starting offset in destination
552
     * @param length maximum bytes to read
553
     * @return number of bytes read, or -1 if end of stream
554
     */
555
    public int read(byte[] b, int offset, int length) throws IOException;
556
    
557
    /**
558
     * Skip bytes efficiently
559
     * @param n number of bytes to skip
560
     * @return number of bytes actually skipped
561
     */
562
    public long skip(long n) throws IOException;
563
    
564
    /**
565
     * Set stream position
566
     * @param newPosition the new position
567
     */
568
    public void position(long newPosition) throws IOException;
569
    
570
    /**
571
     * Get current stream position
572
     * @return the current position
573
     */
574
    public long position() throws IOException;
575
    
576
    /**
577
     * Get stream length
578
     * @return the stream length
579
     */
580
    public long length() throws IOException;
581
}
582

583
/**
584
 * File-cached input stream with inspection capabilities
585
 */
586
public class InspectableFileCachedInputStream extends InputStream {
587
    /**
588
     * Create from input stream with specified cache size
589
     * @param is the source input stream
590
     * @param bufferSize size of memory buffer before file caching
591
     */
592
    public InspectableFileCachedInputStream(InputStream is, int bufferSize);
593
    
594
    /**
595
     * Create from input stream with default cache size
596
     * @param is the source input stream
597
     */
598
    public InspectableFileCachedInputStream(InputStream is);
599
    
600
    /**
601
     * Read a single byte
602
     * @return the byte value (0-255), or -1 if end of stream
603
     */
604
    public int read() throws IOException;
605
    
606
    /**
607
     * Read bytes into array
608
     * @param b destination array
609
     * @param offset starting offset in destination
610
     * @param length maximum bytes to read
611
     * @return number of bytes read, or -1 if end of stream
612
     */
613
    public int read(byte[] b, int offset, int length) throws IOException;
614
    
615
    /**
616
     * Get current position in cached data
617
     * @return the current position
618
     */
619
    public long position();
620
    
621
    /**
622
     * Get total amount of data cached so far
623
     * @return number of bytes cached
624
     */
625
    public long size();
626
    
627
    /**
628
     * Inspect cached data at specified position
629
     * @param position position to inspect
630
     * @return byte value at position, or -1 if not yet cached
631
     */
632
    public int inspect(long position);
633
    
634
    /**
635
     * Close the stream and clean up cache files
636
     */
637
    public void close() throws IOException;
638
}
639
```
640

641
**Usage Examples:**
642

643
```java
644
import it.unimi.dsi.fastutil.io.*;
645

646
// Working with multi-array input for big data
647
byte[][] bigData = new byte[1000][]; // Array of arrays
648
for (int i = 0; i < bigData.length; i++) {
649
    bigData[i] = ("Segment " + i + " data\n").getBytes();
650
}
651

652
FastMultiByteArrayInputStream multiInput = new FastMultiByteArrayInputStream(bigData);
653
long totalSize = multiInput.length();
654
System.out.println("Total data size: " + totalSize);
655

656
// Position anywhere in the big data
657
multiInput.position(totalSize / 2);
658
byte[] sample = new byte[20];
659
multiInput.read(sample);
660
System.out.println("Middle sample: " + new String(sample));
661

662
// File-cached stream for large network downloads
663
try (InspectableFileCachedInputStream cached = 
664
     new InspectableFileCachedInputStream(
665
         new URL("http://example.com/largefile.dat").openStream())) {
666
    
667
    // Read data (automatically cached to file when memory buffer fills)
668
    byte[] buffer = new byte[8192];
669
    int totalRead = 0;
670
    int read;
671
    while ((read = cached.read(buffer)) != -1) {
672
        totalRead += read;
673
        if (totalRead % 1000000 == 0) {
674
            System.out.println("Read " + totalRead + " bytes, cached " + cached.size());
675
        }
676
    }
677
    
678
    // Inspect previously read data
679
    long pos = cached.position();
680
    if (pos > 1000) {
681
        int byte1000 = cached.inspect(1000);
682
        System.out.println("Byte at position 1000: " + byte1000);
683
    }
684
}
685
```
686

687
## Performance Features
688

689
### Key Optimizations
690

691
1. **Unsynchronized Operations**: All classes are designed for single-threaded access, eliminating synchronization overhead
692
2. **Direct Buffer Management**: Manual buffer management for optimal performance
693
3. **True Skipping**: `skip()` methods actually skip data instead of reading and discarding
694
4. **Efficient Repositioning**: Direct position manipulation where supported by underlying streams
695
5. **Memory Mapping**: Automatic detection and use of FileChannel for repositioning when available
696

697
### Usage Guidelines
698

699
1. **Thread Safety**: These classes are NOT thread-safe. Use external synchronization if needed
700
2. **Resource Management**: Always use try-with-resources for proper cleanup
701
3. **Buffer Sizing**: Choose buffer sizes appropriate for your access patterns
702
4. **File Operations**: FastBuffered streams automatically detect and use FileChannel optimizations
703
5. **Memory Constraints**: Use multi-array streams for data larger than single array limits