0
# Amazon S3 Service APIs
1
2
Amazon Simple Storage Service (S3) is an object storage service that offers scalability, data availability, security, and performance. The AWS SDK for Java provides comprehensive S3 APIs for bucket and object operations, access control, multipart uploads, and pre-signed URLs.
3
4
## Package
5
6
**Service Package**: `com.amazonaws.services.s3`
7
**Model Package**: `com.amazonaws.services.s3.model`
8
**Transfer API**: `com.amazonaws.services.s3.transfer`
9
**Maven Artifact**: `aws-java-sdk-s3`
10
11
## Client Creation
12
13
```java { .api }
14
package com.amazonaws.services.s3;
15
16
// S3 client builder
17
class AmazonS3ClientBuilder extends AwsSyncClientBuilder<AmazonS3ClientBuilder, AmazonS3> {
18
static AmazonS3ClientBuilder standard();
19
static AmazonS3 defaultClient();
20
AmazonS3ClientBuilder enablePathStyleAccess();
21
AmazonS3ClientBuilder disableChunkedEncoding();
22
AmazonS3ClientBuilder enableAccelerateMode();
23
AmazonS3ClientBuilder enablePayloadSigning();
24
AmazonS3ClientBuilder enableForceGlobalBucketAccess();
25
AmazonS3 build();
26
}
27
```
28
29
###
30
31
Usage
32
33
```java
34
import com.amazonaws.services.s3.AmazonS3;
35
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
36
import com.amazonaws.regions.Regions;
37
38
// Create default S3 client
39
AmazonS3 s3 = AmazonS3ClientBuilder.defaultClient();
40
41
// Create S3 client with specific region
42
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
43
.withRegion(Regions.US_WEST_2)
44
.build();
45
46
// Create S3 client with acceleration enabled
47
AmazonS3 s3 = AmazonS3ClientBuilder.standard()
48
.withRegion(Regions.US_EAST_1)
49
.enableAccelerateMode()
50
.build();
51
```
52
53
## Main Service Interface
54
55
```java { .api }
56
package com.amazonaws.services.s3;
57
58
// Main S3 interface (thread-safe)
59
interface AmazonS3 {
60
// Configuration
61
void setEndpoint(String endpoint);
62
void setRegion(Region region);
63
void setS3ClientOptions(S3ClientOptions clientOptions);
64
65
// Bucket operations
66
Bucket createBucket(String bucketName);
67
Bucket createBucket(CreateBucketRequest createBucketRequest);
68
void deleteBucket(String bucketName);
69
void deleteBucket(DeleteBucketRequest deleteBucketRequest);
70
List<Bucket> listBuckets();
71
boolean doesBucketExistV2(String bucketName);
72
HeadBucketResult headBucket(HeadBucketRequest headBucketRequest);
73
String getBucketLocation(String bucketName);
74
75
// Object operations
76
PutObjectResult putObject(String bucketName, String key, File file);
77
PutObjectResult putObject(String bucketName, String key, InputStream input,
78
ObjectMetadata metadata);
79
PutObjectResult putObject(PutObjectRequest putObjectRequest);
80
S3Object getObject(String bucketName, String key);
81
S3Object getObject(GetObjectRequest getObjectRequest);
82
ObjectMetadata getObject(GetObjectRequest getObjectRequest, File destinationFile);
83
String getObjectAsString(String bucketName, String key);
84
ObjectMetadata getObjectMetadata(String bucketName, String key);
85
void deleteObject(String bucketName, String key);
86
void deleteObject(DeleteObjectRequest deleteObjectRequest);
87
DeleteObjectsResult deleteObjects(DeleteObjectsRequest deleteObjectsRequest);
88
CopyObjectResult copyObject(String sourceBucketName, String sourceKey,
89
String destinationBucketName, String destinationKey);
90
CopyObjectResult copyObject(CopyObjectRequest copyObjectRequest);
91
boolean doesObjectExist(String bucketName, String objectName);
92
93
// Listing operations
94
ObjectListing listObjects(String bucketName);
95
ObjectListing listObjects(String bucketName, String prefix);
96
ObjectListing listObjects(ListObjectsRequest listObjectsRequest);
97
ObjectListing listNextBatchOfObjects(ObjectListing previousObjectListing);
98
ListObjectsV2Result listObjectsV2(String bucketName);
99
ListObjectsV2Result listObjectsV2(String bucketName, String prefix);
100
ListObjectsV2Result listObjectsV2(ListObjectsV2Request listObjectsV2Request);
101
102
// Access control
103
AccessControlList getBucketAcl(String bucketName);
104
void setBucketAcl(String bucketName, AccessControlList acl);
105
void setBucketAcl(String bucketName, CannedAccessControlList acl);
106
AccessControlList getObjectAcl(String bucketName, String key);
107
void setObjectAcl(String bucketName, String key, AccessControlList acl);
108
void setObjectAcl(String bucketName, String key, CannedAccessControlList acl);
109
110
// Multipart uploads
111
InitiateMultipartUploadResult initiateMultipartUpload(
112
InitiateMultipartUploadRequest request);
113
UploadPartResult uploadPart(UploadPartRequest request);
114
CompleteMultipartUploadResult completeMultipartUpload(
115
CompleteMultipartUploadRequest request);
116
void abortMultipartUpload(AbortMultipartUploadRequest request);
117
PartListing listParts(ListPartsRequest request);
118
MultipartUploadListing listMultipartUploads(ListMultipartUploadsRequest request);
119
120
// Pre-signed URLs
121
URL generatePresignedUrl(String bucketName, String key, Date expiration);
122
URL generatePresignedUrl(String bucketName, String key, Date expiration,
123
HttpMethod method);
124
URL generatePresignedUrl(GeneratePresignedUrlRequest generatePresignedUrlRequest);
125
126
// Waiters
127
AmazonS3Waiters waiters();
128
}
129
```
130
131
## Bucket Operations
132
133
### Creating and Deleting Buckets
134
135
```java
136
import com.amazonaws.services.s3.model.Bucket;
137
import com.amazonaws.services.s3.model.CreateBucketRequest;
138
139
// Create bucket in default region
140
Bucket bucket = s3.createBucket("my-bucket");
141
142
// Create bucket in specific region
143
Bucket bucket = s3.createBucket(new CreateBucketRequest("my-bucket", "us-west-2"));
144
145
// Check if bucket exists
146
boolean exists = s3.doesBucketExistV2("my-bucket");
147
148
// Get bucket location
149
String location = s3.getBucketLocation("my-bucket");
150
151
// List all buckets
152
List<Bucket> buckets = s3.listBuckets();
153
for (Bucket b : buckets) {
154
System.out.println(b.getName() + " - " + b.getCreationDate());
155
}
156
157
// Delete bucket (must be empty)
158
s3.deleteBucket("my-bucket");
159
```
160
161
### Bucket Configuration
162
163
```java { .api }
164
package com.amazonaws.services.s3;
165
166
interface AmazonS3 {
167
// Versioning
168
BucketVersioningConfiguration getBucketVersioningConfiguration(String bucketName);
169
void setBucketVersioningConfiguration(
170
SetBucketVersioningConfigurationRequest setBucketVersioningConfigurationRequest);
171
172
// Lifecycle
173
BucketLifecycleConfiguration getBucketLifecycleConfiguration(String bucketName);
174
void setBucketLifecycleConfiguration(String bucketName,
175
BucketLifecycleConfiguration configuration);
176
void deleteBucketLifecycleConfiguration(String bucketName);
177
178
// CORS
179
BucketCrossOriginConfiguration getBucketCrossOriginConfiguration(String bucketName);
180
void setBucketCrossOriginConfiguration(String bucketName,
181
BucketCrossOriginConfiguration configuration);
182
void deleteBucketCrossOriginConfiguration(String bucketName);
183
184
// Tagging
185
BucketTaggingConfiguration getBucketTaggingConfiguration(String bucketName);
186
void setBucketTaggingConfiguration(String bucketName,
187
BucketTaggingConfiguration bucketTaggingConfiguration);
188
void deleteBucketTaggingConfiguration(String bucketName);
189
190
// Encryption
191
GetBucketEncryptionResult getBucketEncryption(GetBucketEncryptionRequest request);
192
SetBucketEncryptionResult setBucketEncryption(SetBucketEncryptionRequest request);
193
DeleteBucketEncryptionResult deleteBucketEncryption(DeleteBucketEncryptionRequest request);
194
195
// Policies
196
BucketPolicy getBucketPolicy(String bucketName);
197
void setBucketPolicy(String bucketName, String policyText);
198
void deleteBucketPolicy(String bucketName);
199
}
200
```
201
202
## Object Operations
203
204
### Upload Objects
205
206
```java { .api }
207
package com.amazonaws.services.s3.model;
208
209
// Put object request
210
class PutObjectRequest extends AmazonWebServiceRequest {
211
PutObjectRequest(String bucketName, String key, File file);
212
PutObjectRequest(String bucketName, String key, InputStream input,
213
ObjectMetadata metadata);
214
PutObjectRequest(String bucketName, String key, String content);
215
216
PutObjectRequest withCannedAcl(CannedAccessControlList cannedAcl);
217
PutObjectRequest withStorageClass(StorageClass storageClass);
218
PutObjectRequest withMetadata(ObjectMetadata metadata);
219
PutObjectRequest withTagging(ObjectTagging tagging);
220
PutObjectRequest withSSEAwsKeyManagementParams(SSEAwsKeyManagementParams sseParams);
221
PutObjectRequest withSSECustomerKey(SSECustomerKey sseKey);
222
}
223
224
// Put object result
225
class PutObjectResult {
226
String getETag();
227
String getVersionId();
228
String getContentMd5();
229
ObjectMetadata getMetadata();
230
Date getExpirationTime();
231
String getExpirationTimeRuleId();
232
}
233
```
234
235
### Upload Usage
236
237
```java
238
import com.amazonaws.services.s3.model.*;
239
import java.io.File;
240
import java.io.InputStream;
241
242
// Upload file
243
PutObjectResult result = s3.putObject("my-bucket", "my-key", new File("/path/to/file"));
244
245
// Upload from InputStream with metadata
246
ObjectMetadata metadata = new ObjectMetadata();
247
metadata.setContentType("text/plain");
248
metadata.setContentLength(data.length);
249
metadata.addUserMetadata("title", "My Document");
250
251
InputStream inputStream = new ByteArrayInputStream(data);
252
PutObjectResult result = s3.putObject("my-bucket", "my-key", inputStream, metadata);
253
254
// Upload with access control
255
PutObjectRequest request = new PutObjectRequest("my-bucket", "my-key", file)
256
.withCannedAcl(CannedAccessControlList.PublicRead)
257
.withStorageClass(StorageClass.StandardInfrequentAccess);
258
s3.putObject(request);
259
260
// Upload with server-side encryption (SSE-S3)
261
ObjectMetadata metadata = new ObjectMetadata();
262
metadata.setSSEAlgorithm(ObjectMetadata.AES_256_SERVER_SIDE_ENCRYPTION);
263
s3.putObject(new PutObjectRequest("my-bucket", "my-key", file).withMetadata(metadata));
264
265
// Upload with server-side encryption (SSE-KMS)
266
s3.putObject(new PutObjectRequest("my-bucket", "my-key", file)
267
.withSSEAwsKeyManagementParams(new SSEAwsKeyManagementParams("kms-key-id")));
268
```
269
270
### Download Objects
271
272
```java { .api }
273
package com.amazonaws.services.s3.model;
274
275
// Get object request
276
class GetObjectRequest extends AmazonWebServiceRequest {
277
GetObjectRequest(String bucketName, String key);
278
GetObjectRequest(String bucketName, String key, String versionId);
279
280
GetObjectRequest withVersionId(String versionId);
281
GetObjectRequest withRange(long start, long end);
282
GetObjectRequest withSSECustomerKey(SSECustomerKey sseKey);
283
GetObjectRequest withResponseHeaders(ResponseHeaderOverrides responseHeaders);
284
}
285
286
// S3 object (implements Closeable)
287
class S3Object implements Closeable {
288
String getBucketName();
289
String getKey();
290
ObjectMetadata getObjectMetadata();
291
S3ObjectInputStream getObjectContent();
292
void setObjectContent(S3ObjectInputStream objectContent);
293
void close();
294
}
295
296
// Object metadata
297
class ObjectMetadata {
298
long getContentLength();
299
String getContentType();
300
String getContentEncoding();
301
String getContentDisposition();
302
String getETag();
303
Date getLastModified();
304
Map<String, String> getUserMetadata();
305
void addUserMetadata(String key, String value);
306
void setContentLength(long contentLength);
307
void setContentType(String contentType);
308
}
309
```
310
311
### Download Usage
312
313
```java
314
import com.amazonaws.services.s3.model.*;
315
import java.io.InputStream;
316
import java.io.OutputStream;
317
import java.io.FileOutputStream;
318
319
// Download object to memory
320
S3Object object = s3.getObject("my-bucket", "my-key");
321
try (InputStream content = object.getObjectContent()) {
322
// Process content
323
byte[] buffer = new byte[1024];
324
int bytesRead;
325
while ((bytesRead = content.read(buffer)) != -1) {
326
// Process bytes
327
}
328
} // Auto-close the stream
329
330
// Download object to file
331
ObjectMetadata metadata = s3.getObject(
332
new GetObjectRequest("my-bucket", "my-key"),
333
new File("/path/to/destination")
334
);
335
336
// Download as string (small objects only)
337
String content = s3.getObjectAsString("my-bucket", "my-key");
338
339
// Download object metadata only (no data transfer)
340
ObjectMetadata metadata = s3.getObjectMetadata("my-bucket", "my-key");
341
System.out.println("Size: " + metadata.getContentLength());
342
System.out.println("Type: " + metadata.getContentType());
343
System.out.println("ETag: " + metadata.getETag());
344
345
// Download specific byte range
346
GetObjectRequest request = new GetObjectRequest("my-bucket", "my-key")
347
.withRange(0, 1023); // First 1KB
348
S3Object object = s3.getObject(request);
349
350
// Download specific version
351
S3Object object = s3.getObject("my-bucket", "my-key", "version-id");
352
```
353
354
### Delete and Copy Objects
355
356
```java { .api }
357
package com.amazonaws.services.s3.model;
358
359
// Delete single object
360
class DeleteObjectRequest extends AmazonWebServiceRequest {
361
DeleteObjectRequest(String bucketName, String key);
362
DeleteObjectRequest withBypassGovernanceRetention(boolean bypassGovernanceRetention);
363
}
364
365
// Batch delete request
366
class DeleteObjectsRequest extends AmazonWebServiceRequest {
367
DeleteObjectsRequest(String bucketName);
368
DeleteObjectsRequest withKeys(String... keys);
369
DeleteObjectsRequest withQuiet(boolean quiet);
370
void setKeys(List<KeyVersion> keys);
371
372
static class KeyVersion {
373
KeyVersion(String key);
374
KeyVersion(String key, String version);
375
}
376
}
377
378
// Delete result
379
class DeleteObjectsResult {
380
List<DeletedObject> getDeletedObjects();
381
List<DeleteError> getErrors();
382
}
383
384
// Copy object request
385
class CopyObjectRequest extends AmazonWebServiceRequest {
386
CopyObjectRequest(String sourceBucketName, String sourceKey,
387
String destinationBucketName, String destinationKey);
388
CopyObjectRequest withNewObjectMetadata(ObjectMetadata newObjectMetadata);
389
CopyObjectRequest withStorageClass(StorageClass storageClass);
390
CopyObjectRequest withCannedAccessControlList(CannedAccessControlList cannedACL);
391
}
392
393
// Copy object result
394
class CopyObjectResult {
395
String getETag();
396
Date getLastModifiedDate();
397
String getVersionId();
398
}
399
```
400
401
### Delete and Copy Usage
402
403
```java
404
import com.amazonaws.services.s3.model.*;
405
406
// Delete single object
407
s3.deleteObject("my-bucket", "my-key");
408
409
// Batch delete (up to 1000 objects)
410
DeleteObjectsRequest deleteRequest = new DeleteObjectsRequest("my-bucket");
411
List<KeyVersion> keys = new ArrayList<>();
412
keys.add(new KeyVersion("key1"));
413
keys.add(new KeyVersion("key2"));
414
keys.add(new KeyVersion("key3", "version-id"));
415
deleteRequest.setKeys(keys);
416
417
DeleteObjectsResult result = s3.deleteObjects(deleteRequest);
418
System.out.println("Deleted: " + result.getDeletedObjects().size());
419
420
// Copy object within S3
421
CopyObjectResult result = s3.copyObject(
422
"source-bucket", "source-key",
423
"dest-bucket", "dest-key"
424
);
425
426
// Copy with metadata replacement
427
ObjectMetadata newMetadata = new ObjectMetadata();
428
newMetadata.setContentType("application/json");
429
430
CopyObjectRequest copyRequest = new CopyObjectRequest(
431
"source-bucket", "source-key",
432
"dest-bucket", "dest-key"
433
).withNewObjectMetadata(newMetadata)
434
.withStorageClass(StorageClass.StandardInfrequentAccess);
435
436
s3.copyObject(copyRequest);
437
```
438
439
## Listing Objects
440
441
```java { .api }
442
package com.amazonaws.services.s3.model;
443
444
// List objects request (V1)
445
class ListObjectsRequest extends AmazonWebServiceRequest {
446
ListObjectsRequest();
447
ListObjectsRequest withBucketName(String bucketName);
448
ListObjectsRequest withPrefix(String prefix);
449
ListObjectsRequest withDelimiter(String delimiter);
450
ListObjectsRequest withMarker(String marker);
451
ListObjectsRequest withMaxKeys(Integer maxKeys);
452
}
453
454
// List objects result (V1)
455
class ObjectListing {
456
List<S3ObjectSummary> getObjectSummaries();
457
List<String> getCommonPrefixes();
458
boolean isTruncated();
459
String getNextMarker();
460
}
461
462
// List objects V2 request (RECOMMENDED)
463
class ListObjectsV2Request extends AmazonWebServiceRequest {
464
ListObjectsV2Request();
465
ListObjectsV2Request withBucketName(String bucketName);
466
ListObjectsV2Request withPrefix(String prefix);
467
ListObjectsV2Request withDelimiter(String delimiter);
468
ListObjectsV2Request withContinuationToken(String continuationToken);
469
ListObjectsV2Request withStartAfter(String startAfter);
470
ListObjectsV2Request withMaxKeys(Integer maxKeys);
471
}
472
473
// List objects V2 result
474
class ListObjectsV2Result {
475
List<S3ObjectSummary> getObjectSummaries();
476
List<String> getCommonPrefixes();
477
boolean isTruncated();
478
String getContinuationToken();
479
String getNextContinuationToken();
480
}
481
482
// Object summary
483
class S3ObjectSummary {
484
String getBucketName();
485
String getKey();
486
long getSize();
487
Date getLastModified();
488
String getETag();
489
String getStorageClass();
490
Owner getOwner();
491
}
492
```
493
494
### Listing Usage
495
496
```java
497
import com.amazonaws.services.s3.model.*;
498
499
// Simple list (V1 API)
500
ObjectListing listing = s3.listObjects("my-bucket");
501
for (S3ObjectSummary summary : listing.getObjectSummaries()) {
502
System.out.println(summary.getKey() + " - " + summary.getSize());
503
}
504
505
// List with prefix and pagination (V1)
506
ObjectListing listing = s3.listObjects("my-bucket", "photos/");
507
do {
508
for (S3ObjectSummary summary : listing.getObjectSummaries()) {
509
System.out.println(summary.getKey());
510
}
511
listing = s3.listNextBatchOfObjects(listing);
512
} while (listing.isTruncated());
513
514
// List with V2 API (recommended)
515
ListObjectsV2Request request = new ListObjectsV2Request()
516
.withBucketName("my-bucket")
517
.withPrefix("documents/")
518
.withMaxKeys(1000);
519
520
ListObjectsV2Result result;
521
do {
522
result = s3.listObjectsV2(request);
523
for (S3ObjectSummary summary : result.getObjectSummaries()) {
524
System.out.printf("%s (size: %d bytes)%n",
525
summary.getKey(), summary.getSize());
526
}
527
request.setContinuationToken(result.getNextContinuationToken());
528
} while (result.isTruncated());
529
530
// Hierarchical listing with delimiter
531
ListObjectsV2Request request = new ListObjectsV2Request()
532
.withBucketName("my-bucket")
533
.withPrefix("photos/")
534
.withDelimiter("/");
535
536
ListObjectsV2Result result = s3.listObjectsV2(request);
537
538
// Files in current level
539
for (S3ObjectSummary summary : result.getObjectSummaries()) {
540
System.out.println("File: " + summary.getKey());
541
}
542
543
// Subdirectories
544
for (String prefix : result.getCommonPrefixes()) {
545
System.out.println("Directory: " + prefix);
546
}
547
```
548
549
## Access Control
550
551
```java { .api }
552
package com.amazonaws.services.s3.model;
553
554
// Access Control List
555
class AccessControlList {
556
AccessControlList();
557
void grantPermission(Grantee grantee, Permission permission);
558
List<Grant> getGrantsAsList();
559
Owner getOwner();
560
void setOwner(Owner owner);
561
}
562
563
// Canned ACL (pre-defined policies)
564
enum CannedAccessControlList {
565
Private, // Owner gets FULL_CONTROL, no one else has access
566
PublicRead, // Owner gets FULL_CONTROL, everyone gets READ
567
PublicReadWrite, // Owner gets FULL_CONTROL, everyone gets READ and WRITE
568
AuthenticatedRead, // Owner gets FULL_CONTROL, authenticated users get READ
569
LogDeliveryWrite, // For S3 log delivery
570
BucketOwnerRead, // Object owner gets FULL_CONTROL, bucket owner gets READ
571
BucketOwnerFullControl, // Both object and bucket owner get FULL_CONTROL
572
AwsExecRead // For Amazon EC2 to GET AMI bundle
573
}
574
575
// Grant
576
class Grant {
577
Grant(Grantee grantee, Permission permission);
578
Grantee getGrantee();
579
Permission getPermission();
580
}
581
582
// Permission enum
583
enum Permission {
584
FullControl,
585
Read,
586
Write,
587
ReadAcp,
588
WriteAcp
589
}
590
```
591
592
### Access Control Usage
593
594
```java
595
import com.amazonaws.services.s3.model.*;
596
597
// Set canned ACL on object
598
s3.setObjectAcl("my-bucket", "my-key", CannedAccessControlList.PublicRead);
599
600
// Upload with canned ACL
601
PutObjectRequest request = new PutObjectRequest("my-bucket", "my-key", file)
602
.withCannedAcl(CannedAccessControlList.PublicRead);
603
s3.putObject(request);
604
605
// Get object ACL
606
AccessControlList acl = s3.getObjectAcl("my-bucket", "my-key");
607
608
// Modify ACL to grant permission
609
acl.grantPermission(new CanonicalGrantee("user-id"), Permission.Read);
610
s3.setObjectAcl("my-bucket", "my-key", acl);
611
612
// Set bucket ACL
613
s3.setBucketAcl("my-bucket", CannedAccessControlList.Private);
614
```
615
616
## Multipart Upload
617
618
```java { .api }
619
package com.amazonaws.services.s3.model;
620
621
// Initiate multipart upload
622
class InitiateMultipartUploadRequest extends AmazonWebServiceRequest {
623
InitiateMultipartUploadRequest(String bucketName, String key);
624
InitiateMultipartUploadRequest withObjectMetadata(ObjectMetadata objectMetadata);
625
InitiateMultipartUploadRequest withCannedACL(CannedAccessControlList acl);
626
InitiateMultipartUploadRequest withStorageClass(StorageClass storageClass);
627
}
628
629
class InitiateMultipartUploadResult {
630
String getBucketName();
631
String getKey();
632
String getUploadId();
633
}
634
635
// Upload part
636
class UploadPartRequest extends AmazonWebServiceRequest {
637
UploadPartRequest withBucketName(String bucketName);
638
UploadPartRequest withKey(String key);
639
UploadPartRequest withUploadId(String uploadId);
640
UploadPartRequest withPartNumber(int partNumber);
641
UploadPartRequest withFile(File file);
642
UploadPartRequest withFileOffset(long fileOffset);
643
UploadPartRequest withPartSize(long partSize);
644
UploadPartRequest withInputStream(InputStream inputStream);
645
}
646
647
class UploadPartResult {
648
int getPartNumber();
649
String getETag();
650
PartETag getPartETag();
651
}
652
653
// Complete multipart upload
654
class CompleteMultipartUploadRequest extends AmazonWebServiceRequest {
655
CompleteMultipartUploadRequest(String bucketName, String key,
656
String uploadId, List<PartETag> partETags);
657
}
658
659
class CompleteMultipartUploadResult {
660
String getLocation();
661
String getBucketName();
662
String getKey();
663
String getETag();
664
String getVersionId();
665
}
666
667
// Abort multipart upload
668
class AbortMultipartUploadRequest extends AmazonWebServiceRequest {
669
AbortMultipartUploadRequest(String bucketName, String key, String uploadId);
670
}
671
672
// Part ETag
673
class PartETag {
674
PartETag(int partNumber, String eTag);
675
int getPartNumber();
676
String getETag();
677
}
678
```
679
680
### Multipart Upload Usage
681
682
```java
683
import com.amazonaws.services.s3.model.*;
684
import java.io.File;
685
import java.util.ArrayList;
686
import java.util.List;
687
688
File file = new File("/path/to/large-file");
689
long partSize = 5 * 1024 * 1024; // 5 MB
690
691
// 1. Initiate multipart upload
692
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest(
693
"my-bucket", "large-file");
694
InitiateMultipartUploadResult initResult = s3.initiateMultipartUpload(initRequest);
695
String uploadId = initResult.getUploadId();
696
697
try {
698
// 2. Upload parts
699
List<PartETag> partETags = new ArrayList<>();
700
long filePosition = 0;
701
int partNumber = 1;
702
703
while (filePosition < file.length()) {
704
partSize = Math.min(partSize, file.length() - filePosition);
705
706
UploadPartRequest uploadRequest = new UploadPartRequest()
707
.withBucketName("my-bucket")
708
.withKey("large-file")
709
.withUploadId(uploadId)
710
.withPartNumber(partNumber++)
711
.withFileOffset(filePosition)
712
.withFile(file)
713
.withPartSize(partSize);
714
715
UploadPartResult uploadResult = s3.uploadPart(uploadRequest);
716
partETags.add(uploadResult.getPartETag());
717
718
filePosition += partSize;
719
}
720
721
// 3. Complete multipart upload
722
CompleteMultipartUploadRequest completeRequest = new CompleteMultipartUploadRequest(
723
"my-bucket", "large-file", uploadId, partETags);
724
CompleteMultipartUploadResult completeResult = s3.completeMultipartUpload(completeRequest);
725
726
System.out.println("Upload complete: " + completeResult.getLocation());
727
728
} catch (Exception e) {
729
// 4. Abort on failure
730
s3.abortMultipartUpload(new AbortMultipartUploadRequest(
731
"my-bucket", "large-file", uploadId));
732
throw e;
733
}
734
```
735
736
## Pre-signed URLs
737
738
```java { .api }
739
package com.amazonaws.services.s3.model;
740
741
// Generate pre-signed URL request
742
class GeneratePresignedUrlRequest {
743
GeneratePresignedUrlRequest(String bucketName, String key);
744
GeneratePresignedUrlRequest(String bucketName, String key, HttpMethod method);
745
746
GeneratePresignedUrlRequest withMethod(HttpMethod method);
747
GeneratePresignedUrlRequest withExpiration(Date expiration);
748
GeneratePresignedUrlRequest withContentType(String contentType);
749
GeneratePresignedUrlRequest withContentMd5(String contentMd5);
750
GeneratePresignedUrlRequest withVersionId(String versionId);
751
GeneratePresignedUrlRequest withSSECustomerKey(SSECustomerKey sseCustomerKey);
752
GeneratePresignedUrlRequest withResponseHeaders(ResponseHeaderOverrides responseHeaders);
753
}
754
```
755
756
### Pre-signed URL Usage
757
758
```java
759
import com.amazonaws.HttpMethod;
760
import com.amazonaws.services.s3.model.GeneratePresignedUrlRequest;
761
import java.net.URL;
762
import java.util.Date;
763
764
// Generate pre-signed URL for GET (download) - valid for 1 hour
765
Date expiration = new Date();
766
expiration.setTime(expiration.getTime() + 3600 * 1000);
767
768
URL url = s3.generatePresignedUrl("my-bucket", "my-key", expiration);
769
System.out.println("Pre-signed URL: " + url.toString());
770
771
// Generate pre-signed URL for PUT (upload)
772
GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(
773
"my-bucket", "my-key", HttpMethod.PUT)
774
.withExpiration(expiration)
775
.withContentType("image/jpeg");
776
777
URL uploadUrl = s3.generatePresignedUrl(request);
778
779
// Users can now upload by sending PUT request to this URL
780
781
// Generate pre-signed URL with custom response headers
782
ResponseHeaderOverrides responseHeaders = new ResponseHeaderOverrides()
783
.withContentDisposition("attachment; filename=\"download.txt\"")
784
.withContentType("text/plain");
785
786
GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(
787
"my-bucket", "my-key")
788
.withExpiration(expiration)
789
.withResponseHeaders(responseHeaders);
790
791
URL url = s3.generatePresignedUrl(request);
792
```
793
794
## Transfer Manager (High-Level API)
795
796
```java { .api }
797
package com.amazonaws.services.s3.transfer;
798
799
// Transfer Manager builder
800
class TransferManagerBuilder {
801
static TransferManagerBuilder standard();
802
TransferManagerBuilder withS3Client(AmazonS3 s3);
803
TransferManagerBuilder withExecutorFactory(ExecutorFactory executorFactory);
804
TransferManagerBuilder withMinimumUploadPartSize(Long minimumUploadPartSize);
805
TransferManagerBuilder withMultipartUploadThreshold(Long multipartUploadThreshold);
806
TransferManagerBuilder withMultipartCopyThreshold(Long multipartCopyThreshold);
807
TransferManagerBuilder withMultipartCopyPartSize(Long multipartCopyPartSize);
808
TransferManager build();
809
}
810
811
// Transfer Manager
812
class TransferManager {
813
// Upload operations
814
Upload upload(String bucketName, String key, File file);
815
Upload upload(String bucketName, String key, InputStream input, ObjectMetadata metadata);
816
Upload upload(PutObjectRequest putObjectRequest);
817
MultipleFileUpload uploadDirectory(String bucketName, String virtualDirectoryKeyPrefix,
818
File directory, boolean includeSubdirectories);
819
MultipleFileUpload uploadFileList(String bucketName, String virtualDirectoryKeyPrefix,
820
File directory, List<File> files);
821
822
// Download operations
823
Download download(String bucket, String key, File file);
824
Download download(GetObjectRequest getObjectRequest, File file);
825
MultipleFileDownload downloadDirectory(String bucketName, String keyPrefix,
826
File destinationDirectory);
827
828
// Copy operations
829
Copy copy(String sourceBucketName, String sourceKey,
830
String destinationBucketName, String destinationKey);
831
Copy copy(CopyObjectRequest copyObjectRequest);
832
833
// Pause and resume
834
PersistableUpload pauseUpload(Upload upload);
835
Upload resumeUpload(PersistableUpload persistableUpload);
836
PersistableDownload pauseDownload(Download download);
837
Download resumeDownload(PersistableDownload persistableDownload);
838
839
// Shutdown
840
void shutdownNow();
841
void shutdownNow(boolean shutDownS3Client);
842
}
843
844
// Transfer objects
845
interface Transfer {
846
boolean isDone();
847
void waitForCompletion() throws InterruptedException;
848
TransferProgress getProgress();
849
void abort();
850
}
851
852
class Upload implements Transfer {
853
PutObjectResult waitForUploadResult() throws InterruptedException;
854
PersistableUpload pause();
855
}
856
857
class Download implements Transfer {
858
ObjectMetadata getObjectMetadata();
859
PersistableDownload pause();
860
}
861
862
class MultipleFileUpload implements Transfer {
863
Collection<? extends Upload> getSubTransfers();
864
}
865
866
class TransferProgress {
867
long getBytesTransferred();
868
long getTotalBytesToTransfer();
869
double getPercentTransferred();
870
}
871
```
872
873
### Transfer Manager Usage
874
875
```java
876
import com.amazonaws.services.s3.transfer.*;
877
import java.io.File;
878
879
// Create Transfer Manager
880
TransferManager tm = TransferManagerBuilder.standard()
881
.withS3Client(s3)
882
.build();
883
884
try {
885
// Upload file with automatic multipart upload
886
Upload upload = tm.upload("my-bucket", "large-file", new File("/path/to/large-file"));
887
888
// Monitor progress
889
while (!upload.isDone()) {
890
System.out.printf("Progress: %.2f%%%n",
891
upload.getProgress().getPercentTransferred());
892
Thread.sleep(1000);
893
}
894
895
// Wait for completion
896
upload.waitForCompletion();
897
System.out.println("Upload complete");
898
899
// Download file with automatic multipart download
900
Download download = tm.download("my-bucket", "large-file",
901
new File("/path/to/destination"));
902
download.waitForCompletion();
903
904
// Upload entire directory
905
MultipleFileUpload dirUpload = tm.uploadDirectory(
906
"my-bucket",
907
"backup/",
908
new File("/path/to/directory"),
909
true // Include subdirectories
910
);
911
dirUpload.waitForCompletion();
912
913
// Download entire directory
914
MultipleFileDownload dirDownload = tm.downloadDirectory(
915
"my-bucket",
916
"backup/",
917
new File("/path/to/destination")
918
);
919
dirDownload.waitForCompletion();
920
921
// Pause and resume upload
922
Upload upload = tm.upload("my-bucket", "file", new File("/path/to/file"));
923
Thread.sleep(2000);
924
PersistableUpload persistableUpload = upload.pause();
925
926
// Save persistableUpload.serialize() to file/database
927
928
// Later, resume the upload
929
Upload resumedUpload = tm.resumeUpload(persistableUpload);
930
resumedUpload.waitForCompletion();
931
932
} finally {
933
// Shutdown Transfer Manager
934
tm.shutdownNow();
935
}
936
```
937
938
## Storage Classes
939
940
```java { .api }
941
package com.amazonaws.services.s3.model;
942
943
// Storage class enum
944
enum StorageClass {
945
Standard, // Frequently accessed data
946
ReducedRedundancy, // Non-critical, reproducible data (deprecated)
947
Glacier, // Long-term archive
948
StandardInfrequentAccess, // STANDARD_IA - Infrequently accessed data
949
OneZoneInfrequentAccess, // ONEZONE_IA - Infrequently accessed, single AZ
950
IntelligentTiering, // Automatic cost optimization
951
DeepArchive, // DEEP_ARCHIVE - Lowest cost archive
952
GlacierInstantRetrieval, // GLACIER_IR - Archive with instant retrieval
953
Outposts, // S3 on Outposts
954
Snow // AWS Snowball Edge
955
}
956
```
957
958
## Best Practices
959
960
### Object Operations
961
- **Always close S3Object streams**: Use try-with-resources to prevent connection pool exhaustion
962
- **Use Transfer Manager for large files**: Automatically handles multipart uploads (>5MB recommended)
963
- **Set appropriate metadata**: Content-Type, Content-Encoding, etc.
964
- **Use versioning**: Enable versioning for important buckets to protect against accidental deletion
965
966
### Performance
967
- **Reuse client instances**: AmazonS3 client is thread-safe and expensive to create
968
- **Use multipart uploads**: For files > 100MB
969
- **Consider Transfer Acceleration**: For global uploads/downloads
970
- **Batch operations**: Use deleteObjects for batch deletion (up to 1000 objects)
971
972
### Security
973
- **Use IAM roles**: For EC2/ECS/Lambda instead of embedding credentials
974
- **Enable encryption**: Use SSE-S3, SSE-KMS, or SSE-C
975
- **Use pre-signed URLs**: For temporary access without exposing credentials
976
- **Implement bucket policies**: Control access at bucket level
977
- **Enable MFA Delete**: For versioned buckets containing critical data
978
979
### Cost Optimization
980
- **Choose appropriate storage class**: Use Intelligent-Tiering or lifecycle policies
981
- **Enable lifecycle policies**: Transition to cheaper storage or delete old objects
982
- **Use S3 Select**: Retrieve only needed data instead of entire objects
983
- **Delete incomplete multipart uploads**: Set lifecycle policy to abort incomplete uploads after 7 days
984