AWS S3 Java SDK - Downloading an encrypted GetObject by Stream of a Selling Partner (SP) API url

I started using SellingPartner (SP) recently and I am kind confused how they provide us S3 reports to download.

When I fetch a Report Document from SP API I get this return (omitted):

GetReportDocumentResponse class:
{
  "payload": {
    "reportDocumentId": "amzn1.tortuga.3.OMITTED.OMITTED",
    "url": "https://tortuga-prod-na.s3-external-1.amazonaws.com/%2FOMITED/amzn1.tortuga.3.OMITTED.OMITTED?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20201025T163212Z&X-Amz-SignedHeaders=host&X-Amz-Expires=300&X-Amz-Credential=OMITED%2F20201025%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=OMITED",
    "encryptionDetails": {
      "standard": "AES",
      "initializationVector": "OMITED==",
      "key": "+OMITED="
    },
    "compressionAlgorithm": null
  },
  "errors": null
}

If I copy/paste the payload.url directly in my browser, it downloads an encrypted document, which looks fine (I couldn't decrypt it though, snippet in the end).

I am trying to download using the AWS S3 Java SDK and I keep getting software.amazon.awssdk.services.s3.model.S3Exception: Access Denied

I have this snippet:

public String getReportFile(String reportDocumentId) throws IOException {
    GetReportDocumentResponse response = getReport(reportDocumentId);
    ReportDocumentEncryptionDetails encryptionDetails = response.getPayload().getEncryptionDetails();
    
    GetObjectRequest request =
        GetObjectRequest.builder()
            .key(reportDocumentId)
            .bucket("tortuga-prod-na") //hardcoding here, thats the bucket on the URL, right?
            .sseCustomerAlgorithm(encryptionDetails.getStandard())
            .sseCustomerKey(encryptionDetails.getKey())
            // .sseCustomerKeyMD5() should I apply it? Is that the Initialization Vector field?
            .build();
    
//I tried both without Credentials, and using accessKey and secretKey from my personal account, not sure if should be another one related to the URL, what should I use for credentials if the URL works fine in my browser?
    StaticCredentialsProvider credentialsProvider =
        StaticCredentialsProvider.create(AwsBasicCredentials.create(accessKey, secretKey));
    BufferedReader br =
        new BufferedReader(
            new InputStreamReader(
                S3Client.builder()
                    .credentialsProvider(credentialsProvider)
                    .region(Region.US_EAST_1)
                    .build()
                    .getObject(request)));
    

My end goal is to download this file in chunks (as it may have over 500mb) and process a few hundred lines at a time. Would that be possible if it's encrypted? I would like to download it already decrypted and be able to process it in chunks.

I wonder how to make the same request using S3Client like the URL coming from the JSON. Do we have a way to just paste a URL on S3Client, include the encryption settings and make a call?

About the downloaded file from the browser, I tried to decrypt it doing this:

byte[] bytes = FileUtils.readFileToByteArray(new File("encrypted_file"));

Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5PADDING");
SecretKeySpec secretKey = new SecretKeySpec(Base64.getDecoder().decode(<payload.encryptionDetails.key String value>), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey);
System.out.println(new String(cipher.doFinal(bytes)));

which throws exception:
Exception in thread "main" javax.crypto.BadPaddingException: Given final block not properly padded. Such issues can arise if a bad key is used during decryption.

Thanks in advance.

> If I copy/paste the payload.url directly in my browser, it downloads an encrypted document, which looks fine (I couldn't decrypt it though, snippet in the end).

It means the object is not using SSE-C, otherwise you wouldn't be able to download it. Seems the content is encrypted in the client side (outside s3 api) and the encrypted content is uploaded as a normal object. So please check in your code, where the content is uploaded, for encryption parameters .

If you are able to directly download the object from the browser, then just download the content as a normal object.

Note: proper use of the AWS S3 Client Side Encryption

> About the downloaded file from the browser, I tried to decrypt it doing this

wherever you got this code, please do not use it. Just using the AES/ECB/PKCS5PADDING mode is not safe.

Using the IV (initializationVector) implies using different encryption mode of operation. You have to find out which is it is from the code or service which encrypts the content.

private static final String SYMMETRIC_KEY_ALG = "AES";
// find out the correct value, could be AES/CBC/PKCS5Padding
private static final String SYMMETRIC_CIPHER_NAME = "???";

IvParameterSpec ivParamSpec = new IvParameterSpec(encryptionParams.getIv());
SecretKey symmetricKey = new SecretKeySpec(encryptionParams.getKey(), SYMMETRIC_KEY_ALG);

Cipher cipher = Cipher.getInstance(SYMMETRIC_CIPHER_NAME);
cipher.init(Cipher.DECRYPT_MODE, symmetricKey, ivParamSpec);

byte[] decrypted = cipher.doFinal(encryptionParams.getCiphertext());

However - you really need to find out how the content is encrypted. It could be possible the aes-gcm mode is used and then part of the ciphertext can be an authentication hash. So here you should not make assumptions and find the real data.