AWS S3响应失败,尽管文件已完全上传。

huangapple go评论82阅读模式
英文:

AWS S3 response FAILED, even though file is fully uploaded

问题

I'm just trying to upload an image to S3 via an Android app. What I concluded so far is that images that are bigger than 5MB are uploaded without a problem, however, images that are smaller than 5MB are FULLY UPLOADED and can be downloaded from S3 manually, however, the response i get is FAILED. I just want to remind you that 5MB is the bottom threshold for issuing a multipart request (SDK uploads using multipart automatically when the file is above 5MB, I checked this in their code).

I also checked the md5 sum by hand, and the sums from two files (uploaded and selected for upload) MATCH.

I tested uploading small images using a AWS S3 CLI and it works without a problem.

This is the exception I get:

AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash didn't match hash calculated by Amazon S3. You may need to delete the data stored in Amazon S3.

These are the libraries I am using:

com.amazonaws:aws-android-sdk-s3:$version

com.amazonaws:aws-android-sdk-mobile-client:$version

Any help would be much appreciated.

英文:

I'm just trying to upload an image to S3 via an Android app. What I concluded so far is that images that are bigger than 5MB are uploaded without a problem, however, images that are smaller than 5MB are FULLY UPLOADED and can be downloaded from S3 manually, however, the response i get is FAILED. I just want to remind you that 5MB is the bottom threshold for issuing a multipart request (SDK uploads using multipart automatically when the file is above 5MB, I checked this in their code).

I also checked the md5 sum by hand, and the sums from two files (uploaded and selected for upload) MATCH.

I tested uploading small images using a AWS S3 CLI and it works without a problem.

This is the exception I get:

AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash didn't match hash calculated by Amazon S3. You may need to delete the data stored in Amazon S3.

These are the libraries I am using:

com.amazonaws:aws-android-sdk-s3:$version

com.amazonaws:aws-android-sdk-mobile-client:$version

Any help would be much appreciated.

答案1

得分: 1

基本上,问题是图片上传的存储桶被加密了,一旦我们创建了一个新的未加密的存储桶,上传就正常了。
似乎两个文件的哈希值不同,因为一个哈希计算是在加密文件上进行的,另一个是在未加密文件上进行的。看起来(虽然我不是完全确定),多部分请求只是绕过了MD5校验和检查。

值得注意的是,只有在使用Android SDK时才会出现这个问题,iOS的SDK正常工作(不知道具体的实现差异是什么,以后可能可以更新这方面的信息)。

在引入加密之前的几个月里,它都是正常工作的。

我将进一步调查这个问题,并在有原因的情况下通知AWS支持。

英文:

Basically, the problem was that bucket where images were uploaded was encrypted, as soon as we created a new unencrypted bucket, the uploads were fine.
It seems that hash of two files was different because one hash calculation was done on an encrypted file and one on an unencrypted file. It seems (although I'm not completely sure) that multipart request just bypasses the md5 checksum check.

Important to mention, this was only happening while using the Android SDK, the iOS one works fine (don't know what are the exact differences in implementation, I could maybe update on that later).

It worked for months before the encryption was introduced.

I will investigate this issue further and notify AWS support if there is a reason for that.

huangapple
  • 本文由 发表于 2020年5月29日 05:54:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/62075180.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定