英文:
AWS S3 response FAILED, even though file is fully uploaded
问题
I'm just trying to upload an image to S3 via an Android app. What I concluded so far is that images that are bigger than 5MB are uploaded without a problem, however, images that are smaller than 5MB are FULLY UPLOADED and can be downloaded from S3 manually, however, the response i get is FAILED. I just want to remind you that 5MB is the bottom threshold for issuing a multipart request (SDK uploads using multipart automatically when the file is above 5MB, I checked this in their code).
I also checked the md5 sum by hand, and the sums from two files (uploaded and selected for upload) MATCH.
I tested uploading small images using a AWS S3 CLI and it works without a problem.
This is the exception I get:
AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash didn't match hash calculated by Amazon S3. You may need to delete the data stored in Amazon S3.
These are the libraries I am using:
com.amazonaws:aws-android-sdk-s3:$version
com.amazonaws:aws-android-sdk-mobile-client:$version
Any help would be much appreciated.
英文:
I'm just trying to upload an image to S3 via an Android app. What I concluded so far is that images that are bigger than 5MB are uploaded without a problem, however, images that are smaller than 5MB are FULLY UPLOADED and can be downloaded from S3 manually, however, the response i get is FAILED. I just want to remind you that 5MB is the bottom threshold for issuing a multipart request (SDK uploads using multipart automatically when the file is above 5MB, I checked this in their code).
I also checked the md5 sum by hand, and the sums from two files (uploaded and selected for upload) MATCH.
I tested uploading small images using a AWS S3 CLI and it works without a problem.
This is the exception I get:
AmazonClientException: Unable to verify integrity of data upload. Client calculated content hash didn't match hash calculated by Amazon S3. You may need to delete the data stored in Amazon S3.
These are the libraries I am using:
com.amazonaws:aws-android-sdk-s3:$version
com.amazonaws:aws-android-sdk-mobile-client:$version
Any help would be much appreciated.
答案1
得分: 1
基本上,问题是图片上传的存储桶被加密了,一旦我们创建了一个新的未加密的存储桶,上传就正常了。
似乎两个文件的哈希值不同,因为一个哈希计算是在加密文件上进行的,另一个是在未加密文件上进行的。看起来(虽然我不是完全确定),多部分请求只是绕过了MD5校验和检查。
值得注意的是,只有在使用Android SDK时才会出现这个问题,iOS的SDK正常工作(不知道具体的实现差异是什么,以后可能可以更新这方面的信息)。
在引入加密之前的几个月里,它都是正常工作的。
我将进一步调查这个问题,并在有原因的情况下通知AWS支持。
英文:
Basically, the problem was that bucket where images were uploaded was encrypted, as soon as we created a new unencrypted bucket, the uploads were fine.
It seems that hash of two files was different because one hash calculation was done on an encrypted file and one on an unencrypted file. It seems (although I'm not completely sure) that multipart request just bypasses the md5 checksum check.
Important to mention, this was only happening while using the Android SDK, the iOS one works fine (don't know what are the exact differences in implementation, I could maybe update on that later).
It worked for months before the encryption was introduced.
I will investigate this issue further and notify AWS support if there is a reason for that.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论