英文:
How to list and copy s3 keys with non alphanumeric characters?
问题
我有一段代码，使用ListObjectsPages->Contents->Key列出键，并使用CopyObject复制这些键。这通常可以工作，但对于某些键，它会报错NoSuchKey: The specified key does not exist.。它抱怨的键集包括带有+的键。
ListObjectsPages返回键"foo+bar"。- 对于"foo+bar"，
CopyObject会出现NoSuchKey错误。 - 对于"foo bar"（未转义的），
CopyObject会出现NoSuchKey错误。
奇怪的是，如果我在命令行中使用aws s3 cp复制"foo+bar"，复制操作可以成功。但我不能使用命令行。我需要使用SDK。
我正在使用v1.8.11版本。
正如Rayfen提到的，加号字符可能是空格替换的结果。
更新：
这里的一切都在https://github.com/aws/aws-sdk-go/issues/1438中讨论清楚了。Rayfen关于需要使用QueryEscape的说法是正确的。我将向目前唯一的回答提供奖励，因为它提供了有用的信息，但不会将其选为正确答案。
英文:
I have code which lists keys using ListObjectsPages->Contents->Key and copies those keys using CopyObject. This works in general but for some keys it's complaining NoSuchKey: The specified key does not exist. The set of keys it's complaining about include keys with +.
ListObjectsPagesreturns key "foo+bar".CopyObjecton "foo+bar" gives theNoSuchKeyerror.CopyObjecton "foo bar" (unescaped) gives theNoSuchKeyerror.
Oddly, if I use the CLI: aws s3 cp on "foo+bar", the copy works. But I can't use the CLI. I need to use the sdk.
I'm using v1.8.11
As Rayfen mentioned, the plus characters could be the result of space replacement.
Update:
Everything was hashed out here https://github.com/aws/aws-sdk-go/issues/1438. Rayfen was right about needing to QueryEscape. I'm going to award the only current answer with the bounty since it adds useful information, but not select it as correct.
答案1
得分: 1
对象键和元数据文档 很清楚：
> 以下字符集通常可安全用于键名：
> - 字母数字字符 [0-9a-zA-Z]
- 特殊字符
!,-,_,.,*,',(, 和)
不仅 + 会被转换为空格，而且根据同一页的“可能需要特殊处理的字符”部分，: 也应该从空格转换回来，而 QueryUnescape 并不会这样做（它只会将空格转换回 +）。
请检查你的键是否包含其他需要小心处理的特殊字符，比如 :（也会被替换为空格），@ 或 =（被替换为 ;），或者 , 和 ?。
特别要检查从 QueryUnescape 获得的键是否在原始键中用 + 代替了 :：这可能是一个错误地“取消转义”的空格。
英文:
The object key and metadata document is clear:
> The following character sets are generally safe for use in key names:
> - Alphanumeric characters [0-9a-zA-Z]
- Special characters
!,-,_,.,*,',(, and)
Not only + would be converted into space with, but, from the section "Characters That Might Require Special Handling" of the same page, ':' should also be converted back from space, which QueryUnescape does not do (it only convert space back to +).
Check if your keys include other special characters to be handled with care, like : (also replaced by space), @ or = (replaced by ;), or , and ?.
Check in particular if the key obtained from QueryUnescape has a + instead of a ':' in the original key: that could be a space incorrectly "unescaped".
答案2
得分: 0
这是发生的情况的详细说明：https://github.com/aws/aws-sdk-go/issues/1438
- 主要问题是
CopySource需要进行URL编码，而不是Key字段，这让我感到惊讶（我两个都进行了URL编码）。 - 另一个问题是我使用了
path.Join，它会去掉末尾的\。这是一个问题，因为S3键可以有末尾的\，它表示一种文件夹的形式。
英文:
This details what was happening: https://github.com/aws/aws-sdk-go/issues/1438
- The main issue was that
CopySourceneeds to be url-encoded and not theKeyfield, which was surprising to me. (I was url-encoding both.) - The other issue was that I was using
path.Joinwhich strips out trailing\. This is a problem because s3 keys can have trailing\- which represents a sort of folder.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论