英文:
Copying multiple s3 objects at once without --recursive
问题
以下是翻译好的部分：
我有一个S3对象列表，我想从一个存储桶复制到另一个存储桶。这是情景：
| 存储桶1中的对象 | 存储桶2中的对象 |
|---|---|
| s3:/bucket1/path/to/object1 | s3:/bucket2/different/path/{date}/{mode}/object1 |
| s3:/bucket1/path/to/object2 | s3:/bucket2/different/path/{date}/{mode}/object2 |
| s3:/bucket1/path/to/object3 | s3:/bucket2/different/path/{date}/{mode}/object3 |
我不能使用 --recursive 选项，因为我完全更改了对象键。我每个文件执行一次 aws s3 cp，速度相对较慢。是否有更好的方法？
英文:
I have a list of s3 object that I want to copy from one bucket to another. This is the scenario:
| Object int Bucket 1 | Object in Bucket 2 |
|---|---|
| s3:/bucket1/path/to/object1 | s3:/bucket2/different/path/{date}/{mode}/object1 |
| s3:/bucket1/path/to/object2 | s3:/bucket2/different/path/{date}/{mode}/object2 |
| s3:/bucket1/path/to/object3 | s3:/bucket2/different/path/{date}/{mode}/object3 |
I can not use the option --recursive as I'm changing completely the objects keys. I'm executing one aws s3 cp per file and it's pretty slow. Is there a better way to do it?
答案1
得分: 1
你可以通过在命令的末尾添加&来在计算机上并行运行该命令的3个实例：
aws s3 cp s3:/bucket1/path/to/object1 s3:/bucket2/different/path/{date}/{mode}/object1 &
aws s3 cp s3:/bucket1/path/to/object2 s3:/bucket2/different/path/{date}/{mode}/object2 &
aws s3 cp s3:/bucket1/path/to/object3 s3:/bucket2/different/path/{date}/{mode}/object3 &
这将在后台同时运行这些命令，而不是一个接一个地运行。您的屏幕上的消息可能会有些重叠和混淆，但它会正常工作。
英文:
You can run 3x that command in parallel on your computer by adding a & at the end of the command:
aws s3 cp s3:/bucket1/path/to/object1 s3:/bucket2/different/path/{date}/{mode}/object1 &
aws s3 cp s3:/bucket1/path/to/object2 s3:/bucket2/different/path/{date}/{mode}/object2 &
aws s3 cp s3:/bucket1/path/to/object3 s3:/bucket2/different/path/{date}/{mode}/object3 &
This will run the commands 'in the background' and at the same time rather than one-at-a-time. The messages on your screen will be a bit overlapping and confusing, but it will work.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论