英文:
Running AWS Batch Job on the JAR artifact from S3 bucket
问题
- amazon:linux镜像上没有安装AWS客户端，因此无法在作业定义中使用
aws s3 cp命令将可执行文件复制到EC2实例上。出现以下错误：
CannotStartContainerError: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "aws": executable file not found in $PATH: unknown
- 使用
wget命令可以从S3桶下载可执行文件到EC2，但在作业定义的Bash命令中，无法区分下一个命令。比如wget <source-url> && java -jar executable.jar，这里尝试处理Bash中的'&&'，尝试多种方式将两个命令放在单个Bash行中，但都没有成功。出现以下错误：
wget: unable to resolve host address java<- 在作业定义的Bash中不使用'&'符号
wget: unable to resolve host address ‘&&’<- 在Bash命令中使用'&'等分隔符来分隔命令时发生的错误。
英文:
In one of specific requirement, we need to use a Jar executable provided by other team in one of S3 bucket in a aws batch. The plan is to configure a aws batch which will spawn a EC2 instance, will copy the jar file from the s3 bucket and will run the java -jar command on the same jar with intended command line parameters.
Problem is,
- the amazon:linux image does not have aws client installed so we are not able to use the aws s3 cp command in the job definition to copy the executable to the EC2 instance. getting following error,
CannotStartContainerError: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "aws": executable file not found in $PATH: unknown
- with the wget command, the executable gets downloaded to EC2 from the s3 bucket; but the bash command in the job definition, is not able to distinguish the next command. Say wget <source-url> && java -jar executable.jar -> here it tries to process wget on '&&', tried multiple ways to provide two commands in single bash line but did not work. Getting following errors,
wget: unable to resolve host address java<- without using ampersand in bash of job definition
wget: unable to resolve host address ‘&&’<- with ampersand etc. separators for commands in bash
答案1
得分: 0
如果我理解你的情况正确，我相信你正在将命令wget <url> -O executable.jar && java -jar executable.jar 作为可执行命令传递。容器不会在shell中运行它，它只将其视为带有一组奇怪参数的单个命令。相反，你可以在shell中运行它，使用bash -c "wget <url> -O executable.jar && java -jar executable.jar"。如果你的容器中没有bash，你可能需要使用sh。
关于AWS CLI，你应该能够在Amazon Linux中使用yum install awscli来安装它。我相信它会从容器中获取凭据，你可以在Batch中配置它以访问你的S3存储桶。
英文:
If I'm understanding your scenario right, I believe you're passing the command wget <url> -O executable.jar && java -jar executable.jar as the executable command. The container doesn't run this in a shell; it just views it as a single command with a strange set of arguments. Instead you could run it in a shell with bash -c "wget <url> -O executable.jar && java -jar executable.jar". You might need to use sh if bash isn't available in your container.
Regarding the AWS CLI, you should be able to install it in Amazon Linux with yum install awscli. I believe it would pick up the credentials from your container, which you can configure in Batch to have access to your S3 bucket.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论