如何高效处理AWS Lambda和编译后的库?

huangapple go评论92阅读模式
英文:

How to deal efficiently with lambda AWS and compiled libraries?

问题

我最近在集成一个Lambda函数时遇到了一些问题,该函数使用了在运行环境中默认未安装的库。因此,我按照AWS文档中的说明进行了操作。文档指出,默认未安装的库应该被上传为一个zip文件,并且这些库必须编译为Linux操作系统。

由于我使用的是Windows,我使用了Windows Subsystem for Linux(WLS)来生成一个包含我的Lambda函数所需库的zip文件。然后,将这个zip文件上传到S3并与Lambda函数关联。

上述说明的过程可以运行。然而,这不是一个容易自动化的过程。自动化过程对于维护目的非常重要。是否有一种自动化的过程可以实现相同的功能?

英文:

I recently encountered some problems with the integration of a Lambda function using libraries that are not installed by default in the running environment. As results, I followed the procedure explained by AWS in its documentation. The documentation says libraries not installed by default should by uploaded as a zip file and these libraries must be compiled for linux OS.

Since I am on windows, I used the WLS to generate a zip file containing libraries required by my Lambda function. After that, the zip file is uploaded on a S3 and linked to the Lambda function.

The procedure explained above works. However, it is not a procedure that can be automated easily. The automation process is important for maintenance purpose. Is there an automated procedure achieving the same thing?

答案1

得分: 2

你可以将你的依赖项提取到一个 Lambda Layer 中,如果你的主 Lambda 函数不太复杂,只需在AWS控制台中进行更新和测试即可。此外,正如指南中所提到的,你还可以使用 SAM 来自动化大部分部署步骤,并确保代码在Lambda环境中正常运行。

英文:

You can extract your dependencies into a Lambda Layer and if you main Lambda is not heavy just update and test it in the AWS console. Also, as mentioned in the guide, you can use SAM to automate most of the deployment steps and be sure that the code works fine in the Lambda environment.

答案2

得分: 1

我们使用Jenkins来创建包含paramiko以及其他非本地Python包的构建,然后将它们打包并部署到Lambda。

我们在Artifactory中存储了一个基本的ZIP文件,其中包含paramiko包的依赖项,以及一个包含Lambda中主要处理程序方法的base/blank function.py文件。这个ZIP文件是通过创建一个包含包名称/版本的requirements.txt文件,然后运行以下命令来制作的:
pip install -r requirements.txt -t .

这将把所有依赖项的必要文件放入运行该命令的目录中。然后,您可以将该目录打包成ZIP文件:
zip -r9 ../lambda.zip * -x "bin/*" requirements.txt setup.cfg

您可以使用这个基本的ZIP文件来自动创建使用相同包的多个Lambda函数 - 您只需要让Jenkins复制要使用的Lambda函数文件。因此,每次运行Jenkins作业以构建/更新Lambda函数时,它会下载我们在上一步中制作的基本ZIP文件(如前面所提,我们将其存储在Artifactory中,但您可以将其存储在任何地方),然后运行以下命令来更新该ZIP文件以包含新的Lambda函数:
zip -g ../lambda.zip /path/tocheckedoutlambda/new_lambda_function.py

然后,Jenkins使用AWS插件上传到AWS。

这显然是针对Python的,但您应该能够使用类似的方法来处理其他编程语言。

详情请参阅:
https://alexharv074.github.io/2018/08/18/creating-a-zip-file-for-an-aws-lambda-python-function.html

了解有关在Python中创建Lambda ZIP文件的更多详细信息。

英文:

We use jenkins to create our builds that include paramiko as well as other non native python packages, zip them up and then deploy them to lambda.

We have a base zip stored in artifactory that contains the paramkiko package dependencies as well as a base/blank function.py file that contains our main handler method used in the lambda. This zip was made by making a requirments.txt file with the package name/version in it and running:
pip install -r requirements.txt -t .

This puts all the neccessary files from the dependencies into the directory that command was run from. You can then zip up that directory: zip -r9 ../lambda.zip * -x "bin/*" requirements.txt setup.cfg

You can use this base zip to automate creation of multiple lambdas that use the same packages - you just need to have jenkins copy in the lambda function file you want to use. So every time a jenkins job is run to build/update a lambda function it downloads that base zip we made in the previous step (which as mentioned we store in artifactory but you could store it anywhere) then runs the below command to update that zip with the new lambda:
zip -g ../lambda.zip /path/tocheckedoutlambda/new_lambda_function.py

Jenkins then uses the aws plugin to upload to aws.

This is for python obviously, but you should be able to use similar methods for other languages.

See:
https://alexharv074.github.io/2018/08/18/creating-a-zip-file-for-an-aws-lambda-python-function.html

For more details on making zips for lambda in python.

huangapple
  • 本文由 发表于 2020年1月6日 18:16:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/59610261.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定