英文:
can I serve multiple models thru the same port using MLflow?
问题
我正在阅读有关使用MLflow时开箱即用的部署工具。似乎您可以通过执行类似以下操作从注册表中提供模型：
mlflow models serve -m runs:/<run_id>/model <some server>:<some port>。
然后，通过执行以下操作发送一些数据并获得预测：
curl http:/<some server>:<some port>/invocations -H 'Content-Type: application/json' -d '{
"inputs": {<some data>}
}'.
这很不错，但这是否意味着您只能在一个端口上提供一个模型？这是最佳实践还是技术限制，还是情况并非如此？
英文:
I was reading about the deployment tools you get out of the box when using MLflow.
It seems you can serve a model from the registry by doing something like
mlflow models serve -m runs:/<run_id>/model <some server>:<some port>.
and then, you send some data and get back a prediction by doing
curl http:/<some server>:<some port>/invocations -H 'Content-Type: application/json' -d '{
"inputs": {<some data>}
}'.
This is great but does this mean you can only serve one model on one port? Is this a best practice or a technical limitation or not the case?
答案1
得分: 1
You can serve any model using MLflow, if you want to use a single endpoint for multiple models you can package all your models into a single model using pyfunc module and deploy this one. Here an example of this.
In that example, there are two models, and wrapper is created around these models, then, during inference time we choose which model get predictions from using input parameters.
Here you can also find an example published for Azure Databricks.
英文:
You can serve any model using MLflow, if you want to use a single endpoint for multiple models you can package all your models into a single model using pyfunc module and deploy this one. Here an example of this.
In that example, there are two models, and wrapper is created around these models, then, during inference time we choose which model get predictions from using input parameters.
Here you can also find an example published for Azure Databricks
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论