有关摄入数值的问题,增加了2倍。

huangapple go评论74阅读模式
英文:

Issue with ingest values, 2x more

问题

当我将值添加到特征集时,流水线被调用了两次更多(我使用的是 MLRun 版本 1.2.1)。看起来像是个问题,你知道为什么吗?

我使用了这段代码:

import mlrun
import mlrun.feature_store as fstore

# mlrun: start-code
import math

def calc(x):
    x['fn2'] = math.sin(x['fn2']) * 100.0
    print('calc')
    return x

# mlrun: end-code

mlrun.set_env_from_file("mlrun-nonprod.env")
project = mlrun.get_or_create_project(project_name, context='./', user_project=False)
feature_derived = fstore.get_feature_set(f"{project_name}/{feature_derivedName}")
# ...
# dataFrm 只有两个值
feature_derived.graph.to(name="calc", handler='calc')
fstore.ingest(feature_derived, dataFrm)

我得到了以下输出(calc 方法被调用了四次),针对只有两个值的 dataFrm:

> calc
> calc
> calc
> calc
英文:

When I ingested values to the feature set, the pipeline was called 2x more (I used MLRun version 1.2.1). It seems as the issue, do you know why?

I used this code:

import mlrun
import mlrun.feature_store as fstore

# mlrun: start-code
import math

def calc(x):
    x['fn2']=math.sin(x['fn2'])*100.0
    print('calc')
    return x

# mlrun: end-code

mlrun.set_env_from_file("mlrun-nonprod.env")
project = mlrun.get_or_create_project(project_name, context='./', user_project=False)
feature_derived = fstore.get_feature_set(f"{project_name}/{feature_derivedName}")
...
# dataFrm has only two values
feature_derived.graph.to(name="calc", handler='calc')
fstore.ingest(feature_derived, dataFrm)

I got this output (method calc was called four times) for dataFrm with two values:

> calc
> calc 
> calc
> calc

答案1

得分: 0

解决方法很简单,只需在数据摄取方法中设置infer_options=0来关闭预览模式。请查看代码的一部分:

...
feature_derived.graph.to(name="calc", handler='calc')
fstore.ingest(feature_derived, dataFrm, infer_options=0)
...

输出仅包含两个值(如所请求):

> calc
> calc
英文:

The solution is easy, it is enough to switch-off preview mode based on setting infer_options=0 in ingest method. See part of the code

...
feature_derived.graph.to(name="calc", handler='calc')
fstore.ingest(feature_derived, dataFrm, infer_options=0)
...

The output has only two values (as requested):

> calc
> calc 

huangapple
  • 本文由 发表于 2023年2月27日 03:13:40
  • 转载请务必保留本文链接:https://go.coder-hub.com/75574407.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定