通配符字符在读取流.meta文件期间的行为如何?

huangapple go评论63阅读模式
英文:

Wild card character behavior during read stream .meta files?

问题

I am working on real-time streaming. When I tried to read all JSON files which will end with .json, readstream also reads files ending with .json.meta. How can I get rid of .meta files?

英文:

I am working on real time streaming. When I tried to read all json files which will ends with .json readstream also reads files ending with .json.meta. How can get rid of .meta files.

通配符字符在读取流.meta文件期间的行为如何?

In the storage account I have same files one is ending with .json and another one is ending with .json.meta

通配符字符在读取流.meta文件期间的行为如何?

通配符字符在读取流.meta文件期间的行为如何?

答案1

得分: 1

I reproduced your code, and I got the files only having the extension .json as below.

通配符字符在读取流.meta文件期间的行为如何?

Below are the files I am having.

通配符字符在读取流.meta文件期间的行为如何?

So, try restarting your cluster and try.

If you still get these .json.meta files, filter out as below.

filtered_streaming_df = dfProduc.filter(dfProduc["FileName"].like("%.json"))

First, I read all files as below, since I am getting correct JSON files.

通配符字符在读取流.meta文件期间的行为如何?

Then I filtered them.

Output:

通配符字符在读取流.meta文件期间的行为如何?

英文:

I reproduced your code, and i got the files only having extension .json as below.

通配符字符在读取流.meta文件期间的行为如何?

Below are files I am having.

通配符字符在读取流.meta文件期间的行为如何?

So, try restarting your cluster and try.

If you still get this.json.meta files, filter out as below.

filtered_streaming_df = dfProduc.filter(dfProduc["FileName"].like("%.json"))

First, I read all files as below, since i am getting correct json files.

通配符字符在读取流.meta文件期间的行为如何?

Then I filtered them.

Output:

通配符字符在读取流.meta文件期间的行为如何?

huangapple
  • 本文由 发表于 2023年6月8日 21:22:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/76432303.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定