英文:
Wild card character behavior during read stream .meta files?
问题
I am working on real-time streaming. When I tried to read all JSON files which will end with .json, readstream also reads files ending with .json.meta. How can I get rid of .meta files?
英文:
I am working on real time streaming. When I tried to read all json files which will ends with .json readstream also reads files ending with .json.meta. How can get rid of .meta files.
In the storage account I have same files one is ending with .json and another one is ending with .json.meta
答案1
得分: 1
I reproduced your code, and I got the files only having the extension .json
as below.
Below are the files I am having.
So, try restarting your cluster and try.
If you still get these .json.meta
files, filter out as below.
filtered_streaming_df = dfProduc.filter(dfProduc["FileName"].like("%.json"))
First, I read all files as below, since I am getting correct JSON files.
Then I filtered them.
Output:
英文:
I reproduced your code, and i got the files only having extension .json
as below.
Below are files I am having.
So, try restarting your cluster and try.
If you still get this.json.meta
files, filter out as below.
filtered_streaming_df = dfProduc.filter(dfProduc["FileName"].like("%.json"))
First, I read all files as below, since i am getting correct json files.
Then I filtered them.
Output:
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论