英文:
How to use unnest function in data fusion
问题
My input table is like this:
and I need output like this in data fusion (i.e if if qty=2 records need to repeated 2 times, if its 1 repeated once, if its n n times).
can you please any one help me in Cloud-datafusion what wrangler directive i can use to get the data in this format.
英文:
My input table is like this:
and I need output like this in data fusion (i.e if if qty=2 records need to repeated 2 times, if its 1 repeated once, if its n n times).
can you please any one help me in Cloud-datafusion what wrangler directive i can use to get the data in this format.
答案1
得分: 1
以下是翻译好的部分：
- 使用以下指令来创建所需大小的列表，然后展开它并删除字段后：
- set-column :arr l=new('java.util.ArrayList');i=0;while(i lt qty) {l.add(i);i=i+1}; return l;
- flatten :arr
- drop :arr
请注意，从单个行生成的整个行列表应适应执行器内存，因此适用于较小的数量（最多约1000个）。如果要处理更大的数量，您需要生成一个数据集并使用高级连接。
英文:
You can use next directives to create a list of needed size, then explode it and remove the field after:
- set-column :arr l=new('java.util.ArrayList');i=0;while(i lt qty) {l.add(i);i=i+1}; return l;
- flatten :arr
- drop :arr
Please note that the whole list of rows produced from a single row should fit into executor memory, so it's practical for smaller quantities (up to ~1000). To go bigger numbers, you would need to generate a dataset and use an advanced join.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论