英文:
Why does the speed in Pentaho insert/update step became 0 r/s?
问题
我正在进行数据迁移工作，但在插入/更新时，速度非常慢。甚至在20分钟后速度降至0。
英文:
I'm working a Data migration job and when it comes to the insert/update, the speed is really slow. And it even reached 0 after 20min.
答案1
得分: 1
在这些情况下，我所做的是将流程分开并执行三个转换：一个转换将要插入的行和要更新的行分开（存储在两个单独的文件/临时表中，或者用插入/更新标志标记它们并将它们存储在同一个文件/临时表中）。第二个转换执行插入操作，第三个转换执行更新操作。
我发现插入/更新非常慢，所以一旦目标达到某个（对我来说未知的）大小，将插入和更新分开并分别执行会快得多。
英文:
In these cases what I do is separate the flow and execute three transformations: One transformation separates the rows to insert and the rows to update (into two separate files/temporary tables or mark them with insert/update flag and store them in the same file/temp table). A second transformation perform the inserts and a third transformation performs the updates.
I found the Insert/update very slow, so as soon as the target reaches some (unknown to me) size, separating the inserts from the updates and performing them separately is much quicker.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论