英文:
Jooq Java Code to handle more than 200k records from a table
问题
fetchRecords() 函数中如何只返回一部分记录？我应该将其编写为异步函数吗？这样处理结果是否正确？或者是否有更好的处理方法？
class RecordService {RecordPersistence recordPersistence = new RecordPersistence();public void processRecords() {List<Record> records = recordPersistence.fetchRecords();// 处理这些记录的代码}}class RecordPersistence {public List<Record> fetchRecords(){Cursor<Record> records = dsl.select...fetchLazy();while (records.hasNext()) {records.fetchNext(1000);}return records;}}
英文:
The table in my database has more than 200k records and I don't want to fetch all of them at once in the resultSet to process. I am trying to write a function in a service that calls another persistence class function to bring only 1000 records to process.
Once it processes the 1000 records, it should fetch the following thousand records. I am using the JOOQ database library in my Java code. I am just sharing a sample skeleton code that I had in my mind.
class RecordService {RecordPersistence recordPersistence = new RecordPersistence();public void processRecords() {List<Record> records = recordPersistence.fetchRecords();// Code to process them}}class RecordPersistence {public List<Record> fetchRecords(){Cursor<Record> records = dsl.select...fetchLazy();while (records.hasNext()) {records.fetchNext(1000);}return records}}
How to return only a few records from the fetchRecords() function? Should I write this is an async function? Is this the correct way to process result? or is there a better way to handle my case?
答案1
得分: 0
如评论中所提到的，问题在于您将处理逻辑放在了获取逻辑之外。只需将其移到内部，例如：
class RecordService {RecordPersistence recordPersistence = new RecordPersistence();public void processRecords() {recordPersistence.fetchRecords((List<Record> records) -> {// 处理记录的代码});}}class RecordPersistence {public List<Record> fetchRecords(Consumer<? super List<Record>> consumer) {try (Cursor<Record> cursor = dsl.select...fetchLazy()) {while (cursor.hasNext()) {consumer.accept(cursor.fetchNext(1000));}}}}
对于这种类型的流式工作，您不需要任何范式转变（例如异步、响应式等）。只需从外部迭代转换为内部迭代。
英文:
As mentioned in the comments, the problem is that you're keeping the processing logic outside of the fetch logic. Just move it inside, e.g. as follows:
class RecordService {RecordPersistence recordPersistence = new RecordPersistence();public void processRecords() {recordPersistence.fetchRecords((List<Record> records) -> {// Code to process them});}}class RecordPersistence {public List<Record> fetchRecords(Consumer<? super List<Record>> consumer) {try (Cursor<Record> cursor = dsl.select...fetchLazy()) {while (cursor.hasNext()) {consumer.accept(cursor.fetchNext(1000));}}}}
You don't need any paradigm shift (e.g. async, reactive, etc.) for this type of streaming to work. Just switch from external to internal iteration
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论