英文:
Jooq Java Code to handle more than 200k records from a table
问题
fetchRecords() 函数中如何只返回一部分记录？我应该将其编写为异步函数吗？这样处理结果是否正确？或者是否有更好的处理方法？
class RecordService {
RecordPersistence recordPersistence = new RecordPersistence();
public void processRecords() {
List<Record> records = recordPersistence.fetchRecords();
// 处理这些记录的代码
}
}
class RecordPersistence {
public List<Record> fetchRecords(){
Cursor<Record> records = dsl.select...fetchLazy();
while (records.hasNext()) {
records.fetchNext(1000);
}
return records;
}
}
英文:
The table in my database has more than 200k records and I don't want to fetch all of them at once in the resultSet to process. I am trying to write a function in a service that calls another persistence class function to bring only 1000 records to process.
Once it processes the 1000 records, it should fetch the following thousand records. I am using the JOOQ database library in my Java code. I am just sharing a sample skeleton code that I had in my mind.
class RecordService {
RecordPersistence recordPersistence = new RecordPersistence();
public void processRecords() {
List<Record> records = recordPersistence.fetchRecords();
// Code to process them
}
}
class RecordPersistence {
public List<Record> fetchRecords(){
Cursor<Record> records = dsl.select...fetchLazy();
while (records.hasNext()) {
records.fetchNext(1000);
}
return records
}
}
How to return only a few records from the fetchRecords() function? Should I write this is an async function? Is this the correct way to process result? or is there a better way to handle my case?
答案1
得分: 0
如评论中所提到的，问题在于您将处理逻辑放在了获取逻辑之外。只需将其移到内部，例如：
class RecordService {
RecordPersistence recordPersistence = new RecordPersistence();
public void processRecords() {
recordPersistence.fetchRecords(
(List<Record> records) -> {
// 处理记录的代码
}
);
}
}
class RecordPersistence {
public List<Record> fetchRecords(Consumer<? super List<Record>> consumer) {
try (Cursor<Record> cursor = dsl.select...fetchLazy()) {
while (cursor.hasNext()) {
consumer.accept(cursor.fetchNext(1000));
}
}
}
}
对于这种类型的流式工作，您不需要任何范式转变（例如异步、响应式等）。只需从外部迭代转换为内部迭代。
英文:
As mentioned in the comments, the problem is that you're keeping the processing logic outside of the fetch logic. Just move it inside, e.g. as follows:
class RecordService {
RecordPersistence recordPersistence = new RecordPersistence();
public void processRecords() {
recordPersistence.fetchRecords(
(List<Record> records) -> {
// Code to process them
}
);
}
}
class RecordPersistence {
public List<Record> fetchRecords(Consumer<? super List<Record>> consumer) {
try (Cursor<Record> cursor = dsl.select...fetchLazy()) {
while (cursor.hasNext()) {
consumer.accept(cursor.fetchNext(1000));
}
}
}
}
You don't need any paradigm shift (e.g. async, reactive, etc.) for this type of streaming to work. Just switch from external to internal iteration
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论