英文:
Spring boot JPA batch inserts Exception Handling
问题
我正在处理一个实时用例，需要使用Spring Boot JPA将一批消息加载到SQL Server表中，将所有的模型对象添加到列表中，并使用repository.saveAll(list)进行批量加载。由于性能原因，我不能逐条插入记录。我正在寻找以下选项。
是否有任何方法可以读取导致错误的消息，并继续处理其他记录以插入表中。我必须将此错误消息存储在某个地方，不应将其传递给下游应用程序。
英文:
I am working on a real time use case that needs to load batch of messages into SQL server table with spring boot JPA adding all the model objects to the list and doing this for batch loads repository.saveAll(list). Due to performance, I cannot do record by record inserts. I am looking for below option.
Is there any way I can read the error causing message and continue with other records to insert into table. I have to store this error message somewhere and shouldn't pass to Downstream applications.
答案1
得分: 2
saveAll方法，属于Spring Data JPA，实际上是逐个保存记录的列表：
/*
* (non-Javadoc)
* @see org.springframework.data.jpa.repository.JpaRepository#save(java.lang.Iterable)
*/
@Transactional
@Override
public <S extends T> List<S> saveAll(Iterable<S> entities) {
Assert.notNull(entities, "Entities must not be null!");
List<S> result = new ArrayList<S>();
for (S entity : entities) {
result.add(save(entity));
}
return result;
}
然而，自己处理消息循环和让Spring Data JPA来处理的区别在于：saveAll方法使用属性spring.jpa.properties.hibernate.jdbc.batch_size=4来确定是否启用批处理，根据批处理的大小，插入操作可以更有效率。
我认为你只需要自己循环处理消息并捕获错误：
for(EntityType entity: list){
try{
repository.save(entity);
} catch (Exception e){
// 自行处理
}
}
英文:
The saveAll method, of Spring data JPA, is actually saving the list of records one by one:
/*
* (non-Javadoc)
* @see org.springframework.data.jpa.repository.JpaRepository#save(java.lang.Iterable)
*/
@Transactional
@Override
public <S extends T> List<S> saveAll(Iterable<S> entities) {
Assert.notNull(entities, "Entities must not be null!");
List<S> result = new ArrayList<S>();
for (S entity : entities) {
result.add(save(entity));
}
return result;
}
Still, the difference between handling the loop of messages yourself or let it to Spring data JPA to do the job: saveAll uses the property spring.jpa.properties.hibernate.jdbc.batch_size=4 to determine if batching is activated or not, and depending on the size of the batch, the insert can perform better.
I think you just need to loop on the messages yourself and catching the errors youself:
for(EnitityType entity: list){
try{
repository.save(entity);
} catch (Exception e){
// Do what you want
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论