英文:
How to run a SQL only once which is required for processing the read data in spring batch
问题
Here is the translated content without the code parts:
我是Spring Batch的新手,我有一个Spring Boot应用程序,在其中运行一个Spring Batch作业,该作业执行一系列步骤。在其中的一个步骤中,我从数据库中读取数千条记录并处理它们,然后将其写入文件。在处理过程中,我需要从另一个表中获取数据,然后将其存储在一个列表中,我可以在将它们再次写入文件之前使用它来处理所有读取的数据。
是否有一种方法可以只运行一次SQL,对所有数据都有效?
我希望这有助于您的理解。如果您需要更多信息,请随时告诉我。
英文:
I am new to Spring Batch, I have a spring boot app inn which i run a Spring batch job which executes a series of step, on one of the steps, I read thousands of record from a DB and process them and write it to a file. while processing I need a data from another table which i need to store in it a list, which i can use to process all the read data before writing them to a file again.
Is there a way where i can run that SQL only once common to all data?
`
@Configuration
public class JDBCStep {
@Bean("JdbcStep")
public Step step() {
return stepBuilderFactory.get("JdbcStep")
.<ReadObject, WriteObject>chunk(10)
.reader(reader)
.processor(new processor())
.writer(writer)
.listener(new CustomStepListener())
.build();
}
@Bean
public ItemReader<StudentDTO> itemReader(DataSource dataSource,
PagingQueryProvider queryProvider) {
return new JdbcPagingItemReaderBuilder<StudentDTO>()
.name("pagingItemReader")
.dataSource(dataSource)
.pageSize(1)
.queryProvider(queryProvider)
.rowMapper(new BeanPropertyRowMapper<>(SampleDTO.class))
.build();
}
public class Processor implements ItemProcessor<SampleDTO, OutputDTO>{
@override
public OutputDTO process(SampleDTO input){
return ProcessService.processData(input)
}
}
}
public class CustomStepListener implements StepExecutionListener {
@Override
public void beforeStep(StepExecution stepExecution) {
System.out.println("StepExecutionListener - beforeStep");
}
@Override
public ExitStatus afterStep(StepExecution stepExecution) {
System.out.println("StepExecutionListener - afterStep");
return null;
}
}
@Service
public class ProcessService{
public OutputDTO processData(SampleDTO s){
**Here For processing this data s, I need a List of ID from another table which is common for all the sampleDTO data to be processed. If i make a DB call in this method, It makes a DB call for all the record. Is there a way i can make a DB call only once to process all the data**
Query q = entityManager.createQuery(query);
List results = q.getResultList();
//I need the results for all the records to be processed.
}
答案1
得分: 0
You can implement StepExecutionListener to your processor class.
> StepExecutionListener 表示 Step 执行的最通用监听器。它允许在 Step 开始之前和结束之后进行通知。https://docs.spring.io/spring-batch/docs/current/reference/html/step.html
在处理器步骤之前,您可以一次加载将由所有处理数据使用的 IDs。
public class Processor implements ItemProcessor<SampleDTO, OutputDTO>, StepExecutionListener {
private static List<String> ids = new ArrayList<>();
@Override
public OutputDTO process(SampleDTO input) throws Exception {
return new ProcessService().processData(input, ids);
}
@Override
public void beforeStep(StepExecution stepExecution) {
Query q = entityManager.createQuery(query);
ids = q.getResultList();
}
@Override
public ExitStatus afterStep(StepExecution stepExecution) {
ids.clear();
return stepExecution.getExitStatus();
}
}
如果您正在使用自定义的 StepExecutionListener,还可以使用 @BeforeStep 注解,该注解仅在执行步骤之前调用一次。
public class Processor implements ItemProcessor<SampleDTO, OutputDTO> {
private final List<String> ids;
public Processor() {
ids = new ArrayList<>();
}
@BeforeStep
public void beforeStepExecution() {
Query q = entityManager.createQuery(query);
ids = q.getResultList();
}
@Override
public OutputDTO process(SampleDTO input) throws Exception {
return new ProcessService().processData(input, ids);
}
}
英文:
You can implement StepExecutionListener to your processor class.
> StepExecutionListener represents the most generic listener for Step
> execution. It allows for notification before a Step is started and
> after it ends. https://docs.spring.io/spring-batch/docs/current/reference/html/step.html
Before the processor step, you can load ids once which will be used by all process data.
public class Processor implements ItemProcessor<SampleDTO, OutputDTO>, StepExecutionListener {
private static List<String> ids = new ArrayList<>();
@Override
public OutputDTO process(SampleDTO input) throws Exception {
return new ProcessService().processData(input, ids);
}
@Override
public void beforeStep(StepExecution stepExecution) {
Query q = entityManager.createQuery(query);
ids = q.getResultList();
}
@Override
public ExitStatus afterStep (StepExecution stepExecution) {
ids.clear();
return stepExecution.getExitStatus();
}
}
If you are using a custom StepExecutionListener you can also use the @BeforeStep annotation which will only be called once before the step is executed.
public class Processor implements ItemProcessor<SampleDTO, OutputDTO> {
private final List<String> ids;
public Processor(){
ids = new ArrayList<>();
}
@BeforeStep
public void beforeStepExecution() {
Query q = entityManager.createQuery(query);
ids = q.getResultList();
}
@Override
public OutputDTO process(SampleDTO input) throws Exception {
return new ProcessService().processData(input, ids);
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论