运行 SQL 仅一次,这对于在 Spring Batch 中处理读取的数据是必需的。

huangapple go评论70阅读模式
英文:

How to run a SQL only once which is required for processing the read data in spring batch

问题

Here is the translated content without the code parts:

我是Spring Batch的新手,我有一个Spring Boot应用程序,在其中运行一个Spring Batch作业,该作业执行一系列步骤。在其中的一个步骤中,我从数据库中读取数千条记录并处理它们,然后将其写入文件。在处理过程中,我需要从另一个表中获取数据,然后将其存储在一个列表中,我可以在将它们再次写入文件之前使用它来处理所有读取的数据。

是否有一种方法可以只运行一次SQL,对所有数据都有效?

我希望这有助于您的理解。如果您需要更多信息,请随时告诉我。

英文:

I am new to Spring Batch, I have a spring boot app inn which i run a Spring batch job which executes a series of step, on one of the steps, I read thousands of record from a DB and process them and write it to a file. while processing I need a data from another table which i need to store in it a list, which i can use to process all the read data before writing them to a file again.

Is there a way where i can run that SQL only once common to all data?

`

@Configuration
public class JDBCStep {

  @Bean("JdbcStep")
  public Step  step() {
    
    return stepBuilderFactory.get("JdbcStep")
                .<ReadObject, WriteObject>chunk(10)
                .reader(reader)
                .processor(new processor())
                .writer(writer)
                .listener(new CustomStepListener())
                .build();
  }

  @Bean
  public ItemReader<StudentDTO> itemReader(DataSource dataSource,
                                             PagingQueryProvider queryProvider) {
        return new JdbcPagingItemReaderBuilder<StudentDTO>()
                .name("pagingItemReader")
                .dataSource(dataSource)
                .pageSize(1)
                .queryProvider(queryProvider)
                .rowMapper(new BeanPropertyRowMapper<>(SampleDTO.class))
                .build();
    } 

   public class Processor implements ItemProcessor<SampleDTO, OutputDTO>{
        @override
        public OutputDTO process(SampleDTO input){
           return ProcessService.processData(input)
        }
   }
}

public class CustomStepListener implements StepExecutionListener {

	@Override
	public void beforeStep(StepExecution stepExecution) {
		System.out.println("StepExecutionListener - beforeStep");
	}

	@Override
	public ExitStatus afterStep(StepExecution stepExecution) {
		System.out.println("StepExecutionListener - afterStep");
		return null;
	}

}


@Service
public class ProcessService{
    public OutputDTO processData(SampleDTO s){

        **Here For processing this data s, I need a List of ID from another table which is common for all the sampleDTO data to be processed. If i make a DB call in this method, It makes a DB call for all the record. Is there a way i can make a DB call only once to process all the data**
     
     Query q = entityManager.createQuery(query);
     List results = q.getResultList();

//I need the results for all the records to be processed.

}

答案1

得分: 0

You can implement StepExecutionListener to your processor class.

> StepExecutionListener 表示 Step 执行的最通用监听器。它允许在 Step 开始之前和结束之后进行通知。https://docs.spring.io/spring-batch/docs/current/reference/html/step.html

在处理器步骤之前,您可以一次加载将由所有处理数据使用的 IDs。

public class Processor implements ItemProcessor<SampleDTO, OutputDTO>, StepExecutionListener {

    private static List<String> ids = new ArrayList<>();

    @Override
    public OutputDTO process(SampleDTO input) throws Exception {
        return new ProcessService().processData(input, ids);
    }

    @Override
    public void beforeStep(StepExecution stepExecution) {
        Query q = entityManager.createQuery(query);
        ids = q.getResultList();
    }

    @Override
    public ExitStatus afterStep(StepExecution stepExecution) {
        ids.clear();
        return stepExecution.getExitStatus();
    }
}

如果您正在使用自定义的 StepExecutionListener,还可以使用 @BeforeStep 注解,该注解仅在执行步骤之前调用一次。

public class Processor implements ItemProcessor<SampleDTO, OutputDTO> {

    private final List<String> ids;

    public Processor() {
        ids = new ArrayList<>();
    }

    @BeforeStep
    public void beforeStepExecution() {
        Query q = entityManager.createQuery(query);
        ids = q.getResultList();
    }

    @Override
    public OutputDTO process(SampleDTO input) throws Exception {
        return new ProcessService().processData(input, ids);
    }

}
英文:

You can implement StepExecutionListener to your processor class.

> StepExecutionListener represents the most generic listener for Step
> execution. It allows for notification before a Step is started and
> after it ends. https://docs.spring.io/spring-batch/docs/current/reference/html/step.html

Before the processor step, you can load ids once which will be used by all process data.

  public class Processor implements ItemProcessor&lt;SampleDTO, OutputDTO&gt;, StepExecutionListener {

    private static List&lt;String&gt; ids = new ArrayList&lt;&gt;();

    @Override
    public OutputDTO process(SampleDTO input) throws Exception {
        return new ProcessService().processData(input, ids);
    }

    @Override
    public void beforeStep(StepExecution stepExecution) {
        Query q = entityManager.createQuery(query);
        ids = q.getResultList();
    }

    @Override
    public ExitStatus afterStep (StepExecution stepExecution) {
        ids.clear();
        return stepExecution.getExitStatus();
    }
}

If you are using a custom StepExecutionListener you can also use the @BeforeStep annotation which will only be called once before the step is executed.

public class Processor implements ItemProcessor&lt;SampleDTO, OutputDTO&gt; {

    private final List&lt;String&gt; ids;

    public Processor(){
        ids = new ArrayList&lt;&gt;();
    }

    @BeforeStep
    public void beforeStepExecution() {
        Query q = entityManager.createQuery(query);
        ids = q.getResultList();
    }

    @Override
    public OutputDTO process(SampleDTO input) throws Exception {
        return new ProcessService().processData(input, ids);
    }

}

huangapple
  • 本文由 发表于 2023年4月20日 05:33:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/76058969.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定