spring batch MultiResourceItemWriter:如何唯一标识每个已写入的文件

huangapple go评论101阅读模式
英文:

spring batch MultiResourceItemWriter : How to uniquely identify each of the written files

问题

使用MultiResourceItemWriter在步骤中编写多个CSV文件。在步骤中写入各个文件之后,我希望能够使用StepExecutionListener将这些文件作为附件通过电子邮件发送给相应的接收者。

然而,问题在于我如何知道应该将哪些文件发送到哪个电子邮件?文件名还是后缀(我有一个自定义的ResourceSuffixCreator,但它只能获得一个索引,不能有助于有用地区分一个文件和另一个文件。)

使用Spring Boot 2.2.7。

更新
假设在此作业中有多个步骤。成功步骤使用MultiResourceItemWriter为每个电子邮件目标输出与成功相关的文件。

return this.stepBuilderFactory.get("generateSuccessRecords")
    .<SuccessReport, SuccessReport>chunk(1)
    .reader(successReportItemReader(null, null))
    .processor(itemProcessor)
    .writer(successReportItemWriter(null))
    .build();

successReportItemWriter是一个代理到MultiResourceItemWriter的方法。

return new MultiResourceItemWriterBuilder<SuccessReport>()
    .name("successReportItemWriter")
    .itemCountLimitPerResource(1)
    .delegate(individualSuccessReportItemWriter())
    .resource(new FileSystemResource(jobReportDirectory + "/successReport"))
    .resourceSuffixCreator(suffixCreator)
    .build();

individualSuccessReportItemWriter()如下所示。

FlatFileItemWriter<SuccessReport> itemWriter = new FlatFileItemWriter<>();
itemWriter.setName("individualSuccessReportItemWriter");
itemWriter.setHeaderCallback(new SuccesssReportHeaderCallback());
itemWriter.setLineAggregator(new SuccessReportLineAggregator());

在成功步骤生成SuccessReport之后,Fallout步骤将从数据库查询并重复上述步骤,以使用MultiResourceItemWriter为每个电子邮件目标创建FalloutReport .csv文件。

目标是能够将每个电子邮件目标的成功报告和故障报告作为附件发送。假设有25个电子邮件目标。作为运行2个步骤(成功和故障)的结果,将生成25个成功的CSV文件和25个故障的CSV文件。每个电子邮件目标将作为附件获取1个成功和1个故障的CSV文件。

类SuccessReport和FalloutReport在生成.csv文件时都有电子邮件目标,但由于后缀创建器不允许相应地命名文件,因此无法将文件命名为这样的名称。

英文:

Writing multiple CSV files using MultiResourceItemWriter in a Step. After writing the various files in the Step, I want to be able to email these files to the appropriate receiver using the StepExecutionListener.

The problem, though, is how do I know which of the files should be sent to which email ? The filename or the suffix (I have a custom ResourceSuffixCreator but it only gets an index which can't help identify 1 file from the other usefully.)

Using spring boot 2.2.7.

Thanks for any help.

UPDATE
I have, say, steps in this job. The success step outputs success related files for each email target using MultiResourceItemWriter.

    return this.stepBuilderFactory.get(&quot;generateSuccessRecords&quot;)
            .&lt;SuccessReport, SuccessReport&gt;chunk(1)
            .reader(successReportItemReader(null, null))
            .processor(itemProcessor)
            .writer(successReportItemWriter(null))
            .build();

successReportItemWriter is a MultiResourceItemWriter that delegates to a

    return new MultiResourceItemWriterBuilder&lt;SuccessReport&gt;()
            .name(&quot;successReportItemWriter&quot;)
            .itemCountLimitPerResource(1)
            .delegate(individualSuccessReportItemWriter())
            .resource(new FileSystemResource(jobReportDirectory + &quot;/successReport&quot;))
            .resourceSuffixCreator(suffixCreator)
            .build();

individualSuccessReportItemWriter() is as below.

    FlatFileItemWriter&lt;SuccessReport&gt; itemWriter = new FlatFileItemWriter&lt;&gt;();
    itemWriter.setName(&quot;individualSuccessReportItemWriter&quot;);
    itemWriter.setHeaderCallback(new SuccesssReportHeaderCallback());
    itemWriter.setLineAggregator(new SuccessReportLineAggregator());

After the Success step generates the SuccessReport , the Fallout step will query from the DB and repeat the above to create the FalloutReport .csv files for each Email Target, again using MultiResourceItemWriter.

The goal is to be able to email each Email target the Success Report and the Fallout Report .csv files as attachment. Suppose, there are 25 Email Targets. There will be 25 Success .CSV files and 25 Failure .CSV files generated as a result of running the 2 Steps (Success and Fallout). Each Email target will get 1 success and 1 fallout .csv file as attachment.

The class SuccessReport and FalloutReport has the email Target while it generates the .csv file -- however unable to name the files as such because the suffix Creator does not allow for naming them accordingly.

答案1

得分: 3

我想出了一个我认为有些粗糙的方法,但希望能有所帮助。根据您的示例代码,似乎您正在为每个“List”元素使用大小为1的块大小,并为每个元素使用单独的输出文件(这对应于您对每个报告分配一个目标电子邮件的描述)。在这种情况下,以下建议似乎是足够安全的。

如果您的ResourceSuffixCreator也是ItemWriteListener,您可以将目标电子邮件提取到字段变量中,随后您可以用于后续的后缀生成。如果您对目标电子邮件执行了某种哈希(MD5),那么您可以期望相同的哈希将出现在发送到给定电子邮件目标的每个报告中。

例如:

public class SuccessReportSuffixCreator 
    implements ItemWriteListener<SuccessReport>, ResourceSuffixCreator {

    private String emailTarget = "";

    @Override
    public String getSuffix(int i) {
        return Md5Crypt.md5Crypt(this.emailTarget.getBytes()) + "." + i + ".csv";
    }

    @Override
    public void beforeWrite(List<? extends SuccessReport> list) {
        this.emailTarget = list.get(0).getEmailTarget();
    }

    @Override
    public void afterWrite(List<? extends SuccessReport> list) {
        // 清除 emailTarget?
    }

    @Override
    public void onWriteError(Exception e, List<? extends SuccessReport> list) {
        // 清除 emailTarget?
    }
}

我不知道您的两种报告类型的类层次结构是什么样的,所以您可能需要为每种报告类型创建两个不同的类,或者为具有电子邮件目标数据的父类型创建单个类。

您还需要确保在“Step”上注册此ItemWriteListener,以便可以适当地调用其事件处理程序。

希望这对您有用,或者为您解决问题提供了替代思路。

英文:

I've come up with what I feel to be a somewhat crude hack, but hopefully it will help. From your example code, it appears that you are using a chunk size of 1 and a separate output file for each List element (which corresponds to your description of one target email to each report). In that case, the following seems safe enough to suggest.

If your ResourceSuffixCreator were also an ItemWriteListener, you could extract the target email into a field variable that you could subsequently use for suffix generation. If you performed a hash of some kind (MD5) on the target email, then you could expect the same hash to be present for each of the reports destined to a given email target.

For example:

public class SuccessReportSuffixCreator 
    implements ItemWriteListener&lt;SuccessReport&gt;, ResourceSuffixCreator {

    private String emailTarget = &quot;&quot;;

    @Override
    public String getSuffix( int i ) {
        return Md5Crypt.md5Crypt( this.emailTarget.getBytes() ) + &quot;.&quot; + i + &quot;.csv&quot;;
    }

    @Override
    public void beforeWrite( List&lt;? extends SuccessReport&gt; list ) {
        this.emailTarget = list.get( 0 ).getEmailTarget();
    }

    @Override
    public void afterWrite( List&lt;? extends SuccessReport&gt; list ) {
        // clear emailTarget?
    }

    @Override
    public void onWriteError( Exception e, List&lt;? extends SuccessReport&gt; list ) {
        // clear emailTarget?
    }
}

I don't know what your class hierarchy looks like for the two report types, so you'll either need two distinct classes for each report type or a single class for a parent type that has the email target data.

You will also need to make sure and register this ItemWriteListener on the Step so its event handlers will be called appropriately.

I hope this works for you or gives you an alternative idea for how to solve your problem.

huangapple
  • 本文由 发表于 2020年8月31日 23:23:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/63673616.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定