Spring Batch中ClassifierCompositeItemWriter的用法

huangapple
go评论106阅读模式
英文:

Usage of ClassifierCompositeItemWriter in Spring Batch

问题

我有一个传统的数据库，其中的数据存储为CLOB列中的JSON。需要将CLOB中的JSON数据规范化并放入相应的表中，以支持新的解决方案。

CLOB数据将生成1个公司记录，多个部门记录，每个部门都可能有任意数量的员工。所有这些数据都要持久化到3个表中，即公司、部门和员工。我希望只读取一次源数据，然后在一个Spring Batch的项目写入器类中执行“数据拆分”操作。

在这种情况下，哪种项目写入器更适合，ClassifierCompositeItemWriter还是CompositeItemWriter？

处理器似乎很简单：

  1. public class ClientProfileEVProcessor implements ItemProcessor<ClientProfile, ClientCompositeVO> {
  2.     
  3.     @Override
  4.     public ClientCompositeVO process(ClientProfile item) throws Exception {
  5.         ClientEntity client = populateClientProfile(item);
  6.         List<DeptEntity> deptEntities = populateDept(item);
  7.         List<Employee> evEntities = populateEmployee(item);
  8.         return new ClientCompositeVO(client, deptEntities, evEntities);
  9.     }
  10. }

以下是我的分类器尝试。我得到了编译错误 clazz instanceof List。在下面的代码中，我应该如何检查 clazz是List.get(0).class的实例？

  1. public class ClientClassifier<T> implements Classifier<T, ItemWriter<? super T>> {
  2.     
  3.     private final Map<Class<T>, ItemWriter<T>> itemWriterMap;
  4.     
  5.     public EVProfileClassifier(Map<Class<T>, ItemWriter<T>> itemWriterMap) {
  6.         super();
  7.         this.itemWriterMap = itemWriterMap;
  8.     }
  10.     @Override
  11.     public ItemWriter<? super T> classify(T classifiable) {
  12.         Class<?> clazz = classifiable.getClass();
  13.         if (this.itemWriterMap.containsKey(clazz)) {
  14.             return this.itemWriterMap.get(clazz);
  15.         } else if (clazz instanceof List) {
  16.             List<?> list = (List<?>) classifiable;
  17.             return this.itemWriterMap.get(list.get(0).getClass());
  18.         } else {
  19.             throw new IllegalArgumentException("No writer found for domain class: " + clazz.getTypeName());
  20.         }
  21.     }
  23.     public Map<Class<T>, ItemWriter<T>> getItemWriterMap() {
  24.         return itemWriterMap;
  25.     }
  26. }

任何简化解决方案的建议也欢迎。

英文:

I have a legacy database which has data stored as JSON in clob columns. The JSON in clob needs to be normalized and put into respective tables to support a new solution.

The clob data will produce 1 Company record, multiple department records and each department will have any number of employees. All this data is to be persisted into 3 tables i.e. Company, Department & Employee. I want to read the source data only once and perform the "data split" operation in one item-writer class using Spring Batch

Which item-writer will be more appropriate for this use case ClassifierCompositeItemWriter or CompositeItemWriter?

The processor seems to be simple:

  1. public class ClientProfileEVProcessor implements ItemProcessor&lt;ClientProfile, ClientCompositeVO&gt; {
  3. 	/**
  4. 	 * 
  5. 	 */
  6. 	@Override
  7. 	public ClientCompositeVO process(ClientProfile item) throws Exception {
  8. 		ClientEntity client = populateClientProfile(item);
  9. 		List&lt;DeptEntity&gt; deptEntities = populateDept(item);
  10. 		List&lt;Employee&gt; evEntities = populateEmployee(item);
  11. 		return new ClientCompositeVO(client, deptEntities, evEntities);
  12. 	}
  13. }

Here is my attempt of the Classifier. I am getting compilation error clazz instanceof List. How should I check if the clazz instance of list.get(0).class in the below code?

  1.     public class ClientClassifier&lt;T&gt; implements Classifier&lt;T, ItemWriter&lt;? super T&gt;&gt; {
  2.     
  3. 	private final Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; itemWriterMap;
  4. 	
  5. 	public EVProfileClassifier(Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; itemWriterMap) {
  6. 		super();
  7. 		this.itemWriterMap = itemWriterMap;
  8. 	}
  10. 	@Override
  11. 	public ItemWriter&lt;? super T&gt; classify(T classifiable) {
  12. 		Class&lt;?&gt; clazz = classifiable.getClass();
  13. 		if (this.itemWriterMap.containsKey(clazz)) {
  14. 			return this.itemWriterMap.get(clazz);
  15. 		} else if(clazz instanceof List) {
  16. 			List&lt;?&gt; list = (List&lt;?&gt;) classifiable;
  17. 			return this.itemWriterMap.get(list.get(0).getClass());
  18. 		} else {
  19. 			throw new IllegalArgumentException(&quot;No writer found for domainn class: &quot; + clazz.getTypeName());
  20. 		}
  21. 	}
  23. 	public Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; getItemWriterMap() {
  24. 		return itemWriterMap;
  25. 	}
  26. 	
  27. }

Spring Batch中ClassifierCompositeItemWriter的用法

Any suggestions for a simpler solution approach are also welcome.

答案1

得分: 0

以下是翻译好的部分：

以下解决方案有效。我的领域类扩展自一个通用的 BaseEntity。我为三个实体编写了写入器，即 Company、Department 和 Employee。

我编写了一个委托写入器，如下所示：

  1. public class DelegateItemWriter implements ItemWriter<List<CCTBaseEntity>> {
  2.     
  3.     private final LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap;
  4.     
  5.     public DelegateItemWriter(
  6.         LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
  7.         this.itemWriterMap = itemWriterMap;
  8.     }
  9.     
  10.     @Override
  11.     public void write(Chunk<? extends List<BaseEntity>> chunk) throws Exception {
  12.         List<BaseEntity> listEntities = chunk.getItems().get(0);
  13.         for (BaseEntity b : listEntities) {
  14.             this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
  15.         }
  16.     }
  17. }

itemWriterMap 和 DelegateItemWriter 的 Bean 定义如下：

  1. @Bean
  2. public LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap(MongoTemplate mongoTemplate) {
  3.     LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap = new LinkedHashMap<>();
  4.     itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
  5.     itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
  6.     itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
  7.     return itemWriterMap;
  8. }
  10. @Bean
  11. public DelegateItemWriter delegateItemWriter(LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
  12.     return new DelegateItemWriter(itemWriterMap);
  13. }

步骤定义：

  1. @Bean
  2. public Step step1(@Qualifier(value = "dataSource") final DataSource dataSource,
  3.     final JobRepository jobRepository, 
  4.     final PlatformTransactionManager transactionManager,
  5.     final DelegateItemWriter delegateItemWriter) {
  6.     return new StepBuilder("step1", jobRepository).<ClientProfile, List<BaseEntity>>chunk(10, transactionManager)
  7.         .reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
  8.         .build();
  9. }
英文:

the following solution works. My domain classes extend from a common BaseEntity. I wrote writers for the 3 entities i.e., Company, Department & Employee

I wrote a Delegate Writer as follows:

  1. public class DelegateItemWriter implements ItemWriter&lt;List&lt;CCTBaseEntity&gt;&gt; {
  3. 	private final LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap;
  5. 	public DelegateItemWriter(
  6. 			LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap) {
  7. 		this.itemWriterMap = itemWriterMap;
  8. 	}
  10. 	@Override
  11. 	public void write(Chunk&lt;? extends List&lt;BaseEntity&gt;&gt; chunk) throws Exception {
  12. 		List&lt;BaseEntity&gt; listEntities = chunk.getItems().get(0);
  13. 		for (BaseEntity b : listEntities) {
  14. 			this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
  15. 		}
  16. 	}
  17. }

Bean definitions for itemWriterMap & DelegateItemWriter:

  1. @Bean
  2. public LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap(MongoTemplate mongoTemplate) {
  3. 	LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap = new LinkedHashMap&lt;&gt;();
  4. 	itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
  5. 	itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
  6. 	itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
  7. 	return itemWriterMap;
  8. }
  10. @Bean
  11. public DelegateItemWriter delegateItemWriter(LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap) {
  12. 	return new DelegateItemWriter(itemWriterMap);
  13. }

Step Definition:

  1. @Bean
  2. 	public Step step1(@Qualifier(value = &quot;dataSource&quot;) final DataSource dataSource,
  3. 			final JobRepository jobRepository, 
  4. 			final PlatformTransactionManager transactionManager,
  5. 			final DelegateItemWriter delegateItemWriter) {
  6. 		return new StepBuilder(&quot;step1&quot;, jobRepository).&lt;ClientProfile, List&lt;BaseEntity&gt;&gt;chunk(10, transactionManager)
  7. 				.reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
  8. 				.build();
  9. 	}

huangapple
  • 本文由 发表于 2023年7月12日 23:21:02
  • 转载请务必保留本文链接：https://go.coder-hub.com/76672159.html
  • java
  • spring
  • spring-batch
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定