英文:
Usage of ClassifierCompositeItemWriter in Spring Batch
问题
我有一个传统的数据库,其中的数据存储为CLOB列中的JSON。需要将CLOB中的JSON数据规范化并放入相应的表中,以支持新的解决方案。
CLOB数据将生成1个公司记录,多个部门记录,每个部门都可能有任意数量的员工。所有这些数据都要持久化到3个表中,即公司、部门和员工。我希望只读取一次源数据,然后在一个Spring Batch的项目写入器类中执行“数据拆分”操作。
在这种情况下,哪种项目写入器更适合,ClassifierCompositeItemWriter还是CompositeItemWriter?
处理器似乎很简单:
public class ClientProfileEVProcessor implements ItemProcessor<ClientProfile, ClientCompositeVO> {
@Override
public ClientCompositeVO process(ClientProfile item) throws Exception {
ClientEntity client = populateClientProfile(item);
List<DeptEntity> deptEntities = populateDept(item);
List<Employee> evEntities = populateEmployee(item);
return new ClientCompositeVO(client, deptEntities, evEntities);
}
}
以下是我的分类器尝试。我得到了编译错误 clazz instanceof List。在下面的代码中,我应该如何检查 clazz是List.get(0).class的实例?
public class ClientClassifier<T> implements Classifier<T, ItemWriter<? super T>> {
private final Map<Class<T>, ItemWriter<T>> itemWriterMap;
public EVProfileClassifier(Map<Class<T>, ItemWriter<T>> itemWriterMap) {
super();
this.itemWriterMap = itemWriterMap;
}
@Override
public ItemWriter<? super T> classify(T classifiable) {
Class<?> clazz = classifiable.getClass();
if (this.itemWriterMap.containsKey(clazz)) {
return this.itemWriterMap.get(clazz);
} else if (clazz instanceof List) {
List<?> list = (List<?>) classifiable;
return this.itemWriterMap.get(list.get(0).getClass());
} else {
throw new IllegalArgumentException("No writer found for domain class: " + clazz.getTypeName());
}
}
public Map<Class<T>, ItemWriter<T>> getItemWriterMap() {
return itemWriterMap;
}
}
任何简化解决方案的建议也欢迎。
英文:
I have a legacy database which has data stored as JSON in clob columns. The JSON in clob needs to be normalized and put into respective tables to support a new solution.
The clob data will produce 1 Company record, multiple department records and each department will have any number of employees. All this data is to be persisted into 3 tables i.e. Company, Department & Employee. I want to read the source data only once and perform the "data split" operation in one item-writer class using Spring Batch
Which item-writer will be more appropriate for this use case ClassifierCompositeItemWriter or CompositeItemWriter?
The processor seems to be simple:
public class ClientProfileEVProcessor implements ItemProcessor<ClientProfile, ClientCompositeVO> {
/**
*
*/
@Override
public ClientCompositeVO process(ClientProfile item) throws Exception {
ClientEntity client = populateClientProfile(item);
List<DeptEntity> deptEntities = populateDept(item);
List<Employee> evEntities = populateEmployee(item);
return new ClientCompositeVO(client, deptEntities, evEntities);
}
}
Here is my attempt of the Classifier. I am getting compilation error clazz instanceof List. How should I check if the clazz instance of list.get(0).class in the below code?
public class ClientClassifier<T> implements Classifier<T, ItemWriter<? super T>> {
private final Map<Class<T>, ItemWriter<T>> itemWriterMap;
public EVProfileClassifier(Map<Class<T>, ItemWriter<T>> itemWriterMap) {
super();
this.itemWriterMap = itemWriterMap;
}
@Override
public ItemWriter<? super T> classify(T classifiable) {
Class<?> clazz = classifiable.getClass();
if (this.itemWriterMap.containsKey(clazz)) {
return this.itemWriterMap.get(clazz);
} else if(clazz instanceof List) {
List<?> list = (List<?>) classifiable;
return this.itemWriterMap.get(list.get(0).getClass());
} else {
throw new IllegalArgumentException("No writer found for domainn class: " + clazz.getTypeName());
}
}
public Map<Class<T>, ItemWriter<T>> getItemWriterMap() {
return itemWriterMap;
}
}
Any suggestions for a simpler solution approach are also welcome.
答案1
得分: 0
以下是翻译好的部分:
以下解决方案有效。我的领域类扩展自一个通用的 BaseEntity。我为三个实体编写了写入器,即 Company、Department 和 Employee。
我编写了一个委托写入器,如下所示:
public class DelegateItemWriter implements ItemWriter<List<CCTBaseEntity>> {
private final LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap;
public DelegateItemWriter(
LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
this.itemWriterMap = itemWriterMap;
}
@Override
public void write(Chunk<? extends List<BaseEntity>> chunk) throws Exception {
List<BaseEntity> listEntities = chunk.getItems().get(0);
for (BaseEntity b : listEntities) {
this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
}
}
}
itemWriterMap 和 DelegateItemWriter 的 Bean 定义如下:
@Bean
public LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap(MongoTemplate mongoTemplate) {
LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap = new LinkedHashMap<>();
itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
return itemWriterMap;
}
@Bean
public DelegateItemWriter delegateItemWriter(LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
return new DelegateItemWriter(itemWriterMap);
}
步骤定义:
@Bean
public Step step1(@Qualifier(value = "dataSource") final DataSource dataSource,
final JobRepository jobRepository,
final PlatformTransactionManager transactionManager,
final DelegateItemWriter delegateItemWriter) {
return new StepBuilder("step1", jobRepository).<ClientProfile, List<BaseEntity>>chunk(10, transactionManager)
.reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
.build();
}
英文:
the following solution works. My domain classes extend from a common BaseEntity. I wrote writers for the 3 entities i.e., Company, Department & Employee
I wrote a Delegate Writer as follows:
public class DelegateItemWriter implements ItemWriter<List<CCTBaseEntity>> {
private final LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap;
public DelegateItemWriter(
LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
this.itemWriterMap = itemWriterMap;
}
@Override
public void write(Chunk<? extends List<BaseEntity>> chunk) throws Exception {
List<BaseEntity> listEntities = chunk.getItems().get(0);
for (BaseEntity b : listEntities) {
this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
}
}
}
Bean definitions for itemWriterMap & DelegateItemWriter:
@Bean
public LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap(MongoTemplate mongoTemplate) {
LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap = new LinkedHashMap<>();
itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
return itemWriterMap;
}
@Bean
public DelegateItemWriter delegateItemWriter(LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
return new DelegateItemWriter(itemWriterMap);
}
Step Definition:
@Bean
public Step step1(@Qualifier(value = "dataSource") final DataSource dataSource,
final JobRepository jobRepository,
final PlatformTransactionManager transactionManager,
final DelegateItemWriter delegateItemWriter) {
return new StepBuilder("step1", jobRepository).<ClientProfile, List<BaseEntity>>chunk(10, transactionManager)
.reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
.build();
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论