Spring Batch中ClassifierCompositeItemWriter的用法

huangapple go评论71阅读模式
英文:

Usage of ClassifierCompositeItemWriter in Spring Batch

问题

我有一个传统的数据库,其中的数据存储为CLOB列中的JSON。需要将CLOB中的JSON数据规范化并放入相应的表中,以支持新的解决方案。

CLOB数据将生成1个公司记录,多个部门记录,每个部门都可能有任意数量的员工。所有这些数据都要持久化到3个表中,即公司、部门和员工。我希望只读取一次源数据,然后在一个Spring Batch的项目写入器类中执行“数据拆分”操作。

在这种情况下,哪种项目写入器更适合,ClassifierCompositeItemWriter还是CompositeItemWriter?

处理器似乎很简单:

public class ClientProfileEVProcessor implements ItemProcessor<ClientProfile, ClientCompositeVO> {
    
    @Override
    public ClientCompositeVO process(ClientProfile item) throws Exception {
        ClientEntity client = populateClientProfile(item);
        List<DeptEntity> deptEntities = populateDept(item);
        List<Employee> evEntities = populateEmployee(item);
        return new ClientCompositeVO(client, deptEntities, evEntities);
    }
}

以下是我的分类器尝试。我得到了编译错误 clazz instanceof List。在下面的代码中,我应该如何检查 clazz是List.get(0).class的实例

public class ClientClassifier<T> implements Classifier<T, ItemWriter<? super T>> {
    
    private final Map<Class<T>, ItemWriter<T>> itemWriterMap;
    
    public EVProfileClassifier(Map<Class<T>, ItemWriter<T>> itemWriterMap) {
        super();
        this.itemWriterMap = itemWriterMap;
    }

    @Override
    public ItemWriter<? super T> classify(T classifiable) {
        Class<?> clazz = classifiable.getClass();
        if (this.itemWriterMap.containsKey(clazz)) {
            return this.itemWriterMap.get(clazz);
        } else if (clazz instanceof List) {
            List<?> list = (List<?>) classifiable;
            return this.itemWriterMap.get(list.get(0).getClass());
        } else {
            throw new IllegalArgumentException("No writer found for domain class: " + clazz.getTypeName());
        }
    }

    public Map<Class<T>, ItemWriter<T>> getItemWriterMap() {
        return itemWriterMap;
    }
}

任何简化解决方案的建议也欢迎。

英文:

I have a legacy database which has data stored as JSON in clob columns. The JSON in clob needs to be normalized and put into respective tables to support a new solution.

The clob data will produce 1 Company record, multiple department records and each department will have any number of employees. All this data is to be persisted into 3 tables i.e. Company, Department & Employee. I want to read the source data only once and perform the "data split" operation in one item-writer class using Spring Batch

Which item-writer will be more appropriate for this use case ClassifierCompositeItemWriter or CompositeItemWriter?

The processor seems to be simple:

public class ClientProfileEVProcessor implements ItemProcessor&lt;ClientProfile, ClientCompositeVO&gt; {

	/**
	 * 
	 */
	@Override
	public ClientCompositeVO process(ClientProfile item) throws Exception {
		ClientEntity client = populateClientProfile(item);
		List&lt;DeptEntity&gt; deptEntities = populateDept(item);
		List&lt;Employee&gt; evEntities = populateEmployee(item);
		return new ClientCompositeVO(client, deptEntities, evEntities);
	}
}

Here is my attempt of the Classifier. I am getting compilation error clazz instanceof List. How should I check if the clazz instance of list.get(0).class in the below code?

    public class ClientClassifier&lt;T&gt; implements Classifier&lt;T, ItemWriter&lt;? super T&gt;&gt; {
    
	private final Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; itemWriterMap;
	
	public EVProfileClassifier(Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; itemWriterMap) {
		super();
		this.itemWriterMap = itemWriterMap;
	}

	@Override
	public ItemWriter&lt;? super T&gt; classify(T classifiable) {
		Class&lt;?&gt; clazz = classifiable.getClass();
		if (this.itemWriterMap.containsKey(clazz)) {
			return this.itemWriterMap.get(clazz);
		} else if(clazz instanceof List) {
			List&lt;?&gt; list = (List&lt;?&gt;) classifiable;
			return this.itemWriterMap.get(list.get(0).getClass());
		} else {
			throw new IllegalArgumentException(&quot;No writer found for domainn class: &quot; + clazz.getTypeName());
		}
	}

	public Map&lt;Class&lt;T&gt;, ItemWriter&lt;T&gt;&gt; getItemWriterMap() {
		return itemWriterMap;
	}
	
}

Spring Batch中ClassifierCompositeItemWriter的用法

Any suggestions for a simpler solution approach are also welcome.

答案1

得分: 0

以下是翻译好的部分:

以下解决方案有效。我的领域类扩展自一个通用的 BaseEntity。我为三个实体编写了写入器,即 Company、Department 和 Employee。

我编写了一个委托写入器,如下所示:

public class DelegateItemWriter implements ItemWriter<List<CCTBaseEntity>> {
    
    private final LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap;
    
    public DelegateItemWriter(
        LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
        this.itemWriterMap = itemWriterMap;
    }
    
    @Override
    public void write(Chunk<? extends List<BaseEntity>> chunk) throws Exception {
        List<BaseEntity> listEntities = chunk.getItems().get(0);
        for (BaseEntity b : listEntities) {
            this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
        }
    }
}

itemWriterMap 和 DelegateItemWriter 的 Bean 定义如下:

@Bean
public LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap(MongoTemplate mongoTemplate) {
    LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap = new LinkedHashMap<>();
    itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
    itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
    itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
    return itemWriterMap;
}

@Bean
public DelegateItemWriter delegateItemWriter(LinkedHashMap<Class<? extends BaseEntity>, ItemWriter<? extends BaseEntity>> itemWriterMap) {
    return new DelegateItemWriter(itemWriterMap);
}

步骤定义:

@Bean
public Step step1(@Qualifier(value = "dataSource") final DataSource dataSource,
    final JobRepository jobRepository, 
    final PlatformTransactionManager transactionManager,
    final DelegateItemWriter delegateItemWriter) {
    return new StepBuilder("step1", jobRepository).<ClientProfile, List<BaseEntity>>chunk(10, transactionManager)
        .reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
        .build();
}
英文:

the following solution works. My domain classes extend from a common BaseEntity. I wrote writers for the 3 entities i.e., Company, Department & Employee

I wrote a Delegate Writer as follows:

public class DelegateItemWriter implements ItemWriter&lt;List&lt;CCTBaseEntity&gt;&gt; {

	private final LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap;

	public DelegateItemWriter(
			LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap) {
		this.itemWriterMap = itemWriterMap;
	}

	@Override
	public void write(Chunk&lt;? extends List&lt;BaseEntity&gt;&gt; chunk) throws Exception {
		List&lt;BaseEntity&gt; listEntities = chunk.getItems().get(0);
		for (BaseEntity b : listEntities) {
			this.itemWriterMap.get(b.getClass()).write(new Chunk(Arrays.asList(b)));
		}
	}
}

Bean definitions for itemWriterMap & DelegateItemWriter:

@Bean
public LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap(MongoTemplate mongoTemplate) {
	LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap = new LinkedHashMap&lt;&gt;();
	itemWriterMap.put(CompanyEntity.class, companyItemWriter(mongoTemplate));
	itemWriterMap.put(DepartmentEntity.class, deptEntityWriter(mongoTemplate));
	itemWriterMap.put(Employee.class, employeeItemWriter(mongoTemplate));
	return itemWriterMap;
}

@Bean
public DelegateItemWriter delegateItemWriter(LinkedHashMap&lt;Class&lt;? extends BaseEntity&gt;, ItemWriter&lt;? extends BaseEntity&gt;&gt; itemWriterMap) {
	return new DelegateItemWriter(itemWriterMap);
}

Step Definition:

@Bean
	public Step step1(@Qualifier(value = &quot;dataSource&quot;) final DataSource dataSource,
			final JobRepository jobRepository, 
			final PlatformTransactionManager transactionManager,
			final DelegateItemWriter delegateItemWriter) {
		return new StepBuilder(&quot;step1&quot;, jobRepository).&lt;ClientProfile, List&lt;BaseEntity&gt;&gt;chunk(10, transactionManager)
				.reader(reader(dataSource)).processor(profileProcessor()).writer(delegateItemWriter)
				.build();
	}

huangapple
  • 本文由 发表于 2023年7月12日 23:21:02
  • 转载请务必保留本文链接:https://go.coder-hub.com/76672159.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定