Java: Flat List<Map<String, Object>> to Hierarchical List<Map<String, Object>>

This issue seems to be pretty complex so here I'm posting this issue for any possible way to solve this.

I have list of Maps. I want again a list of map, but making sure the map is converted into some sort of hierarchy.

Original data: (List<Map<String, Object>>)

[
{
"studentId": 101,
"name": "John",
"subjectId": 2001,
"marks": 85,
"street": "Bakers Street",
"state": "LA"
},
{
"studentId": 101,
"name": "John",
"subjectId": 2002,
"marks": 75,
"street": "Bakers Street",
"state": "LA"
},
{
"studentId": 102,
"name": "Shae",
"subjectId": 3001,
"marks": 96,
"street": "Howards",
"state": "NYC"
}
]

This list of map to be converted to below list of map: (List<Map<String, Object>>)

[
{
"studentId": 101,
"name": "John",
"academics":
[
{
"subjectId": 2001,
"marks": 85
},
{
"subjectId": 2002,
"marks": 75
}
],
"address":
{
"street": "Bakers Street",
"state": "LA"
}
},
{
"studentId": 102,
"name": "Shae",
"academics":
[
{
"subjectId": 3001,
"marks": 96
}
],
"address":
{
"street": "Howards",
"state": "NYC"
}
}
]

As a naive solution I tried to process them manually (really boring), so looking for any efficient and clean way of doing it using streams or any other possible ways.

UPDATE
The naive solution is as below

public List<Map<String, Object>> transformResultSet(List<Map<String, Object>> flatDataList) {
List<Map<String, Object>> hierarchicalDataList = new ArrayList<Map<String, Object>>();
Map<String, List<Map<String, Object>>> studentIdToStudentDataListMap = new LinkedHashMap<>();
for (Map<Integer, Object> flatData : flatDataList) {
if (studentIdToStudentDataListMap.get(flatData.get("student_id")) == null) {
studentIdToStudentDataListMap.put(Integer.valueOf(flatData.get("student_id").toString()), new ArrayList<Map<String, Object>>());
}
studentIdToStudentDataListMap.get(Integer.valueOf(flatData.get("student_id").toString())).add(flatData);
}
for (Map.Entry<Integer, List<Map<String, Object>>> studentFlatDataList : studentIdToStudentDataListMap.entrySet()) {
Map<String, Object> studentHierarchicalDataMap = new LinkedHashMap<String, Object>();
Map<String, Object> studentFlatDataMap = studentFlatDataList.getValue().get(0);
studentHierarchicalDataMap.put("studentId", studentFlatDataMap.get("studentId"));
studentHierarchicalDataMap.put("name", studentFlatDataMap.get("name"));
List<Map<String, Object>> academicsList = new ArrayList<Map<String, Object>>();
for (Map<String, Object> studentDetailAcademic : studentFlatDataList.getValue()) {
Map<String, Object> academic = new LinkedHashMap<String, Object>();
academic.put("subjectId", studentDetailAcademic.get("subjectId"));
academic.put("marks", studentDetailAcademic.get("marks"));
academicsList.add(academic);
}
studentHierarchicalDataMap.put("academics", academicsList);
Map<String, Object> address = new LinkedHashMap<String, Object>();
address.put("street", studentFlatDataMap.get("street"));
address.put("state", studentFlatDataMap.get("state"));
studentHierarchicalDataMap.put("address", address);
hierarchicalDataList.add(studentHierarchicalDataMap);
}
return hierarchicalDataList;
}

From your json sample it seems that you have List<Object> not List<Map<String, Object>>.
So, just to give you an idea create 2 objects, let's say StudentDto and MarkDto.

Assuming that the input object is Student and StudentDto having List<MarkDto> as a member:

Map<String, List<Student>>  map = list.stream().collect(groupingBy(Student::studentId)); 
Map<String, StudentDto>  dtoMap = new HashMap<>();
for(Map.Entry<String, List<Student>> entry : map.entrySet()) {
StudentDto stud = new StudentDto();
//assign other studentDto properties
for(Student std : entry.getValue()) {
MarkDto mark = new MarkDto();
mark.setSubjectId(std.getStudentid());
mark.setMark(entry.getMark()));
stud.add(mark);
}
dtoMap.put(String.valueOf(stud.getId()), stud);
}
return dtoMap.stream().collect(Collectors.toList()); // or return the map itself

You can split your algorithm on a few steps:

1. Extract subjectId, marks to a new academics Map.
2. Extract street, state to a new address Map.
3. Wrap academics Map with a List.
4. Merge data by studentId. When collision occurs, we need to merge academics List.

Steps 1. and 2. are the same except key names. We can extract it to a new class to avoid duplication of method references:

class ExtractKeysToMap implements Function<Map<String, Object>, Map<String, Object>> {
private final List<String> keys;
private final String newKey;
ExtractKeysToMap(String newKey, List<String> keys) {
this.newKey = Objects.requireNonNull(newKey);
this.keys = Objects.requireNonNull(keys);
}
@Override
public Map<String, Object> apply(Map<String, Object> map) {
Map<String, Object> academics = new HashMap<>();
keys.forEach(key -> {
Object value = map.remove(key);
if (value != null) academics.put(key, value);
});
map.put(newKey, academics);
return map;
}
}

Since we have first and second step implemented we can use it in below example:

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.type.CollectionType;
import java.io.File;
import java.util.Arrays;
import java.util.Collection;
import java.util.Collections;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Objects;
import java.util.function.Function;
import java.util.stream.Collectors;
public class JsonApp {
public static void main(String[] args) throws Exception {
File jsonFile = new File("./src/main/resources/test.json");
ObjectMapper mapper = new ObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT);
CollectionType jsonType = mapper.getTypeFactory().constructCollectionType(List.class, Map.class);
List<Map<String, Object>> response = mapper.readValue(jsonFile, jsonType);
final String academicsKey = "academics";
Collection<Map<String, Object>> result = response
.stream()
.map(new ExtractKeysToMap(academicsKey, Arrays.asList("subjectId", "marks")))
.map(new ExtractKeysToMap("address", Arrays.asList("street", "state")))
.peek(map -> map.computeIfPresent(academicsKey, (k, v) -> new LinkedList<>(Collections.singletonList(v))))
.collect(Collectors.toMap(
map -> map.get("studentId"),
map -> map,
(map0, map1) -> {
((List<Object>) map0.get(academicsKey)).addAll((List<Object>) map1.get(academicsKey));
return map0;
}))
.values();
mapper.writeValue(System.out, result);
}
}

Above code prints:

[ {
"studentId" : 101,
"name" : "John",
"academics" : [ {
"subjectId" : 2001,
"marks" : 85
}, {
"subjectId" : 2002,
"marks" : 75
} ],
"address" : {
"street" : "Bakers Street",
"state" : "LA"
}
}, {
"studentId" : 102,
"name" : "Shae",
"academics" : [ {
"subjectId" : 3001,
"marks" : 96
} ],
"address" : {
"street" : "Howards",
"state" : "NYC"
}
} ]

It seems you need to group by student, while also changing the json structure.

Higher-level, you could do this (we'll see the details later):

Map<Integer, Map<String, Object>> grouped = flatDataList.stream()
.collect(Collectors.toMap(
s -> (Integer) s.get("studentId"),
s -> transformToHierarchicalStudent(s),
(oldS, newS) -> mergeHierarchicalStudents(oldS, newS)));

So, this creates a map of students with hierarchical format, grouped by studentId. We are delegating to two methods: one that creates a hierarchical student out from a flat student and another one that will merge two hierarchical students that have the same studentId.

The transformToHierarchicalStudent method would be as follows:

Map<String, Object> transformToHierarchicalStudent(Map<String, Object> flat) {
Map<String, Object> student = new LinkedHashMap<>();
student.put("studentId", flat.get("studentId"));
student.put("name", flat.get("name"));
Map<String, Object> address = new LinkedHashMap<>();
address.put("street", flat.get("street"));
address.put("state", flat.get("state"));
student.put("address", address);
List<Map<String, Object>> academics = new ArrayList<>();
Map<String, Object> subject = new LinkedHashMap<>();
subject.put("subjectId", flat.get("subjectId"));
subject.put("marks", flat.get("marks"));
academics.add(subject);
student.put("academics", academics);
return student;
}

And the mergeHierarchicalStudents method:

Map<String, Object> mergeHierarchicalStudents(
Map<String, Object> oldSt, Map<String, Object> newSt) {
// We only need to merge the subjects
List<Map<String, Object>> oldAcademics = 
(List<Map<String, Object>>) oldSt.get("academics");
List<Map<String, Object>> newAcademics = 
(List<Map<String, Object>>) newSt.get("academics");
oldAcademcis.addAll(newAcademics);
return oldS;
}

This assumes there are no duplicate subjects for the same student in the original flat list.

Finally, if you need a List of hierarchical students, just grab the map values:

List<Map<String, Object>> hierarchicalStudents = new ArrayList<>(grouped.values());