如何将一个未标准化的CSV文件转换为一个复杂的JSON或Java对象

huangapple
go评论99阅读模式
英文:

How to change an unnormalized csv file to a complex json or java object

问题

我有以下未规范化的 CSV 文件：

  1. user_id,nickname,joinDate,product_id,price
  2. 1,kmh,2023-07-24,P131,3000
  3. 1,kmh,2023-07-24,P132,4000
  4. 1,kmh,2023-07-24,P133,7000
  5. 1,kmh,2023-07-24,P134,9000
  6. 2,john,2023-07-24,P135,2500
  7. 2,john,2023-07-24,P136,6000
  8. 3,alice,2023-07-25,P137,4500
  9. 3,alice,2023-07-25,P138,8000

我将把它转换为以下的 JSON 格式（或 Java 对象）：

  1. [
  2.     {
  3.         "user_id": 1,
  4.         "nickname": "kmh",
  5.         "joinDate": "2023-07-24",
  6.         "orders": [
  7.             {
  8.                 "product_id": "P131",
  9.                 "price": 3000
  10.             },
  11.             {
  12.                 "product_id": "P132",
  13.                 "price": 4000
  14.             },
  15.             {
  16.                 "product_id": "P133",
  17.                 "price": 7000
  18.             },
  19.             {
  20.                 "product_id": "P134",
  21.                 "price": 9000
  22.             }
  23.         ]
  24.     },
  25.     {
  26.         "user_id": 2,
  27.         "nickname": "john",
  28.         "joinDate": "2023-07-24",
  29.         "orders": [
  30.             {
  31.                 "product_id": "P135",
  32.                 "price": 2500
  33.             },
  34.             {
  35.                 "product_id": "P136",
  36.                 "price": 6000
  37.             }
  38.         ]
  39.     },
  40.     {
  41.         "user_id": 3,
  42.         "nickname": "alice",
  43.         "joinDate": "2023-07-25",
  44.         "orders": [
  45.             {
  46.                 "product_id": "P137",
  47.                 "price": 4500
  48.             },
  49.             {
  50.                 "product_id": "P138",
  51.                 "price": 8000
  52.             }
  53.         ]
  54.     }
  55. ]

我已经搜索了相当长的时间，但没有找到可以实现这个功能的库或工具。

我有很多不同类型的 CSV 文件，我需要工具或库来转换所有这些。是否有任何可以实现这一目标的库或工具？

英文:

I have the following unnormalized csv file

  1. user_id,nickname,joinDate,product_id,price
  2. 1,kmh,2023-07-24,P131,3000
  3. 1,kmh,2023-07-24,P132,4000
  4. 1,kmh,2023-07-24,P133,7000
  5. 1,kmh,2023-07-24,P134,9000
  6. 2,john,2023-07-24,P135,2500
  7. 2,john,2023-07-24,P136,6000
  8. 3,alice,2023-07-25,P137,4500
  9. 3,alice,2023-07-25,P138,8000

I'm going to change this to the following json format (or java object).

  1. [
  2.     {
  3.         "user_id": 1,
  4.         "nickname": "kmh",
  5.         "joinDate": "2023-07-24",
  6.         "orders": [
  7.             {
  8.                 "product_id": "P131",
  9.                 "price": 3000
  10.             },
  11.             {
  12.                 "product_id": "P132",
  13.                 "price": 4000
  14.             },
  15.             {
  16.                 "product_id": "P133",
  17.                 "price": 7000
  18.             },
  19.             {
  20.                 "product_id": "P134",
  21.                 "price": 9000
  22.             }
  23.         ]
  24.     },
  25.     {
  26.         "user_id": 2,
  27.         "nickname": "john",
  28.         "joinDate": "2023-07-24",
  29.         "orders": [
  30.             {
  31.                 "product_id": "P135",
  32.                 "price": 2500
  33.             },
  34.             {
  35.                 "product_id": "P136",
  36.                 "price": 6000
  37.             }
  38.         ]
  39.     },
  40.     {
  41.         "user_id": 3,
  42.         "nickname": "alice",
  43.         "joinDate": "2023-07-25",
  44.         "orders": [
  45.             {
  46.                 "product_id": "P137",
  47.                 "price": 4500
  48.             },
  49.             {
  50.                 "product_id": "P138",
  51.                 "price": 8000
  52.             }
  53.         ]
  54.     }
  55. ]

I've been searching for quite a long time and haven't found a library or tool that enables this .

I have so many different types of csv that I need tools or libraries to change all of these. Are there any libraries or tools that make this possible?

答案1

得分: 0

以下是您要的翻译部分：

  1. 你只需要一种方法将CSV解析为Java对象。您可以手动执行此操作，或使用现有库。
  2. 例如，您可以使用Jackson与CSV数据格式如下：
  3. class MyRecord {
  4. @JsonProperty("user_id")
  5. private int userId;
  6. private String nickname;
  7. private LocalDate joinDate;
  8. @JsonProperty("product_id")
  9. private String productId;
  10. // 获取器和设置器
  11. // 有意义的toString方法
  12. }
  13. public class Main {
  14. public static void main(String[] args) throws IOException {
  15. String csv = "user_id,nickname,joinDate,product_id,price\n" +
  16. "1,kmh,2023-07-24,P131,3000\n" +
  17. "1,kmh,2023-07-24,P132,4000\n" +
  18. "1,kmh,2023-07-24,P133,7000\n" +
  19. "1,kmh,2023-07-24,P134,9000\n" +
  20. "2,john,2023-07-24,P135,2500\n" +
  21. "2,john,2023-07-24,P136,6000\n" +
  22. "3,alice,2023-07-25,P137,4500\n" +
  23. "3,alice,2023-07-25,P138,8000";
  24. CsvSchema schema = CsvSchema.emptySchema().withHeader(); // 使用CSV标题读取模式
  25. ObjectMapper mapper = new CsvMapper().registerModule(new JavaTimeModule()); // 用于反序列化Java 8 LocalDate
  26. MappingIterator<MyRecord> resultIterator = mapper.readerFor(MyRecord.class).with(schema).readValues(csv);
  27. while (resultIterator.hasNext()) {
  28. System.out.println(resultIterator.next());
  29. }
  30. resultIterator.close();
  31. }
  32. } 
  33. 打印输出如下：
  34. MyRecord[userId=1, nickname='kmh', joinDate=2023-07-24, productId='P131', price=3000]
  35. MyRecord[userId=1, nickname='kmh', joinDate=2023-07-24, productId='P132', price=4000]
  36. MyRecord[userId=1, nickname='kmh', joinDate=2023-07-24, productId='P133', price=7000]
  37. MyRecord[userId=1, nickname='kmh', joinDate=2023-07-24, productId='P134', price=9000]
  38. MyRecord[userId=2, nickname='john', joinDate=2023-07-24, productId='P135', price=2500]
  39. MyRecord[userId=2, nickname='john', joinDate=2023-07-24, productId='P136', price=6000]
  40. MyRecord[userId=3, nickname='alice', joinDate=2023-07-25, productId='P137', price=4500]
  41. MyRecord[userId=3, nickname='alice', joinDate=2023-07-25, productId='P138', price=8000]
英文:

All you need is a way to parse the CSV to a Java object. You can do this manually, or by using an existing library.

For example, you can use Jackson with the CSV data format like this:

  1. class MyRecord {
  2. @JsonProperty(&quot;user_id&quot;)
  3. private int userId;
  4. private String nickname;
  5. private LocalDate joinDate;
  6. @JsonProperty(&quot;product_id&quot;)
  7. private String productId;
  8. // getters and setters
  9. // a meaningful toString method
  10. }
  11. public class Main {
  12. public static void main(String[] args) throws IOException {
  13. String csv = &quot;user_id,nickname,joinDate,product_id,price\n&quot; +
  14. &quot;1,kmh,2023-07-24,P131,3000\n&quot; +
  15. &quot;1,kmh,2023-07-24,P132,4000\n&quot; +
  16. &quot;1,kmh,2023-07-24,P133,7000\n&quot; +
  17. &quot;1,kmh,2023-07-24,P134,9000\n&quot; +
  18. &quot;2,john,2023-07-24,P135,2500\n&quot; +
  19. &quot;2,john,2023-07-24,P136,6000\n&quot; +
  20. &quot;3,alice,2023-07-25,P137,4500\n&quot; +
  21. &quot;3,alice,2023-07-25,P138,8000&quot;;
  22. CsvSchema schema = CsvSchema.emptySchema().withHeader(); // uses CSV header to read the schema
  23. ObjectMapper mapper = new CsvMapper().registerModule(new JavaTimeModule()); // to deserialise Java 8 LocalDate
  24. MappingIterator&lt;MyRecord&gt; resultIterator = mapper.readerFor(MyRecord.class).with(schema).readValues(csv);
  25. while (resultIterator.hasNext()) {
  26. System.out.println(resultIterator.next());
  27. }
  28. resultIterator.close();
  29. }
  30. }

Which prints:

  1. MyRecord[userId=1, nickname=&#39;kmh&#39;, joinDate=2023-07-24, productId=&#39;P131&#39;, price=3000]
  2. MyRecord[userId=1, nickname=&#39;kmh&#39;, joinDate=2023-07-24, productId=&#39;P132&#39;, price=4000]
  3. MyRecord[userId=1, nickname=&#39;kmh&#39;, joinDate=2023-07-24, productId=&#39;P133&#39;, price=7000]
  4. MyRecord[userId=1, nickname=&#39;kmh&#39;, joinDate=2023-07-24, productId=&#39;P134&#39;, price=9000]
  5. MyRecord[userId=2, nickname=&#39;john&#39;, joinDate=2023-07-24, productId=&#39;P135&#39;, price=2500]
  6. MyRecord[userId=2, nickname=&#39;john&#39;, joinDate=2023-07-24, productId=&#39;P136&#39;, price=6000]
  7. MyRecord[userId=3, nickname=&#39;alice&#39;, joinDate=2023-07-25, productId=&#39;P137&#39;, price=4500]
  8. MyRecord[userId=3, nickname=&#39;alice&#39;, joinDate=2023-07-25, productId=&#39;P138&#39;, price=8000]

答案2

得分: 0

对于您的情况，您可以直接将CSV反序列化为JsonNode，而无需创建POJO类。然后，使用JSON库"Josson"来通过group()函数转换JSON。

  1. String csv = "user_id,nickname,joinDate,product_id,price\n" +
  2.         "1,kmh,2023-07-24,P131,3000\n" +
  3.         "1,kmh,2023-07-24,P132,4000\n" +
  4.         "1,kmh,2023-07-24,P133,7000\n" +
  5.         "1,kmh,2023-07-24,P134,9000\n" +
  6.         "2,john,2023-07-24,P135,2500\n" +
  7.         "2,john,2023-07-24,P136,6000\n" +
  8.         "3,alice,2023-07-25,P137,4500\n" +
  9.         "3,alice,2023-07-25,P138,8000";
  10. ArrayNode arrayNode = Josson.createArrayNode();
  11. CsvSchema schema = CsvSchema.emptySchema().withHeader();
  12. try (MappingIterator<JsonNode> it = new CsvMapper().readerFor(JsonNode.class).with(schema).readValues(csv)) {
  13.     arrayNode.addAll(it.readAll());
  14. }
  15. Josson josson = Josson.create(arrayNode);
  16. JsonNode grouped = josson.getNode(
  17.         "group(map(user_id, nickname, joinDate), map(product_id, price))" +
  18.         ".map(**:key, orders:elements)");
  19. System.out.println(grouped.toPrettyString());

函数 group()

  1. 按{user_id, nickname, joinDate}的"key"分组
  2. 使用{product_id, price}的"elements"

函数 map()

  1. 提取对象"key"中的值
  2. 添加从"elements"复制的字段

输出如下：

  1. [{
  2.   "user_id": "1",
  3.   "nickname": "kmh",
  4.   "joinDate": "2023-07-24",
  5.   "orders": [{
  6.     "product_id": "P131",
  7.     "price": "3000"
  8.   }, {
  9.     "product_id": "P132",
  10.     "price": "4000"
  11.   }, {
  12.     "product_id": "P133",
  13.     "price": "7000"
  14.   }, {
  15.     "product_id": "P134",
  16.     "price": "9000"
  17.   }]
  18. }, {
  19.   "user_id": "2",
  20.   "nickname": "john",
  21.   "joinDate": "2023-07-24",
  22.   "orders": [{
  23.     "product_id": "P135",
  24.     "price": "2500"
  25.   }, {
  26.     "product_id": "P136",
  27.     "price": "6000"
  28.   }]
  29. }, {
  30.   "user_id": "3",
  31.   "nickname": "alice",
  32.   "joinDate": "2023-07-25",
  33.   "orders": [{
  34.     "product_id": "P137",
  35.     "price": "4500"
  36.   }, {
  37.     "product_id": "P138",
  38.     "price": "8000"
  39.   }]
  40. }]
英文:

For your case, you can deserialize the csv into JsonNode directly without creating POJO class. And then use JSON library Josson to transform the JSON by function group().

  1. String csv = &quot;user_id,nickname,joinDate,product_id,price\n&quot; +
  2. &quot;1,kmh,2023-07-24,P131,3000\n&quot; +
  3. &quot;1,kmh,2023-07-24,P132,4000\n&quot; +
  4. &quot;1,kmh,2023-07-24,P133,7000\n&quot; +
  5. &quot;1,kmh,2023-07-24,P134,9000\n&quot; +
  6. &quot;2,john,2023-07-24,P135,2500\n&quot; +
  7. &quot;2,john,2023-07-24,P136,6000\n&quot; +
  8. &quot;3,alice,2023-07-25,P137,4500\n&quot; +
  9. &quot;3,alice,2023-07-25,P138,8000&quot;;
  10. ArrayNode arrayNode = Josson.createArrayNode();
  11. CsvSchema schema = CsvSchema.emptySchema().withHeader();
  12. try (MappingIterator&lt;JsonNode&gt; it = new CsvMapper().readerFor(JsonNode.class).with(schema).readValues(csv)) {
  13. arrayNode.addAll(it.readAll());
  14. }
  15. Josson josson = Josson.create(arrayNode);
  16. JsonNode grouped = josson.getNode(
  17. &quot;group(map(user_id, nickname, joinDate), map(product_id, price))&quot; +
  18. &quot;.map(**:key, orders:elements)&quot;);
  19. System.out.println(grouped.toPrettyString());

Function group()

  1. Group by "key" of {user_id, nickname, joinDate}
  2. With "elements" of {product_id, price}

Functoin map()

  1. Extract values inside object "key"
  2. Add field "orders" copy from "elements"

Output

  1. [ {
  2. &quot;user_id&quot; : &quot;1&quot;,
  3. &quot;nickname&quot; : &quot;kmh&quot;,
  4. &quot;joinDate&quot; : &quot;2023-07-24&quot;,
  5. &quot;orders&quot; : [ {
  6. &quot;product_id&quot; : &quot;P131&quot;,
  7. &quot;price&quot; : &quot;3000&quot;
  8. }, {
  9. &quot;product_id&quot; : &quot;P132&quot;,
  10. &quot;price&quot; : &quot;4000&quot;
  11. }, {
  12. &quot;product_id&quot; : &quot;P133&quot;,
  13. &quot;price&quot; : &quot;7000&quot;
  14. }, {
  15. &quot;product_id&quot; : &quot;P134&quot;,
  16. &quot;price&quot; : &quot;9000&quot;
  17. } ]
  18. }, {
  19. &quot;user_id&quot; : &quot;2&quot;,
  20. &quot;nickname&quot; : &quot;john&quot;,
  21. &quot;joinDate&quot; : &quot;2023-07-24&quot;,
  22. &quot;orders&quot; : [ {
  23. &quot;product_id&quot; : &quot;P135&quot;,
  24. &quot;price&quot; : &quot;2500&quot;
  25. }, {
  26. &quot;product_id&quot; : &quot;P136&quot;,
  27. &quot;price&quot; : &quot;6000&quot;
  28. } ]
  29. }, {
  30. &quot;user_id&quot; : &quot;3&quot;,
  31. &quot;nickname&quot; : &quot;alice&quot;,
  32. &quot;joinDate&quot; : &quot;2023-07-25&quot;,
  33. &quot;orders&quot; : [ {
  34. &quot;product_id&quot; : &quot;P137&quot;,
  35. &quot;price&quot; : &quot;4500&quot;
  36. }, {
  37. &quot;product_id&quot; : &quot;P138&quot;,
  38. &quot;price&quot; : &quot;8000&quot;
  39. } ]
  40. } ]

huangapple
  • 本文由 发表于 2023年7月24日 19:45:15
  • 转载请务必保留本文链接：https://go.coder-hub.com/76754157.html
  • csv
  • java
  • json
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定