如何将大型 JSON 文件正确映射到 Java（POJO）？

huangapple
go评论84阅读模式
英文:

How to correctly map huge json file to Java (pojo)?

问题

大型的JSON文件可用（从100 MB到2 GB）。

我该如何从JSON创建Java（POJO）？我需要获取所有字段的名称。例如，一个类可能有超过1000个字段。

尝试了以下库和在线服务：

JsonSchema2Pojo工作得很好，但一次只能处理约50,000个字符。由于存在限制，我无法完全映射类。需要每次部分复制JSON，然后删除重复的字段。

http://www.jsonschema2pojo.org/

JsonToJava
https://github.com/astav/JsonToJava

还尝试了其他几个在线服务。它们也有限制或不能正常工作。

JSON文件示例：

（以下为JSON文件内容的翻译，由于内容较长，我只提供部分翻译）

感谢您的理解。

英文:

Large json files are available (from 100 MB to 2 GB).

How I can create java (pojo) from json? I need to get the names of all fields. For example, one class can have more than 1000 fields.

Tried the following libraries & online services:

JsonSchema2Pojo works well, but has a limit of about 50,000 characters at a time. Due to the presence of limits, I cannot completely map the class. Need to partially copy the json each time and then remove the duplicate fields.

http://www.jsonschema2pojo.org/

JsonToJava
https://github.com/astav/JsonToJava

Also tried several other online services. They also have limits or do not work correctly.

Example of json file:

{"type":"FeatureCollection","features":[
{"type":"Feature","geometry":{"type":"Point","coordinates":[27.5618791,53.902334]},"properties":{"ele":"280","name":"Минск","place":"city","capital":"yes","name:ar":"مينسك","name:be":"Мінск","name:bg":"Минск","name:bo":"མིན་སིཀ།","name:cs":"Minsk","name:cu":"Мѣньскъ","name:cv":"Минск","name:de":"Minsk","name:el":"Μινσκ","name:en":"Minsk","name:eo":"Minsko","name:es":"Minsk","name:et":"Minsk","name:fa":"مینسک","name:fi":"Minsk","name:fr":"Minsk","name:ga":"Minsc","name:gl":"Minsk - Мінск","name:he":"מינסק","name:hi":"मिन्‍स्‍क","name:hr":"Minsk","name:hu":"Minszk","name:hy":"Մինսկ","name:ia":"Minsk","name:io":"Minsk","name:is":"Minsk","name:it":"Minsk","name:ja":"ミンスク","name:ka":"მინსკი","name:kk":"Минск","name:kn":"ಮಿನ್ಸ್ಕ್","name:ko":"민스크","name:ku":"Mînsk","name:kv":"Минск","name:ky":"Минск","name:la":"Minscum","name:lt":"Minskas","name:lv":"Minska","name:mk":"Минск","name:ml":"മിൻസ്ക്","name:mr":"मिन्‍स्‍क","name:nl":"Minsk","name:no":"Minsk","name:oc":"Minsk","name:os":"Минск","name:pl":"Mińsk","name:pt":"Minsk","name:ru":"Минск","name:sk":"Minsk","name:sl":"Minsk","name:sr":"Минск","name:sv":"Minsk","name:ta":"மின்ஸ்க்","name:tg":"Минск","name:th":"มินสก์","name:tt":"Минск","name:ug":"مىنىسكى","name:uk":"Мінськ","name:ur":"منسک","name:vi":"Minxcơ","name:vo":"Minsk","name:yi":"מינסק","name:zh":"明斯克","website":"https://minsk.gov.by/","int_name":"Minsk","name:ast":"Minsk","name:ckb":"مینسک","name:csb":"Mińsk","name:jbo":"misk","name:mhr":"Минск","name:myv":"Минск ош","name:nds":"Minsk","name:pnb":"منسک","name:rue":"Мінск","name:sah":"Минскай","name:szl":"Mińsk","name:udm":"Минск","name:wuu":"明斯克","nat_name":"Мінск","old_name":"Менск","wikidata":"Q2280","wikipedia":"ru:Минск","population":"1982444","admin_level":"2","alt_name:be":"Менск","alt_name:vi":"Minsk;Minxcơva","name:prefix":"город","old_name:be":"Менск","addr:country":"BY","name:bat-smg":"Minsks","name:sr-Latn":"Minsk","wikipedia:be":"Мінск","wikipedia:en":"Minsk","wikipedia:pl":"Mińsk","addr:postcode":"220000","is_in:country":"Belarus","name:be-tarask":"Менск","source:name:oc":"Lo Congrès","is_in:continent":"Europe","population:date":"2018-01-01","capital_ISO3166-1":"yes","source:population":"Белстат","is_in:country_code":"BY"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5861178,53.9510657],[27.5861196,53.9510294],[27.5862319,53.9510313],[27.5862302,53.9510676],[27.5861178,53.9510657]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5885075,53.9512721],[27.5885398,53.9511614],[27.5887363,53.9511813],[27.588704,53.9512919],[27.5885075,53.9512721]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5881428,53.9512622],[27.5881704,53.9511135],[27.588421,53.9511296],[27.5883935,53.9512783],[27.5881428,53.9512622]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5892179,53.9516538],[27.5893274,53.9516011],[27.5893778,53.9516374],[27.5894352,53.9516098],[27.5895358,53.9516823],[27.5893689,53.9517626],[27.5892179,53.9516538]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.588949,53.9517486],[27.5891273,53.9516621],[27.5892764,53.9517686],[27.5890981,53.9518551],[27.588949,53.9517486]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5867595,53.9500825],[27.5867648,53.9499951],[27.5869291,53.9499985],[27.5869238,53.9500859],[27.5867595,53.9500825]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5830382,53.9503042],[27.5830422,53.9502013],[27.5832783,53.9502045],[27.5832743,53.9503074],[27.5830382,53.9503042]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5820906,53.9497446],[27.5820973,53.9496721],[27.5822371,53.9496765],[27.5822305,53.949749],[27.5820906,53.9497446]]]]},"properties":{"building":"yes"}},
{"type":"Feature","geometry":{"type":"MultiPolygon","coordinates":[[[[27.5477403,53.8940388],[27.5482144,53.8939744],[27.5482459,53.8940791],[27.5482929,53.8940736],[27.5483083,53.8941424],[27.5482593,53.8941471],[27.5482741,53.8942048],[27.5485718,53.8943067],[27.5487113,53.8943028],[27.5487556,53.8943755],[27.548895,53.8944261],[27.5487985,53.894513],[27.5488079,53.8945841],[27.5483385,53.8943889],[27.548317,53.8943202],[27.5481439,53.8942549],[27.5481045,53.8941843],[27.5479863,53.8942063],[27.5479953,53.8942214],[27.5478248,53.8942506],[27.5477403,53.8940388]]]]},"properties":{"name":"Инженерный корпус метрополитена","building":"yes"}}
]}

Thanks in advance!

答案1

得分: 0

感谢您的建议。
以下是实现的方式：
我使用了JsonAnySetter注解来设置所有字段，而不是创建包含所有可能字段的POJO。
以下是我的DTO：

FeatureDto

@Getter
@ToString
public class FeatureDto {

  @JsonProperty("type")
  private String type;
  @JsonProperty("geometry")
  private GeometryDto geometry;
  @JsonProperty("properties")
  private PropertiesDto properties;

}

PropertiesDto

@Getter
@ToString
public class PropertiesDto {

  private List<FieldDto> fieldDtos = new ArrayList<>();

  @JsonAnySetter
  public void setProperties(String code, Object value) {

    if (value instanceof ArrayList) {
      ArrayList<Object> list = (ArrayList<Object>) value;
      StringBuffer stringBuffer = new StringBuffer();
      list.forEach(o -> stringBuffer.append(o).append(", "));
      fieldDtos.add(new FieldDto(code, stringBuffer.toString()));
      return;
    }

    fieldDtos.add(new FieldDto(code, value.toString()));
  }

}

FieldDto

@Data
@ToString
@JsonIgnoreProperties(ignoreUnknown = true)
@AllArgsConstructor
public class FieldDto {

  @JsonProperty("code")
  private String code;
  @JsonProperty("value")
  private String value;

}

这是JSON源：

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Point",
        "coordinates": [27.5618791, 53.902334]
      },
      "properties": {
        "ele": "280",
        "name": "Минск",
        ...
        // 更多属性
      }
    },
    {
      "type": "Feature",
      "geometry": {
        "type": "MultiPolygon",
        "coordinates": [...]
      },
      "properties": {
        "building": "yes"
      }
    },
    ...
    // 更多特征
  ]
}
英文:

Thanks a lot for your advices.
Implemented in the following way:
I used JsonAnySetter annotation to set all fields instead creating pojo with all of the possible fields.
Here is my Dto's

FeatureDto

@Getter
@ToString
public class FeatureDto {
@JsonProperty(&quot;type&quot;)
private String type;
@JsonProperty(&quot;geometry&quot;)
private GeometryDto geometry;
@JsonProperty(&quot;properties&quot;)
private PropertiesDto properties;
}

PropertiesDto

@Getter
@ToString
public class PropertiesDto {
private List&lt;FieldDto&gt; fieldDtos = new ArrayList&lt;&gt;();
@JsonAnySetter
public void setProperties(String code, Object value)
{
if(value instanceof ArrayList) {
ArrayList&lt;Object&gt; list = (ArrayList&lt;Object&gt;) value;
StringBuffer stringBuffer = new StringBuffer();
list.forEach(o -&gt; stringBuffer.append(o).append(&quot;, &quot;));
fieldDtos.add(new FieldDto(code, stringBuffer.toString()));
return;
}
fieldDtos.add(new FieldDto(code, value.toString()));
}
}

FieldDto

@Data
@ToString
@JsonIgnoreProperties(ignoreUnknown = true)
@AllArgsConstructor
public class FieldDto {
@JsonProperty(&quot;code&quot;)
private String code;
@JsonProperty(&quot;value&quot;)
private String value;
}

Here is the json source:

{&quot;type&quot;:&quot;FeatureCollection&quot;,&quot;features&quot;:[
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;Point&quot;,&quot;coordinates&quot;:[27.5618791,53.902334]},&quot;properties&quot;:{&quot;ele&quot;:&quot;280&quot;,&quot;name&quot;:&quot;Минск&quot;,&quot;place&quot;:&quot;city&quot;,&quot;capital&quot;:&quot;yes&quot;,&quot;name:ar&quot;:&quot;مينسك&quot;,&quot;name:be&quot;:&quot;Мінск&quot;,&quot;name:bg&quot;:&quot;Минск&quot;,&quot;name:bo&quot;:&quot;མིན་སིཀ།&quot;,&quot;name:cs&quot;:&quot;Minsk&quot;,&quot;name:cu&quot;:&quot;Мѣньскъ&quot;,&quot;name:cv&quot;:&quot;Минск&quot;,&quot;name:de&quot;:&quot;Minsk&quot;,&quot;name:el&quot;:&quot;Μινσκ&quot;,&quot;name:en&quot;:&quot;Minsk&quot;,&quot;name:eo&quot;:&quot;Minsko&quot;,&quot;name:es&quot;:&quot;Minsk&quot;,&quot;name:et&quot;:&quot;Minsk&quot;,&quot;name:fa&quot;:&quot;مینسک&quot;,&quot;name:fi&quot;:&quot;Minsk&quot;,&quot;name:fr&quot;:&quot;Minsk&quot;,&quot;name:ga&quot;:&quot;Minsc&quot;,&quot;name:gl&quot;:&quot;Minsk - Мінск&quot;,&quot;name:he&quot;:&quot;מינסק&quot;,&quot;name:hi&quot;:&quot;मिन्‍स्‍क&quot;,&quot;name:hr&quot;:&quot;Minsk&quot;,&quot;name:hu&quot;:&quot;Minszk&quot;,&quot;name:hy&quot;:&quot;Մինսկ&quot;,&quot;name:ia&quot;:&quot;Minsk&quot;,&quot;name:io&quot;:&quot;Minsk&quot;,&quot;name:is&quot;:&quot;Minsk&quot;,&quot;name:it&quot;:&quot;Minsk&quot;,&quot;name:ja&quot;:&quot;ミンスク&quot;,&quot;name:ka&quot;:&quot;მინსკი&quot;,&quot;name:kk&quot;:&quot;Минск&quot;,&quot;name:kn&quot;:&quot;ಮಿನ್ಸ್ಕ್&quot;,&quot;name:ko&quot;:&quot;민스크&quot;,&quot;name:ku&quot;:&quot;M&#238;nsk&quot;,&quot;name:kv&quot;:&quot;Минск&quot;,&quot;name:ky&quot;:&quot;Минск&quot;,&quot;name:la&quot;:&quot;Minscum&quot;,&quot;name:lt&quot;:&quot;Minskas&quot;,&quot;name:lv&quot;:&quot;Minska&quot;,&quot;name:mk&quot;:&quot;Минск&quot;,&quot;name:ml&quot;:&quot;മിൻസ്ക്&quot;,&quot;name:mr&quot;:&quot;मिन्‍स्‍क&quot;,&quot;name:nl&quot;:&quot;Minsk&quot;,&quot;name:no&quot;:&quot;Minsk&quot;,&quot;name:oc&quot;:&quot;Minsk&quot;,&quot;name:os&quot;:&quot;Минск&quot;,&quot;name:pl&quot;:&quot;Mińsk&quot;,&quot;name:pt&quot;:&quot;Minsk&quot;,&quot;name:ru&quot;:&quot;Минск&quot;,&quot;name:sk&quot;:&quot;Minsk&quot;,&quot;name:sl&quot;:&quot;Minsk&quot;,&quot;name:sr&quot;:&quot;Минск&quot;,&quot;name:sv&quot;:&quot;Minsk&quot;,&quot;name:ta&quot;:&quot;மின்ஸ்க்&quot;,&quot;name:tg&quot;:&quot;Минск&quot;,&quot;name:th&quot;:&quot;มินสก์&quot;,&quot;name:tt&quot;:&quot;Минск&quot;,&quot;name:ug&quot;:&quot;مىنىسكى&quot;,&quot;name:uk&quot;:&quot;Мінськ&quot;,&quot;name:ur&quot;:&quot;منسک&quot;,&quot;name:vi&quot;:&quot;Minxcơ&quot;,&quot;name:vo&quot;:&quot;Minsk&quot;,&quot;name:yi&quot;:&quot;מינסק&quot;,&quot;name:zh&quot;:&quot;明斯克&quot;,&quot;website&quot;:&quot;https://minsk.gov.by/&quot;,&quot;int_name&quot;:&quot;Minsk&quot;,&quot;name:ast&quot;:&quot;Minsk&quot;,&quot;name:ckb&quot;:&quot;مینسک&quot;,&quot;name:csb&quot;:&quot;Mińsk&quot;,&quot;name:jbo&quot;:&quot;misk&quot;,&quot;name:mhr&quot;:&quot;Минск&quot;,&quot;name:myv&quot;:&quot;Минск ош&quot;,&quot;name:nds&quot;:&quot;Minsk&quot;,&quot;name:pnb&quot;:&quot;منسک&quot;,&quot;name:rue&quot;:&quot;Мінск&quot;,&quot;name:sah&quot;:&quot;Минскай&quot;,&quot;name:szl&quot;:&quot;Mińsk&quot;,&quot;name:udm&quot;:&quot;Минск&quot;,&quot;name:wuu&quot;:&quot;明斯克&quot;,&quot;nat_name&quot;:&quot;Мінск&quot;,&quot;old_name&quot;:&quot;Менск&quot;,&quot;wikidata&quot;:&quot;Q2280&quot;,&quot;wikipedia&quot;:&quot;ru:Минск&quot;,&quot;population&quot;:&quot;1982444&quot;,&quot;admin_level&quot;:&quot;2&quot;,&quot;alt_name:be&quot;:&quot;Менск&quot;,&quot;alt_name:vi&quot;:&quot;Minsk;Minxcơva&quot;,&quot;name:prefix&quot;:&quot;город&quot;,&quot;old_name:be&quot;:&quot;Менск&quot;,&quot;addr:country&quot;:&quot;BY&quot;,&quot;name:bat-smg&quot;:&quot;Minsks&quot;,&quot;name:sr-Latn&quot;:&quot;Minsk&quot;,&quot;wikipedia:be&quot;:&quot;Мінск&quot;,&quot;wikipedia:en&quot;:&quot;Minsk&quot;,&quot;wikipedia:pl&quot;:&quot;Mińsk&quot;,&quot;addr:postcode&quot;:&quot;220000&quot;,&quot;is_in:country&quot;:&quot;Belarus&quot;,&quot;name:be-tarask&quot;:&quot;Менск&quot;,&quot;source:name:oc&quot;:&quot;Lo Congr&#232;s&quot;,&quot;is_in:continent&quot;:&quot;Europe&quot;,&quot;population:date&quot;:&quot;2018-01-01&quot;,&quot;capital_ISO3166-1&quot;:&quot;yes&quot;,&quot;source:population&quot;:&quot;Белстат&quot;,&quot;is_in:country_code&quot;:&quot;BY&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5861178,53.9510657],[27.5861196,53.9510294],[27.5862319,53.9510313],[27.5862302,53.9510676],[27.5861178,53.9510657]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5885075,53.9512721],[27.5885398,53.9511614],[27.5887363,53.9511813],[27.588704,53.9512919],[27.5885075,53.9512721]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5881428,53.9512622],[27.5881704,53.9511135],[27.588421,53.9511296],[27.5883935,53.9512783],[27.5881428,53.9512622]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5892179,53.9516538],[27.5893274,53.9516011],[27.5893778,53.9516374],[27.5894352,53.9516098],[27.5895358,53.9516823],[27.5893689,53.9517626],[27.5892179,53.9516538]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.588949,53.9517486],[27.5891273,53.9516621],[27.5892764,53.9517686],[27.5890981,53.9518551],[27.588949,53.9517486]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5867595,53.9500825],[27.5867648,53.9499951],[27.5869291,53.9499985],[27.5869238,53.9500859],[27.5867595,53.9500825]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5830382,53.9503042],[27.5830422,53.9502013],[27.5832783,53.9502045],[27.5832743,53.9503074],[27.5830382,53.9503042]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5820906,53.9497446],[27.5820973,53.9496721],[27.5822371,53.9496765],[27.5822305,53.949749],[27.5820906,53.9497446]]]]},&quot;properties&quot;:{&quot;building&quot;:&quot;yes&quot;}},
{&quot;type&quot;:&quot;Feature&quot;,&quot;geometry&quot;:{&quot;type&quot;:&quot;MultiPolygon&quot;,&quot;coordinates&quot;:[[[[27.5477403,53.8940388],[27.5482144,53.8939744],[27.5482459,53.8940791],[27.5482929,53.8940736],[27.5483083,53.8941424],[27.5482593,53.8941471],[27.5482741,53.8942048],[27.5485718,53.8943067],[27.5487113,53.8943028],[27.5487556,53.8943755],[27.548895,53.8944261],[27.5487985,53.894513],[27.5488079,53.8945841],[27.5483385,53.8943889],[27.548317,53.8943202],[27.5481439,53.8942549],[27.5481045,53.8941843],[27.5479863,53.8942063],[27.5479953,53.8942214],[27.5478248,53.8942506],[27.5477403,53.8940388]]]]},&quot;properties&quot;:{&quot;name&quot;:&quot;Инженерный корпус метрополитена&quot;,&quot;building&quot;:&quot;yes&quot;}}
]}

huangapple
  • 本文由 发表于 2020年7月23日 09:14:14
  • 转载请务必保留本文链接：https://go.coder-hub.com/63045393.html
  • jackson
  • java
  • json
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定