如何比较两个CSV文件中的Json字段

huangapple go评论83阅读模式
英文:

How to compare two CSV's with Json's fields

问题

以下是翻译好的内容:

我有一个任务需要比较两个CSV文件,它们的字段分别是:字符串、JSON对象、JSON对象、日期、字符串、字符串。如何打开这些文件并进行比较?到目前为止,我已经尝试了以下这些方法来处理一个CSV文件,那么接下来该怎么做呢?

public class MainApp {

    public static void main(String[] args) throws IOException {

        Collection<Object> lines = new ArrayList<>();

        String filePath = ".\\sheet.csv";

        Reader reader = Files.newBufferedReader(Paths.get(filePath));
        CSVParser csvParser = new CSVParser(reader, CSVFormat.DEFAULT.withFirstRecordAsHeader().withIgnoreHeaderCase().withTrim());

        for (CSVRecord csvRecord : csvParser) {
            String name = csvRecord.get("name");
            String email = csvRecord.get("jobSpecification");
            String phone = csvRecord.get("assignedProject");
            String country = csvRecord.get("lastUpdated");
            String id = csvRecord.get("id");
            String comments = csvRecord.get("comments");

            Data row = new Data(name, email, phone, country, id, comments);
            lines.add(row);
        }

        for (Object d : lines) {
            System.out.println(d.toString());
        }
    }
}

数据对象:

public class Data {
    String name;
    String email;
    String phone;
    String country;
    String id;
    String comments;

    public Data(String name, String email, String phone, String country, String id, String comments) {
        this.name = name;
        this.email = email;
        this.phone = phone;
        this.country = country;
        this.id = id;
        this.comments = comments;
    }

    @Override
    public String toString() {
        return "name: " + name + " email: " + email + " phone: " + phone + " country: " + country + " id: " + id + " comments: " + comments;
    }
}

现在我认为只剩下比较的部分了。我制作的CSV文件的样式如这里所示。

英文:

I have a work to compare two CSV's that the fields are : string json object json object date string string How to open the files and compare them? I tried till now for one CSV first of all these methods, so whats next?:

public class MainApp {

public static void main(String[] args) throws IOException{

	Collection&lt;Object&gt; lines = new ArrayList&lt;&gt;();

	String filePath = &quot;.\\sheet.csv&quot;;

	Reader reader = Files.newBufferedReader(Paths.get(filePath));
	CSVParser csvParser = new CSVParser(reader,CSVFormat.DEFAULT.withFirstRecordAsHeader().withIgnoreHeaderCase().withTrim());
	{
		for (CSVRecord csvRecord : csvParser) {
			String name = csvRecord.get(&quot;name&quot;);
			String email = csvRecord.get(&quot;jobSpecification&quot;);
			String phone = csvRecord.get(&quot;assignedProject&quot;);
			String country = csvRecord.get(&quot;lastUpdated&quot;);
			String id = csvRecord.get(&quot;id&quot;);
			String comments = csvRecord.get(&quot;comments&quot;);

			Data row = new Data(name, email, phone, country, id, comments);
			
			lines.add(row);

		}
	}
	for (Object d : lines) {
		System.out.println(d.toString());
	}

}

and the object:

public class Data {
String name;
String email ;
String phone ;
String country;
String id;
String comments;

public Data(String name, String email, String phone, String country, String id, String comments) {
	this.name = name;
	this.email = email;
	this.phone = phone;
	this.country = country;
	this.id = id;
	this.comments = comments;
}

@Override
public String toString() {
	return &quot;name: &quot;+name+&quot; &quot;+&quot; email: &quot;+email+&quot; phone: &quot;+phone+&quot; country: &quot;+country+&quot; id: &quot;+id+&quot; comments: &quot;+comments;
}

}

so now I think that left only the compare
the CSV(that i made) looks like these

答案1

得分: 2

以下是你要翻译的内容:

为了比较两个CSV文件,我建议在包含CSV字段的类上使用equals方法,并逐行迭代这些文件,但是您必须在Data类中实现自己的equals和hashcode方法。

有一件事情需要考虑 - 比较两个JSON并不那么容易,有许多库,但几乎所有库都使用严格的比较方法,这意味着比较具有不同字段顺序、嵌套字段顺序或嵌套数组项顺序的两个JSON将导致这些JSON对象之间的假负比较。

允许宽松比较的库是jsonassert库:

<dependency>
    <groupId>org.skyscreamer</groupId>
    <artifactId>jsonassert</artifactId>
    <version>1.5.0</version>
    <scope>compile</scope>
</dependency>

下面是基本实现,它检查两个JSON是否深度相等(不考虑字段顺序、数组项顺序) - 以comments字段为例。

// Data类
@Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
Data data = (Data) o;
return Objects.equals(name, data.name) &&
Objects.equals(email, data.email) &&
Objects.equals(phone, data.phone) &&
Objects.equals(country, data.country) &&
Objects.equals(id, data.id) &&
deepJSONsEquals(comments, data.comments);
}

private boolean deepJSONsEquals(String firstJson, String secondJson) {
try {
return JSONCompare.compareJSON(firstJson, secondJson, JSONCompareMode.LENIENT).passed();
} catch (JSONException e) {
e.printStackTrace();
return false;
}
}

@Override
public int hashCode() {
return Objects.hash(name, email, phone, country, id, comments);
}

MainApp类:

List linesFromFirstCsv = new ArrayList<>();
List linesFromSecondCsv = new ArrayList<>();

// 逐行将CSV中的值添加到相关列表中(创建Data对象)

if (linesFromFirstCsv.size() == linesFromSecondCsv.size()) {
for (int i = 0; i < Math.min(linesFromFirstCsv.size(), linesFromSecondCsv.size()); i++) {
System.out.println(linesFromFirstCsv.get(i).equals(linesFromSecondCsv.get(i)));
}
} else {
System.out.println("not equals");
}

这将在比较作为comments值传递的示例JSON时返回true结果(数组中的不同字段顺序和数组项顺序):

{
    "comments":
    [
        {
            "timestamp": 123123123,
            "id": "1212",
            "body": "lorem ipsum"
        },
        {
            "body": "lorem ipsum2",
            "id": "1213",
            "timestamp": 123123125
        },
        {
            "id": "1214",
            "body": "lorem ipsum3",
            "timestamp": 123123129
        }
    ]
}
{
    "comments":
    [
        {
            "id": "1213",
            "body": "lorem ipsum2",
            "timestamp": 123123125
        },
        {
            "id": "1212",
            "body": "lorem ipsum",
            "timestamp": 123123123
        },
        {
            "id": "1214",
            "body": "lorem ipsum3",
            "timestamp": 123123129
        }
    ]
}
英文:

To compare two CSV's I suggest to use equals method on class which contains csv's fields and iterate these files line by line,
but you have to implement own equals & hashcode methods in Data class.

There is one thing you need to consider - comparison of two jsons is not so easy, there are many libraries, but almost all libraries use strict comparison method, this means that comparing two
jsons with different field order, or nested field order, or nested array items order will results in false negative comparison between these json objects.

The library that allows lenient comparison is jsonassert library:

        &lt;dependency&gt;
            &lt;groupId&gt;org.skyscreamer&lt;/groupId&gt;
            &lt;artifactId&gt;jsonassert&lt;/artifactId&gt;
            &lt;version&gt;1.5.0&lt;/version&gt;
            &lt;scope&gt;compile&lt;/scope&gt;
        &lt;/dependency&gt;

Below you can find basic implementation, it checks if two JSONs are deeply equals (regardless the fields order, arrays items order) - used comments field as example.

//Data class
@Override
public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    Data data = (Data) o;
    return Objects.equals(name, data.name) &amp;&amp;
            Objects.equals(email, data.email) &amp;&amp;
            Objects.equals(phone, data.phone) &amp;&amp;
            Objects.equals(country, data.country) &amp;&amp;
            Objects.equals(id, data.id) &amp;&amp;
            deepJSONsEquals(comments, data.comments);
}

private boolean deepJSONsEquals(String firstJson, String secondJson) {
    try {
        return JSONCompare.compareJSON(firstJson, secondJson, JSONCompareMode.LENIENT).passed();
    } catch (JSONException e) {
        e.printStackTrace();
        return false;
    }
}

@Override
public int hashCode() {
    return Objects.hash(name, email, phone, country, id, comments);
}

MainApp class:

List&lt;Data&gt; linesFromFirstCsv = new ArrayList&lt;&gt;();
List&lt;Data&gt; linesFromSecondCsv = new ArrayList&lt;&gt;();

// add here line by line values from csv to relevant lists. (create Data objects)

    if(linesFromFirstCsv.size() == linesFromSecondCsv.size()) {
        for (int i = 0; i &lt; Math.min(linesFromFirstCsv.size(), linesFromSecondCsv.size()); i++) {
            System.out.println(linesFromFirstCsv.get(i).equals(linesFromSecondCsv.get(i)));
        }
    } else {
        System.out.println(&quot;not equals&quot;);
    }

This will return a true result in comparison of example jsons passed as comments value(different field order in array, and array items order):

{
    &quot;comments&quot;:
	[
	    {
		&quot;timestamp&quot;: 123123123,
        &quot;id&quot;: &quot;1212&quot;,
        &quot;body&quot;: &quot;lorem ipsum&quot;
		},
		{
        &quot;body&quot;: &quot;lorem ipsum2&quot;,
		&quot;id&quot;: &quot;1213&quot;,
        &quot;timestamp&quot;: 123123125
		},
		{
        &quot;id&quot;: &quot;1214&quot;,
        &quot;body&quot;: &quot;lorem ipsum3&quot;,
        &quot;timestamp&quot;: 123123129
		}
	]
}
{
    &quot;comments&quot;:
	[
		{
        &quot;id&quot;: &quot;1213&quot;,
        &quot;body&quot;: &quot;lorem ipsum2&quot;,
        &quot;timestamp&quot;: 123123125
		},
		{
        &quot;id&quot;: &quot;1212&quot;,
        &quot;body&quot;: &quot;lorem ipsum&quot;,
        &quot;timestamp&quot;: 123123123
		},
		{
        &quot;id&quot;: &quot;1214&quot;,
        &quot;body&quot;: &quot;lorem ipsum3&quot;,
        &quot;timestamp&quot;: 123123129
		}
	]
}

huangapple
  • 本文由 发表于 2020年10月19日 01:34:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/64416324.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定