用Java 8逐行读取Spring的MultipartFile的最佳方法是什么?

huangapple go评论52阅读模式
英文:

Best way to read Spring Multipartfile line by line in Java 8

问题

如何处理CSV Spring MultipartFile中的最佳句柄?
我以前使用过类似于以下内容的方法:

public void handleFile(MultipartFile multipartFile){
    try{
        InputStream inputStream = multipartFile.getInputStream();
        IOUtils.readLines(inputStream, StandardCharsets.UTF_8)
                .stream()
                .forEach(this::handleLine);
    } catch (IOException e) {
        // 处理异常
    }
}

private void handleLine(String s) {
    // 按行处理内容
}

据我所知,这会首先将整个文件加载到内存中的列表中,然后再处理它,对于具有成千上万行的文件来说,这可能需要相当长的时间。

是否有一种方法可以在不手动实现迭代(例如使用read()hasNext()等)的情况下逐行处理它?我正在寻找类似于以下用于文件系统中的文件的示例的简洁方法:

try (Stream<String> stream = Files.lines(Paths.get("file.csv"))) {
        stream.forEach(this::handleLine);
} catch (IOException e) {
    // 处理异常
}
英文:

What is the best handle a csv Spring Multipartfile?
I have used something like this before:

public void handleFile(MultipartFile multipartFile){
    try{
        InputStream inputStream = multipartFile.getInputStream();
        IOUtils.readLines(inputStream, StandardCharsets.UTF_8)
                .stream()
                .forEach(this::handleLine);
    } catch (IOException e) {
        // handle exception
    }
}

private void handleLine(String s) {
    // do stuff per line
}

As far as I know, this first loads the whole file into a list in memory before processing it, which will probably take quite some time for files with tens of thousends of lines.

Is there a way to handle it line by line without the overhead of implementing the iteration by hand (i.e. using stuff like read(), hasNext(), ...)?
I am looking for something concise similar to this example for files from the file system:

try (Stream&lt;String&gt; stream = Files.lines(Paths.get(&quot;file.csv&quot;))) {
        stream.forEach(this::handleLine);
} catch (IOException e) {
    // handle exception
}

答案1

得分: 9

以下是您要翻译的内容:

在具有InputStream的情况下,您可以使用以下方式:

InputStream inputStream = multipartFile.getInputStream();
new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8))
        .lines()
        .forEach(this::handleLine);

在其他情况下:

无论是多部分文件还是有多个独立文件,都有许多方法可以在Java 8中使用Stream API来处理它们:

解决方案1:

如果您的文件位于不同的目录中,可以按以下方式操作:

假设您有一个包含文件路径的List,如下所示:

List<String> files = Arrays.asList(
        "/test/test.txt",
        "/test2/test2.txt");

然后,您可以按如下方式读取上述文件的所有行:

files.stream().map(Paths::get)
        .flatMap(path -> {
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        }).forEach(System.out::println);

解决方案2:

您还可以使用以下方式读取位于/test/ehsan目录中存在的所有文件的所有行:

try (Stream<Path> stream = Files.walk(Paths.get("/test/ehsan"), 1)) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -> {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

如果要递归读取/test/ehsan目录中的所有文件的所有行,可以按以下方式操作:

try (Stream<Path> stream = Files.walk(Paths.get("/test/ehsan"))) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -> {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

正如您所看到的,Files.walk的第二个参数指定要访问的最大目录级别,如果不传递它,将使用默认值,即Integer.MAX_VALUE

解决方案3:

我们可以进一步进行,假设我们想要读取两个完全不同目录中存在的所有文件的所有行,例如/test/ehsan/test2/ehsan1

我们可以做到,但应该谨慎,Stream不应该太长(因为它会降低程序的可读性),最好将它们分解为单独的方法。但由于无法在此处编写多个方法,我将在一个地方写出如何执行:

假设您有一个包含目录路径的List,如下所示:

List<String> dirs = Arrays.asList(
        "/test/ehsan",
        "/test2/ehsan1");

然后,我们可以按如下方式操作:

dirs.stream()
        .map(Paths::get)
        .flatMap(path -> {
            try {
                return Files.walk(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .filter(Files::isRegularFile)
        .flatMap(path -> { 
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .forEach(System.out::println);

希望这些翻译对您有所帮助。

英文:

In cases when you have InputStream you can use this one:

InputStream inputStream = multipartFile.getInputStream();
new BufferedReader(new InputStreamReader(inputStream, StandardCharsets.UTF_8))
                    .lines()
                    .forEach(this::handleLine);

In other cases:

No matter whether it is multipart file or you have multiple independent files, there are many approaches to do it in Java 8 using Stream API:

Solution 1:

If your files are in different directories you can do it this way:

Imagine you have a List of String which contains paths of your files like below:

List&lt;String&gt; files = Arrays.asList(
                &quot;/test/test.txt&quot;,
                &quot;/test2/test2.txt&quot;);

Then you can read all lines of above files as below:

files.stream().map(Paths::get)
        .flatMap(path -&gt; {
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        }).forEach(System.out::println);

Solution 2:

You can also read all lines of files that exist in /test/ehsan directory using Files.walk in the following way:

try (Stream&lt;Path&gt; stream = Files.walk(Paths.get(&quot;/test/ehsan&quot;), 1)) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -&gt; {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

And if you want to read all lines of files in /test/ehsan directory recursively you can do it this way:

try (Stream&lt;Path&gt; stream = Files.walk(Paths.get(&quot;/test/ehsan&quot;))) {
    stream.filter(Files::isRegularFile)
            .flatMap(path -&gt; {
                try {
                    return Files.lines(path);
                } catch (IOException e) {
                    e.printStackTrace();
                }
                return Stream.empty();
            })
            .forEach(System.out::println);
} catch (IOException e) {
    e.printStackTrace();
}

As you can see the second parameter to Files.walk specifies the maximum number of directory levels to visit and if you don't pass it the default will be used which is Integer.MAX_VALUE.

Solution 3:

Lets not stop here, we can go further. what if we wanted to read all lines of files exist in two completely different directories for example /test/ehsan and /test2/ehsan1?

We can do it but we should be cautious, Stream should not be so long( because it reduces readability of our program) it will be better to break them in separate methods, However because it is not possible to write multiple methods here I will write in one place how to do that:

Imagine you have a List of String which contains paths of your directories like below

list&lt;String&gt; dirs = Arrays.asList(
                &quot;/test/ehsan&quot;,
                &quot;/test2/ehsan1&quot;);

Then we can do that this way:

dirs.stream()
        .map(Paths::get)
        .flatMap(path -&gt; {
            try {
                return Files.walk(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .filter(Files::isRegularFile)
        .flatMap(path -&gt; { 
            try {
                return Files.lines(path);
            } catch (IOException e) {
                e.printStackTrace();
            }
            return Stream.empty();
        })
        .forEach(System.out::println);

答案2

得分: 0

public static List readCSV(String fileName) throws IOException {
List records = new ArrayList<>();
try (BufferedReader br = new BufferedReader(
new FileReader(fileName))) {
String line;
while ((line = br.readLine()) != null) {
records.add(line);
}
}
return records;
}

英文:
public static List&lt;String&gt; readCSV(String fileName) throws IOException {
        List&lt;String&gt; records = new ArrayList&lt;&gt;();
        try (BufferedReader br = new BufferedReader(
                new FileReader(fileName))) {
            String line;
            while ((line = br.readLine()) != null) {
                records.add(line);
            }
        }
        return records;
}

huangapple
  • 本文由 发表于 2020年7月29日 00:56:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/63139140.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定