无法以非阻塞方式在 Docker 容器内读取类路径资源文件。

huangapple go评论83阅读模式
英文:

Unable to read classpath resource file in non-blocking way inside docker container

问题

我试图从 src/main/resources 目录加载一个名为 sample.json 的 JSON 文件。

我必须将该 JSON 映射到一个 Java 对象。我的应用程序是响应式的,我正在使用 Spring WebFlux。

我已经按照 Simon's Blog 的指导完成了以下代码:

    SimpleModule module = new SimpleModule();
    module.addDeserializer(OffsetDateTime.class, new JsonDeserializer<>() {
      @Override
      public OffsetDateTime deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
        return OffsetDateTime.parse(p.getValueAsString());
      }
    });

    ObjectMapper objectMapper = new ObjectMapper();
    objectMapper.registerModule(module);

    return Flux.using(() -> Files.lines(Paths.get(ClassLoader.getSystemResource("sample.json").toURI())),
        Flux::fromStream,
        BaseStream::close
    )
        .collectList()
        .map(lines -> String.join("\n", lines))
        .map(jsonContent -> {
          try {
            return objectMapper.readValue(jsonContent, MyPojo.class);
          } catch (JsonProcessingException e) {
            e.printStackTrace();
            return null;
          }
        })
        .map(MyPojo::getValues());

这在本地运行正常,但在 Docker 容器内运行时失败。(我使用 gradle 构建生成 JAR 文件,然后从中构建 Docker 镜像)

部分错误堆栈跟踪:

> java.nio.file.FileSystemNotFoundException: null
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:169) ~[jdk.zipfs:na]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:

英文:

I am trying to load a JSON file sample.json from src/main/resources directory.

I have to map that json to a Java Object. And my application is reactive, I am using Spring webflux.

I have followed Simon's Blog to come up with this:

    SimpleModule module = new SimpleModule();
    module.addDeserializer(OffsetDateTime.class, new JsonDeserializer<>() {
      @Override
      public OffsetDateTime deserialize(JsonParser p, DeserializationContext ctxt) throws IOException, JsonProcessingException {
        return OffsetDateTime.parse(p.getValueAsString());
      }
    });

    ObjectMapper objectMapper = new ObjectMapper();
    objectMapper.registerModule(module);


    return Flux.using(() -> Files.lines(Paths.get(ClassLoader.getSystemResource("sample.json").toURI())),
        Flux::fromStream,
        BaseStream::close
    )
        .collectList()
        .map(lines -> String.join("\n", lines))
        .map(jsonContent -> {
          try {
            return objectMapper.readValue(jsonContent, MyPojo.class);
          } catch (JsonProcessingException e) {
            e.printStackTrace();
            return null;
          }
        })
        .map(MyPojo::getValues());

This works fine in local but fails when running inside a docker container. (I am using gradle build to build the jar file and then build the docker image from it)

Part of the error stacktrace:

> java.nio.file.FileSystemNotFoundException: null
at jdk.zipfs/jdk.nio.zipfs.ZipFileSystemProvider.getFileSystem(ZipFileSystemProvider.java:169) ~[jdk.zipfs:na]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:

答案1

得分: 3

当运行Docker镜像时,您正在尝试从Jar内部访问文件。似乎直接将资源URI传递给Paths.get是行不通的。

您可以参考以下关于同一问题的stackoverflow讨论:

  1. 这个答案
  2. 回答这个问题
  3. 回答这个问题

我认为您应该查看Spring的这个类,它使用Jackson 2.9将字节流解码为JSON,并转换为对象,利用非阻塞解析。

以下代码应该满足您的需求:

首先以Spring的方式获取对Resource的引用

  @Value("classpath:/sample.json")
  Resource sampleFile;

然后进行以下操作:

return new Jackson2JsonDecoder()
        .decodeToMono(
            DataBufferUtils.read(sampleFile, new DefaultDataBufferFactory(), 4096),
            ResolvableType.forClass(MyPojo.class),
            null,
            null)
        .map(object -> (MyPojo) object)
        .map(MyPojo::getValues);

通过这种方式,您可以使用DataBufferUtils.read以非阻塞方式读取文件,还可以将JSON映射到您的POJO中。

英文:

When running the docker image, you are trying to access the file from inside a Jar. And it seems that passing a resource URI directly to Paths.get won't work.

You can refer to the following stackoverflow discussions regarding the same:

  1. This answer
  2. Answers to this question
  3. Answers to this question

I think you should take a look at this class of Spring which decodes a byte stream into JSON and convert to Object's with Jackson 2.9, leveraging non-blocking parsing.

This code should serve your needs:

First get reference to the Resource in Spring's way

  @Value("classpath:/sample.json")
  Resource sampleFile;

Then do this:

return new Jackson2JsonDecoder()
        .decodeToMono(
            DataBufferUtils.read(sampleFile, new DefaultDataBufferFactory(), 4096),
            ResolvableType.forClass(MyPojo.class),
            null,
            null)
        .map(object -> (MyPojo) object)
        .map(MyPojo::getValues)

In this way, you can read the file in a non-blocking manner using DataBufferUtils.read and also map the json to your POJO.

huangapple
  • 本文由 发表于 2020年8月10日 17:37:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/63337681.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定