ImmutableMap.Builder有限制吗?

huangapple go评论67阅读模式
英文:

Does ImmutableMap.Builder have a limit?

问题

我有一个类,大致如下所示:

public enum Animal {
BEAR,
SHEEP,
LION;

private static final Map<String, Animal> MAPPING = new ImmutableMap.Builder<String, Animal>()
.put("BLACK_BEAR", BEAR)
.put("WHITE_BEAR", BEAR)
...
.put("African Lion", LION)
.build()
}

大约有1500个映射条目。

我尝试构建我的应用程序,但出现以下错误:

[javac] An annotation processor threw an uncaught exception.
[javac] Consult the following stack trace for details.
[javac] java.lang.StackOverflowError
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
[javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
[javac] 	at lombok.core.AST.buildWithField(AST.java:284)
[javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
[javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
[javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
[javac] 	at lombok.core.AST.buildWithField(AST.java:284)
[javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
[javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
[javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
[javac] 	at lombok.core.AST.buildWithField(AST.java:284)
[javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
[javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
[javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
[javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
[javac] 	at lombok.core.AST.buildWithField(AST.java:284)

我尝试将映射的大小减少到大约15个,可以成功构建。在静态不可变映射中,我们能够硬编码的大小是否有限制?我在文档中没有看到任何限制。问题可能是什么?
英文:

I have a class that looks something like this:

public enum Animal {
   BEAR,
   SHEEP,
   LION;

   private static final Map&lt;String, Animal&gt; MAPPING = new ImmutableMap.Builder&lt;String, Animal&gt;()
      .put(&quot;BLACK_BEAR&quot;, BEAR)
      .put(&quot;WHITE_BEAR&quot;, BEAR)
      ...
      .put(&quot;African Lion&quot;, LION)
      .build()
}

About 1500 map entries.

I try to build my application and I get the following error:

    [javac] An annotation processor threw an uncaught exception.
    [javac] Consult the following stack trace for details.
    [javac] java.lang.StackOverflowError
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
    [javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
    [javac] 	at lombok.core.AST.buildWithField(AST.java:284)
    [javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
    [javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
    [javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
    [javac] 	at lombok.core.AST.buildWithField(AST.java:284)
    [javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
    [javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
    [javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
    [javac] 	at lombok.core.AST.buildWithField(AST.java:284)
    [javac] 	at lombok.javac.JavacAST.drill(JavacAST.java:374)
    [javac] 	at lombok.javac.JavacAST.buildStatementOrExpression(JavacAST.java:340)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:200)
    [javac] 	at lombok.javac.JavacAST.buildTree(JavacAST.java:67)
    [javac] 	at lombok.core.AST.buildWithField0(AST.java:386)
    [javac] 	at lombok.core.AST.buildWithField(AST.java:284)

I tried reducing the size of the map to 15ish and it builds fine. Is there a limit to the size we can harcode a static immutable map? I didn't see any limitation in the documentation. What could the issue be?

答案1

得分: 2

这不是ImmutableBuilder的问题;问题在于您的源文件现在包含了大量的AST节点,导致lombok的处理占用了太多内存,我猜测这实际上是lombok的一个技术性缺陷,尽管javac可以处理,但从技术上讲是lombok的一个bug。换句话说,编译器无法将您的java文件转换为类文件,但如果可以的话,它应该能正常工作*。

我通常认为,任何试图在源代码中定义如此多数据的源文件都是代码异味。

我建议您将这些数据放入一个文本文件中。

将此文本文件与您的源文件一起放置;如果您使用maven或更标准的源目录结构,可以这样做:

src/main/java/com/foo/yourpkg/Animal.java
src/main/resources/com/foo/yourpkg/AnimalData.txt

这将自动使txt文件与您的类文件位于相同的位置,即使在jar包中也是如此。然后,在您的animal文件中读取它:

public enum Animal {

    private static final Map&lt;String, Animal&gt; MAPPING;
    static {
        try {
            Map&lt;String, String&gt; example = new HashMap&lt;String, String&gt;();
            try (var in = Animal.class.getResourceAsStream("AnimalData.txt")) {
                BufferedReader br = new BufferedReader(new InputStreamReader(
                  in, StandardCharsets.UTF_8));

                for (String line = br.readLine(); line != null; line = br.readLine()) {
                    String[] p = line.split("\t", 2);
                    map.put(p[0], p[1]);
                }
            }
            MAPPING = Collections.unmodifiableMap(map);
        } catch (IOException e) {
            throw new InternalError("data corrupted");
        }
    }
}

这个假设性的示例要求您在文本文件中放置以制表符分隔的字符串对。您可以使用这个示例并根据自己的需求进行调整。

免责声明:我是lombok的核心贡献者,但这个问题似乎是最低优先级的问题之一,我们有很多bug报告要处理,然后才能处理“如果在源代码中尝试创建一个1500大小的immutablemap builder会导致内存耗尽”的问题。

*) 可能;类文件也有限制,如果您删除lombok或增加内存,仍然有可能在稍后的编译阶段因为超过64k字节码限制而拒绝编译这段代码。

英文:

It's not ImmutableBuilder; it's the fact that your source file is now such a gigantic pile of AST nodes that lombok's processing ends up taking too much memory, which, I guess, given that javac can handle it, is technically a bug in lombok. In other words, the compiler can't manage to turn your java file into a class file, but if it could, it would work fine*.

I would in general consider any source file that tries to define that much data in source as a code smell.

I suggest you take whatever data this is and put it in a text file.

Put this text file together with your source files; if you use maven / the more or less standard source dir structure, you'd have, say:

src/main/java/com/foo/yourpkg/Animal.java
src/main/resources/com/foo/yourpkg/AnimalData.txt

which will automatically result in the txt file being in the same place as your class file is, even in jars. Then, to read it, from within your animal file:

public enum Animal {

    private static final Map&lt;String, Animal&gt; MAPPING;
    static {
        try {
            Map&lt;String, String&gt; example = new HashMap&lt;String, String&gt;();
            try (var in = Animal.class.getResourceAsStream(&quot;AnimalData.txt&quot;)) {
                BufferedReader br = new BufferedReader(new InputStreamReader(
                  in, StandardCharsets.UTF_8));

                for (String line = br.readLine(); line != null; line = br.readLine()) {
                    String[] p = line.split(&quot;\t&quot;, 2);
                    map.put(p[0], p[1]);
                }
            }
            MAPPING = Collections.unmodifiableMap(map);
        } catch (IOException e) {
            throw new InternalError(&quot;data corrupted&quot;);
        }
    }
}

This hypothetical example would require you to put tab-separated string pairs in the text file. You can use this example and adjust it to your needs.

DISCLAIMER: I'm a core contributor of lombok, but this is about as low a priority as I can imagine, and we have plenty of bug reports to dig through before we'd get to 'out of memory issues if you try to do a 1500-size immutablemap builder in source'.

*) Possibly; class files have limits too, it's possible if you remove lombok or add memory, that the compiler will in a later phase still end up refusing to compile this code because it would blow past the 64k bytecode limit on your initialization.

答案2

得分: 0

  1. 只需将您的堆栈大小增加,通过在运行构建的JVM中添加-Xss4m(或更高)。
  2. 编辑 鉴于枚举本身很小,您可以这样做:
package org.example;

import java.util.HashMap;
import java.util.Map;

public enum Animal {
    BEAR("白熊", "黑熊", "棕熊"),
    DOG("杂种", "混血"),
    CAT("blah3");

    private final String[] blah1;

    private static final Map<String, Animal> MAPPING = new HashMap<>();

    static {
        for (Animal animal : Animal.values()) {
            for (String tag : animal.blah1) {
                MAPPING.put(tag, animal);
            }
        }
    }

    Animal(String... blah1) {
        this.blah1 = blah1;
    }

    public static void main(String... s) {
        String animal = "黑熊";
        System.out.println(animal + "是" + MAPPING.get(animal));
    }
}
英文:

I agree with @rzwitserloot above, but there are some things you could try:

  1. Just increase your stack size by adding -Xss4m (or higher) to the jvm that's running your build.
  2. edit As the enum itself is small, you could do this:
package org.example;

import java.util.HashMap;
import java.util.Map;

public enum Animal {
    BEAR(&quot;white bear&quot;, &quot;black bear&quot;, &quot;brown bear&quot;),
    DOG(&quot;mongrel&quot;, &quot;cross&quot;),
    CAT(&quot;blah3&quot;);

    private final String[] blah1;

    private static final Map&lt;String, Animal&gt; MAPPING = new HashMap&lt;&gt;();

    static {
        for (Animal animal: Animal.values()) {
            for (String tag : animal.blah1) {
                MAPPING.put(tag, animal);
            }
        }
    }

    Animal(String... blah1) {
        this.blah1 = blah1;
    }


    public static void main(String... s) {
        String animal = &quot;black bear&quot;;
        System.out.println(&quot; A &quot; + animal + &quot; is a &quot; + MAPPING.get(animal));
    }
}

huangapple
  • 本文由 发表于 2020年10月9日 11:40:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/64273611.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定