并发调用单例类方法会产生不一致的结果。

huangapple go评论72阅读模式
英文:

Concurrent calls to singleton class method produces inconsistent results

问题

我有一个单例类,它有一个单一的方法,用于从目录中读取所有文件。configRootDirContentType(一个用于类型引用的枚举)被传入。readAllConfigsFromLocalDisk 方法列出目录中的所有文件,然后根据 ContentType 参数逐个处理,将文件内容映射到预期的对象类型。

// 配置类型引用
public enum ConfigType {
    MY_TYPE, MY_OTHER_TYPE
}

// 单例类
public class Singleton {
    private static Singleton instance;
    private Map<String, MyType> myTypeMap = new HashMap();
    private Map<String, MyOtherType> myOtherTypeMap = new HashMap();

    private Singleton() {}

    public synchronized static Singleton getSingleton() {
        if (instance == null)
            instance = new Singleton();
        return instance;
    }

    public Map<String, MyType> getMyTypeMap(String filePath, ConfigType configType, String filePattern){
        myTypeMap.clear();
        readAllConfigsFromLocalDisk(configRootDir, configType, filePattern);
        return myTypeMap;
    }

    public Map<String, MyOtherType> getMyOtherTypeMap(String filePath, ConfigType configType, String filePattern){
        myOtherTypeMap.clear();
        readAllConfigsFromLocalDisk(configRootDir, configType, filePattern);
        return myOtherTypeMap;
    }

    /**
     * 获取配置根目录中的所有文件并逐个解析
     * @param configRootDir 配置根目录
     * @param configType 配置类型
     * @param filePattern 文件模式
     */
    private void readAllConfigsFromLocalDisk(String configRootDir, ConfigType configType, String filePattern) {
        try (Stream<Path> walk = Files.walk(Paths.get(configRootDir))) {
            Pattern pattern = Pattern.compile(filePattern);
            List<Path> filePaths = getLocalFilePaths(walk, pattern);

            if (!filePaths.isEmpty()) {
                for (Path filePath : filePaths) {
                    String relativePath = filePath.toString();
                    parseConfigFile(relativePath, configType);
                }
            }
        } catch (IOException ex) {
            logger.error("指定的配置根目录未找到。", ex);
        }
    }

    /**
     * 从本地磁盘读取给定的配置文件并映射到指定的配置类型
     *
     * @param filePath 本地磁盘上的配置文件的相对路径
     * @param configType 配置类型(MY_TYPE 或 MY_OTHER_TYPE)
     */
    private void parseConfigFile(String filePath, ConfigType configType ){
        String configContent = Files.readString(Paths.get(filePath), Charsets.UTF_8);
        
        // 根据配置类型解析并覆盖映射
        switch (configType) {
            case MY_TYPE:
                MyTypeConf myTypeConf = Core.getMapper().readValue(configContent, MyTypeConf.class);
                List<MyType> myTypeRefs = myTypeConf.getMyTypeList();
                myTypeMap.putAll(myTypeRefs.stream().collect(Collectors.toMap(MyType::getId, Function.identity())));
            case MY_OTHER_TYPE:
                MyOtherTypeConf myOtherTypeConf = Core.getMapper().readValue(configContent, MyOtherTypeConf.class);
                List<MyOtherType> myOtherTypeRefs = myOtherTypeConf.getMyOtherTypeList();
                myOtherTypeMap.putAll(myOtherTypeRefs.stream().collect(Collectors.toMap(MyOtherType::getId, Function.identity())));
        }
    }

    /**
     * 获取配置根目录中存在的所有匹配文件的文件路径,包括子文件夹。
     *
     * @param walk    配置根目录中路径的流
     * @param pattern 发现文件时要匹配的模式
     * @return 所有匹配模式的文件的Path对象列表。
     */
    private List<Path> getLocalFilePaths(Stream<Path> walk, Pattern pattern) {
        return walk.filter(Files::isRegularFile).filter(p -> {
            String fileName = p.getFileName().toString();
            Matcher matcher = pattern.matcher(fileName);
            return matcher.matches();
        }).collect(Collectors.toList());
    }
}

Akka actors 并发调用两个公共方法 getMyTypeMapgetMyOtherTypeMap 时,有时会出现 com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException

似乎原因是在尝试将文件内容映射到对象时,configContent 实际上是 MyType 可解析的,当尝试映射到 MyOtherType 时出现混淆。

我查看了一些其他地方,但无法完全理解。我正在尝试理解并发调用 readFile 时发生了什么,以及为什么会混淆文件内容。有人可以帮我理解吗?提前感谢。

英文:

I have a singleton class that has a single method that reads all files from a directory. The configRootDir and ContentType (An Enum for type reference) are passed in. readAllConfigsFromLocalDisk method lists all files in the directory and process one by one to map file content to an expected object type according to ContentType parameter.

// Config type reference
public enum ConfigType {
    MY_TYPE, MY_OTHER_TYPE
}

// Singleton class
public class Singleton {
    private static Singleton instance;
    private Map&lt;String, MyType&gt; myTypeMap = new HashMap();
    private Map&lt;String, MyOtherType&gt; myOtherTypeMap = new HashMap();

    private Singleton() {}

    public synchronized static Singleton getSingleton() {
        if (istance == null)
            istance = new Singleton();
        return istance;
    }

    public Map&lt;String,MyType&gt; getMyTypeMap(String filePath, ConfigType configType, String filePattern){
        myTypeMap.clear();
        readAllConfigsFromLocalDisk(configRootDir, configType, filePattern);
        return myTypeMap;
    }

    public Map&lt;String,MyOtherType&gt; getMyOtherTypeMap(String filePath, ConfigType configType, String filePattern){
        myOtherTypeMap.clear();
        readAllConfigsFromLocalDisk(configRootDir, configType, filePattern);
        return myOtherTypeMap;
    }

    /**
     * Get all files in config root directory and parse one by one
     * @param configRootDir Root directory for configurations
     * @param configType Configuration type
     * @param filePattern File pattern
     */
    private void readAllConfigsFromLocalDisk(String configRootDir, ConfigType configType, String filePattern) {
        try (Stream&lt;Path&gt; walk = Files.walk(Paths.get(configRootDir))) {
            Pattern pattern = Pattern.compile(filePattern);
            List&lt;Path&gt; filePaths = getLocalFilePaths(walk, pattern);

            if (!filePaths.isEmpty()) {
                for (Path filePath : filePaths) {
                    String relativePath = filePath.toString();
                    parseConfigFile(relativePath, configType);
                }
            }
        } catch (IOException ex) {
            logger.error(&quot;Specified config root directory not found.&quot;, ex);
        }
    }

    /**
     * Read a given configuration file  from local disk and map to specified config type
     *
     * @param configFile Relative path to config file on local disk
     * @param configType Configuration type (MY_TYPE or MY_OTHER_TYPE)
     */
    private void parseConfigFile(String filePath, ConfigType configType ){
        String configContent = Files.readString(Paths.get(filePath), Charsets.UTF_8);
        
        // Parse based on config type and overwrite map
        switch (configType) {
            case MY_TYPE:
                MyTypeConf myTypeConf = Core.getMapper().readValue(configContent, MyTypeConf.class);
                List&lt;MyType&gt; myTypeRefs = myTypeConf.getMyTypeList();
                myTypeMap.putAll(myTypeRefs.stream().collect(Collectors.toMap(MyType::getId, Function.identity())));
            case MY_OTHER_TYPE:
                MyOtherTypeConf myOtherTypeConf = Core.getMapper().readValue(configContent, MyOtherTypeConf.class);
                List&lt;MyOtherType&gt; myOtherTypeRefs = myOtherTypeConf.getMyOtherTypeList();
                myOtherTypeMap.putAll(myOtherTypeRefs.stream().collect(Collectors.toMap(MyOtherType::getId, Function.identity())));
        }
    }

    /**
     * Get file paths of all matching files exist in configured streaming directory and sub folders from disk.
     *
     * @param walk    Stream of paths in config root directory.
     * @param pattern Pattern to math when discovering files.
     * @return List of Path objects for all files matching the pattern.
     */
    private List&lt;Path&gt; getLocalFilePaths(Stream&lt;Path&gt; walk, Pattern pattern) {
        return walk.filter(Files::isRegularFile).filter(p -&gt; {
            String fileName = p.getFileName().toString();
            Matcher matcher = pattern.matcher(fileName);
            return matcher.matches();
        }).collect(Collectors.toList());
    }
}

Two public methods getMyTypeMap and getMyOtherTypeMap are called concurrently by a set of Akka actors. I get com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException when mapping the file content to Objects in some occasions.

It seems the reason is configContent was actually MyType parsable when trying to map it to MyOtherType and vise versa.

I looked at a few other places but unable to get the full picture of it. I'm trying to understand what happens when readFile is called concurrently and why it mixup file content. Can someone help me to understand this? Thanks in advance.

答案1

得分: 2

以下是您要翻译的内容:

您已经声明了两个共享变量:

private Map&lt;String, MyType&gt; myTypeMap = new HashMap();
private Map&lt;String, MyOtherType&gt; myOtherTypeMap = new HashMap();

由于HashMap不是线程安全的,当多个线程同时访问它的实例(至少有一个线程正在修改它)时,可能会发生奇怪的事情。

使用线程安全的映射不会解决语义问题,因为所有对getMyTypeMap的调用都返回相同的映射实例并对其进行操作,因此调用者不能可靠地使用返回的映射,因为其他线程仍然在执行getMyTypeMap并对其进行更改。对getMyOtherTypeMap的并发调用也是如此。

由于每个方法都以clear()调用开始,似乎不打算在方法的不同调用之间共享数据,因此这些方法不应共享数据。

似乎,您的主要障碍是如何重用代码以获取不同的结果类型。不要使用那个enum类型:

public class Singleton {
    /**
     * 类已经在第一次getSingleton()调用时延迟初始化
     */
    private static final Singleton instance = new Singleton();

    private Singleton() {}

    public static Singleton getSingleton() {
        return instance;
    }

    public Map&lt;String, MyType&gt; getMyTypeMap(String configRootDir){
        return readAllConfigsFromLocalDisk(configRootDir, "my-type-file-pattern",
            MyTypeConf.class, MyTypeConf::getMyTypeList, MyType::getId);
    }

    public Map&lt;String, MyOtherType&gt; getMyOtherTypeMap(String configRootDir){
        return readAllConfigsFromLocalDisk(configRootDir, "my-other-type-file-pattern",
            MyOtherTypeConf.class,MyOtherTypeConf::getMyOtherTypeList,MyOtherType::getId);
    }

    /**
     * 从本地磁盘读取所有配置文件并逐个解析
     * @param configRootDir 配置文件的根目录
     * @param filePattern 文件模式
     * @param confType 配置类型(MyTypeConf.class或MyOtherTypeConf.class)
     * @param getList 配置类型特定的列表访问器方法
     * @param getId 映射键的结果类型特定的Id访问器
     */
    private <T,C> Map<String,T> readAllConfigsFromLocalDisk(
        String configRootDir, String filePattern,
        Class<C> confType, Function<C,List<T>> getList, Function<T,String> getId) {

        try(Stream<Path> walk = Files.walk(Paths.get(configRootDir))) {
            Pattern pattern = Pattern.compile(filePattern);

            return getLocalFilePaths(walk, pattern)
                .flatMap(p -> this.parseConfigFile(p, confType, getList))
                .collect(Collectors.toMap(getId, Function.identity()));

        } catch(IOException|UncheckedIOException ex) {
            logger.error("未找到指定的配置根目录。", ex);
            return Collections.emptyMap();
        }
    }

    /**
     * 从本地磁盘读取给定的配置文件并映射到指定的配置类型
     *
     * @param configFile 本地磁盘上配置文件的路径
     * @param configType 配置类型(MyTypeConf.class或MyOtherTypeConf.class)
     * @param getList 配置类型特定的列表访问器方法
     */
    private <T,C> Stream<T> parseConfigFile(
        Path configFile, Class<C> configType, Function<C,List<T>> getList) {

        try {
            C conf=Core.getMapper().readValue(Files.readString(configFile), configType);
            List<T> tRefs = getList.apply(conf);
            return tRefs.stream();
        } catch(IOException ex) {
            throw new UncheckedIOException(ex);
        }
    }

    /**
     * 获取存在于配置的流目录中的所有匹配文件的文件路径
     * 并从磁盘中的子文件夹中获取。
     *
     * @param walk    配置根目录中路径的流。
     * @param pattern 发现文件时匹配的模式。
     * @return 所有匹配模式的文件的Path对象的流。
     */
    private Stream<Path> getLocalFilePaths(Stream<Path> walk, Pattern pattern) {
        return walk.filter(Files::isRegularFile).filter(p -> {
            String fileName = p.getFileName().toString();
            Matcher matcher = pattern.matcher(fileName);
            return matcher.matches();
        });
    }
}

希望这对您有所帮助。如果您需要进一步的翻译或解释,请随时告诉我。

英文:

You have declared two shared variables:

private Map&lt;String, MyType&gt; myTypeMap = new HashMap();
private Map&lt;String, MyOtherType&gt; myOtherTypeMap = new HashMap();

Since HashMap is not thread-safe, the strangest things can happen when multiple threads access an instance of it concurrently (and at least one thread is modifying it).

Using a thread safe map would not fix the semantic issue, as all invocations of getMyTypeMap return the same map instance and manipulate it, so a caller can not use the returned map reliably as other threads still executing getMyTypeMap are changing it (again). The same applies to concurrent calls of getMyOtherTypeMap.

Since each method starts with a clear() invocation, it seems that sharing data between different invocations of the method is not intended, therefore, these method should not share data.

It seems, the main obstacle to you was how to reuse the code for getting different result types. Don’t use that enum type:

public class Singleton {
/**
* Classes are already lazily initialized, on first getSingleton() call
*/
private static final Singleton instance = new Singleton();
private Singleton() {}
public static Singleton getSingleton() {
return instance;
}
public Map&lt;String, MyType&gt; getMyTypeMap(String configRootDir){
return readAllConfigsFromLocalDisk(configRootDir, &quot;my-type-file-pattern&quot;,
MyTypeConf.class, MyTypeConf::getMyTypeList, MyType::getId);
}
public Map&lt;String, MyOtherType&gt; getMyOtherTypeMap(String configRootDir){
return readAllConfigsFromLocalDisk(configRootDir, &quot;my-other-type-file-pattern&quot;,
MyOtherTypeConf.class,MyOtherTypeConf::getMyOtherTypeList,MyOtherType::getId);
}
/**
* Get all files in config root directory and parse one by one
* @param configRootDir Root directory for configurations
* @param filePattern File pattern
* @param confType Configuration type (MyTypeConf.class or MyOtherTypeConf.class)
* @param getList Configuration type specific list accessor method
* @param getId Result type specific Id accessor for the map key
*/
private &lt;T,C&gt; Map&lt;String,T&gt; readAllConfigsFromLocalDisk(
String configRootDir, String filePattern,
Class&lt;C&gt; confType, Function&lt;C,List&lt;T&gt;&gt; getList, Function&lt;T,String&gt; getId) {
try(Stream&lt;Path&gt; walk = Files.walk(Paths.get(configRootDir))) {
Pattern pattern = Pattern.compile(filePattern);
return getLocalFilePaths(walk, pattern)
.flatMap(p -&gt; this.parseConfigFile(p, confType, getList))
.collect(Collectors.toMap(getId, Function.identity()));
} catch(IOException|UncheckedIOException ex) {
logger.error(&quot;Specified config root directory not found.&quot;, ex);
return Collections.emptyMap();
}
}
/**
* Read a given configuration file from local disk and map to specified config type
*
* @param configFile Path to config file on local disk
* @param configType Configuration type (MyTypeConf.class or MyOtherTypeConf.class)
* @param getList Configuration type specific list accessor method
*/
private &lt;T,C&gt; Stream&lt;T&gt; parseConfigFile(
Path configFile, Class&lt;C&gt; configType, Function&lt;C,List&lt;T&gt;&gt; getList) {
try {
C conf=Core.getMapper().readValue(Files.readString(configFile), configType);
List&lt;T&gt; tRefs = getList.apply(conf);
return tRefs.stream();
} catch(IOException ex) {
throw new UncheckedIOException(ex);
}
}
/**
* Get file paths of all matching files exist in configured streaming directory
* and sub folders from disk.
*
* @param walk    Stream of paths in config root directory.
* @param pattern Pattern to math when discovering files.
* @return Stream of Path objects for all files matching the pattern.
*/
private Stream&lt;Path&gt; getLocalFilePaths(Stream&lt;Path&gt; walk, Pattern pattern) {
return walk.filter(Files::isRegularFile).filter(p -&gt; {
String fileName = p.getFileName().toString();
Matcher matcher = pattern.matcher(fileName);
return matcher.matches();
});
}
}

huangapple
  • 本文由 发表于 2020年8月11日 11:21:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/63351002.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定