通过Spring Data存储库保存内部包含大型Map的POJO导致StackOverflowError。

huangapple go评论62阅读模式
英文:

Save POJO with big Map inside via Spring Data repository leads to StackOverflowError

问题

一般情况下,我从Kafka Stream中读取序列化对象(以JSON格式),然后尝试使用Spring Data repository将其保存到Redis中。
在调用repository.save()两次之后(对象未保存到Redis中),我会收到StackOverFlowError错误:

Exception in thread "processOffers-applicationId-1c24ef63-baae-47b9-beb7-5e6517736bc4-StreamThread-1" java.lang.StackOverflowError
at org.springframework.data.util.Lazy.get(Lazy.java:94)
at org.springframework.data.mapping.model.AnnotationBasedPersistentProperty.usePropertyAccess(AnnotationBasedPersistentProperty.java:277)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:134)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:115)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:601)
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:353)
at org.springframework.data.redis.core.convert.MappingRedisConverter.writeInternal(MappingRedisConverter.java:597)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:639)

序列化后的POJO类如下:

@Data
@With
@NoArgsConstructor
@AllArgsConstructor
@RedisHash("students")
public class Student {
    @Id
    @JsonProperty("student_id")
    private long id;

    @JsonProperty("entities")
    private Map<String, Object> entities = new HashMap<>();
}

entities映射包含100多个条目,其中包含嵌套的映射(对象)。
有趣的部分是:如果我将映射设置为空,一切都能正常工作,数据会立即保存到Redis中。

与POJO相对应的存储库:

@Repository
public interface StudentRepository extends CrudRepository<Student, Long> {
}

此外,我已经为长整型id字段定义了RedisCustomConversion:

@Component
@ReadingConverter
public class BytesToLongConverter implements Converter<byte[], Long> {
    @Override
    public Long convert(final byte[] source) {
        ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
        buffer.put(source);
        buffer.flip();
        return buffer.getLong();
    }
}

@Component
@WritingConverter
public class LongToBytesConverter implements Converter<Long, byte[]> {
    @Override
    public byte[] convert(final Long source) {
        ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
        buffer.putLong(source);
        return buffer.array();
    }
}

Redis配置类如下所示:

@Configuration
@EnableRedisRepositories
public class RedisConfiguration {
    @Bean
    @Primary
    public RedisProperties redisProperties() {
        return new RedisProperties();
    }

    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        var config = new RedisStandaloneConfiguration();
        var props = redisProperties();
        config.setHostName(props.getHost());
        config.setPort(props.getPort());
        return new JedisConnectionFactory(config);
    }

    @Bean
    public RedisTemplate<String, Object> redisTemplate() {
        var template = new RedisTemplate<String, Object>();
        template.setConnectionFactory(redisConnectionFactory());
        template.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
        return template;
    }

    @Bean
    public RedisCustomConversions redisCustomConversions(LongToBytesConverter longToBytes,
                                                         BytesToLongConverter bytesToLong) {
        return new RedisCustomConversions(Arrays.asList(longToBytes, bytesToLong));
    }
}

更新
我在Spring Data Redis Jira上找到了这个问题,但是解决方案被设定为“已修复”,所以这对我来说看起来有些奇怪。

英文:

Generally: i'm reading serialized object (as JSONs) from Kafka Stream and trying to save it to Redis using Spring Data repository.
<br>
After a two calls (objects has not been saved to Redis) to repository.save() i get StackOverFlowError:

Exception in thread &quot;processOffers-applicationId-1c24ef63-baae-47b9-beb7-5e6517736bc4-StreamThread-1&quot; java.lang.StackOverflowError
at org.springframework.data.util.Lazy.get(Lazy.java:94)
at org.springframework.data.mapping.model.AnnotationBasedPersistentProperty.usePropertyAccess(AnnotationBasedPersistentProperty.java:277)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:134)
at org.springframework.data.mapping.model.BeanWrapper.getProperty(BeanWrapper.java:115)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:601)
at org.springframework.data.mapping.model.BasicPersistentEntity.doWithProperties(BasicPersistentEntity.java:353)
at org.springframework.data.redis.core.convert.MappingRedisConverter.writeInternal(MappingRedisConverter.java:597)
at org.springframework.data.redis.core.convert.MappingRedisConverter.lambda$writeInternal$2(MappingRedisConverter.java:639)

<br>

Serialized POJO look like this:

@Data
@With
@NoArgsConstructor
@AllArgsConstructor
@RedisHash(&quot;students&quot;)
public class Student {
    @Id
    @JsonProperty(&quot;student_id&quot;)
    private long id;

    @JsonProperty(&quot;entities&quot;)
    private Map&lt;String, Object&gt; entities = new HashMap&lt;&gt;();
}

Map entities contains 100+ Entries, with nested maps (objects).
<br>
Interesting part: if i make map empty everything works fine and data instantly saved to Redis.
<br>
<br>
Corresponding repository for POJO:

@Repository
public interface StudentRepository extends CrudRepository&lt;Student, Long&gt; {
}

Also, i've defined RedisCustomConversion for Long id field:

@Component
@ReadingConverter
public class BytesToLongConverter implements Converter&lt;byte[], Long&gt; {
    @Override
    public Long convert(final byte[] source) {
        ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
        buffer.put(source);
        buffer.flip();
        return buffer.getLong();
    }
}

@Component
@WritingConverter
public class LongToBytesConverter implements Converter&lt;Long, byte[]&gt; {
    @Override
    public byte[] convert(final Long source) {
        ByteBuffer buffer = ByteBuffer.allocate(Long.BYTES);
        buffer.putLong(source);
        return buffer.array();
    }
}

Redis configuration class looks like this:

@Configuration
@EnableRedisRepositories
public class RedisConfiguration {
    @Bean
    @Primary
    public RedisProperties redisProperties() {
        return new RedisProperties();
    }

    @Bean
    public RedisConnectionFactory redisConnectionFactory() {
        var config = new RedisStandaloneConfiguration();
        var props = redisProperties();
        config.setHostName(props.getHost());
        config.setPort(props.getPort());
        return new JedisConnectionFactory(config);
    }

    @Bean
    public RedisTemplate&lt;String, Object&gt; redisTemplate() {
        var template = new RedisTemplate&lt;String, Object&gt;();
        template.setConnectionFactory(redisConnectionFactory());
        template.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
        return template;
    }

    @Bean
    public RedisCustomConversions redisCustomConversions(LongToBytesConverter longToBytes,
                                                         BytesToLongConverter bytesToLong) {
        return new RedisCustomConversions(Arrays.asList(longToBytes, bytesToLong));
    }
}

UPD:
I've found this issue on Spring Data Redis Jira, but the resolution set as "Fixed", so it's seems strange to me.

答案1

得分: 0

我已经为我的POJO内部映射定义了自定义的WritingConverter和ReadingConverter,使用了GenericJackson2JsonRedisSerializer,一切都运行正常!

@Component
@WritingConverter
public class FieldsToBytesConverter implements Converter<Map<String, Object>, byte[]> {
    private final RedisSerializer serializer;

    public FieldsToBytesConverter() {
        serializer = new GenericJackson2JsonRedisSerializer();
    }

    @Override
    public byte[] convert(Map<String, Object> value) {
        return serializer.serialize(value);
    }
}

@Component
@ReadingConverter
public class BytesToFieldsConverter implements Converter<byte[], Map<String, Object>> {
    private final GenericJackson2JsonRedisSerializer serializer;

    public BytesToFieldsConverter() {
        serializer = new GenericJackson2JsonRedisSerializer();
    }

    @Override
    public Map<String, Object> convert(byte[] value) {
        return (Map<String, Object>) serializer.deserialize(value);
    }
}
英文:

I've defined custom WritingConverter and ReadingConverter for my inner map in POJO using GenericJackson2JsonRedisSerializer and everything worked out!
Code:

@Component
@WritingConverter
public class FieldsToBytesConverter implements Converter&lt;Map&lt;String, Object&gt;, byte[]&gt; {
    private final RedisSerializer serializer;

    public FieldsToBytesConverter() {
        serializer = new GenericJackson2JsonRedisSerializer();
    }

    @Override
    public byte[] convert(Map&lt;String, Object&gt; value) {
        return serializer.serialize(value);
    }
}

@Component
@ReadingConverter
public class BytesToFieldsConverter implements Converter&lt;byte[], Map&lt;String, Object&gt;&gt; {
    private final GenericJackson2JsonRedisSerializer serializer;

    public BytesToFieldsConverter() {
        serializer = new GenericJackson2JsonRedisSerializer();
    }

    @Override
    public Map&lt;String, Object&gt; convert(byte[] value) {
        return (Map&lt;String, Object&gt;) serializer.deserialize(value);
    }
}

huangapple
  • 本文由 发表于 2020年8月18日 01:06:49
  • 转载请务必保留本文链接:https://go.coder-hub.com/63455478.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定