将单个节点转换为使用Hazelcast的分布式Java应用程序。

huangapple go评论70阅读模式
英文:

Convert a single node to distributed java application with Hazelcast

问题

我们有一个当前的Java应用程序,使用Vert.x构建,使用简单的本地HashMaps。为了在多个节点上分发我们的应用程序,我们想要将本地的HashMaps替换为一个Hazelcast的IMap

基本上,我们有两个HashMaps

Map<String, Group> // <groupId - group>
Map<String, Set<String>  // <memberId - set of groupIds>

我们有两个HashMaps,以便可以轻松地通过它们的ID查询我们的组,并知道我们的成员属于哪些组。

然而,使用IMap,根据文档,应该可以创建一个映射:

IMap<String, Group> // <groupId - group>

class Group {
  String id;
  Set<Member> members;
}

class Member {
  String id;
  // 一些数据
}

并且我们应该能够执行:

Collection<Group> groups = imap.values(Predicates.in("members", varargs of member ids))

所以,我有4个问题:

  1. 你能确认这种行为在IMap中是可能的吗?
  2. 如果是的话,imap.values返回的组集合是否可能包含重复项(相同的groupId)?我是否需要执行一种“distinct”操作?
  3. 我们不确定是应该使用Vertx + Hazelcast,还是将所有处理代码移到Hazelcast Jet。仅迁移到Hazelcast Jet有什么优势?
  4. 如果我们使用Hazelcast Jet,是否可以在StreamStage中像这样使用IMap.values(Predicate)功能:
StreamStage.flatMap(memberIds -> Traversers.traverseIterable(imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0]))))

非常感谢!

英文:

We have a current java application built with Vert.x using simple local HashMaps. In order to distribute our application on several nodes, we would like to replace local HashMaps by a single Hazelcast IMap.

Basically we have our 2 HashMaps :

Map<String, Group> // <groupId - group>
Map<String, Set<String>  // <memberId - set of groupIds>

We have 2 HashMaps in order to query easily our groups by their ids and to know to which groups belong our members.

However, with IMap, according to documentation, it should be possible to create only one map :

IMap<String, Group> // <groupId - group>

class Group {
  String id;
  Set<Member> members;
}

class Member {
  String id;
  // some data
}

and we should be able to perform :

Collection<Group> groups = imap.values(Predicates.in("members", varargs of member ids))

So, I have 4 questions :

  1. Could you confirm this behavior is possible with IMap ?
  2. If yes, is it possible the collection of groups returned by imap.values contain duplicates (same groupId) ? Do I have to perform a kind of distinct operation ?
  3. We are not sure if we should use Vertx + Hazelcast, or move all the processing code to Hazelcast Jet. What would be the advantages to move to Hazelcast Jet only ?
  4. If we use Hazelcast Jet, is it possible to use IMap.values(Predicate) feature inside StreamStage like that :
StreamStage.flatMap(memberIds -> Traversers.traverseIterable(imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0])))

Thanks a lot !

答案1

得分: 2

  1. 是的

  2. 如果您的原始映射中不包含重复项,并且IMap没有并发更新,它不应该包含重复项。

  3. 这取决于您的用例,我不能一般性地告诉您。

  4. 这不太好,因为imap.values调用是阻塞的,而在flatMap中的函数不能阻塞。我所说的阻塞是指它执行阻塞的IO操作。另外,您不能从局部变量中捕获imap实例,因为它不是可序列化的,Jet无法将其发送到成员。但是您可以使用mapUsingService,并像这样使用ServiceFactories.imapService

 .mapUsingService(
         ServiceFactories.iMapService("my_map").toNonCooperative(),
         (imap, memberIds) -> 
             imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0]))))
英文:
  1. yes

  2. it should not contain duplicates if your original map doesn't contain them and the IMap isn't concurrently updated

  3. depends on your use case, I can't tell in general

  4. This is not good as the imap.values call is blocking and the function in flatMap must not block. By blocking I mean it does blocking IO operations. Also you can't capture the imap instance from a local variable because it's not serializable and Jet can't send it to the member. But you can use mapUsingService and use ServiceFactories.imapService like this:

 .mapUsingService(
         ServiceFactories.iMapService("my_map").toNonCooperative(),
         (imap, memberIds) -> 
             imap.values(Predicates.in("memberIds", memberIds.toArray(new String[0]))))

huangapple
  • 本文由 发表于 2020年8月16日 17:25:01
  • 转载请务必保留本文链接:https://go.coder-hub.com/63435207.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定