Hazelcast 实例尝试直接访问 Kubernetes Pods,而不是负载均衡器的外部 IP。

huangapple go评论49阅读模式
英文:

Hazelcast instance trying to access Kubernetes pods directly instead of load balancer's external IP

问题

我在Minikube中部署了一个包含2个成员的Hazelcast集群,并设置了一个负载均衡器服务来访问这些成员。我的Spring Boot应用程序部署在Minikube之外,使用以下配置初始化Hazelcast实例:

public Config getHazelcastConfig() {
    Config config = new Config("default");

    config.getNetworkConfig().setPort(5701);
    config.getNetworkConfig().setPortCount(1);
    config.getNetworkConfig().setPortAutoIncrement(false);

    config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
    config.getNetworkConfig()
            .getJoin()
            .getTcpIpConfig()
            .addMember("external-ip:port")
            .setEnabled(true);
    return config;
}

@Bean
public HazelcastInstance hazelcastInstance() {
    Config config = getHazelcastConfig();
    return Hazelcast.newHazelcastInstance(config);
}

当我在重新构建后首次启动Spring Boot应用程序时,我可以成功将Minikube成员添加到集群中,总共有3个成员,正如预期的那样。但是,当我重新启动应用程序时,它试图直接使用它们的私有IP地址访问Kubernetes pods。以下是相关的日志:

2023-07-02 20:39:16.831  INFO 32243 --- [.IO.thread-in-0] c.h.i.server.tcp.TcpServerConnection     : [10.31.0.28]:5701 [dev] [4.2.1] Initialized new cluster connection between /192.168.105.1:50190 and /10.108.94.145:5701
2023-07-02 20:39:16.898  INFO 32243 --- [cached.thread-3] c.h.i.server.tcp.TcpServerConnector      : [10.31.0.28]:5701 [dev] [4.2.1] Connecting to /10.244.0.2:5701, timeout: 10000, bind-any: true
2023-07-02 20:39:26.902  INFO 32243 --- [cached.thread-3] c.h.i.server.tcp.TcpServerConnector      : [10.31.0.28]:5701 [dev] [4.2.1] Could not connect to: /10.244.0.2:5701. Reason: IOException[null to address /10.244.0.2:5701]
2023-07-02 20:39:26.903  INFO 32243 --- [cached.thread-3] c.h.internal.cluster.impl.TcpIpJoiner    : [10.31.0.28]:5701 [dev] [4.2.1] [10.244.0.2]:5701 is added to the blacklist.
2023-07-02 20:39:36.939  INFO 32243 --- [cached.thread-3] c.h.internal.cluster.impl.TcpIpJoiner    : [10.31.0.28]:5701 [dev] [4.2.1] [10.244.0.2]:5701 is added to the blacklist.

IP 10.244.0.2 对应于Kubernetes pods(成员)之一。但是,我的应用程序应该仅通过负载均衡器的外部IP访问成员。我不确定为什么它试图直接访问pods。

如何强制我的Hazelcast实例始终通过负载均衡器的外部IP访问成员?任何指导或建议将不胜感激。

注意:我没有使用ClientConfig,因为我正在尝试将部署在Kubernetes中的成员与启动我的Spring Boot应用程序的成员一起添加为新成员。
我已经按照这个链接设置了我的Minikube

英文:

I have a Hazelcast cluster deployed in Minikube with 2 members. I have set up a load balancer service to access these members. My Spring Boot application, which is deployed outside of Minikube, uses the following configuration to initialize the Hazelcast instance:

public Config getHazelcastConfig() {
    Config config = new Config("default");

    config.getNetworkConfig().setPort(5701);
    config.getNetworkConfig().setPortCount(1);
    config.getNetworkConfig().setPortAutoIncrement(false);

    config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false);
    config.getNetworkConfig()
            .getJoin()
            .getTcpIpConfig()
            .addMember("external-ip:port")
            .setEnabled(true);
    return config;
}
@Bean
public HazelcastInstance hazelcastInstance() {
    Config config = getHazelcastConfig();
    return Hazelcast.newHazelcastInstance(config);
}

When I start the Spring Boot application for the first time after rebuilding it, I can successfully add the Minikube members to the cluster, resulting in a total of 3 members as expected. However, when I restart the application, it tries to access the Kubernetes pods directly using their private IP addresses. Here are the relevant logs:

2023-07-02 20:39:16.831  INFO 32243 --- [.IO.thread-in-0] c.h.i.server.tcp.TcpServerConnection     : [10.31.0.28]:5701 [dev] [4.2.1] Initialized new cluster connection between /192.168.105.1:50190 and /10.108.94.145:5701
2023-07-02 20:39:16.898  INFO 32243 --- [cached.thread-3] c.h.i.server.tcp.TcpServerConnector      : [10.31.0.28]:5701 [dev] [4.2.1] Connecting to /10.244.0.2:5701, timeout: 10000, bind-any: true
2023-07-02 20:39:26.902  INFO 32243 --- [cached.thread-3] c.h.i.server.tcp.TcpServerConnector      : [10.31.0.28]:5701 [dev] [4.2.1] Could not connect to: /10.244.0.2:5701. Reason: IOException[null to address /10.244.0.2:5701]
2023-07-02 20:39:26.903  INFO 32243 --- [cached.thread-3] c.h.internal.cluster.impl.TcpIpJoiner    : [10.31.0.28]:5701 [dev] [4.2.1] [10.244.0.2]:5701 is added to the blacklist.
2023-07-02 20:39:36.939  INFO 32243 --- [cached.thread-3] c.h.internal.cluster.impl.TcpIpJoiner    : [10.31.0.28]:5701 [dev] [4.2.1] [10.244.0.2]:5701 is added to the blacklist.

The IP 10.244.0.2 corresponds to one of the Kubernetes pods (members). However, my application should only access the members through the load balancer's external IP. I'm not sure why it's trying to access the pods directly.

How can I enforce my Hazelcast instance to always access the members via the load balancer's external IP? Any guidance or suggestions would be greatly appreciated.

Note: I am not using the ClientConfig because Im trying to add the members deployed in Kubernetes as new members along with the one member which starts with my spring boot application.
I have setup my Minikube referring this
Thank you.

答案1

得分: 1

Cluster members maintain open sockets to each other, a single LoadBalancer can't make two members inside Kubernetes accessible to one member outside.

英文:

Ongoing discussion on Hazelcast Community Slack

Cluster members maintain open sockets to each other, a single LoadBalancer can 't make two members inside Kubernetes accessible to one member outside.

huangapple
  • 本文由 发表于 2023年7月3日 03:16:15
  • 转载请务必保留本文链接:https://go.coder-hub.com/76600442.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定