Kafka代理可以通过多个端口进行连接吗?

huangapple go评论45阅读模式
英文:

Can Kafka brokers connect through more than one port

问题

我们有很多微服务连接到一个 Kafka 代理集群,作为企业消息总线。
在我们的可靠性测试中,我们想测试系统如何从基础设施故障中恢复,我们使用 Gremlin 来实现这个目的。如果我们想模拟 Kafka 的故障,Gremlin 将切断到端口 9092 的网络连接,这是 Kafka 通信的端口。
问题在于,当切断该端口上的网络连接时,所有连接到 Kafka 的微服务都会受到影响,而不仅仅是我们想要测试的那个微服务。
一个想法是让 Kafka 暴露多个端口,比如 9092 和 9093,并且让我们的应用通过 9092 连接,而所有其他微服务将继续通过 9093 运行。

我不知道这是否可行,这是我的问题。
如果这不可行,您还有哪些其他想法来实现我们想要的目标呢?

提前感谢您。

英文:

We have quite a few micro-services connecting to a cluster of Kafka brockers as a enterprise message bus.
Part of our resiliency testing we want to test how a given system recovers from infrastructure failure and we use Gremlin for this purpose. If we want to simulate a Kafka failure Gremlin will cut off the network connection to port 9092 which is the port Kafka communicates through.
The issue with this is that when cutting off the networking on that port all our micro-services connecting to Kafka are impacted not only the one we want to test.
One idea would be to have Kafka expose more than one port, say 9092 and 9093 and have our app connecting via 9092 whereas all other micro-services will keep running via 9093.

I don't know if this is possible and this is my question about.
If this is not possible, what other ideas to achieving what we want would you have.

Thank you in advance.

答案1

得分: 0

Kafka listeners 服务器配置可以绑定服务器到多个端口,但这不适用于你所要求的多路复用用例。bootstrap.servers 应该仅连接到一个主机:端口,每个服务器,作为CSV列表。而是,这用于公开不同的协议,如SSL / SASL_SSL / SASL_PLAINTEXT等。

所有连接到Kafka的微服务都受到影响,不仅仅是我们想要测试的那一个

不清楚你是在路由器上切断网络,还是在应用程序上,还是在一个代理上。如果在代理上,那么听起来你没有正确设置一些主题的复制因子,因为Kafka客户端将始终重新连接到同步分区的领导者。

英文:

Kafka listeners server config can bind the server to multiple ports, yes, but this is not for a multiplexing use case as you seem to be asking for. bootstrap.servers should only connect to one host:port, per server, as a CSV list. Rather, this is used to expose different protocols like SSL / SASL_SSL / SASL_PLAINTEXT, etc

> all our micro-services connecting to Kafka are impacted not only the one we want to test

Unclear if you're cutting off the network at the router, or at the app, or from one broker. If at the broker, then sounds like you've not setup replication factors correctly on some topics, as Kafka clients will always reconnect to in sync partition leaders

huangapple
  • 本文由 发表于 2023年6月6日 08:50:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/76410790.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定