In micronaut-kafka, How can I use JAAS config to two different consumers from one application?

huangapple go评论54阅读模式
英文:

In micronaut-kafka, How can I use JAAS config to two different consumers from one application?

问题

我尝试了官方文档中提到的覆盖Kafka客户端配置中引导服务器的方法。

然而,在我的情况下,我从其他bean依赖项获取了一个JaaS配置,另一个在秘密路径中可用。拥有自定义配置只加载一个JaaS配置,另一个消费者会断开连接。

例如,

kafka:
  sasl:
    mechanism: PLAIN
    jaas:
      config: >-
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="$ConnectionString"
        password="%s";
  security:
    protocol: SASL_SSL
  consumers:
    abc-consumer-client:
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="$ConnectionString"
            password="%s";
      security:
        protocol: SASL_SSL
    xyz-client:
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="$ConnectionString"
            password="%s";
      security:
        protocol: SASL_SSL

我将micronaut-kafka与Spring Kafka实现关联起来。似乎我需要覆盖这些bean来实现这一点,但我陷入了应该以什么顺序覆盖以及以什么顺序来实现这一点的困境。

类似的Spring Kafka参考- https://stackoverflow.com/questions/60825373/spring-kafka-application-properties-configuration-for-jaas-sasl-not-working

我尝试了micronaut文档中列出的选项,但在我的情况下,其他JaaS配置来自其他bean依赖项,该依赖项进行GRPC调用以获取引导URL和JaaS配置。

我在这里寻找的是以哪种顺序覆盖micronaut-kafka bean,以实现两个消费者连接到两个不同的引导服务器,每个引导服务器都有自己的JaasConfig,而其他JaasConfig依赖于其他服务调用。

英文:

I have tried the approach mentioned on official documentations to override the bootstrap servers in kafka client config.

https://micronaut-projects.github.io/micronaut-kafka/latest/guide/#kafkaClient

However, in my case, I am getting one JaaS config from other bean dependency and other one is available in secret path. Having a custom configuration just load one JaaS config and other consumer get disconnected.

For example,

kafka:
  sasl:
    mechanism: PLAIN
    jaas:
      config: >-
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="$ConnectionString"
        password="%s";
  security:
    protocol: SASL_SSL
  consumers:
    abc-consumer-client:
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="$ConnectionString"
            password="%s";
      security:
        protocol: SASL_SSL
    xyz-client:
      sasl:
        mechanism: PLAIN
        jaas:
          config: >-
            org.apache.kafka.common.security.plain.PlainLoginModule required
            username="$ConnectionString"
            password="%s";
      security:
        protocol: SASL_SSL

I related the micronaut-kafka with spring kafka implementation. It seems like I will have to override the beans to achieve this but getting stuck on which order I should override and in which sequence to achieve this.

Similar Spring Kafka reference- https://stackoverflow.com/questions/60825373/spring-kafka-application-properties-configuration-for-jaas-sasl-not-working

I have tried the options listed on micronaut documentation but in my case, other jaas config is coming from other bean depedency which makes a GRPC call to fetch bootstrap URL and JaaS config.

What am I looking for here is the order in which I should override the micronaut-kafka beans to achieve two consumer connecting to two different bootstrap servers with each having its own JaasConfig and other JaasConfig is dependent on other service call.

答案1

得分: 0

最后,我成功解决了这个问题。

为了解决这个问题,我不得不覆盖KafkaDefaultConfiguration bean,并根据groupID在环境变量中包含属性。

实际上,您可以在bean实例化阶段运行时覆盖属性。
例如,

environment.addPropertySource(PropertySource.of("overrides",
                Map.of("kafka.consumers." + GroupId + ".bootstrap.servers", baseUrl,
                        "kafka.consumers." + GroupId + ".sasl.jaas.config", newJaasConfig )));
英文:

Finally, I was able to figure out this problem myself.

In order to solve this problem, I had to override the KafkaDefaultConfiguration bean and include the properties in the environment variable based on groupID.

You can actually override properties at runtime during the bean instantiation phase.
For instance,

environment.addPropertySource(PropertySource.of("overrides",
                Map.of("kafka.consumers." + GroupId + ".bootstrap.servers", baseUrl,
                        "kafka.consumers." + GroupId + ".sasl.jaas.config", newJaasConfig )));

huangapple
  • 本文由 发表于 2023年7月7日 00:54:06
  • 转载请务必保留本文链接:https://go.coder-hub.com/76631019.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定