连接到 Azure EventHub(类似 Kafka)并在 Spring Boot 中使用连接字符串。

huangapple go评论67阅读模式
英文:

Connect to Azure EventHub(Kafka like) with Spring Boot using connection string

问题

我需要使用Spring Boot连接到启用Kafka的事件中心,并且我有连接字符串和命名空间应该连接到哪里。

我正在使用以下依赖项:

<dependency>
    <groupId>com.microsoft.azure</groupId>
    <artifactId>spring-cloud-azure-eventhubs-stream-binder</artifactId>
    <version>1.2.7</version>
</dependency>

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream-binder-kafka</artifactId>
</dependency>

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-stream</artifactId>
</dependency>

我找到了一个教程,其中我需要从我的本地机器使用az login登录到Azure并创建auth文件,但是我提供了连接字符串,我应该使用哪个,所以是否有一种方式来指定连接字符串和命名空间,就像这样:

spring.cloud.azure.eventhub.connection-string
spring.cloud.azure.eventhub.namespace

因为现在它在抱怨缺少资源组。

我应该如何连接到EventHub?

英文:

I need to connect to event hub with enabled kafka with Spring Boot, and I have connection string and name space where should I connect.

I'm using such dependencies

 &lt;dependency&gt;
        &lt;groupId&gt;com.microsoft.azure&lt;/groupId&gt;
        &lt;artifactId&gt;spring-cloud-azure-eventhubs-stream-binder&lt;/artifactId&gt;
        &lt;version&gt;1.2.7&lt;/version&gt;
    &lt;/dependency&gt;

    &lt;dependency&gt;
        &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
        &lt;artifactId&gt;spring-cloud-stream-binder-kafka&lt;/artifactId&gt;
    &lt;/dependency&gt;

    &lt;dependency&gt;
        &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
        &lt;artifactId&gt;spring-cloud-stream&lt;/artifactId&gt;
    &lt;/dependency&gt;

I found a tutorial where I need to login into azure from my local machine with az login and create auth file, BUT I was provided with connection string which should I use, so is there any way to specify
ONLY connection string with namespace like this :

spring.cloud.azure.eventhub.connection-string
spring.cloud.azure.eventhub.namespace

Because now it is complaining that is is missing resource-group.

How should I connect to EventHub?

答案1

得分: 3

Here's the translation of your provided content:

"我的问题包含了错误的依赖关系,我添加了两个绑定器,这是不正确的。当您启动应用程序时,Spring Cloud Stream 不知道哪一个是主要的。所以您需要选择一个。

所以,因为我想要使用 Event Hub,但之前没有相关经验,但有 Kafka 的经验,而 Event Hub 可以通过 Kafka 协议进行工作,所以我开始朝这个方向进行研究。所有来自 Microsoft 的教程都无法工作(对我来说很遗憾)。它们已经过时了。

因此,我开始思考,如果它是通过 Kafka 协议工作的,也许我可以将 Event Hub 视为带有一些配置更改的简单 Kafka。在搜索后,我找到了很多关于如何做的教程。

您只需要创建常规的 Kafka 消费者/生产者。我使用 Spring Cloud Stream 来完成它。

@Slf4j
@EnableBinding(Sink.class)
public class KafkaSink {

    @StreamListener(Sink.INPUT)
    public void consumerMessage(TestMessage testMessage) {
        log.info(\"{}\", testMessage);
    }
}

@Component
@EnableBinding(Source.class)
public class KafkaSource {

    private final MessageChannel output;

    @Autowired
    public KafkaSource(MessageChannel output) {
        this.output = output;
    }

    public void send(TestMessage testMessage) {
        output.send(MessageBuilder.withPayload(testMessage).build());
    }
}

然后,只需将适当的 JAAS 配置添加到 application.* 文件中。您需要获取您的 Event Hub 的连接字符串。

我的 YAML 文件:

spring:
  cloud:
    stream:
      bindings:
        input:
          destination: my-topic
        output:
          destination: my-topic
      kafka:
        binder:
          auto-create-topics: true
          brokers: ${EVENT_HUB_KAFKA_BROKER}
          configuration:
            sasl:
              jaas:
                config: ${EVENT_HUB_CONNECTION_STRING}
              mechanism: PLAIN
            security:
              protocol: SASL_SSL

一个重要的事情是,EVENT_HUB_KAFKA_BROKER 应该是 Event Hub 的地址,类似于 blablabla.servicebus.windows.net:9093(不要忘记端口)。对于 EVENT_HUB_CONNECTION_STRING,您应该指定将连接字符串解析为密码的模块,它应该类似于 org.apache.kafka.common.security.plain.PlainLoginModule required username=\"$ConnectionString\" password=\"{your_connection_string}\"。"

英文:

tl;dr

My question contains incorrect dependencies, I've added two binders, which incorrect. When you start app spring cloud stream don't know what is primary. So you need to choose only one.

So as I want to work with Event Hub, but had not previous experience with it, but had experience with Kafka and Event Hub has mode to work by Kafka protocol I started to look in that way. All tutorial from Microsoft are not working (sad for me). They are outdated.

So, I started to think if it is working by Kafka protocol, maybe I can thread Event Hub as simple Kafka with some configuration changes. After googling I found a lot of tutorial how to do it.

All you need is to create regular Kafka consumer/producer. I've done it with Spring Cloud Stream

@Slf4j
@EnableBinding(Sink.class)
public class KafkaSink {

    @StreamListener(Sink.INPUT)
    public void consumerMessage(TestMessage testMessage) {
        log.info(&quot;{}&quot;, testMessage);
    }
}

@Component
@EnableBinding(Source.class)
public class KafkaSource {

    private final MessageChannel output;

    @Autowired
    public KafkaSource(MessageChannel output) {
        this.output = output;
    }

    public void send(TestMessage testMessage) {
        output.send(MessageBuilder.withPayload(testMessage).build());
    }
}

And then just add proper jaas configuration into application.* file. You need to get connection string for your Event Hub

My yaml file:

spring:
  cloud:
    stream:
      bindings:
        input:
          destination: my-topic
        output:
          destination: my-topic
      kafka:
        binder:
          auto-create-topics: true
          brokers: ${EVENT_HUB_KAFKA_BROKER}
          configuration:
            sasl:
              jaas:
                config: ${EVENT_HUB_CONNECTION_STRING}
              mechanism: PLAIN
            security:
              protocol: SASL_SSL

One important thing EVENT_HUB_KAFKA_BROKER should be Event Hub address, something like blablabla.servicebus.windows.net:9093 (don't forget port). For EVENT_HUB_CONNECTION_STRING you hould specify module which will be parsing connection string as password and it should be something like org.apache.kafka.common.security.plain.PlainLoginModule required username=&quot;$ConnectionString&quot; password=&quot;{your_connection_string}&quot;\

huangapple
  • 本文由 发表于 2020年8月3日 16:50:51
  • 转载请务必保留本文链接:https://go.coder-hub.com/63226387.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定