英文:
Can I send output of a Kafka Stream to a REST API endpoint
问题
我目前有一个应用程序，它使用Consumer API从Kafka主题中获取消息，然后将这些消息发布到一个REST API端点。我想知道是否可以使用Kafka Streams API来实现相同的功能。这是否可能？
英文:
I currently have an app that consumes from a Kafka topic using Consumer API and publish these messages to a REST API endpoint. I wanted to see if I can use Kafka Streams API do achieve the same. Is that possible?
答案1
得分: 1
从技术上讲，是的，你可以。代码会看起来类似于这样（为了清晰起见，省略了一些细节）
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> inputStream = builder.stream("topic-name");
inputStream.foreach((key, value) -> {
// 使用键和值执行某些操作，以发布到REST API
});
但这不是 Kafka Streams 的设计初衷；它更适用于处理来自 Kafka 的记录，将结果再次发送到 Kafka，而不是与外部系统进行通信。
我看到这种方法的最大问题是，在使用 Kafka Streams 时，你无法直接控制提交。我想你只有在成功发送了一堆记录到 REST API 终端，包括一些重试等情况后才会想要提交。因此，像你已经拥有的普通消费者应用程序更适合执行这样的操作。
希望有所帮助。
英文:
Technically speaking, yes, you can. It would look something like this (with several details omitted for clarity)
StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> inputStream = builder.stream("topic-name");
imputStream.foreach((key, value) -> {
// Do something with key and value to publish to REST API
});
But this isn't what Kafka Streams was designed for; it's meant more for processing records from Kafka, producing the results back to Kafka, and not reaching out to external systems.
The biggest issue I see with this approach is that with Kafka Streams, you don't have direct control over committing. I imagine you'd only want to commit after a bunch of records have been successfully sent to the REST API endpoint, including some retries, etc. So doing something like that is better served with a plain consumer application like you already have.
HTH
答案2
得分: 0
存在一个可在线获取的Kafka Connect HTTP Sink插件，对于这个目的可能更加优化。否则，如果您控制所述HTTP服务器的代码，您还可以选择在其中嵌入一个Kafka Consumer。
英文:
There exists a Kafka Connect HTTP Sink plugin available online, which would be more optimized for this purpose.
Otherwise, if you control the code of said HTTP server, you could also chose to embed a Kafka Consumer within that.
答案3
得分: 0
你应该使用一个连接器来实现这个目的：
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论