英文:
Is there a better way to consume data from event hubs?
问题
1 - 我是新手，尝试实现一个类似以下链接的事件中心消费者：
但我有一些问题：
1 - 我使用了一个后台服务，但当我为任务创建延迟（await Task.Delay(TimeSpan.FromSeconds(30))），如果在任务即将关闭时接收到一个批处理，我会收到一个错误，尝试消费，因为处理器在消费过程中停止，是否有更好的方法来实现这一点？
2 - 如果我有多个要消费的事件中心，我需要创建另一个执行相同操作但针对不同中心的后台服务吗？
英文:
I am new to event hub and tried to implement a consumer like the following
But I have some questions:
1 - I used a service background, but when I create a delay (await Task.Delay(TimeSpan.FromSeconds(30))) for the task, if I receive a batch while the task is about to close, I get a error trying to consume cause the processor is stopped in the middle of consuming, there's a better approach to implement this?
2 - If I have more than a event hub to consume, I'll need to create another service background doing the same thing but for a diferent hub?
答案1
得分: 0
感谢 @ Magnus 的评论。
在从事件中心（Event Hubs）获取数据时，需要遵循以下步骤。
- 不要使用
Task.Delay的固定延迟，可以使用长轮询方式来消耗事件。 - 这涉及在从事件中心接收事件时设置较长的超时时间。
- 您可以将
ReceiveTimeout属性设置为较大的值，以便接收操作在超时之前等待较长时间的事件。
这样可以避免在仍在消耗事件时关闭处理器的问题。
-
不要依赖于服务后台任务来消耗事件，
您可以创建一个专用于事件处理的单独任务或线程。这有助于将事件消耗逻辑与服务后台解耦，并确保即使出于某种原因停止或暂停服务后台任务，事件仍然会被持续处理。 -
根据所使用的编程语言，可能会提供用于从事件中心消耗事件的更高级别框架。这些框架通常处理了
管理连接、重试和事件处理的复杂性，使您可以专注于业务逻辑。
在 .NET 中，您可以考虑使用 Azure Event Hubs SDK 或 Azure Functions Event Hubs 触发器，它们提供了简化事件消费的方式。
关于您的问题，您可以使用单个事件处理器主机实例从多个事件中心（Event Hubs）消耗事件。
您可以在 EventProcessorHost 类的构造函数 中指定事件中心的名称。
有关更多信息，请参阅 MSDoc。
英文:
Thanks @ Magnus for the comments.
When consuming data from Event Hubs, you need to follow the below.
- Instead of using a fixed delay with
Task.Delay, you can use a long-polling approach to consume events. - This involves setting a longer timeout when receiving events from the Event Hub.
- You can set the
ReceiveTimeoutproperty to a larger value so that the receive operation waits for events for a longer period before timing out.
You can avoid the issue of closing the processor while events are still being consumed.
-
Instead of relying on the service background's task to consume events,
you can create a separate taskorthread dedicated to event processing. This helps to decouple the event consumption logic from the service background and ensures that events are continuously processed even if the service background task is paused or stopped for some reason. -
Based on the programming language using, there will be higher-level frameworks available that provide abstractions for consuming events from Event Hubs. And these frameworks often handle the complexities of
managing connections, retries, and event processing, allowing you to focus on the business logic.
In .NET, you can consider using the Azure Event Hubs SDK or Azure Functions Event Hubs trigger, which provides easy way of event consumption.
>For your question, you can use a single Event Processor Host instance to consume events from multiple Event Hubs.
You can specify the name of the Event Hub in the constructor of the EventProcessorHost class.
For more information refer to MSDoc.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论