NServiceBus:通过SQS订阅AWS Textract事件

huangapple go评论78阅读模式
英文:

NServiceBus: Subscribe to AWS Textract event via SQS

问题

以下是翻译好的内容:

如主题所述:我正在使用Amazon Textract来检测扫描文档上的文本。当Textract完成时,它通过SNS触发通知,而SQS队列订阅该通知。

我可以使用AmazonSQSClient接收“全部完成”事件,但由于解决方案的其余部分是构建在NServiceBus之上的,我也想通过NServiceBus处理本机AWS事件。

我如何告诉SqsTransport订阅单个已存在的SQS队列?我不希望nsb创建新的队列,也不打算通过这个队列发送消息。此端点仅用于“仅接收”。

更新

我的问题不是“如何从网络中获取本机AWS事件”,而是“如何配置NSB以使用我的现有队列而不是创建自己的队列”。

更新2
事实证明,我的问题不仅是如何配置nsb。从网络中获取本机AWS事件是另一个障碍,文档上关于本机集成的说明并没有完全解决。

英文:

As the subject states: I'm using Amazon Textract to detect text on scanned documents. When Textract is done, it fires of a notification via SNS and the SQS queue subscribes to the notification.

I can reveive the "all done" event with the AmazonSQSClient but since the rest of the solution is built on top of NServiceBus I want to handle the native AWS events via NServiceBus as well.

How do I tell the SqsTransport to subscribe to a single, already existing SQS queue? I don't want nsb to create new queues and I don't intend to send messages via this queue. This endpoint is meant to be "receive only".


UPDATE

My problem is not "how do I get a native AWS event off the wire" but "how do I configure NSB to use my existing queue instead of creating its own".

UPDATE2
Turns out my problems is not only how to configure nsb. Getting the native AWS event off the wire is another obstacle that the documentation on native integration does not completely solve.

答案1

得分: 4

默认情况下,NServiceBus将端点名称翻译为队列名称。因此,如果您的端点名称是,例如,Sales.OrderProcessing,Amazon SQS传输将将其翻译为Sales-OrderProcessing,以使其与传输特定的“限制”兼容。

可以通过覆盖端点的本地地址来将端点名称与队列名称解耦。

endpointConfiguration.OverrideLocalAddress("someothername");

https://docs.particular.net/nservicebus/endpoints/specify-endpoint-name

英文:

By default, NServiceBus translates the endpoint name into the queue name. So if your endpoint name is, for example, Sales.OrderProcessing this will be translated to Sales-OrderProcessing by the Amazon SQS transport to make it compatible with the transport specific "restrictions".

It is possible to decouple the endpoint name from the queue name by overriding the local address of the endpoint

endpointConfiguration.OverrideLocalAddress("someothername");

https://docs.particular.net/nservicebus/endpoints/specify-endpoint-name

答案2

得分: 0

这会有帮助吗?
您将需要执行原生到NSB的消耗操作:
https://docs.particular.net/transports/sqs/native-integration

英文:

Will this help?
You will have to do the native to NSB consuming:
https://docs.particular.net/transports/sqs/native-integration

答案3

得分: 0

虽然@DanielMarbach的回答有所帮助,但并没有解决所有问题。
我不得不在传输上设置一个自定义的QueueNameGenerator以适应我的现有队列。

对于延迟消息队列的超时设置的验证(在我特定情况下,老实说,我既不需要也不想要)相当难以理解,因为NSB内部的验证并不真正匹配错误消息。延迟为00:01:00,但必须小于00:15:00... 嗯,什么?!我不得不将超时设置为SQS允许的最大值,即15分钟。

更难的障碍是,SqsTransport(特别是InputQueuePump)要求将MessageTypeFullName作为本机AWS消息的MessageAttributes的一部分(或者至少我找不到通过消息接收迷宫的其他方式,适用于我从Textract收到的通知)。我尝试了自定义的Behavior,但找不到在管道中足够早的阶段适用于我。我试图自定义TransportMessage的反序列化,但消息泵使用了一个不关心全局消息序列化的设置或自定义的内部System.Text.Json.JsonSerializer实例。
我最终做的是为传输使用的IAmazonSqs客户端编写了一个装饰器,并拦截了ReceiveMessageAsync方法。因为我只计划将SQS用于这些通知,所以只需将消息的AQTN添加为接收到的消息的MessageAttribute就足够了。
长话短说:这个过程比我希望的要复杂得多,但现在我可以通过NSB连接到Textract通知了。

英文:

While @DanielMarbach's answer helped it did not solve all the problems.
I had to set a custom QueueNameGenerator on the transport to fit my existing queues.

The validation of the timeout settings for the delayed message queue (which I honestly neither need nor want in my specific case) were rather hard to figure out because the validation inside NSB does not really fit the error message. Delay is 00:01:00 but must be less than 00:15:00... eh, what?! I had to set the timeout to the maximum of 15 minutes that SQS allows me.

The harder obstacle was that the SqsTransport (specifically the InputQueuePump) requires the MessageTypeFullName as part of the MessageAttributes of the native AWS message (or at least I could not find another way through the maze of message intake that worked with the notifications I get from Textract).
I tried a custom Behavior but could not find a stage early enough in the pipeline to work for me. I tried to customize the deserialization into TransportMessage but the message pump uses an internal System.Text.Json.JsonSerializer instance that does not care about settings or customizations of the global message serialization.
What I ended up doing was to write a decorator for the IAmazonSqs client the transport uses and intercept the ReceiveMessageAsync method. Since I only plan to use SQS for these notifications it was as simple as adding the AQTN of the message as a MessageAttribute of the received message.
Long story short: It is a much more involved process than I hoped but now I can hook up to the Textract notifications via NSB.

huangapple
  • 本文由 发表于 2023年7月3日 19:11:13
  • 转载请务必保留本文链接:https://go.coder-hub.com/76604186.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定