如何使用Kafka主题测量异步过程的总时间?

huangapple go评论76阅读模式
英文:

How to measure the total time of an asynchronous process using Kafka topics?

问题

我有以下微服务场景

微服务1

1 - 收集 System.getCurrentTimeMillis

2 - 在数据库中的操作后生成一个 id

3 - 以下面的格式将一个对象以 json 发送到主题 INICIO_PROCESSAMENTO

{
   "uuid": "b45ebe6f-d59a-4e37-9706-f546b32c68c3",
   "milliSeconds": 1600258507450
}

微服务7

1 - 使用在第一个微服务中生成的 ID 完成处理消息

2 - 收集 System.getCurrentTimeMillis

3 - 将下面的 Json 发送到主题 FIM_PROCESSAMENTO

{
   "uuid": "b45ebe6f-d59a-4e37-9706-f546b32c68c3",
   "milliSeconds": 1600258508980
}

在这种情况下,如何测量异步事务的总时间?一个建议是使用 Kafka Streams,也许还涉及 KTables,但我不太明白这是如何工作的。

一个重要的问题是,Kafka 中的消息 ID 是由 Kafka/驱动程序生成的长整型。我的操作 ID 只存在于发送到主题的消息体内。

最终的目标是能够收集处理的总时间以进行分析,最初我们将使用 opentracing 实现,但由于我们在处理过程中与第三方进行集成,并且我们依赖于他们没有事务 ID 的返回,所以我们无法沿着这条路走下去。

英文:

I have the following microservices scenario

Microservice 1

1 - Collect a System.getCurrentTimeMillis

2 - Generate an id after an operation in the DB

3 - Send an object in json in the format below to the topic INICIO_PROCESSAMENTO

{
   "uuid": "b45ebe6f-d59a-4e37-9706-f546b32c68c3",
   "milliSeconds": 1600258507450
}

Microservice 7

1 - Finish processing the message with the ID generated in the first microservice

2 - Collect a System.getCurrentTimeMillis

3 - Send Json below for the topic FIM_PROCESSAMENTO

{
   "uuid": "b45ebe6f-d59a-4e37-9706-f546b32c68c3",
   "milliSeconds": 1600258508980
}

In this case, how can I measure the total time of the asynchronous transaction? One suggestion was to use Kafka Streams and maybe KTables, but I didn't understand how that would work.

An important point the message ID in Kafka is a long generated by Kafka / Driver. The ID of my operation exists only within the body of messages sent to topics.

The final objective is to be able to collect the total time of the process for analysis, initially we would use opentracing for this, but as we have an integration with third parties in the middle of the process and we depend on their return that comes without the transaction id we were unable to follow that way.

答案1

得分: 0

在这种情况下,我使用 opentelemetry 解决了这个问题。

英文:

In this case I solved it using opentelemetry

huangapple
  • 本文由 发表于 2020年9月21日 22:34:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/63994444.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定