使用nlog将TraceId写入日志

huangapple go评论108阅读模式
英文:

Write TraceId to logs with nlog

问题

I'm trying to setup my net6 project to send traces and logs to grafana loki and tempo for better observability.
我正在尝试设置我的net6项目,以便将跟踪和日志发送到Grafana Loki和Tempo以实现更好的可观察性。

I'm successfully setup sending traces to tempo using opentelemetry.instrumentation(All traces have TraceId).
我已成功设置使用opentelemetry.instrumentation将跟踪发送到Tempo(所有跟踪都有TraceId)。

Now I'm trying to get TraceId for other logs, but can't get it.
现在我正在尝试获取其他日志的TraceId,但无法获取它。

I'm using NLog.Web.AspNetCore(5.3.2) and NLog.DiagnosticSource(5.0.0) packages and as mentioned here - https://github.com/NLog/NLog.DiagnosticSource - I can get TraceId via ${activity:property=TraceId} variable, but when I use layout like "consoleLayout": "${activity:property=TraceId} [${time} ${uppercase:${level}]} ${logger} ${aspnet-request-method}..."
我正在使用NLog.Web.AspNetCore(5.3.2)和NLog.DiagnosticSource(5.0.0)包,并且正如在这里所提到的 - https://github.com/NLog/NLog.DiagnosticSource - 我可以通过${activity:property=TraceId}变量获取TraceId,但是当我使用类似于"consoleLayout"的布局时:"${activity:property=TraceId} [${time} ${uppercase:${level}]} ${logger} ${aspnet-request-method}...",这个变量是空的。

I did not find any information that would need to configure something else. As far as I understand, TraceID is present by default, and I only need to get it.
我没有找到任何需要配置其他内容的信息。就我所理解,TraceID默认存在,我只需要获取它。

What I missed?
我错过了什么?

英文:

I'm trying to setup my net6 project to send traces and logs to grafana loki and tempo for better observability.
I'm successfully setup sending traces to tempo using opentelemetry.instrumentation(All traces have TraceId).
Now I'm trying to get TraceId for other logs, but can't get it.
I'm using NLog.Web.AspNetCore(5.3.2) and NLog.DiagnosticSource(5.0.0) packages and as mentioned here - https://github.com/NLog/NLog.DiagnosticSource - I can get TraceId via ${activity:property=TraceId} variable, but when I use layout like "consoleLayout": "${activity:property=TraceId} [${time} ${uppercase:${level}]} ${logger} ${aspnet-request-method}..." this variable is empty.

I did not find any information that would need to configure something else. As far as I understand, TraceID is present by default, and I only need to get it.
What I missed?

答案1

得分: 2

我能够在添加了以下代码之后在NLog中获取TraceId:

builder.Services.AddLogging(o => o
    .Configure(x => x.ActivityTrackingOptions = ActivityTrackingOptions.TraceId | ActivityTrackingOptions.SpanId));
英文:

I was able to get TraceId in nlog after I have added

builder.Services.AddLogging(o => o
    .Configure(x => x.ActivityTrackingOptions = ActivityTrackingOptions.TraceId | ActivityTrackingOptions.SpanId));

答案2

得分: 0

可能 TraceId 字段可能会包含大量不同的唯一值,跨不同的日志行 - 每个启用了跟踪的处理请求都有一个唯一值。Grafana Loki为每组唯一的日志字段(排除消息和时间戳字段)创建一个唯一的日志流。参见这些文档。这意味着它将为每个唯一的 TraceId 字段值创建一个唯一的日志流。Grafana Loki将每个流的数据存储在单独的文件中。它还将每个流的索引数据保留在内存中。这意味着在存在数百万个唯一的 StreamId 值的情况下,Grafana Loki可能会受到大量小文件和高内存使用的影响。这被称为 高基数问题

解决方案是将 TraceId 存储在消息本身中,而不是将其提取到单独的字段中。然后,您可以使用行过滤器查找具有所需 TraceId 的消息。

附言:提供的解决方案在可用性方面并不是最佳的。您可以将 StreamId 存储在专用字段中,而无需担心在使用诸如ElasticSearchVictoriaLogs(我在此上工作)等备用日志管理系统时会出现高基数问题。这些系统支持对高基数字段进行快速全文搜索。ElasticSearch得到了广泛应用,而VictoriaLogs与ElasticSearch相比,对相同数量的生产日志需要的RAM少30倍,磁盘空间少15倍。

英文:

It is likely the TraceId field may contain very big number of unique values across different log lines - an unique value per each processed request with the enabled tracing. Grafana Loki creates an unique log stream per each unique set of log fields (excluding message and timestamp fields). See these docs. This means it will create an unique log stream per each unique TraceId field value. Grafana Loki stores data for every stream in a separate file. It also keeps index data per each stream in memory. This means that Grafana Loki may suffer from high number of small files and high memory usage in the case of millions of unique StreamId values. This is called high cardinality issue.

The solution is to store the TraceId in the message itself, and do not extract it into a separate field. Then you can use line filters for finding messages with the needed TraceId.

P.S. The provided solution isn't the best from usability perspective. You can store StreamId in a dedicated field without worrying about high cardinality issues when using alternative log management systems such as ElasticSearch or VictoriaLogs (I work on it). These systems support fast full-text search over high-cardinality fields. ElasticSearch is in widespread use, while VictoriaLogs requires 30x less RAM and 15x less disk space than ElasticSearch for the same amounts of production logs.

huangapple
  • 本文由 发表于 2023年7月6日 16:59:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/76627146.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定