Golang go-workers自定义日志中间件?

huangapple go评论89阅读模式
英文:

Golang go-workers custom logging middleware?

问题

我正在构建一个使用Golang编写的应用程序,该应用程序实现了一个与Sidekiq兼容的jrallison/go-workers工作队列,并在Sirupsen/logrus周围实现了一个自定义的日志包装器,用于处理序列化的JSON日志。

现在,我的应用程序(除了go-workers)都在一个中心位置使用我的日志包装器,以确保其输出的100%与JSON兼容。

请注意,第1行和第2行是来自我们中心日志记录器的正确JSON,但是当go-workers初始化时,我们看到第3行来自错误的纯文本日志记录器。

<!-- language: lang-json -->

{"db":{"Mapper":{}},"instance_id":"1","level":"info","msg":"Db: Connected to MySQL","time":"2015-05-27T04:15:15-04:00"}
{"concurrency":10,"instance_id":"1","level":"info","msg":"Worker: Commencing work","time":"2015-05-27T04:15:15-04:00"}
workers: 2015/05/27 04:15:15.211217 processing queue contact with 10 workers.

当我们发送关闭程序的信号时,我们首先看到第1行错误的纯文本日志记录器,然后是来自我们中心日志记录器的正确JSON的第2行。

<!-- language: lang-json -->

^C
workers: 2015/05/27 04:15:17.197504 quitting queue contact (waiting for 0 / 10 workers).
{"instance_id":"1","level":"info","msg":"Closed correctly","time":"2015-05-27T04:15:17-04:00"}

我似乎无法将自定义的MiddlewareLogging替换为go-workers的默认日志中间件。

<!-- language: lang-golang -->

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware(
    &WorkMiddleware{ App: a },
    )
}

type WorkMiddleware struct{
  App *App
}

func (m *WorkMiddleware) Call(queue string, message *workers.Msg, next func() bool) (acknowledge bool) {
  // before each message is processed:
  job_json := message.Args().GetIndex(0).MustString()
  job := ContactJob{}
  err := json.Unmarshal([]byte(job_json), &job)
  if err != nil {
  m.App.Log.WithFields( log.Fields{
    "job_json": job_json,
    }).Fatal("Worker: Could not Unmarshal job JSON")
    return
  }
  SetMessageJob(message, job)
  start := time.Now()
  m.App.Log.WithFields( log.Fields{
    "job_id": message.Jid(),
    "queue": queue,
    "args": message.Args(),
    }).Print("Work: Job Starting")
  defer func() {
    if e := recover(); e != nil {
      buf := make([]byte, 4096)
      buf = buf[:runtime.Stack(buf, false)]
      m.App.Log.WithFields( log.Fields{
          "job_id": message.Jid(),
          "queue": queue,
          "duration": time.Since(start),
          "error": e,
          "stack": buf,
        }).Fatal("Work: Job Failed")
    }
  }()
  acknowledge = next()
  result := GetMessageResult(message)
  m.App.Log.WithFields( log.Fields{
    "job_id": message.Jid(),
    "result": result,
    "queue": queue,
    "duration": time.Since(start),
    }).Print("Work: Job Done")
  return
}

是否真的可以替换go-workers的默认日志中间件以处理这些行?

英文:

I'm building a Golang app that implements a Sidekiq-compatible jrallison/go-workers work queue and a custom logging wrapper around Sirupsen/logrus for marshaled JSON logs.

Now, all of my app (except for go-workers so far) uses my logger-wrapper in a central place to ensure that 100% of its output is JSON compatible.

Note that lines #1 and #2 are proper JSON from our central logger, but when go-workers initializes we see line #3 come from the wrong logger in plain text.

<!-- language: lang-json -->

{&quot;db&quot;:{&quot;Mapper&quot;:{}},&quot;instance_id&quot;:&quot;1&quot;,&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;Db: Connected to MySQL&quot;,&quot;time&quot;:&quot;2015-05-27T04:15:15-04:00&quot;}
{&quot;concurrency&quot;:10,&quot;instance_id&quot;:&quot;1&quot;,&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;Worker: Commencing work&quot;,&quot;time&quot;:&quot;2015-05-27T04:15:15-04:00&quot;}
workers: 2015/05/27 04:15:15.211217 processing queue contact with 10 workers.

And when we send the signal to close the program, we see first on line #1 the wrong logger in plain text, followed by the proper JSON from our central logger on line #2.

<!-- language: lang-json -->

^C
workers: 2015/05/27 04:15:17.197504 quitting queue contact (waiting for 0 / 10 workers).
{&quot;instance_id&quot;:&quot;1&quot;,&quot;level&quot;:&quot;info&quot;,&quot;msg&quot;:&quot;Closed correctly&quot;,&quot;time&quot;:&quot;2015-05-27T04:15:17-04:00&quot;}

I cannot seem to get this custom MiddlewareLogging to replace the go-workers default logging middleware.

<!-- language: lang-golang -->

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware(
    &amp;WorkMiddleware{ App: a },
    )
}

type WorkMiddleware struct{
  App *App
}

func (m *WorkMiddleware) Call(queue string, message *workers.Msg, next func() bool) (acknowledge bool) {
  // before each message is processed:
  job_json := message.Args().GetIndex(0).MustString()
  job := ContactJob{}
  err := json.Unmarshal([]byte(job_json), &amp;job)
  if err != nil {
  m.App.Log.WithFields( log.Fields{
    &quot;job_json&quot;: job_json,
    }).Fatal(&quot;Worker: Could not Unmarshal job JSON&quot;)
    return
  }
  SetMessageJob(message, job)
  start := time.Now()
  m.App.Log.WithFields( log.Fields{
    &quot;job_id&quot;: message.Jid(),
    &quot;queue&quot;: queue,
    &quot;args&quot;: message.Args(),
    }).Print(&quot;Work: Job Starting&quot;)
  defer func() {
    if e := recover(); e != nil {
      buf := make([]byte, 4096)
      buf = buf[:runtime.Stack(buf, false)]
      m.App.Log.WithFields( log.Fields{
          &quot;job_id&quot;: message.Jid(),
          &quot;queue&quot;: queue,
          &quot;duration&quot;: time.Since(start),
          &quot;error&quot;: e,
          &quot;stack&quot;: buf,
        }).Fatal(&quot;Work: Job Failed&quot;)
    }
  }()
  acknowledge = next()
  result := GetMessageResult(message)
  m.App.Log.WithFields( log.Fields{
    &quot;job_id&quot;: message.Jid(),
    &quot;result&quot;: result,
    &quot;queue&quot;: queue,
    &quot;duration&quot;: time.Since(start),
    }).Print(&quot;Work: Job Done&quot;)
  return
}

Is it actually possible to replace the default go-workers logging middleware for those lines?

答案1

得分: 1

引用 jrallisongithub.com/jrallison/go-workers/issues/50 上的评论:

> 看起来你正在替换工作程序的日志中间件(用于记录工作程序处理的每个消息的信息)。
>
> 我在你的输出中只看到了来自管理器的处理/退出日志条目,这些日志在启动和关闭时记录。
>
> 链接
>
> 如果你不希望记录这些日志,你可以将 workers.Logger 替换为满足 WorkersLogger 接口的对象,但实际上不记录任何内容(... 或者根据你的用例,以 JSON 格式记录):

问题通过我的最终代码得到解决:

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware(&WorkMiddleware{App: a})
  workers.Logger = &WorkersLogger{App: a}
}

type WorkersLogger struct {
  App *App
}

func (l *WorkersLogger) Println(args ...interface{}) {
  l.App.Log.WithFields(log.Fields{
    "instanceId": l.App.Id,
  }).Println(args...)
}

func (l *WorkersLogger) Printf(fmt string, args ...interface{}) {
  l.App.Log.WithFields(log.Fields{
    "instanceId": l.App.Id,
  }).Printf(fmt, args...)
}
英文:

Quoting jrallison from github.com/jrallison/go-workers/issues/50:

> It looks like you're replacing the worker logging middleware (which logs information about each and every message processed by your workers).
>
> The only logging I see in your output is the processing/quitting log entries from the manager which are logged on startup and shutdown.
>
> https://github.com/jrallison/go-workers/blob/571e6b3b7be959e99024ec12b83b2ad261b06ff7/manager.go#L47
>
> If you'd like these to not be logged, you may want to replace workers.Logger with an object that meets the WorkersLogger interface, but doesn't actually log anything (... or perhaps for your use-case, logs it in JSON format):

Issue is resolved by my final code:

<!-- language: lang-golang -->

func (a *App) ConfigureWorkers() {
  workers.Middleware = workers.NewMiddleware( &amp;WorkMiddleware{ App: a } )
  workers.Logger = &amp;WorkersLogger{ App: a }
}

type WorkersLogger struct {
  App *App
}

func (l *WorkersLogger) Println(args ...interface{}) {
  l.App.Log.WithFields( log.Fields{
    &quot;instanceId&quot;: l.App.Id,
    }).Println(args...)
}

func (l *WorkersLogger) Printf(fmt string, args ...interface{}) {
  l.App.Log.WithFields( log.Fields{
    &quot;instanceId&quot;: l.App.Id,
    }).Printf(fmt, args...)
}

huangapple
  • 本文由 发表于 2015年5月27日 16:25:39
  • 转载请务必保留本文链接:https://go.coder-hub.com/30476990.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定