如何优雅地使用配置文件配置日志记录器,同时支持日志轮转。

huangapple go评论81阅读模式
英文:

How to elegantly configure a Logger with a configuration file while supporting log rotation

问题

问题描述

  • 函数:Test1() 是一个在官方文档中推荐使用的日志轮转和切割库 gopkg.in/natefinch/lumberjack.v2
  • 函数:Test2() 是一个使用 yaml 读取配置的日志记录器,基于官方文档中的基本配置。

在执行主函数后,

在控制台输出中:

2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

在日志文件 foo.log 中输出:

{"level":"info","ts":1684111756.5545945,"msg":"logger construction succeeded:lumberjack.Logger"}

这两个日志明显是不同的。

我目前的要求:

  1. 两者都支持使用配置文件 config_log_zap.yaml,以便所有配置都能生效,并让 lumberjack 完成日志轮转和切割工作。

  2. 控制台和日志文件的输出应该是相同的,这样我就可以通过配置文件快速应用所需的内容。之所以需要控制台和日志文件都有输出,是因为我需要在开发过程中关注和记录过去的输出消息。

    在控制台输出中:

    2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

    在日志文件 foo.log 中输出:

    2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

  3. 如何将 Test1()Test2() 合并为一个函数 Test0(),以满足上述两个要求?

请给我一些帮助,我已经研究了很长时间。

main.go

package main

import (
	"gopkg.in/yaml.v3"
	"os"

	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"gopkg.in/natefinch/lumberjack.v2"
)

func Test1() {
	// lumberjack.Logger is already safe for concurrent use, so we don't need to
	// lock it.
	w := zapcore.AddSync(&lumberjack.Logger{
		Filename:   "./foo.log",
		MaxSize:    500, // megabytes
		MaxBackups: 3,
		MaxAge:     28, // days
	})
	core := zapcore.NewCore(
		zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
		w,
		zap.InfoLevel,
	)
	logger := zap.New(core)
	logger.Info("logger construction succeeded:lumberjack.Logger")
}

func Test2() {
	var cfg zap.Config
	yamlFile, _ := os.ReadFile("./config_log_zap.yaml")
	if err := yaml.Unmarshal(yamlFile, &cfg); err != nil {
		panic(err)
	}

	logger := zap.Must(cfg.Build())
	defer logger.Sync()

	logger.Info("logger construction succeeded:config from yaml")
}

func main() {
	Test1()
	Test2()
}

config_log_zap.yaml

# For the full description for the configuration, see
# https://github.com/uber-go/zap/blob/382e2511e51cda8afde24f9e6e741f934308edfa/config.go#L58-L94
level: 'debug'
development: true
disableCaller: true
disableStacktrace: false
sampling:
  initial: 100
  thereafter: 100
encoding: 'console'
encoderConfig:
  messageKey: 'msg'
  levelKey: 'level'
  timeKey: 'ts'
  nameKey: 'logger'
  callerKey: 'caller'
  functionKey: 'function'
  stacktraceKey: 'stacktrace'
  skipLineEnding: false
  lineEnding: "\n"
  levelEncoder: 'capital'
  timeEncoder: 'iso8601'
  durationEncoder: 'string'
  callerEncoder: 'full'
  nameEncoder: 'full'
  consoleSeparator: ' | '
outputPaths:
  - 'stdout'
  - './foo.log'
errorOutputPaths:
  - 'stderr'
  - './error_logs'
initialFields:
  app: 'jpz'
英文:

Problem Description

  • Function: Test1() is a log rotation and cutting library gopkg.in/natefinch/lumberjack.v2 recommended in the official documentation.
  • Function: Test2() is a Logger that uses yaml to read configuration based on the basic configuration in the official documentation.

After executing the main function,

in the console output:

2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

in the log file foo.log output:

{"level":"info","ts":1684111756.5545945,"msg":"logger construction succeeded:lumberjack.Logger"}

These two logs are definitely different.

My current requirements:

  1. Both support using the configuration file config_log_zap.yaml so that all configurations can take effect, and let lumberjack complete the log rotation and splitting work.

  2. The output of both the console and the log file should be the same, so that I can quickly apply the desired content through the configuration file. The reason why both the console and the log file are needed is because I need to pay attention to and record past output messages during development.

    In the console output:

    2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

    In the log file foo.log output:

    2023-05-15T08:49:16.555+0800 | INFO | logger construction succeeded:config from yaml | {"app": "jpz"}

  3. How should I merge Test1() and Test2() into one function Test0() to meet the above two requirements?

Please give me some help, I've been studying it for a long time.

main.go

package main

import (
	"gopkg.in/yaml.v3"
	"os"

	"go.uber.org/zap"
	"go.uber.org/zap/zapcore"
	"gopkg.in/natefinch/lumberjack.v2"
)

func Test1() {
	// lumberjack.Logger is already safe for concurrent use, so we don't need to
	// lock it.
	w := zapcore.AddSync(&lumberjack.Logger{
		Filename:   "./foo.log",
		MaxSize:    500, // megabytes
		MaxBackups: 3,
		MaxAge:     28, // days
	})
	core := zapcore.NewCore(
		zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
		w,
		zap.InfoLevel,
	)
	logger := zap.New(core)
	logger.Info("logger construction succeeded:lumberjack.Logger")
}

func Test2() {
	var cfg zap.Config
	yamlFile, _ := os.ReadFile("./config_log_zap.yaml")
	if err := yaml.Unmarshal(yamlFile, &cfg); err != nil {
		panic(err)
	}

	logger := zap.Must(cfg.Build())
	defer logger.Sync()

	logger.Info("logger construction succeeded:config from yaml")
}

func main() {
	Test1()
	Test2()
}

config_log_zap.yaml

# For the full description for the configuration, see
# https://github.com/uber-go/zap/blob/382e2511e51cda8afde24f9e6e741f934308edfa/config.go#L58-L94
level: 'debug'
development: true
disableCaller: true
disableStacktrace: false
sampling:
  initial: 100
  thereafter: 100
encoding: 'console'
encoderConfig:
  messageKey: 'msg'
  levelKey: 'level'
  timeKey: 'ts'
  nameKey: 'logger'
  callerKey: 'caller'
  functionKey: 'function'
  stacktraceKey: 'stacktrace'
  skipLineEnding: false
  lineEnding: "\n"
  levelEncoder: 'capital'
  timeEncoder: 'iso8601'
  durationEncoder: 'string'
  callerEncoder: 'full'
  nameEncoder: 'full'
  consoleSeparator: ' | '
outputPaths:
  - 'stdout'
  - './foo.log'
errorOutputPaths:
  - 'stderr'
  - './error_logs'
initialFields:
  app: 'jpz'

答案1

得分: 0

使用zap.RegisterSinklumberjack日志记录器注册为新的sink:

package main

import (
	"net/url"
	"os"
	"strconv"
	"strings"

	"gopkg.in/yaml.v3"

	"go.uber.org/zap"
	"gopkg.in/natefinch/lumberjack.v2"
)

type lumberjackSink struct {
	lumberjack.Logger
}

func (l *lumberjackSink) Sync() error {
	return nil
}

func parseNumber(s string, fallback int) int {
	v, err := strconv.Atoi(s)
	if err == nil {
		return v
	}
	return fallback
}

func Test0() {
	if err := zap.RegisterSink("lumberjack", func(u *url.URL) (zap.Sink, error) {
		// 从URL中读取参数:
		// lumberjack://localhost/foo.log?maxSize=500&maxBackups=3&maxAge=28
		filename := strings.TrimLeft(u.Path, "/")
		if filename == "" {
			filename = "foo.log"
		}
		q := u.Query()
		l := &lumberjackSink{
			Logger: lumberjack.Logger{
				Filename:   filename,
				MaxSize:    parseNumber(q.Get("maxSize"), 500),
				MaxBackups: parseNumber(q.Get("maxBackups"), 3),
				MaxAge:     parseNumber(q.Get("maxAge"), 28),
			},
		}
		return l, nil
	}); err != nil {
		panic(err)
	}

	var cfg zap.Config
	yamlFile, _ := os.ReadFile("./config_log_zap.yaml")
	if err := yaml.Unmarshal(yamlFile, &cfg); err != nil {
		panic(err)
	}

	logger := zap.Must(cfg.Build())
	defer logger.Sync()

	logger.Info("logger construction succeeded:config from yaml")
}

func main() {
	Test0()
}

并修改配置文件以设置outputPaths,如下所示:

outputPaths:
  - stdout
  - lumberjack://localhost/foo.log?maxSize=500&maxBackups=3&maxAge=28
英文:

Use zap.RegisterSink to register the lumberjack logger as a new sink:

package main

import (
	"net/url"
	"os"
	"strconv"
	"strings"

	"gopkg.in/yaml.v3"

	"go.uber.org/zap"
	"gopkg.in/natefinch/lumberjack.v2"
)

type lumberjackSink struct {
	lumberjack.Logger
}

func (l *lumberjackSink) Sync() error {
	return nil
}

func parseNumber(s string, fallback int) int {
	v, err := strconv.Atoi(s)
	if err == nil {
		return v
	}
	return fallback
}

func Test0() {
	if err := zap.RegisterSink("lumberjack", func(u *url.URL) (zap.Sink, error) {
		// Read parameters from URL:
		// lumberjack://localhost/foo.log?maxSize=500&maxBackups=3&maxAge=28
		filename := strings.TrimLeft(u.Path, "/")
		if filename == "" {
			filename = "foo.log"
		}
		q := u.Query()
		l := &lumberjackSink{
			Logger: lumberjack.Logger{
				Filename:   filename,
				MaxSize:    parseNumber(q.Get("maxSize"), 500),
				MaxBackups: parseNumber(q.Get("maxBackups"), 3),
				MaxAge:     parseNumber(q.Get("maxAge"), 28),
			},
		}
		return l, nil
	}); err != nil {
		panic(err)
	}

	var cfg zap.Config
	yamlFile, _ := os.ReadFile("./config_log_zap.yaml")
	if err := yaml.Unmarshal(yamlFile, &cfg); err != nil {
		panic(err)
	}

	logger := zap.Must(cfg.Build())
	defer logger.Sync()

	logger.Info("logger construction succeeded:config from yaml")
}

func main() {
	Test0()
}

And modify the config file to set outputPaths like this:

outputPaths:
  - stdout
  - lumberjack://localhost/foo.log?maxSize=500&maxBackups=3&maxAge=28

huangapple
  • 本文由 发表于 2023年5月15日 10:06:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/76250460.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定