Python使用多进程记录到命令行日志

huangapple go评论67阅读模式
英文:

Python logging to cmd with multiprocessing

问题

I am having issues with info log messages from a spawned process not coming through.
The code below shows the warning message coming from the worker, but not the info. I am running on windows 10, with python 3.10.5

from multiprocessing import Process, Queue
import logging

class worker():
	def __init__(self, log_queue):
		self.logger = logging.getLogger(__name__)
		self.logger.setLevel(logging.INFO)

		self.logger.info('worker info')
		self.logger.warning('worker warning')
    

class boss():
	def __init__(self):
		self.logger = logging.getLogger()
		self.logger.info('boss here')


if __name__ == '__main__':
	queue = Queue()

	logging.basicConfig(level=logging.DEBUG)

	worker_p = Process(target=worker, args=(queue,))
	worker_p.start()
	boss_instance = boss()
	worker_p.join()

The code above gives:

INFO:root:boss here
worker warning

Is there a simple way to get messages from the worker show in the command window/main log?

I tried with a logging queue and handlers, but it seems I am just missing a command to get the spawned process to issue info logs.

英文:

I am having issues with info log messages from a spawned process not coming through.
The code below shows the warning message coming from the worker, but not the info. I am running on windows 10, with python 3.10.5

from multiprocessing import Process, Queue
import logging

class worker():
	def __init__(self, log_queue):
		self.logger = logging.getLogger(__name__)
		self.logger.setLevel(logging.INFO)

		self.logger.info('worker info')
		self.logger.warning('worker warning')
    

class boss():
	def __init__(self):
		self.logger = logging.getLogger()
		self.logger.info('boss here')


if __name__ == '__main__':    
	queue = Queue()

	logging.basicConfig(level=logging.DEBUG)

	worker_p = Process(target=worker, args=(queue,))
	worker_p.start()
	boss_instance = boss()
	worker_p.join()

The code above gives:

INFO:root:boss here
worker warning

Is there a simple way to get messages from the worker show in the command window/main log?

I tried with a logging queue and handlers, but it seems I am just missing a command to get the spawned process to issue info logs.

答案1

得分: 2

以下是要翻译的代码部分:

# 为每个进程初始化单独的记录器
# 你的代码中缺少为进程中的记录器添加处理程序的部分
# 而是使用根记录器的流处理程序,具有一些对我来说难以理解的配置

# 简单的解决方案是在每个进程中使用具有自己配置的单独记录器

from multiprocessing import Process, Queue
import logging


class worker():
    def __init__(self, log_queue):
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.addHandler(logging.StreamHandler())
        self.logger.setLevel(logging.INFO)

        self.logger.info('worker info')
        self.logger.warning('worker warning')


class boss():
    def __init__(self):
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.addHandler(logging.StreamHandler())
        self.logger.setLevel(logging.INFO)
        self.logger.info('boss here')


if __name__ == '__main__':
    queue = Queue()
    worker_p = Process(target=worker, args=(queue,))
    worker_p.start()
    boss_instance = boss()
    worker_p.join()

如果只需将日志记录到标准输出,那么这应该是可以的。但是,如果你想要实现实时日志记录,那么你需要实现一个日志记录队列:https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes

英文:

Simplest way is probably to initialize a separate logger for each process. I think what is missing from your code is that you haven't added any handlers for your loggers in the processes, which use the root logger stream handler instead and have some hard for me to understand configuration.

Easy solution is to have separate loggers in each process with their own configuration.

from multiprocessing import Process, Queue
import logging


class worker():
    def __init__(self, log_queue):
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.addHandler(logging.StreamHandler())
        self.logger.setLevel(logging.INFO)

        self.logger.info('worker info')
        self.logger.warning('worker warning')


class boss():
    def __init__(self):
        self.logger = logging.getLogger(self.__class__.__name__)
        self.logger.addHandler(logging.StreamHandler())
        self.logger.setLevel(logging.INFO)
        self.logger.info('boss here')


if __name__ == '__main__':
    queue = Queue()
    worker_p = Process(target=worker, args=(queue,))
    worker_p.start()
    boss_instance = boss()
    worker_p.join()

This should be fine for just logging to standard output, but if you want add life logging, then you need to implement a logging queue: https://docs.python.org/3/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes

huangapple
  • 本文由 发表于 2023年4月19日 17:57:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/76053163.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定