Python paho MQTT的`loop_forever()`:如何在脚本运行时将输出重定向到文件?

huangapple go评论69阅读模式
英文:

Python paho MQTT loop_forever(): how to redirect output to a file while the script is running?

问题

I'm running a script to subscribe to topics of an MQTT broker and fetch the data associated with them. I run the script like this:

$ python3 test_mqtt_client.py

The problem is that when I try to output to a file using this command:

$ python3 test_mqtt_client.py >> /tmp/test_mqtt_client.log

I don't get any content in the file until I interrupt the script using Ctrl+C. How can I get the script's output in /tmp/test_mqtt_client.log while the script is running, before interrupting it?

英文:

I'm running a script to subscribe to topics of an MQTT broker and fetch the data associated to them. I run the script like this:

$ python3 test_mqtt_client.py

import paho.mqtt.client as paho
import ssl
import random

from config import BROKER_ADDRESS, PORT, CLIENT_CERT, CLIENT_KEY, CA_KEY, SUBSCRIBED_TOPICS, DEST_FOLDER

#"on_message" callback
def on_message(client, userdata, message):
  
  print("received message =",str(message.payload.decode("utf-8")))


  filename = str(random.randint(0,4294967295))

  file = open(DEST_FOLDER + filename + '.json', 'w+')
  file.write(str(message.payload.decode("utf-8")))
  file.close()


client=paho.Client() 
client.on_message=on_message

print("connecting to broker")

client.tls_set(
    CA_KEY,
    certfile=CLIENT_CERT,
    keyfile=CLIENT_KEY,
    cert_reqs=ssl.CERT_REQUIRED,
    tls_version=ssl.PROTOCOL_TLSv1_2,
    ciphers=None
)

client.tls_insecure_set(True)

client.connect(BROKER_ADDRESS, PORT, 60)

for x in SUBSCRIBED_TOPICS:
  client.subscribe(x)
  print('Subscribed to topic "' + x + '".')

client.loop_forever()

time.sleep(1)

The problem: if I try to output to a file like this:

$ python3 test_mqtt_client.py >> /tmp/test_mqtt_client.log

I don't get any content on the file untill I interrupt the script using Ctrl+C.

How can I get the output of the script inside /tmp/test_mqtt_client.log while the script is running? I mean, before interrupting it.

答案1

得分: 3

默认情况下,输出到标准输出是缓冲的,这意味着它实际上不会被立即刷新到文件,直到输出缓冲区已满,这又意味着当将标准输出重定向到文件时,通常在你的代码生成足够的输出以溢出输出缓冲区之前,你可能不会在文件中看到任何内容。

有各种处理此行为的方法:

  1. 可以选择将输出发送到标准错误(stderr) 而不是标准输出(stdout)标准错误(stderr) 默认情况下不会缓冲,你应该立即看到输出。这是例如 logging 模块的默认行为,通常比使用 print 输出长时间运行程序的日志更好。当然,你也可以在 print 命令中设置 file=sys.stderr

    print("这是一个测试", file=sys.stderr)
    

    (如果这样做,请记得在命令行上将标准错误(stderr) 重定向,而不是标准输出(stdout)。)

  2. 你可以通过在命令行中添加 -u 标志来以非缓冲模式运行Python:

    python -u test_mqtt_client.py >> test_mqtt_client.log
    
  3. 你可以在每个 print 语句后显式刷新输出缓冲区:

    print("一些日志消息")
    sys.stdout.flush()
    
英文:

By default, output to stdout is buffered: that means it's not actually flushed to a file until the output buffer is full, which in turns means that when redirecting stdout to a file, you often won't see anything in the file until your code generates sufficient output to overflow the output buffer.

There are various ways of dealing with with this behavior:

  1. Just output to stderr instead of stdout. stderr is not buffered by default and you should see output immediately. This is the default behavior of e.g. the logging module, which is generally a better idea than print for logging output from long-running programs. You can, of course, just set file=sys.stderr in your print command instead:

    print("this is a test", file=sys.stderr)
    

    (If you do this, remember to redirect stderr instead of stdout on the command line.)

  2. You can run Python in unbuffered mode by adding the -u flag to the command line:

    python -u test_mqtt_client.py >> test_mqtt_client.log
    
  3. You can explicitly flush the output buffer after each print statement:

    print("some log message")
    sys.stdout.flush()
    

huangapple
  • 本文由 发表于 2023年4月11日 00:12:18
  • 转载请务必保留本文链接:https://go.coder-hub.com/75978731.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定