英文:
Python paho MQTT loop_forever(): how to redirect output to a file while the script is running?
问题
I'm running a script to subscribe to topics of an MQTT broker and fetch the data associated with them. I run the script like this:
$ python3 test_mqtt_client.py
The problem is that when I try to output to a file using this command:
$ python3 test_mqtt_client.py >> /tmp/test_mqtt_client.log
I don't get any content in the file until I interrupt the script using Ctrl+C. How can I get the script's output in /tmp/test_mqtt_client.log
while the script is running, before interrupting it?
英文:
I'm running a script to subscribe to topics of an MQTT broker and fetch the data associated to them. I run the script like this:
$ python3 test_mqtt_client.py
import paho.mqtt.client as paho
import ssl
import random
from config import BROKER_ADDRESS, PORT, CLIENT_CERT, CLIENT_KEY, CA_KEY, SUBSCRIBED_TOPICS, DEST_FOLDER
#"on_message" callback
def on_message(client, userdata, message):
print("received message =",str(message.payload.decode("utf-8")))
filename = str(random.randint(0,4294967295))
file = open(DEST_FOLDER + filename + '.json', 'w+')
file.write(str(message.payload.decode("utf-8")))
file.close()
client=paho.Client()
client.on_message=on_message
print("connecting to broker")
client.tls_set(
CA_KEY,
certfile=CLIENT_CERT,
keyfile=CLIENT_KEY,
cert_reqs=ssl.CERT_REQUIRED,
tls_version=ssl.PROTOCOL_TLSv1_2,
ciphers=None
)
client.tls_insecure_set(True)
client.connect(BROKER_ADDRESS, PORT, 60)
for x in SUBSCRIBED_TOPICS:
client.subscribe(x)
print('Subscribed to topic "' + x + '".')
client.loop_forever()
time.sleep(1)
The problem: if I try to output to a file like this:
$ python3 test_mqtt_client.py >> /tmp/test_mqtt_client.log
I don't get any content on the file untill I interrupt the script using Ctrl+C.
How can I get the output of the script inside /tmp/test_mqtt_client.log
while the script is running? I mean, before interrupting it.
答案1
得分: 3
默认情况下,输出到标准输出是缓冲的,这意味着它实际上不会被立即刷新到文件,直到输出缓冲区已满,这又意味着当将标准输出重定向到文件时,通常在你的代码生成足够的输出以溢出输出缓冲区之前,你可能不会在文件中看到任何内容。
有各种处理此行为的方法:
-
可以选择将输出发送到标准错误(stderr) 而不是标准输出(stdout)。标准错误(stderr) 默认情况下不会缓冲,你应该立即看到输出。这是例如
logging
模块的默认行为,通常比使用print
输出长时间运行程序的日志更好。当然,你也可以在print
命令中设置file=sys.stderr
:print("这是一个测试", file=sys.stderr)
(如果这样做,请记得在命令行上将标准错误(stderr) 重定向,而不是标准输出(stdout)。)
-
你可以通过在命令行中添加
-u
标志来以非缓冲模式运行Python:python -u test_mqtt_client.py >> test_mqtt_client.log
-
你可以在每个
print
语句后显式刷新输出缓冲区:print("一些日志消息") sys.stdout.flush()
英文:
By default, output to stdout is buffered: that means it's not actually flushed to a file until the output buffer is full, which in turns means that when redirecting stdout to a file, you often won't see anything in the file until your code generates sufficient output to overflow the output buffer.
There are various ways of dealing with with this behavior:
-
Just output to stderr instead of stdout. stderr is not buffered by default and you should see output immediately. This is the default behavior of e.g. the
logging
module, which is generally a better idea thanprint
for logging output from long-running programs. You can, of course, just setfile=sys.stderr
in yourprint
command instead:print("this is a test", file=sys.stderr)
(If you do this, remember to redirect stderr instead of stdout on the command line.)
-
You can run Python in unbuffered mode by adding the
-u
flag to the command line:python -u test_mqtt_client.py >> test_mqtt_client.log
-
You can explicitly flush the output buffer after each
print
statement:print("some log message") sys.stdout.flush()
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论