Python文件写入需要手动中断程序才能完成。

huangapple go评论62阅读模式
英文:

Python File Writing Requires Manual Program Interruption for Completion

问题

I'm having trouble with my program that writes parts of a pcap file to a new json file. The issue is that the last 100 lines of the file are only visible when I manually stop the program in the terminal. It only writes the last lines if I forcefully kill it.

file_name = "my_file.pcap"
output_file = 'output_json.json'

start_time = datetime.datetime(2023, 3, 24, 11, 57, 39)
end_time = datetime.datetime(2023, 3, 24, 11, 57, 40)

command = [
        "tshark", "-r", file_name, "-Y",
        f'(oran_fh_cus.extType == 11) && (frame.time >= "{start_time.strftime("%Y-%m-%d %H:%M:%S")}" && frame.time <= "{end_time.strftime("%Y-%m-%d %H:%M:%S")}")',
        "-T", "json", "-e", "frame.time", "-e", "frame.number", "-e", "oran_fh_cus.extType",
        "-e", "oran_fh_cus.bfwI", "-e", "oran_fh_cus.bfwQ"
    ]

with open(output_file, "w") as f:
    subprocess.run(command, stdout=f)

I encounter the following errors before stopping the program:

"Unexpected end of string"
"Expected comma or closing bracket"

However, when I manually stop the program, the last lines are added successfully, and the line that triggered the error appears to be completely normal.

英文:

I'm having trouble with my program that writes parts of a pcap file to a new json file. The issue is that the last 100 lines of the file are only visible when I manually stop the program in the terminal. It only writes the last lines if I forcefully kill it.

file_name = &quot;my_file.pcap&quot;
output_file = &#39;output_json.json&#39;

start_time = datetime.datetime(2023, 3, 24, 11, 57, 39)
end_time = datetime.datetime(2023, 3, 24, 11, 57, 40)

command = [
        &quot;tshark&quot;, &quot;-r&quot;, file_name, &quot;-Y&quot;,
        f&#39;(oran_fh_cus.extType == 11) &amp;&amp; (frame.time &gt;= &quot;{start_time.strftime(&quot;%Y-%m-%d %H:%M:%S&quot;)}&quot; &amp;&amp; frame.time &lt;= &quot;{end_time.strftime(&quot;%Y-%m-%d %H:%M:%S&quot;)}&quot;)&#39;,
        &quot;-T&quot;, &quot;json&quot;, &quot;-e&quot;, &quot;frame.time&quot;, &quot;-e&quot;, &quot;frame.number&quot;, &quot;-e&quot;, &quot;oran_fh_cus.extType&quot;,
        &quot;-e&quot;, &quot;oran_fh_cus.bfwI&quot;, &quot;-e&quot;, &quot;oran_fh_cus.bfwQ&quot;
    ]

with open(output_file, &quot;w&quot;) as f:
    subprocess.run(command, stdout=f)

I encounter the following errors before stopping the program:

"Unexpected end of string"
"Expected comma or closing bracket"

However, when I manually stop the program, the last lines are added successfully, and the line that triggered the error appears to be completely normal.

答案1

得分: 0

这是由于两个原因导致的:

  1. 您的输出被缓冲。 您可以使用 -l 标志来禁用缓冲(请参阅下面的手册文档)。这将使它在接收到每个数据包时打印出来。
  2. 正如 @jasonharper 在评论中解释的那样,由于您的输出是以 JSON 形式输出的,在 JSON 数据包数组被关闭之前,您的输出不能真正有效/完成,需要使用最后的 ]。如果这对您不起作用,我认为您将不得不切换到其他输出格式。

-l

    在打印每个数据包的信息后刷新标准输出。 (严格来说,如果指定了 -V,则不是行缓冲;然而,如果未指定 -V,由于每个数据包只打印一行,并且通常在将实时捕获管道传递到程序或脚本时使用 -l,以便数据包一旦被看到和解析,它就会立即显示输出,所以它应该像真正的行缓冲一样工作。我们这样做是为了解决 Microsoft Visual C++ C 库的一个不足。)

    当将 TShark 的输出导流到另一个程序时,这可能会有用,因为这意味着输出导流到的程序将在 TShark 看到数据包并生成输出后立即看到数据包的解析数据,而不是仅在包含该数据的标准输出缓冲区填满时才看到它。
英文:

This is happening for two reasons:

  1. Your output is buffered. You can disable that with the -l flag (see man page docs below). That will make it print each packet as they are received.
  2. As @jasonharper explained in comments, since you are emitted JSON as the output, your output can't be truly valid/finished until the array of JSON packets is closed, with the last ]. If that doesn't work for you, I think you will have to switch to some other output format.

-l

    Flush the standard output after the information for each packet is printed. (This is
    not, strictly speaking, line-buffered if -V was specified; however, it is the same as
    line-buffered if -V wasn&#39;t specified, as only one line is printed for each packet, and,
    as -l is normally used when piping a live capture to a program or script, so that
    output for a packet shows up as soon as the packet is seen and dissected, it should
    work just as well as true line-buffering. We do this as a workaround for a deficiency
    in the Microsoft Visual C++ C library.)

    This may be useful when piping the output of TShark to another program, as it means
    that the program to which the output is piped will see the dissected data for a packet
    as soon as TShark sees the packet and generates that output, rather than seeing it
    only when the standard output buffer containing that data fills up.

huangapple
  • 本文由 发表于 2023年6月5日 20:39:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/76406508.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定