How to both redirect output and have a console window for a subprocess?

I want to use
subprocess.Popen(["start", "/wait", "program"], shell=True) to have my program open in a new window. At the same time, I want to be able to capture output (stdout), for example by using subprocess.PIPE. However, if I simply add the pipe directive (stdout=.. in popen) , the pipe is of the start program which prints nothing.

I tried to follow https://stackoverflow.com/questions/60275345/how-to-redirect-output-from-start-ed-console and use :

            v = ["start" ,"/wait", "program", "^>1"]

but it didn't work (hangs on readline).

I know I can use log file, but kind of inclined against (because of buffering mainly).

The start command will try to start whatever program is as a new process and if program is console-based, a new console window will open for it to write output (and errors) to and possibly receive input from.

Assuming your application program is console-based, it only has a single stdout stream, which the console window will receive and echo on the window.

As you found, redirecting the stdout of the start command itself doesn't help, since it doesn't actually write anything to standard out (when all is well) - it only serves to launch the other program.

For example, if you tried something like this:

start /wait C:\Windows\system32\PING.EXE 8.8.8.8 -t

... you would see the output in a new console, and nothing in in the calling window / process.

If you tried to redirect the output like this:

start /wait C:\Windows\system32\PING.EXE 8.8.8.8 -t > test.txt

... that wouldn't work either, since it would just be redirecting the output from start, which is nothing.

But this also doesn't work:

start /wait C:\Windows\system32\PING.EXE 8.8.8.8 -t ^> test.txt

... since there isn't anything to apply the escaped redirect to. That's why the question you linked suggests this:

start /wait cmd /c C:\Windows\system32\PING.EXE 8.8.8.8 -t ^> test.txt

This almost works, in that it starts the process, opens the window and runs the program, and captures the output in a file. But it does all of this from the new process, not in the calling process.

However, you want the text from the program to still show up on the new console window, as well as have live access to the same output in your script (in your case not to redirect it to a file, but presumably use it in some other way).

You can get close by starting the command in the new console with PowerShell, using Start-Process instead of cmd /c and using Tee-Object which is similar to the Linux tee command.

However, this only succeeds in writing the output to a file as well as to the console, all from the new console. You could of course monitor the file for changes from your Python script, but this would still only get you new data whenever Tee-Object flushes the file buffer to disk. I don't think there's a way for you to get this to work using normal console windows in Windows, simply due to the way those work, and the fact that you can't just redirect a stream from one process to another using console windows.

The solution here might be to write a Python script that runs in the new console window, both echoing the data to the console as well as processing it. You just launch this helper script from your original script and the effect is exactly what you need. You'd have to provide more details to explain why that wouldn't be a suitable solution in your case.

start is a shell command, not a separate program. The Popen function redirects the output of the program it runs, not the output of shell commands.

We could try to create a batch file that starts your program and redirects its output to a file, and then run this batch file with Popen.

Lets do a start.bat batch file which contain:

start /wait program > output.txt

Then we can run it with Popen like that

import subprocess

p = subprocess.Popen(["start.bat"], shell=True)