In Go, when running exec.Command with /usr/bin/script before a command, an error is thrown: /usr/bin/script: invalid option -- 'P'

I am using Go to automate IBM Aspera uploads. I would like to capture the stdout/stderr percentage progress while this happens.

I am running this on an ubuntu:latest docker image

This is the entire command that I am trying to run in go.

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

This is how I am calling this command in my Go project

cmd := exec.Command("/usr/bin/script", "-e", "-c", "'ascp", "-P <port>", "-m 3g", "-l 3g", "-i <secret>", "local_folder", "<user>@<host>:<remote-folder>'", "> percentage.txt", "2>&1")

I have found I have to use /usr/bin/script to capture the output of the ascp command, otherwise I am unable to capture the upload percentage from the stdout/stderr. I am only able to capture the success message and total bytes transferred.

Example percentage.txt output WITHOUT using /usr/bin/script:

Completed: 2389K bytes transferred in 1 seconds
 (10275K bits/sec), in 3 files, 1 directory.

Example percentage.txt output WITH using /usr/bin/script. As you can see, I am able to preserve the upload percentage:

Script started, output log file is 'typescript'.
<sample_file>				                                                                        100% 2194KB 13.3Mb/s    00:01    
<sample_file>				                                                                        100%  192KB 13.3Mb/s    00:01    
<sample_file>				                                                                        100% 3329   13.3Mb/s    00:01    
Completed: 2389K bytes transferred in 1 seconds
 (12268K bits/sec), in 3 files, 1 directory.
Script done.

When I take the raw command above, and run it directly on the cli of my docker instance, I have no issue and it works as expected.

However, when I attempt to run the command through the exec.Command() function, I receive this output

/usr/bin/script: invalid option -- 'P'
Try 'script --help' for more information.


exit status 1


panic: exit status 1

goroutine 1 [running]:
main.main()
        /project_dir/main.go:40 +0x3e7
exit status 2

When I run println(cmd.String()) this is my output:

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

And if I copy and paste the outputted command string from the cmd.string into my terminal, instead of running it through exec.Command(), it works and the percentage.txt captures the upload status.

What am I doing wrong here? Why is go's exec.Command() unable to run this, but my shell prompt is?

Full code:

func main() {
	f, err := os.OpenFile("percentage.txt", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
	if err != nil {
		panic(err)
	}
	defer f.Close()

	mwriter := io.MultiWriter(f, os.Stdout)

	err = os.Setenv("SHELL", "bash")
	if err != nil {
		return
	}
	cmd := exec.Command("/usr/bin/script", "-e", "-c", "'ascp", "-P <port>", "-m 3g", "-l 3g", "-i <secret>", "local_folder", "<user>@<host>:<remote-folder>'", "> percentage.txt", "2>&1")
	println(cmd.String())

	cmd.Stderr = mwriter
	cmd.Stdout = mwriter

	err = cmd.Run()
	if err != nil {
		println("\n\n" + err.Error() + "\n\n")
		panic(err)
	}
}

You have mis-tokenized your command line. If this is the shell command:

/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1

Then the corresponding exec.Command would almost be:

cmd := exec.Command("/usr/bin/script", "-e", "-c", "ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>")

That is, everything enclosed by single quotes on the command line is a single argument.

Using this version of the command, you will need to handle output redirection yourself (by reading the output from the command and writing it to a file).

If you really don't want to deal with reading the output from the command, you could explicitly call out to a shell:

cmd := exec.Command("/bin/sh", "-c", "/usr/bin/script -e -c 'ascp -P <port> -m 3g -l 3g -i <secret> local_folder <user>@<host>:<remote-folder>' > percentage.txt 2>&1")

...but this is typically less efficient and less robust, because now you need to be careful about characters (like >) that have special meaning to the shell and you need to be careful about properly quoting everything.

Using ascp directly will bring complexity and you will try to parse logs that may anyway change any time.

Have look to the Aspera Transfer SDK.

This SDK provides a GRPC interface and Go stubs.

The SDK provides auto-resume of failed transfers, progress monitoring, multi-session, etc...

The SDK is free and can be downloaded from the IBM developer site.