Can't reach stdout from Docker using Go client

I have a small project in which my go server copies the C files send by http into Docker containers, where they are compiled and run.
However, I am not able to obtain any data send to stdout in the container.

I have determined that file is sent into Docker container, what's more - any problems with compilation are shown on the error stream. However, sending data through stderr in C program also didn't show any results until I have, playing with Dockerfile, used '>&2 echo ""' which somehow pushed data through the stream and I was able to read it.

Right now, as mentioned above, I can only read stderr and solely thanks to a workaround. Any idea why I can't do it using standard methods?

Go server

package main
import (
"fmt"
"net/http"
"io"
"os"
"os/exec"
"log"
"encoding/json"
"github.com/docker/docker/client"
dockertypes "github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"golang.org/x/net/context"
"time"
"bytes"
)
type Result struct {
CompilationCode int
RunCode int
TestsPositive int
TestsTotal int
}
func upload(w http.ResponseWriter, r *http.Request) {
log.Println("method:", r.Method)
if r.Method == "POST" {
log.Println("Processing new SUBMISSION.")
// https://github.com/astaxie/build-web-application-with-golang/blob/master/de/04.5.md
r.ParseMultipartForm(32 << 20)
file, handler, err := r.FormFile("file")
if err != nil {
fmt.Println(err)
return
}
defer file.Close()
baseName:= os.Args[1]
f, err := os.OpenFile(baseName+handler.Filename, os.O_WRONLY|os.O_CREATE, 777)
if err != nil {
fmt.Println(err)
return
}
defer f.Close()
io.Copy(f, file)
if err != nil {
fmt.Println(err)
return
}
compilationCode, runCode, testsPositive, testsTotal := processWithDocker(baseName + handler.Filename, handler.Filename)
result := Result{
CompilationCode: compilationCode,
RunCode: runCode,
TestsPositive:testsPositive,
TestsTotal:testsTotal,
}
resultMarshaled, _ := json.Marshal(result)
w.Write(resultMarshaled)
} else {
w.Write([]byte("GO server is active. Use POST to submit your solution."))
}
}
// there is assumption that docker is installed where server.go is running
// and the container is already pulled
// TODO: handle situation when container is not pulled
// TODO: somehow capture if compilation wasn't successful and
// TODO: distinguish it from possible execution / time limit / memory limit error
// http://stackoverflow.com/questions/18986943/in-golang-how-can-i-write-the-stdout-of-an-exec-cmd-to-a-file
func processWithDocker(filenameWithDir string, filenameWithoutDir string) (int, int, int, int) {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
cli, err := client.NewEnvClient()
if err != nil {
panic(err)
}
var hostVolumeString = filenameWithDir
var hostConfigBindString = hostVolumeString  + ":/WORKING_FOLDER/" + filenameWithoutDir
var hostConfig = &container.HostConfig{
Binds: []string{hostConfigBindString},
}
resp, err := cli.ContainerCreate(ctx, &container.Config{
Image: "tusty53/ubuntu_c_runner:twelfth",
Env: []string{"F00=" + filenameWithoutDir},
Volumes: map[string]struct{}{
hostVolumeString: struct{}{},
},
}, hostConfig, nil, "")
if err != nil {
panic(err)
}
if err := cli.ContainerStart(ctx, resp.ID, dockertypes.ContainerStartOptions{}); err != nil {
panic(err)
}
fmt.Println(resp.ID)
var exited = false
for !exited {
json, err := cli.ContainerInspect(ctx, resp.ID)
if err != nil {
panic(err)
}
exited = json.State.Running
fmt.Println(json.State.Status)
}
normalOut, err := cli.ContainerLogs(ctx, resp.ID, dockertypes.ContainerLogsOptions{ShowStdout: true, ShowStderr: false})
if err != nil {
panic(err)
}
errorOut, err := cli.ContainerLogs(ctx, resp.ID, dockertypes.ContainerLogsOptions{ShowStdout: false, ShowStderr: true})
if err != nil {
panic(err)
}
buf := new(bytes.Buffer)
buf.ReadFrom(normalOut)
sOut := buf.String()
buf2 := new(bytes.Buffer)
buf2.ReadFrom(errorOut)
sErr := buf2.String()
log.Printf("start\n")
log.Printf(sOut)
log.Printf("end\n")
log.Printf("start error\n")
log.Printf(sErr)
log.Printf("end error\n")
var testsPositive=0
var testsTotal=0
if(sErr!=""){
return 0,0,0,0
}
if(sOut!=""){
fmt.Sscanf(sOut, "%d %d", &testsPositive, &testsTotal)
return 1,1,testsPositive,testsTotal
}
return 1,0,0,0
}
// Creates examine directory if it doesn't exist.
// If examine directory already exists, then comes an error.
func prepareDir() {
cmdMkdir := exec.Command("mkdir", os.Args[1])
errMkdir := cmdMkdir.Run()
if errMkdir != nil {
log.Println(errMkdir)
}
}
func main() {
prepareDir()
go http.HandleFunc("/submission", upload)
http.ListenAndServe(":8123", nil)
}

Dockerfile

FROM ubuntu
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update && \
apt-get -y install gcc
COPY . /WORKING_FOLDER
WORKDIR /WORKING_FOLDER
CMD ["./chain"] 

Chain file

#!/bin/bash
gcc -Wall $F00  -o hello
./hello
>&2 echo ""

I believe the following method can be used to obtain the stdout and stderr of running containers.

import "github.com/docker/docker/pkg/stdcopy"

import this package from the docker SDK.

    data, err := cli.ContainerLogs(ctx, resp.ID, types.ContainerLogsOptions{ShowStdout: true, ShowStderr: true})
if err != nil {
panic(err)
}

get the logs from the running container and store it to data.Now create two buffers to store the streams.

	// Demultiplex stdout and stderror
// from the container logs
stdoutput := new(bytes.Buffer)
stderror := new(bytes.Buffer)

now use the imported stdcopy to save the two streams to the buffers.

	stdcopy.StdCopy(stdoutput, stderror, data)
if err != nil {
panic(err)
}

You should consider running the Docker CLI as a separate process and just reading its stdout and stderr, for example:

cmd := exec.Command("docker", "run", "-v=<volumes-to-mount>", "<image>")
out, err := cmd.CombinedOutput()
if err != nil {
return err
}
// `out` now contains the combined stdout / stderr from Docker

I'm not familiar with the Go but the code seems to be reading from the docker log stream. It could be that your Docker daemon is configured to use a different log driver. The docker logs command is not available for drivers other than json-file and journald. You can check with the following command:

$ docker info |grep 'Logging Driver'

You can change the logging driver on a per container basis like so:

$ docker run -it --log-driver json-file <image> <command>

You should be able to pass this parameter via the api when creating the container as well.