Send a chunked HTTP response from a Go server

I am creating a test Go HTTP server, and I am sending a response header of Transfer-Encoding: chunked so I can continually send new data as I retrieve it. This server should write a chunk to this server every one second. The client should be able to receive them on demand.

Unfortunately, the client(curl in this case), receives all of the chunks at the end of the duration, 5 seconds, rather than receiving one chunk every one second. Also, Go seems to send the Content-Length for me. I want to send the Content-Length at the end, and I want the the header's value to be 0.

Here is the server code:

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "time"
)

func main() {
    http.HandleFunc("/test", HandlePost);
    log.Fatal(http.ListenAndServe(":8080", nil))
}

func HandlePost(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Connection", "Keep-Alive")
    w.Header().Set("Transfer-Encoding", "chunked")
    w.Header().Set("X-Content-Type-Options", "nosniff")

    ticker := time.NewTicker(time.Second)
    go func() {
        for t := range ticker.C {
            io.WriteString(w, "Chunk")
            fmt.Println("Tick at", t)
        }
    }()
    time.Sleep(time.Second * 5)
    ticker.Stop()
    fmt.Println("Finished: should return Content-Length: 0 here")
    w.Header().Set("Content-Length", "0")
}

The trick appears to be that you simply need to call Flusher.Flush() after each chunk is written. Note also that the "Transfer-Encoding" header will be handled by the writer implicitly, so no need to set it.

func main() {
  http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    flusher, ok := w.(http.Flusher)
    if !ok {
      panic("expected http.ResponseWriter to be an http.Flusher")
    }
    w.Header().Set("X-Content-Type-Options", "nosniff")
    for i := 1; i <= 10; i++ {
      fmt.Fprintf(w, "Chunk #%d\n", i)
      flusher.Flush() // Trigger "chunked" encoding and send a chunk...
      time.Sleep(500 * time.Millisecond)
    }
  })

  log.Print("Listening on localhost:8080")
  log.Fatal(http.ListenAndServe(":8080", nil))
}

You can verify by using telnet:

<!-- language: lang-none -->

$ telnet localhost 8080
Trying ::1...
Connected to localhost.
Escape character is '^]'.
GET / HTTP/1.1

HTTP/1.1 200 OK
Date: Tue, 02 Jun 2015 18:16:38 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked

9
Chunk #1

9
Chunk #2

...

You might need to do some research to verify that http.ResponseWriters support concurrent access for use by multiple goroutines.

Also, see this question for more information about the "X-Content-Type-Options" header.

It looks like httputil provides a NewChunkedReader function

I think your question title is a bit misleading. The answer in your scenario has to cater to the unique need of waiting a second before sending each chunk rather than sending the chunks as fast as possible, which would be easy to do with io.Copy and req.Header.Set("Transfer-Encoding", "chunked")

There are a couple of things that you are doing that are unnecessary (don't need a goroutine, don't need specific headers). But, for educational purposes, I'll show the minimum way to get your code working, which includes 2 changes:

package main

import (
	"fmt"
	"io"
	"log"
	"net/http"
	"time"
)

func main() {
	http.HandleFunc("/test", HandlePost)
	log.Fatal(http.ListenAndServe(":8080", nil))
}

func HandlePost(w http.ResponseWriter, r *http.Request) {
    // #1 add flusher
	flusher, ok := w.(http.Flusher)
	if !ok {
		panic("expected http.ResponseWriter to be an http.Flusher")
	}
    w.Header().Set("Connection", "Keep-Alive")
    // Don't need these this bc manually flushing sets this header
    // w.Header().Set("Transfer-Encoding", "chunked")
    
    // make sure this header is set
	w.Header().Set("X-Content-Type-Options", "nosniff")

	ticker := time.NewTicker(time.Millisecond * 500)
	go func() {
		for t := range ticker.C {
            // #2 add '\n'
			io.WriteString(w, "Chunk\n")
			fmt.Println("Tick at", t)
			flusher.Flush()
		}
	}()
	time.Sleep(time.Second * 5)
	ticker.Stop()
    // Don't need this
    // fmt.Println("Finished: should return Content-Length: 0 here")
    // w.Header().Set("Content-Length", "0")
}

1. As @maetrics suggested, you need to use a flusher to force the ResponseWriter to send the chunk, even though the writer's internal buffer isn't filled yet. If you didn't call flush manually, it would call flush internally once the 2048 byte internal buffer is full.

2. As the HTTP Transfer-Encoding protocol suggests. You need to add a \n to each chunk you want to send. Otherwise, the flush call won't work as expected, and you will receive the whole message together rather than in 1-second increments, as intended.

Notes:

• As @morganbaz noted in the comments, you need to make sure the X-Content-Type-Options: nosniff header is set. See @morganbaz's comment for the explanation.

• Since you are manually calling flush, there is no need to set the Transfer-Encoding header.

• You shouldn't set the content-length header when the Transfer-Encoded: chunked header is set. See the HTTP Transfer-Encoding Protocol.