How to write to a file in-memory, and then send it via HTTP without using an intermediate file?

I need to request, process and format data to CSV, and then send them via a web service on request.

Assuming that the requested and processed data are in data below, I managed to do that using an intermediate temporary file:

package main

import (
	"encoding/csv"
	"io/ioutil"
	"net/http"
	"os"
	"strconv"

	"github.com/go-chi/chi/v5"
)

type Data struct {
	Name string
	Age  int
}

func main() {
	data := []Data{
		{"John", 30},
		{"Jane", 20},
	}
	tempFileName := "temp.csv"
	// create temporary intermediate file
	file, err := os.Create(tempFileName)
	defer file.Close()
	if err != nil {
		panic(err)
	}
	w := csv.NewWriter(file)
	var csvData [][]string
	for _, record := range data {
		row := []string{record.Name, strconv.Itoa(record.Age)}
		csvData = append(csvData, row)
	}
	w.WriteAll(csvData)

	// read temporary intermediate file to send it via HTTP
	fileBytes, err := ioutil.ReadFile(tempFileName)
	if err != nil {
		panic(err)
	}
	// send the file on request
	router := chi.NewRouter()
	router.Get("/data", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/octet-stream")
		w.WriteHeader(http.StatusOK)
		w.Write(fileBytes)
	})
	http.ListenAndServe(":8087", router)
}

The request is successful:

PS C:\temp> wget http://localhost:8087/data
--2023-06-13 15:34:00--  http://localhost:8087/data
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:8087... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16 [application/octet-stream]
Saving to: 'data'
data                          100%[=================================================>]      16  --.-KB/s    in 0s
2023-06-13 15:34:00 (523 KB/s) - 'data' saved [16/16]
PS C:\temp> cat data
John,30
Jane,20

So far so good - now I would like to get rid of the intermediate file by

• using an in-memory "container" for the CSV data to be written to
• write its contents as bytes to the HTTP writer

I still do not have a good understanding of how to use io.* and bufio.* to land on the required types:

• csv.NewWriter() requires an *os.File and returns a *csv.Writer
• this *csv.Writer contents somehow need to be extracted as byte ...
• ... in order to feed the http.ResponseWriter.Write()

What are the general mechanics (and the ones specific to this case) to approach such a problem of different types of in-memory file manipulation?

csv.NewWriter takes an io.Writer, so you can pass the instance of http.ResponseWriter to it without writing the content to a file or memory.

Here is the demo:

package main

import (
	"encoding/csv"
	"fmt"
	"net/http"
	"strconv"

	"github.com/go-chi/chi/v5"
)

type Data struct {
	Name string
	Age  int
}

func main() {
	router := chi.NewRouter()
	router.Get("/data", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/octet-stream")
		w.WriteHeader(http.StatusOK)
		data := []Data{
			{"John", 30},
			{"Jane", 20},
		}
		csvWriter := csv.NewWriter(w)
		var csvData [][]string
		for _, record := range data {
			row := []string{record.Name, strconv.Itoa(record.Age)}
			csvData = append(csvData, row)
		}
		if err := csvWriter.WriteAll(csvData); err != nil {
			// handle the error
			fmt.Println(err)
		}
	})
	http.ListenAndServe(":8087", router)
}

Thanks to JimB's comment (to use bytes.Buffer) as well as an answer about using pointers to Buffer) I managed to get rid of the temporary file:

package main

import (
	"bytes"
	"encoding/csv"
	"net/http"
	"strconv"

	"github.com/go-chi/chi/v5"
)

type Data struct {
	Name string
	Age  int
}

func main() {
	data := []Data{
		{"John", 30},
		{"Jane", 20},
	}
	var file bytes.Buffer
	w := csv.NewWriter(&file)
	var csvData [][]string
	for _, record := range data {
		row := []string{record.Name, strconv.Itoa(record.Age)}
		csvData = append(csvData, row)
	}
	w.WriteAll(csvData)

	// send the file on request
	router := chi.NewRouter()
	router.Get("/data", func(w http.ResponseWriter, r *http.Request) {
		w.Header().Set("Content-Type", "application/octet-stream")
		w.WriteHeader(http.StatusOK)
		w.Write(file.Bytes())
	})
	http.ListenAndServe(":8087", router)
}

The better answer is Zeke Lu's one.