How to return hash and bytes in one step in Go?

I'm trying to understand how I can read content of the file, calculate its hash and return its bytes in one Go. So far, I'm doing this in two steps, e.g.

// calculate file checksum
hasher := sha256.New()
f, err := os.Open(fname)
if err != nil {
    msg := fmt.Sprintf("Unable to open file %s, %v", fname, err)
    panic(msg)
}
defer f.Close()
b, err := io.Copy(hasher, f)
if err != nil {
    panic(err)
}
cksum := hex.EncodeToString(hasher.Sum(nil))

// read again (!!!) to get data as bytes array
data, err := ioutil.ReadFile(fname)

Obviously it is not the most efficient way to do this, since read happens twice, once in copy to pass to hasher and another in ioutil to read file and return list of bytes. I'm struggling to understand how I can combine these steps together and do in one go, read data once, calculate any hash and return it along with list of bytes to another layer.

If you want to read a file, without creating a copy of the entire file in memory, and at the same time calculate its hash, you can do so with a TeeReader:

hasher := sha256.New()
f, err := os.Open(fname)
data := io.TeeReader(f, hasher)
// Now read from data as usual, which is still a stream.

What happens here is that any bytes that are read from data (which is a Reader just like the file object f is) will be pushed to hasheras well.

Note, however, that hasher will produce the correct hash only once you have read the entire file through data, and not until then. So if you need the hash before you decide whether or not you want to read the file, you are left with the options of either doing it in two passes (for example like you are now), or to always read the file but discard the result if the hash check failed.

If you do read the file in two passes, you could of course buffer the entire file data in a byte buffer in memory. However, the operating system will typically cache the file you just read in RAM anyway (if possible), so the performance benefit of doing a buffered two-pass solution yourself rather than just doing two passes over the file is probably negligible.

You can write bytes directly to the hasher. For example:

package main

import (
	"crypto/sha256"
	"encoding/hex"
	"io/ioutil"
)

func main() {
	hasher := sha256.New()

	data, err := ioutil.ReadFile("foo.txt")
	if err != nil {
		panic(err)
	}

	hasher.Write(data)
	cksum := hex.EncodeToString(hasher.Sum(nil))

	println(cksum)
}

As the Hash interface embeds io.Writer. This allows you to read the bytes from the file once, write them into the hasher then also return them out.

Do data, err := ioutil.ReadFile(fname) first. You'll have your slice of bytes. Then create your hasher, and do hasher.Write(data).

If you plan on hashing files you shouldn't read the whole file into memory because... there are large files that don't fit into RAM. Yes, in practice you very rarely will run into such out-of-memory issues but you can easily prevent them. The Hash interface is an io.Writer. Usually, the Hash packages have a New function that return a Hash. This allows you to read the file in blocks and continuously feed it to the Write method of the Hash you have. You may also use methods like io.Copy to do this:

h := sha256.New()
data := &bytes.Buffer{}
data.Write([]byte("hi there"))
data.Write([]byte("folks"))
io.Copy(h, data)
fmt.Printf("%x", h.Sum(nil))

io.Copy uses a bufer of 32KiB internally so using it requires around 32KiB of memory max.