Releasing memory from large objects

I came across something that I don't understand. Hope you guys can help!

Resources:

1. https://medium.com/@chaewonkong/solving-memory-leak-issues-in-go-http-clients-ba0b04574a83
2. https://www.golinuxcloud.com/golang-garbage-collector/

I read in several articles the suggestion that we can make the job of the GC easier by setting large slices and maps (I guess this applies to all reference types) to nil after we no longer need them. Here is one of the examples I read:

func ProcessResponse(resp *http.Response) error {
    data, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return err
    }
    // Process data here

    data = nil // Release memory
    return nil
}

It is my understanding that when the function ProcessResponse finishes the data variable will be out of scope and basically will no longer exist. The GC will then verify there is no reference to the []byte slice (the one that data pointed to) and will clear the memory.

How setting data to nil improves garbage collection?

Thanks!

As others have pointed out already: setting data = nil right before returning doesn't change anything in terms of GC. The go compiler will apply optimisations, and golang's garbage collector works in distinct phases. In the simplest of terms (with many omissions and over-simplifications): setting data = nil, and removing all references to the underlying slice is not going to trigger an atomic style release of the memory that is no longer referenced. Once the slice is no longer referenced, it'll be marked as such, and the associated memory won't be released until the next sweep.

Garbage collection is a hard problem, in no small part due to the fact that it's not the sort of problem that has an optimal solution that will produce the best results for all use-cases. Over the years, the go runtime has evolved quite a lot, with significant work being done precisely on the runtime garbage collector. The result is that there are very few situations where a simple someVar = nil will make even a small difference, let alone a noticeable one.

If you are looking for some simple rule-of-thumb type tips that can impact the runtime overhead associated with garbage collection (or runtime memory management in general), I do know of one that seems to be vaguely covered by this sentence in your question:

> suggestion that we can make the job of the GC easier by setting large slices and maps

This is something that can produce noticeable results, when profiling code. Say you're reading a large chunk of data that you need to process, or you're having to perform some other type of batch operation and return a slice, it's not uncommon to see people write things like this:

func processStuff(input []someTypes) []resultTypes {
    data := []resultTypes{}
    for _, in := range input {
        data = append(data, processT(in))
    }
    return data
}

This can be optimised quite easily by changing the code to this:

func processStuff(input []someTypes) []resultTypes {
    data := make([]resultTypes, 0, len(input)) // set cap
    for _, in := range input {
        data = append(data, processT(in))
    }
    return data
}

What happens in the first implementation is that you create a slice with len and cap of 0. The first time append is called, you're exceeding the current capacity of the slice, which will cause the runtime to allocate memory. As explained here, the new capacity is calculated rather simplistically, the memory is allocated and the data is copied over:

t := make([]byte, len(s), (cap(s)+1)*2)
copy(t, s)

Essentially, each time you call append when the slice you're appending to is full (ie len == cap), you'll allocate a new slice that can hold: (len + 1) * 2 elements. Knowing that, in the first example, data starts out with len and cap == 0, let's work see what that means:

1st iteration: append creates slice with cap (0+1) *2, data is now len 1, cap 2
2nd iteration: append adds to data, now has len 2, cap 2
3rd iteration: append allocates a new slice with cap (2 + 1) *2, copies the 2 elements from data to this slice and adds the third, data is now reassigned to a slice with len 3, cap 6
4th-6th iterations: data grows to len 6, cap 6
7th iteration: same as 3rd iteration, although cap is (6 + 1) * 2, everything is copied over, data is reassigned a slice with len 7, cap 14

If the data structures in your slice are on the larger side (ie many nested structures, lots of indirection, etc...) then this frequent re-allocating and copying can become quite expensive. If your code contains lots of these kind of loops, it will begin to show up in pprof (you'll start seeing a lot of time being spent calling gcmalloc). Moreover, if you're processing 15 input values, your data slice will end up looking like this:

dataSlice {
    len: 15
    cap: 30
    data underlying_array[30]
}

Meaning you'll have allocated memory for 30 values, when you only needed 15, and you'll have allocated that memory in 4 increasingly large chunks, with copying data each realloc.

By contrast, the second implementation will allocate a data slice that looks like this before the loop:

data {
    len: 0
    cap: 15
    data underlying_array[15]
}

It's allocated in one go, so no re-allocations and copying is needed, and the slice that is returned will take up half the space in memory. In that sense, we start out by allocating a larger slab of memory at the start, to cut down on the number of incremental allocation and copy calls required later on, which will, overall, cut down on runtime costs.

What if I don't know how much memory I need

That's a fair question. This example is not always going to apply. In this case we knew how many elements we'd need, and we could allocate memory accordingly. Sometimes, that's just not how the world works. If you don't know how much data you'll end up needing, then you can:

1. Make an educated guess: GC is difficult, and unlike you, the compiler and go runtime lack the fuzzy logic people have to come up with a realistic, reasonable guesstimate. Sometimes it'll be as simple as: "Well, I'm getting data from that data source, where we only ever store the last N elements, so worst case scenario, I'll be handling N elements", sometimes it's a bit more fuzzy, for example: you're processing a CSV containing a SKU, product name, and stock count. You know the length of the SKU, you can assume stock count will be an integer between 1 and 5 digits long, and a product name will on average be 2-3 words long. English words have an average length of 6 characters, so you can have a rough idea of how many bytes make up a CSV line: say SKU == 10 characters, 80 bytes, product description 2.5 * 6 * 8 = 120 bytes, and ~4 bytes for the stock count + 2 commas and a line break, makes for an average expected line length of 207 bytes, let's call it 200 to err on the side of caution. Stat the input file, divide its size in bytes by 200 and you should have a serviceable, slightly conservative estimate of the number of lines. Add some logging at the end of that code comparing the cap to the estimate, and you can tweak your prediction calculation accordingly.
2. Profile your code. It happens from time to time that you'll find yourself working on a new feature, or an entirely new project, and you don't have historical data to fall back on for a guesstimate. In that case, you can simply guess, run some test scenario's, or spin up a test environment feeding your version of the code production data and profile the code. When you're in the situation where you're actively profiling memory usage/runtime costs for just one or two slices/maps, I must stress that this is optimisation. You should only be spending time on this if this is a bottleneck or noticeable issue (e.g. overall profiling is impeded by runtime memory allocation). In the vast, vast majority of cases, this level of optimisation would fall firmly under the umbrella of micro-optimisation. Adhere to the 80-20 principle

Recap

No, setting a simple slice variable to nil won't make much of a difference in 99% of cases. When creating and appending to maps/slices, what is more likely to make a difference is to cut back on extraneous allocations by using make() + specifying a sensible cap value. Other things that can make a difference is using pointer types/receivers, although that's an even more complex topic to delve in to. For now, I'll just say that I've been working on a code base that has to operate on numbers far beyond the range of your typical uint64, and we have to unfortunately be able to use decimals in a way that is more precise than float64 will allow. We've solved the uint64 issue by using someting like holiman/uint256, which uses pointer receivers, and tackle the decimal problem with shopspring/decimal, which uses value receivers and copies everything. After spending a lot of time optimising the code, we've reached the point where the performance impact of the constant copying of values when using decimals has become an issue. Look at how these packages implement simple operations like addition and try to work out which operation is more costly:

// original
a, b := 1, 2
a += b
// uint256 version
a, b := uint256.NewUint(1), uint256.NewUint(2)
a.Add(a, b)
// decimal version
a, b := decimal.NewFromInt(1), decimal.NewFromInt(2)
a = a.Add(b)

These are just a couple of things that, in my recent work, I've spent time on optimising, but the single most important thing to take away from this is:

Premature optimisation is the root of all evil

When you're working on more complex problems/code, then getting to a point where you're looking in to allocation cycles for slices or maps as potential bottlenecks and optimisations takes a lot of effort. You can, and arguably should, take measures to avoid being too wasteful (e.g. setting a slice cap if you know what the eventual length of said slice will be), but you shouldn't waste too much time hand-crafting every line until the memory footprint of that code is as small as it possibly can be. The cost will be: code that is more fragile/harder to maintain and read, potentially deteriorated overall performance (seriously, you can trust the go runtime to do a decent job), lots of blood, sweat, and tears, and a steep decrease in productivity.