golang errors with bind address already in use even though nothing is running on the port

I have a setup in golang which basically gets a free port from OS and then starts a http sever on it. It started to give random errors with port signup failures. I simplified it into the following program which seems to error after grabbing a few free ports. It happens very randomly and there is no real process running on the port it errors. Doesn't make sense at all to me on why this has to error. Any help would be appreciated.

Output of the program:

..<br/>
..<br/>
58479<br/>
..<br/>
..<br/>
58867<br/>
58868<br/>
58869<br/>
..<br/>
bound well! 58867<br/>
bound well! 58868<br/>
bound well! 58869<br/>
..<br/>
..<br/>
..<br/>
2015/04/28 09:05:09 Error while binding port: listen tcp :58479: bind: address already in use<br/>

I made sure to check that the free port that came out never repeated.

package main

 import (
    "net"
    "net/http"
    "log"
    "fmt"
)

func main() {

    for {
        l, _ := net.Listen("tcp", ":0")
        var port = l.Addr().String()[5:]
        l.Close()
        fmt.Println(port)
        go func() {
                l1, err := net.Listen("tcp", ":"+port)
                if (err != nil) {
                    log.Fatal("Error while binding port: ", err.Error())
                } else {
                    fmt.Println("bound well! ", port)
                }
                http.Serve(l1, nil)
            }()
    }
}

What you do is checking whether the port is free at one point and then you try to use it basing on the fact it was free in the past. This is not going to work.

What happens: with every iteration of a for loop, you generate a port number and make sure it's free. Then you spawn a routine with intention of using this port (which is already released back to the pool of free ports). You don't really know when the routine kicks in. It might be activated while the main routine (the for loop) has just generated another free port – maybe the same one again? Or maybe another process has taken this port in a meantime. Essentially you can have a race condition on a single port.

After some more research:

There's a small caveat though. Two different sockets can be bound to the same ip+port as long as the local+remote pair is unique. I've once written a response about it. So when I've been creating listener with :0 I was able to get a "collision"; proved by netstat -an:

10.0.1.11.65245        *.*                    LISTEN
10.0.1.11.65245        17.143.164.101.5223    ESTABLISHED

Now, the thing is that if you want to explicitly bind the socket the port being used, this is not possible. Probably because you would be able to specify local address only and remote address wouldn't be known until call to listen or connect (we're talking about syscalls now, not the Go interface). In other words, when you leave port unspecified, OS has a wider choice. So if it happens you got a local address that is also being used by another socket, you're unable to bind to it manually.

How to sovle it:

As I've mentioned in the comments, your server process should be using :0 notation in order to be able to choose available resource from OS. Once it's listening, the address should be announced to interested processes. You can do it, for example, through a file or a standard output.

Firstly I check the port:

$ lsof -i :8080

The results are:

COMMAND     PID USER   FD   TYPE             DEVICE SIZE/OFF NODE NAME
WeChat     1527 wonz  241u  IPv4 0xbfe135d7b32e86f1      0t0  TCP wonzdembp:63849->116.128.133.101:http-alt (ESTABLISHED)
__debug_b 41009 wonz    4u  IPv6 0xbfe135e149b1e659      0t0  TCP *:http-alt (LISTEN)

So I kill PID:

$ kill 41009

Then it works.

It is possible you were previously running or debugging an application on this port, and it did not shut down cleanly. The process might still be hanging out in your system's memory. Kill that process completely, and any other network daemons that might be lurking in the shadows, and try to run your application again.

If you haven't checked for this already, you can use (if using Linux) top, htop, or any GUI system monitor like Windows' Task Manager, Gnome3's System Monitor or KDE's KSysGuard to search for the offending process.

For an example, I have observed that Visual Studio Code's debugger/runner utility (F5/Ctrl+F5) does not always clean up the process, especially if you hit F5 too quickly and the old debugger did not shut down.

Use reuseport to afftectively use the port for listen.

"github.com/libp2p/go-reuseport"

l, err := reuseport.Listen("tcp", ":"+strconv.Itoa(tcpPort))

instead of

l1, err := net.Listen("tcp", ":"+port)