Goroutine I/O scheduling

Golangs goroutines present an interface of blocking I/O to the goroutine (-programmer). Behind the scenes the runtime naturally uses some kind of non-blocking I/O to prevent the OS from suspending the OS-thread, so that the runtime can run another goroutine on top of the OS thread while the I/O is performed.

When does the runtime consider the I/O performed so that it can reschedule the goroutine?

To make it clear, assuming I have a net.TCPConn that I call Write on, when can I expect the goroutine to be rescheduled?

conn, err := net.Dial("tcp", serverAddr)
conn.Write(buffer)
timestamp = time.Now()

That is when can I expect the timestamp to be taken?

• When the buffer has been copied to the golang runtime?
• When the buffer has been copied to the runtime and to the OS's kernel space?
• When the buffer has been copied to the runtime, kernel space and additionally to the NIC's send buffer?
• When the buffer has been sent over the network/from the NIC?
• When the buffer has been acknowledged by the recieve ends TCP stack?

You can have a look in file https://github.com/golang/go/blob/master/src/net/fd_unix.go (Write function).

Basically, it depends whether the socket buffer has enough space or not.

If there is enough space in the socket buffer to accommodate the size of your write operation, the data will be immediately written to the socket buffer. I guess this corresponds to your second answer. Additionally, the kernel may actually send the packet (or add it to the NIC queues), but it is independent from the Go runtime.

If there is not enough space in the socket buffer to accommodate the whole write operation, only part of the data will be immediately written to the socket buffer. Then, the call will block (via the runtime polling engine) until the kernel has made some space in the socket buffer (by sending some packets). As soon as some space is available, and all the data have been copied, the call will unblock.

You should consider the timestamp is taken when the net package has written the whole buffer in the socket buffer via a system call.

The following article describes how the netpoller works:

>Whenever a goroutine tries to read or write to a connection, the networking code will do the operation until it receives such an error, then call into the netpoller, telling it to notify the goroutine when it is ready to perform I/O again. The goroutine is then scheduled out of the thread it's running on and another goroutine is run in its place.
>
> When the netpoller receives notification from the OS that it can perform I/O on a file descriptor, it will look through its internal data structure, see if there are any goroutines that are blocked on that file and notify them if there are any. The goroutine can then retry the I/O operation that caused it to block and succeed in doing so."

Thus we conclude that the goroutine can be rescheduled whenever the underlying system call completes writing for the entire buffer. In the case of Linux, it seems to bee when the message has been copied to the kernel space send buffer: https://stackoverflow.com/questions/5407182/blocking-sockets-when-exactly-does-send-return. Which in turn is my second original option "When the buffer has been copied to the runtime and to the OS's kernel space"; also consistent with Didier Spezia's answer.