Maintaning more than 65535 connections on single IP

Reading the following article: 10M concurrent websockets

So, there are 1000 websocket servers listening on ports 10000-11000. When a connection is made to one of these servers, I assume they continue communication from a random established TCP connection with random ports. So, as one IP is used, and there are 64K ports, how can one maintain 10M connections? Are connections identified by IP-Port pairs? Can two different connections from different IPs to same port be established? How does this work under the hood?

> When a connection is made to one of these servers, I assume they continue communication from a random established TCP connection with random ports.

Wrong assumption. They communicate with the clients using the same local port number they are listening on.

> So, as one IP is used, and there are 64K ports, how can one maintain 10M connections?

Not a problem.

> Are connections identified by IP-Port pairs?

Yes.

> Can two different connections from different IPs to same port be established?

Yes.

> How does this work under the hood?

See above. IP:port pairs. You answered your own question.

Sorry for totally changing my answer.

Linux can easily support millions of open sockets if the machine has enough memory and processing power. The TCP/IP stack allows this because the socket the OS targets for a given TCP packet is determined by the source and destination IP and port tuple.

The server implementing the websocket protocol need only listen to a single TCP socket, often defined by the HTTP or HTTPS port number, but not in this example. As part of standard TCP handshaking, the server OS and application open a unique socket for the TCP connection to the new client when the HTTP request which is a websocket request is received. The websocket package takes care of upgrading the protocol used on this new socket from standard HTTP to websocket.

In the example, a goroutine is started for each websocket socket.

The client side, the side initiating the TCP connections, is limited by the number of ephemeral ports its OS can open for a given destination host and port. Honestly, I don't know if this is a limitation of the client OS or the TCP/IP specification itself.

I think the part you are missing is a TCP connection is actually two pairs of IP:PORT.

One for the server, one for the client.

The listening side of a tcp socket is generally always the same IP/Port pair.

Example: net.Listen("tcp", ":8080") is listening on port 8080 (on all interfaces in this case)

The connecting (client) side is usually uses a single outgoing IP along with a random port.

Example: net.Dial("tcp","server:8080) Selects a random available ephemeral port and then attempts to connect to server:8080.

So, in the above example, that connection is: client.ip:32768 -> server.ip:8080 (where 32768 is the ephemeral port selected)

the two pairs combined make a unique connection.

The server side can take as many connections from a single client as there are available (client side) ports. It can also take as many clients are there are IP addresses.

Think of it as, for one listening socket, you can theoretically have 2^16(ports) * 2^32(ipv4 addrs) connections.

In reality, there are reserved IPs, ports, memory limitations, etc so the number is far smaller.

For exmaple, the ephemeral port range on Linux is 32768 - 61000. Which means I'll start getting errors if I net.Dial("tcp", "server:8080") more than 28232 times as I will have exhausted my ephemeral port range for the given server address. But if the server is listening on 2 separate ports, I can do 28232 to the first port, and another 28232 to the second port.

When you see people do the 10MM connection tests, they have to use multiple client IPs or multiple server IPs/Ports to achieve this (or a combo of both to get 10MM unique client:ip/server:ip pairs)