> Suppose that typically 5% of the client connections are actually active, and the other 95% are idle with no pending reads or writes.
I suppose the intent is to then use that 95% memory savings for other work.
So what's the point of making this optimization? The system is going to be hugged to death if the number of client connections approaches 100%, because the system will not have enough available memory.
If 100% client connections is causing memory problems, rather than keeping a pool of buffers so that inactive client connections have less footprint, it seems like a better solution is to decrease the number of available client connections. Then, provision more servers or more memory if more client connections are required.
Provisioning for the worst possible case is sometimes necessary (e.g. hard real-time use cases), but can also be very expensive.
Since HTTP connections are typically kept alive between requests, it's very common for them to be idle. It could be the case that 99.9% of the time, X bytes of memory is enough to avoid transfer delays, but to cover the 100% case, you'd need 10X that.
I think it's reasonable to decide that avoiding delays in the 100% case is not worth 10X the cost in memory.
The answer can always be "throw more servers/resources at it", and that's exactly why we end up with simple websites/programs reacting slower than their equivalent two decades ago. I have too many apps on my phone that don't work properly and take ages to load or perform simple tasks. I tried to report this for some apps, the answer usually is that my phone is too old or my internet too slow (I know not directly related to a server, but I guess developers wouldn't think differently there).
If you use a magnitude fewer resources, you can combine more services on one machine, ultimately allowing faster servicing and a better user experience.
And as I understand that's actually how many Go programmers think. If you don't care about resources, you can set up a simple server in Python. The beauty about Go is that it gives a good developer experience while still allowing to write efficient, optimized programs.
It uses a pool of memory. If the pool is full because too many clients become active at the same time, only a part of the clients will be served in a timely manner. The other clients will have to wait for their reads to be processed. For a proxy handling long-running connections, this seems acceptable to have a slight delay in the 1% of cases where too many clients wake up at the same time. Buying 20× the number of servers don't seem a sensible solution.
I suppose the intent is to then use that 95% memory savings for other work.
So what's the point of making this optimization? The system is going to be hugged to death if the number of client connections approaches 100%, because the system will not have enough available memory.
If 100% client connections is causing memory problems, rather than keeping a pool of buffers so that inactive client connections have less footprint, it seems like a better solution is to decrease the number of available client connections. Then, provision more servers or more memory if more client connections are required.