You're right. But there will realistically still be some protocol-level buffering in user-mode for every socket.
Although that might be a good use-case for a hash table of buffers, as most of the sockets will not have outstanding partial packets most of the time in typical (frequent small message) applications.