Tag: websockets
All the articles with the tag "websockets".
Streaming Tokens to the User: SSE, WebSockets, and the Wait the Reader Feels
Updated:Streaming doesn't make a model generate any faster. It moves the wait the reader feels from the whole answer to the first token. A streaming server on Python's standard library, delivering a real model's tokens three ways: one response, server-sent events, and a WebSocket written by hand. Measured: time to first text, bytes on the wire, reading speed against generation speed, what happens when the user leaves, and how a buffering proxy quietly undoes all of it.