Server-Sent events, why didn't I know about these?

Ok, I've just had an epiphany. I've been contemplating a chat system for my latest project. The options I had up until a few moments ago were:

Long polling - whereby the game client sends an AJAX request every second or so to see if there are any new chats it needs to grab. While this system definitely works, I can see it turning a web server into a puddle of goo when you have 1000 clients polling for chats every second along with trying to serve up web pages and responding to other AJAX requests.

Websockets - whereby I write some websocket server in node.js or c# and have it run as an application on a server somewhere with the game client logged into it. While this would be duplex, instantaneous and efficient, trying to configure a wss server and write the code for it sounds like a nightmare. Not to mention the fact that it would pretty much required a dedicated server somewhere so that it could run continuously.

And then my epiphany:

Server-Sent events - I just read about these here:

https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events

and decided to test it out and ummmm... wow.

So, I dropped an HTMLElement plugin on my layout and put this in the text field:

<ul><div id="screen"></div></ul>

var evtSource = new EventSource("//api.example.com/ssedemo.php", { withCredentials: true } ); 

evtSource.onmessage = function(e) {
 var newElement = document.createElement("li");
 var eventList = document.getElementById('screen');

 newElement.innerHTML = "message: " + e.data;
 eventList.appendChild(newElement);
}

 evtSource.addEventListener("ping", function(e) {
 var newElement = document.createElement("li");
 var eventList = document.getElementById('screen');
 
 var obj = JSON.parse(e.data);
 newElement.innerHTML = "ping at " + obj.time;
 eventList.appendChild(newElement);
}, false);

I haven't used server sent events before, but they look potentially useful for some situations. I had a quick look at how to implement them at a node.js level and I don't think your going to see any less system meltdown than long poll or websockets, they are very similar.

Long poll: Client makes network request to server, server holds onto request until data comes in or request times out ( normally ~ 30s ) then responds.

Short poll: Client makes network request to server, server responds immediately with "no change" or data.

Server sent events: Client makes network request to server, server holds onto connection and writes data to connection when it comes in.

Websocket: Client makes network request to server, server holds onto connection and writes data to connection when comes in, client can also send data to server.

I've used both short poll and websockets on commercial products and can say they work very well, with reasonably minimal complexity.

The only place I've seen long poll before is the Dropbox REST API, where you can subscribe to notifications on files changing. It's quite a pain to implement, on both client and server side as you need to prevent "stampede" situations. Responding closes the network connection so if you send a message to each client they will all close, then immediately reopen a connection causing a large bump in system resource usage.

If I were to implement a instant chat service I would either use short poll or websockets. Websockets have obvious advantages; the server knowing when a client connects/leaves, clients can send messages to the server without needing another request, users receive messages "instantly". The biggest issue with websockets is normally the server running out of sockets for connections. Short poll allows for more users than sockets ( in theory ) as they don't all connect at the same time. Also you can avoid needing an extra request for sending messages from the client by adding pending messages to the next poll request. A poll interval of between 1 - 5 seconds should be fast enough for most situations.

Personally I'd go with a WebSocket - the node ws module is pretty straightforward (assuming you're already familiar with HTTP), and setting up a secure WebSocket is no harder than setting up a secure website - a WebSocket connection starts with a HTTP request, after all.

Well what I was getting at is that with short poll you have 1000 users doing brief ( maybe 50ms ) requests every 5 seconds, but at different time offsets. So you don't actually have 1000 concurrent requests at any one time. Under worse case scenario you could have 1000 concurrent for 1000 users, but all the users would have to start at exactly the same time and have the same network latency.

The project I worked on had a cache in front of it, as the data wasn't specific to a user a proxy server/s served the same result to multiple requests for 1 second. So the proxy only had to make up to 1 request to the main server per second for polling, no matter how many users were connected. If you wanted to split users up into groups/chatrooms then you can have different end points per group, then you have up to 1 request per endpoint per second instead. I can't really remember how many users we got but for read only data that technique scales linearly with how many caching servers you have. Sending a message has to go direct to the server, but we can assume the message density will be a lot lower than the poll requests. So your requests would be (groupCount + sentMessagesPerSecond) for your caching period.

Much of the limit your looking at comes down to how PHP works, I would presume there's a way to pool processes to reduce start up time and limit how many new processes the server has to start up. I've never worked on a commercial PHP project so I don't know much about optimisation for it unfortunately.

As Ashley says websockets are likely to be the easiest solution, I would expect a node.js instance with the "ws" library to scale in the 10s of thousands with a single process without much difficulty. But as you say, that requires you to learn JS and pay for a VPS... I vaguely remember a few hosting providers that will give you a really slow VPS on a free intro tier, might be worth a look? Just keep an eye on your server usage, these things can rack up without you realising.

Thanks for the input guys. Obviously I have a bit of homework to do.