Since my first blog I've been putting together the overall architecture of the game. You can see some of the initial work in the commit log on GitHub. This can be slow going at the start: these initial decisions are very important and can end up very difficult to change later on. In this post I'll cover the overall technical design I'm aiming for.
The game server
For multiplayer games there is generally a server that has the true state of the game, and clients aim to sync with that. The GameServer class will represent the authoritative state of the game. The GameClient class is the local counterpart that aims to sync with the state on GameServer.
Construct's Multiplayer feature works via peer-to-peer connections. This actually avoids the need to run dedicated servers, which helps make it cheap to run! (You can run TURN servers to improve connectivity, but those will only deal with a fraction of the bandwidth, as most connections can still run peer-to-peer.) In a multiplayer game one player will act as the host, and that player will also run GameServer and act as the server for all players.
One interesting design point is that when the host has a local GameServer, it will still run the same code to sync the local GameClient with the GameServer. However the link between the two will have zero latency and unlimited bandwidth, so it should sync perfectly. The main reason for this is it actually saves on coding. If there was some special case to handle being both a server and client at the same time, that means writing a lot of code to handle that mode separately. Since there will already be this whole system to manage a remote client, it can just be re-used in its entirety for a local client. Here's a diagram that shows how it will work with two players.
Two players in a multiplayer game. Each player has a GameClient. The host has GameServer.In fact this approach extends to single-player games too! Instead of writing a whole special single player mode, there can just be a local GameServer on the same system, and it will run a game for one player and sync perfectly with GameClient (essentially just the "Player 1" box in the diagram above). I'd like to have a single-player mode for this project too if possible, and that can definitely be done. It's an interesting design point that going from a single-player design to a multiplayer design is incredibly hard - often infeasible - but going from a multiplayer design to a single-player design can actually be pretty straightforward. So thinking about the multiplayer design is still the right place to start.
Another benefit of single-player mode working the same as multiplayer is I can simulate latency and packet loss on the messages passed between GameClient and GameServer in single-player mode. This means the resilience of syncing over a poor network can also be tested in single-player mode without actually using networking, which should make testing the multiplayer code easier.
Multithreading
A cool part of this design is we can also use multithreading. In short, GameServer can go in a Web Worker.
Not all game development tools support multithreading, but it's another thing JavaScript has had for years in the form of workers. A worker provides an independent JavaScript context that runs in parallel to everything else. The system can schedule it to run on its own CPU core. It can safely communicate with other JavaScript contexts via message passing (using postMessage()) without any risk of nightmare shared memory concurrency bugs that some "unsafe" languages are subject to.
GameServer will be designed to communicate with GameClients over the network, and this also makes it easy to use a worker: instead of sending messages over a network, it sends messages to and from a worker. That's like a perfect network link with zero latency and unlimited bandwidth.
GameServer may well be running very heavy amounts of game logic to simulate hundreds or even thousands of independent units all interacting in different ways. Using a multithreaded architecture moves all this work to its own thread so it won't affect the performance of the game for the local player, as GameClient is in a different thread and so won't be slowed down even if GameServer is using a full CPU core. That's pretty cool and potentially significantly increases the upper limit of how intensive a game it can run, especially since JavaScript has outstanding performance. It will be interesting to see how far this can be pushed. I will definitely be trying some stress tests later on!
Construct can host its own runtime in a Web Worker, off the main browser thread (aka the DOM). Adding a second worker for GameServer means there are now three threads on the host system: one for the browser, one for the Construct runtime, and one for GameServer. I think this is a relatively sophisticated multithreaded architecture for a hand-coded browser game.
Diagram of three threads on the host system, and which communicate with each other.All technology choices are a trade-off though, and there are some downsides. The main one is that by running in an entirely separate JavaScript context, there is no direct access to the Construct runtime. That means it can't rely on Construct for things like collision detection - I will have to code everything in GameServer in pure JavaScript. I don't think there will be too much of that kind of re-implementation though as GameServer doesn't have to handle anything like rendering or user interaction. I guess it might be an interesting exercise in game coding.
Funnily enough that disadvantage also turns in to a potential advantage: if GameServer is entirely self-contained, then it could even be run independently with a tool like Node.js or Deno. That provides a way to host a dedicated server in future, which would solve the problem of the game ending if the host quits, but means somebody needs to pay to run those servers. This also highlights a great strength of JavaScript: it's supported so widely that the same code can be re-used directly in different places such as both servers and clients, rather than needing to rewrite things in another language entirely.
Summary
So overall I would say the advantages of this approach are:
- It covers both single-player and multiplayer games without needing separate modes for each
- The same messages can be used for both network messages for multiplayer games and worker messages to talk to a local GameServer
- Multiplayer code can be tested in single-player mode without actually using networking, by simulating latency and packet loss
- It should work for both peer-to-peer and dedicated server network architectures, although it'll just be peer-to-peer to start with, as that's cheaper to run.
- Multithreading avoids janking the host player's game and could significantly enhance overall performance
The disadvantages are:
- GameServer will have to be coded fully independently, without using Construct for things like collision detection.
- Overall this is a fairly complicated design. But hey, I signed up for a challenge.
In software, as in most engineering things, there's probably not a perfect solution that ticks all boxes - everything involves tradeoffs. But I like this set of tradeoffs and I think it will work well for this project.
Why not use event blocks?
One of the main questions people asked after my first blog post was basically: why not do this with Construct's visual event blocks? I talked a bit about the goals of the project in the first blog post, but it's worth going in to a bit more detail about them.
- First of all, it's not possible to use event sheets with multithreading. So it's impossible to do the architecture I just outlined here with just event sheets! I wrote a blog post back in 2015 on Why do events only run on one core? which addresses some of the technical complications.
- Ever since the introduction of the JavaScript coding feature, Construct has been about both event sheets and coding. We've long been emphasizing that if you do want to learn to program, Construct is a great way to do that: you can start with event sheets, add a few snippets of code in event sheets, and move on to using full coding - and you'll learn a professional programming language along the way. I think the coding part of Construct is under-utilized and is strongly competitive with other more coding-focused tools. I want to help shine a light on the coding side of Construct and prove the point it can be used like a coding-focused tool as well.
- Related to that is dispelling the myth that Construct is "just a toy". Building a sophisticated fully-coded game that can do things impossible in some other coding-focused tools on the market should help prove that point too. It'll help demonstrate that you don't have to move on to a different tool if you want to start writing serious code.
- This project is about pushing Construct to its limits, answering the question: what's the most an advanced user can do with Construct? As we would expect the most advanced users to shift from event sheets to coding, developing a full coding project is the best way to answer that question.
- This project will likely also need a focus on extreme performance to be able to handle potentially thousands of independently interacting units. While event sheets are still highly efficient and can handle many games superbly, when you start to reach this kind of scale, directly coding in a high-performance language like JavaScript helps to go even further in that respect. It also demonstrates what the incredible performance of JavaScript can do beyond the performance limits of other much slower custom languages used in some other tools.
- Event sheets can integrate with JavaScript, but it would probably be difficult to do that to a large extent as event blocks and JavaScript code are completely different paradigms (for example, the concept of "picking" does not exist in JavaScript). I suspect trying to half-and-half lots of complex code and event sheets could actually make things more complicated than they already are. Using event sheets too much could also put off coders interested in following a coding project, and they're a group I'd like to appeal to with this project.
Most of these goals can't be achieved by focusing on event sheets. That's why I'm going for mostly coding for this project. But who knows - if this works out well, maybe there could be a similar project in future focusing on the event system instead.
It's also worth adding:
- There is much more to Construct than the event system, and so there's still a lot of value in dogfooding. In fact since starting I've already fixed two small bugs in the Layout View and Project Bar and made two adjustments to the scripting feature. So it is definitely still helping improve Construct for everyone, including both visual and coding users, along the way.
- We built many of Construct's 280+ built-in example projects, most of which focus on event sheets. That combined with a lot of testing and prototyping means we have used, and do use, the event sheet system a fair bit already. We've already made lots of improvements from that kind of usage and still occasionally do. On the other hand scripting is something we've spent less time using ourselves.
- Despite all the above, I think this project will likely use event sheets to some extent anyway. In particular menu systems are pretty boring to code and work independently of the main game, so I'll probably just use event sheets for those as they really are the best tool for the job. Maybe there are other areas, such as the game user interface or particular interactions, that will be easy to do with event sheets. I don't know yet though! However I'm sure one way or another this project will involve use of event sheets too.
More to come
I hope that helps explain the goals of this project. It's early days yet and there's a lot more to come. I'm aiming to get to something at least minimally playable as quickly as possible - so far there's been a lot of just laying foundations to build on top of, so there's not much to see in action just yet. But you can see all the code so far on the GitHub repository and I'll keep on blogging about it here! Let's see how long it takes to get a multiplayer game where two people can just move their own units around and see everything updating. Then we'll start building gameplay from there.