Hopefully projects like Construct 3 are exactly what browser makers like Google are hoping to see: ambitious Progressive Web Apps (PWAs) that are fully-featured cross-platform replacements for their native counterparts. We've had great feedback on it too, with some users even saying they forget they're in a browser and not a native app. Ports like this still appear to be uncommon as the web platform is still developing and maturing the necessary APIs. Few other companies are willing to take on such a vast and potentially risky project, or if they do, they appear to downplay their PWA as a "Lite" version. We make no such apology; our PWA is the real deal. Therefore I think that our web development experience is particularly notable, especially since our approach to Web Components is pretty much the reverse of what everyone else does from what I gather reading around the web.
Web Components in an ambitious PWA
Web Components are made up of four key technologies: Custom Elements, HTML Templates, Shadow DOM and HTML Imports. Across thousands of lines of code, style and markup, Construct 3 makes minimal use of the first three. They just don't seem to be particularly important to this kind of app with our development approach. However Construct 3 is entirely architected around HTML Imports. They're the glue that holds the whole app together; we use around 300 separate HTML imports. Construct 3 is a huge app and imports work beautifully to break it down in to manageable components that are genuinely a joy to work with.
Guess which Web Component Google wants to remove?
That's right! HTML imports are down for possible removal, or later replacement by something different. We'll be fine — we've successfully polyfilled HTML imports, so even if they're removed, Construct 3 will continue on regardless. But like most polyfills it has some pitfalls, particularly for performance. I'm stunned that such a key part of the future web platform is being disregarded, but I can sort of understand how it's come to this. So part of the reason I'm writing this post is to try and push back against that.
What's so great about HTML imports?
The best demonstration of how beautifully HTML imports can work is when defining a dialog. Construct 3 has over 50 dialogs ranging from a simple OK dialog to a fully-featured image and animations editor implemented in a <dialog>. So this alone is a significant part of the product covering a whole spectrum of features. Let's look at our typical approach to developing a new dialog. We:
- Create a new import for it
- Write the dialog markup in the HTML
- Add dialog-specific styles to its own CSS file, and link to that from the import
Here's a simplified version of what the import looks like:
<!-- Dialog styles -->
<link rel="stylesheet" href="settingsDialog.css">
<!-- Dialog markup -->
<input id="option1" type="checkbox">
<input id="option2" type="checkbox">
<!-- Dialog logic -->
This allows us to define a new piece of functionality in our web app in a simple, modular and isolated way. It even allows a sort of poor-man's scoping for CSS — just prefix every selector with #settingsDialog and you know it will only apply within that dialog. We also have a mini dialog framework that handles moving the <dialog> element to the main document, displaying it, running transitions, handling OK/Cancel and so on.
This approach is rolled out over the entire app. It's so effective, we use it everywhere. Each separate pane in the main view has its own import. The main menu has its own import. The account component lives in an import. Each kind of object in the game development IDE (which we call a Plugin) is defined in its own import. It covers everything, because it's so simple, intuitive and effective.
What about other Web Components?
We make minimal use of all other components. Perhaps it might make our code a bit cleaner in some cases, or be more academically correct/modular, but we get by easily enough. Here's a quick run-down of what else we use:
- HTML Templates: these are handy for stamping out a chunk of DOM repeatedly. I counted, and we use the template element exactly four times in our entire app. So a nice thing to have around, but hardly critical infrastructure in our case.
- Shadow DOM: we don't use this at all. I'd guess this is much more applicable if you actually use Custom Elements, or if you're developing isolated components intended for third-party use in a library. We developed our own UI library because none existed that did what we needed (open Construct 3 and you should see what I mean), so this again this kind of isolation isn't particularly important since it's generally all our own code and markup. If we rewrote our whole UI library, I might experiment with this though.
Part of developing Construct 3 involved designing our own comprehensive UI library. This covers a windowing system (of which dialogs are a subset), tabs, toolbars, icons, menus, notifications and tips, tree controls, icon view controls, table controls, property grids and more. Custom Elements and Shadow DOM could potentially make these more modular, but we've come this far and it's worked fine, so these do not seem to be critical components for a large web app. That contrasts with HTML Imports, which are fundamental to the overall architecture.
I must add that I don't at all assume that everyone develops web apps like us. I am sure that for other kinds of app, these other web components will be critical. That's fine, and this is not meant to dismiss these as unnecessary technologies. My point is to emphasise that in at least some cases, HTML imports are by far the most important of the set.
What went wrong for HTML imports?
Basically, I think HTML imports are ahead of their time.
We started development of Construct 3 about three years ago. Before that we actually made a prototype even further back, using the traditional block layout, used jQuery, and so on. The prototype was so obviously going to be a huge mess with that approach that we decided we'd have to bet on all the modern web platform features for it to be feasible. Construct 3 uses to name but a few: CSS Grid, CSS variables (aka custom properties), CSS Containment (also critical for layout performance), the Dialog element, Service Worker, WebGL and WebGL 2, Web Animations, and more. Notice many of these features only recently became available even in just Chrome. In other words, web apps on the scale of Construct 3 have only just become feasible to release. HTML Imports were first released in Chrome 36 in mid-2014. That was just too soon for many web apps of Construct 3's scale to be around.
Finally, as outlined above, I think one of the best use cases for HTML imports is with dialogs. However the <dialog> element itself is still currently only supported in Chrome. So this feature which is a particularly compelling case for imports has no cross-browser support either. This probably also reduces the perceived utility of imports.
However it's now been so long, the Chrome team (and some other browser makers) appear to be taking the view that it's a failed feature. I hope this blog post helps counter that perception, and demonstrate the real utility of the feature.
Why not polyfill it?
We can, and do. However the main reason is performance. Browsers are good at parsing HTML and can start resource fetches ahead of the time they're actually needed. Consider this case:
<link rel="import" href="sub-import.html">
Ideally we can fetch and even start parsing and pre-compiling script.js while sub-import is still fetching. This ensures maximum performance since as soon as sub-import.html finishes parsing, we can immediately execute a fully parsed copy of script.js. This feature is very difficult to polyfill. The script can be pre-fetched as text, but it has to use ugly blob URLs and still isn't parsed or compiled until it is added to the DOM. Experiments with link preload tags made things slightly worse, not better.
It seems only the browser has the power to control the precise scheduling of fetch, parse and execute for scripts. Our polyfill ends up having to wait until sub-import finishes loading in its entirety before even requesting script.js.
What other options are there?
I've heard of HTML Modules as a possible replacement for HTML Imports, but I can't find much information on what's different about them or how they'd work for a web developer. I do also worry a bit that it's just an exercise in tweaking and renaming it in order to present other browser vendors with something new to consider. In my view, HTML imports in their original form are already very effective.
import doc from "./import.html"
This would be the same as fetching "import.html" as type "document", and assigning the resulting Document to
doc. Now you can access the import document from the script, and perform typical calls like
doc.getElementById(...). This actually looks to me like a pretty elegant way to pull in DOM content to a module script. It's also straightforward to statically analyse.
Going one step further though, we can actually re-define
<link rel="import"> in terms of JS modules. We could say that this:
<link rel="import" href="import.html">
import doc from "./import.html"
In this case 'doc' is not used, but it causes the fetch and propagates through the import's own dependencies. Now your dependencies can also propagate through HTML — but it's entirely based on the JS module system. That means only one module system is used, only one set of deduplication applies, and so on. Hopefully this is an idea worth consideration from browser vendors.
You can also find our polyfill for HTML Imports on GitHub, which is robust enough to work with Construct 3.