This blog post is licensed Creative Common Attribution Required v4.0.
We've done two big performance tests in the past: once in 2012 and again in 2014. Given it's been about another two years and technology is moving as fast as ever, it's time to take another look. As before, we've gone through most of the devices we have available for testing (skipping our growing pile of devices which are either defunct or eventually died), and run our standard performance test (updated to use the latest Construct 2 runtime). We've also tested some of the newer publishing options like Cordova and Crosswalk for comparison.
We're focusing on mobile devices because that's the main concern for performance. Mobile devices tend to be weaker (although this is changing!), and in the past have had less mature software (also less of an issue now). Additionally, HTML5 game performance on desktop basically hasn't been a problem for a few years now. The generally more capable hardware combined with the maturity of desktop software meant that even in our 2014 test all our desktop systems basically got top scores. By now it would be surprising to find a desktop-grade device that didn't get a top-score of 60 FPS on our test. I can't remember seeing any complaints from users about performance in a desktop browser in recent times, so we're not even bothering to test desktops this time around. This in itself is a very positive sign of the maturity of HTML5.
As before, our favourite test is the venerable sbperftest. It's based on a real-world game using our engine making it a realistic test, and it's automated so user input cannot vary the results. It measures the frames-per-second (FPS) once a second and the average FPS over the test duration of about one minute is the resulting score. There are some extra measurements for interest relating to jank and CPU utilisation, but these do not affect the score. It's also a relatively demanding test: at some points there are quite a lot of sprites on-screen at once, and a lot of blending explosion effects. As a result we find it is quite a good indicator of a platform's performance - if sbperftest gets a good score, many other Construct 2 games should perform well too.
So, after a day of running the test on lots of devices, here are our results!
All of the devices performed well - even old devices such as the 5-year-old iPad 2 and the 4-year-old Nexus 7. The only wart was I was surprised to find the Nexus 9 doing pretty badly with the system webview. None of the other Android devices had trouble with their system webviews, so it seems to be a device-specific issue. I filed bug 594154 so hopefully Google can look in to that. The great thing about the Android system webview is it now auto-updates from the Google Play store, updating regularly like Chrome does. So Google can fix this and have the fix in a later webview update. Gone are the days of waiting for an OS update to update the webview! (At least on newer Androids, anyway.)
I think the great thing about these results (ignoring the Nexus 9 issue) is the consistency. The results are good across a range of different operating systems, browser engines and wrappers. In 2012 many devices failed to reach 30 FPS, in 2014 it was a mixed bag, but now the results are mostly in the 55-60 FPS range. Many of the devices that scored 58 or 59 could in fact generally hit 60 FPS, but timer accuracy, the odd v-sync skip or a one-off jank bumped down the score a little. Whether games are playable is no longer in question; it's only a matter of how smooth it is.
The recent generation of devices are already practically perfect. Even the iPad Pro with its gigantic screen aced the test with a steady 60 FPS. It appears around 2014 most hardware reached the point where performance was perfectly sufficient. Devices coming out more recently rival the specs of some laptops, so there is starting to be overlap between mobile and desktop performance. This will only get even better. In future, there will be hardly any reason to consider mobile devices as weaker. They are already closing in on desktop-grade performance!
We are definitely happy to see the end of the non-browser wrappers like directCanvas and Ejecta. These proved very difficult to support and were always missing some major features, and we deprecated them all last year. Now we are pleased to see Cordova with real browser engines actually exceeding the performance of the non-browser engines. A relatively limited directCanvas could score 54 FPS on an iPad 3 in 2012; Cordova with WKWebView now exceeds that with a feature-complete browser engine. Again, Ejecta even with WebGL support scored 51 FPS on an iPhone 4S in 2014; now with Cordova, 56 FPS. So we have reached the point where the non-browser wrappers - which were specifically designed to be faster than webviews - are actually slower than a webview! We are glad to see we made the right move to deprecate the limited non-browser engines.
I don't know if it would be too controversial to call HTML5 performance on mobile a solved problem now. There is still a large but shrinking set of old Android devices, and GPU blacklisting remains an increasingly uncommon but annoying problem. However I don't think it's controversial to say that HTML5 performance on mobile will soon inevitably be a solved problem, just like it is on desktop. With mature software and hardware close to desktop performance, mobile is no longer missing anything. Modern devices are already great and as ever the technology will only continue to improve even further.
I think this proves that HTML5 is a great technology for games. It's a consistently fast and mature cross-platform technology, even when wrapped as a native app. The standards compliance of modern browsers is excellent, too. Construct 2's engine is mature as well, having been one of the earliest-released HTML5 engines in 2011, and now with five years of upgrades and maintenance is probably one of the fastest and most reliable HTML5 engines on the market. It's great to see this carrying over to native apps on both desktop and mobile. We're intrigued to see how things will keep advancing in future!
Get emailed when there are new posts!