0 Favourites

Memory Management with CocoonJS

  • Memory Management with CocoonJS

    We have created a general blogpost about memory management that we highly recommend you to read (http://blog.ludei.com/techniques-to-optimize-memory-use-in-ludeis-canvas-environment/). However, we want to clarify some Construct2 specific topics regarding memory management and our relationship with Scirra in this post.

    First of all, we understand that CocoonJS Canvas+ might not have worked as expected in some situations. You need to understand that the Canvas+ environment is a homemade subset of a browser created from scratch, which is a big challenge for a startup company. We dared to create a multiplatform (iOS and Android) canvas 2D and webgl implementation and compete in performance against browsers/webviews like safari and chrome, which have thousands of contributors including companies like Apple and Google with unlimited resources and money. CocoonJS' Canvas+ environment have been with no doubt the best performing cross-platform HTML5 execution environment for years and we are really proud of it.

    It’s true that Canvas+ is not and will never be a complete web browser and that it does not render any DOM elements apart from canvas, but a complete browser was never our goal. Canvas+ was the first and is still the only platform to date that allows WebGL native app deployment in both iOS 5.0+ Android 2.3+. It is still the only platform to date compatible with wearable devices (we have talked with Chrome WebGL engineers about Canvas+ on Android Wear, and they are amazed about its results on those devices, soon to be announced!). It also supports some extensions like accelerated box2d implementation, multiplayer, social and others that are not supported by other HTML5 gaming wrappers. More than 4000 games have been published using Canvas+, and big companies like Disney, Nickelodeon and others have used Canvas+ to accelerate and publish their games. And if your game doesn’t fit into the Canvas+ runtime, remember that CocoonJS has evolved and it now supports other runtimes like the system Webview and Webview+ (for both iOS using the WKWebView and Android using Chromium), which are full browsers and your game will work in a plug and play way.

    Ludei has been providing great tools and a cloud compiler for HTML5 developers for the last couple of years. And it has been a totally free product. Construct2 requires a paid licence to export to CocoonJS but we have never got one cent of the money from them nor have we asked for it. Other users using engines like Phaser or Pixi have successfully published their games for free using CocoonJS. On the other hand, we have benefited from many new users and published a lot of games thanks to Construct2 and we are glad about this mutual benefit and we hope that both Construct2 and CocoonJS will keep growing and becoming more successful. We recognize that the user support and the documentation have not been the best, but we are making an effort to improve it. We have released support for new runtimes, new plugin APIs and new documentation. As stated before we are a startup working on a technologically complex project competing against companies with unlimited resources and giving a free product and support, so please try to appreciate the challenges we face and know that we are doing the best we can with the resources we have.

    That being said, it’s time to clarify the memory management issues. Ashley and some users have blamed us for loading all the assets at startup, for the lack of memory management support in CocoonJS and even for ignoring cooperation with Scirra. It’s true that Ashley and us had different opinions on the matter and we want to publicly explain our stance.

    First of all, it’s false that Canvas+ lacks memory management. As explained in the above mentioned blogpost, we offer many APIs which give developers even more precise memory control than standard browsers (like the immediate "dispose" method). We might not have a perfect implementation but big companies like Nickelodeon, Disney, Non Stop Games (acquired by King.com) and others have published resource demanding games using a lot of high resolution images and audio files and are really happy to have these memory management features. For example the Nickelodeon app allows for loading more than 50 games in the same app execution, and if you check the app using a profiler, CocoonJS keeps the memory usage stable. Some heavy games even work on devices like the iPad 1, which doesn’t support more than two 2048x2048 textures at the same time in memory. This would be impossible if CocoonJS lacked memory management support.

    The main issue between the CocoonJS Canvas+ and Construct2 marriage has been the assets loading at startup issue. We have explained in the blogpost that for a game engine implementation loading everything at start, relying on the browser/garbage-collector internal implementation (which may be different in each browser or JavaScript virtual machine) and forgetting about effective memory management is the easiest implementation possible. It works for small games that fit into the available memory but it’s not the best option for heavier games.

    Construct2 loads all the assets at startup, and even though this approach may seem to work on desktop browsers due to unlimited memory and browser magic tricks, CocoonJS Canvas+ is not responsible for making the initial image load decisions. Canvas+ just loads an image when the "src" property is set from JavaScript, which is the responsibility of the game engine. The paramount disagreement between Ashley and CocoonJS is about how browsers handle image loading. He states that a browser doesn’t load the image into GPU memory when the "src" property is set but only when it’s rendered for the first time, and Construct2 relies on that expected behaviour to handle the memory usage.

    Clearly we have different philosophical views on the matter. The CocoonJS core team has a proven track record developing successful videogames with millions of downloads (check Slide Soccer, iBasket, etc) and they have proven strong C++/OpenGL skills. The entire team agrees that preloaded OpenGL textures make HTML5 games feel more native-like. Reading an image from disk and uploading a texture to the GPU is an expensive operation, which may lead to noticeable glitches or pauses if the process is carried out in the middle of a render animation. That’s why we are going to keep this approach as the default in Canvas+. Do you prefer the other approach? We have added the "cocoonLazyLoad" property to enable it. Just use the one that works better for you.

    Finally, we have performed some tests with a heavy Construct2 project provided by ArcadEd (thanks!). We are going to share a comparison between Canvas+ and the system WebView on iOS to demonstrate that the blame for the lack of memory management on Canvas+ are unfounded. All the tests have been done using CocoonJS Launcher v2.1, iPhone 5 using iOS 8.0.2 and XCode profiler (which measures memory usage in a reliable way).

    Test using WebGL Context

    The test is performed using default settings in both Canvas+ and Webview.

    Surprise, Construct2 running with WebGL eats almost twice the memory on WebView than on Canvas+. Good thing that Canvas+ has no support for memory management!

    Test using Canvas2D context (WebGL disabled in the code)

    As explained in the blogpost CocoonJS uses POT textures by default in Canvas2D contexts for performance reasons, while system WebView uses NPOT textures. The Construct2 engine or the provided test .capx doesn’t pack textures, so to make the comparison reliable we have enabled NPOT textures calling to the the "CocoonJS.Util.setNPOTAllowed(true)" method.

    The memory peak is similar in both environments. It seems that Images are not properly released specially on Canvas+ even if the capx uses layout by layout. Inspecting the code with Google Chrome profiler we have found that Image references are strongly retained in the Construct2 engine and are not disposed of (when using canvas2D context). This is bad for the garbage collector. Here is an image of the google profiler, which detects 52 retained image objects:

    Let’s enable the new Canvas+ "setMaxMemory" feature and see the results again:

    With "setMaxMemory" set to around 100MB Canvas+ uses a lot less memory than the WebView!

    Next Steps

    We have released CocoonJS 2.1. Running a Construct2 game with WebGL mode has a good memory usage using default settings.

    Running a Construct2 game with WebGL disabled (using Canvas2D) has the retained images problem. We have released the demanded "cocoonLazyLoad" feature, but the retained images on Construct2 are never released. So the problem is not fixed, it just happens later. We recommend that Ashley adds the "image.dispose && image.dispose();" method call in the layout by layout code. With this minimal change, the memory improvements would be amazing. Otherwise you can use the "setMaxMemory" utility.

    We have added this code to CocoonJS Construct2 plugin initialization:

    CocoonJS.App.setNPOTEnabled(false); //change to true to reduce memory usage. It might reduce performance on old GPUs.
    //CocoonJS.App.setMaxMemory(70); //Uncomment this line to set the maxMemory threshold in Megabytes.[/code:33qxzywh]
    
    You can set the values that work better for your game.
    
    [h2][b]Conclusion[/b][/h2]
    
    We have shown that Canvas+ is able to manage memory efficiently with a heavy capx. It even has a better memory usage than the system webview in some scenarios with both webgl and canvas2d contexts. We have improved and added new tools in CocoonJS 2.1, and we’ll keep improving it. We hope that this post clarifies the misunderstandings about the lack of memory management support in Canvas+. As we have improved CocoonJS we also think that Ashley can still improve some things in the Construct2 engine that could benefit not only Canvas+ but any other platform/runtime. Things like avoiding retained images, disposing them on Canvas 2D, packaging textures or using texture compression on WebGL would be very helpful for everyone.
    
    We are always open to your suggestions. Remember that if you have any problem with memory usage in Canvas+ and you think that the problem is on our side you can send us a testcase and we’ll be glad to help you.
  • Please Ashley

    I will need to read all of this a few times to make sure I understand all the options before updating any of my games. Thank you ludei for taking the time to explain it.

  • Construct 3

    Buy Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Buy Now Construct 3 users don't see these ads
  • Thanks for the info ludei, very interesting results!

  • ludei

    So if I compile with 2.1 on the cloud compiler, what is the default action? When I tried last night I noticed my loading layout was gone, but there was longing loading times between layouts. If I wanted it how it was before 2.1, how is that achieved?

    Secondly, and this might be a question for ashley, but will there be a way to show a loading progress from one layout to the next. For example in the demo capx I gave you, it takes a few seconds when you touch the screen for the next layout to load. Is there a way to let the user know it's loading?

  • Hello.

    So if I compile with 2.1 on the cloud compiler, what is the default action? When I tried last night I noticed my loading layout was gone, but there was longing loading times between layouts.

    Please, send us a testcase and we will have a look at it. We couldn't reproduce that issue here.

    If I wanted it how it was before 2.1, how is that achieved?

    Not sure what you mean. You can compile it using the 2.0.2. That option is still available.

    Regards.

  • We recognize that the user support and the documentation have not been the best, but we are making an effort to improve it.

    i personally very much like that new approach, its cool how many good informations you post in the last couple of weeks!

  • ludei working on adding a loading screen to the memory demo I sent earlier. Just wanna test it with both compilers first.

    I'll email it to support and post here when I do.

  • ludei I just sent the latest memory demo with a loading layout to your support email.

  • I can confirm you we received it . Thanks a lot.

  • As explained in the above mentioned blogpost, we offer many APIs which give developers even more precise memory control than standard browsers (like the immediate "dispose" method).

    The "dispose" method does not solve the problem by itself. By the time you can call it, the image is already in memory, which means it might already have crashed, and it still unnecessarily loaded it which increases the loading time.

    [quote:l5tfqw08]Construct2 loads all the assets at startup

    No, it only downloads them. It's a web based engine after all and the game needs to be downloaded before starting. CocoonJS is unique in going ahead and decompressing everything in to memory when we only wanted to download. In a local app, since there is nothing to download, it should simply amount to "remember the path".

    [quote:l5tfqw08]Canvas+ just loads an image when the "src" property is set from JavaScript

    This is precisely the problem: in browsers that is the directive to download a resource.

    [quote:l5tfqw08]The entire team agrees that preloaded OpenGL textures make HTML5 games feel more native-like. Reading an image from disk and uploading a texture to the GPU is an expensive operation, which may lead to noticeable glitches or pauses if the process is carried out in the middle of a render animation.

    The key problem is all browsers disagree with you, even mobile browsers. Anyone designing any web-based engine must deal with the fact that all browsers will have precisely that problem with lazy-loading causing jank mid-game. Therefore, anyone with interest in designing an engine that works well on real browsers will have already mitigated that problem. Construct 2 has done exactly this, by some pre-rendering code before start of layout in canvas2d mode, and texture creation in WebGL mode. This brings a smooth, native-like feel to real browsers even though they lazy-load resources. By choosing a different approach, CocoonJS creates two problems:

    1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers

    2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.

    So I'd have thought it would actually be favourable to CocoonJS to copy what the browsers do and provide advice on how to work around lazy loading in a portable, cross-platform manner. This solves both problems.

    I think the key problem is that you perceive this change as some kind of performance improvement, whereas really it throws a spanner in the works of any engine that's already designed for real browsers, and will trip up anyone porting their game either to or from CocoonJS.

    [quote:l5tfqw08]That’s why we are going to keep this approach as the default in Canvas+. Do you prefer the other approach? We have added the "cocoonLazyLoad" property to enable it. Just use the one that works better for you.

    At last! I'll add that in as the default for images the next build. But as argued above, I feel this ought to be the default!

    [quote:l5tfqw08]Surprise, Construct2 running with WebGL eats almost twice the memory on WebView than on Canvas+.

    I think all this proves is the WebView has a full browser engine (including features like DOM, web audio API, WebRTC etc), and a full browser engine takes up a bit more memory.

    [quote:l5tfqw08]It seems that Images are not properly released specially on Canvas+ even if the capx uses layout by layout. Inspecting the code with Google Chrome profiler we have found that Image references are strongly retained in the Construct2 engine and are not disposed of (when using canvas2D context). This is bad for the garbage collector.

    It shouldn't matter, because in real browsers holding a Javascript reference to an Image object does not mean the Image is decompressed in memory. We can't drop the reference, because we might need it again if the player comes back to that layout later. If we drop references and re-create them we might invoke downloads in real browsers, wasting bandwidth and causing delays, and a separate codepath also defeats the point of having a portable engine that runs the same everywhere. To me your graph simply identifies the canvas+ bug.

    I'd also point out that it's sometimes difficult to make good sense of garbage-collected memory use graphs, since they can spike high and appear to be using lots of memory, but the next collection is able to release it all. That's an essential point: it is more important that it can release memory when it's high, than how much the highest memory use actually is. This allows the game to continue running as opposed to crash, as evidenced by the larger games which tend to crash when ported to Canvas+ when they run fine in a real browser (even with higher memory use!). The 2nd graph also confirms this: despite peaking higher, the web view finishes with lower memory use than canvas+. So I don't consider the peak memory usage particularly relevant, a more important question is "does it crash?"

  • Ashley

    [quote:1yosbz1e]1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers

    2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.

    1. But isn't Canvas+ not a real browser? Shouldn't the CJS exporter simply work for CJS and not worry about anything else?

    2. I don't understand why anyone would do this. In our C2 world anyway.

    Most of this stuff is way over my head, I am just trying to make sense of it as a user of both C2 and CJS.

    I really feel like we are getting somewhere and the bridge of communication has been reopened. I hope some kind of compromise is met, because CJS just works and the last thing I want is for it to break . Right now, I prefer the way things were done in 2.0.2, but only because there seems to an issue when loading a large layout, not having a way to tell the user it's loading. Once in game, my game did run better in 2.1

  • ArcadEd

    "1. But isn't Canvas+ not a real browser? Shouldn't the CJS exporter simply work for CJS and not worry about anything else?"

    I think the issue is that maintaining two codebase where one should be sufficient is the problem for such basic functionnalities as loading a image into memory vs downloading it, also, CocoonJS is supposed to work with the html5 export too, but at least indeed the communication is up, so anything good can happen, and no more break hopefully.

  • Aphrodite

    I gotcha, I just didn't know if it was something that could be exported/compiled differently on export that would not take a lot of man power to do. It's probably something that is a huge undertaking, but I was just curious.

    I just want harmony .

  • [quote:13xpov47]The "dispose" method does not solve the problem by itself. By the time you can call it, the image is already in memory, which means it might already have crashed, and it still unnecessarily loaded it which increases the loading time.

    The dispose method is just an extension to provide a more precise control over the memory usage. Canvas+ also disposes unretained images when the Garbage Collector comes into action. On a WebGL context you can release a texture immediately, but you can’t do that on a Canvas2D context, and that’s a desirable feature. By not using dispose method on Canvas+ or cleaning up retained references on a real browser you are not giving a hint to the browser and you’re totally relying on the internal memory management heuristics. Immediate dispose or cleaning references is always better than letting the browser guess when it should dispose images.

    Thanks to the dispose method some resource demanding games that crash (white screen) on a real webview were able to work on devices like the iPad 1.

    [quote:13xpov47]This is precisely the problem: in browsers that is the directive to download a resource.

    The key problem is all browsers disagree with you, even mobile browsers. Anyone designing any web-based engine must deal with the fact that all browsers will have precisely that problem with lazy-loading causing jank mid-game. Therefore, anyone with interest in designing an engine that works well on real browsers will have already mitigated that problem. Construct 2 has done exactly this, by some pre-rendering code before start of layout in canvas2d mode, and texture creation in WebGL mode. This brings a smooth, native-like feel to real browsers even though they lazy-load resources. By choosing a different approach, CocoonJS creates two problems:

    1) our approach, designed for real browsers, doesn't work, nor will any other engine which has similarly mitigated the problem for real browsers

    2) anyone designing a game for CocoonJS, then porting it to a real browser, will find it suddenly suffers from mid-game jank due to lazy loading resources the first time they are used.

    So I'd have thought it would actually be favourable to CocoonJS to copy what the browsers do and provide advice on how to work around lazy loading in a portable, cross-platform manner. This solves both problems.

    Copying what a mobile browsers does was not our goal because they were not initially designed for gaming. That’s why Canvas+ came to life. Before the latest Android 4.4 or iOS 7.0+ webviews, a native-feel like game using a mobile webview was almost impossible, even using the tricks you are talking about. However, the native-feel like experience was easily achieved with Canvas+, which was specifically designed for gaming.

    You might need the pre-rendering code to mitigate the mid-game janks of real browsers but that pre-rendering code just works on Canvas+. A blitting operation with preloaded textures is negligible in Canvas+, you don’t need to maintain two code bases.

    The good news is that from CocoonJS 2.1 we support both approaches: preloaded textures or lazy loading. So any engine or user can choose the approach that works better for him. The real browser vs CocoonJS design issues that you are talking about are not a problem anymore.

    [quote:13xpov47]At last! I'll add that in as the default for images the next build. But as argued above, I feel this ought to be the default!

    It’s already the default on C2 because we have added a duplicate called idtkLoadDisposed for retro compatibility with the current C2 version. But we like the cocoonLazyLoad name better. If you use the new one we will remove the other on a future release.

    We would like to add a lazyLoad check into the CocoonJS C2 Plugin. Some users might want the old behaviour because C2 seems to have a problem showing progress bars when lazyLoading is enabled.

    [quote:13xpov47]I think all this proves is the WebView has a full browser engine (including features like DOM, web audio API, WebRTC etc), and a full browser engine takes up a bit more memory.

    It also proves that CocoonJS doesn’t deserve to be blamed for lack of memory management.

    [quote:13xpov47]It shouldn't matter, because in real browsers holding a Javascript reference to an Image object does not mean the Image is decompressed in memory. We can't drop the reference, because we might need it again if the player comes back to that layout later. If we drop references and re-create them we might invoke downloads in real browsers, wasting bandwidth and causing delays, and a separate codepath also defeats the point of having a portable engine that runs the same everywhere. To me your graph simply identifies the canvas+ bug.

    It matters. We have created a testcase to show the difference on a real browser. You can drop image references. If you need them again just create a new Image object. A modern browser has advanced caching mechanisms, creates temporary files and is intelligent enough to avoid unneeded network downloads. The problem you are talking about is more negligible in mobile apps indeed. As all the assets are available locally, the browser doesn’t need to download anything from the network.

    We have created a simple test to prove that retained images matter. You can download it from here: https://dl.dropboxusercontent.com/u/20744811/retain_testcase.zip

    The test loads five 2048x2048 images, renders them during some seconds and then renders a red quad. In the first test we drop the images array, in the other one we don’t. The next graph shows the memory difference. Tested on the latest Safari Webview on a iPhone 5 running iOS 8.1:

    In the first graph memory usage decreases when garbage collector comes into action, it takes some seconds to happen. In the second graph memory usage never decreases, because the browser can’t guess that the images will no longer be rendered and the garbage collector finds retained references. If the webview had the dispose method as Canvas+ does we could have decreased the memory amount immediately, without waiting for the seconds needed to the next garbage collector pass.

    The conclusion (for Canvas2D contexts) is that dispose extension is the most effective way to decrease memory usage immediately. Dropping image references works at the expense of waiting for the next GC pass. Finally, keeping retained image references causes unneeded high memory usage until the browser enters in a dire situation and starts wiping out everything it can (similar behaviour can be achieved with the optional maxMemory feature in Canvas+)

    Dropping image references in C2 might involve a bit of work but we really think that it would be a good improvement for both real browsers and CocoonJS. It might also help with the asset loading progress bar issue that some users are talking about. Anyway, the quickest solution is just to call dispose method if available when a layout changes (only one line of code).

    [quote:13xpov47]I'd also point out that it's sometimes difficult to make good sense of garbage-collected memory use graphs, since they can spike high and appear to be using lots of memory, but the next collection is able to release it all. That's an essential point: it is more important that it can release memory when it's high, than how much the highest memory use actually is. This allows the game to continue running as opposed to crash, as evidenced by the larger games which tend to crash when ported to Canvas+ when they run fine in a real browser (even with higher memory use!). The 2nd graph also confirms this: despite peaking higher, the web view finishes with lower memory use than canvas+. So I don't consider the peak memory usage particularly relevant, a more important question is "does it crash?"

    That the WebView does not crash does not mean that a game will run. Webviews can die in high memory usage situations too. If the entire apps doesn’t crash is because some webviews (like the Chromium on Android 4.4) do the job on a separated process from the app. Is the same as what happens in Google Chrome. Each tab is a separate process and if one of them dies the Chrome app might remain alive.

    We know that some customers were using the webview and the games were using so much memory that the webview just went blank. They started requesting the user to reboot the device and close all the other applications. We really think this is not the appropriate behaviour.

    Thanks to these memory management tools we just released, we have been able to provide solutions for low memory devices. We strongly believe that mobile game development needs to move a bit away from this idea of "the browser will handle everything for me" and do a good memory management understanding the issues any native app always face. JavaScript and HTML5 are awesome but successful mobile deployment requires a bit more knowledge. We still read things like "it works on my desktop browser, it should work on CocoonJS Canvas+ or even any mobile browser" and that is a tricky statement.

  • Ashley, you say webview supports WebRTC, but it doesn't. Why not?

Jump to:
Active Users
There are 1 visitors browsing this topic (0 users and 1 guests)