Certain Memory does not get released in texture allocation

Problem Description

Some sort of memory are not deallocated when Sprite objects are no longer presented.

Attach a Capx

http://www.mediafire.com/download/k4fq5gq1r22wj1w/spriteLeak.capx

Description of Capx

The capx contains 3 layouts. Layout 1 is blank. Layout 2 and 3 contains 5 and 3 Sprites respectively. All sprites are 2048 x 2048 pixels, and they just contain different solid colors.

You can press 1,2,3 to change to Layout 1,2,3.

Steps to Reproduce Bug

• Step 1 : run Performance Monitor or tools that can monitor memory consumption of processes.
• Step 2 : run capx.
• Step 3 : Change to Layout 2 and then 3 by pressing 2 and 3. You should see memory consumption for the process going up. In this step, you can try go in only one layout and see difference in memory consumption as well.
• Step 4: Press 1 to come back to Layout 1, where there are no sprites at all. In a brief moment, Garbage Collector will release some memory, but you will see that not all memory is released.

Observed Result

Test on Firefox:

Figure 1

On Firefox, going to layout 2 with those big Sprites and back, the memory consumption comes back only half way. It never goes down ever.

Test on Chrome:

Figure 2

On Chrome, we see similar development just like Firefox.

However, you can force garbage collector to do its job by minimizing Chrome, and this is what we get:

Figure 3

Usually for Chrome and Node Webkit, there are 3 processes, so each line above goes to each process. The green line is the main process, we believe. When we change layouts around, it goes up. When we do nothing, it goes down. When we minimize Chrome, it goes way to the bottom. However, what is interesting is the blue line. We noticed that whenever we allocated more texture memory (ie. sprites are presented when we switch to another layout), the blue line rises up. However, it never comes back down.

We have also checked against different number of Sprite in capx. We tried only one Sprite instead of five, and switching to this layout only. We got the following result:

Note that the blue lines in figure 4 and 5 are the main process (basically green line in Figure 3) and the red line is actually the same process as blue line in Figure 3. (we accidentally close Performance Monitor and we didn't change color back when we start over)

1 sprite:

Figure 4

5 sprites:

Figure 5

This is to demonstrate that the blue line in Figure 3 correlates with texture memory.

We also recalled from the section "A word on texture loading" in https://www.scirra.com/blog/112/remembe ... our-memory :

[quote:38engof9]When the layout ends, it will free all the images in memory not used by the next layout, and load any new images the next layout uses.

So we decided to checked against our GPU whether texture is deallocated or not.

Here are our results:

Preview Chrome:

Exported Node Webkit:

Preview Firefox: (NOTE: the reason that you see it going down here is because that was leftover from our other experiment. Firefox process begins when it rises up again at half way on the graph)

Preview Node Webkit:

We see memory juggling in Preview Chrome and Exported Node Webkit as we switched between the 3 layouts. It always get back down when there are no Sprites. On the other hands, we see memory going up but never come down in the other two.

Perhaps we could say : Preview Chrome and Exported Node Webkit have texture deallocated when Sprites are no longer presented. Preview Firefox and Preview Node Webkit do not have texture deallocated.

This is ok for us, as our focus is Chrome and exported Node Webkit. Nevertheless, recall the blue line from Figure 3, could you please check whether anything can be done?

Expected Result

No longer required memory should be released.

Affected Browsers

• Chrome: YES
• FireFox: YES
• Node Webkit: YES

Operating System and Service Pack

Win 7 64 bits Service Pack 1

Construct 2 Version ID

r173

I don't think there is anything to be done here. Memory management in modern browser engines is complex and measurements may not give an accurate picture of what is happening. There are a lot of subtleties to making these kinds of measurements.

I am pretty sure our memory management code works and that textures really are released by our engine when switching layouts. We have dealt with bugs in other engines like Ejecta where the engine itself failed to release textures and caused out of memory errors, then we fixed it and the out of memory errors went away. (None of this involved changes to Construct 2 - only Ejecta.) When the memory management feature was first introduced, it also enabled large games that previously would not start up to run smoothly throughout.

Some of the complexities involved in these measurements are issues like:

• the "working set" is not exactly the same as "amount of memory allocated by the program". MSDN defines the working set as "the set of pages in the virtual address space of the process that are currently resident in physical memory". This is subject to the OS virtual memory management and it will probably be making decisions based on caching for performance with respect to the total amount of memory in the system and its current usage.
• modern browser engines make heavy use of caching to improve performance. While our engine may explicitly release a resource, the browser may keep it around if it sees no harm in doing so, so subsequent requests can be faster. If the system really gets close to using all its resources (which your test does not appear to come close to), the browser may start freeing its caches to prioritise memory use over performance.
• modern javascript engines have two or more levels of garbage collection. Typically there is a small, fast and regular generational collection aimed at releasing short-lived objects, and then there is a separate large, infrequent, slow full-heap sweep aimed at releasing long-lived objects. There may be multiple stages in-between and they may be separate sweeps aimed at different sized allocations. This means identifying a drop in memory use does not mean the browser has released all unused resources - it may have only done a partial sweep.

Minimising Chrome probably causes a full sweep since it thinks it may not be looked at for a while, and it's a good opportunity since doing it while scrolling or playing a game can cause a momentary pause (jank). Your data appears to support this, where the blue line also represents some kind of cache that isn't being freed, perhaps because Chrome or the OS has identified there are still plenty of free resources on the system so doing so would only reduce performance.

While it's interesting to investigate such issues, it is difficult to draw firm conclusions from data like this. I think we have to limit memory use bug reports to cases where the system can be shown to fail due to exhausting available resources, which suggests a memory leak. (I haven't investigated the savegame bug report yet, but the javascript error suggests it's more likely there's a real issue there.)

Also note it's good to check the latest betas if possible - we may have fixed or altered the issue in recent releases, either deliberately or accidentally.

Sorry for posting in the closed topics,

I was about to submit a bug report, but searched first and this is the same bug.

I see this is an old bug and this behavior is exactly what I've seen in my game, and what I was talking about a few days ago.

Every layout transition, the nw.exe process keeps growing larger and larger (~3-4MB), beyond all the assets of game so there's some kind of duplication in memory going on for recent used/destroyed assets. Basically that process never reduces in memory use, only grows larger over time.

After around 1 hour of play, the process gets to around 1.5GB of memory and the game starts to stutter. A close and relaunch resolves the issue.

Tested on both Chrome, NW10.5 and NW12.

If people's games are large enough, after multiple layout transitions, it could blow up to beyond the 32 bit OS limit of ~3.4GB memory (the rest out of 4GB, reserved for OS) and the game crashes as Tinimations has encountered with Klang.

I suspect it is Chromium not releasing its cache as it should, under normal operations, it will release the cache once system ram limits are approached but it may not be functional for NW.JS/C2 usage.

Edit: Just tested in IE (Win 7 64 Enterprise). This behavior does NOT happen. The iexplorer.exe process caps out around 600MB, but it drops down lower based on the layout. It does not exhibit the ever rising memory usage issue. Destroying images will reduce the memory usage of the process according to the asset size. Thus, whatever is the problem is between C2 and NW.JS/Chromium.

— usually, closed bugs like this are ignored. I suggest you to open a new bug report, to get some attention.