Huh, this is interesting - I could actually reproduce a leak in Chrome by pressing F1 and F2 pretty quickly. I got it up to 1 GB at one point. The video plugin is destroyed when you change layout, and it does run cleanup code, which includes pausing the video if playing (there's no "stop" in JS), and throwing away the video (just dropping the reference, there's no "destroy" or "cleanup" command in JS for video). However this does not seem to be enough to have it released from memory. I found one tiny tweak which seems to fix it though: now when the video is cleaned up we reset the video src to empty, so it drops the reference to the video file. This appears to properly clean it up and prevents Chrome heaping up memory. It seems to work OK in NW.js as well.
Here's the tweaked video plugin runtime.js if you want to test it yourself: https://dl.dropboxusercontent.com/u/15217362/video_runtime.zip
(Extract over exporters\html5\plugins\video\runtime.js and restart C2)
I think this is a workaround to what is basically a browser memory leak. If it works, this could save a lot of memory!