If the layout remains the same, it should not have occasional lagspikes. Basically the asset loaded will remain in memory until layout change.
I've noticed upon layout change, ~5 to 10 seconds, gc kicks in, there's a small stutter as memory is cleaned of un-used assets.
What I also do is have a function that calls regularly used explosions, sprites particles etc off the screen on start of layout, this acts as a "pre-load" mechanic by putting them into memory first, so when they are used and called, they won't cause loading stutter.
Because this is the way gc works you have to design your game around it. Basically any big sprites should be called on start of layout (then destroyed if its not needed right then).