You cannot use directories/subdirectories to organize layers in C2, so putting everything in one layout is not a good idea - I totally agree with that and discourage to do this. That is simply not a good development practice. Another discouraging fact is that if you ever think of moving this game to mobile devices, too many objects in one layout would kill the mobile probably on the start.
One of the most important thing in development is to correctly (also sematically if possible) organize your project elements. C2 gives us layout and layers for that and I recomment to use it.
There is a delay while switching the layout - that is true - but this is totally normal, because all layout objects have to be loaded. It's not a C2 flaw - this is simply how it works everywhere. Check out other games, even the big ones like Diablo etc., everytime you swich between some distinct "screens" there is a loading screen and you have to wait. The faster PC/mobile you have the faster loading is, but it is there anyway.
So don't try to workaround loading game elements by loading all of them at once because this is a bad practice and can kill weaker devices. Instead you should do the same thing as "the best devs" do for decades - use loading screens. Loading screen is not frustrating or anything like that. Players used to wait for game/screen to get loaded.