You can set the layer as Global, then all objects from it will appear on all layouts where you add a layer with this name.
Another option is to use hierarchies. Select all objects of the help window, right-click on the parent object (for example WindowBackground sprite) and add all other objects to the hierarchy. Then you will be able to create the entire hierarchy in runtime - WindowBackground will be created with all its children. This way you can store the whole help window on an unused layout and only spawn it when needed.
I actually have a separate layout "DialogueAssets" in my project with dozens of hierarchies for different menus, dialogue and popup windows etc.