As with import statements, in the import map you need to start paths with ./ for them to refer to files in the project. If they start with /, that is (I think) interpreted as a URL relative to the root of the origin, which will happen to work in preview as that runs on preview.construct.net, but won't work on export.
So you should start those paths with ./. I'm not sure what you mean by this:
This makes you revert back to relative paths, but now you can't use the same import map key because the path to the file is different from different directories.
You should be able to use the same import map key everywhere. The location you use it doesn't matter.