Yes, thank you I completely forgot about that. 
Yet, the problem with the restarting audio still persisted. I didn't mention that my prototype is an audio-only game. Since the player won't see the objects anyway, I ended up rotating the whole world around the player, this way the audio doesn't restart. 
This might not be a solution if you are working with visuals, though.