Have you tried it on Chrome? If the pauses aren't there, it's a problem with Safari, not our code, so there's probably not much we can do.
I would guess it's to do with the audio playback - in my testing, that's the main cause of pauses (browsers loading sound effects). Once most sounds have already played and the game is "warmed up", it should be running fairly smoothly. Do you find that it gets better later on?