The problem lies in the fact that a bullet will move with respect to dt. dt will change all the time by tiny amounts, which means the bullets will move different distances every tick. If the speed of the bullet is high, it will move farther every tick.
The bullet will register its collision with a wall only once a tick. So, every bullet will probably move a different distance 'into the wall' before it executes the bounce code. This will give naturally inconsistent bouncing if you are at an angle.
There's not much you can do, apart from using physics with 'framerate independent' set to off.