If you are launching the ball from a tube or similar, it could be that the impulse force is bouncing off of the object it's launching from. In which case it would be an idea to move it a few pixels away so it has a 'clean' launch area before the impulse force. This shouldn't be too obvious due to the quick frame refresh.
With regards to the arrow - that's a different question to what you originally posted, it's worth a separate topic.
If you don't want to ball to fall, then set gravity to 0. Physics works without gravity as well i.e. as in space.