The golden rule of performance is to make measurements. If you can't measure a difference, it doesn't matter either way, and you're probably wasting your time until you find something that makes a measurable difference.
Note that testing a powerful desktop machine might make it difficult to measure any changes, since it will always be fast enough - ideally you want the weakest device that you intend to be targeting to test on. (For example for a Steam desktop game the lowest-end device is probably still a decent-ish gaming laptop, whereas for a mobile game the lowest-end device would be a cheap Android phone.)