It sounds like we already have keyboard shortcuts that cover most of that:
Oh! and when selecting objects on different z depths it would be nice if the actual object on a lower z depth I've got my mouse over is picked instead of another object that is above it
That's: Hold Tab and click a selected object to select the next object underneath in the Z order
Oh and yes, when moving objects it would be nice if we could use the arrow keys to move objects in the layoutview instead of relying on just the mouse all the time would be great for accurate alignments.
And that's already supported - did you try it?