I recently was working on doing something very similar - although I was also trying to use the entire screen (regardless of device size or orientation) by using Scale Outer and Unbounded scrolling (which added an extra level of complexity that took me an entire long weekend to work out).
it is just a matter of calculating how far the layout is allowed to scroll, and keeping within those limits.
So, here is how I would do what you were asking - no plugins required:
this version does the double tap to zoom you mentioned, but I also included how I track multiple touch points, and do pinch to zoom... I also added a button to turn the limit feature on and off so that you can see that it really works.