After some testing, if you lift the balll too high, and overlapping the wall, it fails ...
I gave the ball the behaviour bound by layout now too, and it stopped that ...
So far xD
Good to know that I can use it, so I can get rid of the walls on every level. I can probably add this behaviour to the group with all the objects, so no walls are required, and it doesn't fail to place objects close to the walls, it actually gets as far as it can get.
Problems with physics still remain tho
Not disabling collisions when touching restart solve the problem of the ball going through other objects, but I can't leave it like that because when I drag it, it just pushes other objects away.