In the image editor, at all time the X axis is highlighted, or actually the put at the start of the value. Selecting a polygon point and pressing Delete on the keyboard only removes the number on the x axis and moves the point to the left.
The bigger problem is that right clicking on the point and clicking Delete doesn't do anything.
I guess I'll need to put a reminder to change my collision polygons with the next release :P