When I first lift a movie to a Menu, the Thumbnail is rectangular as expected.
However, when I double click a Button style, even if the chosen style is itself rectangular, the Thumbnail turns square, and not even dragging the side handle will change this.
No solution but some insight. If you select the button Mask property under the graphics menu heading in the properties window and change the mask file to a rectangular file you create in Paintshop Pro or Photoshop then you can retain the rectangular shape to your button.
If you use the mask supplied by DVDA2 for your chosen button then it will force it to turn into a square.