I just ran across this same issue today when adding any fx to a clip. Since the OP didn't post a screen shot, I will here. This started in the middle of a project when I was adding some green screen photos. The first few photos were fine the every photo added after that had the rectangle.
I restarted Vegas PRO 14, rebooted the computer, moved the preview from one screen to the other, Opened a new project. Nothing helps.
Here's a shot of a new project
Former user
wrote on 2/6/2018, 7:54 PM
Turn off the FX Split Screen, it is to the left of where it says "good"