Is it possible to "select" a part with a mask (let's say for simplicity sake, square) and instead of everything else getting cut out, invert the section so it is cut out?
I'm sure I missed something and it is simple, but my brain isn't working right now.
What sort of mask are you using? Cookie Cutter allows "cut away section" or "cut away all but section". If you are using a compositing mask you can add the Invert filter to the mask. If you are creating the mask yourself as an image you can invert it before using it in Vegas.