Could be a number of things but in the past Vegas always based it's external preview on the internal preview window. So, if you set the preview window to auto and then sized it way down, vegas would output that reduced image and then scale it up. This makes it look kind of bad.