Best to do this before encoding. For some reason DVD players seem to 'legalise' the video on output. I've dropped the encoded stream back into Vegas and on the scopes it looks the same yet when you play it out the video has been compressed into the legal range.
Best advice seems to be to push the gamma so the video is pretty 'hot' and then encode that. Don't print to VHS like that though our you're likely to hit problems.