Just putting this out there because it wasted a day of my time, hope to help someone else with this problem.
I'm running Vegas 14 with a GeForce GTX 580 card and GPU processing enabled. This worked until recently, when suddenly any and all mp4 renders would fail (at 0% completed, with Vegas lockup). Other file formats rendered fine.
After much trial and error, here's what I found: the first fix was, not surprisingly, to disable the GPU processing, and then mp4s would render, but more slowly. Next, I rolled back the NVIDIA driver to Version 372.90 (from 390.65) and was again able to render mp4s with GPU processing enabled. (Also, using the 390 driver, I saw lots of weird issues with Titles and Text.)
So I guess I'm staying with 2016 display drivers for the foreseeable future...