I believe this is what you want.
The latter part of this post came from musicvid.

Encode set to AC3;
Click on custom tab;
On the first tab set Dialog normalization to "-31";
On the last tab marked preprocessing;
Set the Line Mode & RF mode profiles to "None";
Now save this as a preset.


If you're doing AC-3 Stereo, you can ignore everything I say about the Phase Shift option, it's greyed out of course.

Dialog Norm at -31 (this sets it at unity gain)
RF and Line Mode DRC Off (WYHIWYG)
DC High Pass Filter On (this operates at <=3Hz and takes DC bias out of the mix)
Bandwidth Low Pass Filter On (this takes aliasing noise at >20kHz out of the encode, but it also lowered the rendered volume by -0.1dB in my tests)
90 Deg Phase Shift On (makes it easier for Pro Logic decoding on older equipment, Dolby says it is inaudible in 5.1 except in "very rare" cases)

Everything else can be left just as it is in the 5.1 DVD template; the only difference I can determine compared to the Studio encoder is the 90 Deg Phase Shift, but turning it off replicates the downmix pan bug I reported in the Audio forum.

Mike