best "archiving" codec

Mindmatter wrote on 1/4/2016, 8:21 AM
Hi all,

I've been asked to deliver a " master quality" ( meaning least possible compression) version of a recent documentary I made with a Sony PMW 320. I was thinking of a DNxHD185 export. I had already rendered an internet version in 16mbs .mp4 ( Sony AVC), which apparently is not good enough.
My quesion is, as the sony records in mpeg2 with about 35mb/s, does it make sense at all to render in such a high bitrate? if not , what codec or bitrate should I choose?

Thanks!

AMD Ryzen 9 5900X, 12x 3.7 GHz
32 GB DDR4-3200 MHz (2x16GB), Dual-Channel
NVIDIA GeForce RTX 3070, 8GB GDDR6, HDMI, DP, studio drivers
ASUS PRIME B550M-K, AMD B550, AM4, mATX
7.1 (8-chanel) Surround-Sound, Digital Audio, onboard
Samsung 970 EVO Plus 250GB, NVMe M.2 PCIe x4 SSD
be quiet! System Power 9 700W CM, 80+ Bronze, modular
2x WD red 6TB
2x Samsung 2TB SSD

Comments

musicvid10 wrote on 1/4/2016, 8:47 AM
The best archive is a smart render (or as close as possible) of the original.
Mpeg-2 will not become obsolete in our lifetimes.
Former user wrote on 1/4/2016, 10:37 AM
I would ask that they be more specific about what format they want. The best master quality, other than original files, would be uncompressed. Rendering in a good codec at a higher bitrate to a compressed file will not improve the quality, but it can make quality loss minimal. So yes, I think higher bitrates do make sense. I try to do as much uncompressed as I can. Even my intermediates.
Jamon wrote on 1/4/2016, 10:48 AM
Search for "lossless video codec".
musicvid10 wrote on 1/4/2016, 11:12 AM
Rendering to uncompressed only adds air to your decompressed bitstream, so there is no gain by inflating its size. The colorspace conversion to rgb causes additional losses, so I would stick with the closest yuv codec, in this case mpeg 2.
Former user wrote on 1/4/2016, 11:19 AM
If you can do a smart render to mpeg2, I would agree. But to render to any other format, it has to be uncompressed anyway.
musicvid10 wrote on 1/4/2016, 11:36 AM
That's decompressed, not uncompressed.
Post some tests if you don't see a difference rendering YUV->RGB colorspace.
Actually, rendering a 10 bit yuv archive would be my first choice to save space while preventing further banding.
Former user wrote on 1/4/2016, 11:55 AM
Yep, decompressed. I thought about that after I typed it. I am not disagreeing about the yuv-rgb, but there are "uncompressed" yuv codecs.

Mindmatter wrote on 1/4/2016, 3:19 PM
Thanks for your answers!
I just did some tests - the smart rendered file ( template EX HQ 35mbs VBR ) looks , at least to my eyes right now, the exact same as the original coming from the PMW 320. I also can't see a difference to DNXHD185.
What struck me tho is that comparing the smart rendered and the original in mediainfo, the original says 37,7 mbs and the smart rendered 27.9 mbs.

Why is there such a difference?

Also, to my surprise, mediainfio says the original clips are in 4:2:0. I was sure, and rechecked the Sony PMW 320 specs online, that it should have recorded 50mbs in 4:2.2 ??

AMD Ryzen 9 5900X, 12x 3.7 GHz
32 GB DDR4-3200 MHz (2x16GB), Dual-Channel
NVIDIA GeForce RTX 3070, 8GB GDDR6, HDMI, DP, studio drivers
ASUS PRIME B550M-K, AMD B550, AM4, mATX
7.1 (8-chanel) Surround-Sound, Digital Audio, onboard
Samsung 970 EVO Plus 250GB, NVMe M.2 PCIe x4 SSD
be quiet! System Power 9 700W CM, 80+ Bronze, modular
2x WD red 6TB
2x Samsung 2TB SSD

musicvid10 wrote on 1/4/2016, 5:02 PM
yeah anything that will smart render is your best bet.
Donald and i both agree on that
PeterDuke wrote on 1/4/2016, 6:58 PM
"What struck me tho is that comparing the smart rendered and the original in mediainfo, the original says 37,7 mbs and the smart rendered 27.9 mbs.

Why is there such a difference?"

There may be metadata such as aperture, shutter, date, GPS, etc. that has been stripped during render.

Vegas does not fully smart render MPEG2 in tests that I have done, but that is unlikely to account for the difference you see.

Did you see a "no recompression required" message when you rendered?

If not, it didn't smart render.
musicvid10 wrote on 1/4/2016, 7:32 PM
Vegas Pro can smart-render vbr mpeg-2 with different minimum bitrates than the source without re-rendering, and that is at least one of the reasons for the discrepancy you see; camera transport streams run generally higher minimum or even cbr.
videoITguy wrote on 1/4/2016, 9:17 PM
Mindmatter's OP question begs to know more about what the focus of the direction is for the receiver of the file.

Archiving codec is not really about compression or no compression - its about viability in decoding platforms of the future.

1) A great example of a compressed but math lossless ( thus visually identical) compare to the original. MagicYUV

2) A fair comparison example of the recent past is Cineform with compression and minimal visual loss over at least 5 generations ...
PeterDuke wrote on 1/5/2016, 1:30 AM
As I recall, Cineform only handles stereo audio. If you want 5.1 or higher, you will have to save the audio separately.
royfphoto wrote on 1/5/2016, 3:55 AM
Who's doing the asking? If for broadcast most networks have a TOS sheet
I shoot for PBS and follow:

In this case I can use: XDCAM 422 50mb, or DNxHD 145, 220 or 220x OP1a file
Mindmatter wrote on 1/5/2016, 4:32 AM
Thanks again for the information everyone.

Peter, i didn't see a " no recompression needed " info when I rendered.

As to the purpose of the archiving - in Luxembourg, we have the CNA ( Centre national de l'Audiovisuel). By law, every officially broadcast or presented production is being archived in order to keep a national / historical audiovisual archive. So when we were approached by the CNA, they asked for a master qualiity export, expecting something like a prores uncompressed / minimally compressed high res codec render directly from the prores files, what is probably what most digital cinema productions give them.

After consulting the forum here, I informed them about the codec of my original files and opted for the smart rendered version. I'll see what they say.

royfphoto, I can't find the 50mbs 4.2.2.option, where exactly do you have that? My best option in the EX templates is HQ 11920x1080 35 mbps VBR, and I can't edit any of the values there.

AMD Ryzen 9 5900X, 12x 3.7 GHz
32 GB DDR4-3200 MHz (2x16GB), Dual-Channel
NVIDIA GeForce RTX 3070, 8GB GDDR6, HDMI, DP, studio drivers
ASUS PRIME B550M-K, AMD B550, AM4, mATX
7.1 (8-chanel) Surround-Sound, Digital Audio, onboard
Samsung 970 EVO Plus 250GB, NVMe M.2 PCIe x4 SSD
be quiet! System Power 9 700W CM, 80+ Bronze, modular
2x WD red 6TB
2x Samsung 2TB SSD

musicvid10 wrote on 1/5/2016, 8:00 AM
If you don't see "no decompression needed" on the unchanged footage, that is not a smart-render.
JohnnyRoy wrote on 1/5/2016, 1:33 PM
> "...they asked for a master qualiity export, expecting something like a prores uncompressed / minimally compressed high res codec render directly from the prores files, what is probably what most digital cinema productions give them."

Yup. That's what I was just going to point out. When a broadcaster asks for a "Master File" they are usually referring to Apple ProRes 422 HQ which Vegas Pro cannot create.

I would use the Sony MXF encoder with the HD422 1920x1080-60i 50 Mbps. template. This is what we give PBS and they are happy with it. (of course, if you are in a PAL country use the 50i template instead)

Ask if they will accept MXF files first.

~jr
relaxvideo wrote on 1/5/2016, 3:01 PM
I love Canopus HQ codec (can smartrender from Vegas)

#1 Ryzen 5-1600, 16GB DDR4, Nvidia 1660 Super, M2-SSD, Acer freesync monitor

#2 i7-2600, 32GB, Nvidia 1660Ti, SSD for system, M2-SSD for work, 2x4TB hdd, LG 3D monitor +3DTV +3D projectors

Win10 x64, Vegas22 latest

Mindmatter wrote on 1/6/2016, 12:46 PM
"If you don't see "no decompression needed" on the unchanged footage, that is not a smart-render."

I guess that as soon as there are fades, text, FX etc, there has to be a reprocessing / recompression of the material at those exact points on the timeline. As to the rest, I see no reason why tat should not simply be smartrendered mpeg4. What's your view on that?

AMD Ryzen 9 5900X, 12x 3.7 GHz
32 GB DDR4-3200 MHz (2x16GB), Dual-Channel
NVIDIA GeForce RTX 3070, 8GB GDDR6, HDMI, DP, studio drivers
ASUS PRIME B550M-K, AMD B550, AM4, mATX
7.1 (8-chanel) Surround-Sound, Digital Audio, onboard
Samsung 970 EVO Plus 250GB, NVMe M.2 PCIe x4 SSD
be quiet! System Power 9 700W CM, 80+ Bronze, modular
2x WD red 6TB
2x Samsung 2TB SSD

musicvid10 wrote on 1/6/2016, 2:27 PM
All render settings must match the source.
You are correct; none of your transitions,effects, generated text even exist except as a set instructions-- they must be rendered, but only once.
Chienworks wrote on 1/6/2016, 2:48 PM
To be even broader (is that now a non-PC term?), anything that alters the original video image in any way will cause recompression. This also includes things like pan/crop, track motion, compositing, titles on top of the footage, any effects, any composite/transparency levels, any processing, crossfades, transitions, cuts, tapping your toe while rendering ... etc.

As far as that the render settings must match the source, this is of course a given. Considering how critical it is it seems a pity that there isn't an option for this in the rendering/template dialog. There should be a big box you can check for "i'm going to try smart rendering so just use the source settings for my output please!" It seems very rude that Vegas makes you supply that information, and won't even tell you you're wrong until you notice (or maybe don't notice) the lack of "no recompression required" in the preview window.

This of course also leads to a problem if you mix sources in your project. If you use clips of multiple formats then you can only choose one format to smart render to, and everything else will be recompressed.
royfphoto wrote on 1/6/2016, 4:23 PM
check view all options> Render> Sony MFX> scroll down to the bottom
PeterDuke wrote on 1/6/2016, 5:11 PM
There is also a flag in Preferences called "Enable no-recompress rendering" that is normally ticked. If not ticked, Vegas won't smart render (except in version 9!).
PeterDuke wrote on 1/6/2016, 5:34 PM
"I guess that as soon as there are fades, text, FX etc, there has to be a reprocessing / recompression of the material at those exact points on the timeline. As to the rest, I see no reason why tat should not simply be smartrendered mpeg4. What's your view on that?"

While it may be theoretically possible to smart render unchanged segments, the encoder has to be written to do that. For a while (Versions 9b and 9c I think), Vegas would (sort of) smart render AVCHD, a form of MPEG4. In later versions they removed that capability, presumably because the implementation had some problems that they didn't wish to spend time on trying to fix.

Adobe Premiere doesn't smart render at all as far as I know.