Whats the best intermediary File Format to minimize generational loss?

chas-chas wrote on 9/15/2016, 5:24 AM

Whats the best intermediary File Format to minimize generational loss?

thanks

 

I look at the outputs for Vegas Pro 11.700 (32-bit).....I see a lot of -50i/-60i.....are they the interlaced formats? 

There are only 2x progressive, 24p and 25p file formats?

 

 

 

Comments

Wolfgang S. wrote on 9/15/2016, 10:12 AM

Typical intermediate codecs are Cineform or Canopus HQ(X). Cineform will work very well in Vegas, also with 10bit. 

That should not be mixed up with progressive versus interlaced, what is a complete different story. There are 23,976p, 24,000p, 25p, 30p, 50p, 60p or even 120p or higher. To use intermediate codecs to avoid losses - without other information I would stay in the format of the original footage. If one deinterlaces may depend more on how the footage will be used. 

Desktop: PC AMD 3960X, 24x3,8 Mhz * RTX 3080 Ti (12 GB)* Blackmagic Extreme 4K 12G * QNAP Max8 10 Gb Lan * Resolve Studio 18 * Edius X* Blackmagic Pocket 6K/6K Pro, EVA1, FS7

Laptop: ProArt Studiobook 16 OLED * internal HDR preview * i9 12900H with i-GPU Iris XE * 32 GB Ram) * Geforce RTX 3070 TI 8GB * internal HDR preview on the laptop monitor * Blackmagic Ultrastudio 4K mini

HDR monitor: ProArt Monitor PA32 UCG-K 1600 nits, Atomos Sumo

Others: Edius NX (Canopus NX)-card in an old XP-System. Edius 4.6 and other systems

ritsmer wrote on 9/15/2016, 12:09 PM

Agree - but also a format that will smart-render can be used. There has been some discussion if smart-rendering works - or not - but check the manual for the conditions - and then try it out with your normally used delivery formats - just to be sure.

I have successfully smart-rendered recently...

... and then we have the lossless Lagarith too. But it takes up very much space, of course.

Finally, if you decide for, say a mp4 format - you can mostly raise the Mbps significently before rendering.

chas-chas wrote on 9/15/2016, 1:19 PM

what version of cineform, is it free and where can i get it?

@ritsmer, what codec can smart render? thanks

Musicvid wrote on 9/15/2016, 2:23 PM

I tend to divide "best" into three categories -- size, quality, encoding speed.

General rule: pick any two of the above.

Are you looking for a mathematically lossless encoder, or a visully lossless encoder that produces smaller files?

 

chas-chas wrote on 9/15/2016, 2:41 PM

no i need a good codec for a 32bit system that can mitigate the generation loss....

SO i need ALL of them at reasonable levels :P

thanks so much :P

NormanPCN wrote on 9/15/2016, 3:01 PM

what version of cineform, is it free and where can i get it?

Download the GoPro studio software. It is free. I'm not sure if it supports 32-bit beyond Quicktime of course.

 

Musicvid wrote on 9/15/2016, 3:09 PM

Yes, you will want a mathematically lossless RGB encoder,, not YUV.

My previous tests on the warhorse encoders are here, but not Sony or Magic YUV, because they will start to show wear around 6-10 generations. Be aware that some of the following will also encode yuv, but that is not what you want.

https://www.vegascreativesoftware.info/us/forum/intermediates-part-i-seven-lossless-codecs--84932/

Wolfgang S. wrote on 9/15/2016, 4:17 PM

Cineform can be installed as codec in your system if you install the free GoPro Software. And the decoding of Cineform has been implement with Vegas 6 or 7 I think - so that works for 10bit too.

Another nice codec would be XAVC-I today.

Desktop: PC AMD 3960X, 24x3,8 Mhz * RTX 3080 Ti (12 GB)* Blackmagic Extreme 4K 12G * QNAP Max8 10 Gb Lan * Resolve Studio 18 * Edius X* Blackmagic Pocket 6K/6K Pro, EVA1, FS7

Laptop: ProArt Studiobook 16 OLED * internal HDR preview * i9 12900H with i-GPU Iris XE * 32 GB Ram) * Geforce RTX 3070 TI 8GB * internal HDR preview on the laptop monitor * Blackmagic Ultrastudio 4K mini

HDR monitor: ProArt Monitor PA32 UCG-K 1600 nits, Atomos Sumo

Others: Edius NX (Canopus NX)-card in an old XP-System. Edius 4.6 and other systems

chas-chas wrote on 9/15/2016, 5:28 PM

can anyone link me up to a download thanks, i cannot find it

 

https://gopro.com/help

vtxrocketeer wrote on 9/15/2016, 6:55 PM

can anyone link me up to a download thanks, i cannot find it

 

https://gopro.com/help

Just download and install the free GoPro Studio.  The Cineform codecs will be installed when you do this.  You need not use Studio for anything, but you have to install it to get Cineform.  It is a lossy codec, as noted above, but it is visually lossless for several generations.  It's a superb intermediate codec, plays well with Vegas, and is relatively compact. 

Musicvid wrote on 9/15/2016, 10:05 PM

GoPro Studio is awesome for some tasks, including unpacking ProTune

 

Former user wrote on 9/15/2016, 10:11 PM

I still don't get why people avoid uncompressed for their intermediates. If you are working in video, you know you need a lot of storage so that shouldn't be an issue. You can always delete the intermediate when you are finished. It renders several times faster than a compressed format.(which is an advantage if you need to recreate the intermediate). And is lossless.

NormanPCN wrote on 9/15/2016, 11:41 PM

Data rate probably. 8-bit 1080p30 video has a data rate of 186MB per second. Well beyond HD bandwidth for a single media file. SSD and/or RAID territory. "Sony YUV" would save some data rate and is uncompressed but not lossless as it is 422 chroma subsample.

 

NickHope wrote on 9/16/2016, 2:20 AM

...but not... ...Magic YUV, because they will start to show wear around 6-10 generations.

It shouldn't do if the default "Compress as is (no conversion)" is set, should it? When I render to MagicYUV and bring it back to the timeline to compare with the original, not a single pixel moves on any of the scopes, which says to me it's 100% lossless.

Musicvid wrote on 9/16/2016, 5:42 AM

It's  just the chroma subsampling  that makes the big initial hit, then whatever compression losses in subsequent generations. If one is running Magic in 4:4:4 or direct copy, I doubt one would detect a difference after any reasonable number of generations. 

All yuv-only encoders were ruled out of that initial (only?) round of tests, and comparing the belle-nuit chart with 4:4:4 rgb will instantly show the differences. Generational losses I wouldn't expect to be much different than Huffy, which showed up after the fifth generation in one guy's tests.

What would be interesting would be to see tests comparing Sony YUV and Magic in 8 and 10 bit modes using the scopes and charts, Sony being the gold standard in every test I ran. Still, a yuv intermediate would be my second choice if I knew I would be rendering more than a few consecutive generations.

 

Former user wrote on 9/16/2016, 9:04 AM

Data rate is an issue only if you are trying to play it real time. If you are making an intermediate, to me that means you are moving between programs such as Vegas to Resolve, etc. I find uncompressed much faster to render and move around between programs and don't have to be concerned with any loss.

chas-chas wrote on 9/16/2016, 10:20 AM

1. where is this setting thanks? "Compress as is (no conversion)"

Also this is for use on a laptop so uncompressed is no good: big HDD and fast CPU would be needed...

Donations welcome :P

2. Btw, Vegas does try to preview in full mode...HOW to turn that preview OFF ?

PS. there is NO Audio in any of the SONY bluray and AVCHD outputs? never seem to render for me....

SO most of Vegas' outputs are interlaced? or do i just switch the settings to NONE(Progessive)? 

PPS. Lagarith is giving me 1.5+Gb per 60 sec footage, how much could cineform help?

 

NickHope wrote on 9/16/2016, 1:39 PM

@chas-chas You'll find it here... Choose Render As > Video for Windows > Choose a template > Choose MagicYUV from the Video format dropdown menu > Configure...  I use defaults for everything in there (I think).

@Musicvid As I understand it, MagicYUV does a direct RGB encode, not YUV, if it's set like in these screenshots. The "TIP" at the bottom of the tooltip implies (to me) that direct RGB will be used if you don't select a YUV variant. Unfortunately MediaInfo doesn't reveal whether the rendered file is RGB or YUV. I guess I could do some multi-generational tests to confirm it.

Eagle Six wrote on 9/16/2016, 1:40 PM

1 minute 1080p event:

CineForm YUV 4:2:2 Filmscan 1 = 2,050,021 KB

CineForm RGB 4:4:4 Filmscan1 = 4,206,236 KB

Edit:  Lagarith Lossless RGB = 4,299,185 KB (on my desktop the CineForm render times are about 25% faster than Lagarith)

Best Regards......George

Last changed by Eagle Six on 9/16/2016, 1:50 PM, changed a total of 1 times.

System Specs......
Corsair Obsidian Series 450D ATX Mid Tower
Asus X99-A II LGA 2011-v3, Intel X99 SATA 6 Gb/s USB 3.1/3.0 ATX Intel Motherboard
Intel Core i7-6800K 15M Broadwell-E, 6 core 3.4 GHz LGA 2011-v3 (overclocked 20%)
64GB Corsair Vengeance LPX DDR4 3200
Corsair Hydro Series H110i GTX 280mm Extreme Performance Liquid CPU Cooler
MSI Radeon R9 390 DirectX 12 8GB Video Card
Corsair RMx Series RM750X 740W 80 Plus Gold power pack
Samsung 970 EVO NVMe M.2 boot drive
Corsair Neutron XT 2.5 480GB SATA III SSD - video work drive
Western Digitial 1TB 7200 RPM SATA - video work drive
Western Digital Black 6TB 7200 RPM SATA 6Bb/s 128MB Cache 3.5 data drive

Bluray Disc burner drive
2x 1080p monitors
Microsoft Window 10 Pro
DaVinci Resolve Studio 16 pb2
SVP13, MVP15, MVP16, SMSP13, MVMS15, MVMSP15, MVMSP16

NickHope wrote on 9/16/2016, 1:52 PM

I did some tests on lossless and near-lossless codecs here: https://www.vegascreativesoftware.info/us/forum/magicyuv-codec-coming-on-strong-for-4k--98668/#ca606764 Unfortunately didn't include Lagarith or Cineform but you should get a significant saving in file size with any of the near-lossless codecs (Cineform etc.) compared to the lossless codecs (MagicYUV, UT Video Codec, Lagarith). The thing I really like about MagicYUV compared to the other lossless codecs is the decode speed, which makes for smooth playback. If you're pushed for power and space then don't discount Sony XAVC Intra. It may be lossless enough for you and it's small and plays smoothly and of course natively supported in Vegas. I would test that against Cineform if I were you.

Musicvid wrote on 9/16/2016, 6:38 PM

Unfortunately MediaInfo doesn't reveal whether the rendered file is RGB or YUV. I guess I could do some multi-generational tests to confirm it.

A simple render of the Belle-Nuit 1080p chart will tell you instantly if it's something other than 4:4:4.

http://www.belle-nuit.com/test-chart

NickHope wrote on 9/16/2016, 8:57 PM

Unfortunately MediaInfo doesn't reveal whether the rendered file is RGB or YUV. I guess I could do some multi-generational tests to confirm it.

A simple render of the Belle-Nuit 1080p chart will tell you instantly if it's something other than 4:4:4.

http://www.belle-nuit.com/test-chart


I just did so with the following settings and not a single pixel on the scopes moves between the test-chart tif file and the rendered MagicYUV file.

John_Cline wrote on 9/16/2016, 11:19 PM

I use the UTvideo lossless codec if for no other reason than it is handled natively in Handbrake.