Interlaced HD to DVD AGAIN - some test renders

Back to post

Comments

NickHope wrote on 5/13/2011, 5:41 AM
Didn't you try that in Method 2 at the beginning of the thread? Most of us didn't seem to like the results very much.

Yes. I wasn't a big fan of the result either. But there's a lot more difference in that method than just the spline36 resize. I don't believe it's the spline36 resize that is the drawback.

Also, why are the forum members over at Doom 9 relying on the rendered results of the Belle Nuit test chart so much? It is after all a still image that doesn't seem relevant to real world moving image comparisons IMO.

I agree and I didn't notice that they are. Rather it seems to be us in this thread that have been over-relying on it. However it does usefully show up a significant and fundamental difference between doing a bob using TDeint(mode=1) and other methods. i.e. grey vs alternating black/white.
craftech wrote on 5/13/2011, 6:02 AM
I am also noticing that if I set the first line as:

SetMemoryMax(800)

It renders with about 1850 MB or so memory utilization. If I set it to 1000 the memory utilization is higher (2200 +), but it crashes.

If I comment the line out (#), Windows will utilize an average of 2085 MB and it doesn't crash. Seems to be a tricky setting that may be worth commenting out because It doesn't seem to change the rendering time at all.

Not sure.

I am using the utility CpuMon

John
johnmeyer wrote on 5/13/2011, 11:17 AM
Don't use SetMemoryMax() and don't use the Distributor() statements unless you cannot fix the crashing any other way. I never even knew about the Distributor() statement until recently and have never needed it, even with QTGMC, which is one of the least stable scripts I have tried to run with multi-threading (it runs just fine without multi-threading and, to its credit, can in fact be multi-threaded, whereas some scripts use plugins that simply do not run with the multi-threaded AVISynth hack).

So, view these as a "last resort," rather than something you routinely put into every multi-threaded script.

craftech wrote on 5/17/2011, 11:59 AM
Question:

This setting in the AviSynth script:

ColorYUV(levels="TV->PC")

appears to darken the video.

Is that the correct setting?

Thanks,

John
NickHope wrote on 5/17/2011, 10:43 PM
ColorYUV(levels="TV->PC") widens the color space from [16,235] to [0,255], so it should darken lowlights and brighten highlights. In other words it should increase contrast. This is necessary because serving in YUY2 squeezes the color space from [0,255] to [16,235] and generally we want to reverse that.

If your Vegas project was set up to use the whole range from 0 to 255, as might be the case with dSLR footage, then the squeeze might be a good thing so that you end up with legal levels for DVD. But in the case of your stage show footage, the scopes show blacks at around 16 and highlights well below 235, so the levels are already legal, so I expect you'll want to reverse the squeeze, otherwise your DVD will end up wishywashy.

Apologies for my terminology. I have terrible trouble knowing the correct technical terms for this stuff.
craftech wrote on 5/18/2011, 2:56 AM
Thanks Nick,

So you are saying that for my stage footage ColorYUV(levels="TV->PC") already reverses the squeeze so leave it there? Or are you saying I should reverse it to ColorYUV(levels="PC>TV")?

The blacks are a bit crushed.

If you notice in the script I used I left out QTGMC.

Would that have done anything to change the lowlights had I left it in?

John
NickHope wrote on 5/18/2011, 4:46 AM
So you are saying that for my stage footage ColorYUV(levels="TV->PC") already reverses the squeeze so leave it there?
Yes

The blacks are a bit crushed.
QTGMC doesn't do anything to change the luminance.

Looking at the histogram of the short clip you sent me, the minimum luminance of the blacks is 16 and the "knee" is at 18, which is about perfect, so I wouldn't do anything.

In theory the blacks shouldn't be getting crushed in that clip since they are not darker than 16. If you're not sure what is going on, render a few seconds of Belle Nuit test chart along with some of your footage, burn to DVD, and study the result. On a TV the areas on the left labelled 235, 239, 251 and 255 should all be the same (white). You should just be able to distinguish 231 as very pale grey. On the right, 0, 4, 12 and 16 should all be the same (black). You should just be able to distinguish 20 as very dark grey.

If you didn't include ColorYUV(levels="TV->PC") somewhere in the script then all those areas would be identifiable from each other (which is not what you want in this case).
craftech wrote on 5/18/2011, 5:00 AM
Nick,

On further examination, the problem appears to be display dependent.

1. Two different upscaling DVD players to Plasma via HDMI - Blacks are fine
2. Non-upscaling DVD player to Plasma via S-Video. Blacks are fine.
3. Non-upscaling DVD player to CRT via S-Video. Blacks are crushed.
4. Upscaling DVD player to a different CRT via Composite. Blacks are fine.
5. Non-upscaling DVD player to another CRT via composite. Blacks are crushed.

Note: I do realize that upscaling does not take place over all those connections. Upscaling vs non-upscaling was just for reference. I have a bunch of DVD players and TVs for testing.

It's probably insane to try to get this to work on everything like DV to SD DVD does.
But I want to keep trying.

I am working on a different script that I concocted. I'll keep you posted. I am also working on an intermediate no one has tried yet (AFAIK).

Thanks,

John
NickHope wrote on 5/18/2011, 6:38 AM
Lots of variables in the mix there! I don't have so many scenarios to test with. Generally I'm checking with a 5-year old DVD player and CRT TV via composite.

It's probably insane to try to get this to work on everything like DV to SD DVD does.

Are you saying that you can make a DVD from DV footage in a way that behaves the same (regarding blacks) in all those 5 scenarios you mentioned, unlike the results you've had with these scripts?
craftech wrote on 5/19/2011, 2:39 AM
Are you saying that you can make a DVD from DV footage in a way that behaves the same (regarding blacks) in all those 5 scenarios you mentioned, unlike the results you've had with these scripts?
=================
Unlike the results I have had with everything to do with HD to SD. No colorspace problems. No downscaling problems. No square pixel conversions. Barely any color correction (enhancement).

Yes.

John
craftech wrote on 5/26/2011, 6:36 PM
I tried the script with QTGMC in it, but Procoder reads the frameserved avi (avs) signpost file as 852 x 56 every time and that the source is 10 minutes long when in fact it is an hour and 21 minutes.

That's too far off to even take a chance on rendering it.

John

NickHope wrote on 5/26/2011, 7:32 PM
Sounds strange. Just render a few seconds first as a test, rather than the whole thing.

Because it's receiving a frameserved dummy avi file, I wouldn't expect Procoder to know how long the whole video is. But the resolution should be read correctly. Are your project properties 1920x1080? If you paste your script here I'll see if I can work out what's going on.

craftech wrote on 5/27/2011, 3:51 AM
Are your project properties 1920x1080?

Yes. It's actually the entire musical that the clip was from that I sent you for this project.

If you paste your script here I'll see if I can work out what's going on.

It's method 3. Same script.

What I think is going on is that the installation of QTGMC or it's plugin package isn't right. Let me look for error messages in Procoder or Virtual Dub.

Thanks,

John

EDIT: Nick. I was just wading through the posts about QTGMC over at Doom 9 when I came across one of your posts from April where you said that you had the same issue with CCE.

I'm getting errors. CCE tells me "Frame size 852 x 56 is not supported".

How did you fix it?
NickHope wrote on 5/27/2011, 7:03 AM
Ah yes, I remember now. Well I think that error can be indicative of a number of different problems and the frame size that the error throws can be fairly meaningless. In the case referred to in that thread I was using DirectShowSource instead of AviSource, but I didn't have a Direct Show decoder in operation on this computer, so the Direct Show couldn't decode the source. I already had ffdshow installed but I hadn't activated H.264 decoding in it. Once I did that, the script worked.

Incidentally ffdshow gets a bit of a bad press sometimes on the forum, as it tends to get lumped in with codec packs, but I've never had a problem with it. It does take a bit of setting up however.

I've also had that type of error for daft mistakes like the filename in my script not matching the frameserved filename, or simply forgetting to start the frameserver.

Incidentally I haven't given up on a version 2 of the DVD project, but I need to get some other work done first. I've got a few different things to try.
craftech wrote on 5/28/2011, 4:50 AM
The error message that shows up in Procoder and Virtual Dub is:

There is no function named Interleaved2Planar.

That's part of QTGMC.

I put QTGMC-3.32.avsi into the plugins folder for AviSynth 2.5 and I think I followed the other things necessary for the plugins for QTGMC.

But maybe not. Something is wrong.

John

EDIT: I searched this on Doom 9. Someone else had this problem and it was because masktool2 and SSE3Tools were not installed.

I'll check that out.
craftech wrote on 5/30/2011, 1:12 PM
OK. QTGMC is up and running.

There were a bunch of dll files that weren't properly installed.

The multi threading line that I added to the script doesn't seem to run as fast as some other MT scripts when using QTGMC. I tried Method 3 with and without MT, with and without the "Distributor" line. Nothing gets it to render above 60 - 65% CPU utilization. Other MT scripts I have tried will get me to up to 100% CPU utilization, but not this one.

Method 3 with Multi threading is rendering at around .05% of realtime.

John

John
johnmeyer wrote on 5/30/2011, 2:56 PM
Nothing gets it to render above 60 - 65% CPU utilization. Other MT scripts I have tried will get me to up to 100% CPU utilization, but not this one.I haven't been reading everything in this thread the past month, so this may not be of help, but the following script, which resizes 1080i HDV to 720p runs at exactly 50% of realtime on my PC, and uses about 80% of my eight cores. I am using AVISynth 2.6 dated 9/19/2009. The version of AVISynth used makes a big difference in performance of QTGMC.

Hopefully you can get some help by looking at the settings in the simple script below.

#Script to deinterlace video based on QTGMC deinterlacing script

#Designed to produce 1280x720 progressive video



loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")

loadplugin("C:\Program Files\AviSynth 2.5\plugins\SSE3Tools.dll")

loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")



Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC-3.31.avsi") #



#Serve out from Vegas in YUY2

SetMemoryMax(1024)



SetMTMode(5,4)

AVISource("e:\frameserver.avi").assumetff().ConvertToYV12(interlaced=true,matrix="PC.709")SetMTMode(2)



LanczosResize(1280,height)



QTGMC( Preset="Very Fast", FPSDivisor=1, EdiThreads=0, ShowSettings=false ).SelectEven() #12.96 fps through MeGUI



LanczosResize(width,720).Levels(16, 1, 235, 0, 255, coring=false) #Use this with MeGUI


NickHope wrote on 5/30/2011, 9:30 PM
When I finally get back on this project I want to try this script that henryho_hk suggested, or something similar. I think QTGMC is generally OTT for this application and doing the bob with AviSynth's standard bob is going to be way faster. I'm intrigued by the options he's using in the bob function, as well the next line in his script.

At present I don't have my head around the 704 vs 720 height issue for SD, so I need to do some reading on that too. Would welcome any clues on that issue before I get started.

Another thing that popped into my head is that, because TDeint does a strange bob by making all the frames grey in section 1, maybe a script using that needs less or even no low pass filtering, so it might be possible to get rid of the blur/sharpen function without introducing flicker. Something else I plan to try.
NickHope wrote on 5/31/2011, 2:25 AM
As I understand it, LaTo just added a way to use SmoothLevels losslessly for YUY2 clips with version 2 beta 4. This should only work in AviSynth version 2.6 because 2.5 does not support YV16, but it seems like AviSynth 2.6 is preferred anyway for most people doing multi-threaded QTGMC.

So, using SmoothLevels towards the end of the script like this:

ConvertToYV16() #Lossless

SmoothLevels(preset="pc2tv")

ConvertToYUY2() #Lossless


...may well be preferable to using ColorYUV at the start like this:

ColorYUV(levels="TV->PC")


SmoothLevels gives a smooth histogram and hopefully should get rid of the slight banding that was noted in the grey gradient in the tests. Also, it might be preferable to do the ConvertToYV16() early in the script and the other 2 lines towards the end. It depends whether the filters such as QTGMC, TDeint etc. support YV16. I don't know until I try it, or what the speed difference might be.
craftech wrote on 5/31/2011, 4:55 AM
the following script, which resizes 1080i HDV to 720p runs at exactly 50% of realtime on my PC, and uses about 80% of my eight cores. I am using AVISynth 2.6 dated 9/19/2009. The version of AVISynth used makes a big difference in performance of QTGMC.
==================================
Thanks for the reply John.
I am using AVISynth 2.5.

To test Method 3 with multi-threading all I did was add the following to the head of the script:
SetMTMode(5, 4) for my Quad Core editor.

Thus the script was:

Method 3 with Multi-threading


SetMTMode(5, 4)

AviSource("d:\fs.avi")

ColorYUV(levels="TV->PC") 

AssumeTFF

TDeint(mode=1)

bicubicresize(1440,960)

bicubicresize(720,960,-.8,.6)

p1 = bicubicresize(720,480,-.8,.6)

p2 = p1.QTGMC(TR0=1,TR1=1,TR2=2,InputType=1)

p2.blur(0,1).sharpen(0,.51).blur(0,1).sharpen(0,.85)

i1 = assumetff().separatefields().selectevery(4,0,3).weave()

return(last)

Distributor()


It was faster with the Distributor line than without.

It rendered at .05% of realtime and utilized between 60 and 68% CPU.

John

EDIT: NOTE: THIS SCRIPT IS INCORRECTLY WRITTEN


craftech wrote on 5/31/2011, 5:04 AM
This should only work in AviSynth version 2.6 because 2.5 does not support YV16, but it seems like AviSynth 2.6 is preferred anyway for most people doing multi-threaded QTGMC.

Nick,

When I installed the .dll files for the MT version it appeared that it was version 2.5

MT support page.

Or are you suggesting the basic AviSynth 2.6 with the modified 2.5 MT dlls?

John
johnmeyer wrote on 5/31/2011, 10:34 AM
I don't know why you are getting .05% of realtime. 1% would be 1/100 of real time, so 0.05% would be 1/2000 of real time. Thus, one second of video would take 2,000 seconds or 33 minutes to render. Is this really true? I suspect you just mis-typed this.

A more serious issue is that your script is not doing what you think it is doing. The "return (last)" statement will return the results created on this line:

bicubicresize(720,960,-.8,.6)

All subsequent lines are explicitly assigned to variables. The "last" variable will always refer to the last line in the script that was NOT assigned to a variable. You can confirm this by temporarily adding this line to the end of any statement:

.Levels(0, 1, 255, 255, 0)

This inverts the video (makes it a negative) so you can tell immediately what is going on. So, what I am suggesting is that you temporarily change the bicubicresize line to this:

bicubicresize(720,960,-.8,.6).Levels(0, 1, 255, 255, 0)

Open that and you'll see that the video is negative. No surprise there. However, then change this line back to the way it was, and then add the levels statement to the "i1" line:

i1 = assumetff().separatefields().selectevery(4,0,3).weave().Levels(0, 1, 255, 255, 0)

This is where you will get, I suspect, a surprise. What you'll find is that you don't get reversed video. Thus, none of the statements in your script after the bicubicresize are being executed.

Simple fix: change return(last) to return(i1).

FWIW, with your script, with the proper return statement, I get 9% of real time when I do a video analysis pass in VirtualDub.



craftech wrote on 5/31/2011, 2:02 PM
I don't know why you are getting .05% of realtime

That's what Procoder said. In the end it ended up taking 1560 minutes to render a [144 minute video]..CORRECTION: . 82 minute video.

Thus, none of the statements in your script after the bicubicresize are being executed.

That really complicates this entire thread. Method 3 above (which seems to be at least a bit preferred) would be affected by that problem as well since all I did was add the MT statement to the beginning of it.

Or do I have something wrong in the way I used Nick's Script (Method 3)?

John
johnmeyer wrote on 5/31/2011, 2:12 PM
As I stated in my post yesterday, I haven't been reading every post in this thread, so I cannot comment on where the error in the script first began. However, if you do what I suggest, you'll see that the script doesn't return the results of QTGMC and therefore isn't even deinterlacing.

As for what Procoder states, I can't come up with an explanation. Perhaps I just don't understand their terminology. To me, if something renders a 144 minute video in 144 minutes, I would say that it was 100% of realtime. If it took 1,440 minutes, I would say it took 10% of realtime, and so on.