Video Super Sampling ... SPOT please clarify

Comments

farss wrote on 4/13/2004, 2:52 PM
Now all of this makes sense, certainly for generated media. Clearly there's no point to supersampling where the source and project frame rates are the same, even though supersampling will create additional frames they'll never be used.
But this is where it starts to get confusing:
1) If the source media is say 15fps and the project is 30fps, from what SPOT is demoing there should be some advantages.
2) If I'm slomoing 30fps to 50% ideally I need a source frame rate of 60 fps right, so applying supersampling of 2 should create that, admittedly they'll only be interpolated frames.
But so far no one seems to be seeing any improvement in either of these scenarios, which indicates that those additionaol frames are being created but not used.
Now could the answer lie in the use of motion blur. Motion blur of 1 forces Vegas to use the previous frame, we've seen good demos of how this works in conjunction with generated media that has motion. Would this not also work with real video where we are effectively uping the frame rate?
Just to clarify, I'm suggesting that its the combination of supersampling AND motion blur.
I just hope I'm not clouding the issue!
johnmeyer wrote on 4/13/2004, 4:23 PM
Thanks Chienworks,randybrown, cheesehole, and farss. It is all starting to come into focus. The only thing that still puzzles me is that I could have sworn (based on tests I did six months ago, and posted in this forum) that I was able to reduce the flicker in slow motion (25% of normal speed) by using Supersample set to 3.

I will be the first to admit that I may have blown the test. Certainly from everything I now understand after reading this thread, Supersampling should be a complete waste of time and CPU power if I am merely slowing down standard NTSC DV footage.

Anyway, that is the only thing that still confuses me.

BTW, I just did a search on the original Vegas 4.0 announcement literature and came across these descriptions:

Video Super-Sampling envelope
Improve the appearance of computer-generated animation by calculating intermediate frames between the project's frame rate, allowing you to create smoother motion blurring or motion from sources such as track motion, event pan/crop, transitions, or keyframable effects.

Smart resampling
No more guesswork when time-stretching or mixing video framerates. Smart Resampling eliminates the need to manually set resample switches, and works with all formats, helping to prevent strobing and other output problems.
filmy wrote on 4/13/2004, 4:30 PM
As I had said before I have seen some improvment in generated media - in my case roll credits at 24p - by supersample. Also in going 30i > 24p I have seen a quality change. I was only supersampling at 2, so I did not try to see what 8 would be compared to lower amounts.

Going back the the thread of a year ago I did do some tests with 15fps material going up to 24/30 and 720x480 frame size. There was a noticable change in the in between frames, and by all acounts that is what supersample does well - in between frames. Aside from that the whole quality issues came up and really that was the issue a lot of people were confused with. It seemed as though you could take a low quality source and send it through supersample and magicly restore it...now this part has been solved I think.
Cheesehole wrote on 4/13/2004, 4:37 PM
filmy>Going back the the thread of a year ago I did do some tests with 15fps material going up to 24/30 and 720x480 frame size. There was a noticable change in the in between frames, and by all acounts that is what supersample does well - in between frames.

Not in between frames of video. There's simply nothing there. When we say Ithat super sample is good at sampling in between frames we are talking about project frames. Supersampling needs something to sample in order to create a better image. There is no magic here, no sophisticated image algorithms, just plain old blending. Using it on frame rate changes may get you a slightly "different" look than resample, but it isn't any more sophisticated than resample or motion blur.
johnmeyer wrote on 4/13/2004, 4:56 PM
There is even more information on these features in a document at this link:

http://mediasoftware.sonypictures.com/products/Vegas4NewFeatures.pdf

However, the PDF file is corrupt (Sony, you may want to check this). Fortunately, I was able to get enough of it in Google that I could type the results into my word processor and transfer here. Unfortunately, it once again seems to contain information that contradicts some of what I thought I was hearing earlier in this thread. Here are the paragraphs from that 15-month-old Sonic Foundry marketing piece (I added the italics):
====================================================
Video Supersampling
Video Supersampling calculates intermediate frames between the project's frame rate to improve the appearance of computer-animated motion. This works especially well with video that contains slow motion, and also works in conjunction with the motion blur envelope to further improve the appearance of computer-generated animation sequences. Choose Video Supersampling from the shortcut menu, then adjust the envelope to achieve the results you want. Keep in mind that video Supersampling can significantly increase your project's render time since you are asking your project to render more frames per second. You'll need to find the right trade-off between the benefits of Supersampling and increased render times.

Smart Resampling
Many times, the frame rate of a video event does not match the Vegas project frame rate. This can happen when the video in the event was shot at a different frame rate than you are using in your project, or when you use a velocity envelope to create slow motion. You can determine how Vegas treats video with frame rates that differ from the project rate.

Right-click on an event, and choose Switches from the shortcut menu. At the bottom of the shortcut menu, notice that Vegas defaults to Smart Resample. With this option chosen, Vegas determines whether the calculated frame rate of the video in the event matches the project frame rate. If it does, Vegas does no resampling. If the frame rates are different (and your project frame rate is at least 24 frames per second), Vegas resamples the video in the event. This means that Vegas looks at two frames, and interpolates to create a new frame that acts as a transition between them. This process can help solve the problem of video that looks "jittery" due to low frame rates.
Cheesehole wrote on 4/13/2004, 5:22 PM
The phrase you highlighted was probably a mistake, or was very poorly worded. Read the Vegas help or the Vegas manual for accurate information about these features.
johnmeyer wrote on 4/13/2004, 6:00 PM
cheesehole,

If the information from the Sony marketing literature that I quoted in my previous post was a mistake, that would explain a lot of the confusion.
Spot|DSE wrote on 4/13/2004, 6:27 PM
Downloadable media, and re-done screenshots are up on the tutorial page for you to play with at home. I opened the .mov file in Quicktime rather than Vegas, it's displayed at double size and screenshot at same. Similar frame is displayed in Vegas, post process and render. You can download the media file directly from the Library of Congress site (the link is in the tutorial) or you can download it directly from the Sundance site. Unfortunately, the finished file cannot be located, nor can the veg, so I built this project again. You'll have to decide for yourself whether this is snake oil or not. To my knowledge, there is no other location to grab this video clip, so you'll have to take it for what it's worth. Download the original and the processed clip.
Regarding Supersampling, I don't believe I said it was a spatial process. If I ever did, I'd like to know where, and if I did, I apologize. It's pretty clear in the manual and original white paper, wherever might now reside. I couldn't find it.
Thank you to the three persons who sent mail with name calling and the one racial slur. (Of course, you've sent that one before, can't you be more creative?)
Skevos, thank you for a civil dialog.
For those that didn't clearly understand the tutorial, my apologies. It was written on a long flight from Sydney to Hong Kong. Maybe I was tired, maybe I was on a caffeine high. I don't recall. It was over a year ago. But you can see the original and finished results for yourselves on the tutorial now that the media is there to download. http://www.sundancemediagroup.com/tutorials/supersample.htm
philfort wrote on 4/13/2004, 9:35 PM
Well I guess you sort of imply it is a spatial process here:

http://www.sundancemediagroup.com/tutorials/supersample.htm

by using the term "low resolution" along with low framerate:

"When used to upsample low resolution/low framerate video such as this original mov file seen in Fig 1"

and also by showing a before and after frame, where the after picture is of much higher spartial resolution. That seems to imply that you can improve the spatial resolution of footage with super-sampling (maybe only slightly, as others have indicated, if you combine many frames of footage that is looking at the same objects).

Also, the links to the before and after video are different videos. "before" is "Buffalo Dance", and after is "Sioux Ghost Dance", so there is nothing compare - except to note that the resolution of the original "Buffalo Dance" is much higher than your very pixelated "before" screenshot of "Sioux Ghost Dance", so maybe it is your screenshot that is somehow messed up, and the original video was of much higher quality that what is depicted in the screenshot.

It seems like the tutorial is suggesting that supersampling turned the "before" into the "after". There is just no way I can believe that (unless I could see if for myself). (For example, that rope or tie hanging down from the vest on the guy in the middle doesn't exist in the "before" footage).
mark2929 wrote on 4/13/2004, 9:41 PM
I Think the important thing is what the blur is doing many times I have seen drastic improvements in footage using commotion really taking out the jaggies then with supersampling I believe it is possible to achieve higher resolution. I say resolution because the blur is removing the straight jagged lines. I think without the source footage and the original experiment... Exactly documented... Its wrong to say with "certainty" it can be proved right or wrong, another case for the X Files...

My TV HAS Sony Digital Reality Creation. Which adds more Pixels to the Picture. I was wondering if that is an algorithm that could somehow be incorporated into Vegas... Perhaps not.... I dont know the Logistics .
philfort wrote on 4/13/2004, 9:49 PM
Each pixel of video in your "before" screenshot is three pixels wide, which suggests something is wrong with the screenshot (I could understand if they were 2 pixels wide, since you say it is showing at 200% ... however - even then, quicktime upsamples so you don't see the pixelation evident in your screenshot... the pixels will be blurred, not sharp - at least my quicktime player does that at 200%)
Spot|DSE wrote on 4/13/2004, 10:22 PM
In my hurry I linked to the wrong file. I uploaded many of these that came from the same Black Maria period. So, if you want more comparisons, there are a few more up there.
Cheesehole wrote on 4/14/2004, 3:01 AM
Spot> You'll have to decide for yourself whether this is snake oil or not.
Why don't you just give us something reproduceable? Are you just playing games now?

Spot>Regarding Supersampling, I don't believe I said it was a spatial process. If I ever did, I'd like to know where, and if I did, I apologize.

Spot are you serious? Take a look at your own tutorial pages. All you had to do is say oops... I made a mistake, and then fix it.

I'm disappointed by the way you were treated - you certainly need a thick skin to do what you do, but in one breath you tell us how many people you've demonstrated this effect to, and in the next you say you never said it does that. Who do you think you are kidding?

You are still using misleading (wrong) images and you are still making claims that are not backed up by any reproduceable results.

This page is sort of a mix of *bs* and facts, the end result being totally misleading:
http://www.sundancemediagroup.com/tutorials/supersample.htm

THE RESULTS YOU CLAIM CANNOT AND DID NOT OCCUR. SUPERSAMPLING HAS NO EFFECT ON EXISTING VIDEO AND CANNOT "UPSAMPLE STILLS"

I've downloaded your examples, and followed your steps (including the very latest set of steps in your very latest tutorial), and in every single case the results did not (and could not) achieve the results you claim.

Sony claims:
"Video supersampling can improve the appearance of computer-generated animation by calculating intermediate frames between the project's frame rate, allowing you to create smoother motion blurring or motion from sources such as

My results match Sony's claims.

And THIS page is just total *bs*.
http://www.sundancemediagroup.com/tutorials/supersample_2.htm

That text could never have gone from being a blurry mess - with letters bleeding into each other - to nice sharp text. At least not the way you claim. What really happened was It was sharp in the original video, and then it was blurred by the bad resize algorithm in the preview window (which you grabbed a shot of) giving the impression that Vegas had turned a blurry mess into a perfectly watchable film.

We assumed it was an accident but now if you want us to decide whether it is snake oil - well it is. No different from when mechanics pour oil under your car and then "fix" your oil leak.
BD wrote on 4/14/2004, 4:18 AM
Here's a case where Supersampling actually can improve a single video frame:

Using Pan/Crop to "zoom" during a wedding ceremony generated an individual frame that was pure white in alternating scan lines, occasionally when the photographer's flash popped. SuperSampling at the max (8) setting fixed this, rendering those frames to look normal instead of obviously artificial.

Brandon's Dad
Jay Gladwell wrote on 4/14/2004, 5:15 AM
GEE GOLLY DARN WHIZ! For those like me that are technically challenged, this thread certainly has been a whopper!

You guys and your grasp of this stuff never cease to amaze me. Each and every one of you have my awe and respect, truly. If I can understand ten or fifteen percent of what you've said here, I consider myself lucky!

This will require a printing out of this thread and then pouring over it many times to get to where it can be understood (by me, that is!).

Thanks to one and all!

J--

Spot|DSE wrote on 4/14/2004, 7:35 AM
Like I said earlier, you're right. It's all BS. I invented it. Apparently I get kicks out of doing a project in another app and then claiming I did it in Vegas or something. What on earth could I possibly gain by faking this process? You've got the original whether it comes from my site or from the LOC site. You've got the finished MPEG. The screenshots on the updated page are exact screenshots from ALT+PrintScreen, no editing excepting where I cut off the top of the video preview window for NDA reasons.
You tell me how I did it.
Cheesehole wrote on 4/14/2004, 7:50 AM
dis·in·gen·u·ous
adj.
1. Not straightforward or candid; insincere or calculating
2. Pretending to be unaware or unsophisticated; faux-naïf.
Spot|DSE wrote on 4/14/2004, 7:58 AM
Simply amazing how quickly you resort to name-calling when you can't successfully complete something. It's my fault you can't do it?
BTW, the second supersample page posted while I was in Hawaii, was created from a downloaded file in a hotel room. I did that on my laptop. Both screenshots are from the Vegas preview window, no resizing. Excuse my butt for trying to answer a question while I was on vacation.
Again, you've got the original file (I assume) and you have the finished MPEG, I assume. You tell me how I arrived there. You are so damn sure of how I DIDN'T arrive there, explain to me how I got there. To my knowledge, there is only one means of accessing that quicktime file.
Either way, I've tried hard enough at this point. There are other things to be done this week.
winrockpost wrote on 4/14/2004, 8:33 AM
Thanks a lot guys,, I' sittin here trying to come up with a way to deduct a diving trip to the Bahamas,, and I start reading this thread,, now I have to drive back to the studio, follow a tutorial, and see what kind of results I get.

Guess I'll file an extension
JohnnyRoy wrote on 4/14/2004, 8:38 AM
I think I have found a logical explanation. please read:

> You'll have to decide for yourself whether this is snake oil or not.

Supersampling is not snake oil. I have used it on projects and have been impressed with its ability to clean up low-res video. I do believe, however, that the method in which it is being presented in the tutorials is what’s causing all the confusion.

> I opened the .mov file in Quicktime rather than Vegas, it's displayed at double size and screenshot at same

Actually this is unscientific and it explains a lot. We are no longer comparing apples to apples. You have introduced another variable (the Quicktime viewer) and in order for this to be a fair comparison, you should render file final results to a Quicktime MOV file and view the results in the same viewer. If you don’t think this is significant, read on...

So what is the explanation?

I have to agree with Skevos that the second set of pictures Spot is showing in Supersample 2 are misleading. I have recreated Spot’s experiment EXACTLY and can explain what’s happening. You can try this at home:

The shots in Spot’s Supersample 2 tutorial are taken directly from the Vegas preview window which is set to 360x240 on frame 60. When I view 4024.mov in my Vegas preview it looks like a blurry mess just like Spot’s tutorial. However, if you tear the preview window off so it floats and view the file at 720x480 you will see that cheeshole was right, there is much, much more information there. Almost as good as the the supersampled version without ANY processing whatsoever. (although super sampling does smooth it out nicely and make a better result in the end)

For some reason, once you process the file with supersampling at 720x480 MPEG2 and drop it on the Vegas timeline, the preview looks pristine? So what you are seeing is the Vegas preview window’s inability to display the original 160x120 file clearly at 320x240. This makes the original look much worse that it really is.

So there is no magic here. The original file contains more information than the preview and tutorial screen shots are leading you to believe and the preview favors the processed file (I’ll tell you why later in the extra credit section).

The scientific method :

For this experiment to be scientific, we must compare apples to apples and have supersampling be the only variable. If you take the original 4024.mov file and do NOTHING to it other than to render it as MPEG2 NTSC DVD. Now drop that MPEG file on the Vegas timeline next to 4024.mov and you will get almost the EXACT same results as the one that was supersampled in the tutorial. One preview will be blurry and the other pristine.

Now process the 4024.mov file with supersampling, render it to MPEG2 NTDC DVD and compare it to the unprocessed MPEG NTSC DVD file you just made. The difference is what supersampling did. THAT my friends, is the scientific results of super sampling. Apples to apples, 720x480 MPEG2 to 720x480 MPEG2. An improvement? Yes. As dramatic as the tutorial pictures? No.

Extra credit work:

I wanting to understand what’s going on here I did a further experiment and rendered 4024.mov to an uncompressed 160x120 AVI file and low and behold it looked great in the Vegas preview window. So the pixilation you are seeing in the preview window screen shots of the tutorial are a direct result of Vegas not being able to upscale a Quicktime files in the preview window! Probably a Quicktime library deficiency.

I truly believe that Spot was not aware of this deficiency in previewing Quicktime and was only reporting what his eyes saw.

Mystery solved Jinkies Scooby... all this talk about super sampling is making me hungry. I can go for a super sample smorgasbord down at the greasy spoon. Let's eat!

~jr
johnmeyer wrote on 4/14/2004, 9:06 AM
JohnnyRoy,

Thanks for clarifying what is going on.
=================================
Sony.

The message to your employees from this thread should be loud and clear: Even the most talented and knowledgeable users can't fully figure out what Supersampling and Resampling are doing, and when to use them. It is up to you -- either in the help file, manual, readme, or white paper -- to provide a thorough, clear explanation that can be understood and used by not only the video intelligentsia, but by mere mortals as well.

You have even more than the usual obligation to do this because, as I showed in the quotes from your marketing literature in my previous quotes, much of this confusion has resulted from your own mis-statements.
Skevos_Mavros wrote on 4/14/2004, 9:41 AM
Hi again Spot,

You said:

(SNIP)
Skevos, thank you for a civil dialog.
(SNIP)

Heh - well, one of us was civil! :-) On the other hand, one of us said rather hostile and rude things in private emails, including pretty crude name calling. I'll let others guess who was the rude one out of us two... ;-)

ANOTHER ATTEMPT TO EXPLAIN
In any event, there's no hard feelings here Spot, especially since your updated tutorial page merely confirms your errors (notice carefully that I say you committed errors - I'm not accusing you of deception or lying).

Rather than type five thousand words here explaining it one more time, I've typed slightly less words on this temporary web page, explaining where you've got it inadvertantly and honestly wrong. Sorry to take my full reply off the forum but if the forum allowed inline images it would have been so much easier, and having the images right on the page make it clear what I'm talking about.

SUMMARY
Here's the short version of that page for those contemplating whether or not to bother going there and reading it:

. Spot did not use that small Quicktime file to create that higher-res MPEG video, he used a larger MPEG file (of the same footage) as his original video (the large MPEG file is available on the same Library of Congress server as the small Quicktime). When creating his tutorials I assume he simply forgot how he originally did it and honestly but erroneously used an image of the small Quicktime as his original or "before" source. Details and links are on my temp web page.

. In his second tutorial Spot inadvertantly used a before image that looks much worse than the video it refers to actually is, due to a small bug in the Vegas Preview Window. Details are on my temp web page.

HAPPY TIMES ARE HERE AGAIN
I hope that clears things up, and we can get back to recognising supersampling as a great feature for what it can really do.

And one last time Spot - I like you man! Stop being so stubborn! :-)

All the best,


Skevos Mavros
http://www.mavart.com
Spot|DSE wrote on 4/14/2004, 10:00 AM
1. I did NOT use an MPEG from the site, and the two files now on the tutorial page were created yesterday and this morning from QT files downloaded from the LOC site.
2. I did not call any names. I said that "attacking me when I'm not around to defend myself is chickensh**. I feel that kicking at someone when they're not around is indeed poor form and cowardly behavior.

On the other hand, your post contains false statements. I've not used the MPEGs don't have any of them on my server, but do have a number of the quicktimes because they were part of a disk we used in a school training situation.
While I may not have explained well, and maybe even have misunderstood the tool, I don't believe that grants you the right to put words in my mouth or state that I used media I didn't use. Further, that doesn't grant anyone the right to call me "prairie nigg**" "Liar" "Thief" and any number of other names that came both privately and in posts.
You know, this is kinda interesting....
It's like standing in a room completely naked, and everybody commenting on the size of my di**. If I had a big di**, some would comment on how I used a patch or something else to make it big. If I had a little di**, people would be laughing at that. But no one else dares take their pants down. And instead, they make comments about how their own di** arrived at a particular size, even posting false, BS webpages about how they arrived at having a big di**.
Post my mail to you, Skevos, i've nothing to hide. All you are doing is adding to the bullsh** by posting what you THINK I did.
Tell you what. At NAB I'm busy as hell. but if you'll find me at the show, I'll do this for you in person, live on the floor.
Cheesehole wrote on 4/14/2004, 10:03 AM
Skevos I thought I was losing my mind. Your methodology is commendable, and more importantly... indisputable :D I knew there had to be another source for that file.