Supersample affects slo-mo. Yes it does!

johnmeyer schrieb am 16.05.2007 um 02:16 Uhr
OK, lots of post over the years about how the supersample envelope on the video bus does not affect video, but instead only acts on media generated by Vegas.

Well, I have now found out that this is not true.

1. Take an m2t HDV file (although my guess is that this will work with regular DV as well).

2. Set the playback rate to 0.25, and put a velocity envelope on the event and set it to 25%. Thus, we are going to play back at 1/16 normal speed.

3. Disable resample for the event.

Now, render to an MPEG-2 file using the normal DVD Architect NTSC (SD) template and view the result (after preparing to a DVD) on a normal TV monitor. In fast motion sequences, you should see "double vision" because both fields are being displayed simultaneously.

Now, add a supersample envelope to the video bus, and set it to "2." Repeat the render as in the previous paragraph.

Note that there is no longer any "double vision"!!

This is actually the effect I was trying to achieve, namely a continuous series of still photos.

Here are the results. The first is without supersample. Notice the "double vision" on the club head at the top of the swing. By contrast, the second version, with supersample set to 2, shows a perfectly clean club head motion. These two files, when encoded to interlaced MPEG-2 and displayed on an NTSC monitor show exactly the same thing as in these YouTube clips.

No Supersample



Supersample set to 2



Kommentare

farss schrieb am 16.05.2007 um 02:36 Uhr
Ok, so in the second sample it looks like you've lost one field.
What I'd like Supersample to do is create 'motion blur' between those frames like it does with Vegas generated media.

Bob.
TheHappyFriar schrieb am 16.05.2007 um 04:20 Uhr
i believe you use motion blur + super sampling to get that effect. I don't have any good footage handy to test it though.
johnmeyer schrieb am 16.05.2007 um 04:29 Uhr
i believe you use motion blur + super sampling to get that effect.

No, the ONLY difference between those two clips was the addition of the supersample envelope on the video bus, which I then dragged upwards until the setting read "2." I most definitely did not use motion blur on either clip (the YouTube video looks blurry, but that is courtesy of YouTube, not Vegas).
jrazz schrieb am 16.05.2007 um 04:38 Uhr
John, is that 60i footage that you started with? What was your shutter speed set to?

j razz
johnmeyer schrieb am 16.05.2007 um 05:58 Uhr
John, is that 60i footage that you started with? What was your shutter speed set to?

Yes, I started with HDV from my FX1, 1440x1080 60i video. I took it this morning in the Carmel fog, and didn't have time to set up my lights, so I had to crank the gain way up, and even then could only manage either 1/250 or 1/500 (can't remember now, although I could look at the info on the tape ... ).
farss schrieb am 16.05.2007 um 06:06 Uhr
John,
he was replying to my post, how to smooth out the motion. What I'm talking about is getting interpolation between the frames that the camera captured.
In fact the first sample looks closer to what I'd want than the second, the second video looks like you've lost every second field.

Bob.
johnmeyer schrieb am 16.05.2007 um 06:42 Uhr
In fact the first sample looks closer to what I'd want than the second, the second video looks like you've lost every second field.

Yes, you are right. I was trying to get the "60p from 60i" trick by converting fields to frames (losing 1/2 the vertical resolution), but it appears that supersampling -- which definitely did affect the results -- actually eliminated the odd or even fields.

I finally figured out how to get the effect I wanted (see below). Pretty good, eh? (This is 1/2 speed, not 1/16 as in the previous example. However, I was able to keep both fields).





farss schrieb am 16.05.2007 um 06:49 Uhr
That looks better!
Bob.
DJPadre schrieb am 16.05.2007 um 07:11 Uhr
supersampling has ALWAYS affected video "changes" ie, multiple passes and recalculations prior ot export.

from slow mo, to frame rate conversion, supersampling is one of teh main reasons we sold so many copies of vegas to those that transcode for other formats (ie PAL to NTSC, 50i to 24p etc etc)

I dont understand why the big deal now considering this feature has been in existance since v4 (from memory)

thing about slowmotin is that when you do slow the footag down not only are u emulating the slower frame rate, but ur also emulating the slower shutter speed.
anythign less than 1/120 will give u blur if u slow it down, and even if u were to interpolat ur fields into frames, those fields wil be blurred also, hence the blurred frames, so teh slowmotion is in fact accurate based on the shutter speed.
There is only one program which DOESNT affect the shutter and that is dynapels slowmotion, which optically interpolates each frame/field. Ie, its NOT predictive. it analyses the frames prior to and after the interpolated frame. Now this is what we al believe vegas does, but it doesnt, UNLESS u supersample..

Supersampling works much like audio oversamplings.. in fact, from what i understand its identical to audio oversampling... only done with video
The multiple passes made offer a far more accurate rendition for teh final output.

The predictive analysis in vegas cannot be determined even with supersampling, as im yet to be advised by anyone in the know whether it is predictive (ie, what should be in the "in between" frames based on the before and after field/frame) or optical based, which analyses the entire frame (be it progressive or interlaced) and draws a new frame from scratch.
Vegas interpolation is sometimes weak and there have been many instances where i have experienced field or frame doubling.
Try slowing down a progressive scan clip slower than 80% and tell me if u think that its acceptable.
I personally dont.
However, run the same clip through Dynapel SLowmo and you WILL see the difference..
Its not to say Vegas sux.. it doesnt.. however like most apps, it cannot handle complete frame regeneration within a slow mo'd clip.. unless ur refering to frame regeneration using an interlaced source for a progressive output.

The amazing thing about it is taht its all in realtime and u can also preview the results prior to rendering.. (FCP doesnt even do that.. )

but yeah.. i mean fair enough if u offer vegas an even multiplication to decipher interlaced slowmo @ 50% and run SS u will def get good results... albeit the renders will take as long as the amount of passes you choose.. (ie supersampling at 8, ur render will take 8 times longer to complete)
But if u try that same thing with progressive footage, it will puke.

So the question here is what kind of sampleing and regeneration algorythms are used for vegas slowmotion?

In addition, IF the shot has a hilight or blow oout, that highlight will indeed begin to flicker.. and thats not becuase its blown out, its becuase Vegas hasnt or cannot calculate the correct frame data for taht inbetween frame.

Perfect example is this golf shot.. If teh sky was overcast, and was nigh on clipping the 255 peak, you would notice flicker, because the newly drawn frames created by vegas will inadvertantly exceed that threshold. Dotn ask me why.. it jsut doesnt calculate highlgihts very well.. in all honest, i dont think it can calculate thresholds going over the legal limits.

So this in itself proves that vegas interpolation isnt as accurate as we'd like it to be.. maybe this is an 8bit thing.. i dunno but ive never had issues with the properly shots clips, or of there is a blown out clip, u must run Broadcast safe colours.

In additin, a tip here is to NOT use track motion or event pan crop while using slowmo..
do ur slowmotion, render out, then use track motion.
Vegas just cannot handle the maths for some reason..
farss schrieb am 16.05.2007 um 07:58 Uhr
I dont understand why the big deal now considering this feature has been in existance since v4 (from memory)

I think the issue keeps coming up because ever since V4 we've never been told exactly what it does and how it does it. The common wisdom has been that outside of Vegas generated media it does nothing, to date I don't recall anyone proving otherwise and Sony have never said anything that I can recollect. In all the training courses and Vegas presentations I've been to it doesn't even rate a mention.

If there really is some magic in it I'm all ears, I do quite a bit of PAL to NSTC conversion and in my quick tests I've not seen any advantage to using Supersampling.

Bob.
DJPadre schrieb am 16.05.2007 um 09:14 Uhr
Bob, I myself have had several trainees/companies move to Vegas because of it.

From my understanding of supersampling, it works much like oversamlping, where analysis is made and then "reconfirmed' a number of times in turn offering a more accurate rendition on the output as any anomolies in the multiple calculations are found then averaged out for the output.

Much like audio, the samples taken are 2 to 8 fold (in this case) so any changes made to a video clip are processed or calculated up to 8 times..

im sure if calculated, the minute differences would be noticable, but on the real world side of things supersampling DOESNT make much of a difference unless the maths "makes sense" to the app.
Lets say youve supersampled a pan crop and changed teh frame rate and playback rate..

which ever goes first (pan crop in most cases as this is teh central frame the other elements will be working from) is calculated first, however vegas CANT do the additional maths to slow it down, or change the final output.. for some reason, vegas jsut pukes when u throw this basic task to it. It will do the pan crop, but the slowmotion wont be accurate as the frame prediction is processed at the same time.
So by the time the interpolated frame is drawn, the pan crop has "moved" (im refering to Keyframed Pan crop here) so as that frame is slotted where it belongs within its new frame rate and playback rate, the frame itself is out of sync to the rest of the sequence ( due to the pan crop see)

So, what happens is that Vegas sees teh frame All at once.. it does the maths for EACH part.. pan crop
slow mo
frame rate
then slots it in.. so the "new" interpolated frame is where it would have been if the pan crop didnt exist or more accurately the new interpolated frame is where the keyframe is for teh pan crop AT THAT TIME..so in fact its not predictive.. as it hasnt predicted the pan crop..
am i making sense? Then as the next frame comes to play, it calculates it again.. this time the pan crop is now where it is.. in teh new position... and then the slowmo frame and the rest are also calcualted.. and it goes around and does it again..
so again teh slowmo looks wierd because the actual frame is no longer in place as the NEW frame is where it would be BEFORE the pan crop took place..
am i making any sense here?

Dont ask me why supersampling doesnt handle this or whyvegas itself cant handle this, but to me it seems that the 2 are very different algorythms which confuse the maths i guess..
who knows.. i might be totally wrong, but ever since V3, vegas has had issues with doing pan crops with amended frames or playback

Do it in a sequence of renders and theres no problem, do it all at once, and youll have issues
farss schrieb am 16.05.2007 um 10:14 Uhr
Took me a while to get the gist of what pan/crop had to do with PAL to NTSC conversion but finally the penny dropped.
The conversion requires both rescaling of the image and frame rate conversion!
What you're saying is doing these as separate renders works better?

My understanding is that for rescaling to do a good job requires de-interlacing as half the resolution is in each frame however there's the obvious problem of temporal separation and the pixels getting moved around. This is where the boxes from S&W etc do their magic. The best we can do in Vegas is use either Blend or Interpolate else some real nasty jaggies start to appear. In this instance I fail to see where or how Supersampling comes into play.

The frame rate conversion in Vegas is a simple temporal averaging system, it's effectively supersampling anyway. Say I need to create a new frame at 50% of the way between frame 1 and 2. This new frame is an equal mix of those two frames. If it's 20% between then 80% of frame 1 and 20% of frame two are mixed. This would explain the problem you've seen with highlights going bad.

What perpexes me is in all the disussions over the years about PAL to NTSC conversion Supersampling has never been mentioned, setting a De-Interlace method has gotten a lot of press though. I've done plenty of PAL to NTSC conversions with no Supersampling and they've held up to critical examination. All I've used was a Blend which produces slight motion blur , I think I'm the only one that's noticed though.

The only time I've seen Supersampling have an impact is on Vegas generated media when it's being animated. In this case Vegas calculates a number of tween frames based on the Supersampling value to create a crude motion blur. When Vegas has no idea of how things have moved in a regular video I can't see how it can do anything.

Bob.


Edit: The Vegas 7.0 documentation is in fact quite clear on Supersampling "...it CANNOT improve the appearance of existing video"

DJPadre schrieb am 16.05.2007 um 11:05 Uhr
The conversion requires both rescaling of the image and frame rate conversion!
What you're saying is doing these as separate renders works better?

((Precisely))

My understanding is that for rescaling to do a good job requires de-interlacing as half the resolution is in each frame however there's the obvious problem of temporal separation and the pixels getting moved around. This is where the boxes from S&W etc do their magic. The best we can do in Vegas is use either Blend or Interpolate else some real nasty jaggies start to appear. In this instance I fail to see where or how Supersampling comes into play.

((Well in this instance, interpolation will work better.. in the case of an AXIO or a matroxRTx100 or RT2, as THIS element is hardware dependant, those jaggies ring true even more profusely, than they do in Vegas.. as theyre hardware dependant, whereas vegas is SW...
However in premPro2, with a GFX card running.. lets say a page curl, those jaggies arent as profound, as the image is redrawn through the GFX card renderer. In turn being "corrected" through antialias filters on the card itself.. those jaggies are still there, but the card has fixed them...
For vegas, its all CPU maths, for HW, its hardware dependant, depending on teh card and antialiase filters running. some have 2x other 4x, others 8x.. again these work in a similar fashion to supersampling, however the supersample may not affect the "existiing" footage per se (ie u cant make crap look good) however it DOES affect the interpolated or "generated" material such as temporal frames or fields.. thus the ORIGINAL frames (of your slow mo or your frame rate conversion from pal to ntsc) are unafected, however the newly generated temporal frames to achieve the output are supersampled.
In turn, the maths thats happening is based on the original source, BUT only affecting the new generated material.
Phew.. lol ))

The frame rate conversion in Vegas is a simple temporal averaging system, it's effectively supersampling anyway. Say I need to create a new frame at 50% of the way between frame 1 and 2. This new frame is an equal mix of those two frames.
((Theoretically that is correct, however it depends if your refering to interlace or progressive. In progressive, the in between frame (lets call it 1a) isnt drawn correctly.
It never has been. You will notice the luminance, and colour gradation IS an average, however coupled with the orignal material the differenctial of that added new frame is noticable.

U might notice this moreso on slowmotion material, than lets say a pal to ntsc conversion. as the frame rate is slower u see, in addition to the emulation of slower shutter... in addition to the emulated or more precisely, the "new" frame DOES NOT carry the same image data as teh orignal. Therefore, when u supersample, that "new" frame SHOULD be calculated as many times as u specify PRIOR to being drawn.
Now this is all theory.. but im yet to have a specific response from anyone about this.. ))

If it's 20% between then 80% of frame 1 and 20% of frame two are mixed. This would explain the problem you've seen with highlights going bad.

((ya))

What perpexes me is in all the disussions over the years about PAL to NTSC conversion Supersampling has never been mentioned, setting a De-Interlace method has gotten a lot of press though.

((i guess this is due to res and frame differences, as the deinterlacer/blender kicks in whenever there is a change to the original. Im yet to also have verification of this, but again, the theory is sound.. especially considering were actually scaling by doing these type of conversions.
One thing though is that ive stopped doing complete conversions like this unless im providing masters on tape or whatever other foramt may be required for edit. If however im provding DVD, the region free aspect of the discs we create (save for the pressed units) take care of the issue (pal vs ntsc) so we dont have to worry about it. If however were creating glass masters for US and UK, then we have no choice but to make these conversions and the pressing house wont do it. ))

I've done plenty of PAL to NTSC conversions with no Supersampling and they've held up to critical examination.
((Agreed. Even without SS, Vegas works a treat, but there are many out there who are so nitpicky (especially those wanting to output to film.. starting with 50i or 25p) that the supersampling can (to their eyes) make a difference. Personally, i dont SS Pal to NTSC... and the only time i do SS anything is with Interlaced footage which needs slow motion. once i create a new clip, i reimport it.. if its progressive that needs slow mo, i dont even bother with vegas, as it cant handle it))

All I've used was a Blend which produces slight motion blur , I think I'm the only one that's noticed though.
((Thats usually teh case matre.. were our own worst critics... lol))

The only time I've seen Supersampling have an impact is on Vegas generated media when it's being animated. In this case Vegas calculates a number of tween frames based on the Supersampling value to create a crude motion blur. When Vegas has no idea of how things have moved in a regular video I can't see how it can do anything.

((WEll thats the point see.. it only sees THAT frame and WAHT needs to be done to THAT frame.. it doesnt look ahead... theoreticcally interpolated slowmotion SHOULD be silky smooth if it did in fact calculate BOTH before and after frames, but for some reason it cant interpolate furhter than 50%
In fact when training ,i usually tll people to slow mo in increments of 10% Then reimport, tehn slowmo again, and around and around until tehir as slow as theyre happy with.
This way, the 90% clip is actually calculcated from 100%
ie, only 10% of teh new clips is actually processed while the remainig 90% of teh frames are recaulcuted to their new position.
Now do it again.. ie your 90% speed clip is reimported then slowed AGAIN at 10%
this will give you an 80% sped clip... BUT those 80% are far more accurate now as the orignial source (90% clip) has many more source frames to use a reference)
If however you were to drop a 100% clip down to 40% youd see some incredible stutter..
But using the theory of rendering in 10% increments, the interpolatoins SHOULD be far more accurate, as the source material is still "close" based on the 'stretch"
How do i explain this.. umm..
OK, lets say youve slowed somethign down to 75%
25% of what will be seen has to be regenerated to confrom tot he project frame rate..
Now IF you slowed to 87.5% instead of 75, only 12.5 needs regeneration.
NOW, looking at both of these, the 87.5% WILL be smoother as the material still carried many more SOURCE frames.
However the 75% copy, using twice as many generated frames, WONT be as smooth, as Vegas DOESNT interpolate its own generaled material.
Instead it will say..
"OK i need to slip 3 frames (f2 f3 f4) in here..between 1 and 5.
What i have is this colour here here and here.. "

it then caluclates the WHERE in those 3 frames this colour needs to go BASED on those colours position in Frames 1 and 5...
Its using the 1st and 5th frame as reference.. Its NOT using frame 4 for frames 3 reference.. and this is where accuracy, or lack of comes into play...

Either way, if u use the theory of the 10 percentile for slowmo or actual interpolation (pal ntsc), you'll have many more source frames to work off when it comes to your output. Thus, your output will have more source frames to work off to begin with, in turn being a more accurate representation of the source (coz there is more of it, or of what there IS of it, it in itself is more accurate due to the source being so dense during ITS process... )

Phew.. hope im making sense..

either way, for serious slowmotion, i use Dynapel slowmotion.. im yet to see anything as accurate as that..

farss schrieb am 16.05.2007 um 11:54 Uhr
Remarkably enough, it makes sense to me. I'd probably want to do some simulated tests as I don't have a good enough grounding in maths to construct a mathematical model of what you're proposing.

All that aside when a client wants a NTSC DVD I always say I can send it to a post house with the right gear and a fee to match or I can do an acceptable job for way less. So far they've always taken my method, probably says a lot about the quality of the clients I attract :)

My personal preference is if the material isn't already shot is to shoot 25p, works nicely for PAL and with a 4% slowdown it works just as well for NTSC, especially if it's HD.

I guess we should apologise to John, he's still got a guy trying to hit a ball in slomo and this isn't helping his golf swing.

Bob.
DJPadre schrieb am 16.05.2007 um 12:32 Uhr
lol agreed..
TheHappyFriar schrieb am 16.05.2007 um 13:38 Uhr
The common wisdom has been that outside of Vegas generated media it does nothing,

Ho do you get generated media to "stutter" so you can super sample it? I tried pan/crop, moving via text editor, animating noise texture's & checker boards, then I would slow it down as much as possible via CTRL+Drag but it was always smooth.

Maybe this is something that was "fixed" after Vegas 4/5 (in 6?) and that's why it works the way we think it should?
DJPadre schrieb am 16.05.2007 um 15:48 Uhr
try it with a DV file, not generated media.. u might be able to reproduce it that way
johnmeyer schrieb am 16.05.2007 um 16:51 Uhr
OK, I've finished my tests, and here are my conclusions:

1. Supersampling only affects slow mo IF Smart Resample is disabled for the event that is slowed. If "forced" or "smart" is selected, then supersample has no effect.

2. As Bob pointed out, the effect of the supersample in this case seems to be to eliminate either the odd or even fields (not sure which). That is hardly a great thing.

3. Changing the supersample to higher numbers has absolutely no impact, either positive or negative. Nothing happens at 0 or 1; the field disappears at 2; then nothing more beyond that.

Thus, I think this is an interesting, but not particularly useful, anomaly.

As for how Vegas does slow motion, it is extremely simple: It merely "crossfades" one frame with the next, with the "strength" of the contribution of one frame increasing as you get closer to the next frame. Thus, if Vegas must generate three new frames between two existing frames, then new frame one has 25% opacity from the first frame, plus 75% from the second; then 50/50; and finally 75/25. There is definitely no motion prediction.

By contrast, Dynapel's MotionPerfect or Twixtor or the various plugins for AVISynth like MVTools all use "motion estimation." This is the same technology that is used for MPEG-2 encoding to "predict" where each pixel is going to move in the next frame, so that only difference vectors need to be stored in order to create the next frame. When this technology works, it works incredibly well, but it also can create some horrendous artifacts. Try doing slow motion in Steadyhand of a camera pan across a scene that contains a picket fence: It creates an effect that Salvador Dali would have loved. By contrast, the simple-minded approach used by Vegas never produces any surprises.

However, it also never produces "to die for" slow motion. Since I would argue (I think correctly) that speed change is the single most-used fX in the world, if I were Sony, I would sure invest a LOT of money to provide all sorts of additional technology to give us tools to do better speed change. This includes stop motion capture; time lapse; and motion estimation slow motion.


farss schrieb am 16.05.2007 um 22:29 Uhr
If you want slomo to die for the Phantom HD is the answer, comes with a 'to die for' price. Even so it's getting way cheaper than what it used to cost to shoot extreme slomo, used to be 35mm was the only way.
JJKizak schrieb am 16.05.2007 um 23:37 Uhr
OK. What is the price to die for?
JJK
Spot|DSE schrieb am 17.05.2007 um 00:05 Uhr
$100k for the HD and $200k for the Phantom 65. Not cheap.
least, that's the price from NAB 2006, if I recall round numbers correctly.
farss schrieb am 17.05.2007 um 00:14 Uhr
The remarkable thing is when we first looked at these 2 years ago the limit was a few 100 frames at 1000fps, that was with 8GB of RAM. The color was pretty poor as well. Now they can offer 512GB of RAM with matching record times and excellent color.

The advances in silicon are opening up all manner of new possibilities.

Bob.
DJPadre schrieb am 17.05.2007 um 02:21 Uhr
johnmeyer, if ur theory on Vegas slowmotion being an crossfade mix of opacities were true, then progressive scan slowmotion would be far smoother than what it is
I honestly dont believe that your theory is sound and the progressive scan element and the lack of smoothness during lets say a 50% slowmo would be as bad as it is. witha 50% slowmo, u have a clean A to B frame reference with nothing inbetween. it SHOULD be even smoother than interlaced considering its a full frame and not a field
But its not..
JJKizak schrieb am 17.05.2007 um 12:57 Uhr
Spot/DSE:
Wow! I am dead.

JJK