Field Order bug in V8 & V9

farss wrote on 8/20/2009, 6:19 AM
This is troubling. Years ago in the older versions of Vegas I could easily fix field order problems with Vegas by simply changing the field order of the media in Project Media.

I just struck some media with the usual field order issue and my best efforts with V8 failed to fix the problem. Thinking I had a field dominance issue I checked some regular DV footage and field order comes out perfect with the media's field order either way around. This used not be!

To fully check this out I need to get a CRT on my old system running the earlier versions of Vegas. If somehow the behavior of Vegas got changed years ago then that explains a lot of issues people have been having, the regular advice about how to fix such problems does not work.

Bob.

Comments

farss wrote on 8/21/2009, 6:00 AM
Checked this in V5, it works the same <phew>, last thing I wanted to find was another gremlin that had crept into the works.
However I'm now at even more of a loss to explain what I see Vegas doing.

Simple test.
Take a short SD 50i AVI file.
1 ) Add to PAL 50i project. Render out to 1.avi, PAL 50i LFF
2) Reverse media's field order and render out to 2.avi, PAL 50i LFF
3) Set field order back to correct LFF and render out to 3.avi as PAL 50i LFF.
4) Render out to 4.AVI PAL 50i UFF
Put all rendered clips onto the T/L and play out. Only clip 4.avi views on the external CRT as having the field order wrong. This makes no sense. Vegas reads the field order flags from the source media, shows them correctly and yet ignores them!


How I got to having this problem was pretty simple.
I rendered files from a DVD camcorder from their native 50i SD mpeg-2 UFF to PAL DV 50i LFF. Vegas automatically should have handled the field order swap but didn't. Instead I got an AVI file flagged as LFF that was UFF. Now that I know what Vegas is doing I can fix MY problem easily enough but this is a real trap.

Bob.
farss wrote on 8/21/2009, 10:50 PM
More tests.

I took a small sample of my problem footage and tried to reverse the field order in V8. Complete failure. Rendering this out as LFF or UFF made no difference, obvious field order problem persisted.

I took the same small avi into my old clucker running V5 and I could easily correct the problem by rendering it out as PAL 4:3 50i but with the field order reversed to UFF. Bringing that back into V5 and finally it looks perfect on my CRT.

Only conclusion I can reach is that somewhere between V5 and V8 something got messed up. This may explain several problems I've had and ones other users have had over the last few years. Sadly I think I've been giving people advice that would not have worked, I never considered for a moment that something as simple as this that worked in V5 got messed up in later releases.

Unless anyone has any bright ideas I'll raise a trouble ticket. I imagine I'm going to have some fun explaining this problem.

Bob.
Grazie wrote on 8/21/2009, 11:00 PM
DVD camcorder from their native 50i SD mpeg-2 UFF - Is it? Never used a DVD camcorder, but why UFF, when everything else "smells" of PAL? Isn't PAL always LFF - or am I, yet again, "displaying" my ignorance?

Grazie
farss wrote on 8/21/2009, 11:19 PM
"Isn't PAL always LFF "

No, PAL DV is LFF, PAL Standard is UFF and so is PAL when it's mepg-2. Certainly this is what Vegas reports the original camcorder media as being, UFF. If I drop them straight onto the V8 T/L it plays out and creates a DVD with juddery motion. Importing camcorder disk produces the same problem.

Bob.

Grazie wrote on 8/22/2009, 12:08 AM
What happens when you do Media Matching through Project Settings?

BTW, and just to add to my video/digital brain, just WHY did PAL DV go for the opposite? Why?

Grazie
farss wrote on 8/22/2009, 1:43 AM
"What happens when you do Media Matching through Project Settings?"

From the camera source mpg files I get custom project 704x576 UFF.
Field order still wrong on CRT.

From the AVI file that I rendered some time ago I get PAL 4:3 LFF.
Field order still wrong on CRT.

As far as I can see project field order makes no difference to anything. Maybe it affects gen media when it's animated. I really wish SCS would document some of this stuff.

"BTW, and just to add to my video/digital brain, just WHY did PAL DV go for the opposite? Why?"

Not a clue. Maybe they just flipped a coin. What's strange is that analog PAL is the opposite to digital PAL. Might have something to do with how the early A/D converters worked.

Anyways I've now bounced the original media to the old thunder box and it's rendering it out with the correct field order,
Go V5, I still love ya :)

Bob.
Malcolm D wrote on 8/22/2009, 2:17 AM
Bob
V8 appears to be working for me at least with HDV.
Last week I received a 50i HDV tape exported from FCP.
I imported it into V8c and assumed the field order was UFF.
The NTSC MPEG2 render also set to UFF but when I checked the DVD it was strobing even on an LCD so I checked media properties and found it was set to LFF.
I changed media properties to UFF and re-rendered with the fields set to UFF again. I believe all the render setting does is set the flag. Only changing the media properties makes any difference.
This time it was perfect.
It was actually you who told me how to do this originally and I have had a few ocaissions where I have needed to do it.
The same clients previous job on HDV had correct field setting except for one static logo clip which was not noticed.
When rendering DV to MPEG2 Vegas offers LFF by default and this is ok for a DVD but don't rip the DVD or re-import the MPEG file to an NLE without correcting the setting.
It seems I will have to be more vigilant in checking media properties and not assume that because I captured it the field order will be as expected.
I agree that project field order seems to be irrelevant but maybe it sets the default for a render which could be why an MPEG2 render from a DV project always offers LFF by default.
Malcolm
johnmeyer wrote on 8/22/2009, 10:10 AM
Bob sent a short clip to me. I first verified that it did indeed have the wrong field order (TFF). I then put it into Vegas 7.0d and also Vegas 8.0c. Vegas reported that the field dominance was BFF, but I expected this because no program actually looks at the fields, but instead relies on the header or on the type of codec used.

I rendered without changing the field order of the source file, and Vegas smart rendered to DV PAL AVI. No change. I did the same thing, rendering to uncompressed AVI. No change. By "no change," I mean that field order was still wrong and the result was "juddery."

I then reversed the field order by clicking on the event and changing from lower field first (BFF in my nomenclature) to upper field first (TFF). I then did the same two renders both from 7.0d and 8.0c.

Unlike Bob, the fields DID get reversed. In other words, the problem footage that had the wrong field order had been fixed.

So why can't Bob do this, even though he's done it for years?

I suspect something has changed with some Vegas preference. The "solution" I suggested in a PM was to create a restore point (so you could undo this step) and then reset all the internal preferences by holding the shift key while starting Vegas. It almost has to be something like this because I used his footage and got it to work on my "modern" versions of Vegas, and also because he used to be able to get it to work as well.

It will be interesting to hear if he has any success by changing the preferences.

craftech wrote on 8/22/2009, 11:09 AM
Do you think the default "Smart Resampling" switch could have anything to do with it?
Could "Disable Resampling" possibly help?

John
johnmeyer wrote on 8/22/2009, 4:00 PM
Do you think the default "Smart Resampling" switch could have anything to do with it?It definitely can mess with such things. FWIW, I did not change from the default "smart resample" when I did my tests.
farss wrote on 8/22/2009, 6:59 PM
First off a huge thank you [again] to John Meyer. What a guy.
Secondly and just for the record I did NOT shoot that sample clip :)
I may suck as a cameraman but I can clean a lens.


I have tried everything and all to no avail.
I can fix the problem in V5 and take the bog standard PAL DV clip into my other system running V8 and still the field order there shows as incorrect. I have swapped monitors and A/D converters between these two systems with no change in the outcomes.
Thinking there maybe something truly wierd in the original clip I took a bog standard PAL DV clip which does not have any field order problems and tried reversing the field order in both V8 and V9 with no success. I tried changing field order in both the event and the project media, still no joy.
I take some heart from a post above that the same technique has been used and it worked. I again must thank J.M. for verifying that it works OK for him. I am concerned however that over the past years at least one other user reported trying the same technique and simply could not get it to work. I had hoped to discover why I am now having the same kind of problem but I'm completely out of ideas.
Just to verify what is happening I took all of the clips I'd rendered out of V5,V8 and V9 into Ppro CS3. All of them also show the field order as being wrong. I seriously suspect where ever the problem lies it is not within Vegas, rather it is something systemic.
Did I mention I did a default restore on V8.0c. My V9.0a install is very recent and clean. I do not install codec packs. I have "ignore 3rd party codecs" set. The only thing left is maybe an exorcism or a complete bare metal rebuild of my system that's running V8/9.

Unfortunately I have simply run out of time, jobs are banking up and the grass needs mowing. I am going to take the easy way out and de-interlace the footage. A big thank you to Mike Crash for his Smart Deinterlacer plugin. The results look quite acceptable, given that this is only for someone's CV and the 2 minute DVD will likely be viewed on a PC it's hardly worth worrying about.

Again I really wished I could get to the bottom of this. I suspect the same gremlin could explain all the grief I've had with YUV footage I've ingested via a Decklink card and I do have another job coming up that's all DigiBeta footage.

Bob.
johnmeyer wrote on 8/22/2009, 8:16 PM
Bob,

I have several AVISynth scripts that will do field reversal. I'll be happy to send them to you, if you want. I hate to have you deinterlace and lose all that temporal resolution.
farss wrote on 8/22/2009, 10:52 PM
Please send them, much appreciated.

Here's the odd thing though. After de-interlacing and applying 1 frame of motion blur set to asymetric pyramid it looks quite good. Reason being I suspect the toy camera used to shoot the footage had the shutter speed wound up high because of the bright sun. It also helped a tad in some of the other shots where the light was very low and the noise was getting up there.

Bob.
johnmeyer wrote on 8/22/2009, 11:47 PM
I have sent two AVISynth scripts via PM. They each do field reversal. The first simply changes the header to TFF (matching the actual field order). The second actually does change the field order so that you can then use the traditional (for PAL DV AVI) BFF field dominance.

I also added a "Separatefields()" statement at the end of each script, but commented out. If you remove the comment, you can view your video one field at a time. This will immediately show you if you have field reversal. When the field order is correct, each field progresses smoothly and naturally, and except for the slights up/down motion (because the fields are spatially displaced from each other) you should feel like you are looking at regular frame-based video. However, if the fields are reversed, you get a "jitterbug" effect where one frame goes forward and the next seems to go backwards. Even with very slight motion, the problem is immediately evident.
GS1966 wrote on 8/23/2009, 1:58 AM
Bob, V8 And V9 correctly understand and work with interlaced video, and will perfectly cope with any problem (it not a joke)
Let's recollect a few theories:
1. What is tag Field First? It only a command to the codec as he should decode a videostream.
2. By standards it is certain, how fields weave into a frame. What field in video signal to consider as the first (F1 - odd, or top), and what the second (F2 - lower).
3. In computer appendices, down to occurrence SP2, there was no unity - the tag of a field was put who as wanted. It created mess and chaos.
By your words
From the camera source mpg files I get custom project 704x576 UFF.Field order still wrong on CRT.
From the AVI file that I rendered some time ago I get PAL 4:3 LFF.Field order still wrong on CRT.

It is possible to assume, that you have collided with such variant (Item 3) - the mediafile with UFF has tag LFF (or on the contrary).
To correct simply:
Open properties this event, and on "Media" Tab replace Field First.
The strobe will disappear

Gene
farss wrote on 8/23/2009, 3:31 AM
"It is possible to assume, that you have collided with such variant (Item 3) - the mediafile with UFF has tag LFF (or on the contrary)."

It would seem that is the problem.

"To correct simply:

Done that.
"The strobe will disappear "

It doesn't. Not with V8 or V9 on one system.
It does on another system with V5. Render out a new AVI file on that system, check it, it's fine. Take it to the V8/9 systems, it's still wrong.

Everything else works as well as it does on my V8/9 systems. Many hours edited and rendered on that system.


I should also point out that John tried exactly the same thing with exactly the same media on his system running V8 and it worked correctly. Hence my conclusion that indeed it is not something wrong with V8/9. What it could possibly be is a very interesting question. If there's something that I've installed on the system that's messing with how Vegas handles or see field order that's pretty worrying.

Bob.
megabit wrote on 8/23/2009, 3:45 AM
If there's something that I've installed on the system that's messing with how Vegas handles or see field order that's pretty worrying.

Exactly, Bob.
I'm witnessing another strange Vegas behaviour nobody else can confirm (including SCS support) - mxf files, when smart-rendered out to mxf, cause huge memory leak which freezes the whole system after several minutes rendering. It happens on both my systems (Vista x64 on the main editing station, and Vista x86 on the laptop). All Vegas versions I have currently installed (8.0c, 8.1, 9.0a 32 and 64-bit) are affected.

The same project under XP is rendering happily within the normal memory limits.

This can only lead to the conclusion that something I have installed on both my Vista systems is colliding with Vegas installations. But what? I'm afraid I'll never know...

Sorry for digressing,

Piotr

AMD TR 2990WX CPU | MSI X399 CARBON AC | 64GB RAM@XMP2933  | 2x RTX 2080Ti GPU | 4x 3TB WD Black RAID0 media drive | 3x 1TB NVMe RAID0 cache drive | SSD SATA system drive | AX1600i PSU | Decklink 12G Extreme | Samsung UHD reference monitor (calibrated)

GS1966 wrote on 8/23/2009, 4:50 AM
Please send me a fragment yuor problem video (or give the link), I wish to taste in VegasPro
PS. On Russian site there is a remarkable material - "Definition real Field First in videoevent"

Gene
craftech wrote on 8/23/2009, 5:26 AM
PS. On Russian site there is a remarkable material - "Definition real Field First in videoevent"
================
This is the Babelfish translation of that web page:
------------
There are the situations, when for the installation are used [videofragmenty] of different origin - for example video, taken by [mpeg]- video cameras (from UpperField) or [videofragmenty], created or processed by strange [softom].

Let us examine two interconnected questions:

- as to determine real Field Of first of [videofragmenta]
- as it is correct to install [videofragmenty] with different prevailing fields in order to avoid artifacts after error (smearing on the vertical line or strobe).
Determination of the real Of fieldFirst of [futazha]/[videofragmenta] and bringing FieldFirst of project and [futazhey] to the united value

1. you will preserve project.

2. introduce changes in the installations of project (laying Of video):
- Frame Of rate = 50
- Field Of order = Of none (Progressive)
- Deinterlace of method = Of blend or Interpolate.

Now Vegas will show in the window of the preliminary survey (Video Of preview, the regime Of best-Full) not of two fields simultaneously, but each field (half-field) separately (into Properties of [futazha] must stand Smart of resample).

3. you will switch the regime of [prevyu]- window to Best-Full, and you pass to [taymlayn]:

- " extend” (< --="" --="">) path c by [futazhami] into the length, by key “pointer upward” or in another manner (buzz path). It is not maximally necessary, it will be sufficient to half (by approximately 50-60 percent);

- you will establish cursor to the necessary sequence and press “Alt + - >” (” pointer to the right” on the keyboard).

We look into the window of the preliminary survey:

a) if for each pressure of keys in you occurs change on the screen - this is [interleysnoe] video 50i.
- if motion is sequential, by video interpreted correctly.
- if motion “two steps forward, one step back” - by video interpreted incorrectly, into Properties of [futazha] change interpretation for the opposite (UpperLower< -="">).

b) if picture changes through once after two pressures of keys (although the fragment is interpreted as [interleysnyy], Upper or Lower) - this progressive video 25p, nothing with it made must not be.

4. they did recognize real FieldFirst of [eventa]?
Now, if it differs from the installation Of fieldFirst of project, it must be displaced on the vertical line to 1 pixel upward or downward.
For this make ONE (!) of two:

- For the concrete fragment ([eventa]): Pan/Crop - > Position: Center of position Of y=289 (for [futazha] of 720[kh]576)

OR

- For the entire path: to apply to the track Of track of motion - > Position: Y=1
It is possible one time to raise path, and then in it is simple to place [futazhi] with the different from the project Of fieldFirst.

5. return tuning Frame Of rate and Field Of first conversely (what they were prior to the fulfillment of point 2) you will preserve project, it is better under the new name.

6. You [perezagruzite] Vegas and project.

All, it is possible to take up further work (without any limitations!) with this project.

Prompt:
You can preserve the installations of project for determining the prevailing field under another name.
For example, you work in the project PAL DV (720×576; 25,000 fps). After the introduction of changes (p. 2) you will preserve installations for example by the name PAL DV (720×576; 50,000 fps). Then, for the recovery to the installations of the current project (p. 5) to you it suffices to replace template, to harvest “to use” and “ok”.
------------
John
farss wrote on 8/23/2009, 6:22 AM
Although it's hard to read I think what's being done there is to onvert 50i to 50p and then checking the motion between frames. If the motion moves in the same direction when you step between frames the field order is correct. If motion goes backwards, forwards, backwards then field order is incorrect.

John Meyer uses the same trick using AVISynth. I use a CRT monitor. Same basic thing really.

In my case if I change the media's field order and render to a new PAL DV avi file it stays the same, go figure. GS1966 can try to recreate my problem with ANY PAL DV video. If he can force a change in field order then he doesn't have my problem.

Just to be certain I tried the test again after disabling MacDrive which has another wierd problem with something else but the problem persists.

Bob.

[edit] I just refound this excellent guide to the mysteries of interlaced video. It's about the only one that describes both the problems of field order and field dominance. I've found many in the industry use those terms interchangeably and they are not the same problem.

http://neuron2.net/LVG/interlacing.html
megabit wrote on 8/23/2009, 6:59 AM
Thanks Bob for this excellent link. I understood the notion of field dominance vs order some time ago, but sort of lost it ever since :)

One thing intrigues me though; the article reads:

One way to determine the correct field dominance is to play a segment of your video on a TV. If the motion is jerky, try changing the field dominance so the other field is displayed first (dominant)

Can you enlighten me, how exactly I "change the field dominance" in Vegas ?!!

AMD TR 2990WX CPU | MSI X399 CARBON AC | 64GB RAM@XMP2933  | 2x RTX 2080Ti GPU | 4x 3TB WD Black RAID0 media drive | 3x 1TB NVMe RAID0 cache drive | SSD SATA system drive | AX1600i PSU | Decklink 12G Extreme | Samsung UHD reference monitor (calibrated)

megabit wrote on 8/23/2009, 7:31 AM
This also rises another question that has bitten me for a long time (sorry if a bit off-topic here, Bob):

When I create a 50i version of my 25p source for compliance with DVDA, I am assuming all is done is turning 25p into 25PsF, and flagging as interlaced. BUT: how can I make sure that when played back on a progressive display, the 2 "fields" (or segments, as they're from the same moment in time) originated from a given frame, will be put back together to form the exact frame the originated from? I mean, it can as well combine the "lower"segment of this frame with the "upper" segment from the next frame, and combing will result!

I even remember I did witness combing once, when playing back a 50i m2v file I produced in Vegas from 25p...

It's this problem that forms the weakest link in my confidence as to the workflow I adopted for preparing DVD's and BD's from my 1080/25p material.

AMD TR 2990WX CPU | MSI X399 CARBON AC | 64GB RAM@XMP2933  | 2x RTX 2080Ti GPU | 4x 3TB WD Black RAID0 media drive | 3x 1TB NVMe RAID0 cache drive | SSD SATA system drive | AX1600i PSU | Decklink 12G Extreme | Samsung UHD reference monitor (calibrated)

MarkWWW wrote on 8/23/2009, 9:11 AM
In the dim and distant recesses of my memory I seem to recall that there was a problem with the way Vegas dealt with the field order of SD PAL under certain circumstances that could be fixed with a simple edit to a configuration file. I'll have a search and see if I can come up with the details - they could be relevant in your situation.

LATER

Ah, yes, found it. If you do a search for "profiles.ini" you will find a small number of messages dealing with field order problems specific to SD PAL .avis. The solution to these problems was to change one of the entries in the "vegas profiles.ini" file from:

Key2=0, "None", 720, 576, 25.0, 0
Attributes2="Upper Field", 1.0925925925, "Undefined", 1

to

Key2=0, "None", 720, 576, 25.0, 0
Attributes2="Lower Field", 1.0925925925, "Undefined", 1

I don't know if this is relevant to your problems, but it might be worth seeing if making that change will help you.

(The "vegas profiles.ini" used to be in the main Vegas directory in early versions, but from V.7.0 onwards it has been moved to "C:\Documents and Settings\All Users\Application Data\Sony\Vegas\7.0" or "C:\Documents and Settings\All Users\Application Data\Sony\Vegas Pro\8.0".)

Mark

craftech wrote on 8/23/2009, 9:35 AM
You may be onto something regarding this problem and perhaps others regarding interlacing problems with certain files under certain conditions coming out of Vegas.

Here is the complete vegas profiles.ini from Vegas 8.0c:

[Video Media Profiles]
Key1=0, "None", 720, 480, 29.97, 0
Attributes1="Lower First", 0.9090909090, "Undefined", 1

Key2=0, "None", 720, 576, 25.0, 0
Attributes2="Upper First", 1.0925925925, "Undefined", 1

Key3=0, "None", 720, 486, 29.97, 0
Attributes3="Lower First", 0.9090909090, "Undefined", 1

Key4=0, "None", 704, 480, 29.97, 0
Attributes4="Upper First", 0.9090909090, "Undefined", 1

Key5=0, "None", 640, 480, 29.97, 0
Attributes5="Upper First", 1, "Undefined", 1

Key6=0, "None", 352, 240, 29.97, 0
Attributes6="Progressive", 0.9132420091, "Undefined", 1

Key7=0, "None", 768, 576, 25.0, 0
Attributes7="Upper First", 1, "Undefined", 1

Key8=0, "None", 352, 288, 25.0, 0
Attributes8="Progressive", 1.0920607185, "Undefined", 1

Key9=0, "None", 1440, 1080, 0.0, 0
Attributes9="Progressive", 1.3333333333, "Undefined", 1

Key10=0, "None", 1440, 1080, 29.97, 0
Attributes10="Upper First", 1.3333333333, "Undefined", 1

Key11=0, "None", 1440, 1080, 25.0, 0
Attributes11="Upper First", 1.3333333333, "Undefined", 1

Key12=0, "None", 1920, 1080, 29.97, 0
Attributes12="Upper First", 1, "Undefined", 1

Key13=0, "None", 1920, 1080, 25.0, 0
Attributes13="Upper First", 1, "Undefined", 1


How is Vegas handling changes made to the predefined profiles?
Would perhaps adding additional profiles to Vegas Profiles.ini help the situation?

John