Best way to work with HDV....rewrap, proxies, etc.

mudsmith wrote on 8/6/2013, 7:27 PM
Working with 11, build 701 on my Xeon 8 core, which should still be in my spec list.

I have been out of commission for about 10 months with some medical stuff, but am now diving back into the large documentary project that was loaded prior to this delay. Will need to accelerate on the edit, and have more funding in the wind and potential outlet on PBS or similar, so need to get the efficiencies in place now. Apologies in advance if all this material has been covered (I am sure not for the first time) in my absence.

Most of the extensive footage was shot on HDV/tape with an FX1, which is native 1440X1080. Need help from those who have worked with this format. Although I am an extremely experienced audio guy and conversant with most video issues, I may not be as up to speed on all the format issues of the day as most of you, so please try to be specific with any responses. Thanks in advance.

Question set no. 1:-What is the best method, both for editing/preview stability, and end quality, to work with the footage? Should I "wrap" the files? Should I use "proxies"? If yes to either of these, please give me a specific workflow methodology, including any third party software I should use.......Aside from the massive audio issues (which I have a handle on), there are also plenty of color correction issues that will need to be dealt with inside Vegas, if possible. I could conceivably go outside to another program, but would have to deal with the learning curve, so.......My partner is adept with visuals and manipulations in Photoshop, so could possibly tackle some of this while I stick to the things that I am better at.


Question Set No. 2:

Any suggestions for dealing with the obvious delivery need for 1920X1080? My renders of the trailer we prepped last fall always ended up pillared no matter what I did with project settings. Perhaps that is in the cards anyway, but I would sure like to know if there is any other way around this. The stills were obviously easier to manipulate, but all the HDV footage is another story.

Comments

videoITguy wrote on 8/6/2013, 7:37 PM
I would suggest your problems are not going to be technical but rather political. You suggest your outlet at PBS? Or some potential broadcast form?

THen here is where you start! FIRST - get the spec info from the broadcast outlet regarding their acceptable output for digiital submission, audio level, leader length cues, and most of all, final edited length - like less than 19 minutes for 1/2hour time slot. PBS outlets have a spec (likely in .pdf file) that you need now. The format of length and content will be highly political and is likely to break your efforts. Get the skinny now and then work backwards from that spec.
mudsmith wrote on 8/6/2013, 8:27 PM
I am, and have been, on top of all that since before my medical issues. There are questions to be asked, but I know all the issues for PBS, which are defined much more clearly than for some other major outlets, actually......I have enough contacts inside the PBS organization and family of stations that this communication will not be too difficult.

So.....I still need advice on the best (both for efficiency and end quality) way to edit/preview these files, and the options for delivery in full, or at least pseudo, 1920x1080, if any such options exist.....whatever the ultimate outlet.

You are right that network "politics" matter, but I am used to that issue and have had the latest PBS "Red Book" for over a year, as well as insights into major cable requirements, and need more direct advice with Vegas at this point.
TheHappyFriar wrote on 8/6/2013, 8:31 PM
HDV is perhaps the second best native format to edit in Vegas... DV being the first.

Edit everything @ HDV, then render to 1920x1080. It will just had horizontal pixels, so it won't mess up any interlacing. I capture and edit HDV, I have almost no issues.
musicvid10 wrote on 8/6/2013, 8:35 PM
"

I suggest you do all your editing in native project format (1440x1080 HDV). Not only will it give the most stable workflow and smooth preview, HDV will smart render,.

Save question #2 for the end, when you're sure of your preferred output / delivery format. Two steps are better than three.
videoITguy wrote on 8/6/2013, 9:22 PM
I don't see how the OP suggests his PBS connections are as clear cut as he suggests, otherwise he would learn several of the following criteria 1) There is no way that 90minutes of video submission is going to make it to 90minutes of air. 2) Most material edited and played back at PBS is workflow from MAC platform like FCP NLE. 3) Even if production is started on VegasPro and finalized on MAC - thw workflow is very problematic. 4) Finally the broadcast spec of PBS submission requirements is going to go a long way to suggest good tech quality - which again will be problematic from a source of HDV 1440x1080.

A very slight pillarbox output would be the normal output of VegasPro thru 1440x1080 re-mastered to full HD upon delivery. But PBS engineers can and do mess this up quite often by either over-scanning the screen or worse shrinking the screen in strange ways with the wrong pixel aspect.

Finally color- grading will be a real gotcha at Vegas - the best integrated solution with VegasPro ( version 9 best) would be with the full package of Cineform - 2nd best would be with BlackMagic tools and Resolve... several forum members are good at this. The learning curve is very steep. 2nd Generation mastering on HD with Mac platform could be and is likely to be better result.
musicvid10 wrote on 8/6/2013, 9:24 PM
Undoubtedly, his output will be 1920x1080 at 1.0 PAR. That is not at issue. Logic suggests that step should be saved until the very last, not by performing "surgery" on the source.
mudsmith wrote on 8/7/2013, 10:59 AM
-My understanding of the time formats for PBS is totally clear, and is not something that needs to be hashed out here. I have worked in both TV and radio broadcast formats for 30 years now, so am aware of the issues and potential issues. There are some newer specs that may or may not interfere with delivery to PBS in the end, but I will see about that as we progress. There are multiple other potential and very real delivery destinations that will not have these particular spec problems, and I need to make the edit prior to delivery as efficient as possible......but trust me, I have all the potential delivery spec and "political" issues clearly in my brain.

-Mac vs. PC has nothing to do with workflow, only software platform (and I assume, by Mac, you are largely talking about FCP, since Avid has long been available on both platforms). If my work does, indeed, need to pass through FCP at some point, there is no dearth of good FCP editors available. If Premier Pro is the answer, then there are plenty of PCs available, and perhaps some experience base I can deal with......If color correction needs to happen on someone else's system, then so be it, but this is something I am asking you folks for......In other words, if this is the "problematic workflow" issue, then I am looking for details here.
-In fact, if "workflow" is problematic, this is the only answer I am looking for from anyone who has had to deal with HDV.

-Details from anyone who has delivered product from HDV and/or had issues with editing/fixing in that format that were fixed (or not) by rewrapping the files or working with proxies or such?

-Additionally, one of the other posters seems to be suggesting that it is possible to deliver this in 1920x1080, but is not clear about whether it must be pillared for the end product or not, nor is it indicated how I would ensure this from Vegas. Any thoughts?

Since I am quite certain that FX1 footage has been used for serious documentaries and films in the past, it does seem like there should be a knowledge base of sorts about dealing with the format issue, and I was hoping there would be some sort of knowledge base among Vegas users......I know the whole world has changed multiple times since FX1s were the hot new camera, but.......
mudsmith wrote on 8/7/2013, 11:05 AM
.....And if the full Cineform package is the best answer for color correction, are you saying that it would be applied inside Vegas, or outside Vegas on a clip by clip or file by file basis?

What would this work flow actually entail? When do you suggest Cineform tools by applied in the process?
mudsmith wrote on 8/7/2013, 11:16 AM
.....Another poster suggested that I just need to render in 1920x1080, but, as stated above, my attempts at that last year (albeit very quickly as I went into medical land) always resulted in pillared product.

------I am sure I was doing something very wrong here, so walk me through it. Should the project have been worked in 1920 or 1440? At one point, and with which menu items checked where, should I have done the render, and from what original project settings? I really thought I had explored all the possibilities, but obviously must have missed something.

----Please walk me through it. It is especially important to understand now, obviously, about how the project needs to be set up for efficient editing and at what point I may need to change the project itself over to 1920.

-----All of my earlier render attempts were aimed at BluRay or compressed files for YouTube or delivery via YouSendIt. As the first product was a fairly short trailer, I did have time to try a few things, but none successfully produced an unpillared product.
vtxrocketeer wrote on 8/7/2013, 11:22 AM
I still shoot on HDV (Canon XH-A1). Typically I'll place all captured HDV footage on a 1440x1080p timeline. I'll de-noise (with NeatVideo) where needed, then render all events to 1920x1080p Cineform avi clips.

I edit, color correct, grade, and composite all within in a new Vegas project containing only the Cineform events, lock an edit, then render to whatever I need, usually a Cineform master that I will then render to MPEG-2 for DVD or Blu-ray, MPEG-4 for web, etc.

The Cineform codec does give some headroom for gentle grading. I edit in Vegas' 8-bit color depth, but grade in 32-bit floating color depth. For reasons I have yet to understand, toggling between the two color depth settings produces drastic changes on ungraded HDV footage, but has no effect whatsoever on ungraded Cineform avi's.

At the "high" setting, Cineform avi's edit like butter on the Vegas timeline. At the higher FilmScan 1 setting, timeline playback gets bogged down on my aging machine when I start to pile on FX.

I use Cineform exclusively because my workflow tends to involve multi-generational renders, which is where Cineform shines: it is visually lossless.
musicvid10 wrote on 8/7/2013, 11:34 AM
1440x1080 HDV is just an anamorphic squeeze for 1920x1080 -- that is the Display Aspect Ratio, not two different formats.
No big deal. At some point it ends up 1920x1080.

Same as DVD. 720x480 is neither 4:3 nor 16:9, but it displays as one or the other because it is anamorphic.

vbxrocketeer's comment on the effects of toggling 32-bit project settings on 8-bit and reported 10-bit source is well taken.
I still like working with native HDV because it will smart render.
mudsmith wrote on 8/7/2013, 11:59 AM
Thanks-

A few questions:

-The Cineform avi render gives you an unpillared 1920x1080?

-I am assuming that, in order to be able to do any additional editing in the second project filled by Cineform avi "events" that the "events" you rendered had to be larger sections of the files edited in the original HDV edit in order to allow you the flexibility to change transitions and/or lengthen cuts?


......Most of what you are saying or implying is clear, but it sort of seems that you do have to render (within Cineform) quite a bit more material than pre-edited "events" to pull this off?.....Should I be doing something with the many stills we will be using via Cineform avi as well?....or should those be placed on the timeline after creating the second project?

I suppose I am a little confused by "place all captured footage on a 1440x1080 timeline" since there is no way I will be putting 25 hours of footage on a timeline, and then the term "events" by (which I am assuming you mean denoised video).

I am assuming that I will likely have to follow a slightly different workflow where I render large sections of the loaded files (again, I have something over 20 hours of footage loaded on my drives, along with tons of stills) to Cineform created avis, which you are recommending, but which you also think should possibly be denoised first.

There are a considerable number of clips/tapes that have fairly serious color correction and fixes needed, so am I correct in interpreting your post to say that you find this to work better with the Cineform avis than the original files?

.....If so, then this is definitely the info I was looking for, and will put it to use as soon as I understand the full implications of the implied workflow.

I had purchased some Cineform products last year, so was thinking I had to head down this path.

I am, unfortunately perhaps, at a point where I have a moderately intensive rough cut of about an hour's worth of interview clips on my timeline, so have to figure out how to render all the original files that these come from to Cineform avis as my next step, I suppose, or just spend a couple of days rendering everything I have captured to the new format.....and not lose the editing work I have already done.

Any thoughts from anyone about how to achieve this most efficiently?

Any thoughts from anyone else about the validity of using the Cineform avis or any other part of this process?
mudsmith wrote on 8/7/2013, 12:07 PM
And musicvid10:

If 1440x1920 is just an anamorphic squeeze of 1920x, how do I get it to render unpillared?

Since you prefer the Smart Rendering capacity of HDV, how do I make use of this and end up with a 1920x1080 product without pillars our of Vegas?

If you have specific menu item suggestions, please pass them on. Although everything seems to display properly in Vegas when I am working on the project with the overall settings for 1920, my rendered files definitely do not display properly without pillars on any computer......I think the BluRay did display properly on a monitor via HDMI, but this is all from a year ago and I gave the last copy I had to a producer so won't be able to check for a bit when I can get it back, and a regular DVD will display as 1920x1080 from that same player via HDMI, so.......

I need details, and any details about how you implement Smart Rendering in your workflow.

Thanks in advance.
OldSmoke wrote on 8/7/2013, 12:38 PM
I suspect you have Letterbox enabled in the render dialog. Switch it off and your final render should be fine. I almost exclusively work with HDV and never had an issue resizing it to 1920x1080. All my delivery is in 1920x1080 aside from Internet upload which I recently changed to the lower 1280x720 standard.

Proud owner of Sony Vegas Pro 7, 8, 9, 10, 11, 12 & 13 and now Magix VP15&16.

System Spec.:
Motherboard: ASUS X299 Prime-A

Ram: G.Skill 4x8GB DDR4 2666 XMP

CPU: i7-9800x @ 4.6GHz (custom water cooling system)
GPU: 1x AMD Vega Pro Frontier Edition (water cooled)
Hard drives: System Samsung 970Pro NVME, AV-Projects 1TB (4x Intel P7600 512GB VROC), 4x 2.5" Hotswap bays, 1x 3.5" Hotswap Bay, 1x LG BluRay Burner

PSU: Corsair 1200W
Monitor: 2x Dell Ultrasharp U2713HM (2560x1440)

mudsmith wrote on 8/7/2013, 1:06 PM
Will check this shortly.

All other input HIGHLY welcome with great appreciation and thanks.
vtxrocketeer wrote on 8/7/2013, 2:17 PM
mudsmith,

1. There is no need to render stills to any intermediate video codec. Vegas plays nicely with just about any graphic still, PNG being the most favored. I've used JPEG and PhotoShop files with zero problems.

2. 20 hours of HDV footage is a LOT for a single project. I won't (and, in fact, can't) tell you how to edit. Perhaps you can breakk up your bin of 20h of footage into logical separate projects for editing and grading, e.g., Interview 1, Interview 2, etc., then nest those projects into a single master project for a final render.

3. I only mentioned denoising because it makes sense to denoise any footage before grading, color correcting, and/or compositing. Otherwise you'll be grading noise. Yecch. Hence, on only the HDV clips that need it, I'll apply NeatVideo. With my XH-A1, I really need denoising for mid- to low light scenarios, so I find this workflow good for me.

a. I use Excalibur (dandy script extension) to make each HDV event on the timeline a region. Best practices dictate match project properties to media --> 1440 x 1080p.

b. Then I region render to a Cineform avi (1920x1080). This gives me a directory of Cineform avi's that correspond 1:1 to the original HDV clips, some having noise removed.

c. Those Cineform avi's are then placed into a new project, 1920x1080p. Edit, etc.

As I said above, I chose Cineform because I grade a lot, generate master files, and for DVD production I take a master avi through VirtualDub for a downsize. All these involve successive renders, which is exactly where Cineform (or any lossless codec) shines: there is no real (or perceptible) generational loss of data. Is this "best" for you? Depends. It's bullet proof for me.
musicvid10 wrote on 8/7/2013, 2:25 PM
mudsmith,

1920x1080, 1.0 PAR (square pixel aspect)
That's what goes in your render dialog.
I doubt any broadcaster accepts anamorphic these days.

Best.
mudsmith wrote on 8/7/2013, 2:37 PM
Thanks vtxrocketeer. That is specific enough to get me going.....at least for some test runs here on a smaller project while I figure out how I want to set it up.

In my capacity as a producer, I have frequently found that a minimum of a 10 to 1 ratio for raw footage to end product was common for documentaries or industrial videos, and is certainly not way out of bounds for film work.

It might be that bringing everything into the timeline initially would have been a good practice, but I got into this well beyond the point that it would have made sense. When I started in, the main producer/director had already created a 1 hour 40 minute rough cut in an earlier version of Vegas, and that was inadvertently in standard def......I was thrilled that clicking a couple of buttons brought everything (except the pre-rendered stills and motion effects, etc.) into high def and that I actually now had a starting point to work with.

So I will play with the project for the trailer (only a bit over 5 minutes long) using your suggested workflow as well as musicvid10's and see where I get before asking more questions or diving in.

Thanks again for the tips.

mudsmith wrote on 8/7/2013, 2:46 PM
musicvid10-

Thanks.

FYI- PBS does, in fact, require delivery of your project in an anamorphic version on HDCAM (or perhaps it is a DigiBeta throwback that they are requiring....have to go back and look at the specs, but that makes more sense) designed to translate to standard def, I believe. This is in addition to the other high def versions they require.

Luckily for us, there is a local house that specializes in prepping and transferring for this and any other kind of master you need, so we just budget their services in.

Since almost no editors now have any tape decks at all in their suites, this kind of service has become totally necessary.

Other broadcasters/cable sources seem to be taking files in in a more or less standard manner. Some of them (Discovery is a good example) have very large in house editing and transfer facilities....others do not.
mudsmith wrote on 8/7/2013, 2:52 PM
Additional detail, vtxrocketeer:

When I go into Project Properties, I don't find a line for 1440x1080p, only a setting that indicates 60i for US standards, while there is a 24p available.... Am I missing something?
videoITguy wrote on 8/7/2013, 2:58 PM
mudsmith, your pursuit in this thread seems to be taking two different courses...1) First you seem to be seeking recipes in step by step as the editor and 2) But you seem to wear the hat of the producer looking after funding, logistics, and I note particularly your comment - "Mac vs. PC has nothing to do with workflow, only software platform. If my work does, indeed, need to pass through FCP at some point, there is no dearth of good FCP editors available. If Premier Pro is the answer, then there are plenty of PCs available....If color correction needs to happen on someone else's system, then so be it."

With that second comment it seems like you are more content to wear the producer HAT and let others do the work. I would suggest that you decide at the outset which role will be better under your circumstances and then make inquiries appropriately.

As far as recipes, I think that musicvid, HappyFriar, and vtxrocketeer have all made great suggestions. 1) Edit in 1440 x1080 HDV project settings - render to 1920 x1080 for output file.
2) YOU DO need the specs for the acceptable PBS standard video and audio submission.
3) Introducing a digital intermediate, as I have previously suggested is wise and vtxrocketeer makes several points on how to handle.
4) Color grading and matching scenes can be an integrated and concurrent with editing effort with Cineform Pro or Resolve BMD but steep learning curve with abundant issues in the interface monitoring with VegasPro.
5) You can rewrap Cineform.avi to Cineform.mov to move to the Mac platform - but this can also introduce huge headache in skewing the gamma setting. Get some advice and start an interface with FCP editors now.

Finally to vtxrocketeer points about use for the Cineform DI's. Depending on how you like to work this could take several different trips. I would suggest direct editing only of HDV sources as previously stated reasoning. Only convert an edited timeline of a large amount of finalized HDV source events to the 1st generation Cineform. There is really no need to convert throw-away or all sources to just introduce Cineform to the timeline.
vtxrocketeer wrote on 8/7/2013, 3:19 PM
videoITguy, thanks for the remarks on initial workflow. I may be (mis)remembering something from years ago on HDV (4:2:0) not being the most robust when grading, compared to a Cineform avi (4:2:2). I know that color depth is not created upon transcoding, but I had the impression that the extra headroom, so to speak, in a Cineform avi would allow better for pushing the footage around. Ergo my suggestion. (N.B. I'm about to part ways with my beloved XH-A1 and, consequently, with the workflow I took pains to explain above to mudsmith.)

Mudsmith, I see that my dashed-off replies are too shorthanded. Open a new project, open project properties, click on match-media button, and select one of your HDV clips. Vegas will adjust. Whatever template is highlighted is what you want. In my case, it is HDV 1440x1080 at 24p because that's what I shoot.
videoITguy wrote on 8/7/2013, 3:30 PM
vtxrocketeer - AFAIK you are correct. Here is what I was suggesting as an attempt to keep footage flows and volume space handling most efficient. 1) Produce a cuts only edit of your most valuable footage. THis might take 25hours down to one hour. Leave ample heads and tails in the HDV project timeline of this cuts-edit. Now convert this timeline to 1st generation Cineform and use for source of concurrent grade and tighten edit with effects for a 2nd timeline. This effort would effect (produce) with your comment appropriately.
musicvid10 wrote on 8/7/2013, 3:34 PM
vtxrocketeer,
You are quite correct that upsampling 8-bit 4:2:0 to 10-bit 4:2:2 does not add anything that wasn't already there, however the "headroom" theory is something I'm not quite sure of.

All of the existing video information has already been fully unpacked to RGB in your 32-bit float project, so the source media parameters are no longer operative at that point. "Headroom" is dependent on the source data fitting between the RGB floor and ceiling, consideration being given to the output destination (usually 8-bit YUV).

As I said, your point about more favorable 32-bit project behavior with reported 10-bit source is well taken, and if undertaking some serious grading or generated media, I might be more inclined to try out your workflow for that reason.

OTOH, if it's a bit of color correction and leveling, I think there are still some advantages to sticking with the source for editing, and saving the heavy lifting for last, contingent on knowing where it is destined for.

It's worth mentioning again, that Project settings above the line do not affect the rendered output.