Jay,

Thanks for that, working on my own, it's good to have points of reference, e.g. to check I'm not missing something fundamental.

Having said that, I have started using BPAV-MP4 files directly as an experiment. The only issue so far, of no real importance I hope, is that Vegas puts a .sfk file in the BPAV folder. Pollution, but benign I hope.

Sometimes I use AviSynth (frameserving into Vegas) for double-framerate motion-compensated deinterlacing (better for pan/crop quality). It's great because it can be treated like a proxy file. However so far have had more success with MP4 than with MXF - because for the latter I get the video stream but not the audio stream. So it looks like there are two things in favor of the BPAV/MP4 approach - space-saving and AviSynth.

If you're not getting the MXF clip's audio, then you or your workflow are indeed "missing something fundamental."

That being said, if you're happy with your current work around, and if it's working for you, then more power to you!

This is a topic I'm interested in. Here's my understaning of it to date:

1) If I use Device Explorer to import card images from a HDD Vegas makes another copy.
2) Vegas has no mechanism to correctly join clips that span cards, there is couple of frames lost if I butt the two parts on the T/L. To get around this I still have to use the Clipbrowser to move the clip segments into the one BPAV folder and then export as MXF.

Overall I'm not really that impressed by what Vegas offers in the way of XDCAM EX support. By the time I backup cards, shuffle things around and export out of clipbrowser etc this new 'faster' tapeless workflow is a pain and no faster than working with tape.

Bob.

"... this new 'faster' tapeless workflow is a pain and no faster than working with tape."

Bob, I can't directly address your workflow, as I've not "seen" what you're doing or not doing. But I must disagree with your general assessment. Moving from tape to files has saved me hours with every project, and I know for a fact I'm not the only one.

When it comes to ingesting real time transfer of video tape versus video files alone, there simply isn't any comparison. Video file transfer takes a fraction of time.

I agree with Jay on that.

Before i started using my nanoFlash, I also experimented with Device Explorer's MP4's vs. ClipBrowser's MXF's. But now - if only for consistency reasons - I settled with MXF.

My workflow is simple:
- for the EX1, use ClipBrowser to export to MXF (after combining BPAVs to merge the file chunks)
- for the nanoFlash, simply copy the MXF's to the HDD
- use MXF on HDD for editing.

As to the workflow speed/convenience, I have 2 very important conditions met:

1. LOTS of FAST disk spaces (currently some 8 TB, including external archives)

2. Very fast SxS reader (also in the PC, it's PCIe based - not USB).

3. Equally fast CF card reader (again, using PCIe->ExpressCard->CF adapter).

"2. Very fast SxS reader (also in the PC, it's PCIe based - not USB)."

Just out of curiosity, Piotr (sorry), what are you using?

Jay,

I'm using this one:

http://www.synchrotech.com/products-expc/pcie-expresscard-slot-drive_03.html

- but there is now plenty of other re-branded models for less (AFAIK, they're all made by the ST-Lab).

Piotr

I actually just use the SONY SxS reader to copy the files direct to drive (in this case I store everything on a Drobo). I edit directly from the BPAV structure. Most of the time I'm working on projects from start to finish, but on the odd occasion where I'm passing material to other producers, MXF has been helpful. In those cases I use XDCAM Clip Browser to find the files, and then export them to a separate directory, bundle it up and ship it to whoever needs it.

The features in the Clip Browser (as has been stated before, CRC, de-flash-band, metadata etc) and my mild paranoid streak for untouched source material keeps me using the BPAV/mp4 structure.

In Vegas I've never had a problem with BPAV. I have a multi-monitor setup (timeline window, preview monitor, and then a "junk" monitor where I put utilities and other stuff...and that's where I keep Clip Browser open so I can quickly look at files and then refer to the file/directory number to drag/drop from Sony's viewer. It would be nice if you could see clips in Vegas, but my current workaround works fine.

I find it too time consuming, but perhaps I am doing something wrong.

Unless I shoot a theater production I typically shoot very short segments (a few seconds to a minute). Typically that leaves me with between 60 and 180 .mxf files for an hour of video.

I am using Vegas 8 so when I import the 60 - 180 .mxf files into Vegas I run into the following problems:

1. You can't double click them to add them to the timeline. I am not sure when that feature was removed. I went from Vegas 4 to Vegas 8. I could do that with Vegas 4.

2. If I select a bunch of them and drag them to the timeline there are always gaps in random places between some of them. That results in the following time consuming scenario.

3. Stretch out the timeline really far. Slowly scroll along the timeline looking for the gaps. When I find them - Select events to end then drag the timeline closed and move along. Of course Select events to end means to the end of the next gap so I have to do this roughly 10 times or more depending upon how many .mxf files I have up there on the timeline.


Really time consuming and annoying.

Is there a way to join all the .mxf files before I import them into Vegas? Working with DV was much much easier.

John

Thank you, Piotr.

John, like I said to Bob, I can't imagine what you're doing to get such results, since I haven't watched you work.

I import the BPAV file(s) onto my hard drive (1TB). Using Clip Browser, I convert the video files to MXF.

Then I open Vegas (9.0e), go to the Project Media tab, go to the folder that holds the clips, select them all, if required, which brings them into the All Media window--there they are all already highlighted. I can drag them all onto the TL and they are perfectly aligned, no gaps whatsoever, along with their audio tracks.

For me, it has been as easy as working with DV.

I can fully confirm Jay's experience with mxf files.

For what it's worth, I just took one of my full 16GB SxS cards and copied the BPAV folder onto the hard drive. That took about 8 or 9 minutes.

Then I opened Clip Broswer and coverted about 55 minutes of clips to MXF files in another specified folder. That took about 10 minutes.

I opened Vegas and brought in the clips to the TL and started editing.

Total time, from inserting the SxS card to ready to edit, was right at 20 minutes.

With tape, I'd still have had about 35 minutes to wait for the real time transfer.

There is no way files should be taking as long as tape. If it is, there is something wrong somewhere.

...and I assume you were copying at no faster than some 30 MBps; while the PCIe adapters runs at 90 MBps sustained (with original SxS only, of course).

Piotr

Piotr, now I have to reveal my ignorance. I really don't know the rate of transfer. I was using the Sony SxS card reader which connects via USB.

Yes, I am certain what you're using is considerably faster!

Jay,

In fact, it's so fast that you actually need a pretty fast HDD, or it will become a bottleneck!

One thing to remember about those PCIe->ExpressCard adapters: unlike the laptop ExpressCard readers, they are NOT hot-swappable. Meaning that in order for the OS to recognize an SxS media, you need to boot your PC with an SxS card in the drive.

However this is not a huge problem at all - considering that we usually put our PCs to sleep rather than turn them off for the night. So, at those rare occasions when I have to reboot, I just need to remember to stick an SxS card in, and I'm good to go until the next reboot!

BTW, I don't even remember when I rebooted my PC last time :)

Piotr

Certainly using SxS cards with a PCIe interface reader is FAST.
On the other hand I have 10x 16GB SDHC cards in MxM adaptors and transfer speeds over USB is SLOW. One way I will investigate to speed this up is transferring 4 cards at once in parrallel.

If you want to make an apples to apples comparison of speed you should include all the process and that's not being done.

Time to ingest a 85min HDV tape with media on T/L ready to edit AND and archival copy on the shelf. Around 100 minutes, Vegas does take its time building waveforms. To date using Sony's "HDV" 85 minute tapes not a single dropout. I'm very lucky as I'm able to buy these at half price.

So far no one is including the time to make an on the shelf archival copy of their media nor considering the cost of the media. A 32GB SxS card is around $1,100 down here, SDHC equivalent is 10% of that so one can afford a lot of them. They approach the point of being cheap enough to use for at least medium term archiving but they are slow to access. The UDMA CF cards that Piotr is using in the Nanoflash are faster and more expensive per hour of video.

The problem I see here is people are comparing different needs and from those needs each derives there own workflows, certain aspects maybe mission critical to some and irrelevant to others. This does make it hard to do an apples to apples comparison or even advise on best practices and it gets even harder if you need to consider capital investment.

Clearly if your needs are ENG time to air with a decent budget, tapeless especially using SxS cards is a winner by a huge margin. On the other hand one of our clients shot around 30 hours of EFG footage and to get it into their systems had to transfer it in realtime to Digibeta! Tapeless still won compared to shooting DB, those MxM cards easily fitted into the pocket of a jacket, wrangling 50 DB camera tapes in a war zone would have been an issue.

Bob.

I agree Bob that I have stressed the positives, but please admit that on the hand, you're presenting the worst scenario (SDHC being used instead of SxS, archiving as separate process, etc.).

So I should have added perhaps that before step 1 of my workflow described earlier, I use ShotPut to copy SxS contents into two (or for very important projects - three) HDD locations simultaneously - the new version is very fast indeed. This way I have 1 or 2 archives, and the working copy on the local HDD to do the merging and exporting for mxf from the ClipBrowser...

Piotr

"but please admit that on the hand, you're presenting the worst scenario (SDHC being used instead of SxS, archiving as separate process, etc.)."

Of course, no problem admitting that. In fact feel free to take me to task whenever I don't present worst case scenarios.

On the other hand I also included a best case scenario when the XDCAM EX workflow is the fastest there is.

Bob.

John, like I said to Bob, I can't imagine what you're doing to get such results, since I haven't watched you work.
------------------------------------------------------------------------------------------
Two things Jay;

1. I am doing the exact same thing as you are.

2. I don't have a problem when it is a "few" .mxf files, but when it's 60 - 180 of them I get gaps every time.

How many do you drag to the timeline typically? 10 or 20? I don't have a problem with 10 or 20 either.

How closely have you checked for gaps? They aren't noticeable unless you look for them. They are little blips between events when you preview. Then you have to stretch out the timeline really far to see them.

Since I am doing the exact same thing that you are, it's either that you are using less .mxf files or maybe you aren't looking for the gaps. This problem is consistent on my Vegas 8 timeline with large numbers of .mxf files. If I were able to join them all I would do it in a heartbeat.

I also remember a script of some sort that looked for gaps a few years ago

John