Sony V8 + Sony CF recorder = TERRIBLE

bruceo wrote on 5/27/2008, 5:57 PM
I've posted on this before, but now another 60 days later and 8 projects with about 70 hours of footage recorded by Z7U onto CF and not one project is Vegas able to complete. We can eventually get 8 hours of clips from the CF recorder into vegas and usually start editing, but after a day or two of cutting then Vegas will hit a frame in the thumbnails or preview window and then the video buffer or MCdecoder crashes. Sometimes we can figure out the clip vegas doesn't like and rename it and move on for a while until it fubars again. Any clips that vegas doesn't like play perfect in WMP or VLC player. So we end up wasting all those man hours and having to start from scratch with a recapture of all the source tapes.

So since solid state recording results in multitudes of clips for every start/stop and Vegas takes a reeeediculous amount of time loading those projects because of the large amount of source clips, you'll have to wait 5 or 10 minutes for the project to load then end up watching the thumbnails load in and then crash the project making it almost impossible to figure out the source clip causing the crash.

I also continue to find that Vegas often doesn't even like its own M2T output because it will black frame or flatline audio sections on m2ts dropped into the timeline that were originally renedred output straight form Vegas. WTF SCS?

Is there ANYONE out there ANYWHERE using the Sony CF recorder for weddings or long form creative shoots with 6-8 hours of clips that is having any success?

Comments

farss wrote on 5/27/2008, 6:46 PM
Can't help you directly however turning off thumbnails and waveforms under the View menu may reduce your grief.

Bob.
bruceo wrote on 5/27/2008, 8:13 PM
That works, but then I can't edit efficiently without thumbnails or waveforms. It would be like editing on an avid
Pedro Rocha wrote on 5/27/2008, 8:20 PM
Try disabling the "smart rendering" feature in preferences menu. In some cases it worked for me.
Laurence wrote on 5/27/2008, 8:47 PM
I use the Z7 and while I do mostly shorter promos, I do it from a library of hours of archived footage.

Maybe my workflow might help you.

I find the following limitations to be true when you work with native .m2t clips in Vegas:

1. You can't have too many in a project at once or Vegas will crash and burn.
2. You can however have an equivalent amount of footage in Vegas if it is in fewer separate clips.
3. Vegas doesn't play back .m2t clips that well when you are selecting just one channel of the audio like what often happens when you have a shotgun mic on one channel and nothing on the other.
4. Vegas can smart-render .m2t video but not the audio.
5. Vegas can embed markers in the .m2t clips.
6. Vegas users have access to some excellent scripts including Ultimate-S and Excalibur.

I do mostly documentary style projects where you have lots and lots of source footage from which you need to edit.

I thought about it and came up with the following workflow which seems to work well with Vegas's particular set of strengths and weaknesses.

Here it is:

When I come back from a trip like the one I am going to take to Peru next month, I will have seven or eight tapes full of pretty cool footage, but with lots of junk as well.

1/ I go through the tapes one at a time and with each tape I put all the footage on the timeline.

2/ I edit them down to the stuff I might use, clearing out the bad takes and junk

3/ Then run the "add markers to events" function in the Ultimate-S script.

4/ Some of these clips will be ones that I know I will use. On the important ones I name the markers with descriptive names that will be useful later.

5/ I select the audio for all the clips that sounds best. For example, if there is a shotgun on the left channel and nothing on the right, I choose just the left channel on those clips.

6/ I select all the clips with dialog that might be useful and run the "normalize selected" feature from Ultimate-S.

7/ I smart-render the whole timeline into a new combined clip with embedded markers named after the tape, and maybe something descriptive about that tape.

8/ I repeat the whole process on the next tape.

I vary a little from this if I am doing interviews. In that case I edit everything down to the answers to the questions that I might possibly use and at each marker I put in a title which states the question. Then I smart-render the whole interview to a clip named after the person being interviewed.

What I end up with is a bunch of large clips with names like Costa Rica trip tape 3, or Brent interview, etc. When I load the clips into Vegas, there are markers everywhere the camera starts and stops, If I want I can resplit them using Ultimate-S "split at embedded markers" command. With the interviews, I can immediately go to exact questions. All the audio with dialog is normalized. Editing from there is an absolute treat.

I don't know if this will help you much, but I just love working this way.

With my Z7, I use the CF cards except when I travel. Then I use tape since I can't dump footage until I get back to a computer. At this point I am finding the card recording to be reliable enough that I'm not even bothering to record to both anymore.
farss wrote on 5/27/2008, 8:48 PM
Gosh if that's all it would take to make Vegas like editing on an Avid I'd be overjoyed. It would be nice though if we could control thumbnails and waveforms independantly.
While we're on about waveforms, I recall seeing a quite old DAW that drew real envelopes, not waveforms and they were log not linear.

Bob.
Spot|DSE wrote on 5/28/2008, 7:56 AM
Bruce,
To what cards are you recording? I've seen corrupt headers with slow CF cards, and Sony specifically warns about this as well. I've not had a problem with any card that Sony or Lexar specs as being fast enough.
You also might try this speed test on your cards;
http://www.kenrockwell.com/tech/cf-sd.htm
Guy S. wrote on 5/28/2008, 10:08 AM
<I thought about it and came up with the following workflow which seems to work well with Vegas's particular set of strengths and weaknesses.>>

Great suggestions, and very clear. Thanks for taking the time!

Guy
Coursedesign wrote on 5/28/2008, 10:13 AM
Spot is right, this definitely sounds like corrupt headers from slow CF cards.
Darth A Booey wrote on 5/28/2008, 10:59 AM
Great suggestions, and very clear. Thanks for taking the time!

+1

Thanks Laurence!
bruceo wrote on 5/28/2008, 2:30 PM
Transcend 32 GB cards. I have 6 of them. All of the footage looks fine in WMP or VLC player. Even the ones which Vegas pukes on don't show any glitches. If crappy WMP can playback the footage then Vegas should be able to handle them. I actually used to have the same problems with Vegas crapping out the same way with m2ts also flat line audio and red and black frames and everyone would say that it is something wrong with the M2t's. I was so frustrated that I purchased CS3 & FCS, but then V8 was released at the same time (literally as i was installing CS3) and all of a sudden all of the supposedly bad m2ts worked great now in V8. Now that the solid state HDV recording is here Vegas is puking again with horrible load times and ungraceful crashes. If the footage can be played in finicky WMP then Vegas should be able to handle them without fatally crashing 6 entire projects with 20-40 manhours on each edit.

If someone is doing this kind of volume of work in HDV using this CF recorder I want to come visit you. I also invite anyone to come by my studio and see this demonstrated on 5 different machines and witness it form themselves.
Laurence wrote on 5/28/2008, 3:11 PM
Bruceo, check out http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=454045this old thread[/link] from 2006.

It may not be clear from the thread, but back then, Vegas would crash if you went over about 80 m2t clips on a timeline. You could have as much footage as you wanted, but it had to be spread across less than something like 80 clips. As you will see from the thread, some people like me had a huge amount of trouble with this issue and others who worked with fewer larger clips hadn't even noticed it.

This problem was eventually fixed... sort of. Remember that in working with long GOP, the first frame of the clip that you are working on my not be on an i-frame (a fully drawn frame), and so an editing program like Vegas needs to be able to buffer back at least to the nearest preceding i-frame in order to handle the beginning of that clip. What Madison did finally was to increase the number of clips that Vegas was able to buffer. As you are now noticing, the new number might still not be high enough for some users such as yourself.

What happened with me is that since 2006 I have learned to work with fewer but larger clips, and Vegas seems to be optimized better to do that. It was a pain at first, but now I kind of like it.

If you aren't sure about what I am talking about with i-frames and buffering back to the i-frames, there is a pretty good description http://www.tvtechnology.com/features/Focus-on-editing/f_Long_GOP_Editing.shtmlhere[/link] about the pitfalls of editing long-GOP video.

Of course you could transcode everything into an intra-frame codec like Cineform, but as you know, that takes up a whole lot more hard disc space.
Konrad wrote on 5/28/2008, 3:39 PM
What speed are your Transcend 32 gb cf cards? 133x?
Spot|DSE wrote on 5/28/2008, 4:24 PM
I'm doing very long form work...Typically killing off several 16GB cards a day, transferring them to a Merax Databank, and then dumping them to a storage RAID for editing at day's end. We've yet to see any problems from THESE cards. I have seen problems with the Transcend cards.
How long are the clips? are you sync'd, or are you using CF only?
bruceo wrote on 5/28/2008, 5:51 PM
Laurence, I'll have to try the projects broken into smaller sections and see how that works although in most cases it seems like all the clips will work fine in 1 large timeline and then all of a sudden a few project iterations later it starts crashing.
bruceo wrote on 5/28/2008, 5:59 PM
They vary from a few seconds to 1 hour, but sometimes the clips vegas pukes on aill be 15 seconds or an hour. It seems typically it will crash when it thumbnails a frame that it doesn't like within a clip. So you might be at a certain zoom level with no problems and then you cut and zoom and it rethumbs and then crashes and will never recover wasting the entire project. If I drop all the m2ts and try to render a large m2t the render will freeze, apparently at the frames that Vegas pukes on. The only workaround is to dump the CF footage and capture from the tapes.

As far as synched. The CF recorder fails to sync properly with tape. I noticed the DR60 will usually wait for the tape to record before recording, not so with the CF. If the tape goes to the anoying standby the CF will still record and if you power up from standby and start recording the tape will record faster than the CF and if you started recording on tape before the CF boots up then the CF doesn't notice that the cam is recording and will not record. This has happened quite often and is very annoying for run and gun shooting.
Laurence wrote on 5/28/2008, 8:01 PM
They vary from a few seconds to 1 hour, but sometimes the clips vegas pukes on aill be 15 seconds or an hour. It seems typically it will crash when it thumbnails a frame that it doesn't like within a clip. So you might be at a certain zoom level with no problems and then you cut and zoom and it rethumbs and then crashes and will never recover wasting the entire project. If I drop all the m2ts and try to render a large m2t the render will freeze, apparently at the frames that Vegas pukes on. The only workaround is to dump the CF footage and capture from the tapes.

That sounds like you are getting data errors in the m2t clips. I have had that with tape captures before I learned to turn off my virus protection when capturing from tape, but never yet with stuff recorded to CF card.

I have noticed that the Transcend cards don't start and stop exactly at the same time as the tape. They tend to start earlier and stop later. Maybe you are getting errors because of the fast start and stops where the 133x rate of the Transcend cards isn't keeping up. I have had no problems recording to the Transcend card alone, but the one time I went to both tape and card I did have problems. I have done four or five sessions with just the card recording alone though and had no problems.

Maybe you need to use a faster card when using both card and tape. That would be bad news I suppose seeing as you have so many of the Transcend cards. I have two Transcend cards: a 16GB and a 32GB. I have found both to be quite reliable except on that time when I went to both card and tape. After that I did a computer full reformat (as opposed to the camera quick format) of the card and it seems to have been fine after that, but maybe it was the simultaneous recording rather than the full reformat that made the difference.

I know you aren't going to trust a paid job to just the card, but maybe you can record something like a family event to just the card and see if you are having problems when you record to the card alone or if it is just when you are recording to both at once.
apit34356 wrote on 5/28/2008, 8:54 PM
I think DSE and Coursedesign are correct about it being a CF card problem. Response time on write and read cycles can be misleading. The camera only has limited buffers and really must push data out. The CF card is probably failing as data transfers approaches burst speeds and the write cycle is being cut too short for a few cell bits to be properly set. ie, like 98% of the bits can be accessed and written to at MAX or 120% over, the remaining few bits require speeds of less than 70% of burst mode over a long write sequence. Card is ok for most apps, but not for continuous writing.
Laurence wrote on 5/28/2008, 9:09 PM
Then why am I and several others on the DVInfo forum having such good luck with the same card? Even odder: why is Bruceo having trouble with six separate Transcend cards and why do both mine work? We must be doing something different.

I have done a bunch of short clips with the my Transcends as well as some longer interviews. So far the cards seem quite reliable no matter what I do. The one difference is that I tend to use either the card or the tape whereas Bruceo is capturing to both at the same time.

Then you have Spot who is using both the tape and the card, but using a faster card.
Laurence wrote on 5/28/2008, 10:18 PM
Bruceo, I have started a thread about the possibility that the dual record mode (tape and CF simultaneously) on the Z7 is messed up. You can find it http://www.dvinfo.net/conf/showthread.php?t=122623here.[/link]
Spot|DSE wrote on 5/28/2008, 10:27 PM
As far as synched. The CF recorder fails to sync properly with tape. I noticed the DR60 will usually wait for the tape to record before recording, not so with the CF. If the tape goes to the anoying standby the CF will still record and if you power up from standby and start recording the tape will record faster than the CF and if you started recording on tape before the CF boots up then the CF doesn't notice that the cam is recording and will not record. This has happened quite often and is very annoying for run and gun shooting.

This is a classic description of what Sony BPSD says will happen if slow CF cards are used. In our case, tape and CF are always sync'd, always. But, we don't use larger than 16GB cards, and we use either the Sony cards or the high end Lexar cards.
apit34356 wrote on 5/29/2008, 12:51 AM
Laurence, nice work on the scuba diving video.

"Then why am I and several others on the DVInfo forum having such good luck with the same card" Again, in the real world of manufacturing, production runs of ics and their seconds( almost 100% functional to ic's specif's ), do encounter production errors from materials that are subpar. The specific production run may meet 85% of the performance requirements. With the flash market suffering too much medium size CFs, the profit margin lies with a few large CFs where they get a better price. So, a few companies have been questioned about flooding the CF market with questionable product. Plus, there is a major market of knock-offs occurring as well as companies buying production "seconds" or slightly defective ICs targeted for destruction and reselling them as named brand product.

Of course, his specific Sony camera could be pushing the address lines too quick, where the camera's interface is at fault for this specific product CF.

But there has been a number of reports of CF failures over the last two years.
Laurence wrote on 5/29/2008, 4:08 AM
Bruceo, there are two different ways to setup simultaneous record to CF card and tape.

One is to use the Z7 as it comes set from the factory with the Z7 in tape only mode and the card writer unit in "follow" mode. This is the same way you would use the card writer with a camera not designed for it like a HVR-Z1.

The second is to put the card reader in the proper mode for the Z7 and S270 and put the Z7 in the mode where it records to both card and tape.

One of these might be more reliable than the other. Have you tried both?
bruceo wrote on 5/29/2008, 9:45 AM
This is a classic description of what Sony BPSD says will happen if slow CF cards are used. In our case, tape and CF are always sync'd, always. But, we don't use larger than 16GB cards, and we use either the Sony cards or the high end Lexar cards.

Spot, The major sync issue has nothing at all to do with the speed of the card.

Turn your Z7U on and immediately hit record. The tape mechanism will start recording and the CF will not because the firmware has not even loaded up fully and when it does it does not recognize the camera is recording and will NOT record. This happens by instinct often when something happens that we do not expect and we have to turn the cam on and instinctually we hit record to start record as soon as the tape mech comes online. Now that we know that the CF unit takes longer to boot up, now we have to wait until the CF indicator pops up on the LCD.

CONVERSLY. Let the cam sit for a while without recording so that when you hit record you get the STBY, STBY, STBY for somtimes as long as 10 seconds..... The CF starts recording as soon as you hit record and the tape records as soon as it gets out of standby.

SO just these two factors alone for a real run and gun shooting environment results in quite a bit of not only out of sync clips but usually many minutes more captured on tape and seconds more off in the other direction on CF.
bruceo wrote on 5/29/2008, 9:53 AM
It is messed up, but HDV is not even frame accurate anyway. EVEN Vegas renders do not match up.

Loop a 5 minute section and render to HDV 60i. Take the resulting m2t and drop it back into the loop and drop the opacity 50%.... I've never seen it match up after 100s if not thousands of HDV renders in 4 years. Even back when I used to do proxy renders to avi I noticed that they would always be off, which is why I abandoned proxy rendering back in the day. I bought gearshift back then, but I never used it because at the time i ended up going the cineform route (which had its own set of problems) and I never heard anyone using gearshift bitching about synch issues, so I wondered if Gearshift incorporated a frame shift that corrected this or just the users were OK with having their footage 2-3 frames off their original cuts.