PC Magazine Review - Performance Conclusion Right?

WillFastie wrote on 4/28/2003, 9:49 AM
I'm a bit confused by the performance results described in PC Magazine's review (http://www.pcmag.com/article2/0,4149,1017029,00.asp). I thought Vegas and Premiere had comparable performance, yet the review has Vegas doing rendering tasks at half the speed of Premiere. I also thought both products used the MainConcept encoder, which should level the MPEG-2 playing field.

It looks like the test case the reviewer used was pretty simple. I've seen some posts here that suggest that complicated timelines require more rendering times. Was the reviewer's use of a chroma key a factor?

Can anyone shed any light on this? I just switched to Vegas from Premiere and hope I didn't dig myself into a hole.

Comments

Jason_Abbott wrote on 4/28/2003, 11:16 AM
Others may be able to address render speeds directly ... I would just point out that Vegas, at least for me, is much faster in what matters: work-flow. I can put stuff together a lot faster with Vegas than I did with Premiere.

As for rendering in general, I can load a second instance of Vegas and have it render (just one copy of Premiere open would regularly crash for me) or, for long projects, I let them render overnight, same as I did with Premiere.

And what I do personally is pre-render my timeline as I go, whenever I take a break, so by the time I'm done everything is rendered--a poor man's background render. :)

- Jason
DDogg wrote on 4/28/2003, 11:51 AM
I tried to find an older thread that dealt with "smart resample", but could not find it. If I remember correctly, when resample kicked in the render times went to hell but many times you could turn it off. I guess it wasn't so smart all the time.

While I do think Premiere does some things much faster (try rendering audio out to wav with both, ouch!), I just don't see it as an issue, at least for me - Especially given the productivity increases and a real windows interface that leverages my previous windows experience.

Like the previous poster mentioned, and whether it be Premiere or Vegas, most folks would do the final render on long timelines overnight and any difference is not a big deal. Oh, also, with Satish's new frameserver, external and sometimes faster mpeg encoders can now be used easily.

Hmmm, "WillFastie", that seems to ring a bell from years ago in the software biz...Mag reviewer/commentator? Maybe just my memory going bad. Seems to happen more lately :)
WillFastie wrote on 4/28/2003, 4:32 PM
Creative Computing IBM column starting in 1981, then EIC PC Tech Journal, plus hundreds of other articles until a few years ago.
rextilleon wrote on 4/28/2003, 4:35 PM
The problem with the reviewer is that he didn't give us a sense of what his configuration and what he tried to render-----It's a function of processor speed and I dont recall him even mentioning what he used.
Paradox wrote on 4/28/2003, 4:56 PM
Time and again, I find myself knocking projects out with Vegas much faster than I did with Premiere or Speed Razor. Two reasons: stability and workflow. In the end, I am doing much less rendering. In fact many projects ONLY require final rendering. I find that glitches in setting stuff up in transitions and effects are virtually eliminated with the ease with which I can work with Vegas.

Rendering itself may be the same or even a little slower. But in the end, for me, that's not what makes the big difference.
WillFastie wrote on 4/28/2003, 8:28 PM
In fairness to the reviewer, the small space allotted for the article doesn't leave much room to provide every detail. Even so, he did mention at least a couple of items on the timeline.

What I've heard here so far is that Vegas may, indeed, be slower at rendering but that it doesn't matter because the workflow is more important and it is faster.

That about right?
rextilleon wrote on 4/28/2003, 8:56 PM
I think you are right Will--remember, Vegas is a software solution--If you need rendering speed you need hardware and you need to spend money--Doesn't make any sense to me when you are editing DV.
rextilleon wrote on 4/28/2003, 8:56 PM
One other thing---Render times increase with fx, compositing and transitions---If you do lots of straight cutting then renders are fairly fast.
WillFastie wrote on 4/29/2003, 9:01 AM
I'm not comfortable with the notion that rendering performance is of less importance for software-only NLEs.

The impression I get is that many of the participants here and in other forums are pros or at least semi-pros. Such users can manage time and workflow to accomodate tools, as do professionals in any field.

But the expanding market for digital video is consumers, for which this will be a hobby. The idea of rendering a 1-hour movie overnight isn't exactly compelling stuff in that context.

Anyway, that's a bit off-topic. I'm still wondering if the PC Mag review is right or wrong?
d1editor wrote on 4/29/2003, 10:09 AM
I just wanted to take a moment and reflect on the rendering time for Vegas 4. My partner still uses Adobe Premiere 6.5 and to be honest- the render times are very similar to what I have been experiencing. He cut his teeth on Adobe and it is all what your used to. He "plays" on the V4 machine and he likes what he sees and he may become a V4 convert yet! We do alot of projects for a Fortune 500 company, who has an in-house AV department. We provide 3D animation for them and generally follow the project to completion. They have an Avid Media Composer 9000 XL (2 moths old) on a packed Mac G4 Dual 1.25 GHz. They had to deliver a DVD to their customer- a final length of 8 minutes! It took this hardware intensive Avid 3 hours to render an MPEG 2!!! I happened to have my laptop with me (for any graphic needs) and captured SVideo (and audio) out from the Avid into my system. I took that captured footage into my V4 system and rendered out an MPEG 2 in 12 minutes! I cut a DVD and went back to my customer to compare the quality of the DVD to theirs using the Apple DVD Studio Pro.(Currently I use Sonic DVD Producer) We had the entire department trying to see anything different- and could not! Their editor is looking into V4 for his own use at home!
Also...I edit a weekly TV show using V4- usually 6 video layers and 8 audio layers with tons of keys and compositing. This 28:30 length show usually takes 35 to 45 minutes to render. I have several systems, so the render times (and they are not bad) do not cause any downtime for us.... Will- I think you will be happy with the Vegas 4 package, in my opinion, it's fast, easy and kicks @#@ in the DV edit world!
WillFastie wrote on 4/29/2003, 12:49 PM
Mike, that's an informative post. If I read it correctly, your results are that Vegas is rendering in 150% of the length of the video.

Here's some additional information about PC Mag's test. I corresponded with the author to get this (words in brackets mine, not his):

"The first [clip] had four layers
Bottom video
20 second chroma key
20 second spinning logo (360 degree spin)
A title with animation

"Then I dissolved into clip 2 and performed a 40 second 320x240 image pan from the upper left to the lower right. In used original audio in all tracks with one background audio track. Nothing exotic, but some fairly common design elements. I performed all tests on a Pentium IV 3.06 GB with 1 GB RAM running XP pro."

His result was a rendering time of about 475% of the video length.

So, what about the contents of his test timeline would cause such a difference between his test and your experience?
d1editor wrote on 4/29/2003, 4:37 PM
Will,
From past experience, when you use the Pan and Crop- the rendering time increases. I use this function on 25 meg, 300 to 600 DPI images to cleanly create "camera moves" on still images.
My question would be familiarity the author has with each software package, and how he applied the keys and the moves?? There are correct processes and processes that are not correct, but get the job done at less effeciency. Perhaps the author had an increased rendering time do to the proceedure used to apply the moves! The true test would be to have similar machines (we have) and have knowledgeable users of each software perform the same task and time the results... I find it hard to believe the author knew the best way to implement the task at hand on each platform! As stated, my partner uses Premiere and I use Vegas...each platform is on a similar machines... we found equal rendering times on each. Infact, you do not need to render dissolves and effects to "scrub" the timeline as you need to do in premiere. Not sure why the editor stated 475% rendering ratios- I question the methodology since I have not experienced this. I did a video incorporating 50 scanned slides at 600 DPI to create camera moves on each and every slide, along with color correction and in some cases a gaussian blur... in this case, rendering time increased to 300% ratio---> but---> it was the same for Premiere! Hope this helps... Mike
jetdv wrote on 4/29/2003, 7:03 PM
This thread may be of interest:

http://www.dvinfo.net/conf/showthread.php?s=51fb12f754ba21f74da2e8873b6c2539&threadid=9024
WillFastie wrote on 4/29/2003, 8:40 PM
Mike:

The second clip was a video. The pan was across the video's diagonal, a 320x240 window moving from upper left to lower right over the 40 second period. No stills were involved.

What about chroma key? Does that increase rendering time?

Will
rextilleon wrote on 4/29/2003, 9:15 PM
Sure---and changes to your image will increase render time---motion blur is a killer
WillFastie wrote on 4/29/2003, 9:18 PM
Okay, so this is beginning to sound like the test is legit. If the author built the same timeline into both Premiere and Vegas and timed the result on the same machine, isn't that valid?
d1editor wrote on 4/29/2003, 9:38 PM
Will,
Chroma Keying will slow down the render and increase rendering time in Vegas. We are talking about rendering to the pixel! I will ask my partner to test his 6.5

I understand that it seems legit. that would be my reaction also...but, did he know how to optimize rendering and processes on each platform?? That's all I am saying here. My company has Premiere and Vegas in almost every case- render times were very close to being the same. It's all in the user's zone and being able to optimize your platform for efficiency and speed! We seem to be happy with both platforms...and I am moving my associates towards liking Vegas better than Premiere. I really do not think you will be unhappy with Vegas!
We do a weekly show that is 28:20 with Vegas and have never had complaints about rendering taking to long or causing a missed deadline. We are taling about 45 minutes for our show which has keys, motion graphics created in vegas, soft wipes, color correction, and several audio layers.

I can't say the magazine is "Wrong"- I was not there....I can only talk from history and results within my company.
Mike
Paul_Holmes wrote on 4/29/2003, 9:39 PM
I went from the DV500 and Premiere to Vegas. Somehow I never noticed the slower rendering. I do know that on my Athlon 1800, if I render straight cuts I render an AVI in better than real time. Also, doesn't Premiere use the Microsoft codec, which is inferior to Sonic Foundry's proprietary one. Correct me if I'm wrong on this.
Another important point is the ease of running 1, 2, 3 instances of Vegas and rendering in the background. It's a different way of working than Premiere, but you can easily divide many projects into 3rds, and work on one while another is rendering.

With Premier and DV500 I could render a constant bitrate MPEG 2 in 2&1/2 realtime. With Vegas I render a constant rate MPEG2 in better than real-time (that's from a rendered AVI).

So, there are apples and oranges in all of this, but the long and short of it is that most of us who use Vegas would never go back, or up, or sideways for that matter! :)

Anyways, the point is, that article may have been factual, but as others have said it doesn't reflect a lot of advantages that cancel out the long render (like the better codec). To their credit they did point out a lot of the other wonderful features of Vegas.
d1editor wrote on 4/29/2003, 11:17 PM
Will,
Just finished a demo reel for a motivational speaker (boring - but what the heck!)
We did a 3 camera shoot for the seminar (wide, tight, handheld). For the edit I utilized all 3 cameras as a video layer and faded between them (worked great for this project) ...simple cuts and dissolves and a few supers to key. Had 3 layers of audio....SOT, SFX and Music. The very loooong demo (client forced the length) was 26:44. Rendered the project as NTSC DV....AVI files (5.7 gig final)...wanted to let you know....the render took 8:12 ---> less than 1/3 real time
Thought you should know this Will.....
elCutty wrote on 4/30/2003, 1:39 AM
This performance issue may be related to the integration on the MC encoder into V4.0b. When I encoded a clip using the Satish frameserver and an external MC encoder it took 2.6 times of the timeline. Using the internal MPEG encoder with the same setting it took 5.1 !!! times. So internal rendering does not seam to slow V4 down. Many would have expected that useing the frameserver would take longer.
WillFastie wrote on 4/30/2003, 7:42 AM
I did not understand the last post. You said the external encoder took 2.6x and the internal took 5.1x, which sounds longer. Then you said "internal rendering does not seem to slow V4 down." I'm confused.

Last night I did a quick test of a 40-second video clip with a 320x240 pan from corner to corner (similar to what the reviewer did). It rendered to MPEG-2 (MC default) at 2.65x and AVI (NTSC DV good) at 2.13x on a 2.26GHz P4 system.
d1editor wrote on 4/30/2003, 9:25 AM
Will,
How is your system set up? How much ram? Drives? Do you have separate drives for video capture and edit? Striped raid? System bus speed? Rendering is all hardware...give the particulars....
Mike
DDogg wrote on 4/30/2003, 9:38 AM
"... and AVI (NTSC DV good) at 2.13x on a 2.26GHz P4 system."

So what was the timing for Premiere to render the same clip equivalent to DV?

This brings up the point that maybe the first comparison should be render times for internal DV render times. Seems most of the posts are comparing Mpeg render times. There are so many variables that could be at play in that instance. From a "purist testing standpoint" it introduces a third party into the render question. Yes, I agree at first glance it seems like it is apples to apples since they both use the MC mpeg encoder, but pre-filtering and a ton of other stuff may well effect the timings.

If the third party is removed and straight DV render times are compared it gets a little more easy to regress the actions and identify whether a specific sub-task of the complete render is less efficient between the two packages. To define a bit better, a series of comparison renders to DV, which break up the reviewers full test into sub sets may well show one specific area where one program is massively more efficient than the other. Resizing comes to mind right off the bat. Maybe the pans. This would account for the difficulty Will is having getting a definitive answer. It may well be that the speed differences between one program and the other are entirely based upon the specific sub-tasks involved.

/Add Just to add onto that thought, So let's say we see that the render times for resizing are considerably different. Now the question becomes, "Did one do a better job than the other?". If so, it is no longer apples to apples. Longer render times for a better job are certainly acceptable I would think. If not "better" then it might identify a specific area of weakness that, when invoked, could seriously effect render times. If so, that would be an area SoFo could address and perhaps user testing would help them identify it as it is a time consuming process.
elCutty wrote on 4/30/2003, 9:48 AM
DDogg, you expressed it much more clearely than I did. Comparing render times should only be applied to DV output files.

However, do you have any idea why the MC encoder works twice as fast when beeing served by the frameserver? Normally I would expect a 10% performance reduction compared to the stand alone encoder - which could be measured. But internal encoding should be as fast as the stand alone.