PC Magazine review - Vegas one of the slowest renderers?

WT wrote on 11/4/2003, 1:48 AM
This was in October's PC Magazine:

RENDERING TESTS

SOPHISTICATED 12.5-MINUTE PROJECT

Pinnacle Edition 5
Render to DV - Too fast to measure.
Change transition and rerender (DV) - Too fast to measure
Render to MPEG-2 - 35:53
Change transition and rerender (MPEG-2) - 33:24

Ulead MediaStudio Pro
Render to DV - 26:49
Change transition and rerender (DV) - Too fast to measure
Render to MPEG-2 - 34:30
Change transition and rerender (MPEG-2) - 0:01

Vegas 4.0
Render to DV - 75:57
Change transition and rerender (DV) - 0:18
Render to MPEG-2 - 92:59
Change transition and rerender (MPEG-2) - 89:44

These seem to be pretty damning numbers - especially considering that I have always considered Ulead MediaStudio a toy compared to Vegas. I still love Vegas, but what's going on?

WT

Comments

jcg wrote on 11/4/2003, 2:01 AM
No way. Absolutely not. I am doing complex projects up to 20 minutes.

JCG
craftech wrote on 11/4/2003, 5:22 AM
It depends upon what they used for media generated material, filters, etc. and many other factors. I don't mind the wait except when I have to go back and do it all over again because of a bug or a non-intuitive setting which should have been clicked on or off.

By comparison, Pinnacle products are notorious for bugs and problems and the company is virtually useless for product support. The forum can't hold a candle to this one either. That is what makes Vegas worth having to wait to finish a render at times and/or redo a project. It's the people providing tech support and the people on this forum who are anxious to help whenever they can.

John
PeterWright wrote on 11/4/2003, 6:09 AM
A large part of the problem is the priority that rendering times get in these half-arsed reviews.

Myth - It's important that I get my renders yesterday.

Reality - I use Vegas for my living every day, and even if Vegas took two or three times longer to render than it currently does, I wouldn't dream of using any other program.
Rendering is a reality, so it gets planned for.

That doesn't mean I wouldn't use faster speeds - but it really isn't that critical. There's too many other things to be getting on with.
Tampa wrote on 11/4/2003, 6:15 AM
My numbers are more like jcg's, but I have only used a DV camera as input! I haven't seen the article yet. Was their source a DV camera, or files already rendered by another application?
TheHappyFriar wrote on 11/4/2003, 7:27 AM
This wa actuatly posed back in September. Sorry to tell ya. :)
farss wrote on 11/4/2003, 7:38 AM
BTW why didn't they include MGI Videowave in their review.
It renders faster than anything I've ever come accross. The again its limited to one video track and the audio needs to be extracted to a separate .wav file if you want to do anything with it. But render, about 2 minutes for 15 minutes of video.

I only mention it to show silly some of these comparisons are.

However some of the competition is using the GPU on the graphics card. The first time I watched the demo of Serious Sam it occurred to me that if a video game can do all those amazing things at over 100fps why isn't that horsepower being used in video. Damn, should have patented the idea!
Grazie wrote on 11/4/2003, 7:50 AM
farss - Now it's my turn . . I couldn't have put it better . . . well said . . it's something that has been puzzling me too.

Grazie
JJKizak wrote on 11/4/2003, 8:00 AM
All I know is after rendering a 78 minute slide show with P3 1 gig it took 7:05 hrs.. I saved the project over the network to my P4 2.8, 800 with Hyperthreading then rendered the project again and it took 2 hrs 23 minutes.
Am I happy? You better believe it.

JJK
Spot|DSE wrote on 11/4/2003, 8:14 AM
1> This was an old review.
2> They didn't use media captured with Vegas, they used media that was captured via some other app or media that was already in computer
3> Vegas is indeed a render pig, but it's not as exaggerated as most of these reviews make it out to be.

Funny, in this month's DV magazine, Adam Wilt actually uses Vegas to talk about 24P, rendering to and from 24P, and how fast Vegas does it. In the past, Adam has been so anti-Vegas it was funny. Many trade shows, I've talked with him and he's lambasted Vegas several times. Now, it's accepted and loved by him. Just that alone says how the market's perception is shifting. Maybe one of the consumer-oriented mags like PC will finally get an educated reviewer who actually understands video, knows what 'codec' means, and knows what DV means vs uncompressed. (jan ozer's review compared rendering to Vegas uncompressed to Premiere's DV render)
BillyBoy wrote on 11/4/2003, 8:44 AM
I think SPOT summed it up best. When, actually if PC MAG ever gets an "educated" reviewer capable of reviewing NLE software fairly and objectively, then maybe they'll have some creditability. Till then, I'd give Homer Simpson's review of video editing software as much or more weight. BTW, I haven't subscribed to that rag for years. Duh!
TheHappyFriar wrote on 11/4/2003, 9:33 AM
What really annyos me is that OpenGL has been out for years from SGI, started being used in Games in the mid-late 90's, games started supporting DUEL processors in the laste 90's (Quake 3 did in 99), and most NLE's don't supprt any hardware features unless you buy a SPECIAL hardware card! Just redicilus!

WT wrote on 11/4/2003, 1:15 PM
Hey there,

I too wonder about the exact circumstances of their testing methodology. Because of this, and the fact that PC Mag isn't exactly a serious NLE or industry publication, I take this article with a few grains (or a whole shaker!) of salt. Even IF these rendering numbers are true, there's no way it's enough of a reason for me to leave the brilliant interface to spend oodles of money on an otherwise lousier package. I'd waste way more time learning how to use a competitor's interface than I'd waste waiting for a render.

I wasn't trolling, I assure you (you'll be hard pressed to find a bigger supporter than me of Vegas on both audio and video sides) - however, it is kinda disturbing that such a popular consumer magazine would print something like this that would cast doubt on quite a few potential buyers. In fact, fairly recently another local freebie consumer mag/paper in my area also published some bad looking rendering numbers for Vegas.

Is it really that hard to do a fair standardized rendering benchmark?

BTW, sorry if this was a repost Friar, I'm usually so happy with Vegas that believe it or not, I don't even need to check the video forum. Believe me, that's an outstanding achievement! ; ))

WT
BillyBoy wrote on 11/4/2003, 1:24 PM
I gave up on PC Magazine years ago because rarely did they do a serious or objective review regardless if reviewing hardware or software. This of course if just my personal opinion, however after reading several PC Mag reviews of hardware/ software I personally own and am very familar with, I just couldn't and wouldn't take them seriously. They just miss too many things. Sad, because when the magazine first came out and awhile after when it was in its heyday where it came out twice a month and would run 600 pages or so it was top notch. Not for a long time. Ditto for Windows Mag. which also once upon a time was decent and before it went belly-up it too went Hollywood, lacking meat and having mostly fluff.
Maverick wrote on 11/4/2003, 2:22 PM
Could it be possible that, given all else is equal in Hardware/Software apart from the diting apps, the faster rendering times could actually produce inferior results beause of the algorithms used?
busterkeaton wrote on 11/4/2003, 2:30 PM
That is certainly true. It doesn't mention that that is the case here, but the preformance metric should not be how fast, but how long does it take to get good output? Even Video magazine sometimes forget to mention this, let alone something like PC mag.

You see this with DVD drives too, they always give you a performance graph, but do they also talk about the final quality and compatibility of the burned DVDs?
BillyBoy wrote on 11/4/2003, 4:10 PM
That's the point. With PC Mag, they rarely if ever mention what they used for testing and they don't usually show source, or final render or say exactly what they did in the way of manipulating the image so just saying product A renders in X minutes and product B renders in Y minutes is next to useless. Of course they rarely mention the QUALIFICATIONS of the guy doing the testing either. Usually its just some freelance writer that don't know boo about what he suppose to be reviewing, Fed-X drops off the application, he installs, and a few weeks later you seen the review in the magazine. It not like UL labs or even consumer magazine that actually has a lab and puts whatever they're testing through a set of rigiid tests under carefully controlled condtions where they also publish the testing methods to the nth detail. They USED to do that, years ago. Now its more likely some Joe Blow sitting in his underwear in his apartment fooling around with software he's never seen before.
kentwolf wrote on 11/4/2003, 7:35 PM
Years ago, I used to believe what PC Mag said. I would make purchases based on their word.

That all stopped some years ago... When they gave awards to apps that were horrible (like Pinnacle), they lost all credibility with me.

The best place for *real* reviews are forums just like this...
24Peter wrote on 11/4/2003, 10:55 PM
I tend to agree with BillyBoy and Kent on this one. I cancelled my sub to PC Mag a couple of years ago because of their cursory testing and abrupt slant to corporate IT issues. Even PC World (which I still read) blew it recently in their review of MPEG-2 encoding software for DVD's. I ran out and downloaded NeoDVD (Mediostream I think) because their encoding times (according to PC World) were nearly twice as fast as the next nearest app. Problem was, the quality sucked. I mean, it was awful. It was hard to believe they were selling this product. And you couldn't adjust any of the settings - not even "good" "better" "best" type of thing much less actually choose a bit rate. I got my money back from the company and now disregard most of the PC World's video related reviews.
kentwolf wrote on 11/4/2003, 11:33 PM
...another side note...

I have actually seen where:

1.) Reviews were done. "Hey everybody, the product's great."
2.) People buy, then see the product is garbage.
3.) The reviewer in question gets called on it (which is extremely rare) in Letters To The Editor, that kind of thing...
4.) The explanation is given (words to the effect of):

"Well, we only had a beta version to test so we could get it, the review, in by press time. [The SoftwareMaker] assured that thus-and-such issue would be fixed by the final release, and apparently it wasn't. We're sorry. Had we known..."

...and that is how some reviews are done by Big Name computing magazines. I read it myself...not an urban legend...

Too bad...

Back in the day when you had to actually pay money for ZDNet on CompuServe, it was pretty good. No more.

Someone ought to write a review on computing magazine reviews... Are the reviews accurate, thorough, do they work as advertised, etc...

...that would be interesting...
BillyBoy wrote on 11/5/2003, 12:07 AM
That reminds me.... Anyone remember that picutre of Peter Norton (in a white shirt with his sleves rolled up and his arms crossed? He used to write a column for PC Mag and others.

I started to sour on PC Mag way back in the late 80's I think it was when some fly by night company came out with some "turbo" thing that was suppose to really boost throughput for Windows doing something. So long ago, I forget exactly what it was suppose to do.. Originally they "tested it" in PC Mag Labs and said something to the effect oh boy, this is right up there with sliced bread, you got to get it.

Then the letters to the editor started to pour in saying this is crap it don't do s.... and then they were forced to retrack a lot of what they said and they ended up ripping the product proving it didn't do anything. Nothing. All smoke and mirrors. Just some phony gauges...all eye candy.

Oh... originally this compnay was a big adveriser. You think that had anything to do with the orignal glowing reviews. You think?