8.1 has been out for quite some time now....

Why on Earth would Sony not update Cinescore to work with their own software?

This really makes me wonder how this could be overlooked for so long.

"This really makes me wonder how this could be overlooked for so long."

64-bit operating systems have been out for quite a while and still isn't much 64-bit software available, so it's not just Sony dragging their feet. It's not that Sony has overlooked this, it just isn't particularly easy to do and they have had other priorities.

"64-bit operating systems have been out for quite a while and still isn't much 64-bit software available, so it's not just Sony dragging their feet. It's not that Sony has overlooked this, it just isn't particularly easy to do and they have had other priorities."

The last patch to Cinescore (1.0c) was released almost two years ago. So irrespective of 64bit, how long do you figure qualifies for "overlooked" status?

Jim

>> The last patch to Cinescore (1.0c) was released almost two years ago

So what you are saying is that what is "overlooked" is the development of Cinescore. Obviously this is a product that Sony has not worked on for quite some time. Doesn't look good. Products that go ignored for this long tend to die.

The problem with Cinescore is that it works too good.
JJK

Terje, another possible explanation is that SCS has bin busy working on other products, for example developing the 64-bit version of Vegas. I guess they are working on 64-bit versions of both Sound Forge and Cinescore.

Jøran Toresen

>> Terje, another possible explanation is that SCS has bin busy
>> working on other products

Actually, that is not really the case. Not in software. If a product sells well enough for there to be a business case it will have to be maintained. If a product has not has a patch release in two years it is not maintained. The problem is that to ramp up development on a product that has gone unseen by developers for two years is very expensive.

Now, if this was the only problem we saw out of SCS I would not worry too much. It is not. Vegas 9 was clearly released with only a cursory amount of internal testing prior to release. There is obviously no regression testing going on and SCS has clearly not invested in automated testing tools. If that was the case, Vegas 9 would not have been released in a state where just selecting "render as" and choosing a encoder that is shipped with the product crashes the entire piece of software. Such banal errors are caught with automated tools easily. Clearly SCS are not employing such and that is a very serious problem for the future quality of Vegas.

If the current Project Manager at Vegas gets in touch with me or anyone in the software industry with some experience we can probably give him a lot of pointers on how do conduct testing. He clearly doesn't know how.

Terje, I see you are still completely full of yourself. I love how you use the words "clearly" and "obviously" with no actual inside knowledge or proof whatsoever.

If the current Project Manager at Vegas gets in touch with me or anyone in the software industry with some experience we can probably give him a lot of pointers on how do conduct testing. He clearly doesn't know how.

Terje's post contains so many errors and misconceptions that it's almost like Terje knows nothing at all about software development. In fact, there's virtually nothing true about anything in that post.

There's no 'rule' about software not being 'maintained' if there hasn't been a patch in two years. Each company and piece of software is different. You can't generalize like that. Terje just made that up.

There's no way you can determine the amount of testing some software went through based upon one or two bugs in it. For all you know it started out with 10,000 bugs and they tested the hell out it in order to release it with 20 known bugs. On the other hand, maybe it started out with 30 bugs, there was little testing, and then they released it with 20. You can't know which one of those scenarios is more correct.

There's also no way you can determine if they used an automated testing tool or not. Maybe they did use an automated tool but the tests passed on their equipment. Automated tools can easily miss 'banal errors' if that test scenario was not anticipated during test case development. Besides, just because some company is using an automated testing tool does not mean that software is bug free. I would not expect a render scenario to be automated, anyways. That's a manual test.

And lastly, Project Managers do not conduct testing. In fact, they usually know very little about it. So Terje giving test pointers to a Project Manager is a waste of time. It's like reading poetry to a monkey. The monkey doesn't care and doesn't want to know.

So Terje clearly doesn't know anything about the SDLC.

+1 WJSD

The whole assertions about testing efforts in SCS is orthogonal to this Cinescore issue and really just a distraction.
For whatever reason Cinescore doesnt work with VP9, I imagine that this was a deferred work item. I would be surpised if it was an oversight. It probably just required too much work given the product schedule and resources were prioritized elsewhere on more important (as they see it) features or bugs.

On the testing issue, its rarely possible to make correct assumptions about a situation that you have not seen from the inside.

I do beleive that they did not allow enough time for testing in the schedule.

If i were to guess at the cause of this (and it is just a guess by mapping our process onto SCS which there is no certainty that they use the same process) I'd say that the developers did not reach "Feature Complete" early enough to allow for complete testing, given the schedule. Developers keep working on new feature code during the time allocated for testing. When this happens, the code keeps churning and invalidating test cases and introducing regressions.

I suspect that many of these were ultimately notices, but the decision was made "we must ship!" for NAB. If NAB specifically was missed, then the product must be available as quickly as possible after NAB to leverage the post-NAB interest and sales. Hence the discounted pre-orders.

These are just guesses of course. However, I dont see how anyone could realistically assume that SCS doesnt know anything about testing.
I do agree with you that it's possible that the Product Manager may not know much about testing. That's reasonable considering that Test is rarely the focus of the Product Manager's job. That would be the Test Manager and test team.

>> There's no 'rule' about software not being 'maintained'
>> if there hasn't been a patch in two years.

This is correct, there is no rule about that whatsoever. On the other hand, if consumer of software is updated that rarely, in the vast majority of cases that is because the software is no longer actively maintained. There might be rare occasions where this is not the case, but I have never seen one.

>> There's no way you can determine the amount of testing
>> some software went through based upon one or two bugs in it.

One major thing to point out here, and this is the penalty you get from posting late at night. My main assertion about testing that is or is not done at SCS has in fact nothing whatsoever to do with Cinescore. I have elaborated on that in other threads. My main point is the fact that basic rendering with one of the most important codecs shipped with Vegas doesn't in fact work at all. I should have elaborated on that obviously not assuming I was still in the same thread where i had talked about this already.

Now - how do you know to what level software has been tested based on bugs? That depends entirely on the type of bug, not the number of bugs. If, to make a silly point, the software will not start at all for any customers, and that is the ONLY bug in the software, the software was clearly not tested at all. Even if that is the only bug. There are other similar bugs, but let me elaborate a little on the specific type of bug we see here.

If a piece of software is expected to regularly perform a particular task - say a task like "Render to H.264" and this task is one of the main functions of the software, then a vital thing to do is to create a test case for this. This can either be done using some sort of Unit testing framework (nUnit or similar) or it can be done using a GUI-testing tool like AutoIt. At the end of the day, doing testing in a GUI tool has some advantages since it simulates a user experience. To my knowledge Vegas is very script friendly, so using a C# based testing tool could be a good start. I do not know which features are exposed to the C# scripting side.

Now - to the regular task. Vegas renders video. It is one of it foremost tasks. It's like Microsoft Word printing a page of text. When you create a test plan for something like Vegas one of the items on that test plan must be "Test rendering with all codecs shipped with Vegas ... elaborate based on number of passes, bitrates etc". This should be one of the main testing points. This is a lot easier than testing Word printing, but at least as important.

When you put together the tests cases "described" in the one sentence above, each will be run with a set of legal parameters testing some border conditions and perhaps even a set of illegal parameters, tests that are designed to fail.

So, let's take a look-see on a particular bug

1 - is encoding to H.264 an important feature for Vegas? - Let's say Yes
2 - is encoding using the Main Concept encoder important? Given that it is the only H.264 encoder that gives you an amount of choice in how to encode, we can safely say Yes
3 - has it been tested with Vegas 9? Perhaps. It seems to work at times (or even often) with one-pass encodes. It seems to never work with two-pass encodes.

>> Project Managers do not conduct testing.

I have never seen anyone who claimed they did. Also, and that should have been reasonably obvious from my post, I did in fact mean to write Product Manager, not Project Manager. I doubt Vegas has a (or a single) Project Manager, but I do hope there is a Product Manager.

>> giving test pointers to a Project Manager is a waste of time.

A Product Manager is responsible for the end result. The buck stops with him. Giving pointers to him will help him either fix or replace his test team.

>> I love how you use the words "clearly" and
>> "obviously" with no actual inside knowledge
>> or proof whatsoever.

I like programming in Ruby. I like one of it's main ideas. Duck Typing. You know the concept. If it walks like a Duck and... well then it is a Duck.

I have watched many a software company mismanage software that eventually went down in flames. Heck, I was part owner of a company that managed to do it with our own software, and we our software had a starting price point of about $120 000. The signs are easy to spot if you've been doing software for a while.

There are some things you always test. For someone making video editing software, encoding video would be one of those things. When an important codec that is included with the product always crashes then it wasn't tested.

How do we know it was not tested - some times you ship with known bugs right? Well, it would be better to ship without the offending codec ("oops, we missed it - sorry, we'll put it back in 9.0a") than with the bug and the cost of doing so would be insignificant. It's an add-on so you take it out or you fix the problem. Easy.