Are you rendering to an external HD via firewire?

Tjglfr,

If you are asking me, no I am using internal ide drives.

Dave T2

Anybody using external?...How many internal HD's can you fit in a
desktop?

On previous desktops, you could normally fit 4 harddrives with the native connectors. With addon cards, such as a Promise controller, you can add 4 more.

With my new motherboard, I could have 4 IDE harddrives and 2 SATA drives. Of course, this means I have no space for a CD/DVD drive so right now I have 3 harddrives and the CD drive on IDE ports.

Dave T2

"How many internal HD's can you fit in a desktop?"

Well, I guess it depends on how big your computer case is, right? I mean... I figured out a way to conceivably fit 5 drives in a Dell Precision workstation, but I can fit about 7 or 8 in my old Antec server case.

I guess I see the point of the separate read/write drives, but I always assumed that the drive was just sending data to the proc, then waiting for the rendered file to return, not really hitting the drive all that hard. But I guess I can see it now that I think about it. I guess I'll give it a try.

Data wise, the harddrive is the slowest device on your computer. But some rendering might require the hd to wait I guess. But I think it is usually the other way around. The proc waiting for the hd.

Dave T2

This is a simple one with a simple answer:

ALWAYS render to a drive that is different from the one that contains your source files.

The difference in time is predictable, and is significant. It is due to the obvious fact that a disk drive cannot simultaneously read and write information. However, your computer is quite capable of having two separate drives doing two operations at the same time.

You can easily compute exactly what the time difference will be if your source (media files) and destination (files you are rendering to) are the same type and roughly the same size. Do the following: Find a file on any disk drive that will take at least fifteen seconds to copy. I'd suggest a 500 MB file. Then, copy that file to a different folder on the same drive. Make sure you copy it (hold the Ctrl key while you drag and drop, or use the Copy/Paste operation in Explorer). Use your watch to measure the time.

Then, reboot the computer (or find another file that is exactly the same size) and do the same thing, except this time copy the file to an external drive (you need to re-boot because otherwise some of the file will be read from RAM memory the second time around and the copy will therefore be faster because of that, and not because of what we are trying to measure).

I just did this with two different, but roughly the same size files, and this is what I got:

18.19 seconds
06.10 seconds

Thus, copying to another hard disk on my computer is roughly three times faster than copying to the same hard disk. This has very little to do with internal vs. external or IDE vs. SATA or Firewire vs. USB. It is simply the fact that the head on the drive can't be in two places at the same time.

The files I used above were 133 MB. If I had one hour of DV video, the file would be 13 GB, or almost exactly 100x larger. The time difference would therefore be 12 seconds times 100 or 1200 seconds. Converting to minutes (1200/60) gives me a difference of 20 minutes.

How much would you pay for an upgrade that would save 20 minutes on your next render??

Like I said, this is not subtle.

One note: If you are rendering to a format that is highly compressed compared to the original (e.g., rendering from DV AVI to MPEG-2), then the difference will not be as large, because the writes are only about 10-30% of the reads (depending on what compression level you choose). Also, if you have a project that requires complex rendering, where it takes hours for each minute of video, then perhaps you will consider the twenty minutes to be "round-off error."

For me, twenty minutes is a lot of time, and if all I have to do is render to a different disk, then by gosh, I'm going to do it all the time, no matter what.

I think that this is so important that I think Sony should have a suggestion that appears when you render reminding you to render to a separate drive, if possible.

Tower:-

C: System

D: Graphics . . Keepers & Short Term

E: CD/DVD

F: Short Term for - Render to . . Render As . . New Track . . SteadyHand . . MPEGS . .

G: Anything audio: Audio Hospital [SF] . . Loops . . ACID projects . . AKM

External f/w:-

H: I: J: K: L: . .. for Capture and Swapping

Total = 1TB ± 100GB

Done!

Grazie

It also makes sense to keep the source files on the fastest drive and render / encode to the slower drive as almost always the source file(s) are larger than the output file.
Bob.

> This is a simple one with a simple answer:

> ALWAYS render to a drive that is different from the one that contains your source files.

> The difference in time is predictable, and is significant. It is due to the obvious
> fact that a disk drive cannot simultaneously read and write information. However,
> your computer is quite capable of having two separate drives doing two
> operations at the same time.


John, I always use separate drives based on the same assumption. However, maybe it's not as straightforward as the file copy test would imply.. If it takes 18 seconds to do a file copy vs. several hours to do a render at "best" quality, wouldn't it seem logical that most of the time is consumed by the CPU, not the drives?

Nonetheless, I still use separate drives for source and render output. 18 seconds is probably an understatement, because the test was for a solid, full-file copy (not pieces and fragments as needed). As much time as the CPU spends on rendering, I would not want it to EVER be delayed because of a hard drive being shared for input and output.

Any advantage I can get my hands on is worth doing.

Okay, so in an internal 3 drive set up like this:

C: system
M: Media (short term storage) / Audio / Render
V: Capture

Would the page file be happier on C, M, or V? I'd like to dedicate a drive for the page file if at all possible.

If it takes 18 seconds to do a file copy vs. several hours to do a render at "best" quality, wouldn't it seem logical that most of the time is consumed by the CPU, not the drives?

Absolutely, 100% correct. If you go back and re-read my post, you'll see that I talked about this where I conceded that if your render time takes days, then the 20 minutes saved might not seem worth it. However, my next sentence was that twenty minutes is twenty minutes, and if it takes zero effort on my part to save it, I'll do it every time.

Your point is a little like what happens when you go to buy a new car and suddenly you find yourself saying, "aw heck, it's only another $500" when the rest of your life you'll spend hours trying to find a place that sells the $500 monitor for $20 less.

$500 is $500, whether it's part of a $30,000 purchase or part of a $1,000 purchase. I'm just as excited to save it (or get it) in either case.

Would the page file be happier on C, M, or V? I'd like to dedicate a drive for the page file if at all possible

I posted about this two months ago:

Page File Setup

Don't expect huge gains. I'm not sure it really made much difference. Didn't hurt anything.

Your point is a little like what happens when you go to buy a new car and suddenly you find yourself saying, "aw heck, it's only another $500" when the rest of your life you'll spend hours trying to find a place that sells the $500 monitor for $20 less.

Yeah, I won't argue with that! I didn't mean to suggest that your post was wrong. As I said, I use separate render/capture drives, too. Every little bit helps. It gets so frustrating when you have to start a render at night and not see results until the next day (usually after work). We must do everything we can to minimize render times.

Last night I did a project for my wife. She wrote a power point with photos and stuff in it, then she did an "oops" and realized they don't have a computer where she'll be doing the presentation, only a DVD player. I exported the slides to jpg and dropped them into a Vegas project, then I added some background music.

The darn thing rendered to MPG2 in a matter of seconds! If only life were this easy for the rest of our projects! I'm currently working on our wedding video (in HDV), with lots of pan/crop, multi-cam, stills tracks, effects, .mov clips with transparency, etc. Rendering 1.5 minutes to the Cineform AVI format took about 6 hours! Yikes!!! And this is on a fast new AMD Athlon XP2 4400+ machine!

All of which emphasises your point to make whatever difference you can that helps (like the separate capture vs. render drives, and other things).

I'm currently working on our wedding video (in HDV), with lots of pan/crop, multi-cam, stills tracks, effects, .mov clips with transparency, etc. Rendering 1.5 minutes to the Cineform AVI format took about 6 hours!

Yes, HDV is going to kill us all. There was a post about this a few days back where someone talked about the sense of relief when he went back and had to do a straight SD project. Everything was so FAST!

Unfortunately, the dirty little secret is that we are not going to see anywhere the rate of increase in computer speed in the coming years that we've come to expect in the past. There are all sorts of fundamental physics problems that are going to be tough to surpass. Lots has been written about this, but most of the articles have focused on the difficulty in shrinking the size of the transistors any further because each transistor is reaching the size of atoms. That is not the only issue, I don't think, because you could take the same part and, if you could make it clock at a higher speed, you could still get performance improvements. Clock speed is certainly related to size, but it is related to many other factors, including all the interconnections.

Bottom line is that most speed gains using traditional electronic-based technology are going to come from exploiting parallelism, such as multi-core, or multi-processor, and also from getting faster I/O. None of these are going to give us 10x improvements within the next few years, and therefore HDV is going to be a rather slow process for a long time to come.

6 hours to render HDV?

Is that using a DI--or straight HDV?

6 hours to render HDV?

The source files were: a track with Cineform AVIs, a track with DV from a second camera, two different tracks with stills downres'd to 1440x1080, a few tracks with some effects AVIs and MOVs (Digitial Hotcakes @ NTSC, some in QuickTime with transparency), 9 tracks in all. Project properties set at HDV 1080@60i.

I wanted to look at some of my work on my HDTV so I rendered to Cineform AVI and then transcoded the output to WMVHD (which my Avel LinkPlayer2 can play at 1080i for the TV).

Rendering about 1.5 minutes off the timeline to the Cineform AVI intermediate took over 6 hours. And that was using my new dual-core computer (AMD Athlon 64 X2 4400+ dual-core). All separate drives (system, capture, render). RAID 0 for the capture drive. Recently defragged. Anti-virus turned off. You know the drill.

In contrast, the transcode to WMVHD (1080i, 8Mbps) took about 34 minutes.

All so I could test 1.5 minutes of the timeline on my big screen TV. But yeah, gotta know what it's going to look like. (No the HDTV isn't calibrated, but at least I get the general idea).

WMVHD (which my Avel LinkPlayer2 can play at 1080i for the TV).

I asked the following question several times in other posts but never got an answer. However, it sounds like you (Jayster) would know the answer right off the top of your head. Here it is:

What settings do you use to create an HD WMV file that looks really good when played on your 1080i HD TV monitor?

I am assuming -- especially because of the HD format wars -- that lots of players are going to do what your Avel player already does, namely play WMV HD files. Therefore, I'd like to archive my HD projects in HD WMV, but since I don't yet have an HD monitor (even though I have an FX1), I can't play around and test various settings myself.

Any help or hints would be appreciated.

Sounds like a pretty impressive bit of computer crushing, eh? I hope it looked good.

Now, if one had used a Black Magic HD card on a similarly spec'd system, save for rendering/encoding the HDV to the black magic codec, I wonder how the system would respond? I mean, could you preview out of the BM's analog HD component into a HDTV set and gotten some real-time preview? I guess then we'd be talking drive throughput issue, not just processor power.

Fast Array + Black Magic HD card + fast procs = real-time (ish) performance of BM encoded footage?

SOME OBSERVATIONS ON RAID vs USB...
Rendering Does Not Push the Hard Drives as Much as a Straight File Copy. Even without any transitions, effects, etc., just a straight AVI to mpeg2 conversion, my USB drives keep up with my RAID drives...

Using an Athlon X2 4400, dual cores at 100%, rendering a 3:31 AVI file to MPEG2 takes [few effects] takes approx. 2:00 minutes regardless the drive [RAID, Internal IDE, Internal SATA, External USB - all are up to handling the mpeg output...]

MY SYSTEM:
RAID0 SATA to RAID0 ATA: mpeg2 creation = 1:59
My RAID to RAID normally transfers files at 120 mb/s

RAID0 SATA to USB 2.0: mpeg2 creation = 2:02
My RAID to USB 2.0 normally transfers files at 24 mb/s

At the time, External USB cases and ATA hard drives were about 1/2 the cost of external SATA, so I chose to render all of my work to external USB drives. This has been a good choice for me because in my case, external SATA would not improve my mpeg2 rendering. HOWEVER, it does take FOREVER when I need to copy a large amount of data from one USB drive to another.

ALSO, make sure that you do not have 2 devices using USB connections that are shared - this can slow USB flowthrough... Sometimes front/back connectors are shared, or USB ports adjacent to each other are shared depending on your mb and case.