Network Transcoding Farm - Is there such a thing?

PixelStuff wrote on 3/29/2016, 9:04 PM

Does anyone know of software to manage transcoding video across multiple machines on a network? Maybe the bandwidth is too intensive for it to be worth it, I don't know.

For example, if I had a weekly show with a lot of footage, shot with multiple GH4 cameras in 4K and down rezed to 1080p 4:4:4 10bit .... what is the fastest way to get that ready for editing if we have access to a couple dozen networked computers in the evenings when most people are sleeping?

Comments

videoITguy wrote on 3/29/2016, 9:50 PM

There are many many ways to do video tasks with networks. What you do and how you do it will be very specific to the situation.

Assume that you are thinking about 1 gig networked workstation class of computers.

Two principles apply:
1) The bandwidth of connections is only sufficient to support file transfers!
and
2) You will need to divide and conquer tasks on separate machines - for example one machine that does title and graphics composition, other machines to do audio beds, and others to do sub-project rendering...

you get the idea

musicvid10 wrote on 3/29/2016, 9:52 PM

Older versions of Vegas had network distributed rendering.

Client renderers were hard to mount and keep online, and you needed identical cores on each machine to realize any advantage, about 20% with diminishing returns the more machines you added, because of system overhead. Still, it beat our 1.4Ghz single core capabilities; at least by enough to keep us trying.

A modern i-Class processor will run circles around distributed network rendering, which went out of fashion with Vegas users about the time we got good fast quads.

Also, due to the complexities of long-GOP encoding, those who have tried independently to engineer this with AVCHD and XAVC have found the task all but insurmountable for common mortals.

Get a reliable six-core and split up your projects to another machine or two; you will have far more bang for your buck without all the messiness.

Hope this helps.

PixelStuff wrote on 3/29/2016, 11:42 PM

Just to be clear, I'm not wanting to know about rendering Vegas project files, but instead transcoding the original camera files into an intermediate file that Vegas can easily work with. Doesn't mater what software is used, but I'm hoping there is something where we can setup a transcoding profile, drop 10-15 hours of video files onto the application, and it will automatically send them out to various computers in the "farm", do the transcoding, resizing, or whatever, then bring the new file back when finished.

Or something along those lines.

Someone needs to invent a network rendering system for Handbrake if such a thing doesn't exist.

musicvid10 wrote on 3/29/2016, 11:49 PM

Parallel rendering, which I "think" you are describing, would seem to be easily accomplished with a shell script.
That said, running multiple simultaneous 4K renders on a LAN . . . I don't know. Got fiber?

As far as distributed rendering in Handbrake, why don't you talk to THIS GUY.

john_dennis wrote on 3/30/2016, 9:54 AM

Do you have a hardware/software budget number for this weekly show?

videoITguy wrote on 3/30/2016, 12:59 PM

Transcoding -particularly if you have Sony cameras orginating will best be deployed ot workstation class machines with Catalyst prepare. With anything of network - 1 gig or less - you need distributed computing solution as I referred to the OP in my earlier post above.