Sony Networking Rendering Clarification

4eyes wrote on 2/12/2008, 3:23 AM

My plan initially is to network 10 dual dual-core Xeon machines together for the sole intent of rendering - possibly dual quad-core machines depending on the price.It depends on what your rendering, that much power you could work on multiple projects.
I've gotten the networking to work by following the help files in Vegas.
Being as your looking to network all these beasts together if they can process data very fast then just how fast can you feed them data. If you want to render at 20X real time then you will have to feed the network at 20X the data rate of the codec your rendering to. I think you need to do some network engineering and network analysis. Setting up windows shares using the UMB protocol to always work reliably takes some planning.
If you have a good file server is the best method (as stated in the help files). Put everything on the server, give the computers & server static IP's that never changes. Edit your hosts configuration files on each computer to resolve UMB (names) to ip addresses.

If you have vegas now you can simulate rendering mpeg2 video via distributed method. Non-distributed works (we know this). Just read the help files, edit the Netxxxxxx.conf file to change ports for each session of the network renderer on your own computer.

This is what I've learned so far, if you tell Pro 8 to render to a HDV 1080-60i (mpeg2) via distributed network it breaks the rendering into 5 sec clips/files (this setting probably can be changed via a configuration file, I'm only guessing though, the length of those clips may be dependent on the size of the file your rendering), after they are rendered (lots of files), one of the network renderer stitches them and back together (I have to double check that, but I pretty sure).
I had to disable the "No recompress long GOP video structures" under options or the stitching process failed.

(Even with the "No Recompress" disabled my videos still look damn good), they always did, even from a timeline render.
I'd like to see how something like your proposing would perform. I still think the final stitching together would be done by one computer. I don't think that would be a problem for a dual-Xeon.

jabloomf1230 wrote on 2/12/2008, 9:15 AM

@bastinado,

Maybe it is a new feature? ;-)

I also have NEVER seen this TCP error previously and despite it's cludgy setup and performance and other limitations, network rendering has always worked for me with previous versions of Vegas. However, I rarely used it, since it seemed to be equivalent of souping up a Yugo.

R0cky wrote on 2/12/2008, 11:35 AM

I have never seen it before either and have had little trouble getting network rendering to work in previous versions.

WRT the MPEG2 network rendering vs. encoding. On previous versions if you uncheck "render using final template" or something like that, it would render as DV/AVI and upload to the stitch host where it would stitch and then encode to MPEG2. This worked fine for me.

If you had a lot of fx etc. this gave me a significant improvement in total time from click to drop even though all of the mpeg encoding was done on the host.

Steve Mann wrote on 2/13/2008, 2:01 PM

Jab said: "In fiddling around with this, I just generated an error:

192.168.2.5 The requested address is not valid in its context 255.255.255.255:53704

I've never seen that one before. In TCP lingo that means either the IP address is invalid or the port is being used by another application. Is this a new feature with Vegas 8P?"

Are you using DHCP to assign the IP address for the networked computers? I haven't done any network rendering yet, but I suspect that your renderfarm computers each need a static IP address.

R0cky wrote on 2/13/2008, 2:46 PM

I do have static IPs and am getting this error with V8. Never saw it before.

Has ANYONE got network rendering to work with V8?

Kennymusicman wrote on 2/13/2008, 4:27 PM

Yes...

255.255.255.255 is an IP broadcast address, part of the subnet mask.

(My initial thoughts would be firewall, or your lmhosts file, or mapping in the network render service)

jabloomf1230 wrote on 2/13/2008, 7:44 PM

255.255.255.0 is the LAN subnet mask. And it doesn't matter whether I use a static or dynamic (DHCP) IP address. I think the "255.255.255.255" part is bogus and the error message is masking the real error.

This would all be very interesting if the same network render hadn't worked for Vegas 5, 6 & 7, but it did.

Oops, I forgot to add that the firewall (both router hardware and Windows software) is open to port 53704. I can ping the various machines fine.

R0cky wrote on 2/14/2008, 11:59 AM

no firewall, i've checked the mapping and share permissions a dozen times. I will look at the hosts file and see if there's anything funny there.

It worked for me using 5, 6, and 7 too.

jabloomf1230 wrote on 2/14/2008, 7:56 PM

Well first of all, network rendering DOES work for me under Vegas 8P. However, sometimes now, I get that error for no apparent reason. Here's what I would suggest that you try:

1) Enable NetBIOS over TCP/IP in your Windows network settings on every machine.

2) Make sure that your router isn't appending a DNS suffix to each machines' full name (like "Joes-PC.Linksys"). If the Vegas server shows names like that, but the actual full computer name in "My Computer" (or just "Computer" in Vista) doesn't show that name, you can sometimes fix that problem by changing the full name to include the suffix. However, you are better off without the extra DNS suffix that some routers tack on to everything. You can also see these DNS suffixes if you run IPConfig /all on each computer from the command prompt. The program nbtstat can also be run from the command prompt and it will show you all the NetBIOS names.

3) As all the tutorials for network rendering both previously mentioned here and elsewhere suggest, all Assets and temporary files should be located in one folder and that all computers can "see" that folder AND have full privileges to read, write, delete, etc. to that location. Using more than one folder is a recipe for problems with network rendering.

4) Delete the configuration file that the Vegas render server uses and let the server recreate it from scratch. The file is in a different location, depending on whether you are using Vista or XP, but you should be able to find it. It's called "VegSrv.settings". Warning- Do not mess with any of the files named .config in the main Vegas folder!

I don't know if the last item is needed, but it doesn't hurt anything.

Reboot all the computers and make sure that each copy of the render server can see the other copies.

See if any of that helps.

R0cky wrote on 2/16/2008, 8:07 AM

2. I have fixed IP addresses and am using the IP address directly rather than NetBIOS names.

3. I am using multiple folders, I will change that and see if it makes a difference. Some assets are on other networked drives. I've tried moving these locally but will check that again.

4. I've done this multiple times.

I've checked visibility and r/w capability on all of the shares multiple times too.

It's good to hear that it can be made to work....

thanks,
Rocky

jabloomf1230 wrote on 2/17/2008, 9:41 AM

It works, but erratically. It never did that before for me until Vegas 8P.

jabloomf1230 wrote on 2/25/2008, 9:30 AM

Rocky,

Did you ever get this to work at all? Now I am in the same "boat" as you. With no additional changes, Vegas now generates that error every time that I try to network render. I suppose that we should ask again if anyone has gotten network rendering to work consistently under Vegas 8P.

Did you contact Sony support about this?

R0cky wrote on 2/25/2008, 10:58 AM

I haven't worked on it for a while but I haven't made it work yet. Sometimes if I send an entire render to a network device it will work, typically an AC3 encode. But not always.

My next step is to try it with all of the assets in a single directory that is shared explicitly (rather than share the drive at the root level.

After that is to connect 2 machines with a separate hub with no other devices on the network, static IPs, so on and so forth taking my server and network out of the equation.

I haven't submitted a support ticket, I have 2 others in that are higher priority for me right now.

Rocky

4eyes wrote on 2/25/2008, 4:53 PM

2. I have fixed IP addresses and am using the IP address directly rather than NetBIOS names.You should have NMB resolutions which are the ip addresses & there corresponding names.
Make sure you are also in the same workgroup.
Goto C:\windows\system32\drivers\etc\hosts on each machine and edit the hosts file.
Editing your hosts file helps NMB resolution, otherwise depending on your network configuration you can get name mangling between the windows masters.
Look at the makeup of a hosts configuration file: It's somewhat self explanatory.
Example:
Name of my computer is POPS(192.168.1.30): The computer I want to connect to is named MOMS:(192.168.1.40)
Here is the line added to the hosts file on POPS to TELL POPS where MOMS is!
192.168.1.40 moms
Here is the hosts file edited on MOMS to TELL MOMS where POPS is!
192.168.1.30 pops

After you save the hosts file goto POPS computer and Ping moms
cmd terminal window: ping moms (from POPS computer)
cmd terminal window: ping pops (from MOMS computer)

The shared directory on POPS is VIDEO therefore it's NMB is:
\\pops\video
The shared directory on MOMS is VIDEO1 therefore it's NMB is:
\\moms\video1

When you connect to the other computer you must have rights to the other computer.
On both computers from explorer "Tools -> Map", when you do this click on the "Connect using a different name" and connect to that computer using that computers username (not the one on your computer). Others way to do this also.

Now when you start networking rendering on each computer you must tell the renderer where the other computer is at.
Just follow the directions in the help file on network rendering.
Got to run, MOMS pinging me!

Musician wrote on 2/26/2008, 10:23 AM

OK guys, I got clarification from Sony about network rendering to mpeg-2. Here is the question that I sent with help from johnmeyer:

"The proper question to have asked would have been, "Can Vegas 8.0b do distributed network rendering to the MPEG-2 format and have the host automatically stitch the results?" That is my reworded question: In one network render pass, can I turn out an mpeg-2 where the stitch host will stitch an mpeg-2 together"

and the reply was:

"I apologize for the delay on this. When you are performing a distributed network render to MPEG-2, the format (including maxGOP size, bitRate size, fps, etc.) of the final file needs to exactly match the intermediate files. When using this feature it is important that the exact same render template is used throughout the render. As long as these settings are set exactly the same the render will work properly."

It looks like it may be a sensitive process where everything has to be set up perfectly, but it also sounds possible. In about a week, I am going to network a couple of my computers together and test this out. I will let you know how it goes.

jabloomf1230 wrote on 2/26/2008, 8:36 PM

I guess it doesn't hurt to use LMHOST also, but you would think that DHCP, NetBIOS and/or IP address would be enough. It was never this complicated in Vegas before version 8P. In any case, if this makes Network Rendering work 100% of of time, then I'm willing to try it. I'll let you know if this eliminates the TCP error message.

MSK wrote on 2/27/2008, 7:12 PM

From my understanding Vegas currently supports up to 4 rendering threads. I don't think 10 machines would work, especially if they're 10 x2 or x4 processors.

jabloomf1230 wrote on 2/28/2008, 6:35 AM

But wouldn't each Vegas Network Server on each networked machine support up to 4 rendering threads? Also, according to the somewhat obtuse Vegas network rendering documentation, you can also set up two instances of the Vegas Network Server on one computer, just by using two different port numbers. What purpose does this serve?

Kennymusicman wrote on 2/28/2008, 6:48 AM

Yes, each machine will render with its own threading, so there is no problem or limit as such - more machines in the farm, mean more processing abilities.
You can indeed set up more than one instance of network rendering. THe purpose is 2 fold. Firslty, you can use it as a form of background rendering, and set priorities on CPU such as a low priority net-render, and a medium priority net-render, allowing you freedom of working. Secondly, some codecs are not multi-core aware, as such, so you could net render to multiple instances which each having a core to itself, thereby allowing you to speed up rendering with such codecs.

Network rendering is one of the coolest parts about Vegas :)

jabloomf1230 wrote on 2/28/2008, 8:21 AM

Have you had any problems with network rendering in Vegas 8P? I and at least one other person in this thread have been getting a strange TCP error message (see several posts above in this thread) on network render setups that worked fine under Vegas 7.

jabloomf1230 wrote on 2/28/2008, 1:26 PM

Adding the IP addresses to the hosts file does fix this problem. No more error, at least for me.

EDIT: Well, the error has come back, this time with a slightly different network render. I'm convinced that it's a bug, in Vegas 8P since I'm not the only one that has reported this error.