Render Farm file size

[Peter Milner]

When I send a job to the render farm, the amount of data transferred is huge.

A drawing of about 6Mb in size translates into about 16Gb of data for each of the four render nodes.

This means it can sometimes take 15 to 20 minutes before the rendering actually starts.

Is this how it should be, or is something not working properly?

[Roy Hirshkowitz]

It can be a lot of data.  For example, solids get converted to meshes which can be much less compact.  Materials and textures are internalized.

Really not much to do here-- make sure your FACETRES setting isn't too high-- make sure you're not using unnecessarily high resolution textures.

This is a common problem.  Professional render farms often have very fast networks for file transport.  It may be possible to locate your shared folder on the array somewhere-- this might (or might not) help the transport issue-- depends on whether the farm machines are linked by something faster.

[Peter Milner]

But why is it so much more than rendering locally?

If I render a 24.7Mb file on my own machine, the RAM usage only goes up by 2.3Gb.

[Roy Hirshkowitz]

It should be the same-- I'm pretty sure it's basically the same code.  You can't correlate drawing size and memory usage directly-- a 6mb drawing could certainly generate a lot more memory than a 25mb drawing-- it all depends on what's in it.  It could, of course, be a bug-- if we've got an example of a drawing that's behaving wildly different I'm interested.

[Peter Milner]

Don't worry Roy, it's me being a bit dense.

The renders I'm sending to the farm are much higher resolution than the ones I run locally, which explains the difference in RAM usage.

[Peter Milner]

Okay, I've found some models which when rendered locally take up about 0.7Gb, but when rendered by the farm take up about 8.7Gb.

The model itself is only about 8Mb in size and I'm rendering at about 1000 x 700 pixels.

[Roy Hirshkowitz]

If you'd like me to take a look you can upload the file using the standard link above.

If you want to investigate further yourself, the shared data file that's written has an extension of .nXtModel.  It's this file size you want to check first-- this is the data that goes flying around to each node.  It should not be out of line with the internal memory usage of the model when rendering locally.

[Peter Milner]

I've just uploaded one of the models.

The nXtModel file is 399Mb. The RAM usage on the farm goes up about 9Gb.

[Peter Milner]

On the other hand, it may be some sort of memory leak.

When a rendering is finished, the RAM usage goes back down to 2Gb. But each time I send a new render it goes up higher than before.

After several renders today, it is now clocking up to over 19Gb.

[Roy Hirshkowitz]

It does sounds like a memory leak of some sort.  I'll take a look this weekend or sooner if I get a little time.

[Roy Hirshkowitz]

Couldn't find anything here-- not sure what's up.

The file you sent was almost 8MB.  It wrote an nXtModel file of 27MB and causes about 200MB in allocation to the nXt64.exe process.  All of this seems normal given the image size, etc..  I was able to fire off several renderings and did not see any changes in the allocation.

The way the farm works is that nXtFarmer64.exe will launch a process called nXt64.exe.  It's the latter that does all the real work, the first just sits and looks for messages.  After nXt64.exe finishes a task it actually kills itself and goes away-- something which will always deallocate any memory it consumed.  nXtFarmer64.exe will notice that it's not there and launch a fresh copy.  All of this mechanism was designed to avoid definitively any persistent memory leaks.

You can see all of this if you launch task manager on one of the nodes and look for these two processes (on the Process tab).  If you see multiple copies of nXt64.exe hanging around then we definitely have a problem.  

If you're getting different behavior than what I described in the first part, we can certainly try updating the farm software in case the last published build was bad.

 

[Peter Milner]

Thanks for checking it out Roy.

It looks like the problem is model specific. I can't get that one to run at all now, but a similar style fresh model has just run perfectly with no problems.

I'll keep investigating.