In computer graphics, there is always the need for more crunch power. Whether it be for pushing higher quality visualizations in economic time frames, pushing through more frames of animation at a time, or just getting your current work done faster, computational horse power has become a pivotal part of the animation pipeline.
The term "renderfarm" has been the standard name for a cluster of computers whose purpose is specifically for rendering animation. In almost any studio, large or small, you will find at least a few systems used specifically for offloading rendering tasks. Building a renderfarm is generally not a small task. There are a few things to keep in mind. Hardware, operating systems, rendering software, network setup and storage.
Hardware of course is quite important. What type of system do you need? PC or Mac, AMD or Intel, how much memory, etc. Generally you need to start at the software level though. Your software will generally dictate the type of hardware you use. Main reason for this, some software is unsupported on Linux or Mac, or some software is supported on a Mac, but isn't compiled as a Universal binary yet.
Personally, I chose to go with a PC based system utilizing the latest Intel processors, under the Linux operating system, with a Windows dual boot. The Windows dual boot was a temporary thing for my first render node, given the issue of one of my renderers not being supported under Linux yet. The choice of going with Intel over AMD was rather simple. Lower wattage and higher performance today. Once a renderfarm starts scaling much larger, going lower wattage means less power draw, and less heat dissipation.
Power draw and heat dissipation become quite important when you are looking at large scale. When you are dissipating a lot of heat, you need more air conditioning on less systems. This of course leads to even more power draw as air conditioning as well is a power hungry process. If you can cool more systems with 10,000 BTUs of cooling, you significantly reduce the amount of power required to keep your renderfarm up and running. Of course, lower wattage systems will also consume less power, saving money in that respect as well.
If you are planning on scalability, you will want to try and pack as much compute power into as dense of space as possible. This generally involves utilizing a rackmount form factor for your system purchases. Generally, a 1U form factor is most popular. You can fit all of the CPU crunch power you need into a 1U case, and rack them up for dense computing environments. As an example, if you have a rack cabinet rated at 45U, you can fit 45 of your 1U computers into a 20" x 30" (approximately) footprint.
Boxx Technologies 1U Intel Xeon RenderBoxx.
When you are selecting the hardware you wish to use for your renderfarm, it is highly suggested that you stick with that type of system throughout your whole renderfarm. For example, if you end up going with AMD Opteron processors, stick with that chip. Going with another chip may cause some inconsistencies in your renders. A few tests I did with the ZooRender Maya benchmark, showed minor inconsistencies not only between different architectures, like an IBM PowerPC processor and an AMD Opteron processor, but there were also slight differences between the 32bit and 64bit versions of the Mental Ray renderer, even on that AMD processor, without changing architectures. There can be little intricate bits of the rendering algorithm that may be sensitive to rounding issues between processor architectures, and these issues may lead to very small variations in colour on the frame.
So sticking with the same processor architecture is quite important throughout your renderfarm. If you are using Intel Xeons based on Intels new Core architecture, stick with those. If you are using their Netburst Xeons, stick with those. Same thing if you are using AMD. Stick to Opterons if you are using those.
The Power Requirements
As already touched on briefly, power is a factor. As your renderfarm starts getting to a larger scale, you need to start being concerned about the load on your electrical system. In many cases in a home, or in an office (in North America), you will be running almost everything off of 15 amp breakers. There are of course a few exceptions, specifically for appliances like a dryer, but for the most part, if you look at a breaker box, you will mostly see 15 amp breakers.
This brings up our first issue. How many computers does it take to trip a 15 amp breaker? Well, general rule of thumb is 100W of power draw is 1 amp. So you can draw up to about 1,500W from a single 15 amp breaker. In todays computers, it is not uncommon for there to be a 500W or 600W power supply, but generally, a lot of the extra power requirements come in with the addition of todays high performance graphics cards. Realistically though, you can't measure your power draw from the rating of your power supply. Your primary factors are generally a combination of your system configuration, and the power supply efficiency.
Today, there are 2 primary competitors in the x86 workstation market. If you are looking into dual CPU systems, your primary targets are Intel Xeon or AMD Opteron. Currently, the Opterons run around 80W to 95W per chip, while the new Xeons run around 65W. This can be a very important factor to consider, as it attributes to both power draw, and heat dissipation. In smaller scales, these numbers may not be very important, however, as your cluster gets larger, the numbers do add up, and you will need to ensure the the proper power infrastructure is in place to handle this kind of setup.
There are a few solutions to consider when putting together a large renderfarm. First is your battery backup solution. When you are running massive jobs, power failure can be catastrophic. Make sure that the battery backup you are using will support the number of nodes you will be connecting to it. Make sure the networking infrastructure is also on battery backup. Proper power distribution will become very important as your renderfarm begins to expand. The power draw will become excessive for basic UPS to handle everything. This may be the point where you may need to talk to the electrical company about your power needs.
Bottom line, a standard plugin that most of you may have your computer plugged into will not be enough as you expand. Breaker capacity and cable types become serious issues. Drawing 30 to 50 amps through cable meant for 15 amps is just a bad idea all around and can become very dangerous. Plan your power needs very carefully when you plan to build your renderfarm.
The 1U Form Factor
Now I've talked a little bit about the 1U form factor, but I want to get a little more in depth with it. This is generally important since most people who are used to assembling their own systems, are used to using a standard heatsink and fan for their processors, which of course don't fit inside the 1U form factor.
Going a little more in depth though, 1U is a term used for rack mount systems. It is measured as 1.75" in height, so it can be racked fairly densely in a cabinet, or just an open rack. In the case of a renderfarm, a 1U form factor is a popular choice. There are other choices, including 2U, 3U, 4U, etc. Basically, what this means, in a rack cabinet, it will take that amount of space, so a 42U rack cabinet with a 2U and a 1U system (or peripheral) in it, will only have 39U of space left.
There are a lot of vendors out there that do sell 1U "blades" for processing, such as Apple, HP, IBM, Boxx Technologies, etc. I do personally recommend going with a vendor. As you can see above, I ended up going with Boxx Technologies, but for those interested in building their own, there are a few things to keep in mind.
First off, a 1U system no longer relies on a fan attached to the heatsink, simply because it will not fit. The case will generally have fans in it that will pass air from the front of the case to the back of the case. No air will be brought in or exhausted from the sides, top or bottom, simply because slide rails may be attached to the sides, and other systems may be pressed against the top and the bottom.
Second, make sure the motherboard you are using is "rackmount ready". Generally, in a single processor setup, this means the processor socket is located close to the front of the case. In dual socket setups, it means the same thing, plus, both sockets are located right next to each other. This is important for cooling since the closer the processors are to the fans, the better.
Finally, you will either need to buy processors with a 1U heatsink, or buy them separately, specific to your processor socket. As I mentioned earlier, a standard heatsink and fan that comes with processors will not fit in a 1U case. There are heatsinks with fans on them as well designed for 1U, but I'm not certain as to how well that will work out.
Just remember, if you choose to go with 1U, you are no longer building a system for computer graphics. Use of any of the PCI slots requires a 1U riser card, and you will only be limited to 1 device. On top of that, noise will become an issue. The small chassis can only fit small 40mm fans. In order to pass the air it needs to pass through to keep your system cool, they have to run very fast, and they are loud. I suggest taking a look around at computer parts vendors to see what they have available for 1U cases. Find what works best for what you wish to build.
.
(Article Info) 
