How it works
FASTBuild was designed from the ground up to support efficient parallel compilation and build distribution. Its self-contained architecture and the absence of external dependencies insure an easy deployment across build machines; so basically you don’t need to install specific development tools on the build machines, FASTBuild takes care of sending the compiler toolchain to the build workers and run the remote compilation work in an isolated environment.
Here is an example of build times using FASTBuild distribution from the FASTBuild website:
Tips:
- Only source code compilation can be distributed across machines at the moment.
- There are some restrictions as far as supported compiler settings and build distribution, you can refer to the FASTBuild offical documentation for more details.
Distribution setup
The setup is straightforward and consists of 4 steps:
1) Creating a shared brokerage folder: this is where build workers register themselves
2) Setting the FASTBUILD_BROKERAGE_PATH environment variable on all build machines, the path must point to the location created in the first step.
3) Starting the FBuildWorker.exe on all build machines.
4) Launching a distributed build.
Assuming we’re running on Windows, here are a few more details about each step:
1. Create the shared brokerage folder:
Workers are discovered through a brokerage location. The brokerage location is a network path accessible from both the host and workers.
a) Choose a computer that is available through the local network, i.e. it is accessible by the other computers participating in the build. We had noticeably fewer problems when all machines were connected through a wired LAN, as wireless performance (and thus build times) tended to degrade rapidly for large builds.
b) Create a new folder named something like FASTBuildShared. This folder will contain all FASTBuild shared files, see the caching instructions for another usage of this folder. Enable Windows sharing on this folder.
c) Give Read/Write permissions to all users and machines that need access the shared location.
Tips/Troubleshooting:
- As mentioned above, distributing builds through WIFI can result in network saturation, slow transfers and timeouts which can negatively impact build times
- If some computers aren’t contributing as expected, be sure to test the access to the shared brokerage folder from other computers: for example try to create and delete a folder to verify that you have full access rights.
2. Setting the FASTBUILD_BROKERAGE_PATH environment variable:
Build hosts and build workers need to know about each other in order to communicate commands and build tasks.
Every active build worker (machines running FBuildWorker.exe), writes a file to the brokerage folder created in Step 1. Upon starting a build, the build host (running Fbuild.exe) scans the files inside the brokerage folder and registers the associated machines as build workers it can use to distribute jobs.
FBuild.exe and FBuildWorker.exe processes know about the brokerage location via the FASTBUILD_BROKERAGE_PATH environment variable which must be set on all machines participating in the build distribution, The value must be the path created in step 1.
In our example it looks something like:
3. Starting FBuildWorker.exe on all build machines:
Start FBuildWorker.exe on all machines participating in the build distribution.
You should see worker UI that offers a couple of customization settings in order to manage its work load.
To verify that the build workers have successfully written their file inside the brokerage folder you should be able to see their corresponding machine name as files inside that location. If we take our previous example, the files can be found inside \\FastBuildShared\main\14
4. Launching a distributed build:
The last step before launching a distributed build is to make sure it is enabled inside the Unreal FastBuild.cs. Verify that the bEnableDistribution attribute is set to true.
Now start a new build and notice some important information logged in the build output window that shows that the build distribution is active:
The build report should also tell us some important stats on how the build was distributed and how much time we saved in the process:
