Near the end of 2015 we were placed in the unpleasant situation of needing to compile two versions of Unreal Engine without the benefit of the ubiquitously speedy Incredibuild.
To make things worse we on platforms using the lightly poky Microsoft C++ compiler, meaning that changing a core header in Unreal was resulting in compile times of over half an hour.
It did not take many of these changes before we decided something had to be done. Being in Montreal we knew that some studios in the city were using the aptly named distributed build and caching solution FASTBuild. As the universe tends to do, as we were cleaning up our solution it appears someone else was also on the trail of making FASTBuild work with unreal! If we had a time machine… FASTBuild with Unreal would have saved us a bit of trouble.
The end goal for what we did as well as what Carl did was the same: produce a .bff file from the UnrealBuildTool actions.
The Unreal FASTBuild duo has some quirks worth mentioning.
Unreal enjoys its precompiled headers (PCHs), this works as expected in FASTBuild except that Unreal also sometimes links with the PCH object file, and the object file FASTBuild generates does not have the appropriate name. So we have a small modification to FASTBuild which allows us to override the PCH object file name.
Another quirk was that some of the obj files (occasionally in the hundreds of MB) generated are so big that the clients were timing out. Yassine did some profiling and somewhat surprisingly it looks like it’s happening at the Windows socket level with the send call telling us the buffer had no room for very small sends of 8 bytes, just after attempting some sends in the tens of megabytes. Bumping up the timeout from 5 seconds to over a minute got rid of all of these issues, but it was a bit of an annoyance.