Multiple Single Job Builds

Implementation

This can be best explained through a simple example. If a build performs compile, unit tests, integration tests and generates installer. Simplifying it further if we can assume that each of them takes 5 minutes each, adding up to build time of 20 minutes. Such a single job build can be broken up into smaller and multiple single job builds. From our example we can breakup the build into three parts doing compile + unit tests, compile + integration tests and compile + generate installer respectively. This in effect reduces the build time to 10 minutes. This approach can be applied when using first generation tools. Second generation tools have lot more options as we would see later.

Limitation

Material Synchronization Issue

Synchronizing different single job builds with same materials is often overlooked when using this pattern. Since the single job builds are acting independent of each other, they can potentially have different materials (can be seen in the diagram as well). The impact of this can be hard to diagnose. From our example only if compile passes in some build and fails in other then the problem becomes visible. In large teams (doing a lot of commits) this can be regularly seen. This problem also plagues the downstream builds and processes which might be using the artifacts generated by these builds. Since other jobs might have failed, these artifacts might not be the right candidate for the downstream activities. For example, if the generate installer is generating the artifact, unit test might have failed for the same version of materials.

For example if we have following situation