Optimized Parallel Jobs Build

Optimized Parallel Jobs Build

Context

Modular System. High build time. Hardware constraint to use Parallel Jobs Build.

Description

In principle it is same as Parallel Jobs Build but avoids the duplicate work. As a result it needs less resources.

Extracts out common elements of different jobs in Parallel Jobs Build into a setup pipeline with single job. The other parts of the job are run in a downstream pipeline which works of the output of setup pipeline. This applies to applications using compiled programming languages when the tests cannot be run before compilation of code.

Limitations

• The continuous integration build is not the same (a setup where binaries are published and consumed) as local build, hence not verifiable by commiter. This can overcome by designing for this problem.
• It creates temporary output which might not be useful for any downstream activities after the build is over but might still occupy the disk space on CI server.
• The build time may be higher than Parallel Jobs Build depending on time taken by the continuous integration server to schedule individual jobs and transfer output between different jobs.