[ILUG] Build Systems

[Darragh Bailey]

Trying to put in place an improved build system here at work. Will most likely
use BuildBot (http://buildbot.sourceforge.net/) for the scheduling and control.
Currently mainly concerned with the underlining steps that decide whats to be
done and how.

Plan to implement all of the decision making part of what to build/rebuild using
make since it has a number of useful features already built in such as
dependency chaining, incremental builds (rebuilding updated stuff only) and
parallel building. Also everyone here has familiarily of using make so if they
want to do a private build it's not too much of a strech to have them use make
to control it.


Before I outline the problem, would like to point out that in order to allow
proper dependency resolution in a sane manner I will be using make in a
non-recusive format. i.e. multiple make files included together to perform the
build rather than calling sub makes. One of the basic reasons for doing this way
is because some components will contain patches that will affect the kernel, so
changes to those patches should cause a kernel rebuild. With the sub make method
AFAICS the makefile that rebuilds the kernel would require explicit knownledge
of those components, whereas with a non recusive make you can create separate
makefiles for the components which add the relevant parts of themselves as
dependencies to the kernel make rule.

i.e.
makefile1
kernel: kernel_srcs
        build_cmds

makefile2
kernel: component_kernel_patches

When you include both makefiles, the rule automatically becomes
kernel: kernel_srcs component_kernel_patches


Anyway, one of the requirements is to be able to change the version number for
subsequent rebuilds of a component. i.e. update a build number so that
individual rebuilds of a component target (rpm) will differ. Only problem is
that it you use make to have the target, an rpm so that it can determine if the
target is newer or older that its source components, the version is in the rpm
file name. This causes a problem since all the rules are resolved when the
makefiles are read before they start excution, so if it determines that it needs
to rebuild a target file you can't suddenly modifiy the build number otherwise
the built rpm won't match up with the make rule target and make will exit.
(Could be wrong here but I don't think you can modify rule targets on the fly in
make).

So far the only solution I can see is to run make with a prep step before the
main build. For each of the rpm targets in the components use a separate file
containing a build number, and set this file to be the target of all the srcs
that make up the rpm. Set up rules so that calling the prep step will run these
rules that will update these files if necessary and increment the number
contained. Then perform a subsequent run of make to build anything needing
building. Each component will extract the number from the files when
constructing the target names to be rebuilt. If the target is missing, then the
component will be rebuilt using the updated build number.


tbh, that solution sounds less than ideal to me so I may have to rethink the use
of make depending on how much an overhead that will cause to the calls to make
in reading all the necessary makefiles.

Estimate approximately 100 makefiles will need to be read, by performing
inclusion before running any step. Expect 60 of those makefiles to need to have
the modifiable build number that will change if a rebuild of the associated
component is necessary.


--
Darragh

"Nothing's foolproof to a sufficently talented fool"