Besides the switch to SQLite there are other big changes regarding the build of stage02, which is the population of the chroot environment:

• LessLinux now supports tracking of dependencies after building packages. This allows to define the order of packages to build.

• As a consequence of the dependency tracking it is now also possible to parallelize the build. On a three core Athlon this brought down build times from around three days to less than 20 hours. With a faster machine you should even get better results.

Prerequesites

I assume that you already built stage01 on the same machine. If not, just create the directories as we did in the last article and mount the hard disk or partition containing sources and the stage01 chroot. The unprivileged user is not required this time.

Building

I prefer redirecting the console messages to a log file (and “tailing” it in another terminal). The build and install of the respective pacakges are separately logged to an package specific log file. I will later show how to analyze and them. So change to the build directory and run:

        ruby -I. builder.rb -s 1,3 -n -u -l -t 7 > /tmp/stage02.log 2>&1

This command uses six threads maximum. It really depends on your machine how many is ideal. As a rule of thumb, take twice the number of processor cores plus one. Less than three do not make sense, and above ten too many seeks may appear. Anyway: If you have a pretty fast box with a six core processor and you are maybe building on a SSD: try 15 or 20 threads and tell me how it compares to just ten.

Trouble

You will occasionally run into a problem where no modified files were found. In this case the builder exits. To find out which package is affected just grep for “no modified” in the log file:

        grep -A2 -i 'no modified' /tmp/stage02.log

This is most likely due to some race condition: The build or install script is not yet closed, but the builder already tries to run it. Take a look at the files /mnt/archiv/LessLinux/llbuild/stage02/build/packagename-version.build.log and packagename-version.install.log. If it reads bad interpreter … text file busy this is the problem. Sometimes the maximum number of open files is the problem, so you should compare the current number of open files with the maximum number during the threaded build.

To re-build the affected package delete the package and the file /tmp/LessLinux_Emergency_Exit and run the builder again:

        rm -rf /mnt/archiv/LessLinux/llbuild/stage02/build/packagename-version* /tmp/LessLinux_Emergency_Exit
        ruby -I. builder.rb -s 1,3 -n -u -l -t 7 >> /tmp/stage02.log 2>&1

When restarting the build all packages already built are ignored and the builder picks up where it left.

The next post will show how to assemble a working live system from the freshly created stage02.

Dieser Eintrag wurde am Thursday, February 3rd, 2011 um 3:11 pm geschrieben und ist unter Howto, Packages and Architecture abgelegt. Antworten können Sie mit dem RSS 2.0-Feed verfolgen. Für diesen Artikel sind weder Trackbacks noch Kommentare erlaubt.