PKG_COMP(8)

NAME
pkg_comp - build packages inside a sandbox

SYNOPSIS
pkg_comp [- ( c|C) conf_file] [-Nn] target [pkg_name ...]

DESCRIPTION
pkg_comp, or Package Compiler in its full name, is a tool that makes easy the compilation of packages inside a clean sandbox. This allows an easy tracking of exact dependencies and the correct behavior of a package in a fresh system installation.

The behavior of pkg_comp is controlled through a small configuration file and a target (keep reading to learn more). The configuration file tells pkg_comp how to configure the new chroot environment, and the target specifies which action to take.

The following options are recognized:

-C conf_file
    Use conf_file as configuration file (full path expected).

-c conf_file
    Use conf_file as configuration file (only base name expected).

-N  With the exception of pkgtools/libkver (see NETBSD_RELEASE) avoid installation of default packages as well as INSTALL_PACKAGES and BUILD_PACKAGES during the creation of the chroot.

-n  Avoid installation of INSTALL_PACKAGES and BUILD_PACKAGES during the creation of the chroot.

  What to use it for?
You can use pkg_comp to achieve many goals when building packages. Here are some ideas:

•   Build packages for other system versions. For example, build packages for NetBSD 4.0 while you are running NetBSD 5.0.

•   Build packages using different mk.conf options than your current system, like changing the threading library, COPTS, placement of configuration files, etc.

•   Debug the build process of a package, checking if buildlinks work properly.

•   Avoid autoconf's side effects by keeping a separate chroot for each project, like one for GNOME2 and another one for KDE3.

•   Schedule builds of package sets for several different machines.

CONTROL DIRECTORY
pkg_comp needs to store several pieces of information when it is running. Instead of using normal system trees, it uses a special directory inside the chroot to avoid polluting the system. It stores there scripts, object files, built packages, etc. This directory is $DESTDIR/pkg_comp; the symbolic link $DESTDIR/p is automatically created to ease pathnames when working inside the chroot.

CONFIGURATION
With pkg_comp you can maintain several configuration files so you can work with different chroot jails easily. To make this easy, configuration files are stored inside $HOME/pkg_comp, followed by the configuration file name and the .conf suffix. The default configuration file is $HOME/pkg_comp/default.conf, and is always used if you do not specify another one. The configuration file name is specified by the argument of the -c option. Alternatively you can specify any pathname as a configuration file with the argument of the -C option.

Configuration files are simple shell scripts that define variables. The default values shown here are those written in the template when issuing a maketemplate.

AUTO_PACKAGES
    A list of packages to automatically build during the auto target. A package is in the form ‘section/name’, like ‘misc/colorls’, or a plain name like ‘colorls’. Defaults to nothing.

AUTO_TARGET
    The pkgsrc target to use when building packages in an automated fashion (using the ‘auto’ target). Should be set to ‘package’ or ‘bin-install’, as other values are useless. Defaults to ‘package’.

BUILD_PACKAGES
    A list of packages to automatically build after the makeroot target. A package is in the form ‘section/name’, like ‘misc/colorls’, or a plain name like ‘colorls’. Defaults to nothing.

BUILD_PKG_COMP_TARGET
    The pkgsrc target to use when building packages. It can contain any target supported by the pkgsrc system, but reasonable values are: ‘install’, ‘package’ and ‘bin-install’. Defaults to ‘package’.

COPYROOTCFG
    If set to ‘yes’, all configuration files (not directories) that reside inside /root are copied to $DESTDIR/root. Defaults to ‘no’.

DESTDIR
    The chroot jail directory. Defaults to /var/chroot/pkg_comp/default.

DISTRIBDIR
    This is the directory which holds binary sets and X sets. Its structure is the same as official release distributions, that is, tarball files must reside inside $DISTRIBDIR/binary/sets. Defaults to /var/pub/NetBSD.

DISTRIB_EXT
    This is the file extension for the tarballs in $DISTRIBDIR. Recent NetBSD releases use tar.xz instead of tgz. Defaults to tgz

EXTRAMK
    Specifies a whitespace-separated list of files that must be appended to $DESTDIR/etc/mk.conf. This is useful to add special items to this configuration file. Defaults to nothing.

INSTALL_PACKAGES
    A list of packages to automatically install after the makeroot and before installing BUILD_PACKAGES. These are also installed within the sandbox created by the auto target, but before anything is built. Each name must be the full package name, including the tgz suffix. Packages are searched inside $REAL_PACKAGES/All. Defaults to nothing.

GENERATE_PKG_SUMMARY
    If set to ‘yes’, generate a new $REAL_PACKAGES/pkg_summary.gz file at the end of every package build by both the ‘auto’ and ‘build’ targets.

LOCALBASE
    Where binary packages get installed. Defaults to /usr/pkg.

MKCONF_VARS
    A list of variable names that will be appended to the generated /etc/mk.conf file, together with their values set in the configuration file. Its default value contains all variables listed here.

NETBSD_RELEASE
    Specifies which version number of NetBSD is installed inside the chroot. If set to ‘no’, no special action is taken (this is useful if the system version inside the chroot matches the outside one). Otherwise, the package pkgtools/libkver will be installed inside the chroot, in a special purpose prefix whose value can be set in $DESTDIR/etc/mk.conf via the configuration file with the LIBKVER_STANDALONE_PREFIX variable. The libkver library will be configured inside the chroot, with the symbolic link $DESTDIR/libkver_osrelease and LD_PRELOAD in default shells environments, so that the NetBSD version specified in NETBSD_RELEASE overrides the host system version. See kver(3) for more information. Defaults to ‘no’.

PKG_DBDIR
    Location of the packages database. Defaults to /var/db/pkg.

PKG_SYSCONFBASE
    Base directory of configuration files. Defaults to /usr/pkg/etc.

PKGSRC_COMPILER
    List of values specifying the chain of compilers to invoke when building packages. Defaults to ‘gcc’. If you are defining REAL_CCACHE, remember to prepend ‘ccache’ to this variable's value.

PKGVULNDIR
    Directory where the vulnerabilities file will be installed (inside the chroot). Defaults to /usr/pkg/share.

REAL_PKGVULNDIR
    Directory where the system-wide vulnerabilities file resides (outside the chroot). Defaults to /usr/pkgsrc/distfiles.

ROOTSHELL
    The shell of the root user. Defaults to /bin/ksh.

SETS
    A list of binary sets to be extracted inside DESTDIR. Defaults to ‘base.tgz comp.tgz etc.tgz kern-GENERIC.tgz text.tgz’. If no kernel is extracted by these sets, an empty /netbsd file is created inside the chroot.

SETS_X11
    A list of binary sets of the X Window system. This has the same behavior as SETS. If this variable is set to ‘no’, no X Window is configured inside the chroot jail and no other X variables take effect. Defaults to ‘xbase.tgz xcomp.tgz xetc.tgz xfont.tgz xserver.tgz’.

SYNC_UMOUNT
    If set to ‘yes’, run sync(8) three times after all file systems have been unmounted. Defaults to ‘no’.

USE_AUDIT_PACKAGES
    If set to ‘yes’, let pkg_comp handle the vulnerabilities file automatically. This means that it will install the system-wide vulnerabilities file inside the chroot when needed, keeping both in sync. Defaults to ‘yes’.

USE_GCC3
    If set to ‘yes’, the GNU C Compiler version 3 will be installed inside the chroot environment and used to build all packages, using the lang/gcc3 package. Defaults to ‘no’.

  Mounted file systems
In order to avoid duplicating huge system trees, pkg_comp takes advantage of file system layers. By default, it uses mount_null(8), which duplicates a file system tree into another directory; although you may want to use mount_union(8), or even mount_overlay(8). If the content of these variables is empty, that file system is not mounted.

You can control which layer to use and which options you want with special configuration options, as explained below.

These file systems are mounted before entering the chroot and unmounted after exiting. In order to know if file systems are mounted or not, the program uses a temporary file, called $DESTDIR/pkg_comp/tmp/mount.stat, which controls the number of pkg_comp processes using the chroot environment. If some of them crashes unexpectedly and you notice it does not try to unmount the file systems, this status file may get out of sync. Be sure to check that NO file systems are mounted when issuing a removeroot.

REAL_CCACHE
    Specifies where a global ccache directory resides in the real system. Defaults to nothing, which disables the global cache. Keep in mind that this is specially useful to keep the cache across rebuilds of the sandbox, but be very careful if you plan to share a cache directory between different sandboxes, as this can lead to problems.

REAL_DISTFILES
    Specifies where distfiles reside in the real system. Defaults to /usr/pkgsrc/distfiles.

REAL_DISTFILES_OPTS
    Mount options. Defaults to -t null -o rw.

REAL_PACKAGES
    Specifies where to build binary packages. This variable is specially useful. Defaults to /usr/pkgsrc/packages.

REAL_PACKAGES_OPTS
    Mount options. Defaults to -t null -o rw.

REAL_PKGSRC
    The pkgsrc tree. This can be useful if you want to use several pkgsrc trees independently. Defaults to /usr/pkgsrc.

REAL_PKGSRC_OPTS
    Mount options. Defaults to -t null -o ro.

REAL_SRC
    The src system tree. Usually useless, but may be needed by some packages. Defaults to /usr/src.

REAL_SRC_OPTS
    Mount options. Defaults to -t null -o ro.

MAKEROOT_HOOKS
    A whitespace separated list of functions or external scripts to be executed after the sandbox is created. Two arguments are given to each of them: $DESTDIR, and the word makeroot. Defaults to nothing.

MOUNT_HOOKS
    A whitespace separated list of functions or external scripts to be executed after file systems are mounted. Two arguments are given to each of them: $DESTDIR, and the word mount. Defaults to nothing.

UMOUNT_HOOKS
    A whitespace separated list of functions or external scripts to be executed before file systems are unmounted. Two arguments are given to each of them: $DESTDIR, and the word umount. Defaults to nothing.

TARGETS
A target specifies what pkg_comp should do (as in make). The following list describes all supported targets, in the logical order you should call them.

maketemplate
    Create a sample conf_file. You should edit it after the creation as you will probably want to change the default configuration, specially paths.

makeroot
    Create the chroot environment, based on the specs of the configuration file. This step is required before trying any other, except maketemplate.

build
    Builds the specified packages inside the chroot. You can pass the package names as a relative path within pkgsrc or as the basename of the package directory (i.e. omitting the directory name).

install
    Install the specified binary packages into the chroot. Package names can contain globs. The package files will be taken from within REAL_PACKAGES.

chroot
    Enters the chroot environment. If no arguments are given, ROOTSHELL is executed, otherwise whatever you typed. If the first argument begins with a word prefixed by pkg_, then the ‘chroot’ argument can be omitted (it is implied).

removeroot
    Remove the entire chroot tree. You should do it with this target because it will take care to umount needed mount points.

auto
    This executes several targets automatically, using AUTO_TARGET as BUILD_PKG_COMP_TARGET during the build. The order is: makeroot, build and removeroot. This is useful to create binary packages of several pkgsrc and their dependencies automatically. For this to be useful, you need to set REAL_PACKAGES and use AUTO_PACKAGES or pass package names through the command line.

    If the magic word ‘resume’ is passed as the unique argument to this target, pkg_comp will attempt to resume a previous automatic build for the given configuration.

NOTES
This program uses nullfs to create virtual copies of real trees inside the chroot environment.

You need to install the security/audit-packages package in the host system (and have an up to date vulnerabilities database) if you want security checks to work inside the chroot environment.

SEE ALSO
pkg_delete(1), pkgsrc(7), mount_null(8), sync(8), sysctl(8)

AUTHORS
Julio M. Merino Vidal <jmmv@NetBSD.org>