Arch linux arch build system

Arch Build System

The Arch build system is a ports-like system for building and packaging software from source code. While pacman is the specialized Arch tool for binary package management (including packages built with the ABS), ABS is a collection of tools for compiling source into installable .pkg.tar.zst packages.

Ports is a system used by *BSD to automate the process of building software from source code. The system uses a port to download, unpack, patch, compile, and install the given software. A port is merely a small directory on the user’s computer, named after the corresponding software to be installed, that contains a few files with the instructions for building and installing the software from source. This makes installing software as simple as typing make or make install clean within the port’s directory.

ABS is a similar concept. A part of ABS is a SVN repository and an equivalent Git repository. The repository contains a directory corresponding to each package available in Arch Linux. The directories of the repository contain a PKGBUILD file (and sometimes other files), and do not contain the software source nor binary. By issuing makepkg inside a directory, the software sources are downloaded, the software is compiled, and then packaged within the build directory. Then you can use pacman to install the package.

Contents

Overview

This article or section needs expansion.

‘ABS’ may be used as an umbrella term since it includes and relies on several other components; therefore, though not technically accurate, ‘ABS’ can refer to the following tools as a complete toolkit:

Repository tree The directory structure containing files needed to build all official packages but not the packages themselves nor the source files of the software. It is available in svn and git repositories. See the section #Repository tree for more information. PKGBUILD A Bash script that contains the URL of the source code along with the compilation and packaging instructions. makepkg A shell command tool which reads the PKGBUILDs, automatically downloads and compiles the sources and creates a .pkg.tar* according to the PKGEXT array in makepkg.conf . You may also use makepkg to make your own custom packages from the AUR or third-party sources. See Creating packages for more information. pacman pacman is completely separate, but is necessarily invoked either by makepkg or manually, to install and remove the built packages and for fetching dependencies. AUR The Arch User Repository is separate from ABS but AUR (unsupported) PKGBUILDs are built using makepkg to compile and package up software. In contrast to the ABS tree which is simply a bare git repository, the AUR exists as a polished website interface with various interactive features. It contains many thousands of user-contributed PKGBUILDs for software which is unavailable as an official Arch package. If you need to build a package outside the official Arch tree, chances are it is in the AUR.

Repository tree

The core, extra, and testing official repositories are in the packages repository for checkout. The community and multilib repositories are in the community repository.

Each package has its own subdirectory. Within it there are repos and trunk directories. repos is further broken down by repository name (e.g., core) and architecture. PKGBUILDs and files found in repos are used in official builds. Files found in trunk are used by developers in preparation before being copied to repos .

For example, the tree for acl looks like this:

The source code for the package is not present in the ABS directory. Instead, the PKGBUILD contains a URL that will download the source code when the package is built.

Use cases

ABS automates certain tasks related to compilation from source. Its use cases are:

• Any use case that requires you to compile or recompile a package.
• Make and install new packages from source of software for which no packages are yet available (see Creating packages) .
• Customize existing packages to fit your needs (e.g. enabling or disabling options, patching).
• Rebuild your entire system using your compiler flags, «à la FreeBSD».
• Cleanly build and install your own custom kernel (see Kernel compilation).
• Get kernel modules working with a custom kernel.
• Easily compile and install a newer, older, beta, or development version of an Arch package by editing the version number in the PKGBUILD.

Usage

Retrieve PKGBUILD source

To retrieve the PKGBUILD required to build a certain package from source, you can either use SVN or a Git-based approach using the asp package which is a thin wrapper around the svntogit repositories. In the following, the svn-based method as well as the git-based method are described.

Retrieve PKGBUILD source using Git

As a precondition, install the asp package. Asp is a tool to retrieve the build source files for Arch Linux packages using the Git interface. Also see the Arch Linux BBS forum thread [1].

To clone the svntogit-repository for a specific package, use:

This will clone the git repository for the given package into a directory named like the package.

To update the cloned git repository, run asp update followed by git pull inside the git repository.

Furthermore, you can use all other git commands to checkout an older version of the package or to track custom changes. For more information on git usage, see the git page.

If you just want to copy a snapshot of the current PKGBUILD for a specific package, use:

Retrieve PKGBUILD source using SVN

Prerequisites

Checkout a repository

To checkout the community and multilib repositories:

In both cases, it simply creates an empty directory, but it does know that it is an svn checkout.

Checkout a package

In the directory containing the svn repository you checked out (i.e., packages or community), do:

This will pull the package you requested into your checkout. From now on, any time you svn update at the top level, this will be updated as well.

If you specify a package that does not exist, svn will not warn you. It will just print something like «At revision 115847», without creating any files. If that happens:

• check your spelling of the package name
• check that the package has not been moved to another repository (i.e. from community to the main repository)
• check https://archlinux.org/packages to see if the package is built from another base package (for example, python-tensorflow is built from the tensorflow PKGBUILD)

You should periodically update all of your checked out packages if you wish to perform rebuilds on more recent revisions of the repositories. To do so, do:

Checkout an older version of a package

Within the svn repository you checked out as described in #Checkout a repository (i.e. «packages» or «community»), first examine the log:

Find out the revision you want by examining the history, then specify the revision you wish to checkout. For example, to checkout revision r1729 you would do:

This will update an existing working copy of package-name to the chosen revision.

You can also specify a date. If no revision on that day exists, svn will grab the most recent package before that time. The following example checks out the revision from 2009-03-03:

It is possible to checkout packages at versions before they were moved to another repository as well; check the logs thoroughly for the date they were moved or the last revision number.

Build package

Configure makepkg for building packages from the PKGBUILDs you have checked out, as explained in makepkg#Configuration.

Then, copy the directory containing the PKGBUILD you wish to modify to a new location. Make the desired modifications there and use makepkg there as described in makepkg#Usage to create and install the new package.

Tips and tricks

Preserve modified packages

Updating the system with pacman will replace a modified package from ABS with the package of the same name from the official repositories. See the following instructions for how to avoid this.

Insert a group array into the PKGBUILD, and add the package to a group called modified .

Add this group to the section IgnoreGroup in /etc/pacman.conf .

If new versions are available in the official repositories during a system update, pacman prints a note that it is skipping this update because it is in the IgnoreGroup section. At this point the modified package should be rebuilt from ABS to avoid partial upgrades.

Источник

Kernel/Arch Build System

See Kernels for the main article.

The Arch Build System can be used to build a custom kernel based on the official linux package. This compilation method can automate the entire process, and is based on a very well tested package. You can edit the PKGBUILD to use a custom kernel configuration or add additional patches.

Contents

Getting the ingredients

Since you will be using makepkg, follow the best practices outlined there first. For example, you cannot run makepkg as root/sudo. Therefore, create a build directory in your user home first.

Install the asp package and the base-devel package group.

You need a clean kernel to start your customization from. Retrieve PKGBUILD source using Git and few other files into your build directory by running:

At this point, the directory tree looks like (there may be a few other files):

Then, get any other file you need (e.g. custom configuration files, patches, etc.) from the respective sources.

Modifying the PKGBUILD

Edit PKGBUILD and look for the pkgbase parameter. Change this to your custom package name, e.g.:

Avoid creating the doc

A large portion of the lengthy compiling effort is devoted to creating the documentation. As of June 16. 2021, the following patch to PKGBUILD avoids its creation:

This patch deletes line #63, and changes line #194. You might have to edit the PKGBUILD file manually if it does not apply cleanly.

Changing prepare()

In prepare() function, you can apply needed kernel patches or change kernel build configuration.

If you need to change a few config options you can edit config file in the source.

Or you can use a GUI tool to tweak the options. Comment make olddefconfig in the prepare() function of the PKGBUILD, and add your favorite tool:

Generate new checksums

#Changing prepare() suggests a possible modification to $_srcname/.config . Since this path is not where downloading the package files ended, its checksum was not checked by makepkg (which actually checked $_srcname/../../config ).

If you replaced the downloaded config with another config file before running makepkg, install the pacman-contrib package and generate new checksums by running:

Compiling

You can now proceed to compile your kernel by the usual command makepkg .

If you have chosen an interactive program for configuring the kernel parameters (like menuconfig), you need to be there during the compilation.

The -s parameter will download any additional dependencies used by recent kernels such as xml and docs.

Installing

The compile step will leave two packages in the

/build/linux folder, one for the kernel and one for the kernel headers. They might have names like:

Best practice is to install both packages together as they might be both needed (e.g. DKMS):

(substitute the actual names of the files you have in the folder)

Boot loader

If you have modified pkgbase in order to have your new kernel installed alongside the default kernel you will need to update your bootloader configuration file and add new entries (‘default’ and ‘fallback’) for your custom kernel and the associated initramfs images.

Updating

Assuming one has an arch kernel source that they want to update, one method to do that is with https://github.com/archlinux/linux. In what follows, the top kernel source directory is assumed at

In general, arch sets an arch kernel source with two local git repositories. The one at archlinux-linux/ is a local bare git repository pointing to https://github.com/archlinux/linux.git . The other one is at src/archlinux-linux/ , pulling from the bare repository. Possible local patches, and building, are expected at src/archlinux-linux/ .

For this example, the HEAD of the locally installed bare git repository source at archlinux-linux/ was initially pointing to

which is somewhere between v5.2.5-arch1 and v5.2.6-arch1.

One can see it fetched v5.2.7-arch1, which was the newest archlinux tag, because it prints what new tags were obtained. If no new tags were obtained then there is no newer archlinux source available.

Now the source can be updated where the actual build will take place.

You can verify you are on track with something like

This shows few specific archlinux patches between Arch Linux kernel v5.2.7-arch1 and Linux 5.2.7 .

The up to date PKGBUILD, as well archlinux kernel configuration file, can be pulled in by the asp command:

Now you should merge files located in

/build/linux/ . Merging can also done manually, or with specific utilities. Review #Changing prepare(), and run manually most, if not all, the shell commands of PKGBUILD::prepare().

At this point, makepkg —verifysource should succeed. While #Compiling, make sure to also add —noextract option to the makepkg command, since it should be able to build the packages as if the source was extracted by makepkg —nobuild . And you are back to #Installing.

Cleanup

One will probably want to remove

/build/linux/linux/ after merging. In addition,

/build/linux/src/archlinux will accumulate branches in the form of 5.2.7-arch1 if more recent updates are done in this fashion. These can be deleted with

Источник

Kernel/Traditional compilation

This article is an introduction to building custom kernels from kernel.org sources. This method of compiling kernels is the traditional method common to all distributions. It can be, depending on your background, more complicated than using the Kernels/Arch Build System. Consider the Arch Build System tools are developed and maintained to make repeatable compilation tasks efficient and safe.

Contents

Preparation

It is not necessary (or recommended) to use the root account or root privileges (i.e. via Sudo) for kernel preparation.

Install the core packages

Install the base-devel package group, which contains necessary packages such as make and gcc . It is also recommended to install the following packages, as listed in the default Arch kernel PKGBUILD: xmlto , kmod , inetutils , bc , libelf , git , cpio , perl , tar , xz .

Create a kernel compilation directory

It is recommended to create a separate build directory for your kernel(s). In this example, the directory kernelbuild will be created in the home directory:

Download the kernel source

Download the kernel source from https://www.kernel.org. This should be the tarball ( tar.xz ) file for your chosen kernel.

It can be downloaded by simply right-clicking the tar.xz link in your browser and selecting Save Link As. , or any other number of ways via alternative graphical or command-line tools that utilise HTTP, TFTP, Rsync, or Git.

In the following command-line example, wget has been installed and is used inside the

/kernelbuild directory to obtain kernel 4.8.6:

You should also verify the correctness of the download before trusting it. First grab the signature, then use that to grab the fingerprint of the signing key, then use the fingerprint to obtain the actual signing key:

Note the signature was generated for the tar archive (i.e. extension .tar ), not the compressed .tar.xz file that you have downloaded. You need to decompress the latter without untarring it. Verify that you have xz installed, then you can proceed like so:

Do not proceed if this does not result in output that includes the string «Good signature».

If wget was not used inside the build directory, it will be necessary to move the tarball into it, e.g.

Unpack the kernel source

Within the build directory, unpack the kernel tarball:

To finalise the preparation, ensure that the kernel tree is absolutely clean; do not rely on the source tree being clean after unpacking. To do so, first change into the new kernel source directory created, and then run the make mrproper command:

Kernel configuration

This is the most crucial step in customizing the default kernel to reflect your computer’s precise specifications. Kernel configuration is set in its .config file, which includes the use of Kernel modules. By setting the options in .config properly, your kernel and computer will function most efficiently.

You can do a mixture of two things:

• Use the default Arch settings from an official kernel (recommended)
• Manually configure the kernel options (optional, advanced and not recommended)

Default Arch configuration

This method will create a .config file for the custom kernel using the default Arch kernel settings. If a stock Arch kernel is running, you can use the following command inside the custom kernel source directory:

Otherwise, the default configuration can be found online in the official Arch Linux kernel package.

Advanced configuration

There are several tools available to fine-tune the kernel configuration, which provide an alternative to otherwise spending hours manually configuring each and every one of the options available during compilation.

Those tools are:

• make menuconfig : Command-line ncurses interface superseded by nconfig
• make nconfig : Newer ncurses interface for the command-line
• make xconfig : User-friendly graphical interface that requires packagekit-qt5 to be installed as a dependency. This is the recommended method — especially for less experienced users — as it is easier to navigate, and information about each option is also displayed.
• make gconfig : Graphical configuration similar to xconfig but using gtk. This requires gtk2 , glib2 and libgladeAUR .

The chosen method should be run inside the kernel source directory, and all will either create a new .config file, or overwrite an existing one where present. All optional configurations will be automatically enabled, although any newer configuration options (i.e. with an older kernel .config ) may not be automatically selected.

Once the changes have been made save the .config file. It is a good idea to make a backup copy outside the source directory. You may need to do this multiple times before you get all the options right.

If unsure, only change a few options between compilations. If you cannot boot your newly built kernel, see the list of necessary config items here.

Running lspci -k # from liveCD lists names of kernel modules in use. Most importantly, you must maintain cgroups support. This is necessary for systemd. For more detailed information, see Gentoo:Kernel/Gentoo Kernel Configuration Guide and Gentoo:Intel#Kernel or Gentoo:Ryzen#Kernel for Intel or AMD Ryzen processors.

Compilation

Compilation time will vary from as little as fifteen minutes to over an hour, depending on your kernel configuration and processor capability. Once the .config file has been set for the custom kernel, within the source directory run the following command to compile:

Installation

Install the modules

Once the kernel has been compiled, the modules for it must follow. First build the modules:

Then install the modules. As root or with root privileges, run the following command to do so:

This will copy the compiled modules into /lib/modules/ — . For example, for kernel version 4.8 installed above, they would be copied to /lib/modules/4.8.6-ARCH . This keeps the modules for individual kernels used separated.

Copy the kernel to /boot directory

The kernel compilation process will generate a compressed bzImage (big zImage) of that kernel, which must be copied to the /boot directory and renamed in the process. Provided the name is prefixed with vmlinuz- , you may name the kernel as you wish. In the examples below, the installed and compiled 4.8 kernel has been copied over and renamed to vmlinuz-linux48 :

Make initial RAM disk

If you do not know what making an initial RAM disk is, see Initramfs on Wikipedia and mkinitcpio.

Automated preset method

An existing mkinitcpio preset can be copied and modified so that the custom kernel initramfs images can be generated in the same way as for an official kernel. This is useful where intending to recompile the kernel (e.g. where updated). In the example below, the preset file for the stock Arch kernel will be copied and modified for kernel 4.8, installed above.

First, copy the existing preset file, renaming it to match the name of the custom kernel specified as a suffix to /boot/vmlinuz- when copying the bzImage (in this case, linux48 ):

Second, edit the file and amend for the custom kernel. Note (again) that the ALL_kver= parameter also matches the name of the custom kernel specified when copying the bzImage :

Finally, generate the initramfs images for the custom kernel in the same way as for an official kernel:

Manual method

Rather than use a preset file, mkinitcpio can also be used to generate an initramfs file manually. The syntax of the command is:

• -k ( —kernel ): Specifies the modules to use when generating the initramfs image. The name will be the same as the name of the custom kernel source directory (and the modules directory for it, located in /usr/lib/modules/ ).
• -g ( —generate ): Specifies the name of the initramfs file to generate in the /boot directory. Again, using the naming convention mentioned above is recommended.

For example, the command for the 4.8 custom kernel installed above would be:

Copy System.map

The System.map file is not required for booting Linux. It is a type of «phone directory» list of functions in a particular build of a kernel. The System.map contains a list of kernel symbols (i.e function names, variable names etc) and their corresponding addresses. This «symbol-name to address mapping» is used by:

• Some processes like klogd, ksymoops, etc.
• By OOPS handler when information has to be dumped to the screen during a kernel crash (i.e info like in which function it has crashed).

If your /boot is on a filesystem which supports symlinks (i.e., not FAT32), copy System.map to /boot , appending your kernel’s name to the destination file. Then create a symlink from /boot/System.map to point to /boot/System.map- :

After completing all steps above, you should have the following 3 files and 1 soft symlink in your /boot directory along with any other previously existing files:

• Kernel: vmlinuz-
• Initramfs: Initramfs- .img
• System Map: System.map-
• System Map kernel symlink

Bootloader configuration

Add an entry for your new kernel in your bootloader’s configuration file. See Arch boot process#Feature comparison for possible boot loaders, their wiki articles and other information.

Источник