Jump to content

User:Knuxify/Draft:Porting guide/Kernel package/Downstream

From postmarketOS Wiki

This part of the guide goes over the process of packaging a downstream (vendor) kernel in postmarketOS.

Finding the downstream kernel source

In order to port postmarketOS to the device, we need to be able to build the Linux kernel from source.

  • If your device has a LineageOS/TWRP/other Android port, most likely someone has uploaded the kernel sources they used to make the port; look on sites like GitHub (search for repositories named android_kernel_(vendor)_(codename).
    • Do not confuse this with the android_device_... repositories, which contain Android-specific build files.
  • If the kernel is not on GitHub, try looking on the manufacturer's page; some manufacturers have open source download centers with OSS source code, including the Linux kernel.
    • If you find the kernel on the manufacturer's website, mirror it somewhere else - there's no guarantee that the link will always be active. Ports with dead kernel source links will be archived.
    • Do not fork the kernel and add patches there. Patches can be applied to the kernel tree in the APKBUILD file; there is no need to create yet another downstream fork, further contributing to the problem of out-of-date forks.
Source code downloads for various manufacturers 
Fairphone "Kernel Source Code" links at code.fairphone.com
Motorola MotorolaMobilityLLC on GitHub
Lenovo support.lenovo.com
Samsung opensource.samsung.com
Sony developerworld.wpp.developer.sony.com
Xiaomi MiCode/Xiaomi_Kernel_OpenSource on GitHub

For many devices, especially from small companies, kernel sources are not publicly available. Per the GPL license, the source code should be open - try e-mailing their support requesting source code.

You may be able to use a kernel for another device with the same SoC; however, you will have to write your own kernel config and it might be missing some drivers. Ask for advice in the porting channel.

Prebuilt kernels, such as kernels extracted directly from the device, will not work. Kernels built for Android are usually missing kernel configuration options needed by postmarketOS. Even if they aren't, a port using a prebuilt kernel will not get accepted into upstream pmaports. If you want to test your luck, there is a wiki page with instructions: Using prebuilt kernels, however it's not guaranteed to work.

Filling the kernel APKBUILD

pmbootstrap generates a very basic template for the kernel package; however, we need to fill in some information to get it to build.

Open the ~/.local/var/pmbootstrap/cache_git/pmaports/device/downstream/linux-vendor-codename directory (replace vendor-codename with your device's vendor/codename). Inside of this directory, you will find the following files:

  • APKBUILD - the main build script;
  • A few .patch files - these contain patches to apply on top of the kernel. By default, pmbootstrap adds some patches that are necessary to get older kernel versions to build with newer gcc versions.

Setting the source location

For the build process to work, and for the CI/package builder to be able to build the kernel, we need to specify where to look for the kernel source code.

pmbootstrap automatically generates a "Sources" section:

# Source
_repository="(CHANGEME!)"
_commit="ffffffffffffffffffffffffffffffffffffffff"
_config="config-$_flavor.$arch"
source="
	$pkgname-$_commit.tar.gz::https://github.com/LineageOS/$_repository/archive/$_commit.tar.gz
    ...
    (xxx).patch
    $_config
    "
  • If your kernel is on GitHub, replace LineageOS in the URL with the user/organization, set the _repository variable to the repository name, and _commit to the latest commit in the repository (see the image to the right for a guide on where to find the commit hash).
  • If your kernel is on GitLab, replace the URL with https://gitlab.com/FIXME/$_repository/-/archive/$_commit.tar.gz, then follow the rest of the instructions above.
  • If your kernel is available as a raw .tar.gz file or other archive, remove the _repository and _commit variables and replace the $ppkgname-$_commit.tar.gz::... line with the URL to the tarball.

Getting the kernel version

The version of the kernel needs to be set in the metadata. To find the kernel version, open the Makefile file in the kernel repository and find the following lines:

VERSION = 6
PATCHLEVEL = 16
SUBLEVEL = 0
EXTRAVERSION = -rc6

In this example, the version number is 6.16.0-rc6. In many cases, the EXTRAVERSION will be empty - in those cases, just leave the 6.10.0 part.

Open the APKBUILD file and update the pkgver variable accordingly.

Note Do not modify pkgrel! This variable is specific to the kernel package and indicates that it has been modified/rebuilt, but without the kernel version changing.

Finding the defconfig

The defconfig file describes the Linux kernel configuration options (i.e. which drivers to enable and some settings related to the kernel). We need to find the defconfig used by the device we're porting.

In the kernel, defconfig files are stored in the arch/{arm,arm64}/configs directory. Look for a file with your device's model number or codename.

  • In CyanogenMod/LineageOS kernels, you can typically find the correct defconfig file prefixed with cyanogenmod_ or lineage_.
  • In some kernels, there's a build.config file in the repository root; that file contains a DEFCONFIG= line that has the defconfig name.
  • If your device has an accompanying android_device_... repository, you can find the defconfig name in the TARGET_KERNEL_CONFIG variable in BoardConfig.mk.

In case you can't find the kernel config in the kernel sources, you might be able to get it from the device itself (note that this requires root access):

  • Connect to the device with adb shell
  • Switch into superuser mode with su
  • Copy the /proc/config.gz file to a location you can pull it from, e.g. /sdcard
    • If the file doesn't exist, this method will not work.
  • Exit the adb shell connection
  • Pull the config.gz file with adb pull
  • Unpack with gzip -d config.gz

Copy the defconfig file into the linux-vendor-codename directory. Rename it to config-vendor-codename.arch (replace arch with the architecture in the device package - usually armv7 or aarch64).

Then, update the comment at the top of the APKBUILD file by replacing FIXME with the original filename of the defconfig file (or "config.gz dumped from device"):

# Kernel config based on: FIXME

Finding the DTB

In the case of devices from 2015 onwards, and for most aarch64 devices, the device-specific configuration is stored in a DTB (device tree blob). This blob is compiled from a DTS (device tree source) that is present in the tree.

You can verify if the kernel uses device trees by booting the device and checking if either /sys/firmware/devicetree or /proc/device-tree exist.

TODO TODO: Do the above directories always exist?

Generating the checksums and downloading the sources

In order to verify that the source files haven't changed, the APKBUILD file stores the sha256sum checksums of each of the source files (see the sha256sum variable at the very bottom of the file).

pmbootstrap can download the source files and generate these checksums automatically. To do this, run:

$ pmbootstrap checksum linux-vendor-codename

This will take a while on first run, as pmbootstrap needs to download the kernel tarball. Future runs will be quicker - the downloaded file will be cached, and used for future checksum calculations as well as the building process.

Note The checksums are only used for source files. If you modify the APKBUILD and don't modify the source files (or add/remove source files to the APKBUILD), you don't need to re-run the checksum generation command.

If at this stage you encounter an error, check if your source URL is set correctly. The actual download URL, with all variables replaced, is visible in the log.

Kernel configuration

There are some kernel configuration options that postmarketOS requires that Android kernels don't set by default. Additionally, some kernel config options are known to not play nice with a non-Android userspace. To make the process of fixing up the kernel config easier, pmbootstrap has the kconfig check command:

$ pmbootstrap kconfig check linux-vendor-codename

If the config check fails, this command will print the options that need to be changed:

WARNING: config-vendor-codename.arch: CONFIG_ANDROID_PARANOID_NETWORK should *not* be set (category:default)
WARNING: config-vendor-codename.arch: CONFIG_DEVTMPFS should be set (category:default)
WARNING: config-vendor-codename.arch: CONFIG_DEVPTS_MULTIPLE_INSTANCES should be set (category:default)
ERROR: kconfig check failed! More info: https://postmarketos.org/kconfig

Copy this list to a text editor and keep it handy - we will need to adjust these config options in a second.

Warning WARNING: Do not modify the kernel config options directly in the config-vendor-codename.arch file! This will not correctly update the dependencies of the config option, and your changes will be silently ignored.

The Linux kernel's build system has an utility for editing the configuration called menuconfig. pmbootstrap provides a convenient that automatically calls menuconfig for the kernel package, and saves the changed config file.

To edit the config, run:

$ pmbootstrap kconfig edit linux-vendor-codename

Removing failed patches

Since menuconfig is ran in the kernel tree, the kernel source code needs to be extracted first. The extraction process also applies the patches - thus, when you run this command for the first time, you might see an error like this in the logs:

>>> linux-wiki-example: Unpacking /var/cache/distfiles/linux-vendor-codename-ffff.tar.gz...
>>> linux-vendor-codename: 02_this_patch_fails.patch
patching file arch/arm/mach-msm/perf_trace_counters.h
Hunk #1 FAILED at 121.
1 out of 1 hunk FAILED -- saving rejects to file arch/arm/mach-msm/perf_trace_counters.h.rej
>>> ERROR: linux-vendor-codename: all failed

If that happens, remove the failing patch from the source= variable in the APKBUILD, delete the patch file from the same folder and correct the checksums:

$ nano pmaports/device/testing/linux-vendor-codename/APKBUILD # remove from source
$ rm pmaports/device/testing/linux-vendor-codename/02_this_patch_fails.patch
$ pmbootstrap checksum linux-vendor-codename

Afterwards, run the kconfig edit command again.

"Install ncurses (ncurses-devel) and try again." message

With some older kernels, pmbootstrap kconfig edit may fail to start, showing a message about missing ncurses. In those cases, you may need to add either the first or both of the following patches:

See Troubleshooting downstream kernel compilation#Patching the kernel for instructions on how to apply them.

Alternatively, you can use nconfig instead of menuconfig (see hint below).

Adjusting the kernel config in menuconfig

If all goes correctly, you should see a TUI menu allowing you to choose various options. Navigate using the arrow up/down keys; enter menus by pressing Enter/Return, go back a level by pressing Escape twice.

For each kernel option that you need to change, find it in the list and use Enter to change the value. [ ] means the option is not selected, [M] means it's built as a module (only for options that enable drivers), [*] means it's enabled (and built-in, in the case of drivers).

The list at Kernel configuration/Specific options shows which menus you have to enter to find the option. If your option is missing from the article, you can press / to open search (enter the config option without the CONFIG_ prefix).

Hint As an alternative to menuconfig, if your kernel supports it, you can use nconfig by providing the -n flag:
$ pmbootstrap kconfig edit -n linux-vendor-codename
A convenient feature of nconfig compared to standard menuconfig is its search function. Pressing / will start searching within the currently opened submenu; A searching: prompt will appear at the top of the terminal. Type in the menu item name and use the arrow keys to jump between search results. Use Escape to exit search mode.

Once you have set all of the kernel configuration options, exit menuconfig and, if asked, save changes. Then, run pmbootstrap kconfig check again:

$ pmbootstrap kconfig check linux-vendor-codename

Repeat until all warnings are gone:

kconfig check succeeded!
NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
DONE!
Note pmbootstrap kconfig edit automatically regenerates the checksums once it's done, so you don't need to run pmbootstrap checksum manually.

Building the kernel and fixing build errors

The next step is to try building the kernel package. To do so, run:

$ pmbootstrap build linux-vendor-codename

The build logs will appear in the pmbootstrap log. Make sure to monitor it for any errors.

It is very likely that the build will fail. postmarketOS uses much newer versions of GCC and other build tools than what your kernel was intended to be compiled with; thus, you will need to patch it to get it to compile succesfully.

Finding errors in the logs

If the build stops because of an error, pmbootstrap build will show this error:

TODO TODO: add example

Open the pmbootstrap log and search for messages that contain "error" or "fail".

The most common errors are compilation errors:

In file included from arch/arm/mach-msm/perf_trace_counters.h:127:0,
                 from arch/arm/mach-msm/perf_trace_counters.c:14:
include/trace/define_trace.h:79:43: fatal error: ./perf_trace_counters.h: No such file or directory

or linker errors, right at the end of the build process:

drivers/built-in.o: In function `.LANCHOR1':
msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock'
msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock'
Makefile:877: recipe for target '.tmp_vmlinux1' failed
make: *** [.tmp_vmlinux1] Error 1

Finding patches

In order to fix the build errors, we will need to patch the kernel.

In many cases, other people have encountered the same issue before, and patches are already available.

  • Look for the error on the Troubleshooting downstream kernel compilation page; that page has hints for fixing issues with downstream kernels beyond just adding patches.
  • You can find patches in pmaports that mention the error with grep -iR "error message" ~/.local/var/pmbootstrap/cache_git/pmaports/device.
TODO TODO: do we want Troubleshooting downstream kernel compilation#Patching the kernel here or on that page?

Continue to /Device package.