User:Knuxify/Draft:Porting guide/Kernel package/Downstream
This part of the guide goes over the process of packaging a downstream (vendor) kernel in postmarketOS.
Finding the downstream kernel source
In order to port postmarketOS to the device, we need to be able to build the Linux kernel from source.
- If your device has a LineageOS/TWRP/other Android port, most likely someone has uploaded the kernel sources they used to make the port; look on sites like GitHub (search for repositories named
android_kernel_(vendor)_(codename)
.- Do not confuse this with the
android_device_...
repositories, which contain Android-specific build files.
- Do not confuse this with the
- If the kernel is not on GitHub, try looking on the manufacturer's page; some manufacturers have open source download centers with OSS source code, including the Linux kernel.
- If you find the kernel on the manufacturer's website, mirror it somewhere else - there's no guarantee that the link will always be active. Ports with dead kernel source links will be archived.
- Do not fork the kernel and add patches there. Patches can be applied to the kernel tree in the APKBUILD file; there is no need to create yet another downstream fork, further contributing to the problem of out-of-date forks.
Source code downloads for various manufacturers | |
---|---|
Fairphone | "Kernel Source Code" links at code.fairphone.com |
Motorola | MotorolaMobilityLLC on GitHub |
Lenovo | support.lenovo.com |
Samsung | opensource.samsung.com |
Sony | developerworld.wpp.developer.sony.com |
Xiaomi | MiCode/Xiaomi_Kernel_OpenSource on GitHub |
For many devices, especially from small companies, kernel sources are not publicly available. Per the GPL license, the source code should be open - try e-mailing their support requesting source code.
You may be able to use a kernel for another device with the same SoC; however, you will have to write your own kernel config and it might be missing some drivers. Ask for advice in the porting channel.
Prebuilt kernels, such as kernels extracted directly from the device, will not work. Kernels built for Android are usually missing kernel configuration options needed by postmarketOS. Even if they aren't, a port using a prebuilt kernel will not get accepted into upstream pmaports. If you want to test your luck, there is a wiki page with instructions: Using prebuilt kernels, however it's not guaranteed to work.
Filling the kernel APKBUILD
pmbootstrap generates a very basic template for the kernel package; however, we need to fill in some information to get it to build.
Open the ~/.local/var/pmbootstrap/cache_git/pmaports/device/downstream/linux-vendor-codename
directory (replace vendor-codename
with your device's vendor/codename). Inside of this directory, you will find the following files:
APKBUILD
- the main build script;- A few
.patch
files - these contain patches to apply on top of the kernel. By default, pmbootstrap adds some patches that are necessary to get older kernel versions to build with newer gcc versions.
Setting the source location
For the build process to work, and for the CI/package builder to be able to build the kernel, we need to specify where to look for the kernel source code.
pmbootstrap automatically generates a "Sources" section:
# Source
_repository="(CHANGEME!)"
_commit="ffffffffffffffffffffffffffffffffffffffff"
_config="config-$_flavor.$arch"
source="
$pkgname-$_commit.tar.gz::https://github.com/LineageOS/$_repository/archive/$_commit.tar.gz
...
(xxx).patch
$_config
"
- If your kernel is on GitHub, replace
LineageOS
in the URL with the user/organization, set the_repository
variable to the repository name, and_commit
to the latest commit in the repository (see the image to the right for a guide on where to find the commit hash). - If your kernel is on GitLab, replace the URL with
https://gitlab.com/FIXME/$_repository/-/archive/$_commit.tar.gz
, then follow the rest of the instructions above. - If your kernel is available as a raw .tar.gz file or other archive, remove the
_repository
and_commit
variables and replace the$ppkgname-$_commit.tar.gz::...
line with the URL to the tarball.
Getting the kernel version
The version of the kernel needs to be set in the metadata. To find the kernel version, open the Makefile
file in the kernel repository and find the following lines:
VERSION = 6
PATCHLEVEL = 16
SUBLEVEL = 0
EXTRAVERSION = -rc6
In this example, the version number is 6.16.0-rc6
. In many cases, the EXTRAVERSION
will be empty - in those cases, just leave the 6.10.0
part.
Open the APKBUILD
file and update the pkgver
variable accordingly.
Do not modify pkgrel ! This variable is specific to the kernel package and indicates that it has been modified/rebuilt, but without the kernel version changing. |
Finding the defconfig
The defconfig file describes the Linux kernel configuration options (i.e. which drivers to enable and some settings related to the kernel). We need to find the defconfig used by the device we're porting.
In the kernel, defconfig files are stored in the arch/{arm,arm64}/configs
directory. Look for a file with your device's model number or codename.
- In CyanogenMod/LineageOS kernels, you can typically find the correct defconfig file prefixed with
cyanogenmod_
orlineage_
. - In some kernels, there's a
build.config
file in the repository root; that file contains aDEFCONFIG=
line that has the defconfig name. - If your device has an accompanying
android_device_...
repository, you can find the defconfig name in theTARGET_KERNEL_CONFIG
variable inBoardConfig.mk
.
In case you can't find the kernel config in the kernel sources, you might be able to get it from the device itself (note that this requires root access):
- Connect to the device with
adb shell
- Switch into superuser mode with
su
- Copy the
/proc/config.gz
file to a location you can pull it from, e.g./sdcard
- If the file doesn't exist, this method will not work.
- Exit the adb shell connection
- Pull the config.gz file with
adb pull
- Unpack with
gzip -d config.gz
Copy the defconfig file into the linux-vendor-codename directory
. Rename it to config-vendor-codename.arch
(replace arch with the architecture in the device package - usually armv7 or aarch64).
Then, update the comment at the top of the APKBUILD file by replacing FIXME
with the original filename of the defconfig file (or "config.gz dumped from device"):
# Kernel config based on: FIXME
Finding the DTB
In the case of devices from 2015 onwards, and for most aarch64 devices, the device-specific configuration is stored in a DTB (device tree blob). This blob is compiled from a DTS (device tree source) that is present in the tree.
You can verify if the kernel uses device trees by booting the device and checking if either /sys/firmware/devicetree
or /proc/device-tree
exist.
TODO: Do the above directories always exist? |
Generating the checksums and downloading the sources
In order to verify that the source files haven't changed, the APKBUILD file stores the sha256sum checksums of each of the source files (see the sha256sum
variable at the very bottom of the file).
pmbootstrap can download the source files and generate these checksums automatically. To do this, run:
$ pmbootstrap checksum linux-vendor-codename
This will take a while on first run, as pmbootstrap needs to download the kernel tarball. Future runs will be quicker - the downloaded file will be cached, and used for future checksum calculations as well as the building process.
The checksums are only used for source files. If you modify the APKBUILD and don't modify the source files (or add/remove source files to the APKBUILD), you don't need to re-run the checksum generation command. |
If at this stage you encounter an error, check if your source URL is set correctly. The actual download URL, with all variables replaced, is visible in the log.
Kernel configuration
There are some kernel configuration options that postmarketOS requires that Android kernels don't set by default. Additionally, some kernel config options are known to not play nice with a non-Android userspace. To make the process of fixing up the kernel config easier, pmbootstrap has the kconfig check
command:
$ pmbootstrap kconfig check linux-vendor-codename
If the config check fails, this command will print the options that need to be changed:
WARNING: config-vendor-codename.arch: CONFIG_ANDROID_PARANOID_NETWORK should *not* be set (category:default)
WARNING: config-vendor-codename.arch: CONFIG_DEVTMPFS should be set (category:default)
WARNING: config-vendor-codename.arch: CONFIG_DEVPTS_MULTIPLE_INSTANCES should be set (category:default)
ERROR: kconfig check failed! More info: https://postmarketos.org/kconfig
Copy this list to a text editor and keep it handy - we will need to adjust these config options in a second.
WARNING: Do not modify the kernel config options directly in the config-vendor-codename.arch file! This will not correctly update the dependencies of the config option, and your changes will be silently ignored. |
The Linux kernel's build system has an utility for editing the configuration called menuconfig
. pmbootstrap provides a convenient that automatically calls menuconfig for the kernel package, and saves the changed config file.
To edit the config, run:
$ pmbootstrap kconfig edit linux-vendor-codename
Removing failed patches
Since menuconfig is ran in the kernel tree, the kernel source code needs to be extracted first. The extraction process also applies the patches - thus, when you run this command for the first time, you might see an error like this in the logs:
>>> linux-wiki-example: Unpacking /var/cache/distfiles/linux-vendor-codename-ffff.tar.gz...
>>> linux-vendor-codename: 02_this_patch_fails.patch
patching file arch/arm/mach-msm/perf_trace_counters.h
Hunk #1 FAILED at 121.
1 out of 1 hunk FAILED -- saving rejects to file arch/arm/mach-msm/perf_trace_counters.h.rej
>>> ERROR: linux-vendor-codename: all failed
If that happens, remove the failing patch from the source=
variable in the APKBUILD
, delete the patch file from the same folder and correct the checksums:
$ nano pmaports/device/testing/linux-vendor-codename/APKBUILD # remove from source
$ rm pmaports/device/testing/linux-vendor-codename/02_this_patch_fails.patch
$ pmbootstrap checksum linux-vendor-codename
Afterwards, run the kconfig edit
command again.
"Install ncurses (ncurses-devel) and try again." message
With some older kernels, pmbootstrap kconfig edit
may fail to start, showing a message about missing ncurses. In those cases, you may need to add either the first or both of the following patches:
- device/.shared-patches/linux/fix-check-lxdialog.patch
- device/.shared-patches/linux/fix-check-lxdialog-makefile.patch
See Troubleshooting downstream kernel compilation#Patching the kernel for instructions on how to apply them.
Alternatively, you can use nconfig instead of menuconfig (see hint below).
If all goes correctly, you should see a TUI menu allowing you to choose various options. Navigate using the arrow up/down keys; enter menus by pressing
/ , go back a level by pressing twice.For each kernel option that you need to change, find it in the list and use Enter to change the value. [ ]
means the option is not selected, [M]
means it's built as a module (only for options that enable drivers), [*]
means it's enabled (and built-in, in the case of drivers).
The list at Kernel configuration/Specific options shows which menus you have to enter to find the option. If your option is missing from the article, you can press to open search (enter the config option without the CONFIG_
prefix).
As an alternative to menuconfig , if your kernel supports it, you can use nconfig by providing the -n flag:
$ pmbootstrap kconfig edit -n linux-vendor-codename
searching: prompt will appear at the top of the terminal. Type in the menu item name and use the arrow keys to jump between search results. Use to exit search mode. |
Once you have set all of the kernel configuration options, exit menuconfig and, if asked, save changes. Then, run pmbootstrap kconfig check
again:
$ pmbootstrap kconfig check linux-vendor-codename
Repeat until all warnings are gone:
kconfig check succeeded!
NOTE: chroot is still active (use 'pmbootstrap shutdown' as necessary)
DONE!
pmbootstrap kconfig edit automatically regenerates the checksums once it's done, so you don't need to run pmbootstrap checksum manually. |
Building the kernel and fixing build errors
The next step is to try building the kernel package. To do so, run:
$ pmbootstrap build linux-vendor-codename
The build logs will appear in the pmbootstrap log. Make sure to monitor it for any errors.
It is very likely that the build will fail. postmarketOS uses much newer versions of GCC and other build tools than what your kernel was intended to be compiled with; thus, you will need to patch it to get it to compile succesfully.
Finding errors in the logs
If the build stops because of an error, pmbootstrap build
will show this error:
TODO: add example |
Open the pmbootstrap log and search for messages that contain "error" or "fail".
The most common errors are compilation errors:
In file included from arch/arm/mach-msm/perf_trace_counters.h:127:0, from arch/arm/mach-msm/perf_trace_counters.c:14: include/trace/define_trace.h:79:43: fatal error: ./perf_trace_counters.h: No such file or directory
or linker errors, right at the end of the build process:
drivers/built-in.o: In function `.LANCHOR1': msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock' msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock' Makefile:877: recipe for target '.tmp_vmlinux1' failed make: *** [.tmp_vmlinux1] Error 1
Finding patches
In order to fix the build errors, we will need to patch the kernel.
In many cases, other people have encountered the same issue before, and patches are already available.
- Look for the error on the Troubleshooting downstream kernel compilation page; that page has hints for fixing issues with downstream kernels beyond just adding patches.
- You can find patches in pmaports that mention the error with
grep -iR "error message" ~/.local/var/pmbootstrap/cache_git/pmaports/device
.
TODO: do we want Troubleshooting downstream kernel compilation#Patching the kernel here or on that page? |
Continue to /Device package.