Mainlining Guide
This is a draft of an idea, not reviewed by people who have actually done some mainlining yet. The commands are not tested at this point. The steps are not meant to be followed until this is reviewed! |
Introduction
This guide will walk you through mainlining your device. Starting by collecting all required information, setting up your build environment and picking a mentor that will guide you through the tricky parts. For the devices where this will work, you should be fine with basic shell knowledge, and you will be rewarded with learning a lot of new stuff and booting the mainline kernel on your device!
Requirements
postmarketOS port
To make the instructions easier, this guide assumes that postmarketOS was already ported with a downstream kernel to your device, and that you can enter a shell after it booted via USB (SSH or telnet). Downstream kernels are the ones from the vendor, that ship with the device, and all derivatives from that (e.g. your typical Android / LineageOS ROM). You can check if it was ported already on the devices page, and start a new port if necessary.
Mainlining section
Please follow these instructions to extend your device's wiki page with a mainlining section that has all required information before proceeding. It will serve both as reference of which peripherals are in the device, and as progress indicator of what the status is on each of these components.
Mentor
The mainlining process varies greatly for each device, which means we can't just write down straight forward instructions here. But luckily there are skilled people who have a bigger picture of a specific SoC and they know how to help you out efficiently.
SoC | Mentors |
---|---|
Qualcomm (msm...) | opendata26 (#postmarketOS), bamse on ##linux-msm and robclark on the same channel for graphics related things. |
OMAP3 | pavel? |
Make sure that your SoC is in the list (if it isn't then we don't have an easy way for mainlining right now, sorry). Let's do some more preparation now, so everything is in place before you kindly ask your mentor to help you with the process.
Preparation
pmbootstrap setup
If you have not done this already, please download and initialize pmbootstrap
. You can stay with the defaults, just make sure to select the right device, and (to save you some time) none
as user interface. Replace ~/code
with the path where you would like to store the source code.
$ cd ~/code
$ git clone https://github.com/postmarketOS/pmbootstrap
$ cd pmbootstrap
$ alias pmbootstrap=$PWD/pmbootstrap.py
$ pmbootstrap init
Generate an initramfs with the debug-shell hook:
$ pmbootstrap initfs hook_add debug-shell
$ pmbootstrap export
Development tools on host system
See also: What tools do I need? - kernelnewbies.org |
The idea of pmbootstrap
is, that you don't need to install anything on your host system, and we do everything in Alpine chroots. But this does not work with the process outlined in this guide yet, so we need to diverge from that idea during this step. Please install the following tools on your host system with your package manager:
- libncurses5-dev
- gcc
- make
- bc
- libssl-dev
- gcc cross-compiler, depending on your device architecture:
aarch64
device: look for aaarch64
/arm64
cross-compilerarmhf
device: look for aarmhf
/armv6
/armv7
cross-compiler
Kernel
Download a copy of linux-next
:
$ cd ~/code
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-next.git linux
$ cd linux
If you have already checked out the regular linux repository, you can add the linux-next
branch as follows:
$ cd ~/code/linux
$ git remote add linux-next https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git
$ git fetch --tags linux-next
$ git checkout -b development linux-next/master
After running the commands of one of the two blocks above, add a new branch for your device (replace lg-mako
with your device's name):
$ cd ~/code/linux
$ git checkout -b "device-lg-mako"
During the mainlining process, you will be required to fire up menuconfig
and to compile everything. Let's do this once to make sure that everything works. You can exit menuconfig right away after it starts. If anything does not run through, you are most likely missing a dependency as listed in the step above.
Menuconfig
armhf
device:
$ export ARCH=arm
$ export CROSS_COMPILE="arm-none-eabi-"
$ make defconfig
$ make menuconfig
aarch64
device:
$ export ARCH=arm64
$ export CROSS_COMPILE="aarch64-none-eabi-"
$ make defconfig
$ make menuconfig
Kernel compilation
Make sure you did the exports from #Menuconfig above! |
Replace 3
with the amount of CPUs on your host machine plus one (e.g. quad-core: means -j5
).
$ make -j3
Got everything?
Please double check with the following checklist that you meet all the requirements before proceeding further. If you need any help with the above, just ask in the channel or on GitHub as usually.
- Device ported to postmarketOS
- Mainlining section added to device page
- Mentor
- Initramfs with debug-shell generated
- Development tools on host system
- Menuconfig tested
- Kernel compilation tested
Contact your mentor
Hop on Matrix or IRC and ask the mentor if they have time to walk you through the further steps. This should be obvious, but remember to be nice and understanding if the mentor does not have time right away, they are supporting you for free so you'll need to be grateful and patient.
The following steps are meant to be taken together with the mentor.
Device Tree Source
Start with an existing file
The DTS file describes how the peripherals of the device are connected to the SoC. Your mentor probably knows which file you should use to start your work on. This may either be a file in the existing Linux source tree, or a file your mentor uploaded somewhere.
Example for copying an existing file:
$ cd ~/code/linux/arch/arm/boot/dts
$ cp qcom-apq8064-asus-nexus7-flo.dts qcom-apq8064-lg-nexus4-mako.dts
Let's commit that directly, so it will be easy to see what you have changed later on by running git diff
(adjust the description of the commit accordingly):
$ git add -A
$ git commit -m "Starting with: qcom-apq8064-asus-nexus7-flo.dts"
Adjust model and compatible
Open the DTS file in an editor and replace the model
and compatible
strings at the top of the file:
// SPDX-License-Identifier: GPL-2.0
#include "qcom-apq8064-v2.0.dtsi"
#include <dt-bindings/gpio/gpio.h>
#include <dt-bindings/input/input.h>
#include <dt-bindings/pinctrl/qcom,pmic-gpio.h>
/ {
model = "Asus Nexus7(flo)";
compatible = "asus,nexus7-flo", "qcom,apq8064";
// ...
In the example, we replace model
with "LG Nexus4(mako)"
and the first string after compatible with "lg,nexus4-mako
(we leave the second string in place, as this is the name of the SoC, which is the same).
And again, let's commit this change. It's recommended to commit after every change you make to the DTS files so you can revert easily to the last known working state if necessary:
$ git add -A
$ git commit -m "Adjusted model and compatible"
Minimal booting version
Your mentor will instruct you how to modify the file further, so you have a minimal configuration that allows you to boot into the mainline kernel if everything went right. Almost nothing will work at this point, except for initializing RAM and serial output. We will work our way to enabling one feature after another throughout the process.
Testing
Defconfig and compilation
Ask your mentor for the right defconfig to use and export the variables as in menuconfig (mind the armhf
and aarch64
differences). Then configure the kernel to use the defconfig and finally build it (like in kernel compilation above). Example:
$ export ARCH=arm
$ export CROSS_COMPILE="arm-none-eabi-"
$ make qcom_defconfig
$ make -j5
Append DTB
Now append the DTB (device tree blob), which has been generated from your DTS file, to the kernel:
$ cat arch/arm/boot/zImage arch/arm/boot/dts/qcom-msm8974-sony-xperia-castor.dtb > zImage-dtb
Execute on the device
This greatly depends on your device, and you can figure out the correct method by looking at your deviceinfo and at the postmarketos-mkinitfs
code if necessary. But typically for fastboot based Android devices, you can patch the boot.img
file as follows.
Fasboot example
Prepare your current shell session once:
$ pmbootstrap chroot -- apk add abootimg android-tools
$ export DEVICE="$(pmbootstrap config device)"
$ export WORK="$(pmbootstrap config work)"
$ export TEMP="$PMBWORK/chroot_native/tmp/mainline/"
$ mkdir -p "$TEMP"
Then run the following everytime you would like to replace the kernel inside the boot.img
file and flash it to the device:
$ cp ~/code/linux/zImage-dtb "$TEMP"
$ cp "/tmp/postmarketOS-export/boot.img-$DEVICE" "$TEMP/boot.img"
$ pmbootstrap chroot
# abootimg -u /tmp/mainline/boot.img -k /tmp/mainline/zImage-dtb
# fastboot boot /tmp/mainline/boot.img
dmesg on your PC
Before booting the first time, run dmesg -w
on your PC. If everything went right, you should see something like the following as soon as the device boots:
[27166.665566] usb 3-9: new high-speed USB device number 16 using xhci_hcd [27166.793111] usb 3-9: New USB device found, idVendor=0525, idProduct=a4a7 [27166.793117] usb 3-9: New USB device strings: Mfr=1, Product=2, SerialNumber=0 [27166.793121] usb 3-9: Product: Gadget Serial v2.4 [27166.793125] usb 3-9: Manufacturer: Linux 4.16.0-rc5-next-20180314-dirty with ci_hdrc_msm [27166.809593] cdc_acm 3-9:2.0: ttyACM0: USB ACM device [27166.809777] usbcore: registered new interface driver cdc_acm [27166.809778] cdc_acm: USB Abstract Control Model driver for USB modems and ISDN adapters
TODO
- "Mainline section" link is just a stub
- Extend initramfs debug-shell hook to dump dmesg to RAM with userspace tools, so it can be read from /proc/lastkmesg from an Android kernel (that way we don't need serial working to get USB going, right?)
- Add steps to upstream the changes made into the kernel (there are already good tutorials out there)
- Add step to integrate the changes with the pmbootstrap source tree (might need changes in pmbootstrap architecture for that)