Mainlining FAQ
"Mainlining isn't always fun, but it's hella rewarding"
Various information collected from #postmarketOS. See The Mainline Kernel for a more howto-style page.
How to start?
- Serial cable highly recommended
- First step will be to find out how much your device is already supported by mainline
- Check if your soc is supported.. that will be basic building block
- There's really no point doing display till you have uart or USB working
Writing dmesg to RAM and reading it out after reboot
During the mainlining process you need to be able to get the dmesg output from the device. Typically this is done with a serial cable (see below). Sometimes making such a cable is relatively easy by using the audio jack, but in other cases it requires opening the device or is not possible at all. Here is a less comfortable alternative.
In Android kernels it is possible to write the output of dmesg to memory, and read it out after reboot from /proc/last_kmsg. So while you are mainlining, you can use Android's ram_console driver from your downstream kernel to dump dmesg to memory as well, then reboot to the downstream kernel and read out /proc/last_kmesg. Quote from decatf:
The Android ram_console does this. It should exist in many if not most Android kernels. The driver is registered the board/machine init. I am still using it even on kernel 4.16. On my device I can force reboot to a downstream recovery kernel and read last_kmsg from a kernel that booted nothing but the bare device tree and the ram_console driver.
- Allocate the same memory block as your downstream or recovery kernel. Do this by specifying reserved-memory in device tree.
- Initialize the ram_console driver with the appropriate memory address and size in the
machine_initfunction for your device.
- Make sure the driver is using the same ram console signature as your downstream kernel.
On more recent devices Android has transitioned over to using the mainline kernels pstore ramoops functionality. If your device is recent enough that the recovery uses pstore then you only need to add a node for the ramoops driver to device tree.
The memory allocation address of the ram_console and pstore are logged to dmesg. The memory address in following example corresponds to the samsung-p4wifi device in the steps above. To find the memory allocation address of ram_console:
$ dmesg | grep ram_console
[ 0.906530] ram_console: got buffer at 2e600000, size 100000
To find the memory allocation address of pstore ramoops:
$ dmesg | grep attached
[ 0.074293] ramoops: attached 0x100000@0x2e500000, ecc: 0/0
See also: Halium docs on dmesg
Serial cable
Can I do it without a serial cable?
- in theory, yes by dumping dmesg to RAM (see above)
- once it reliably boots though, and USB comes up properly, you can use the USB serial gadget in the kernel (see below)
How to get serial output? (serial cable)
- enable the serial device for your kernel (e.g. for OMAP: CONFIG_SERIAL_OMAP and CONFIG_SERIAL_OMAP_CONSOLE)
- add console=ttySAC0,115200 to the kernel cmdline
- it might also be ttyO0 (uppercase O for OMAP chipset)
How to get serial output over USB? (USB serial gadget)
The Linux kernel can act like a serial device, and it's possible to output what you would get on the serial port over that fake serial device directly via USB to your PC.
- set CONFIG_USB_G_SERIAL=y and enable USB as console, and add console=ttyGS0,115200 to the cmdline
Is it possible to boot kernels with a serial cable?
- No. Just flash with a normal cable and switch to the serial cable to read the log output.
- Once it reliably boots though, and USB comes up properly, you can use the USB serial gadget in the kernel
There's no device tree source (dts) in the kernel fork for my device, how do I make one?
Previous kernels used board-*.c files instead of the dts files. These are in folders such as arch/arm/mach-omap2 (example).
The idea is that you'll need to convert these to dts files. Doing so manually for each of them is a huge and error-prone task, better use one of the following techniques:
- Strip down
dtsfrom similar hardware (n900 in case of omap3?) and try to get it to boot - Add support to an existing mainline driver (e.g. mms144 and mms114 are probably pretty similar) rather than forward-porting the old driver
- Forward-port the board files to recent kernels (new kernels still understand board files for some hardware, but not for all!), then strip them down to a bare minimum you can still test (where serial or usb is still working, so you know that the device boots), and then convert that do
dts.
Finding the best approach is very hardware specific. If you know C and think you have tried some of the methods above without success, consider asking the maintainers of the chipset in the kernel. Regarding OMAP, Pavel offered that you could CC him.
See also: #1021
Consider updating your device's kernel to the latest Android stable version first (these instruction might make it easier to convert the board files to dtbs) and then going to real mainline from there.
What options are needed to get USB network?
- USB_ETH IIRC
- also check the defconfig in aports/main/linux-postmarketos-mainline
How to make OpenRC print its output to the serial port?
- just set
RC_LOGGINGto true in /etc/rc.conf - (it's the init system log like Starting etc.....[ok])
How can I compile kernel sources from a local git repo?
 |
A more modern approach is compiling kernels with envkernel.sh. |
- Clone the kernel with git (~5GB!)
- Use the following command to build the mainline kernel package with the checked out sources
$ pmbootstrap build linux-postmarketos-mainline --src=~/code/linux
How can I repackage the initramfs with custom modules?
Run a normal installation to generate the initramfs:
$ pmbootstrap install
And then writing a short shell script, that repackages the postmarketOS initramfs with the modules you compiled. After reading through it, adjust it as needed and execute it as root:
#!/bin/sh
# Config
device="oneplus-bacon"
tmpdir="/tmp/pmOS-initfs-with-mainline-modules"
work="$(pmbootstrap -q config work)"
# Create temp folder
[ -d "$tmpdir" ] && rm -rf "$tmpdir"
mkdir -p "$tmpdir"
# Extract the pmOS initramfs
cp -v "$work/chroot_rootfs_$device/boot/initramfs-"* initramfs.gz
gzip -d initramfs.gz
cpio -i < initramfs
# Remove existing modules
rm -rf initramfs lib/modules
#
# CHANGEME: you need to write the commands that copy the kernel modules from your
# kernel source folder into the initramfs here!
#
# Package the initramfs again
cd "$tmpdir"
find . -print0 | busybox cpio --quiet -o -H newc | gzip -1 > initramfs-repackaged
How to merge kernel configs?
The idea is, that we have a shared linux-postmarketos-stable package, which all mainlined devices use. Check Kernel_configuration#Automatically_merging_kernel_configuration for instructions on how to merge kernel configs somewhat efficiently with a script.
How to flash each postmarketOS partition from the rootfs to a different partition on the phone
Usually pmOS writes the whole rootfs (which has its own /boot and / partitions) to one partition on the device (more). If your mainline kernel isn't able to find the postmarketOS partition by label (even if you have enabled device mapper in the kernel config), then you could use the following workaround to flash each partition from the rootfs image to a real partition on the device.
$ pmbootstrap install
$ pmbootstrap export
$ losetup /dev/loop0 <pathtoimage>
$ dd if=/dev/loop0p1 of=boot_partition.img
$ dd if=/dev/loop0p2 of=root_partition.img
$ fastboot flash system root_partition.img
$ fastboot flash cache boot_partition.img
Display panel
How to let the bootloader initialize the panel?
opendata found a cool hack to let the bootloader initialize the display, which makes everything a lot easier. It only works with Sony Shinano and Rhine devices.
- Make sure to base of downstream 3.4 kernel
- Basically extract the dts from your downstream 3.4 kernel
- Use DTC to convert it to a dts
- Then delete everything after main header apart from display stuff (mdp, gpu) and anything in the spmi node
- Then in the spmi node delete everything apart from wled and pm8941@0 gpios
- That should be enough for bootloader to init display
- Example:
- 1. Stock downstream: https://ptpb.pw/hRmy.dts
- 2. Minimum for display init: https://ptpb.pw/f2ye.dts
- 3. mainline dts with second added into it: https://ptpb.pw/q28L.dts
- Then just add it to simple panel with video mode and video hse
Can I use the simple panel driver?
- If your device shows the boot splash with mainline, simple panel should work
- Adapt this commit to make it use your panels values (hfp hbp etc)
- now in flags add:
MIPI_DSI_MODE_VIDEO | MIPI_DSI_MODE_VIDEO_HSE
Kernel is stuck during boot without any output
- enable debug output with the kernel command line
- consider adding kprint statements to narrow down where it hangs