Porting to a new device: Difference between revisions
m Sideload |
→Alternatives to regular flashing: --no-fde when creating a recovery zip |
||
Line 313: | Line 313: | ||
For Android based devices, where a recovery system such as TWRP is installed, you can generate an Android recovery zip compatible file, and sideload it. | For Android based devices, where a recovery system such as TWRP is installed, you can generate an Android recovery zip compatible file, and sideload it. | ||
<syntaxhighlight lang="shell-session"> | <syntaxhighlight lang="shell-session"> | ||
$ pmbootstrap install --android-recovery-zip | $ pmbootstrap install --no-fde --android-recovery-zip | ||
$ pmbootstrap flasher sideload | $ pmbootstrap flasher sideload | ||
</syntaxhighlight> | </syntaxhighlight> |
Revision as of 01:21, 20 November 2017
Preparation
Please use a spare device for experimenting with postmarketOS! You won't be able to use typical phone features right now, such as making calls, writing SMS, using Bluetooth. Although everything has been tested, there is no guarantee that you won't break your device. |
- Join Matrix/IRC to get help when you're stuck
- Search the supported devices page for your device to avoid duplicate work
- We assume, that you are using Linux. If you don't have Linux installed, please set up a virtual machine with Virtual Box first.
Initialization
Run the following commands in a Linux terminal to get started.
$ git clone https://github.com/postmarketOS/pmbootstrap
$ cd pmbootstrap
$ ./pmbootstrap.py init
Type in the code name of your device to start a new port, and fill out everything else you get asked for. Check out how to find device specific information if necessary.
Target device (either an existing one, or a new one for porting). Available (31): asus-flo, asus-grouper, fairphone-fp2, ... Device [samsung-i9100]: wiki-example You are about to do a new device port for 'wiki-example'. Continue? (y/n) [y]: Device architecture (armhf/aarch64) [armhf]: Who produced the device (e.g. LG)? Manufacturer: Samsung What is the official name (e.g. Google Nexus 5)? Name: Wiki Example Does the device have a hardware keyboard? (y/n) [n]: Does the device have a sdcard or other external storage medium? (y/n) [n]: Which flash method does the device support?
Flash method
It depends on the flashing protocol, that comes with the device. For most Android devices, this is fastboot
, some Samsung models need heimdall
(which is compatible to "Odin") and some Nokia based phones work with 0xffff
. If you don't know which one it is, try to enter the bootloader on your device and check whether it says "Fastboot mode" or "Odin mode". Most of the time, that can be done by pressing the Volume Down
key when it is just booting. In case you can't figure that one out, try to search the internet - there might even be videos on YouTube on how to do that if you're lucky.
Flash method (fastboot/heimdall/0xffff) [fastboot]: Username [user]: Available user interfaces (3):
Interfaces
We recommend, that you start with weston
first, because that has been around the longest in postmarketOS and, while it is not really suitable for a phone, it works as a stable demo interface for making the graphics output work in the first place. After that one works, feel free to try out another interface (but expect, that it will not necessarily work out of the box).
* none: No graphical environment * hildon: (X11) Lightweight GTK+2 UI (optimized for single-touch touchscreens) * weston: (Wayland) Reference compositor (demo, not a phone interface) * xfce4: (X11) Lightweight GTK+2 desktop (stylus recommended) User interface [weston]: Location of the 'work' path. Multiple chroots (native, device arch, device rootfs) will be created in there. Work path [/home/user/.local/var/pmbootstrap]: How many jobs should run parallel on this machine, when compiling? Jobs [3]: Rebuild packages, when the last modified timestamp changed, even if the version did not change? This makes pmbootstrap behave more like 'make'. Timestamp based rebuilds (y/n) [n]: Additional packages that will be installed to rootfs. Specify them in a comma separated list (e.g.: vim,file) or "none" Extra packages [none]: Your host timezone: Europe/London Use this timezone instead of GMT? (y/n) [y]: WARNING: The applications in the chroots do not get updated automatically. Run 'pmbootstrap zap' to delete all chroots once a day before working with pmbootstrap! It only takes a few seconds, and all packages are cached. Done!
Shortcut
./pmbootstrap.py
is the key program for postmarketOS development, so you might want to set up pmbootstrap
as shortcut, that works from any folder:
$ sudo ln -s $PWD/pmbootstrap.py /usr/local/bin/pmbootstrap
The rest of the documentation assumes, that you have this shortcut set up. If you skipped this step, simply use ./pmbootstrap.py
whenever pmbootstrap
is mentioned.
Logging
Open a second terminal and run the following command to see detailed logging (such as compiler output). Whenever reporting, that something does not work, please attach the detailed log output from this window.
$ pmbootstrap log
Device specific package
Open aports/device/device-wiki-example/deviceinfo
in your favorite editor (replace wiki-example
) and adjust:
deviceinfo_screen_width
andheight
deviceinfo_flash_offset_
lines with this guide (only if there are any!)
Then regenerate the checksums and build your device package. Example commands:
$ nano aports/device/device-wiki-example/deviceinfo
$ pmbootstrap checksum device-wiki-example
$ pmbootstrap build device-wiki-example
See deviceinfo reference for a listing of all available variables and their meanings.
Kernel package
While the goal is to use a mainline kernel for all devices in the long run, this certainly can not be done in one step when porting to a new device. What we do in the meantime is shipping a Linux kernel fork, that is known to work with this device (e.g. from Android).
Source code
APKBUILD has nothing to do with Android's app format! |
First of all, find the source code of the known working kernel fork, then and adjust the # Source
section in the aports/device/linux-wiki-example/APKBUILD
file. For LineageOS kernels, you will only need to change _repository
and _commit
variables. Usually it makes sense to use a commit from the most recently updated branch.
# Source
_repository="(CHANGEME!)"
_commit="ffffffffffffffffffffffffffffffffffffffff"
_config="config-${_flavor}.${arch}"
source="
$pkgname-$_commit.tar.gz::https://github.com/LineageOS/${_repository}/archive/${_commit}.tar.gz
$_config
compiler-gcc6.h
01_msm-fix-perf_trace_counters.patch
02_gpu-msm-fix-gcc5-compile.patch
"
builddir="$srcdir/${_repository}-${_commit}"
Kernel version
In the repository with the kernel source code, you will find a Makefile
. Open this file and adjust the pkgver
in the kernel APKBUILD
to the information from the VERSION
, PATCHLEVEL
and SUBLEVEL
variables (e.g. 3.4.0).
Defconfig
Next you will need to find the kernel configuration, that is used to compile that kernel source for your device. Download it and save it as aports/device/linux-wiki-example/config-wiki-example.armhf
(replace wiki-example
and armhf
).
$ wget https://github.com/LineageOS/lge-kernel-mako/raw/cm-14.1/arch/arm/configs/lineageos_mako_defconfig
$ mv lineageos_mako_defconfig aports/device/linux-wiki-example/config-wiki-example.armhf
Download sources and update checksums
Use the checksum action again. It will not only generate the checksums of all source files, but also download them in case they have not been downloaded yet. If this command fails, most likely the download URL is invalid and needs to be adjusted in the APKBUILD
again. Another cause could be, that the defconfig could not be found (e.g. because it was saved with a wrong file name). The actual download URL (with all variables replaced) and the download progress are visible in the log window.
$ pmbootstrap checksum linux-wiki-example
Kernel configuration
Check what you need to adjust in the kernel config before it can be used with postmarketOS. Keep changing the options, until everything passes.
$ pmbootstrap kconfig_check
$ pmbootstrap menuconfig wiki-example
$ pmbootstrap kconfig_check
Kernel compilation
Next up is the kernel compilation, which will fail a few times before you get it working. Run it once to see where it fails. Please note, that before actually compiling the kernel, pmbootstrap will build/install the required dependencies, such as binutils
and gcc
versions for cross-compilation (the plan is to provide them in binary form at some point in the future).
$ pmbootstrap build linux-wiki-example
We most likely use a more modern version of GCC compared to what your device kernel was intended to be compiled with (which is good, we don't want outdated software). However, this means that you will probably need to patch your kernel before it compiles successfully. Don't be scared, we'll talk you through it.
Remove failed patches
You will find some example patches already in the generated kernel package. When a patch does not apply correctly, you will get something like the following in the log window (scroll up, the error is above the ^^^
line).
>>> linux-wiki-example: Unpacking /var/cache/distfiles/linux-wiki-example-ffff.tar.gz... >>> linux-wiki-example: 01_msm-fix-perf_trace_counters.patch patching file arch/arm/mach-msm/perf_trace_counters.h Hunk #1 succeeded at 158 (offset 37 lines). >>> linux-wiki-example: 02_this_patch_fails.patch patching file arch/arm/mach-msm/perf_trace_counters.h Hunk #1 FAILED at 121. 1 out of 1 hunk FAILED -- saving rejects to file arch/arm/mach-msm/perf_trace_counters.h.rej >>> ERROR: linux-wiki-example: all failed
When this happens, remove the failing patch from the source=
variable in the APKBUILD
, and delete the patch file from the same folder. Finally, correct the checksums and build again.
$ nano aports/device/linux-wiki-example/APKBUILD # remove from source
$ rm aports/device/linux-wiki-example/02_this_patch_fails.patch
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap build linux-wiki-example
In the unlikely case, that everything runs through smoothly, continue reading here.
Find the error message
In the log window, scroll up until you find the first message containing error:
or Error
(examples below). Read on for some tips on how to make this easier, if you can't find it right away.
In file included from arch/arm/mach-msm/perf_trace_counters.h:127:0, from arch/arm/mach-msm/perf_trace_counters.c:14: include/trace/define_trace.h:79:43: fatal error: ./perf_trace_counters.h: No such file or directory
drivers/built-in.o: In function `.LANCHOR1': msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock' msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock' Makefile:877: recipe for target '.tmp_vmlinux1' failed make: *** [.tmp_vmlinux1] Error 1
To get an error log, where the actual error message is not mixed with jobs running in parallel, please run the compilation again with only one job enabled. In general, this makes it slower. But don't worry, we use ccache
. Therefore everything you have already compiled once will be faster when compiled again.
$ pmbootstrap -j1 build linux-wiki-example
If you still can't find it, try it with a text editor. The log is stored in ~/.local/var/pmbootstrap/log.txt
(or in whichever work folder you have specified in the initialization step). It is even possible to clear the log file right before compilation, when you press CTRL
+ C
in the log window and use log -c
:
(old log output here)
^C
$ pmbootstrap log -c
Get a patch
From postmarketOS aports
It is highly likely, that we already ran into the same issue with another kernel. Take (parts of) the offending file name and search for it inside aports/device
.
$ grep -r 'perf_trace_counters\.c' aports/device/
aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch:In-tree compilation for arch/arm/mach-msm/perf_trace_counters.c was
aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch: from arch/arm/mach-msm/perf_trace_counters.c:14
$ grep -r 'msm_iommu_sec' aports/device/
aports/device/linux-lg-mako/02_gpu-msm-fix-gcc5-compile.patch:msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock'
aports/device/linux-lg-mako/02_gpu-msm-fix-gcc5-compile.patch:msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock'
When there is a result, copy the patch file you found to your new kernel package, add it to the source
variable in the APKBUILD
and try building again.
$ cp aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch aports/device/linux-wiki-example/
$ nano aports/device/linux-wiki-example/APKBUILD # add it to source
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap build linux-wiki-example
From elsewhere
Fire up your favorite search engine, and look at all results for the error message and variations of it. The ""
feature of most search engines is useful, so it searches for an exact string and not single words. Example queries for the error messages above:
include/trace/define_trace.h: fatal error: ./perf_trace_counters.h: "No such file or directory"
linux "./perf_trace_counters.h: No such file or directory"
android "undefined reference to `kgsl_iommu_sync_unlock'"
In most cases, this will yield a patch that you can apply to your kernel. Save what you have found as a patch file right next to the APKBUILD
of your new kernel package. Mailing list posts are usually in the format of a patch file and can be used directly, while commits or pull requests (PRs) on GitHub can be downloaded as patch when you append .patch
to the URL (e.g. commit, patch).
If what you found on the web can't be used as patch file directly (e.g. sometimes the files are in other folders), but you understood how you would need to patch the source (from the search results or simply because you already knew), please follow this guide.
Patch files can contain arbitrary text before the first line with ---
in the file. Please use this space to link to the source where you found a patch (in case it is from the Internet) and to paste the error message, that your patch fixes (so it is easier to find for other people).
After creating the patch file and adding the source URL and error message, put the file name into source
in your APKBUILD
and give it another shot.
$ nano aports/device/linux-wiki-example/03_description-here.patch
$ nano aports/device/linux-wiki-example/APKBUILD
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap build linux-wiki-example
We can't promise anything, but you shouldn't need to do this more than 2 times or so, before the kernel build finally goes through.
We are happy to help you!
If you get stuck, we are happy to help you in the chat and in the issues tracker. When asking for help, always include the APKBUILD you have written, and any custom patches you have applied, as well as the error message you have gotten from pmbootstrap log
.
Documentation
This is a wiki, so please adjust everything that isn't detailed enough or would have helped you with porting.
More importantly, if you are at this point, it is a good idea to start writing your porting progress to the Devices page, even if you didn't get very far and the kernel does not compile (it will help others to continue from your work). Simply register in the wiki (you don't even need an e-mail address) and follow this tutorial.
Installation
If you want to install to the system partition of an image file, run the following:
$ pmbootstrap install --no-fde
If you want to install to a SD card, insert it and run the following. Replace mmcblk0 with the actual device name (lsblk
is handy to find out the correct device name).
$ pmbootstrap install --no-fde --sdcard=/dev/mmcblk0
The --no-fde
disables full disk encryption. You can enable it once you have graphical output and touchscreen or physical keyboard working, so you can actually type in the password. Additional porting steps to get that working are described here.
Flashing
Before you can run any flash command, you must put your device in the flashing mode. It is usually done holding Volume Down & Power simultaneously when the device is switched off but it may vary depending on the device. For more information see the specific page for your device in the LineageOS Wiki.
If you want to install to the system partition, run the following (not for the SD card installation!):
$ pmbootstrap flasher flash_system
If you have a device, that works with fastboot, you can boot the kernel now without flashing it:
$ pmbootstrap flasher boot
Otherwise, you will need to flash the kernel to the device boot partition:
$ pmbootstrap flasher flash_kernel
Alternatives to regular flashing
If the flashing method does not work, it is also possible to export all generated image files to a specific directory (with symlinks), so you can flash it manually with your host Linux system (or even on Windows with proprietary flashers such as Odin, if this is the only way it works for you):
$ pmbootstrap flasher export
$ pmbootstrap flasher export --odin
For Android based devices, where a recovery system such as TWRP is installed, you can generate an Android recovery zip compatible file, and sideload it.
$ pmbootstrap install --no-fde --android-recovery-zip
$ pmbootstrap flasher sideload
When all else fails, you might have success with installing via netcat or booting via NFS.
USB Network
As soon as usb networking works, it's time to upstream your work into the official pmbootstrap repository. Please make a pull request (PR) or send us a link to your git repository in the chat if you don't want to use GitHub. |
If you are lucky, your screen may give some clues that you are booted into pmOS. If not, do not get discouraged, the graphics on your device may not yet be setup correctly. The next steps are to see if you can connect to the device through SSH over USB (or if that fails, via telnet). See the Boot process and Inspecting the initramfs pages for more details.
Display
To make the display work in the first place, read Troubleshooting:display and Tuning sysfs. Touch screen in weston
is described here.
osk-sdl support
If the display and the touchscreen interface are working, then it's time to add support for osk-sdl
, so that you are able to use full disk encryption on the root filesystem. Instructions for adding support can be found here: Porting osk-sdl to New Devices
Advanced topics
- WiFi (works on many devices now, but the guide needs some work, please expand it and look into the
aports
folder how it works for other devices and ask in the chat) - Cellular modems still need lots of work, see the software stack we have proposed and started to implement in #598.
- User-Interfaces
- Mainline kernel