Difference between revisions of "Porting to a new device"

From postmarketOS
Jump to: navigation, search
(Advanced topics)
(Source code: mention checking _flavor)
 
(55 intermediate revisions by 16 users not shown)
Line 4: Line 4:
 
* Join [[Matrix and IRC|Matrix/IRC]] to get help when you're stuck
 
* Join [[Matrix and IRC|Matrix/IRC]] to get help when you're stuck
 
* Search the [[devices]] page for your device to avoid duplicate work (read potentially existing device pages and check for links to existing branches).
 
* Search the [[devices]] page for your device to avoid duplicate work (read potentially existing device pages and check for links to existing branches).
 +
** Note that there's a [[Generic x64 uefi]] device, which you might be able to use if your target device has a x86_64 CPU.
 +
 +
=== Requirements ===
 +
==== Host PC ====
 
* We assume, that you are using Linux. If you don't have Linux installed, please [http://www.brianlinkletter.com/installing-debian-linux-in-a-virtualbox-virtual-machine/ set up a virtual machine with VirtualBox] first.
 
* We assume, that you are using Linux. If you don't have Linux installed, please [http://www.brianlinkletter.com/installing-debian-linux-in-a-virtualbox-virtual-machine/ set up a virtual machine with VirtualBox] first.
 +
* The host operating system needs to run Linux 3.17 or newer (first kernel with getrandom syscall)
 +
* A few gigabytes of free space
 +
 +
==== Target Device ====
 +
If your target device has an ARM processor, it needs to have an [https://en.wikipedia.org/wiki/Floating-point_unit FPU].
 +
 +
Most phones fulfill this requirement: in the first > 80 ports we have had, only twice we had phones without an FPU ({{issue|1203}}, {{issue|1592}}). These are from 2010 and 2011, so it is [https://unix.stackexchange.com/a/185070 worth checking] if your device is from that time. Otherwise just go ahead.
 +
If your device have ARMv7 CPU it should have FPU, with ARMv6 CPUs it's a 50/50 chance.
  
 
=== Initialization ===
 
=== Initialization ===
Run the following commands in a Linux terminal to get started.
 
  
 +
[[installing pmbootstrap|Install pmbootstrap]], our lightweight postmarketOS development and installation tool. Then run the following:
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ git clone https://github.com/postmarketOS/pmbootstrap
+
$ pmbootstrap init
$ cd pmbootstrap
+
Location of the 'work' path. Multiple chroots (native, device arch, device rootfs) will be created in there.
$ ./pmbootstrap.py init
+
Work path [/home/user/.local/var/pmbootstrap]:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 19: Line 31:
 
<pre>
 
<pre>
 
Target device (either an existing one, or a new one for porting).
 
Target device (either an existing one, or a new one for porting).
Available (31): asus-flo, asus-grouper, fairphone-fp2, ...
+
Available (60): asus-flo, asus-grouper, fairphone-fp2, ...
 
Device [samsung-i9100]: wiki-example
 
Device [samsung-i9100]: wiki-example
 
You are about to do a new device port for 'wiki-example'.
 
You are about to do a new device port for 'wiki-example'.
 
Continue? (y/n) [y]:
 
Continue? (y/n) [y]:
Device architecture (armhf/aarch64) [armhf]:
+
Device architecture (armhf/armv7/aarch64/x86_64/x86) [armhf]: armv7
 
Who produced the device (e.g. LG)?
 
Who produced the device (e.g. LG)?
 
Manufacturer: Samsung
 
Manufacturer: Samsung
Line 34: Line 46:
 
==== Flash method ====
 
==== Flash method ====
 
It depends on the flashing protocol, that comes with the device. For most Android devices, this is <code>fastboot</code>, some Samsung models need <code>heimdall</code> (which is compatible to "Odin") and some Nokia based phones work with <code>0xffff</code>. If you don't know which one it is, try to enter the bootloader on your device and check whether it says "Fastboot mode" or "Odin mode". Most of the time, that can be done by pressing the <code>Volume Down</code> key when it is just booting. In case you can't figure that one out, try to search the internet - there might even be videos on YouTube on how to do that if you're lucky.
 
It depends on the flashing protocol, that comes with the device. For most Android devices, this is <code>fastboot</code>, some Samsung models need <code>heimdall</code> (which is compatible to "Odin") and some Nokia based phones work with <code>0xffff</code>. If you don't know which one it is, try to enter the bootloader on your device and check whether it says "Fastboot mode" or "Odin mode". Most of the time, that can be done by pressing the <code>Volume Down</code> key when it is just booting. In case you can't figure that one out, try to search the internet - there might even be videos on YouTube on how to do that if you're lucky.
 +
 +
For more information see the Wiki page about [[Deviceinfo flash methods]].
  
 
<pre>
 
<pre>
Line 63: Line 77:
 
* xfce4: (X11) Lightweight GTK+2 desktop (stylus recommended)
 
* xfce4: (X11) Lightweight GTK+2 desktop (stylus recommended)
 
User interface [weston]:
 
User interface [weston]:
Location of the 'work' path. Multiple chroots (native, device arch, device rootfs) will be created in there.
+
Build options: Parallel jobs: 3, ccache per arch: 5G
Work path [/home/user/.local/var/pmbootstrap]:
+
Change them? (y/n) [n]:
How many jobs should run parallel on this machine, when compiling?
 
Jobs [3]:
 
Rebuild packages, when the last modified timestamp changed, even if the version did not change? This makes pmbootstrap behave more like 'make'.
 
Timestamp based rebuilds (y/n) [n]:
 
 
Additional packages that will be installed to rootfs. Specify them in a comma separated list (e.g.: vim,file) or "none"
 
Additional packages that will be installed to rootfs. Specify them in a comma separated list (e.g.: vim,file) or "none"
 
Extra packages [none]:
 
Extra packages [none]:
Line 78: Line 88:
 
Done!
 
Done!
 
</pre>
 
</pre>
 
==== Shortcut ====
 
<code>./pmbootstrap.py</code> is the key program for postmarketOS development, so you might want to set up <code>pmbootstrap</code> as shortcut, that works from any folder:
 
 
<syntaxhighlight lang="shell-session">
 
$ sudo ln -s $PWD/pmbootstrap.py /usr/local/bin/pmbootstrap
 
</syntaxhighlight>
 
 
The rest of the documentation assumes, that you have this shortcut set up. If you skipped this step, simply use <code>./pmbootstrap.py</code> whenever <code>pmbootstrap</code> is mentioned.
 
  
 
==== Logging ====
 
==== Logging ====
Line 94: Line 95:
 
$ pmbootstrap log
 
$ pmbootstrap log
 
</syntaxhighlight>
 
</syntaxhighlight>
 
=== Device specific package ===
 
Open <code>aports/device/device-wiki-example/deviceinfo</code> in your favorite editor (replace <code>wiki-example</code>) and adjust:
 
 
* <code>deviceinfo_screen_width</code> and <code>height</code>
 
* <code>deviceinfo_flash_offset_</code> lines ('''only if you skipped the bootimg analyzing'''):
 
** [[#Flash_method|Get a boot.img]]
 
** Run <code>pmbootstrap bootimg_analyze path/to/boot.img</code>
 
 
Then regenerate the checksums and build your device package (<code>-i</code> ignores the dependencies, as we don't have a working kernel package yet):
 
 
<syntaxhighlight lang="shell-session">
 
$ nano aports/device/device-wiki-example/deviceinfo
 
$ pmbootstrap checksum device-wiki-example
 
$ pmbootstrap build -i device-wiki-example
 
</syntaxhighlight>
 
 
See [[deviceinfo reference]] for a listing of all available variables and their meanings. The [[device specific package]] article explains advanced customization of the <code>APKBUILD</code>.
 
  
 
=== Kernel package ===
 
=== Kernel package ===
While the goal is to use a mainline kernel for all devices in the long run, this certainly can not be done in one step when porting to a new device. What we do in the meantime is shipping a Linux kernel fork, that is known to work with this device (e.g. from Android).
+
While the goal is to use a [https://wiki.postmarketos.org/wiki/The_Mainline_Kernel mainline kernel] for all devices in the long run, this certainly can not be done in one step when porting to a new device. What we do in the meantime is shipping a Linux kernel fork, that is known to work with this device (e.g. from Android). The steps below are the straight forward changes you need to compile your kernel. For details see the [[vendor kernel specific package]] article.
  
 
==== Source code ====
 
==== Source code ====
 
{{note|<code>APKBUILD</code> has [[Glossary#apk|nothing to do]] with Android's app format!}}
 
{{note|<code>APKBUILD</code> has [[Glossary#apk|nothing to do]] with Android's app format!}}
First of all, [[How-to-find-device-specific-information#LineageOS_kernel_source_repository|find the source code]] of the known working kernel fork, then and adjust the <code># Source</code> section in the <code>aports/device/linux-wiki-example/APKBUILD</code> file. For LineageOS kernels, you will only need to change <code>_repository</code> and <code>_commit</code> variables. Usually it makes sense to use a commit from the most recently updated branch.
+
First of all, [[How-to-find-device-specific-information#LineageOS_kernel_source_repository|find the source code]] of the known working kernel fork (lowest known working version is 3.x, 2.6 doesn't compile with GCC6, may compile with gcc4, not yet tested), then and adjust the <code># Source</code> section in the <code>aports/device/linux-wiki-example/APKBUILD</code> file. In case you have ''not'' cloned <code>pmbootstrap</code> from git, the aports folder will be in <code>cache_git/pmaports</code> inside your work folder (default: <code>~/.local/var/pmbootstrap/cache_git/pmaports</code>). For LineageOS kernels, you will only need to change <code>_repository</code> and <code>_commit</code> variables. Usually it makes sense to use a commit from the most recently updated branch. For example, if the kernel repo is at <code>https://github.com/LineageOS/android_kernel_motorola_ghost.git</code>, enter: <code>_repository="android_kernel_motorola_ghost"</code>.
  
 
<syntaxhighlight lang="bash">
 
<syntaxhighlight lang="bash">
 +
_flavor="wiki-example" # Be sure it's in form of <vendor-codename>, e.g. motorola-ghost
 +
 
# Source
 
# Source
 
_repository="(CHANGEME!)"
 
_repository="(CHANGEME!)"
Line 128: Line 113:
 
     $pkgname-$_commit.tar.gz::https://github.com/LineageOS/${_repository}/archive/${_commit}.tar.gz
 
     $pkgname-$_commit.tar.gz::https://github.com/LineageOS/${_repository}/archive/${_commit}.tar.gz
 
     $_config
 
     $_config
     compiler-gcc6.h
+
     gcc7-give-up-on-ilog2-const-optimizations.patch
    01_msm-fix-perf_trace_counters.patch
+
     gcc8-fix-put-user.patch
     02_gpu-msm-fix-gcc5-compile.patch
 
 
"
 
"
 
builddir="$srcdir/${_repository}-${_commit}"
 
builddir="$srcdir/${_repository}-${_commit}"
Line 147: Line 131:
  
 
==== Download sources and update checksums ====
 
==== Download sources and update checksums ====
Use the checksum action again. It will not only generate the checksums of all source files, but also download them in case they have not been downloaded yet. If this command fails, most likely the download URL is invalid and needs to be adjusted in the <code>APKBUILD</code> again. Another cause could be, that the defconfig could not be found (e.g. because it was saved with a wrong file name). The actual download URL (with all variables replaced) and the download progress are visible in the log window.
+
Use the checksum command. It will not only generate the checksums of all source files, but also download them in case they have not been downloaded yet. If this command fails, most likely the download URL is invalid and needs to be adjusted in the <code>APKBUILD</code> again. Another cause could be, that the defconfig could not be found (e.g. because it was saved with a wrong file name). The actual download URL (with all variables replaced) and the download progress are visible in the log window.
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
Line 154: Line 138:
  
 
==== Kernel configuration ====
 
==== Kernel configuration ====
The magic command, which lets you change the kernel configuration is <code>pmbootstrap menuconfig</code>. Don't try to manually change the defconfig file, because only by running menuconfig, the dependencies will get resolved properly. <code>menuconfig</code> always runs on the kernel with all patches applied, so we'll run it once to see if we need to remove patch files first.
+
{{note|If <code>pmbootstrap</code> throws an error here, you need to [[#Remove failed patches|remove the example patch]]!}}
 +
The magic command, which lets you change the kernel configuration is <code>pmbootstrap kconfig edit linux-wiki-example</code>. Don't try to manually change the defconfig file, because only by running menuconfig, the dependencies will get resolved properly. <code>menuconfig</code> always runs on the kernel with all patches applied, so we'll run it once to see if we need to remove patch files first.
  
 
===== Remove failed patches =====
 
===== Remove failed patches =====
Line 160: Line 145:
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap menuconfig linux-wiki-example
+
$ pmbootstrap kconfig edit linux-wiki-example
 
</syntaxhighlight>
 
</syntaxhighlight>
 
<pre>
 
<pre>
Line 174: Line 159:
 
</pre>
 
</pre>
  
When this happens, remove the failing patch from the <code>source=</code> variable in the <code>APKBUILD</code>, and delete the patch file from the same folder. Finally, correct the checksums and try menuconfig again.
+
When this happens, remove the failing patch from the <code>source=</code> variable in the <code>APKBUILD</code>, and delete the patch file from the same folder. Finally, correct the checksums and try kconfig edit again.
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
Line 180: Line 165:
 
$ rm aports/device/linux-wiki-example/02_this_patch_fails.patch
 
$ rm aports/device/linux-wiki-example/02_this_patch_fails.patch
 
$ pmbootstrap checksum linux-wiki-example
 
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap menuconfig linux-wiki-example
+
$ pmbootstrap kconfig edit linux-wiki-example
 
</syntaxhighlight>
 
</syntaxhighlight>
  
===== Make kconfig_check happy =====
+
===== Make kconfig check happy =====
 +
{{note|Consider using <code>kconfig edit -x</code> or <code>-g</code> to get a GUI instead of the terminal interface. Details: <code>pmbootstrap kconfig edit -h</code>}}
 
Check what you need to adjust in the kernel config before it can be used with postmarketOS. Keep [[Kernel_configuration|changing the options]], until everything passes.
 
Check what you need to adjust in the kernel config before it can be used with postmarketOS. Keep [[Kernel_configuration|changing the options]], until everything passes.
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap kconfig_check
+
$ pmbootstrap kconfig check
$ pmbootstrap menuconfig wiki-example
+
$ pmbootstrap kconfig edit wiki-example
$ pmbootstrap kconfig_check
+
$ pmbootstrap kconfig check
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 231: Line 217:
 
$ pmbootstrap log -c
 
$ pmbootstrap log -c
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
Another trick to find the error message real quick, is to open <code>log.txt</code> with <code>less</code> and use the follow mode (press <code>F</code> while in <code>less</code> or open less with the <code>+F</code> option: <code>less +F log.txt</code>). This causes <code>less</code> to behave like <code>tail -f</code>. When an error occurs, you can exit the follow mode by pressing <code>CTRL-C</code> and use <code>?</code> to reverse search the output (for example: <code>?error</code>). Press <code>F</code> to re-enable follow mode.
  
 
==== Get a patch ====
 
==== Get a patch ====
Line 276: Line 264:
  
 
We can't promise anything, but you shouldn't need to do this more than 2 times or so, before the kernel build finally goes through.
 
We can't promise anything, but you shouldn't need to do this more than 2 times or so, before the kernel build finally goes through.
 +
 +
===== Using GCC6 =====
 +
If you can't make your kernel compile or boot with the latest GCC from Alpine, and you are pretty sure that the issues are specific to the GCC version, you can also [[Vendor_kernel_specific_package#Use_GCC_6|use GCC6 instead]].
 +
 +
===== Using GCC4 =====
 +
As GCC4 was recently introduced in postmarketOS repos, you can follow the procedure for GCC6 above and replace all gcc6 occurrences with gcc4 and add <nowiki>gcc4 (armv7/armhf/...)-gcc4</nowiki> to your list of dependencies.
 +
 +
=== Device specific package ===
 +
Open the <code>deviceinfo</code> file in an editor (replace <code>wiki-example</code>).
 +
 +
<syntaxhighlight lang="shell-session">
 +
$ nano aports/device/device-wiki-example/deviceinfo
 +
</syntaxhighlight>
 +
 +
Now adjust:
 +
 +
* <code>deviceinfo_screen_width</code>
 +
* <code>deviceinfo_screen_height</code>
 +
 +
In case you were asked for <code>boot.img</code> analyzing, and skipped that section during [[#Flash_method|init]]: You can run <code>pmbootstrap bootimg_analyze path/to/boot.img</code> to get the right values for the <code>deviceinfo_flash_offset_</code> lines now. Filling out manually also works, but that is error-prone and therefore not recommended.
 +
 +
See [[deviceinfo reference]] for a listing of all available variables and their meanings. The [[device specific package]] article explains advanced customization of the <code>APKBUILD</code>.
  
 
=== We are happy to help you! ===
 
=== We are happy to help you! ===
Line 286: Line 296:
  
 
=== Installation ===
 
=== Installation ===
{{note|If you get a "no space left on device" error, please report in {{github|928}}.}}
+
{{note|If you get a "no space left on device" error, please report in {{issue|928}}.}}
 +
 
 +
Create checksums and build the device package once to make sure it doesn't contain errors at this point (we didn't do that earlier, because <code>pmbootstrap</code> would have complained about the invalid <code>pkgver</code> of the kernel package):
 +
<syntaxhighlight lang="shell-session">
 +
$ pmbootstrap checksum device-wiki-example
 +
$ pmbootstrap build device-wiki-example
 +
</syntaxhighlight>
 +
 
 
To install to the system partition of an image file, run the following:
 
To install to the system partition of an image file, run the following:
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap install --no-fde
+
$ pmbootstrap install
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 296: Line 313:
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap install --no-fde --sdcard=/dev/mmcblk0
+
$ pmbootstrap install --sdcard=/dev/mmcblk0
 
</syntaxhighlight>
 
</syntaxhighlight>
 
The <code>--no-fde</code> disables '''f'''ull '''d'''isk '''e'''ncryption. You can enable it once you have graphical output and touchscreen or physical keyboard working, so you can actually type in the password. Additional porting steps to get that working are described [[Osk-sdl#Porting_to_New_Devices|here]].
 
  
 
=== Flashing ===
 
=== Flashing ===
Before you can run any flash command, you must put your device in the flashing mode. It is usually done holding Volume Down &amp; Power simultaneously when the device is switched off but it may vary depending on the device. For more information see the specific page for your device in the [https://wiki.lineageos.org/devices LineageOS Wiki].
+
Before you can run any flash command, you must put your device in the flashing/bootloader mode. It is usually done holding Volume Down &amp; Power simultaneously when the device is switched off but it may vary depending on the device. For more information see the specific page for your device in the [https://wiki.lineageos.org/devices LineageOS Wiki].
  
 
If you want to install to the system partition, run the following (not for the SD card installation!):
 
If you want to install to the system partition, run the following (not for the SD card installation!):
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap flasher flash_system
+
$ pmbootstrap flasher flash_rootfs
 
</syntaxhighlight>
 
</syntaxhighlight>
  
In case your system partition is too small for the generated image (e.g. because you chose Plasma Mobile as UI), it is possible to flash to another partition as well. Just be sure to erase any previous installations of postmarketOS in other partitions, because the init script will start with the first one that it encounters. (To delete a previous version either use <code>dd</code> or simply install the known working OS, e.g. Android, on it). Using multiple partitions with LVM is planned ({{github|60}}).
+
In case your system partition is too small for the generated image (e.g. because you chose Plasma Mobile as UI), it is possible to flash to another partition as well. Just be sure to erase any previous installations of postmarketOS in other partitions, because the init script will start with the first one that it encounters. (To delete a previous version either use <code>dd</code> or simply install the known working OS, e.g. Android, on it). Using multiple partitions with LVM is planned ({{issue|60}}).
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ pmbootstrap flasher flash_system --partition userdata
+
$ pmbootstrap flasher flash_rootfs --partition userdata
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 335: Line 350:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
For Android based devices, where a recovery system such as TWRP is installed, you can generate an Android recovery zip compatible file, and sideload it.
+
For Android based devices, you can do a [[Android_Recovery_Zip_Installation|recovery zip installation]].
<syntaxhighlight lang="shell-session">
 
$ pmbootstrap install --no-fde --android-recovery-zip
 
$ pmbootstrap flasher sideload
 
</syntaxhighlight>
 
  
 
When all else fails, you might have success with installing via [https://github.com/postmarketOS/pmbootstrap/issues/456 netcat] or booting via [https://github.com/postmarketOS/pmbootstrap/pull/547 NFS].
 
When all else fails, you might have success with installing via [https://github.com/postmarketOS/pmbootstrap/issues/456 netcat] or booting via [https://github.com/postmarketOS/pmbootstrap/pull/547 NFS].
  
 
=== USB Network ===
 
=== USB Network ===
{{note|As soon as usb networking works, it's time to upstream your work into the official pmbootstrap repository. Please [https://help.github.com/articles/about-pull-requests/ make a pull request] (PR) or send us a link to your git repository in the chat if you don't want to use GitHub.}}
+
If you are lucky, your screen may give some clues that you are booted into pmOS. If not, do not get discouraged, the graphics on your device may not yet be setup correctly. The next steps are to see if you can connect to the device through [[USB Network|SSH over USB]] (or if that fails, via telnet). See the [[Boot process]] and [[Inspecting the initramfs]] pages for more details.
If you are lucky, your screen may give some clues that you are booted into pmOS. If not, do not get discouraged, the graphics on your device may not yet be setup correctly. The next steps are to see if you can connect to the device through SSH over USB (or if that fails, via telnet). See the [[Boot process]] and [[Inspecting the initramfs]] pages for more details.
 
  
It is quite easy to give your phone [[USB_Internet|Internet access via USB]], when your PC is connected to the Internet.
+
It is quite easy to give your phone [[USB Internet|Internet access via USB]], when your PC is connected to the Internet.
 +
 
 +
In case it looks like your device did not even boot, have a look at [[Troubleshooting:boot]].
 +
 
 +
=== Upstreaming ===
 +
 
 +
As soon as USB networking is functional, or you have another indication that booting the device works successfully, it is time to upstream your work into the official [https://gitlab.com/postmarketOS/pmaports/ pmaports] repository. This way other people can benefit from your work, and build upon it.
 +
 
 +
See the [[git workflow]] article for detailed instructions.
  
 
=== Display ===
 
=== Display ===
Line 354: Line 372:
 
After you have it working, please take some [[Making_good_photos|nice photos]] and add them to your device's wiki page. If you have a Reddit account, consider posting them in [https://www.reddit.com/r/postmarketOS/ /r/postmarketOS] as well.
 
After you have it working, please take some [[Making_good_photos|nice photos]] and add them to your device's wiki page. If you have a Reddit account, consider posting them in [https://www.reddit.com/r/postmarketOS/ /r/postmarketOS] as well.
  
=== osk-sdl support ===
+
=== Changing your installation ===
 +
==== Install packages over SSH ====
 +
Get Internet access via [[USB Internet|USB]] (easier) or [[Wifi]], then use the package manager to install new packages:
 +
<syntaxhighlight lang="shell-session">
 +
$ apk add hello-world
 +
</syntaxhighlight>
 +
 
 +
You can also [[Installing_packages_on_a_running_phone|host packages you have built yourself]] from your PC. Just make sure to increase the package version when you build your own version of a package, that already exists. Otherwise the package manager can not know that your package is the newest one and should be installed. Check the output of <code>apk</code> to see whether it is really installing the package or not.
 +
 
 +
==== Building a new system image ====
 +
Whenever you rebuild a package (or pick a new one to be installed by default in <code>pmbootstrap init</code>), you need to make sure that the package is actually installed in the chroot from which the system image gets generated. Achieve that by running <code>pmbootstrap install</code> again, it should update all outdated packages. Pay attention to the output of <code>pmbootstrap log</code> to see if it actually has been updated.
 +
If something did not work as expected, you can always <code>pmbootstrap zap</code> your chroots to start over with a clean installation.
 +
 
 +
==== Kernel ====
 +
A simple way to recompile your kernel (e.g. because you want to change your kernel config) and flash it to your device is running the following commands:
 +
<syntaxhighlight lang="shell-session">
 +
$ pmbootstrap build --force linux-wiki-example
 +
$ pmbootstrap install
 +
$ pmbootstrap flasher flash_kernel
 +
</syntaxhighlight>
 +
 
 +
The last flash_kernel step is only needed if your device has the kernel stored on an extra boot partition.
 +
Android/fastboot compatible devices have such a partition, Nokia devices (N900 etc.) don't. If your device does ''not'' have a boot partition, update your system partition or SD card.
 +
 
 +
After you've gotten more familiar with postmarketOS, you could try out the <code>postmarketos-update-kernel</code> as well. It allows you to flash a new kernel to the device's <code>boot</code> partition ([[Partition-layout#Android_internal_storage|not to be confused]] with postmarketOS' own boot subpartition <code>systemp1</code>).
 +
 
 +
=== Full disk encryption: osk-sdl support ===
 +
{{note|Our charging application <code>charging-sdl</code> uses the same software stack as <code>osk-sdl</code>, so following the instructions should make both components work.}}
 
If the display and the touchscreen interface are working, then it's time to add support for <code>osk-sdl</code>, so that you are able to use full disk encryption on the root filesystem. Instructions for adding support can be found here: [[Osk-sdl#Porting_to_New_Devices|Porting osk-sdl to New Devices]]
 
If the display and the touchscreen interface are working, then it's time to add support for <code>osk-sdl</code>, so that you are able to use full disk encryption on the root filesystem. Instructions for adding support can be found here: [[Osk-sdl#Porting_to_New_Devices|Porting osk-sdl to New Devices]]
  
 
=== Advanced topics ===
 
=== Advanced topics ===
* [[WiFi]] (works on many devices now, but the guide needs some work, please expand it and look into the <code>aports</code> folder how it works for other devices and ask in the chat)
+
{{note|Out of ideas what to do next with postmarketOS? Check out the [https://postmarketos.org/blog/2018/06/09/one-year/#get-involved get involved] section from the last big blog post!}}
* Cellular modems still need lots of work, see the software stack we have proposed and started to implement in {{github|598}}.
 
 
* [[User-Interfaces]]
 
* [[User-Interfaces]]
 +
* [[WiFi]]
 +
* [[Audio]]
 +
* [[Modem]]
 
* [[Mainline kernel]]
 
* [[Mainline kernel]]
 
* [[Existing Alpine installation]]
 
* [[Existing Alpine installation]]
 +
* [[Hybris]]
 +
 +
[[Category:Guide]]

Latest revision as of 11:48, 31 August 2019

Preparation

Please use a spare device for experimenting with postmarketOS! You won't be able to use typical phone features right now, such as making calls, writing SMS, using Bluetooth. Although everything has been tested, there is no guarantee that you won't break your device.
  • Join Matrix/IRC to get help when you're stuck
  • Search the devices page for your device to avoid duplicate work (read potentially existing device pages and check for links to existing branches).
    • Note that there's a Generic x64 uefi device, which you might be able to use if your target device has a x86_64 CPU.

Requirements

Host PC

  • We assume, that you are using Linux. If you don't have Linux installed, please set up a virtual machine with VirtualBox first.
  • The host operating system needs to run Linux 3.17 or newer (first kernel with getrandom syscall)
  • A few gigabytes of free space

Target Device

If your target device has an ARM processor, it needs to have an FPU.

Most phones fulfill this requirement: in the first > 80 ports we have had, only twice we had phones without an FPU (#1203, #1592). These are from 2010 and 2011, so it is worth checking if your device is from that time. Otherwise just go ahead. If your device have ARMv7 CPU it should have FPU, with ARMv6 CPUs it's a 50/50 chance.

Initialization

Install pmbootstrap, our lightweight postmarketOS development and installation tool. Then run the following:

$ pmbootstrap init
Location of the 'work' path. Multiple chroots (native, device arch, device rootfs) will be created in there.
Work path [/home/user/.local/var/pmbootstrap]:

Type in the code name of your device to start a new port, and fill out everything else you get asked for. Check out how to find device specific information if necessary.

Target device (either an existing one, or a new one for porting).
Available (60): asus-flo, asus-grouper, fairphone-fp2, ...
Device [samsung-i9100]: wiki-example
You are about to do a new device port for 'wiki-example'.
Continue? (y/n) [y]:
Device architecture (armhf/armv7/aarch64/x86_64/x86) [armhf]: armv7
Who produced the device (e.g. LG)?
Manufacturer: Samsung
What is the official name (e.g. Google Nexus 5)?
Name: Wiki Example
Does the device have a hardware keyboard? (y/n) [n]:
Does the device have a sdcard or other external storage medium? (y/n) [n]:

Flash method

It depends on the flashing protocol, that comes with the device. For most Android devices, this is fastboot, some Samsung models need heimdall (which is compatible to "Odin") and some Nokia based phones work with 0xffff. If you don't know which one it is, try to enter the bootloader on your device and check whether it says "Fastboot mode" or "Odin mode". Most of the time, that can be done by pressing the Volume Down key when it is just booting. In case you can't figure that one out, try to search the internet - there might even be videos on YouTube on how to do that if you're lucky.

For more information see the Wiki page about Deviceinfo flash methods.

Which flash method does the device support?
Flash method (fastboot/heimdall/0xffff) [fastboot]:

Depending on your selection, pmbootstrap may ask you to analyze a boot.img file to get offset values required for the flash method. Get the right one for your device from TWRP (twrp-*.img), LineageOS (boot.img inside the zip archive) or by extracting it from within a running Android. If you can't find any, you can skip this step for now by just pressing the return key.

You can analyze a known working boot.img file to automatically fill out the flasher information for your deviceinfo file. Either specify the path to an image or press return to skip this step (you can do it later with 'pmbootstrap bootimg_analyze').
Path: /home/user/Downloads/twrp-3.2.0-0-mako.img
NOTE: You will be prompted for your sudo password, so we can set up a chroot to extract and analyze your boot.img file
(native) install alpine-base
(native) install file unpackbootimg

Interfaces

We recommend, that you start with weston first, because that has been around the longest in postmarketOS and, while it is not really suitable for a phone, it works as a stable demo interface for making the graphics output work in the first place. After that one works, feel free to try out another interface (but expect, that it will not necessarily work out of the box).

Username [user]:
Available user interfaces (5):
* none: No graphical environment
* hildon: (X11) Lightweight GTK+2 UI (optimized for single-touch touchscreens)
* luna: (Wayland) webOS UI, ported from the LuneOS project (Not working yet)
* plasma-mobile: (Wayland) Mobile variant of KDE Plasma, optimized for touchscreen
* weston: (Wayland) Reference compositor (demo, not a phone interface)
* xfce4: (X11) Lightweight GTK+2 desktop (stylus recommended)
User interface [weston]:
Build options: Parallel jobs: 3, ccache per arch: 5G
Change them? (y/n) [n]:
Additional packages that will be installed to rootfs. Specify them in a comma separated list (e.g.: vim,file) or "none"
Extra packages [none]:
Your host timezone: Europe/London
Use this timezone instead of GMT? (y/n) [y]:
WARNING: The applications in the chroots do not get updated automatically.
Run 'pmbootstrap zap' to delete all chroots once a day before working with pmbootstrap!
It only takes a few seconds, and all packages are cached.
Done!

Logging

Open a second terminal and run the following command to see detailed logging (such as compiler output). Whenever reporting, that something does not work, please attach the detailed log output from this window.

$ pmbootstrap log

Kernel package

While the goal is to use a mainline kernel for all devices in the long run, this certainly can not be done in one step when porting to a new device. What we do in the meantime is shipping a Linux kernel fork, that is known to work with this device (e.g. from Android). The steps below are the straight forward changes you need to compile your kernel. For details see the vendor kernel specific package article.

Source code

APKBUILD has nothing to do with Android's app format!

First of all, find the source code of the known working kernel fork (lowest known working version is 3.x, 2.6 doesn't compile with GCC6, may compile with gcc4, not yet tested), then and adjust the # Source section in the aports/device/linux-wiki-example/APKBUILD file. In case you have not cloned pmbootstrap from git, the aports folder will be in cache_git/pmaports inside your work folder (default: ~/.local/var/pmbootstrap/cache_git/pmaports). For LineageOS kernels, you will only need to change _repository and _commit variables. Usually it makes sense to use a commit from the most recently updated branch. For example, if the kernel repo is at https://github.com/LineageOS/android_kernel_motorola_ghost.git, enter: _repository="android_kernel_motorola_ghost".

_flavor="wiki-example" # Be sure it's in form of <vendor-codename>, e.g. motorola-ghost

# Source
_repository="(CHANGEME!)"
_commit="ffffffffffffffffffffffffffffffffffffffff"
_config="config-${_flavor}.${arch}"
source="
    $pkgname-$_commit.tar.gz::https://github.com/LineageOS/${_repository}/archive/${_commit}.tar.gz
    $_config
    gcc7-give-up-on-ilog2-const-optimizations.patch
    gcc8-fix-put-user.patch
"
builddir="$srcdir/${_repository}-${_commit}"

Kernel version

In the repository with the kernel source code, you will find a Makefile. Open this file and adjust the pkgver in the kernel APKBUILD to the information from the VERSION, PATCHLEVEL and SUBLEVEL variables (e.g. 3.4.0).

Defconfig

Next you will need to find the kernel configuration, that is used to compile that kernel source for your device. Download it and save it as aports/device/linux-wiki-example/config-wiki-example.armhf (replace wiki-example and armhf).

$ wget https://github.com/LineageOS/lge-kernel-mako/raw/cm-14.1/arch/arm/configs/lineageos_mako_defconfig
$ mv lineageos_mako_defconfig aports/device/linux-wiki-example/config-wiki-example.armhf

Download sources and update checksums

Use the checksum command. It will not only generate the checksums of all source files, but also download them in case they have not been downloaded yet. If this command fails, most likely the download URL is invalid and needs to be adjusted in the APKBUILD again. Another cause could be, that the defconfig could not be found (e.g. because it was saved with a wrong file name). The actual download URL (with all variables replaced) and the download progress are visible in the log window.

$ pmbootstrap checksum linux-wiki-example

Kernel configuration

If pmbootstrap throws an error here, you need to remove the example patch!

The magic command, which lets you change the kernel configuration is pmbootstrap kconfig edit linux-wiki-example. Don't try to manually change the defconfig file, because only by running menuconfig, the dependencies will get resolved properly. menuconfig always runs on the kernel with all patches applied, so we'll run it once to see if we need to remove patch files first.

Remove failed patches

You will find some example patches already in the generated kernel package. When a patch does not apply correctly, you will get something like the following in the log window (scroll up, the error is above the ^^^ line).

$ pmbootstrap kconfig edit linux-wiki-example
>>> linux-wiki-example: Unpacking /var/cache/distfiles/linux-wiki-example-ffff.tar.gz...
>>> linux-wiki-example: 01_msm-fix-perf_trace_counters.patch
patching file arch/arm/mach-msm/perf_trace_counters.h
Hunk #1 succeeded at 158 (offset 37 lines).
>>> linux-wiki-example: 02_this_patch_fails.patch
patching file arch/arm/mach-msm/perf_trace_counters.h
Hunk #1 FAILED at 121.
1 out of 1 hunk FAILED -- saving rejects to file arch/arm/mach-msm/perf_trace_counters.h.rej
>>> ERROR: linux-wiki-example: all failed

When this happens, remove the failing patch from the source= variable in the APKBUILD, and delete the patch file from the same folder. Finally, correct the checksums and try kconfig edit again.

$ nano aports/device/linux-wiki-example/APKBUILD # remove from source
$ rm aports/device/linux-wiki-example/02_this_patch_fails.patch
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap kconfig edit linux-wiki-example
Make kconfig check happy
Consider using kconfig edit -x or -g to get a GUI instead of the terminal interface. Details: pmbootstrap kconfig edit -h

Check what you need to adjust in the kernel config before it can be used with postmarketOS. Keep changing the options, until everything passes.

$ pmbootstrap kconfig check
$ pmbootstrap kconfig edit wiki-example
$ pmbootstrap kconfig check

Kernel compilation

Next up is the kernel compilation, which will fail a few times before you get it working. Run it once to see where it fails.

$ pmbootstrap build linux-wiki-example

We most likely use a more modern version of GCC compared to what your device kernel was intended to be compiled with (which is good, we don't want outdated software). However, this means that you will probably need to patch your kernel before it compiles successfully. Don't be scared, we'll talk you through it. But in the unlikely case, that everything runs through smoothly, continue reading here.

Find the error message

In the log window, scroll up until you find the first message containing error: or Error (examples below). Read on for some tips on how to make this easier, if you can't find it right away.

In file included from arch/arm/mach-msm/perf_trace_counters.h:127:0,
                 from arch/arm/mach-msm/perf_trace_counters.c:14:
include/trace/define_trace.h:79:43: fatal error: ./perf_trace_counters.h: No such file or directory
drivers/built-in.o: In function `.LANCHOR1':
msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock'
msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock'
Makefile:877: recipe for target '.tmp_vmlinux1' failed
make: *** [.tmp_vmlinux1] Error 1

To get an error log, where the actual error message is not mixed with jobs running in parallel, please run the compilation again with only one job enabled. In general, this makes it slower. But don't worry, we use ccache. Therefore everything you have already compiled once will be faster when compiled again.

$ pmbootstrap -j1 build linux-wiki-example

If you still can't find it, try it with a text editor. The log is stored in ~/.local/var/pmbootstrap/log.txt (or in whichever work folder you have specified in the initialization step). It is even possible to clear the log file right before compilation, when you press CTRL + C in the log window and use log -c:

(old log output here)
^C
$ pmbootstrap log -c

Another trick to find the error message real quick, is to open log.txt with less and use the follow mode (press F while in less or open less with the +F option: less +F log.txt). This causes less to behave like tail -f. When an error occurs, you can exit the follow mode by pressing CTRL-C and use ? to reverse search the output (for example: ?error). Press F to re-enable follow mode.

Get a patch

From postmarketOS aports

It is highly likely, that we already ran into the same issue with another kernel. Take (parts of) the offending file name and search for it inside aports/device.

$ grep -r 'perf_trace_counters\.c' aports/device/
aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch:In-tree compilation for arch/arm/mach-msm/perf_trace_counters.c was
aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch:                     from arch/arm/mach-msm/perf_trace_counters.c:14
$ grep -r 'msm_iommu_sec' aports/device/
aports/device/linux-lg-mako/02_gpu-msm-fix-gcc5-compile.patch:msm_iommu_sec.c:(.data+0x9298): undefined reference to `kgsl_iommu_sync_lock'
aports/device/linux-lg-mako/02_gpu-msm-fix-gcc5-compile.patch:msm_iommu_sec.c:(.data+0x929c): undefined reference to `kgsl_iommu_sync_unlock'

When there is a result, copy the patch file you found to your new kernel package, add it to the source variable in the APKBUILD and try building again.

$ cp aports/device/linux-lg-mako/01_msm-fix-perf_trace_counters.patch aports/device/linux-wiki-example/
$ nano aports/device/linux-wiki-example/APKBUILD # add it to source
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap build linux-wiki-example
From elsewhere

Fire up your favorite search engine, and look at all results for the error message and variations of it. The "" feature of most search engines is useful, so it searches for an exact string and not single words. Example queries for the error messages above:

  • include/trace/define_trace.h: fatal error: ./perf_trace_counters.h: "No such file or directory"
  • linux "./perf_trace_counters.h: No such file or directory"
  • android "undefined reference to `kgsl_iommu_sync_unlock'"

In most cases, this will yield a patch that you can apply to your kernel. Save what you have found as a patch file right next to the APKBUILD of your new kernel package. Mailing list posts are usually in the format of a patch file and can be used directly, while commits or pull requests (PRs) on GitHub can be downloaded as patch when you append .patch to the URL (e.g. commit, patch).

If what you found on the web can't be used as patch file directly (e.g. sometimes the files are in other folders), but you understood how you would need to patch the source (from the search results or simply because you already knew), please follow this guide.

Patch files can contain arbitrary text before the first line with --- in the file. Please use this space to link to the source where you found a patch (in case it is from the Internet) and to paste the error message, that your patch fixes (so it is easier to find for other people).

After creating the patch file and adding the source URL and error message, put the file name into source in your APKBUILD and give it another shot.

$ nano aports/device/linux-wiki-example/03_description-here.patch
$ nano aports/device/linux-wiki-example/APKBUILD
$ pmbootstrap checksum linux-wiki-example
$ pmbootstrap build linux-wiki-example

We can't promise anything, but you shouldn't need to do this more than 2 times or so, before the kernel build finally goes through.

Using GCC6

If you can't make your kernel compile or boot with the latest GCC from Alpine, and you are pretty sure that the issues are specific to the GCC version, you can also use GCC6 instead.

Using GCC4

As GCC4 was recently introduced in postmarketOS repos, you can follow the procedure for GCC6 above and replace all gcc6 occurrences with gcc4 and add gcc4 (armv7/armhf/...)-gcc4 to your list of dependencies.

Device specific package

Open the deviceinfo file in an editor (replace wiki-example).

$ nano aports/device/device-wiki-example/deviceinfo

Now adjust:

  • deviceinfo_screen_width
  • deviceinfo_screen_height

In case you were asked for boot.img analyzing, and skipped that section during init: You can run pmbootstrap bootimg_analyze path/to/boot.img to get the right values for the deviceinfo_flash_offset_ lines now. Filling out manually also works, but that is error-prone and therefore not recommended.

See deviceinfo reference for a listing of all available variables and their meanings. The device specific package article explains advanced customization of the APKBUILD.

We are happy to help you!

If you get stuck, we are happy to help you in the chat and in the issues tracker. When asking for help, always include the APKBUILD you have written, and any custom patches you have applied, as well as the error message you have gotten from pmbootstrap log.

Documentation

This is a wiki, so please adjust everything that isn't detailed enough or would have helped you with porting.

More importantly, if you are at this point, it is a good idea to start writing your porting progress to the Devices page, even if you didn't get very far and the kernel does not compile (it will help others to continue from your work). Simply register in the wiki (you don't even need an e-mail address) and follow this tutorial.

Installation

If you get a "no space left on device" error, please report in #928.

Create checksums and build the device package once to make sure it doesn't contain errors at this point (we didn't do that earlier, because pmbootstrap would have complained about the invalid pkgver of the kernel package):

$ pmbootstrap checksum device-wiki-example
$ pmbootstrap build device-wiki-example

To install to the system partition of an image file, run the following:

$ pmbootstrap install

If you want to install to a SD card, insert it into your PC and run the following. Replace mmcblk0 with the actual device name (lsblk is handy to find out the correct device name). You don't need to format or partition it beforehand, pmbootstrap will take care of that.

$ pmbootstrap install --sdcard=/dev/mmcblk0

Flashing

Before you can run any flash command, you must put your device in the flashing/bootloader mode. It is usually done holding Volume Down & Power simultaneously when the device is switched off but it may vary depending on the device. For more information see the specific page for your device in the LineageOS Wiki.

If you want to install to the system partition, run the following (not for the SD card installation!):

$ pmbootstrap flasher flash_rootfs

In case your system partition is too small for the generated image (e.g. because you chose Plasma Mobile as UI), it is possible to flash to another partition as well. Just be sure to erase any previous installations of postmarketOS in other partitions, because the init script will start with the first one that it encounters. (To delete a previous version either use dd or simply install the known working OS, e.g. Android, on it). Using multiple partitions with LVM is planned (#60).

$ pmbootstrap flasher flash_rootfs --partition userdata

If you have a device, that works with fastboot, you can boot the kernel now without flashing it:

$ pmbootstrap flasher boot

Otherwise, you will need to flash the kernel to the device boot partition:

$ pmbootstrap flasher flash_kernel

Alternatives to regular flashing

If the flashing method does not work, it is also possible to export all generated image files to a specific directory (with symlinks), so you can flash it manually with your host Linux system (or even on Windows with proprietary flashers such as Odin, if this is the only way it works for you):

$ pmbootstrap export
$ pmbootstrap export --odin

For Android based devices, you can do a recovery zip installation.

When all else fails, you might have success with installing via netcat or booting via NFS.

USB Network

If you are lucky, your screen may give some clues that you are booted into pmOS. If not, do not get discouraged, the graphics on your device may not yet be setup correctly. The next steps are to see if you can connect to the device through SSH over USB (or if that fails, via telnet). See the Boot process and Inspecting the initramfs pages for more details.

It is quite easy to give your phone Internet access via USB, when your PC is connected to the Internet.

In case it looks like your device did not even boot, have a look at Troubleshooting:boot.

Upstreaming

As soon as USB networking is functional, or you have another indication that booting the device works successfully, it is time to upstream your work into the official pmaports repository. This way other people can benefit from your work, and build upon it.

See the git workflow article for detailed instructions.

Display

To make the display work in the first place, read Troubleshooting:display and Tuning sysfs. Then configure your touch screen.

After you have it working, please take some nice photos and add them to your device's wiki page. If you have a Reddit account, consider posting them in /r/postmarketOS as well.

Changing your installation

Install packages over SSH

Get Internet access via USB (easier) or Wifi, then use the package manager to install new packages:

$ apk add hello-world

You can also host packages you have built yourself from your PC. Just make sure to increase the package version when you build your own version of a package, that already exists. Otherwise the package manager can not know that your package is the newest one and should be installed. Check the output of apk to see whether it is really installing the package or not.

Building a new system image

Whenever you rebuild a package (or pick a new one to be installed by default in pmbootstrap init), you need to make sure that the package is actually installed in the chroot from which the system image gets generated. Achieve that by running pmbootstrap install again, it should update all outdated packages. Pay attention to the output of pmbootstrap log to see if it actually has been updated. If something did not work as expected, you can always pmbootstrap zap your chroots to start over with a clean installation.

Kernel

A simple way to recompile your kernel (e.g. because you want to change your kernel config) and flash it to your device is running the following commands:

$ pmbootstrap build --force linux-wiki-example
$ pmbootstrap install
$ pmbootstrap flasher flash_kernel

The last flash_kernel step is only needed if your device has the kernel stored on an extra boot partition. Android/fastboot compatible devices have such a partition, Nokia devices (N900 etc.) don't. If your device does not have a boot partition, update your system partition or SD card.

After you've gotten more familiar with postmarketOS, you could try out the postmarketos-update-kernel as well. It allows you to flash a new kernel to the device's boot partition (not to be confused with postmarketOS' own boot subpartition systemp1).

Full disk encryption: osk-sdl support

Our charging application charging-sdl uses the same software stack as osk-sdl, so following the instructions should make both components work.

If the display and the touchscreen interface are working, then it's time to add support for osk-sdl, so that you are able to use full disk encryption on the root filesystem. Instructions for adding support can be found here: Porting osk-sdl to New Devices

Advanced topics

Out of ideas what to do next with postmarketOS? Check out the get involved section from the last big blog post!