Difference between revisions of "Hybris"

From postmarketOS
Jump to: navigation, search
m (Test whether lxc-android works properly and kernel is correctly configured)
(Merging contents of TheKit/Hybris into this page + add more formatting, more detailed explanation.)
 
(10 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
[https://en.wikipedia.org/wiki/Hybris_(software) Libhybris] allows devices lacking FLOSS drivers to make full use of their hardware by using the proprietary drivers from Android. Mainlining and using free software drivers is of course the better approach, but it is not possible for all devices. '''We will not list devices that need hybris as featured devices''' (see {{issue|83|postmarketos.org}}).
 
[https://en.wikipedia.org/wiki/Hybris_(software) Libhybris] allows devices lacking FLOSS drivers to make full use of their hardware by using the proprietary drivers from Android. Mainlining and using free software drivers is of course the better approach, but it is not possible for all devices. '''We will not list devices that need hybris as featured devices''' (see {{issue|83|postmarketos.org}}).
 +
 +
== What is libhybris? ==
 +
 +
'''libhybris''' is a compatibility layer, allowing libraries compiled with bionic libc (used in Android) to be loaded by applications built with different libc (originally glibc, musl in postmarketOS case).
 +
 +
== How it works? ==
 +
'''libhybris''' includes modified Android linker (partially reimplementing it, for several Android versions) and provides <code>android_dlsym</code> function which can be used by host libc applications to load bionic libraries. Bionic libc gets loaded into process space alongside host libc, but with many functions hooked or replaced with host libc ones to avoid various memory structs incompatibilities between two libcs.
 +
 +
However, for this to work, there are few Android-side libraries to be patched. This is necessary to avoid thread-local storage (TLS) slots conflicts, as Android has reserved private slots for system libraries such as '''libEGL''' which do not require allocation through pthread API and accessed directly through CPU register containting thread pointer, which causes host libc data to be ovewritten.
 +
 +
[https://halium.org Halium] and [https://github.com/mer-hybris/ mer-hybris] already include patches to workaround that, but if you use different Android tree, the following needs to be applied: bionic patch mentioned in https://github.com/libhybris/libhybris/pull/395
 +
 +
== Running Android services in LXC container or chroot ==
 +
This is not strictly required to use '''libhybris''' itself, but is generally needed, because Android drivers expect Android <code>init</code> to be running, which provides <code>property_service</code>, does HW-specific initialization via <code>init.$soc.rc</code> script and starts vendor-specific services.
 +
 +
Halium solves that by providing minimal modified Android base which can run inside LXC container and system-side scripts to do that (provided by lxc-android package). If your device supports LineageOS 14.1, it might be easiest to port Halium and use resulting <code>system.img</code> with lxc-android.
 +
 +
Otherwise it is also possible to run Android init in chroot with graphical services such as <code>zygote</code> disabled. [https://github.com/mer-hybris/ mer-hybris] tree (built for SailfishOS ports) is an example of that and contains modifications needed for '''libhybris''' to work.
  
 
== Using libhybris with your device ==
 
== Using libhybris with your device ==
  
''This is a stub copied from chat messages in #postmarketOS. Please extend if you are porting your device to libhybris.''
+
=== libhybris current usage ===
 +
It is used for the [[Planet_Computers_Gemini_PDA_(planet-geminipda)|Gemini PDA]] (non-CAF) and [[Samsung_Galaxy_S4_Mini_dual_sim|Samsung S4 Mini Duos]] (CAF), [[Samsung_Galaxy_S5_(samsung-klte)|Samsung Galaxy S5]] (CAF), [[Xiaomi_RedMi4x_(xiaomi-santoni)|Xiaomi Redmi 4X]] (CAF) device ports.
  
* convert the halium system.img to a mountable image (basically what the [https://github.com/Halium/halium-scripts/blob/master/halium-install#L41-L52 halium install script] does)
+
=== Android system image ===
* install postmarketOS on the device without user interface and copy the new system.img to the device, to <code>/var/lib/lxc/android/system.img</code>
+
 
* install the hybris related packages:
+
First thing to do is convert the halium generated '''system.img''' to a mountable image (basically what the [https://github.com/Halium/halium-scripts/blob/master/halium-install#L41-L52 halium install script] does)
 +
and the new '''system.img''' to the device, to <code>/var/lib/lxc/android/system.img</code>
 +
 
 +
The way to do attain this, is:
 +
* Port halium 7.1 (https://docs.halium.org) to your device. This may take considerable effort. Note that we only have hybris 7.1 packaged at the moment.
 +
* Get it running to a point where you have at least graphics operational
 +
* It works if you run the same kernel source for both, and apply all defconfig changes needed for the halium port to your postmarketos port
 +
* Boot your halium-running device into android recovery, and download the <code>/data/system.img</code> file from it. This is the systemimage you've built for halium, with the modifications that the installation script has done.
 +
* Flash postmarketOS to your device, where you've included the necessary changes for halium
 +
* Upload the system.img file to <code>/var/lib/lxc/android/system.img</code>
 +
 
 +
=== Install the hybris related packages ===
  
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
 
$ sudo apk add lxc-android \
 
$ sudo apk add lxc-android \
              xf86-video-hwcomposer \
 
              xf86-video-hwcomposer-7.1-caf \
 
 
               libhybris \
 
               libhybris \
 +
              libhybris-7.1 \
 
               libhybris-egl \
 
               libhybris-egl \
 
               libhybris-gles \
 
               libhybris-gles \
 
               libhybris-libwayland-egl \
 
               libhybris-libwayland-egl \
              libhybris-7.1-caf \
 
 
               wayland-libs-client \
 
               wayland-libs-client \
 
               wayland-libs-server
 
               wayland-libs-server
 
</syntaxhighlight>
 
</syntaxhighlight>
  
* start the service:
+
For X11 support, install the following, depending if you need software which is built for Code Aurora Forum’s Android for MSM (generally nicknamed “CAF” for short):
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ sudo rc-service lxc-android start
+
$ sudo apk add xf86-video-hwcomposer xf86-video-hwcomposer-7.1-caf             
 
</syntaxhighlight>
 
</syntaxhighlight>
  
== Test whether lxc-android works properly and kernel is correctly configured ==
+
For all the other devices:
 +
<syntaxhighlight lang="shell-session">
 +
$ sudo apk add xf86-video-hwcomposer xf86-video-hwcomposer-7.1             
 +
</syntaxhighlight>
  
1) Ensure you installed the packages above, started the lxc-android service and checked that it's running
+
=== Start the android container ===
 
 
2) Download the Halium reference rootfs from bshah’s personal server: http://bshah.in/halium/halium-rootfs-20170630-151006.tar.gz
 
 
 
3) Extract to a temporary folder
 
 
 
4) Bind some folders into the halium rootfs
 
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ mount --bind /dev /tmp/halium-rootfs/dev
+
$ sudo rc-service lxc-android start
$ mount --bind /sys /tmp/halium-rootfs/sys
 
$ mount --bind /proc /tmp/halium-rootfs/proc
 
$ mount --bind /system /tmp/halium-rootfs/system
 
$ ln -s /system/vendor /tmp/halium-rootfs/vendor
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
5) Chroot into halium
+
=== Configure Xorg ===
<syntaxhighlight lang="shell-session">
+
 
$ chroot /tmp/halium-rootfs /bin/bash
+
You need to create the <code>/etc/X11/xorg.conf.d/10-<device>.conf</code> with the following:
 +
<syntaxhighlight>
 +
Section "Device"
 +
  Identifier "Card0"
 +
  Driver "hwcomposer"
 +
EndSection
 
</syntaxhighlight>
 
</syntaxhighlight>
  
6) Update
+
=== Run hardware composer test (to verify it all works) ===
 +
For a CAF device, use this command:
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
$ echo "deb http://repo.halium.org/caf xenial main" >> /etc/apt/sources.list.d/halium-caf.list
+
$ EGL_PLATFORM=hwcomposer test_hwcomposer_7_caf
$ apt-get update
 
$ apt-get dist-upgrade
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
7) Test
+
For a non-CAF device, use this command:
 
<syntaxhighlight lang="shell-session">
 
<syntaxhighlight lang="shell-session">
 
$ EGL_PLATFORM=hwcomposer test_hwcomposer
 
$ EGL_PLATFORM=hwcomposer test_hwcomposer
 
</syntaxhighlight>
 
</syntaxhighlight>
 +
 +
== Utilizing libhybris in different subsystems ==
 +
=== Graphics ===
 +
Android HAL requires device vendors to provide HWComposer API implementation. That API can be utilized via '''libhybris''' by Wayland compositors or Xorg to output graphical buffers on screen (as opposed to writing to fbdev directly).
 +
 +
The following projects support it:
 +
* kwin - Plasma De https://github.com/KDE/kwin/tree/master/plugins/platforms/hwcomposer (this one is packaging WIP)
 +
* qt5-qpa-hwcomposer-plugin - ''Qt Platform Abstraction'' plugin utilized by SailfishOS, Nemo Mobile and LuneOS - https://github.com/mer-hybris/qt5-qpa-hwcomposer-plugin
 +
* mir-android-platform - Graphics platform for the Mir display server - https://github.com/ubports/mir-android-platform
 +
* xf86-video-hwcomposer - Xorg DDX driver to render through HWComposer API - https://github.com/gemian/xf86-video-hwcomposer (this one is packaged)
 +
 +
=== Sound ===
 +
PulseAudio Droid modules allow PulseAudio to use Android HAL via '''libhybris'''. While it is possible to use ALSA directly, this approach has benefits of reusing Android mixer configs, which gets very tricky on embedded devices such as phones: https://github.com/mer-hybris/pulseaudio-modules-droid. postmarketOS has this packaged as [https://gitlab.com/postmarketOS/pmaports/tree/master/hybris/pulseaudio-modules-droid pulseaudio-modules-droid]. See also [[Audio#libhybris|Audio (libhybris)]].
 +
 +
=== Cellular ===
 +
There are two different implementations of RIL plugin for ofono whichs allows it to communicate with Android's RIL daemon for cellular network connectivity (both data and calls).
 +
 +
* ril driver - developed and maintaned by SailfishOS, available as part of their ofono fork - https://git.merproject.org/mer-core/ofono
 +
* rilmodem driver - was developed by Ubuntu Touch and partly merged to upstream ofono, but is not updated anymore - https://github.com/rilmodem/ofono
 +
 +
=== Camera and media decoding ===
 +
GStreamer plugin is maintained by SailfishOS - https://github.com/sailfishos/gst-droid. It requires https://github.com/sailfishos/droidmedia to be compiled against device Android tree and minimedia daemon running.
 +
 +
The alternative approach utilized by Ubuntu Touch is to use libcamera_compat_layer and libmedia_compat_layer. They comprised of libraries that need to be compiled against Android (compat folder of libhybris) and libhybris wrappers. Does not work with Android 7 and newer yet.
  
 
== See also ==
 
== See also ==
Line 67: Line 116:
 
* [https://postmarketos.org/blog/2018/06/09/one-year/#libhybris-and-optional-proprietary-components One year of postmarketOS: libhybris and optional proprietary components]
 
* [https://postmarketos.org/blog/2018/06/09/one-year/#libhybris-and-optional-proprietary-components One year of postmarketOS: libhybris and optional proprietary components]
 
* [https://postmarketos.org/blog/2017/12/31/219-days-of-postmarketOS/#libre-drivers-and-libhybris 219 days of postmarketOS: libre drivers and libhybris]
 
* [https://postmarketos.org/blog/2017/12/31/219-days-of-postmarketOS/#libre-drivers-and-libhybris 219 days of postmarketOS: libre drivers and libhybris]
 +
 +
 +
[[Category:Technical Reference]]

Latest revision as of 18:48, 17 July 2019

Libhybris allows devices lacking FLOSS drivers to make full use of their hardware by using the proprietary drivers from Android. Mainlining and using free software drivers is of course the better approach, but it is not possible for all devices. We will not list devices that need hybris as featured devices (see postmarketos.org#83).

What is libhybris?

libhybris is a compatibility layer, allowing libraries compiled with bionic libc (used in Android) to be loaded by applications built with different libc (originally glibc, musl in postmarketOS case).

How it works?

libhybris includes modified Android linker (partially reimplementing it, for several Android versions) and provides android_dlsym function which can be used by host libc applications to load bionic libraries. Bionic libc gets loaded into process space alongside host libc, but with many functions hooked or replaced with host libc ones to avoid various memory structs incompatibilities between two libcs.

However, for this to work, there are few Android-side libraries to be patched. This is necessary to avoid thread-local storage (TLS) slots conflicts, as Android has reserved private slots for system libraries such as libEGL which do not require allocation through pthread API and accessed directly through CPU register containting thread pointer, which causes host libc data to be ovewritten.

Halium and mer-hybris already include patches to workaround that, but if you use different Android tree, the following needs to be applied: bionic patch mentioned in https://github.com/libhybris/libhybris/pull/395

Running Android services in LXC container or chroot

This is not strictly required to use libhybris itself, but is generally needed, because Android drivers expect Android init to be running, which provides property_service, does HW-specific initialization via init.$soc.rc script and starts vendor-specific services.

Halium solves that by providing minimal modified Android base which can run inside LXC container and system-side scripts to do that (provided by lxc-android package). If your device supports LineageOS 14.1, it might be easiest to port Halium and use resulting system.img with lxc-android.

Otherwise it is also possible to run Android init in chroot with graphical services such as zygote disabled. mer-hybris tree (built for SailfishOS ports) is an example of that and contains modifications needed for libhybris to work.

Using libhybris with your device

libhybris current usage

It is used for the Gemini PDA (non-CAF) and Samsung S4 Mini Duos (CAF), Samsung Galaxy S5 (CAF), Xiaomi Redmi 4X (CAF) device ports.

Android system image

First thing to do is convert the halium generated system.img to a mountable image (basically what the halium install script does) and the new system.img to the device, to /var/lib/lxc/android/system.img

The way to do attain this, is:

  • Port halium 7.1 (https://docs.halium.org) to your device. This may take considerable effort. Note that we only have hybris 7.1 packaged at the moment.
  • Get it running to a point where you have at least graphics operational
  • It works if you run the same kernel source for both, and apply all defconfig changes needed for the halium port to your postmarketos port
  • Boot your halium-running device into android recovery, and download the /data/system.img file from it. This is the systemimage you've built for halium, with the modifications that the installation script has done.
  • Flash postmarketOS to your device, where you've included the necessary changes for halium
  • Upload the system.img file to /var/lib/lxc/android/system.img

Install the hybris related packages

$ sudo apk add lxc-android \
               libhybris \
               libhybris-7.1 \
               libhybris-egl \
               libhybris-gles \
               libhybris-libwayland-egl \
               wayland-libs-client \
               wayland-libs-server

For X11 support, install the following, depending if you need software which is built for Code Aurora Forum’s Android for MSM (generally nicknamed “CAF” for short):

$ sudo apk add xf86-video-hwcomposer xf86-video-hwcomposer-7.1-caf

For all the other devices:

$ sudo apk add xf86-video-hwcomposer xf86-video-hwcomposer-7.1

Start the android container

$ sudo rc-service lxc-android start

Configure Xorg

You need to create the /etc/X11/xorg.conf.d/10-<device>.conf with the following:

Section "Device"
  Identifier "Card0"
  Driver "hwcomposer"
EndSection

Run hardware composer test (to verify it all works)

For a CAF device, use this command:

$ EGL_PLATFORM=hwcomposer test_hwcomposer_7_caf

For a non-CAF device, use this command:

$ EGL_PLATFORM=hwcomposer test_hwcomposer

Utilizing libhybris in different subsystems

Graphics

Android HAL requires device vendors to provide HWComposer API implementation. That API can be utilized via libhybris by Wayland compositors or Xorg to output graphical buffers on screen (as opposed to writing to fbdev directly).

The following projects support it:

Sound

PulseAudio Droid modules allow PulseAudio to use Android HAL via libhybris. While it is possible to use ALSA directly, this approach has benefits of reusing Android mixer configs, which gets very tricky on embedded devices such as phones: https://github.com/mer-hybris/pulseaudio-modules-droid. postmarketOS has this packaged as pulseaudio-modules-droid. See also Audio (libhybris).

Cellular

There are two different implementations of RIL plugin for ofono whichs allows it to communicate with Android's RIL daemon for cellular network connectivity (both data and calls).

Camera and media decoding

GStreamer plugin is maintained by SailfishOS - https://github.com/sailfishos/gst-droid. It requires https://github.com/sailfishos/droidmedia to be compiled against device Android tree and minimedia daemon running.

The alternative approach utilized by Ubuntu Touch is to use libcamera_compat_layer and libmedia_compat_layer. They comprised of libraries that need to be compiled against Android (compat folder of libhybris) and libhybris wrappers. Does not work with Android 7 and newer yet.

See also