Osk-sdl

WARNING: Osk-sdl is deprecated by Unl0kr (See pmaports#2319)

This is an application developed by the postmarketOS community to unlock the LUKS volume in the beginning of the boot process. It implements a password box and on-screen keyboard directly on the linux framebuffer, but also supports (though largely untested at the moment) any of the other backends supported by SDL2.

It is written in C++14 and uses libcryptsetup directly to unlock the partitions. The osk-sdl package is part of pmOS's aports.

Reporting Issues

Issues with osk-sdl should be reported on the project gitlab issues page.

The osk-sdl-dev room on Matrix can also be used to ask questions, see Matrix_and_IRC.

Porting to New Devices

It's necessary to configure and/or create a few files when porting osk-sdl to a new device. Some of the files mentioned below are not necessary on all devices. Any files created out of necessity in the below steps should be included in your device package. See the nokia-n900 device package for an example of how some of these files have been included/installed.

It's highly recommended that you run osk-sdl from within pmOS, which means installing pmOS without the --fde parameter. Running osk-sdl from within pmOS during the porting process will allow you to run any additional tools required for generating some of the configuration files below, as well as collect additional diagnostic info if you run into any issues.

Running osk-sdl within pmOS

The osk-sdl package is in pmOS's aports directory. To prepare pmOS for running osk-sdl for porting:

$ git clone https://gitlab.postmarketos.org/postmarketOS/pmbootstrap.git
$ cd pmbootstrap
$ ./pmbootstrap init
<select device options>
<select 'none' for UI so that osk-sdl can be run from within pmos without conflicting with weston>
$ ./pmbootstrap install --add osk-sdl

Not using --fde will allow pmOS to boot straight into rootfs without requiring unlocking of the partition. This seems counter-intuitive for porting a keyboard made for initramfs, but this will allow you to manually run osk-sdl with various environment options set and/or osk.conf options for debug.

To run osk-sdl from pmOS (not in initramfs):

$ sudo su
# export DFBARGS="system=fbdev,no-cursor,disable-module=linux_input"
# export TSLIB_TSDEVICE=/dev/input/eventX
# osk-sdl -d a -n a -c /etc/osk.conf -v

Do not use the -t option when porting osk-sdl on a device, this mode of running osk-sdl will not be helpful for this purpose!

Enable hardware acceleration

osk-sdl can use hardware acceleration (instead of slowly drawing to the framebuffer). The necessary setup is automatically done by setting deviceinfo_mesa_driver (see deviceinfo reference) in the device package. The driver that needs to be specified there can be found for example by running:

$ strace kmscube 2>&1 | grep "xorg/modules"
...
openat(AT_FDCWD, "/usr/lib/xorg/modules/dri/sun4i-drm_dri.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 4

In this example, "sun4i-drm" should be specified. In some cases also additional GPU firmware needs to be copied to the initramfs which can be done by putting e.g. /etc/postmarketos-mkinitfs/files/30-gpu-firmware.files into the device package with each file that should be copied on a separate line. See the Fairphone 2 device port for an example where a firmware file is copied.

In addition to setting deviceinfo_mesa_driver, you must also add a dependency on mesa-egl (pmaports#856) OR mesa-dri-gallium for newer devices.

This feature was added in pmaports!1625.

Set touchscreen device

See deviceinfo_dev_touchscreen.

Create fb.modes

Some devices require creating a file (/etc/fb.modes) to inform DirectFB of the correct mode that the display supports.

To calculate the mode for your device you can run fbset, however beware that the mode it returns isn't always supported. Some things to be aware of:

Make sure the last number of the geometry line (the color depth) is not 32 or 24, because this is not supported by directfb. If you need 32 or 24 there, set this value to 16 and set pixelformat in DFBARGS to a supported pixel format with color depth 32 or 24 (see man page of directfbrc for all available pixelformat options).
Normally the geometry line should have twice your resolution (both for physical and virtual resolution). Fbset sometimes returns wrong values with which osk-sdl cannot cope (generates segmentation fault when touching the keyboard).

An example for a display with 1080x1920 resolution and 32-bit color depth:

mode "1080x1920-0"
    # D: 0.001 MHz, H: 0.001 kHz, V: 0.000 Hz
    geometry 1080 1920 1080 1920 16
    timings 898000000 75 129 4 8 5 4
    rgba 8/16,8/8,8/0,8/24
endmode

If you had to create this file for osk-sdl to display on your device, make sure to include the /etc/fb.modes file in your device package.

Source:
• DirectFB modes
• SDL2 modes

Create touchscreen calibration file

Some devices require touchscreen calibration for osk-sdl to properly handle taps on the screen. To create the calibration file necessary for osk-sdl, run the ts_calibrate utility and tap the small X's on screen as accurately as you can. This will generate a file /etc/pointercal. Use ts_test utility to check if touchscreen is working correctly after calibration.

After running the calibration, re-run osk-sdl to confirm that touch input is being appropriately handled.

If you had to create this file for osk-sdl to properly handle the touchscreen input on your device, make sure to include the /etc/pointercal file in your device package.

The file format is quite similar to the calibration for libinput. The file consists of 9 values. It starts with two groups of three values, the first is for the x coordinate and the second for the y coordinate. Each group consists of a multiplier for the x and a multiplier for the y coordinate and then an offset. However, as all values need to be integers, the values are scaled by a scaling factor. This is the 7th number in the file and is always 65536. The last two values are the screen width and height. Similar to the calibration matrix for libinput, some of the first two values of every group should be 0. While one might think that the other values should always be around the scaling factor or, for the offsets, 0 or the scaling factor times device width/height (as for libinput), this is not the case. E.g. for the Google_Nexus_7_2012_(asus-grouper) the x/y multipliers are around 0.6 (see pointercal) even though for Weston no such scaling is necessary. This could actually be the width/height ratio (0.625) or something unrelated. If you have more information on this, please add it here so we can get more precise calibration files!

Porting References

Development

Testing with QEMU

To build osk-sdl from a local source tree and test it in QEMU, run the following once:

$ pmbootstrap zap
$ pmbootstrap config device qemu-amd64
$ pmbootstrap config ui none

Then for every time you want to test (it's faster the second time):

$ pmbootstrap build osk-sdl --src=/path/to/osk-sdl/source/tree
$ yes | pmbootstrap install --fde
$ pmbootstrap qemu

The yes command will set the password to y.

Troubleshooting

Updating osk-sdl without flashing the rootfs

osk-sdl does not get updated when flashing the kernel/initramfs/boot.img to your device. That is, because it is in the initramfs-extra file inside the boot partition of the rootfs. Here's a rough description of the steps you need to take to update it without flashing the entire rootfs (which is the easy way), in case you have flashed a version that doesn't boot up anymore. As usual, just ask in the channel for help if you get stuck.

Install the debug-shell initramfs hook and connect via telnet
Mount the boot partition of the rootfs image
use netcat to upload the new initramfs-extra file via telnet

Unlocking via telnet

If osk-sdl does not work for some reason, but you really need to access your data, you can connect to the initramfs via telnet, enable initramfs-extra, then run the following to unlock it:

# source /init_functions.sh
# cryptsetup luksOpen "$(find_root_partition)" root
Enter passphrase for /dev/mapper/mmcblk0p2:
# killall osk-sdl

ERROR: Library not found: libGL.so

This should be resolved in SDL2 2.0.6, however if this issue occurs then most likely the mesa-gl dependency for osk-sdl did not install properly. Please report this issue, see #Reporting Issues.

Make sure you've exported SDL_VIDEO_GL_DRIVER:

$ sudo su
# export SDL_VIDEO_GL_DRIVER="libGL.so.1"

If you have been testing earlier versions of osk-sdl, you may still have a custom SDL2 package with version 9999 in your packages directory. That is not required anymore, all necessary changes to SDL2 have been upstreamed, we can use the version from Alpine now. Delete it with the following steps:

$ sudo rm -v ~/.local/var/pmbootstrap/packages/*/sdl2*
$ pmbootstrap index

DirectFB/FBDev: No supported modes found in /etc/fb.modes and current mode not supported!

You will need to create a fb.modes file for your device. See the above section #Create fb.modes

ERROR: Unknown dfb pixelformat

osk-sdl is using SDL2 and DirectFB for rendering to the framebuffer (which is what we need to do for most devices, because they only have framebuffer drivers in the kernel, not the more modern DRI/DRM drivers). The error message means, that the SDL2 mode is not mapped to a DirectFB mode. See osk-sdl#85 for detailed instructions on how to patch SDL2 to add a mapping for your mode.

Building osk-sdl from repo

Quote from osk-sdl#90:

Here's the workflow for modifying the code and building it as postmarketOS package:

$ cd ~/code
$ git clone https://gitlab.postmarketos.org/postmarketOS/osk-sdl/
$ cd osk-sdl
$ nvim src/main.cpp # make changes somewhere
$ pmbootstrap build osk-sdl --arch=armhf --src=~/code/osk-sdl

Then you can copy the new package to the device, either with scp or with this guide: Installing_packages_on_a_running_phone