Testing Team

From postmarketOS

The testing team allows community members to directly get involved in the testing process. When there is a shiny new thing to test, users in the table below will get a notification (usually via mention in pmaports). They can install the change and report back whether everything worked as expected, or whether there is a regression.

FAQ

Can I join the testing team?

Thank you for considering! Yes, as long as you ...

  • ... have familiarized yourself enough with pmOS that you feel comfortable testing changes
  • ... make sure you don't lose important data (make backups as needed!)

Benefits:

  • You can dramatically help improving the quality of postmarketOS for the device you are using.
  • You hold the cool new changes in your hands before everyone else!

To join the testing team, add yourself to the table of members at the bottom of this wiki page.

Consider joining the release-party channel, where testing of releases and bigger changes gets coordinated.

Do I need to have stable or edge installed for testing?

The easiest is to just go with what you already have installed. But if you are considering a new installation just for testing, choose depending on what you want to test:

  • Merge requests: having edge installed is best, because almost all merge requests are for edge. (See the branch that a merge request is made against, if it is master, then it is for pmOS edge, otherwise the branch name is the name of the release, e.g. v23.06.)
  • Releases: doing a new installation with the upcoming stable release would be best (so we can see if there are any problems with completely fresh installations). But you can also upgrade or downgrade from the previous release / from edge.
  • Service packs: to test a service pack, you need to be on the current stable release.

Note that you do not need to be constantly swapping between releases when you are in the testing team. Change the release and help out with testing as often as you feel comfortable, you can also only test new releases ahead of time, or only test some of the edge MRs you are interested in etc.

How to test changes?

Merge requests

Install binary packages from merge request pipelines on your device with mrtest.

Make sure to run mrtest zap and apk upgrade -a when done with testing to remove the merge request packages again. Keeping them may cause problems in the future (e.g. conflicts when trying to test another merge request).

Releases

  • New installation
    • Either download an image for your device. Make sure to upgrade all packages before doing any tests.
    • Or build your own image with pmbootstrap. Make sure to select the new release in pmbootstrap init.
  • Upgrade or downgrade from the previous release / from edge.

Doing a new installation may show different problems (e.g. automatic resize of the filesystem not working) than upgrading or downgrading from another release. Make sure to mention which method you used, and consider using one that wasn't tested yet.

Service packs

Service packs can be tested via a staging repository before they get released. Wait for the announcement of the staging repository, then follow these steps.

Note that v23.06 and sp1 are examples, adjust them for the service pack you are testing.

  • Make sure you are on the stable release (v23.06)
  • Edit your /etc/apk/repositories:
    • replace http://mirror.postmarketos.org/postmarketos/v23.06
    • with http://mirror.postmarketos.org/postmarketos/staging/sp1/v23.06
  • Run apk upgrade -a
  • Reboot
  • Verify everything is still working as expected
  • Report back in the service pack issue
  • Revert the change to /etc/apk/repositories and run apk upgrade -a again when done

The staging branch and repository will be deleted after the service pack is merged, so apk will complain about a non-existing repository if you didn't revert the changes in the end.

What about device maintainers?

Typically device maintainers are not part of the testing team. Device maintainers already test on their devices as good as they can, the point of the testing team is to scale testing across many people, so development can be done both faster and with less regressions.

Best practices for editing the table?

  • Keep the entries consistent, so they can be easily searched (same spelling for UI names etc).
  • What to put in "Notes":
    • Revision of your device, if there are multiple ones (e.g. PinePhone revisions)
    • The panel of the device if there are multiple ones
    • Add-ons you would like to test, such as the PinePhone hardware keyboard

Members

Wiki username GitLab username Matrix username Device codenames UIs (or any) Notes
Anjanmomi anjandev @anjan:postmarketos.org xiaomi-beryllium postmarketos-ui-sxmo-de-sway Tianma Panel
Weissnix4711 Weissnix4711 @weissnix4711:matrix.org pine64-pinephonepro postmarketos-ui-gnome-mobile No accessories
EnderNightLord CarbonatedCaffeine @carbonatedcaffeine:matrix.org pine64-pinephone
oneplus-enchilada
samsung-m0
postmarketos-ui-phosh
postmarketos-ui-plasma-mobile
N/A
QC8086 QC8086 @qc8086:matrix.org pine64-pinebookpro
postmarketos-ui-sway (pbp)
N/A
PanzerSajt panzersajt @panzersajt3:matrix.org purism-librem5
pine64-pinephone
pine64-pinephonepro
oneplus-enchilada
xiaomi-beryllium
bq-paella
nokia-n900
postmarketos-ui-phosh
postmarketos-ui-gnome-mobile
L5 Evergreen
PP v1.2b
PinePhone hardware keyboard
Poco F1 EBBG
jrfern jrfern @jrfern:matrix.org fairphone-fp2 postmarketos-ui-phosh otm1902b panel
Crowdtier crowdtier @crowdtier:matrix.org oneplus-enchilada
pine64-pinephone
nokia-n900
postmarketos-ui-gnome-mobile
postmarketos-ui-phosh
maemo leste
postmarketos-ui-plasma-mobile
Pinephone v1.2b
Pinephone Keyboard
Antoniomaciej Antoniomaciej @antoniomaciej:matrix.org shift-axolotl
postmarketos-ui-phosh
postmarketos-ui-plasma-mobile
N/A
TheSame DvaMishkiLapa @thesame.personal:matrix.org google-krane postmarketos-ui-gnome N/A
peerke Peerke @64knl:matrix.org fairphone-fp4 postmarketos-ui-gnome-mobile N/A
Breakfastisready Breakfastisready @breakfastisready:matrix.org pine64-pinephonepro
oneplus-enchilada
samsung-gt58
postmarketos-ui-sxmo-de-sway
postmarketos-ui-phosh
Pinephone Keyboard
Fdelamotte fdlamotte @fdelamotte:matrix.org oneplus-enchilada postmarketos-ui-sxmo-de-sway N/A
Antonok antonok @antonok:antonok.com oneplus-enchilada
purism-librem5
postmarketos-ui-phosh N/A
Hexaheximal hexaheximal @hexaheximal:matrix.org valve-jupiter any N/A
Fancsali fancsali @fancsali:matrix.org google-veyron-minnie postmarketos-ui-xfce
postmarketos-ui-phosh
N/A
pcercuei pcercuei pcercuei (IRC) samsung-i9100 postmarketos-ui-phosh N/A
Nau5ea nau5ea @ensignmorituri:postmarketos.org google-krane postmarketos-ui-phosh N/A
Serg12344 Serg12344 @serg12344:matrix.org xiaomi-mido postmarketos-ui-plasma-mobile N/A
retourmy retourmy @regis:g5.ns0.it pine64-pinephone
pine64-pinephonepro
pine64-pinebookpro
purism-librem5
postmarketos-ui-phosh Pinephone v1.2b
L5 evergreen
LongnoseRob longnoserob @longnoserob:matrix.org asus-tf101
asus-tf201
asus-grouper
postmarketos-ui-mate other UI might also be possible
Apollo3zehn Apollo3zehn @apollo3zehn:matrix.org fairphone-fp4 postmarketos-ui-plasma-mobile N/A
Colaur colorant31 @colorant:matrix.org samsung-a6plte
samsung-a3
postmarketos-ui-phosh N/A
Tommi xplosionmind @tommi:fsfe.org fairphone-fp5 postmarketos-ui-gnome-mobile N/A
Owen TheOwenSZ @owen:fyralabs.com google-hayato postmarketos-ui-gnome N/A
Catto24 catto24 @catto24:matrix.org google-corsola any postmarketOS is cool :D
Mtek89 mtekman @mtekman:matrix.org alcatel-idol347 any N/A
fossdd fossdd @fossdd:matrix.org pine64-pinephonepro any N/A

See also