Difference between revisions of "Pmbootstrap development guide"

From postmarketOS
Jump to: navigation, search
m (Ollieparanoid moved page Development-guide to Development guide)
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Tips ==
+
=== Installation code (<code>pmb.install.*</code>) ===
 +
Whenever you change the installation process, also adjust the recovery zip installer (which performs a good part of the installation procedure directly on the Android device with a shell script).
  
If happens often that we commit changes and then we realize that we forgot to regenerate APKBUILD checksums. This is noticed by Travis ([https://travis-ci.org/postmarketOS/pmbootstrap/builds/261056579 example]) but you might want to avoid it.
+
=== What is this <code>args</code> variable used everywhere? ===
 +
See the big comment at the top of [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/helpers/args.py <code>pmb/helpers/args.py</code>].
  
You can install the following git hook that is run every time before you commit.
+
=== Executing shell commands ===
 +
Use one of the following functions instead of Python's built in <code>subprocess</code>:
  
Copy the following code to <code>.git/hooks/pre-commit</code> inside your <code>pmbootstrap</code> repository:
+
* [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/helpers/run.py <code>pmb.helpers.run.user()</code>]
 +
* [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/helpers/run.py <code>pmb.helpers.run.root()</code>]
 +
* [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/chroot/user.py <code>pmb.chroot.user()</code>]
 +
* [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/chroot/root.py <code>pmb.chroot.root()</code>]
  
<pre class="shell">#!/bin/sh
+
These functions call [https://gitlab.com/postmarketOS/pmbootstrap/blob/master/pmb/helpers/run_core.py <code>pmb.helpers.run_core.core()</code>] internally to write to the log file (that you can read with <code>pmbootstrap log</code>) and timeout when there is no output. A lot of function parameters are passed through to <code>core()</code> as well, see its docstring for a detailed description of what these parameters do.
#
 
# postmarketOS pre-commit hook
 
# Checks for bad checksums
 
  
./test/check_checksums.py</pre>
+
==== Writing to files ====
Now this is what happens when you forgot to regenerate checksums for some aport and try to commit
+
{{note|See also: {{github|1316}} Properly escape commands in pmb.chroot.user()}}
 +
The passed commands gets interpreted as single command, and not executed in a shell. This means something like the following '''will not work''':
  
<pre class="shell">$ git commit -a
+
<syntaxhighlight lang="python">
Checking device-qemu-aarch64 for correct checksums
+
pmb.chroot.root(args, ["echo", "test", ">", "/tmp/test"])
[13:45:23] (native) generate checksums for device-qemu-aarch64
+
</syntaxhighlight>
[13:45:23] Done
 
MM aports/device/device-qemu-aarch64/APKBUILD
 
  
diff --git a/aports/device/device-qemu-aarch64/APKBUILD b/aports/device/device-qemu-aarch64/APKBUILD
+
Instead, if you need shell functionality in the call, you can wrap the call with <code>sh -c</code>. Here's one example with the parameters correctly escaped (you don't need to do that for the strings in this example, but if you pass variables then they should be escaped properly):
index 99b811b..c4cee6c 100644
 
--- a/aports/device/device-qemu-aarch64/APKBUILD
 
+++ b/aports/device/device-qemu-aarch64/APKBUILD
 
@@ -21,4 +21,4 @@ package() {
 
                &quot;$pkgdir&quot;/etc/deviceinfo
 
}
 
 
-sha512sums=&quot;a18ab789c19d8802be6a9e9c842f4c304f267bd78f46b096b4ba6609430d1544e0a9eca9bc22db67a0728efae3aad6c2def9f7e64157d31aeb31b4bb839076d5  deviceinfo&quot;
 
+sha512sums=&quot;8df46383685a40a96b7c604c29d413ecdd4b4a9b92cb61f452283042f189b1ca01cb70632fec0711dd67229e0501729762c6841359ac7ea455f516b512abce9c  deviceinfo&quot;
 
  
** The checksums are not correct</pre>
+
<syntaxhighlight lang="python">
After that, simply do <code>git commit -a --amend -n</code> (<code>-n</code> is necessary so that this time the hook is not triggered).
+
shell_cmd = "echo " + shutil.quote("test") + " > " + shutil.quote("/tmp/test")
 +
pmb.chroot.root(args, ["sh", "-c", shell_cmd])
 +
</syntaxhighlight>
  
== Code ==
+
Another technique (especially if you need to run many commands in the shell at once) is writing all commands down into a temporary file and executing it with the shell, see for example [https://github.com/postmarketOS/pmbootstrap/blob/1c13ca4fd9e32952a8e559987f82cd0925cddee9/pmb/chroot/initfs.py#L66-L76 pmb/chroot/initfs.py].
  
=== pmb.helpers.run ===
+
==== Writing files to the chroot ====
 +
The users in the chroots (root and pmos) have different user IDs than the user of the host system. Therefore we can't just write a file to anywhere in the chroot. But we can write to <code>/tmp</code> and move the file afterwards:
 +
<syntaxhighlight lang="python">
 +
with open("tmp/somefile", "w") as handle:
 +
    handle.write("It works")
 +
pmb.chroot.root(args, ["mv", "/tmp/somefile", "/etc/somefile"])
 +
</syntaxhighlight>
  
Use <code>pmb.helpers.run.root()</code> and <code>pmb.helpers.run.user()</code> instead of <code>subprocess</code> because these logs to the log file.
+
== See also ==
 +
* [[Testsuite]]
  
For security reasons, you cannot write files by redirecting the output of the <code>echo</code> command using <code>pmb.helpers.run</code> functions. The following call will never work as you would expect
+
[[Category:Guide]]
 
 
<pre>pmb.chroot.root(args, [&quot;echo&quot;, &quot;test&quot;, &quot;&gt;&quot;, &quot;/tmp/test&quot;])</pre>
 
In order to write a file inside of a chroot, the recommended way is to write the file normally as you would do in Python and then copy the file to the correct location, like
 
 
 
<pre>with open(&quot;/tmp/somefile&quot;, &quot;w&quot;) as handle:
 
    handle.write(&quot;It works&quot;)
 
pmb.chroot.root(args, [&quot;mv&quot;, &quot;/tmp/somefile&quot;, &quot;/etc/somefile&quot;])</pre>
 
If you need to redirect the input instead, another solution is writing a bash script to a file and then executing it as in [https://github.com/postmarketOS/pmbootstrap/blob/1c13ca4fd9e32952a8e559987f82cd0925cddee9/pmb/chroot/initfs.py#L66-L76 pmb/chroot/initfs.py]
 
 
 
=== pmb.install ===
 
 
 
Whenever you change the installation process, also adjustt the recovery zip installer (which performs a good part of the installation procedure directly on the Android device with a shell script).
 

Latest revision as of 04:23, 17 December 2018

Installation code (pmb.install.*)

Whenever you change the installation process, also adjust the recovery zip installer (which performs a good part of the installation procedure directly on the Android device with a shell script).

What is this args variable used everywhere?

See the big comment at the top of pmb/helpers/args.py.

Executing shell commands

Use one of the following functions instead of Python's built in subprocess:

These functions call pmb.helpers.run_core.core() internally to write to the log file (that you can read with pmbootstrap log) and timeout when there is no output. A lot of function parameters are passed through to core() as well, see its docstring for a detailed description of what these parameters do.

Writing to files

See also: #1316 Properly escape commands in pmb.chroot.user()

The passed commands gets interpreted as single command, and not executed in a shell. This means something like the following will not work:

pmb.chroot.root(args, ["echo", "test", ">", "/tmp/test"])

Instead, if you need shell functionality in the call, you can wrap the call with sh -c. Here's one example with the parameters correctly escaped (you don't need to do that for the strings in this example, but if you pass variables then they should be escaped properly):

shell_cmd = "echo " + shutil.quote("test") + " > " + shutil.quote("/tmp/test")
pmb.chroot.root(args, ["sh", "-c", shell_cmd])

Another technique (especially if you need to run many commands in the shell at once) is writing all commands down into a temporary file and executing it with the shell, see for example pmb/chroot/initfs.py.

Writing files to the chroot

The users in the chroots (root and pmos) have different user IDs than the user of the host system. Therefore we can't just write a file to anywhere in the chroot. But we can write to /tmp and move the file afterwards:

with open("tmp/somefile", "w") as handle:
    handle.write("It works")
pmb.chroot.root(args, ["mv", "/tmp/somefile", "/etc/somefile"])

See also