[bug#49577] [PATCH] Add python-pmbootstrap

[bug#49577] [PATCH] Add python-pmbootstrap

[phodina via Guix-patches via]

---
index d8c3fbec07..0ccd115416 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -985,6 +985,31 @@ and URL-safe UUIDs.  UUIDs are generated using the built-in Python @code{uuid}
 module and then similar looking characters are removed.")
     (license license:bsd-3)))

+(define-public python-pmbootstrap
+  (package
+    (name "python-pmbootstrap")
+    (version "1.32.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "pmbootstrap" version))
+       (sha256
+        (base32
+         "1j796f4wrcdrjd7j9m3w3ziiyqq17kskil9bgy893j6sk73fc5q0"))))
+    (build-system python-build-system)
+    ;; TODO: Running tests fails
+    (arguments
+     '(#:tests? #f))
+    (inputs
+     `(("python-pytest" ,python-pytest) ("python-pyopenssl" ,python-pyopenssl)))
+    (propagated-inputs `(("openssl" ,openssl)))
+    (home-page "https://www.postmarketos.org")
+    (synopsis
+     "A sophisticated chroot / build / flash tool to develop and install postmarketOS")
+    (description
+     "A sophisticated chroot / build / flash tool to develop and install postmarketOS")
+    (license #f)))
+
 (define-public python-logwrap
   (package
     (name "python-logwrap")
--
2.31.1

[bug#49577] [PATCH] Add python-pmbootstrap

[phodina via Guix-patches via]

Here's updated patch with better package description:

---
index d8c3fbec07..a59d065efa 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -985,6 +985,44 @@ and URL-safe UUIDs.  UUIDs are generated using the built-in Python @code{uuid}
 module and then similar looking characters are removed.")
     (license license:bsd-3)))

+(define-public python-pmbootstrap
+  (package
+    (name "python-pmbootstrap")
+    (version "1.32.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "pmbootstrap" version))
+       (sha256
+        (base32
+         "1j796f4wrcdrjd7j9m3w3ziiyqq17kskil9bgy893j6sk73fc5q0"))))
+    (build-system python-build-system)
+    ;; TODO: Running tests fails
+    (arguments
+     '(#:tests? #f))
+    (inputs
+     `(("python-pytest" ,python-pytest) ("python-pyopenssl" ,python-pyopenssl)))
+    (propagated-inputs `(("openssl" ,openssl)))
+    (home-page "https://www.postmarketos.org")
+    (synopsis
+     "A sophisticated chroot / build / flash tool to develop and install
+postmarketOS")
+    (description
+     "Bootstrap program that abstracts everything in chroots and therefore
+basically runs on top of any Linux distribution. Features:
+@enumerate
+@item chroot setup (distro-independent QEMU user emulation
+@item clean chroot shutdown (umount) and zapping
+@item build software as packages
+@item cross-compile all armhf-packages
+@item effective caching out of the box (survives chroot zaps)
+@item installation targets
+@item flasher abstractions
+@item logging
+@item security
+@end enumerate")
+    (license #f)))
+
 (define-public python-logwrap
   (package
     (name "python-logwrap")
--
2.31.1

[bug#49577] [PATCH] Add python-pmbootstrap

[Sarah Morgensen]

Hi!

Thanks for the patch. I have some suggestions for this one as well.

phodina <phodina@protonmail.com> writes:

> Here's updated patch with better package description:
>
> ---
> index d8c3fbec07..a59d065efa 100644
> --- a/gnu/packages/python-xyz.scm
> +++ b/gnu/packages/python-xyz.scm
> @@ -985,6 +985,44 @@ and URL-safe UUIDs.  UUIDs are generated using the built-in Python @code{uuid}
>  module and then similar looking characters are removed.")
>      (license license:bsd-3)))
>
> +(define-public python-pmbootstrap
> +  (package
> +    (name "python-pmbootstrap")
> +    (version "1.32.0")
> +    (source
> +     (origin
> +       (method url-fetch)
> +       (uri (pypi-uri "pmbootstrap" version))
> +       (sha256
> +        (base32
> +         "1j796f4wrcdrjd7j9m3w3ziiyqq17kskil9bgy893j6sk73fc5q0"))))
> +    (build-system python-build-system)
> +    ;; TODO: Running tests fails
> +    (arguments
> +     '(#:tests? #f))

It is not a good idea to skip tests without knowing why they fail, as
there could be subtle bugs in the program even if it seems to run. In
this case, it seems like the reason they're failing is that the pypi
release strips some directories that are required for testing. Perhaps
try from the git repo?

> +    (inputs
> +     `(("python-pytest" ,python-pytest) ("python-pyopenssl" ,python-pyopenssl)))

Typically test packages are put in native-inputs rather than inputs.

> +    (propagated-inputs `(("openssl" ,openssl)))

Is this necessary? If pmbootstrap just uses openssl through pyopenssl,
it shouldn't be necessary.

> +    (home-page "https://www.postmarketos.org")
> +    (synopsis
> +     "A sophisticated chroot / build / flash tool to develop and install
> +postmarketOS")
> +    (description
> +     "Bootstrap program that abstracts everything in chroots and therefore
> +basically runs on top of any Linux distribution. Features:
> +@enumerate
> +@item chroot setup (distro-independent QEMU user emulation
> +@item clean chroot shutdown (umount) and zapping
> +@item build software as packages
> +@item cross-compile all armhf-packages
> +@item effective caching out of the box (survives chroot zaps)
> +@item installation targets
> +@item flasher abstractions
> +@item logging
> +@item security
> +@end enumerate")
> +    (license #f)))

Please make sure not to skip the license field. It looks like this one
is gpl3.

> +
>  (define-public python-logwrap
>    (package
>      (name "python-logwrap")
> --
> 2.31.1

--
Sarah

[bug#49577] [PATCH] Add python-pmbootstrap

[Maxim Cournoyer]

Hi phodina,

Sarah Morgensen <iskarian@mgsn.dev> writes:

> Hi!
>
> Thanks for the patch. I have some suggestions for this one as well.
>
> phodina <phodina@protonmail.com> writes:
>
>> Here's updated patch with better package description:
>>
>> ---
>> index d8c3fbec07..a59d065efa 100644
>> --- a/gnu/packages/python-xyz.scm
>> +++ b/gnu/packages/python-xyz.scm
>> @@ -985,6 +985,44 @@ and URL-safe UUIDs.  UUIDs are generated using the built-in Python @code{uuid}
>>  module and then similar looking characters are removed.")
>>      (license license:bsd-3)))
>>
>> +(define-public python-pmbootstrap
>> +  (package
>> +    (name "python-pmbootstrap")
>> +    (version "1.32.0")
>> +    (source
>> +     (origin
>> +       (method url-fetch)
>> +       (uri (pypi-uri "pmbootstrap" version))
>> +       (sha256
>> +        (base32
>> +         "1j796f4wrcdrjd7j9m3w3ziiyqq17kskil9bgy893j6sk73fc5q0"))))
>> +    (build-system python-build-system)
>> +    ;; TODO: Running tests fails
>> +    (arguments
>> +     '(#:tests? #f))
>
> It is not a good idea to skip tests without knowing why they fail, as
> there could be subtle bugs in the program even if it seems to run. In
> this case, it seems like the reason they're failing is that the pypi
> release strips some directories that are required for testing. Perhaps
> try from the git repo?
>
>> +    (inputs
>> +     `(("python-pytest" ,python-pytest) ("python-pyopenssl" ,python-pyopenssl)))
>
> Typically test packages are put in native-inputs rather than inputs.
>
>> +    (propagated-inputs `(("openssl" ,openssl)))
>
> Is this necessary? If pmbootstrap just uses openssl through pyopenssl,
> it shouldn't be necessary.
>
>> +    (home-page "https://www.postmarketos.org")
>> +    (synopsis
>> +     "A sophisticated chroot / build / flash tool to develop and install
>> +postmarketOS")
>> +    (description
>> +     "Bootstrap program that abstracts everything in chroots and therefore
>> +basically runs on top of any Linux distribution. Features:
>> +@enumerate
>> +@item chroot setup (distro-independent QEMU user emulation
>> +@item clean chroot shutdown (umount) and zapping
>> +@item build software as packages
>> +@item cross-compile all armhf-packages
>> +@item effective caching out of the box (survives chroot zaps)
>> +@item installation targets
>> +@item flasher abstractions
>> +@item logging
>> +@item security
>> +@end enumerate")
>> +    (license #f)))
>
> Please make sure not to skip the license field. It looks like this one
> is gpl3.
>
>> +
>>  (define-public python-logwrap
>>    (package
>>      (name "python-logwrap")

Gentle ping, since the above comments were 42 weeks ago :-)

Thanks,

Maxim

[bug#49577] [PATCH] Add python-pmbootstrap

[phodina via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 211 bytes --]

Hi,

here's an updated version of the patch + 2 more tools around PMOS.

However, there is some issue with the sanity-check phase in pmbootstrap which I disabled as I didn't know how to fix it.

----
Petr

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0003-gnu-Add-boot-deploy.patch --]
[-- Type: text/x-patch; name=0003-gnu-Add-boot-deploy.patch, Size: 2309 bytes --]

From 45fea1b10467dae4310064112b993037b7f827c3 Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Mon, 10 Jan 2022 02:54:17 +0100
Subject: [PATCH 3/3] gnu: Add boot-deploy.

* gnu/packages/android.scm (boot-deploy): New variable.

diff --git a/gnu/packages/android.scm b/gnu/packages/android.scm
index aa9b0e0fe6..0342e30cef 100644
--- a/gnu/packages/android.scm
+++ b/gnu/packages/android.scm
@@ -46,6 +46,7 @@ (define-module (gnu packages android)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (gnu packages)
   #:use-module (gnu packages admin)
+  #:use-module (gnu packages algebra)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages check)
@@ -1023,6 +1024,42 @@ (define-public abootimg
 safest way, on a file image.")
     (license license:gpl2+)))
 
+(define-public boot-deploy
+  (package
+    (name "boot-deploy")
+    (version "0.5")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://gitlab.com/postmarketOS/boot-deploy")
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "0vh8gjprsyp4zwmpgasf4gyxclm3yyr32pf39bf218690m86fijq"))))
+    (build-system gnu-build-system)
+    (arguments
+     (list #:phases
+           #~(modify-phases %standard-phases
+               (delete 'configure)
+               (delete 'build)
+               (replace 'check
+                 (lambda* (#:key tests? #:allow-other-keys)
+                   (when tests?
+                     (invoke "bash" "test_boot_deploy_functions.sh"))))
+               (replace 'install
+                 (lambda* _
+                   (install-file "boot-deploy"
+                                 (string-append #$output "/bin")))))))
+    (native-inputs (list bc))
+    (inputs (list bash-minimal))
+    (synopsis "Boot deploy for PostmarketOS devices")
+    (description
+     "This package provides tools to generate and deploy images for
+PostmarketOS devices.")
+    (home-page "https://gitlab.com/postmarketOS/boot-deploy")
+    (license license:gpl3+)))
+
 (define-public python-androguard
   (package
     (name "python-androguard")
-- 
2.35.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: 0001-gnu-Add-pmbootstrap.patch --]
[-- Type: text/x-patch; name=0001-gnu-Add-pmbootstrap.patch, Size: 3078 bytes --]

From 604e23fe8f2c624a70bdc26cd829dce539778323 Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sun, 18 Jul 2021 12:33:07 +0200
Subject: [PATCH 1/3] gnu: Add pmbootstrap.

* gnu/packages/python-xyz.scm (pmbootstrap): New variable.

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 6132019a53..8c33364c40 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -1178,6 +1178,65 @@ (define-public python-shortuuid
 module and then similar looking characters are removed.")
     (license license:bsd-3)))
 
+(define-public pmbootstrap
+  (package
+    (name "pmbootstrap")
+    (version "1.44.1")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "pmbootstrap" version))
+       (sha256
+        (base32
+         "0hzfgf085b4h2pmbgfkx25sxq9a27m1l77jx7in48vfv8lihmhzh"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:tests? #f
+       #:phases
+       #~(modify-phases %standard-phases
+         (add-after 'unpack 'replace-programs
+           (lambda* (#:key inputs #:allow-other-keys)
+             (substitute* "pmb/chroot/apk_static.py"
+               (("\"openssl\"") (string-append "\"" #$openssl
+                                               "/bin/openssl\"")))
+             (substitute* "pmb/config/__init__.py"
+               (("\"git\"") (string-append "\"" #$git
+                                           "/bin/git\""))
+               (("\"openssl\"") (string-append "\"" #$openssl
+                                               "/bin/openssl\""))
+               (("\"ps\"") (string-append "\"" #$procps
+                                          "/bin/ps\"")))))
+         (replace 'check
+           (lambda* (#:key tests? #:allow-other-keys)
+                                        ;(add-installed-pythonpath inputs outputs)
+             (when tests?
+               ;; To import pmb_test module
+               (setenv "PYTHONPATH"
+                       (string-append (assoc-ref %outputs "out") "/test/pmb_test:"
+                                      (getenv "PYTHONPATH")))
+               (invoke "pytest" "-vv"))))
+         ;; Circular dependency with pmbootstrap
+         (delete 'sanity-check))))
+    (native-inputs (list python-pytest))
+    (inputs (list git procps openssl))
+    (home-page "https://postmarketos.org")
+    (synopsis "Build and flash tool for postmarketOS")
+    (description
+     "Bootstrap program that abstracts everything in chroots and therefore
+basically runs on top of any Linux distribution.  Features:
+@enumerate
+@item chroot setup (distro-independent QEMU user emulation
+@item clean chroot shutdown (umount) and zapping
+@item build software as packages
+@item cross-compile all armhf-packages
+@item effective caching out of the box (survives chroot zaps)
+@item installation targets
+@item flasher abstractions
+@item logging
+@item security
+@end enumerate")
+    (license license:gpl3+)))
+
 (define-public python-logwrap
   (package
     (name "python-logwrap")
-- 
2.35.1


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: 0002-gnu-Add-pmos-installer.patch --]
[-- Type: text/x-patch; name=0002-gnu-Add-pmos-installer.patch, Size: 4795 bytes --]

From 8b1c26dfddf10200dbd72be13322fdd835d257aa Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sun, 9 Jan 2022 19:22:51 +0100
Subject: [PATCH 2/3] gnu: Add pmos-installer.

* gnu/packages/android.scm (pmos-installer): New variable.

diff --git a/gnu/packages/android.scm b/gnu/packages/android.scm
index 04dc4b6499..aa9b0e0fe6 100644
--- a/gnu/packages/android.scm
+++ b/gnu/packages/android.scm
@@ -11,7 +11,7 @@
 ;;; Copyright © 2019 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2020 Sergey Trofimov <sarg@sarg.org.ru>
 ;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
-;;; Copyright © 2021 Petr Hodina <phodina@protonmail.com>
+;;; Copyright © 2021, 2022 Petr Hodina <phodina@protonmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -32,16 +32,25 @@ (define-module (gnu packages android)
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix gexp)
+  #:use-module (guix utils)
+  #:use-module (ice-9 match)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
   #:use-module (guix git-download)
   #:use-module (guix build-system android-ndk)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
+  #:use-module (guix build-system copy)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (gnu packages)
+  #:use-module (gnu packages admin)
+  #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
+  #:use-module (gnu packages cpio)
   #:use-module (gnu packages docker)
   #:use-module (gnu packages gcc)
   #:use-module (gnu packages gnupg)
@@ -1268,3 +1277,64 @@ (define-public enjarify
 applications.  Enjarify can translate the Dalvik bytecode back to equivalent
 Java bytecode, which simplifies the analysis of Android applications.")
     (license license:asl2.0)))
+
+(define-public pmos-installer
+  (package
+    (name "pmos-installer")
+    (version "1.0.0")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://gitlab.com/sdm845-mainline/pmos-installer")
+                    (recursive? #t)
+                    (commit version)))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "0lbdk2453bywdibys35zyf1j27a1m11a7q8jvmgjf99v77l9rrbx"))))
+    (build-system copy-build-system)
+    (arguments
+     `(#:install-plan '(("makeinstaller.sh" "bin/makeinstaller.sh") ("initrd"
+                                                                     "initrd"))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'install 'wrap-shellscript
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out")))
+               (substitute* (string-append out "/bin/makeinstaller.sh")
+                 ;; Typo in name
+                 ;; https://gitlab.com/sdm845-mainline/pmos-installer/-/issues/5
+                 (("vmlinuz-") "vmlinuz")
+                 ;; Path to initrd
+                 (("\"\\$\\(dirname \"\\$0\"\\)\"") (string-append out)))
+               (wrap-program (string-append out "/bin/makeinstaller.sh")
+                             `("PATH" ":" prefix
+                               ,(append (map (lambda (dir)
+                                               (string-append (assoc-ref
+                                                               inputs dir)
+                                                              "/bin"))
+                                             '("coreutils" "cpio"
+                                               "grep"
+                                               "zip"
+                                               "mkbootimg"
+                                               "pmbootstrap"
+                                               "sudo"
+                                               "sed"
+                                               "util-linux"
+                                               "which")))))))))))
+    (inputs (list bash-minimal
+                  coreutils
+                  pmbootstrap
+                  grep
+                  util-linux
+                  mkbootimg
+                  cpio
+                  zip
+                  sed
+                  which
+                  sudo))
+    (home-page "https://gitlab.com/sdm845-mainline/pmos-installer")
+    (synopsis "Flashable installer for postmarketOS")
+    (description "This package provides a flashable installer for
+        postmarketOS.")
+    (license license:gpl3+)))
-- 
2.35.1

[bug#49577] [PATCH] Add python-pmbootstrap

[phodina via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 567 bytes --]

Hi Maxim,

here's update of the package. I moved to git repo as it has test suite packaged.

However, currently 44 tests fail, 80 pass.

The reason is they require network connectivity (mostly git tests, can be added to list of dependencies), then some require chroot (that might be dropped and run in guix isolated environment), sudo (again remove and use TMP location).

The tool is able to produce working PostmarketOS image which can be run on mobile device.

Not sure about the time it will take to patch all the tests. What do you think?

----
Petr

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v4-0001-gnu-Add-pmbootstrap.patch --]
[-- Type: text/x-patch; name=v4-0001-gnu-Add-pmbootstrap.patch, Size: 3195 bytes --]

From 5443a12dc08936a7facab4b1b85953624a29df24 Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sun, 18 Jul 2021 12:33:07 +0200
Subject: [PATCH v4] gnu: Add pmbootstrap.

* gnu/packages/python-xyz.scm (pmbootstrap): New variable.

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 17fbd9de9e..7f409cf118 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -153,6 +153,7 @@ (define-module (gnu packages python-xyz)
   #:use-module (gnu packages)
   #:use-module (gnu packages algebra)
   #:use-module (gnu packages adns)
+  #:use-module (gnu packages admin)
   #:use-module (gnu packages aidc)
   #:use-module (gnu packages attr)
   #:use-module (gnu packages backup)
@@ -1458,6 +1459,56 @@ (define-public python-shortuuid
 module and then similar looking characters are removed.")
     (license license:bsd-3)))
 
+(define-public pmbootstrap
+  (package
+    (name "pmbootstrap")
+    (version "1.50.1")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://gitlab.com/postmarketOS/pmbootstrap")
+                    (commit version)))
+              (sha256
+               (base32
+                "0n26zbpw7nhyh9gq0ydgij7m4pgv2n5wlmyc7g2sr7gqhqv04j2j"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:tests? #f ; TODO: many test fail
+           #:phases #~(modify-phases %standard-phases
+                        (add-after 'install 'fix-paths
+                          (lambda* _
+                            (let ((git (string-append #$git "/bin/"))
+                                  (procps (string-append #$procps "/bin"))
+                                  (openssl (string-append #$openssl "/bin"))
+                                  (sudo "/run/setuid-programs"))
+                              (wrap-program (string-append #$output
+                                                           "/bin/pmbootstrap")
+                                            `("PATH" ":" suffix
+                                              ,(list git procps openssl sudo))))))
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (invoke "pytest")))))))
+    (native-inputs (list python-pytest python-pyopenssl))
+    (inputs (list git procps openssl sudo))
+    (home-page "https://postmarketos.org")
+    (synopsis "Build and flash tool for postmarketOS")
+    (description
+     "Bootstrap program that abstracts everything in chroots and therefore
+basically runs on top of any Linux distribution. Features:
+@enumerate
+@item chroot setup (distro-independent QEMU user emulation
+@item clean chroot shutdown (umount) and zapping
+@item build software as packages
+@item cross-compile all armhf-packages
+@item effective caching out of the box (survives chroot zaps)
+@item installation targets
+@item flasher abstractions
+@item logging
+@item security
+@end enumerate")
+    (license license:gpl3+)))
+
 (define-public python-logwrap
   (package
     (name "python-logwrap")
-- 
2.37.2

[bug#49577] [PATCH] Add python-pmbootstrap

[Maxim Cournoyer]

Hi Petr,

phodina <phodina@protonmail.com> writes:

> Hi Maxim,
>
> here's update of the package. I moved to git repo as it has test suite packaged.

Thank you!

> However, currently 44 tests fail, 80 pass.
>
> The reason is they require network connectivity (mostly git tests, can
> be added to list of dependencies), then some require chroot (that
> might be dropped and run in guix isolated environment), sudo (again
> remove and use TMP location).

Oh.  That's more than half the tests failing.  I guess the whole test
suite could be disabled with a comment, if possible with a link
requesting upstream to make it easy to skip network-requiring tests via
a flag or something.

-- 
Thanks,
Maxim

[bug#49577] [PATCH] Add python-pmbootstrap

[Maxim Cournoyer]

tags 49577 +moreinfo
quit

Hi Jorge,

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi Petr,
>
> phodina <phodina@protonmail.com> writes:
>
>> Hi Maxim,
>>
>> here's update of the package. I moved to git repo as it has test suite packaged.
>
> Thank you!
>
>> However, currently 44 tests fail, 80 pass.
>>
>> The reason is they require network connectivity (mostly git tests, can
>> be added to list of dependencies), then some require chroot (that
>> might be dropped and run in guix isolated environment), sudo (again
>> remove and use TMP location).
>
> Oh.  That's more than half the tests failing.  I guess the whole test
> suite could be disabled with a comment, if possible with a link
> requesting upstream to make it easy to skip network-requiring tests via
> a flag or something.

The status is unchanged; the ball is in Petr's camp, or someone
interested enough in the patch to make the changes suggested.

I'm using this opportunity to gently ping Petr :-).

-- 
Thanks,
Maxim

[bug#49577] [PATCH] Add python-pmbootstrap

[phodina via Guix-patches via]

[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]

Hi Jorge and Maxim,

unfreezing this patch. So I've modified the patch to list the tests that should not be run based on the Nix example.

The package is also updated to the latest version.

However, I'm struggling with the import of the `pmb_test` module.

I've modified the variable `PYTHONPATH` accordingly to fix the error:
```
E   ModuleNotFoundError: No module named 'pmb_test'
``` 

But now I get the following error:
```
wrong-type-arg "string-append" "Wrong type (expecting ~A): ~S" ("string" #f) (#f)
```

So it seems the `PYTHONPATH` env variable is not set. Do you know how to fix this?


----
Petr




Sent with Proton Mail secure email.

------- Original Message -------
On Thursday, May 25th, 2023 at 2:16 AM, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:


> tags 49577 +moreinfo
> quit
> 
> Hi Jorge,
> 
> Maxim Cournoyer maxim.cournoyer@gmail.com writes:
> 
> > Hi Petr,
> > 
> > phodina phodina@protonmail.com writes:
> > 
> > > Hi Maxim,
> > > 
> > > here's update of the package. I moved to git repo as it has test suite packaged.
> > 
> > Thank you!
> > 
> > > However, currently 44 tests fail, 80 pass.
> > > 
> > > The reason is they require network connectivity (mostly git tests, can
> > > be added to list of dependencies), then some require chroot (that
> > > might be dropped and run in guix isolated environment), sudo (again
> > > remove and use TMP location).
> > 
> > Oh. That's more than half the tests failing. I guess the whole test
> > suite could be disabled with a comment, if possible with a link
> > requesting upstream to make it easy to skip network-requiring tests via
> > a flag or something.
> 
> 
> The status is unchanged; the ball is in Petr's camp, or someone
> interested enough in the patch to make the changes suggested.
> 
> I'm using this opportunity to gently ping Petr :-).
> 
> --
> Thanks,
> Maxim

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: v2-0001-gnu-Add-pmbootstrap.patch --]
[-- Type: text/x-patch; name=v2-0001-gnu-Add-pmbootstrap.patch, Size: 7560 bytes --]

From ec3193bfa3320b6b282926807bfb2501362fe11d Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sun, 18 Jul 2021 12:33:07 +0200
Subject: [PATCH v2] gnu: Add pmbootstrap.

* gnu/packages/python-xyz.scm (pmbootstrap): New variable.

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index f7e3f6f538..b14637af73 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -158,6 +158,7 @@ (define-module (gnu packages python-xyz)
   #:use-module (gnu packages)
   #:use-module (gnu packages algebra)
   #:use-module (gnu packages adns)
+  #:use-module (gnu packages admin)
   #:use-module (gnu packages aidc)
   #:use-module (gnu packages attr)
   #:use-module (gnu packages backup)
@@ -1744,6 +1745,115 @@ (define-public python-shortuuid
 module and then similar looking characters are removed.")
     (license license:bsd-3)))
 
+(define-public pmbootstrap
+  (package
+    (name "pmbootstrap")
+    (version "1.53.0")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "pmbootstrap" version))
+              (sha256
+               (base32
+                "13nyjrwi5r8ma9i58cz6qbgy2xyjzs708nq65kdmhrw3ba5z3jgj"))))
+    (build-system python-build-system)
+    (arguments
+     (list #:phases #~(modify-phases %standard-phases
+                        (add-after 'install 'fix-paths
+                          (lambda* _
+                            (let ((git (string-append #$git "/bin/"))
+                                  (procps (string-append #$procps "/bin"))
+                                  (openssl (string-append #$openssl "/bin"))
+                                  (sudo "/run/setuid-programs"))
+                              (wrap-program (string-append #$output
+                                                           "/bin/pmbootstrap")
+                                `("PATH" ":" suffix
+                                  ,(list git procps openssl sudo))))))
+                        (replace 'check
+                          (lambda* (#:key tests? #:allow-other-keys)
+                            (when tests?
+                              (setenv "PYTHONPATH"
+                                      (string-append (getcwd) "/test:"
+                                                     (getenv "PYTHONPATH")))
+                              (invoke "pytest" "-k"
+                                      (string-append "not test_apk_static"
+                                       " and not test_aportgen"
+                                       "test_aportgen_device_wizard"
+                                       " and not test_bootimg"
+                                       " and not test_build_depends_binary_outdated"
+                                       " and not test_build_depends_high_level"
+                                       " and not test_build_depends_no_binary_error"
+                                       " and not test_build_is_necessary"
+                                       " and not test_build_local_source_high_level"
+                                       " and not test_build_src_invalid_path"
+                                       " and not test_can_fast_forward"
+                                       " and not test_check_build_for_arch"
+                                       " and not test_chroot_arguments"
+                                       " and not test_chroot_interactive_shell"
+                                       " and not test_chroot_interactive_shell_user"
+                                       " and not test_clean_worktree"
+                                       " and not test_config_user"
+                                       " and not test_cross_compile_distcc"
+                                       " and not test_crossdirect"
+                                       " and not test_file"
+                                       " and not test_filter_aport_packages"
+                                       " and not test_filter_missing_packages_binary_exists"
+                                       " and not test_filter_missing_packages_invalid"
+                                       " and not test_filter_missing_packages_pmaports"
+                                       " and not test_finish"
+                                       " and not test_folder_size"
+                                       " and not test_get_apkbuild"
+                                       " and not test_get_depends"
+                                       " and not test_get_upstream_remote"
+                                       " and not test_helpers_lint"
+                                       " and not test_helpers_package_get_apkindex"
+                                       " and not test_helpers_repo"
+                                       " and not test_helpers_ui"
+                                       " and not test_init_buildenv"
+                                       " and not test_kconfig_check"
+                                       " and not test_keys"
+                                       " and not test_newapkbuild"
+                                       " and not test_package"
+                                       " and not test_package_from_aports"
+                                       " and not test_pkgrel_bump"
+                                       " and not test_pmbootstrap_status"
+                                       " and not test_print_checks_git_repo"
+                                       " and not test_pull"
+                                       " and not test_qemu_running_processes"
+                                       " and not test_questions_additional_options"
+                                       " and not test_questions_bootimg"
+                                       " and not test_questions_channel"
+                                       " and not test_questions_keymaps"
+                                       " and not test_questions_work_path"
+                                       " and not test_read_config_channel"
+                                       " and not test_recurse_invalid"
+                                       " and not test_run_abuild"
+                                       " and not test_run_core"
+                                       " and not test_shell_escape"
+                                       " and not test_skip_already_built"
+                                       " and not test_switch_to_channel_branch"
+                                       " and not test_version")))))
+                        ;; Circular dependency with pmbootstrap
+                        (delete 'sanity-check))))
+    (native-inputs (list python-pytest python-pyopenssl))
+    (inputs (list git procps openssl sudo))
+    (home-page "https://postmarketos.org")
+    (synopsis "Build and flash tool for postmarketOS")
+    (description
+     "Bootstrap program that abstracts everything in chroots and therefore
+basically runs on top of any Linux distribution. Features:
+@enumerate
+@item chroot setup (distro-independent QEMU user emulation
+@item clean chroot shutdown (umount) and zapping
+@item build software as packages
+@item cross-compile all armhf-packages
+@item effective caching out of the box (survives chroot zaps)
+@item installation targets
+@item flasher abstractions
+@item logging
+@item security
+@end enumerate")
+    (license license:gpl3+)))
+
 (define-public python-logwrap
   (package
     (name "python-logwrap")
-- 
2.39.1

[bug#49577] [PATCH] Add python-pmbootstrap

[Maxim Cournoyer]

Hi Petr,

phodina <phodina@protonmail.com> writes:

> Hi Jorge and Maxim,
>
> unfreezing this patch. So I've modified the patch to list the tests that should not be run based on the Nix example.
>
> The package is also updated to the latest version.
>
> However, I'm struggling with the import of the `pmb_test` module.
>
> I've modified the variable `PYTHONPATH` accordingly to fix the error:
> ```
> E   ModuleNotFoundError: No module named 'pmb_test'
> ``` 
>
> But now I get the following error:
> ```
> wrong-type-arg "string-append" "Wrong type (expecting ~A): ~S" ("string" #f) (#f)
> ```
>
> So it seems the `PYTHONPATH` env variable is not set. Do you know how to fix this?

Maybe because we've moved to use GUIX_PYTHONPATH and not PYTHONPATH?

-- 
Thanks,
Maxim

[bug#49577] [PATCH] Add python-pmbootstrap

[jgart via Guix-patches via]

Hi phodina,

I would try Maxim's suggestion. 

Otherwise, that might be a good Lars question... CC'ing

I would have to read through more code to affirm that for you and I don't have the time at the moment to read down that particular rabbit hole.

Maybe a this weekend project.

There's also a pmbootstrap in guixrus if that helps to crosscheck anything useful or not for this patch:

https://toys.whereis.みんな/?search=pmbootstrap

hope this helps and excuse the lack of further investigation at this moment on my end,

jgart

[bug#49577]

[jgart via Guix-patches via]

Hi,

What's the status on this?

Regarding the tests, you can try disabling the same ones that are disabled in the nixpkgs pmboostrap:

https://github.com/NixOS/nixpkgs/blob/nixos-unstable/pkgs/tools/misc/pmbootstrap/default.nix#L31