[bug#56298] python-cmsis-pack-manager and dependencies

[bug#56298] python-cmsis-pack-manager and dependencies

[Peter Polidoro]

Patch series to follow.

[bug#56298] [PATCH 0/7] Add python-cmsis-pack-manager and dependencies

[peter]

From: Peter Polidoro <peter@polidoro.io>

I attempted to package python-cmsis-pack-manager, but I was not sure how to
create the package properly since it is a combination of Python and Rust
code. I think I included the dependencies properly, but there probably needs
to be some modifications to the build phases in order for the rust code to get
compiled properly and placed somewhere the Python code can find it. Can
someone please take a look when you have a chance and help me finish the
package? This package is itself a dependency for python-pyocd, which I hope to
package to enable flashing of embedded code to various boards using only Guix
packages. Thanks!

Peter Polidoro (7):
  gnu: Add python-milksnake.
  gnu: Add rust-quick-xml-0.17.
  gnu: Add rust-minidom-0.12.
  gnu: rust-rustversion-1: Update to 1.0.7.
  gnu: rust-anyhow-1: Update to 1.0.58.
  gnu: Add rust-loom-0.5.
  gnu: Add python-cmsis-pack-manager.

 gnu/packages/crates-io.scm  | 90 +++++++++++++++++++++++++++++++++----
 gnu/packages/embedded.scm   | 81 +++++++++++++++++++++++++++++++++
 gnu/packages/python-xyz.scm | 21 +++++++++
 3 files changed, 183 insertions(+), 9 deletions(-)

-- 
2.36.1

[bug#56298] [PATCH 1/7] gnu: Add python-milksnake.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/python-xyz.scm (python-milksnake): New variable.
---
 gnu/packages/python-xyz.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/python-xyz.scm b/gnu/packages/python-xyz.scm
index 1d43049776..dbe8dff142 100644
--- a/gnu/packages/python-xyz.scm
+++ b/gnu/packages/python-xyz.scm
@@ -3060,6 +3060,27 @@ (define-public python-mimeparse
 matching them against a list of media-ranges.")
     (license license:expat)))
 
+(define-public python-milksnake
+  (package
+    (name "python-milksnake")
+    (version "0.1.5")
+    (source (origin
+              (method url-fetch)
+              (uri (pypi-uri "milksnake" version ".zip"))
+              (sha256
+               (base32
+                "120nprd8lqis7x7zy72536gk2j68f7gxm8gffmx8k4ygifvl7kfz"))))
+    (build-system python-build-system)
+    (propagated-inputs (list python-cffi))
+    (native-inputs (list unzip))
+    (home-page "https://github.com/getsentry/milksnake")
+    (synopsis
+     "A python library that extends setuptools for binary extensions.")
+    (description
+     "This package provides a python library that extends setuptools for binary
+extensions.")
+    (license license:asl2.0)))
+
 (define-public python-miniboa
   (package
     (name "python-miniboa")
-- 
2.36.1

[bug#56298] [PATCH 2/7] gnu: Add rust-quick-xml-0.17.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/crates-io.scm (rust-quick-xml-0.17): New variable.
---
 gnu/packages/crates-io.scm | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index f9bc9b42de..cdef764993 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -27,6 +27,7 @@
 ;;; Copyright © 2022 Aleksandr Vityazev <avityazev@posteo.org>
 ;;; Copyright © 2022 Marius Bakke <marius@gnu.org>
 ;;; Copyright © 2022 Evgenii Lepikhin <e.lepikhin@corp.mail.ru>
+;;; Copyright © 2022 Peter Polidoro <peter@polidoro.io>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -46159,6 +46160,25 @@ (define-public rust-quick-xml-0.19
         ("rust-memchr" ,rust-memchr-2)
         ("rust-serde" ,rust-serde-1))))))
 
+(define-public rust-quick-xml-0.17
+  (package
+    (inherit rust-quick-xml-0.20)
+    (name "rust-quick-xml")
+    (version "0.17.0")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (crate-uri "quick-xml" version))
+       (file-name (string-append name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1vvjqq2q459jifb0jnn1j22sq9l7m19g0vln7m4pa4npr2ldpz5a"))))
+    (arguments
+     `(#:skip-build? #t
+       #:cargo-inputs
+       (("rust-encoding-rs" ,rust-encoding-rs-0.8)
+        ("rust-memchr" ,rust-memchr-2)
+        ("rust-serde" ,rust-serde-1))))))
+
 (define-public rust-quick-xml-0.12
   (package
     (inherit rust-quick-xml-0.20)
-- 
2.36.1

[bug#56298] [PATCH 3/7] gnu: Add rust-minidom-0.12.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/crates-io.scm (rust-minidom-0.12): New variable.
---
 gnu/packages/crates-io.scm | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index cdef764993..c4a975afd3 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -11679,6 +11679,27 @@ (define-public rust-cmake-0.1
     (license (list license:asl2.0
                    license:expat))))
 
+(define-public rust-minidom-0.12
+  (package
+    (name "rust-minidom")
+    (version "0.12.0")
+    (source (origin
+              (method url-fetch)
+              (uri (crate-uri "minidom" version))
+              (file-name (string-append name "-" version ".tar.gz"))
+              (sha256
+               (base32
+                "06nbqscsv2clc4mvdzzl1syn89plsqvmxn2lqxjfrxbllqar2m7y"))))
+    (build-system cargo-build-system)
+    (arguments
+     `(#:cargo-inputs (("rust-quick-xml" ,rust-quick-xml-0.17))))
+    (home-page "https://gitlab.com/xmpp-rs/xmpp-rs")
+    (synopsis
+     "DOM implementation on top of quick-xml")
+    (description
+     "This package provides a minimal DOM implementation on top of quick-xml
+targeting the subset of XML useful for XMPP.")
+    (license license:mpl2.0)))
 (define-public rust-cocoa-0.24
   (package
     (name "rust-cocoa")
-- 
2.36.1

[bug#56298] [PATCH 4/7] gnu: rust-rustversion-1: Update to 1.0.7.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/crates-io.scm (rust-rustversion-1): Update to 1.0.7.
---
 gnu/packages/crates-io.scm | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index c4a975afd3..d309d212b9 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -51871,7 +51871,7 @@ (define-public rust-rusttype-0.7
 (define-public rust-rustversion-1
   (package
     (name "rust-rustversion")
-    (version "1.0.2")
+    (version "1.0.7")
     (source
      (origin
        (method url-fetch)
@@ -51880,13 +51880,10 @@ (define-public rust-rustversion-1
         (string-append name "-" version ".tar.gz"))
        (sha256
         (base32
-         "1xkr1g792w728py2qpg2zj0vfviv2xzmxkkd9w6035l9d5ss3fxk"))))
+         "1krzk0w1ii52km06qkggwszj4i4836ybbk0wka989lpm533zg9d0"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:cargo-inputs
-       (("rust-proc-macro2" ,rust-proc-macro2-1)
-        ("rust-quote" ,rust-quote-1)
-        ("rust-syn" ,rust-syn-1))))
+     `(#:cargo-development-inputs (("rust-trybuild" ,rust-trybuild-1))))
     (home-page "https://github.com/dtolnay/rustversion")
     (synopsis "Conditional compilation according to rustc compiler version")
     (description
-- 
2.36.1

[bug#56298] [PATCH 5/7] gnu: rust-anyhow-1: Update to 1.0.58.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/crates-io.scm (rust-anyhow-1): Update to 1.0.58.
---
 gnu/packages/crates-io.scm | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index d309d212b9..155d693430 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -2839,7 +2839,7 @@ (define-public rust-antidote-1
 (define-public rust-anyhow-1
   (package
     (name "rust-anyhow")
-    (version "1.0.46")
+    (version "1.0.58")
     (source
      (origin
        (method url-fetch)
@@ -2848,12 +2848,14 @@ (define-public rust-anyhow-1
         (string-append name "-" version ".tar.gz"))
        (sha256
         (base32
-         "0w0zm1bqk5kbk834r4xszlzqiln4vw5k1lnlswyjkh24khi2ia1s"))))
+         "014pw61axl4idagjdh9x96b06cy12vqs5599mzi11gnd7h2x41xv"))))
     (build-system cargo-build-system)
     (arguments
-     `(#:cargo-development-inputs
+     `(#:cargo-inputs (("rust-backtrace" ,rust-backtrace-0.3))
+       #:cargo-development-inputs
        (("rust-futures" ,rust-futures-0.3)
         ("rust-rustversion" ,rust-rustversion-1)
+        ("rust-syn" ,rust-syn-1)
         ("rust-thiserror" ,rust-thiserror-1)
         ("rust-trybuild" ,rust-trybuild-1))))
     (home-page "https://github.com/dtolnay/anyhow")
-- 
2.36.1

[bug#56298] [PATCH 6/7] gnu: Add rust-loom-0.5.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/crates-io.scm (rust-loom-0.5): New variable.
---
 gnu/packages/crates-io.scm | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/gnu/packages/crates-io.scm b/gnu/packages/crates-io.scm
index 155d693430..c1a54bdecf 100644
--- a/gnu/packages/crates-io.scm
+++ b/gnu/packages/crates-io.scm
@@ -32646,6 +32646,38 @@ (define-public rust-logtest-2
     (description "This package tests and asserts log statements.")
     (license (list license:expat license:asl2.0))))
 
+(define-public rust-loom-0.5
+  (package
+    (name "rust-loom")
+    (version "0.5.6")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (crate-uri "loom" version))
+       (file-name (string-append name "-" version ".tar.gz"))
+       (sha256
+        (base32 "1x9hmiv00ir79kypcg0jdw0j0fkd3ymq6rpv6pli6q5qifrfql7z"))))
+    (build-system cargo-build-system)
+    (arguments
+     ;; FIXME: build phase fails with the error: "the
+     ;; `#[track_caller]` attribute is an experimental feature".
+     `(#:skip-build? #true
+       #:cargo-inputs
+       (("rust-cfg-if" ,rust-cfg-if-1)
+        ("rust-futures-util" ,rust-futures-util-0.3)
+        ("rust-generator" ,rust-generator-0.6)
+        ("rust-scoped-tls" ,rust-scoped-tls-1)
+        ("rust-serde" ,rust-serde-1)
+        ("rust-serde-json" ,rust-serde-json-1))))
+    (home-page "https://github.com/tokio-rs/loom")
+    (synopsis "Permutation testing for concurrent code")
+    (description
+     "Loom is a testing tool for concurrent Rust code.  It runs a test many
+times, permuting the possible concurrent executions of that test under the C11
+memory model.  It uses state reduction techniques to avoid combinatorial
+explosion.")
+    (license license:expat)))
+
 (define-public rust-loom-0.4
   (package
     (name "rust-loom")
-- 
2.36.1

[bug#56298] [PATCH 7/7] gnu: Add python-cmsis-pack-manager.

[peter]

From: Peter Polidoro <peter@polidoro.io>

* gnu/packages/embedded.scm (python-cmsis-pack-manager): New variable.
---
 gnu/packages/embedded.scm | 81 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 81 insertions(+)

diff --git a/gnu/packages/embedded.scm b/gnu/packages/embedded.scm
index 6a66fe4753..07087b0feb 100644
--- a/gnu/packages/embedded.scm
+++ b/gnu/packages/embedded.scm
@@ -12,6 +12,7 @@
 ;;; Copyright © 2021 Morgan Smith <Morgan.J.Smith@outlook.com>
 ;;; Copyright © 2022 Mathieu Othacehe <othacehe@gnu.org>
 ;;; Copyright © 2022 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2022 Peter Polidoro <peter@polidoro.io>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -35,6 +36,7 @@ (define-module (gnu packages embedded)
   #:use-module (guix svn-download)
   #:use-module (guix git-download)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (guix build-system cargo)
   #:use-module (guix build-system cmake)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system python)
@@ -48,6 +50,7 @@ (define-module (gnu packages embedded)
   #:use-module (gnu packages boost)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
+  #:use-module (gnu packages crates-io)
   #:use-module (gnu packages cross-base)
   #:use-module (gnu packages dejagnu)
   #:use-module (gnu packages flex)
@@ -65,6 +68,7 @@ (define-module (gnu packages embedded)
   #:use-module (gnu packages python-web)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages rust)
   #:use-module (gnu packages swig)
   #:use-module (gnu packages texinfo)
   #:use-module (gnu packages version-control)
@@ -1580,6 +1584,83 @@ (define-public python-psptool
     (description "PSPTool is a tool for dealing with AMD binary blobs")
     (license license:gpl3+)))
 
+(define-public python-cmsis-pack-manager
+  (package
+    (name "python-cmsis-pack-manager")
+    (version "0.4.0")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/pyocd/cmsis-pack-manager")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "0gv8qnk4qfffa6pshm85jh5wi4rf2pfrxn2jdj8wa19adzdwnwag"))))
+    (build-system python-build-system)
+    (arguments
+     (list
+      #:imported-modules (append %cargo-build-system-modules
+                                 %python-build-system-modules)
+      #:modules `(((guix build cargo-build-system) #:prefix cargo:)
+                  ,@%python-build-system-modules
+                  (srfi srfi-1)
+                  (ice-9 match))
+      ;; #:phases
+      ;; #~(modify-phases (@ (guix build python-build-system) %standard-phases)
+      ;;     (add-before 'build 'configure-cargo
+      ;;       (lambda* (#:key inputs #:allow-other-keys)
+      ;;         ;; Hide irrelevant inputs from cargo-build-system so it does
+      ;;         ;; not try to unpack sanity-check.py, etc.
+      ;;         (let ((cargo-inputs (filter (match-lambda
+      ;;                                       ((name . path)
+      ;;                                        (or (string-prefix? "rust-" name)
+      ;;                                            (string=? "gcc" name))))
+      ;;                                     inputs)))
+      ;;           (with-directory-excursion "rust"
+      ;;             ((assoc-ref cargo:%standard-phases 'unpack-rust-crates)
+      ;;              #:inputs cargo-inputs
+      ;;              #:vendor-dir "guix-vendor")
+      ;;             ((assoc-ref cargo:%standard-phases 'configure)
+      ;;              #:inputs cargo-inputs)
+      ;;             ((assoc-ref cargo:%standard-phases 'patch-cargo-checksums)
+      ;;              #:vendor-dir "guix-vendor"))
+      ;;           (rename-file "rust/.cargo" ".cargo"))))
+      ;;     (replace 'check
+      ;;       (lambda* (#:key tests? #:allow-other-keys)
+      ;;         (when tests?
+      ;;           (invoke "pytest" "-vv" "tests"))))
+      ;;     )
+      ))
+    (inputs (list rust-anyhow-1
+                  rust-bytes-1
+                  rust-futures-0.3
+                  rust-log-0.4
+                  rust-loom-0.5
+                  rust-minidom-0.12
+                  rust-reqwest-0.11
+                  rust-rustc-demangle-0.1
+                  rust-serde-1
+                  rust-serde-json-1
+                  rust-tokio-1))
+    (propagated-inputs (list python-appdirs
+                             python-milksnake
+                             python-pyyaml))
+    (native-inputs (list python-hypothesis
+                         python-jinja2
+                         python-mock
+                         python-pytest
+                         rust
+                         `(,rust "cargo")))
+    (home-page "https://github.com/pyocd/cmsis-pack-manager")
+    (synopsis
+     "Python manager for CMSIS-Pack index and cache with Rust backend")
+    (description
+     "This package provides a python module, Rust crate, and command line utility
+for managing current device information that is stored in many Common
+Microcontroller Software Interface Standard (CMSIS) Packs.")
+    (license license:asl2.0)))
+
 (define-public agent-proxy
   (let ((commit "8927798a71d246871ea8fc22b4512296a3fa1765")
         (revision "0"))
-- 
2.36.1

[bug#56298] [PATCH 2/7] gnu: Add rust-quick-xml-0.17.

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1001 bytes --]


On 29-06-2022 16:46, peter@polidoro.io wrote:
> +     `(#:skip-build? #t
> +       #:cargo-inputs
> +       (("rust-encoding-rs" ,rust-encoding-rs-0.8)
> +        ("rust-memchr" ,rust-memchr-2)
> +        ("rust-serde" ,rust-serde-1))))))

The #:cargo-development-inputs are important for antioxidant (the new 
build system intended to eventually replace cargo-build-system), to 
actually run the tests. Try running "guix import crate quick-xml@0.17", 
IIUC it will add the development-inputs.

(I've sent a bug report about "guix import crate ..." removing 
cargo-development-inputs in the past, but I cannot find it anymore).

Also, likely you don't actually need the old version -- going by the 
code in antioxidant, always using rust-quick-xml@0.22 (instead of older 
versions) does not cause any problems, but since this is 
cargo-build-system, you will have to patch the Cargo.toml of the 
dependent to accept the latest version of rust-quick-xml.

Greetings,
Maxime.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[bug#56298] [PATCH 5/7] gnu: rust-anyhow-1: Update to 1.0.58.

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 953 bytes --]


On 29-06-2022 16:46, peter@polidoro.io wrote:
>       (build-system cargo-build-system)
>       (arguments
> -     `(#:cargo-development-inputs
> +     `(#:cargo-inputs (("rust-backtrace" ,rust-backtrace-0.3))
> +       #:cargo-development-inputs
>          (("rust-futures" ,rust-futures-0.3)
>           ("rust-rustversion" ,rust-rustversion-1)
> +        ("rust-syn" ,rust-syn-1)
>           ("rust-thiserror" ,rust-thiserror-1)
>           ("rust-trybuild" ,rust-trybuild-1))))

I assume you updated rust-rustversion to update rust-anyhow?  If so, I 
have looked at the Cargo.toml of anyhow. It is very picky in the version 
of dependents, so if in the future we update rust-rustversion again, 
rust-anyhow will fail to build. To be robust to that, the Cargo.toml 
needs to be patched to be less picky, or alternatively you could wait 
for antioxidant which ignores the version specifications entirely.

Greetings,
Maxime.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[bug#56298] [PATCH 3/7] gnu: Add rust-minidom-0.12.

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 109 bytes --]

A patch for this has already been submitted, at 
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49451#32>.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

[bug#56298] [PATCH 5/7] gnu: rust-anyhow-1: Update to 1.0.58.

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1434 bytes --]


On 27-07-2022 18:01, Maxime Devos wrote:
> I assume you updated rust-rustversion to update rust-anyhow?  If so, I 
> have looked at the Cargo.toml of anyhow. It is very picky in the 
> version of dependents, so if in the future we update rust-rustversion 
> again, rust-anyhow will fail to build. To be robust to that, the 
> Cargo.toml needs to be patched to be less picky, or alternatively you 
> could wait for antioxidant which ignores the version specifications 
> entirely. 

Nevermind, from 
<https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html>:

> The string |"0.1.12"| is a version requirement. Although it looks like 
> a specific /version/ of the |time| crate, it actually specifies a 
> /range/ of versions and allows SemVer <https://semver.org> compatible 
> updates. An update is allowed if the new version number does not 
> modify the left-most non-zero digit in the major, minor, patch 
> grouping. In this case, if we ran |cargo update -p time|, cargo should 
> update us to version |0.1.13| if it is the latest |0.1.z| release, but 
> would not update us to |0.2.0|. If instead we had specified the 
> version string as |1.0|, cargo should update to |1.1| if it is the 
> latest |1.y| release, but not |2.0|. The version |0.0.x| is not 
> considered compatible with any other version.
>
(if it used = or <=, that would be a different manner)

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 2491 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]