[bug#66690] [PATCH 0/7] Upgrade cwltool

[bug#66690] [PATCH 0/7] Upgrade cwltool

[Arun Isaac]

This patchset upgrades cwltool to the latest version.

Arun Isaac (7):
  gnu: python-ruamel.yaml.clib: Update to 0.2.7.
  gnu: python-ruamel.yaml: Update to 0.17.40.
  gnu: python-schema-salad: Update to 8.4.20230927144413.
  gnu: Add python-cwlformat.
  gnu: Add python-cwl-upgrader.
  gnu: Add python-cwl-utils.
  gnu: cwltool: Update to 3.1.20230601100705.

 gnu/packages/bioinformatics.scm | 232 ++++++++++++++++++++++++--------
 gnu/packages/serialization.scm  |   9 +-
 2 files changed, 178 insertions(+), 63 deletions(-)


base-commit: fb3707762de577074c73d40575fe759095ef83e6
-- 
2.41.0

[bug#66690] [PATCH 1/7] gnu: python-ruamel.yaml.clib: Update to 0.2.7.

[Arun Isaac]

* gnu/packages/serialization.scm (python-ruamel.yaml.clib): Update to 0.2.7.
---
 gnu/packages/serialization.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index e836b4f8b3..c1158b10b3 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2023 Alexey Abramov <levenson@mmer.org>
 ;;; Copyright © 2023 Sharlatan Hellseher <sharlatanus@gmail.com>
 ;;; Copyright © 2023 Vinicius Monego <monego@posteo.net>
+;;; Copyright © 2023 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -705,7 +706,7 @@ (define-public python-ruamel.yaml
 (define-public python-ruamel.yaml.clib
   (package
     (name "python-ruamel.yaml.clib")
-    (version "0.2.6")
+    (version "0.2.7")
     (source
       (origin
         ;; pypi release code has cythonized code without corresponding source.
@@ -716,7 +717,7 @@ (define-public python-ruamel.yaml.clib
         (file-name (string-append name "-" version "-checkout"))
         (sha256
          (base32
-          "05m3y7pjfbaarqbbgw1k6gs6cnnmxnwadjipxvw1aaaqk3s236cs"))
+          "119vvz2f6m5i0443r82j2r4hqqlmryffp25spv2cqky2cwjp5lj0"))
         (modules '((guix build utils)))
         (snippet
          '(begin
-- 
2.41.0

[bug#66690] [PATCH 2/7] gnu: python-ruamel.yaml: Update to 0.17.40.

[Arun Isaac]

* gnu/packages/serialization.scm (python-ruamel.yaml): Update to 0.17.40.
---
 gnu/packages/serialization.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index c1158b10b3..45aaee5a28 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -677,14 +677,14 @@ (define-public python-msgspec
 (define-public python-ruamel.yaml
   (package
     (name "python-ruamel.yaml")
-    (version "0.16.13")
+    (version "0.17.40")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "ruamel.yaml" version))
        (sha256
         (base32
-         "0hm9yg785f46bkrgqknd6fdvmkby9dpzjnm0b63qf0i748acaj5v"))))
+         "0g9irxrlim93z9ccgjyx4bfhbkdlqin0hzmhnn1d8rb7y23bj930"))))
     (build-system python-build-system)
     (native-inputs
      (list python-pytest))
-- 
2.41.0

[bug#66690] [PATCH 3/7] gnu: python-schema-salad: Update to 8.4.20230927144413.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-schema-salad): Update to
8.4.20230927144413.
[propagated-inputs]: Add python-importlib-resources. Replace python-mistune
with python-mistune-next.
[arguments]: Use G-expressions. Set exact version number. Remove arguments
that pytest does not recognize. Skip more tests that require network
access. Skip tests using #:test-flags.
---
 gnu/packages/bioinformatics.scm | 48 +++++++++++++++++++++++----------
 1 file changed, 34 insertions(+), 14 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index a1eac184ad..c6819316ae 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -8,7 +8,7 @@
 ;;; Copyright © 2016, 2020, 2022 Marius Bakke <marius@gnu.org>
 ;;; Copyright © 2016, 2018 Raoul Bonnal <ilpuccio.febo@gmail.com>
 ;;; Copyright © 2017, 2018 Tobias Geerinckx-Rice <me@tobias.gr>
-;;; Copyright © 2017, 2021, 2022 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2017, 2021, 2022, 2023 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2018 Joshua Sierles, Nextjournal <joshua@nextjournal.com>
 ;;; Copyright © 2018 Gábor Boskovits <boskovits@gmail.com>
 ;;; Copyright © 2018-2023 Mădălin Ionel Patrașcu <madalinionel.patrascu@mdc-berlin.de>
@@ -4327,30 +4327,50 @@ (define-public python-pybigwig
 (define-public python-schema-salad
   (package
     (name "python-schema-salad")
-    (version "8.2.20211116214159")
+    (version "8.4.20230927144413")
     (source
       (origin
         (method url-fetch)
         (uri (pypi-uri "schema-salad" version))
         (sha256
          (base32
-          "005dh2y45x92zl8sf2sqjmfvcqr4hrz8dfckgkckv87003v7lwqc"))))
+          "0d0mm1cwrdmz2yplfi7vgqjsfaz4z3rdm2gx2103176jdsiwq610"))))
     (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-before 'check 'skip-failing-tests
-           (lambda _
-             ;; Skip tests that require network access.
-             (substitute* "schema_salad/tests/test_cwl11.py"
-               (("^def test_(secondaryFiles|outputBinding)" all)
-                (string-append "@pytest.mark.skip(reason="
-                               "\"test requires network access\")\n"
-                               all))))))))
+     (list
+      #:test-flags
+      ;; Skip tests that require network access.
+      #~(list "-k"
+              (string-append "not test_secondaryFiles"
+                             " and not test_outputBinding"
+                             " and not test_yaml_tab_error"
+                             " and not test_idmap"
+                             " and not test_idmap2"
+                             " and not test_include"
+                             " and not test_load"
+                             " and not test_load_cwlschema"
+                             " and not test_load_by_yaml_metaschema"
+                             " and not test_load_metaschema"
+                             " and not test_bad_schemas"))
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'set-version
+            (lambda _
+              (substitute* "setup.py"
+                ;; Set exact version number.
+                (("version=\"[[:digit:].]*\"")
+                 (string-append "version=\"" #$version "\"")))))
+          (add-after 'unpack 'fix-pytest-arguments
+            (lambda _
+              (substitute* "tox.ini"
+                ;; pytest does not recognize the -n auto arguments. Remove
+                ;; them.
+                (("addopts=-n auto") "")))))))
     (propagated-inputs
      (list python-cachecontrol
+           python-importlib-resources
            python-lockfile
-           python-mistune
+           python-mistune-next
            python-rdflib
            python-rdflib-jsonld
            python-requests
-- 
2.41.0

[bug#66690] [PATCH 4/7] gnu: Add python-cwlformat.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwlformat): New variable.
---
 gnu/packages/bioinformatics.scm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index c6819316ae..3b2fc8de8a 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -4484,6 +4484,33 @@ (define-public python-scrublet
 doublets in single-cell RNA-seq data.")
     (license license:expat)))
 
+(define-public python-cwlformat
+  (package
+    (name "python-cwlformat")
+    (version "2022.02.18")
+    (source
+     ;; The PyPI tarball is missing Readme.md. Readme.md is required for the
+     ;; build.
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/rabix/cwl-format")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0agkz2w86k91rc9m5vx5hsqi5nm6fcmzkng6j99hjapz0r9233ql"))))
+    (build-system pyproject-build-system)
+    (propagated-inputs
+     (list python-importlib-resources
+           python-ruamel.yaml))
+    (home-page "https://github.com/rabix/cwl-format")
+    (synopsis "Prettifier for CWL code")
+    (description "@code{python-cwlformat} is a specification and a reference
+implementation for a very opinionated @acronym{CWL, Common Workflow Language}
+code formatter.  It outputs CWL in a standardized YAML format.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH 5/7] gnu: Add python-cwl-upgrader.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwl-upgrader): New variable.
---
 gnu/packages/bioinformatics.scm | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 3b2fc8de8a..79dbf0f2e6 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -4511,6 +4511,32 @@ (define-public python-cwlformat
 code formatter.  It outputs CWL in a standardized YAML format.")
     (license license:asl2.0)))
 
+(define-public python-cwl-upgrader
+  (package
+    (name "python-cwl-upgrader")
+    (version "1.2.9")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "cwl-upgrader" version))
+       (sha256
+        (base32
+         "017nfxrrnrf1bacryyigqgx9pprw6g5m4n2wf69j6pfihl10426a"))))
+    (build-system python-build-system)
+    (native-inputs
+     (list python-mypy-extensions
+           python-pytest-cov
+           python-pytest-runner))
+    (propagated-inputs
+     (list python-ruamel.yaml
+           python-schema-salad))
+    (home-page "https://github.com/common-workflow-language/cwl-upgrader")
+    (synopsis "CWL document upgrader")
+    (description "@code{python-cwl-upgrader} is a standalone upgrader for
+@acronym{CWL, Common Workflow Language} documents from version draft-3, v1.0,
+and v1.1 to v1.2.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH 6/7] gnu: Add python-cwl-utils.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwl-utils): New variable.
---
 gnu/packages/bioinformatics.scm | 43 +++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 79dbf0f2e6..2deec5368e 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -4537,6 +4537,49 @@ (define-public python-cwl-upgrader
 and v1.1 to v1.2.")
     (license license:asl2.0)))
 
+(define-public python-cwl-utils
+  (package
+    (name "python-cwl-utils")
+    (version "0.29")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "cwl-utils" version))
+       (sha256
+        (base32
+         "0rfm49wwnn4bcalkfrhvlmn7bhdf6nnwcr8fvx5gqxqyd4b5izf5"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:test-flags
+      #~(list "-k"
+              (string-append "not test_graph_split"
+                             " and not test_load_document_with_remote_uri"
+                             " and not test_remote_packing"
+                             " and not test_remote_packing_github_soft_links"
+                             " and not test_value_from_two_concatenated_expressions"))))
+    (inputs
+     (list node))
+    (native-inputs
+     (list python-mypy-extensions
+           python-pytest
+           python-pytest-mock
+           python-pytest-runner))
+    (propagated-inputs
+     (list python-cwl-upgrader
+           python-cwlformat
+           python-packaging
+           python-rdflib
+           python-requests
+           python-ruamel.yaml
+           python-schema-salad))
+    (home-page "https://github.com/common-workflow-language/cwl-utils")
+    (synopsis "Python utilities for CWL")
+    (description "@code{python-cwl-utils} provides python utilities and
+autogenerated classes for loading and parsing CWL v1.0, CWL v1.1, and CWL v1.2
+documents.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH 7/7] gnu: cwltool: Update to 3.1.20230601100705.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (cwltool): Update to 3.1.20230601100705.
[arguments]: Use G-expressions. Set version in setup.py. Do not attempt to
substitute files cwltool/schemas/v1.1/tests/env-tool1.cwl,
cwltool/schemas/v1.1/tests/env-tool2.cwl and
cwltool/schemas/v1.1/tests/imported-hint.cwl that have been removed in this
release.
[inputs]: Add python-cwl-utils.
---
 gnu/packages/bioinformatics.scm | 88 ++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 45 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 2deec5368e..b4ac896a0d 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -4583,7 +4583,7 @@ (define-public python-cwl-utils
 (define-public cwltool
   (package
     (name "cwltool")
-    (version "3.1.20220119140128")
+    (version "3.1.20231020140205")
     (source (origin
               (method git-fetch)
               (uri (git-reference
@@ -4592,56 +4592,54 @@ (define-public cwltool
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "1jmrm0qrqgka79avc1kq63fgh20gx6g07fc8p3iih4k85vhdyl3f"))))
+                "045n594k7zhm5qnf7mippldmm43wj3x0wnizhhpd1n2ak2iiky88"))))
     (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'loosen-version-restrictions
-           (lambda _
-             (substitute* "setup.py"
-               (("== 1.5.1") ">=1.5.1")))) ; prov
-         (add-after 'unpack 'dont-use-git
-           (lambda _
-             (substitute* "gittaggers.py"
-               (("self.git_timestamp_tag\\(\\)")
-                (string-append "time.strftime('.%Y%m%d%H%M%S', time.gmtime(int("
-                               (string-drop ,version 4) ")))")))))
-         (add-after 'unpack 'modify-tests
-           (lambda _
-             ;; Tries to connect to the internet.
-             (delete-file "tests/test_content_type.py")
-             (delete-file "tests/test_udocker.py")
-             (delete-file "tests/test_http_input.py")
-             (substitute* "tests/test_load_tool.py"
-               (("def test_load_graph_fragment_from_packed")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_load_graph_fragment_from_packed")))
-             (substitute* "tests/test_examples.py"
-               (("def test_env_filtering")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_env_filtering")))
-             ;; Tries to use cwl-runners.
-             (substitute* "tests/test_examples.py"
-               (("def test_v1_0_arg_empty_prefix_separate_false")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_v1_0_arg_empty_prefix_separate_false")))
-
-             (substitute* '("cwltool/schemas/v1.1/tests/env-tool1.cwl"
-                            "cwltool/schemas/v1.1/tests/env-tool2.cwl"
-                            "cwltool/schemas/v1.1/tests/imported-hint.cwl"
-                            "tests/subgraph/env-tool2.cwl"
-                            "tests/subgraph/env-tool2_req.cwl"
-                            "tests/subgraph/env-wf2_subwf-packed.cwl"
-                            "tests/subgraph/env-tool2_no_env.cwl")
-               (("\"/bin/sh\"") (string-append "\"" (which "sh") "\"")))
-             ;; Pytest doesn't know what to do with "-n auto"
-             (substitute* "tox.ini"
-               (("-n auto") "")))))))
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'loosen-version-restrictions
+            (lambda _
+              (substitute* "setup.py"
+                (("== 1.5.1") "> 1.5.1")))) ; prov
+          (add-after 'unpack 'set-version
+            (lambda _
+              ;; Set exact version.
+              (substitute* "setup.py"
+                (("use_scm_version=True")
+                 (string-append "version=\"" #$version "\"")))))
+          (add-after 'unpack 'modify-tests
+            (lambda _
+              ;; Tries to connect to the internet.
+              (delete-file "tests/test_content_type.py")
+              (delete-file "tests/test_udocker.py")
+              (delete-file "tests/test_http_input.py")
+              (substitute* "tests/test_load_tool.py"
+                (("def test_load_graph_fragment_from_packed")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_load_graph_fragment_from_packed")))
+              (substitute* "tests/test_examples.py"
+                (("def test_env_filtering")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_env_filtering")))
+              ;; Tries to use cwl-runners.
+              (substitute* "tests/test_examples.py"
+                (("def test_v1_0_arg_empty_prefix_separate_false")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_v1_0_arg_empty_prefix_separate_false")))
+              (substitute* '("tests/subgraph/env-tool2.cwl"
+                             "tests/subgraph/env-tool2_req.cwl"
+                             "tests/subgraph/env-wf2_subwf-packed.cwl"
+                             "tests/subgraph/env-tool2_no_env.cwl")
+                (("\"/bin/sh\"") (string-append "\"" (which "sh") "\"")))
+              ;; Pytest doesn't know what to do with "-n auto"
+              (substitute* "tox.ini"
+                (("-n auto") "")))))))
     (inputs
      (list python-argcomplete
            python-bagit
            python-coloredlogs
+           python-cwl-utils
            python-mypy-extensions
            python-prov
            python-pydot
-- 
2.41.0

[bug#66690] [PATCH 0/7] Upgrade cwltool

[Ludovic Courtès]

Hi,

Arun Isaac <arunisaac@systemreboot.net> skribis:

> This patchset upgrades cwltool to the latest version.
>
> Arun Isaac (7):
>   gnu: python-ruamel.yaml.clib: Update to 0.2.7.
>   gnu: python-ruamel.yaml: Update to 0.17.40.
>   gnu: python-schema-salad: Update to 8.4.20230927144413.
>   gnu: Add python-cwlformat.
>   gnu: Add python-cwl-upgrader.
>   gnu: Add python-cwl-utils.
>   gnu: cwltool: Update to 3.1.20230601100705.

I don’t see it at qa.guix, but if it builds for you, and if the
dependents of the 3 packages being upgraded work fine, please go ahead!

Ludo’.

[bug#66690] [PATCH 0/7] Upgrade cwltool

[Arun Isaac]

Hi Ludo,

Unfortunately, the upgrade of python-ruamel.yaml breaks conda. We'll
have to upgrade conda too before this patchset can be pushed. And,
that's quite a major upgrade.

Regards,
Arun

[bug#66690] [PATCH v2 0/8] Upgrade cwltool

[Arun Isaac]

In this patchset, I have updated cwltool once more to the latest version. And,
I have split out ruamel-yaml into two separate package variants so that conda
no longer breaks. I will push this patchset if it builds successfully on QA
without breaking other packages.

Arun Isaac (8):
  gnu: python-ruamel.yaml.clib: Update to 0.2.8.
  gnu: python-ruamel-yaml: Update to 0.18.6.
  gnu: python-schema-salad: Update to 8.5.20240102191335.
  gnu: Add python-cwlformat.
  gnu: Add python-cwl-upgrader.
  gnu: Add python-cwl-utils.
  gnu: Add python-spython.
  gnu: cwltool: Update to 3.1.20240112164112.

 gnu/packages/bioinformatics.scm     | 244 ++++++++++++++++++++--------
 gnu/packages/linux.scm              |  46 ++++++
 gnu/packages/package-management.scm |   3 +-
 gnu/packages/serialization.scm      |  21 ++-
 4 files changed, 240 insertions(+), 74 deletions(-)


base-commit: 5ad54df6b09b08b73453cf1071413693f053a0cc
-- 
2.41.0

[bug#66690] [PATCH v2 1/8] gnu: python-ruamel.yaml.clib: Update to 0.2.8.

[Arun Isaac]

* gnu/packages/serialization.scm (python-ruamel.yaml.clib): Update to 0.2.8.

Change-Id: I18172bb4cad80b0e1a5db11fbc8fb937f46a1862
---
 gnu/packages/serialization.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index ccafb8e802..963028c734 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -17,6 +17,7 @@
 ;;; Copyright © 2023 Vinicius Monego <monego@posteo.net>
 ;;; Copyright © 2023 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;; Copyright © 2023 Zheng Junjie <873216071@qq.com>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -749,7 +750,7 @@ (define-public python-ruamel.yaml
 (define-public python-ruamel.yaml.clib
   (package
     (name "python-ruamel.yaml.clib")
-    (version "0.2.6")
+    (version "0.2.8")
     (source
       (origin
         ;; pypi release code has cythonized code without corresponding source.
@@ -760,7 +761,7 @@ (define-public python-ruamel.yaml.clib
         (file-name (string-append name "-" version "-checkout"))
         (sha256
          (base32
-          "05m3y7pjfbaarqbbgw1k6gs6cnnmxnwadjipxvw1aaaqk3s236cs"))
+          "0qspqnk72xrjj17b00hjibbzjk3krsrakzf08wxwz7z908cv6278"))
         (modules '((guix build utils)))
         (snippet
          '(begin
-- 
2.41.0

[bug#66690] [PATCH v2 2/8] gnu: python-ruamel-yaml: Update to 0.18.6.

[Arun Isaac]

* gnu/package/serialization.scm (python-ruamel-yaml): Update to 0.18.6.
(python-ruamel.yaml-0.16): New variable.
* gnu/packages/package-management.scm (conda)[inputs]: Replace
python-ruamel.yaml with python-ruamel.yaml-0.16.

Change-Id: Iff9eeeae043c9547d93a6c780f0c30bd760490d9
---
 gnu/packages/package-management.scm |  3 ++-
 gnu/packages/serialization.scm      | 16 ++++++++++++++--
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index c1ad613bee..5d6bc47b8b 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -23,6 +23,7 @@
 ;;; Copyright © 2022, 2023 Zhu Zihao <all_but_last@163.com>
 ;;; Copyright © 2023 jgart <jgart@dismail.de>
 ;;; Copyright © 2023 Mădălin Ionel Patrașcu <madalinionel.patrascu@mdc-berlin.de>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1251,7 +1252,7 @@ (define-public conda
            python-pyyaml
            python-requests
            python-responses
-           python-ruamel.yaml
+           python-ruamel.yaml-0.16
            python-tqdm
            ;; XXX: This is dragged in by libarchive and is needed at runtime.
            zstd))
diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm
index 963028c734..240646748b 100644
--- a/gnu/packages/serialization.scm
+++ b/gnu/packages/serialization.scm
@@ -721,14 +721,14 @@ (define-public python-msgspec
 (define-public python-ruamel.yaml
   (package
     (name "python-ruamel.yaml")
-    (version "0.16.13")
+    (version "0.18.6")
     (source
      (origin
        (method url-fetch)
        (uri (pypi-uri "ruamel.yaml" version))
        (sha256
         (base32
-         "0hm9yg785f46bkrgqknd6fdvmkby9dpzjnm0b63qf0i748acaj5v"))))
+         "06rimidc9nb3i3r90n3a1zwf0qxw24zqykb3wpxwd1p72yifc9wb"))))
     (build-system python-build-system)
     (native-inputs
      (list python-pytest))
@@ -747,6 +747,18 @@ (define-public python-ruamel.yaml
 style and key ordering are kept, so you can diff the source.")
     (license license:expat)))
 
+(define-public python-ruamel.yaml-0.16
+  (package
+    (inherit python-ruamel.yaml)
+    (version "0.16.13")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "ruamel.yaml" version))
+       (sha256
+        (base32
+         "0hm9yg785f46bkrgqknd6fdvmkby9dpzjnm0b63qf0i748acaj5v"))))))
+
 (define-public python-ruamel.yaml.clib
   (package
     (name "python-ruamel.yaml.clib")
-- 
2.41.0

[bug#66690] [PATCH v2 3/8] gnu: python-schema-salad: Update to 8.5.20240102191335.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-schema-salad): Update to
8.5.20240102191335.
[native-inputs]: Add python-pytest-xdist.
[propagated-inputs]: Add python-importlib-resources and
python-mypy-extensions. Remove python-lockfile, python-rdflib-jsonld and
python-typing-extensions. Replace python-mistune with python-mistune-next.
[arguments]: Use G-expressions. Set exact version number. Skip more tests that
require network access.

Change-Id: I4de4704f58f40fdf9bf4c02e41c2c353beaa3eb2
---
 gnu/packages/bioinformatics.scm | 61 ++++++++++++++++++++-------------
 1 file changed, 38 insertions(+), 23 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 84577ef760..75cfad51a6 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5001,37 +5001,52 @@ (define-public python-pyfasta
 (define-public python-schema-salad
   (package
     (name "python-schema-salad")
-    (version "8.2.20211116214159")
+    (version "8.5.20240102191335")
     (source
-     (origin
-       (method url-fetch)
-       (uri (pypi-uri "schema-salad" version))
-       (sha256
-        (base32
-         "005dh2y45x92zl8sf2sqjmfvcqr4hrz8dfckgkckv87003v7lwqc"))))
+      (origin
+        (method url-fetch)
+        (uri (pypi-uri "schema-salad" version))
+        (sha256
+         (base32
+          "035202p696i3jylb8b3nm9qcxsqby15hhqn1dl4nrz73a17p0ckx"))))
     (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-before 'check 'skip-failing-tests
-           (lambda _
-             ;; Skip tests that require network access.
-             (substitute* "schema_salad/tests/test_cwl11.py"
-               (("^def test_(secondaryFiles|outputBinding)" all)
-                (string-append "@pytest.mark.skip(reason="
-                               "\"test requires network access\")\n"
-                               all))))))))
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'set-version
+            (lambda _
+              ;; Set exact version.
+              (substitute* "setup.py"
+                (("use_scm_version=True")
+                 (string-append "version=\"" #$version "\"")))))
+          (add-before 'check 'skip-failing-tests
+            (lambda _
+              ;; Skip tests that require network access.
+              (let ((skip-test
+                     (lambda (test-pattern)
+                       (string-append "@pytest.mark.skip(reason="
+                                      "\"test requires network access\")\n"
+                                      test-pattern))))
+                (substitute* "schema_salad/tests/test_cg.py"
+                  (("^def test_(load(_by_yaml_metaschema|_metaschema|_cwlschema|)|include|idmap|idmap2)\\(" all)
+                   (skip-test all)))
+                (substitute* "schema_salad/tests/test_cwl11.py"
+                  (("^def test_(secondaryFiles|outputBinding|yaml_tab_error)\\(" all)
+                   (skip-test all)))
+                (substitute* "schema_salad/tests/test_examples.py"
+                  (("^def test_bad_schemas\\(" all)
+                   (skip-test all)))))))))
     (propagated-inputs
      (list python-cachecontrol
-           python-lockfile
-           python-mistune
+           python-importlib-resources
+           python-mistune-next
+           python-mypy-extensions
            python-rdflib
-           python-rdflib-jsonld
            python-requests
-           python-ruamel.yaml
-           python-typing-extensions))
+           python-ruamel.yaml))
     (native-inputs
-     (list python-black python-pytest python-pytest-runner))
+     (list python-black python-pytest python-pytest-runner python-pytest-xdist))
     (home-page "https://github.com/common-workflow-language/schema_salad")
     (synopsis "Schema Annotations for Linked Avro Data (SALAD)")
     (description
-- 
2.41.0

[bug#66690] [PATCH v2 4/8] gnu: Add python-cwlformat.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwlformat): New variable.

Change-Id: Ib2d79e350ab100c61c7afe2d5d3eafc72146e493
---
 gnu/packages/bioinformatics.scm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 75cfad51a6..48e56945b3 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5138,6 +5138,33 @@ (define-public python-scrublet
 doublets in single-cell RNA-seq data.")
     (license license:expat)))
 
+(define-public python-cwlformat
+  (package
+    (name "python-cwlformat")
+    (version "2022.02.18")
+    (source
+     ;; The PyPI tarball is missing Readme.md. Readme.md is required for the
+     ;; build.
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/rabix/cwl-format")
+             (commit version)))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0agkz2w86k91rc9m5vx5hsqi5nm6fcmzkng6j99hjapz0r9233ql"))))
+    (build-system pyproject-build-system)
+    (propagated-inputs
+     (list python-importlib-resources
+           python-ruamel.yaml))
+    (home-page "https://github.com/rabix/cwl-format")
+    (synopsis "Prettifier for CWL code")
+    (description "@code{python-cwlformat} is a specification and a reference
+implementation for a very opinionated @acronym{CWL, Common Workflow Language}
+code formatter.  It outputs CWL in a standardized YAML format.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH v2 5/8] gnu: Add python-cwl-upgrader.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwl-upgrader): New variable.

Change-Id: I2022239840e9fe9e745689ef8b8bf4279be2ff9e
---
 gnu/packages/bioinformatics.scm | 24 ++++++++++++++++++++++++
 1 file changed, 24 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 48e56945b3..438b22800c 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5165,6 +5165,30 @@ (define-public python-cwlformat
 code formatter.  It outputs CWL in a standardized YAML format.")
     (license license:asl2.0)))
 
+(define-public python-cwl-upgrader
+  (package
+    (name "python-cwl-upgrader")
+    (version "1.2.11")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "cwl-upgrader" version))
+       (sha256
+        (base32
+         "12j6z8nvwnzjjyypz59hwj5hmrcri2r6aknw52n9dbj6lbzbdd2p"))))
+    (build-system pyproject-build-system)
+    (native-inputs
+     (list python-pytest))
+    (propagated-inputs
+     (list python-ruamel.yaml
+           python-schema-salad))
+    (home-page "https://github.com/common-workflow-language/cwl-upgrader")
+    (synopsis "CWL document upgrader")
+    (description "@code{python-cwl-upgrader} is a standalone upgrader for
+@acronym{CWL, Common Workflow Language} documents from version draft-3, v1.0,
+and v1.1 to v1.2.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH v2 6/8] gnu: Add python-cwl-utils.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (python-cwl-utils): New variable.

Change-Id: I9347c80c0e866b89a7d432091fbcf1bc009209a5
---
 gnu/packages/bioinformatics.scm | 43 +++++++++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 438b22800c..35c4d7d993 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5189,6 +5189,49 @@ (define-public python-cwl-upgrader
 and v1.1 to v1.2.")
     (license license:asl2.0)))
 
+(define-public python-cwl-utils
+  (package
+    (name "python-cwl-utils")
+    (version "0.32")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "cwl-utils" version))
+       (sha256
+        (base32
+         "06wkw8d8cqm3hnz8xwnysz874gwaym36c358cr7frw5iglhvsj98"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:test-flags
+      #~(list "-k"
+              (string-append "not test_graph_split"
+                             " and not test_load_document_with_remote_uri"
+                             " and not test_remote_packing"
+                             " and not test_remote_packing_github_soft_links"
+                             " and not test_value_from_two_concatenated_expressions"))))
+    (inputs
+     (list node))
+    (native-inputs
+     (list python-mypy-extensions
+           python-pytest
+           python-pytest-mock
+           python-pytest-runner))
+    (propagated-inputs
+     (list python-cwl-upgrader
+           python-cwlformat
+           python-packaging
+           python-rdflib
+           python-requests
+           python-ruamel.yaml
+           python-schema-salad))
+    (home-page "https://github.com/common-workflow-language/cwl-utils")
+    (synopsis "Python utilities for CWL")
+    (description "@code{python-cwl-utils} provides python utilities and
+autogenerated classes for loading and parsing CWL v1.0, CWL v1.1, and CWL v1.2
+documents.")
+    (license license:asl2.0)))
+
 (define-public cwltool
   (package
     (name "cwltool")
-- 
2.41.0

[bug#66690] [PATCH v2 7/8] gnu: Add python-spython.

[Arun Isaac]

* gnu/packages/linux.scm (python-spython): New variable.

Change-Id: I758f879608bc4b0c0411918b4ee0fa9cc3abc639
---
 gnu/packages/linux.scm | 46 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 46 insertions(+)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index d798c44a8f..99bc517a96 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -80,6 +80,7 @@
 ;;; Copyright © 2023 Jaeme Sifat <jaeme@runbox.com>
 ;;; Copyright © 2024 Gabriel Wicki <gabriel@erlikon.ch>
 ;;; Copyright © 2024 Sharlatan Hellseher <sharlatanus@gmail.com>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -5265,6 +5266,51 @@ (define-public singularity
 isolation or root privileges.")
     (license license:bsd-3)))
 
+(define-public python-spython
+  (package
+    (name "python-spython")
+    (version "0.3.13")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (pypi-uri "spython" version))
+       (sha256
+        (base32 "0kly851k6mj7xzcybciav5d0pq5q04pzg7c5a1g712bqbxkha4ck"))))
+    (build-system pyproject-build-system)
+    (arguments
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          ;; Configure absolute path to singularity.
+          (add-after 'unpack 'configure
+            (lambda* (#:key inputs #:allow-other-keys)
+              (let ((singularity (search-input-file inputs "bin/singularity")))
+                (substitute* "spython/utils/terminal.py"
+                  (("software=\"singularity\"")
+                   (string-append "software=\"" singularity "\"")))
+                (substitute* (list "spython/utils/terminal.py"
+                                   "spython/main/help.py"
+                                   "spython/main/base/command.py")
+                  (("\\[\"singularity\"")
+                   (string-append "[\"" singularity "\"")))
+                (substitute* "spython/main/execute.py"
+                  (("shutil.which\\(\"singularity\"\\)")
+                   (string-append "shutil.which(\"" singularity "\")"))))))
+          ;; Skip tests that require network access.
+          (add-before 'check 'skip-tests
+            (lambda _
+              (delete-file "spython/tests/test_client.py"))))))
+    (inputs
+     (list singularity))
+    (native-inputs
+     (list python-pytest
+           python-pytest-runner))
+    (home-page "https://github.com/singularityhub/singularity-cli")
+    (synopsis "Singularity Python client")
+    (description "@code{python-spython} is a Python library to interact with
+Singularity containers.")
+    (license license:mpl2.0)))
+
 (define-public libnvme
   (package
     (name "libnvme")
-- 
2.41.0

[bug#66690] [PATCH v2 8/8] gnu: cwltool: Update to 3.1.20240112164112.

[Arun Isaac]

* gnu/packages/bioinformatics.scm (cwltool): Update to 3.1.20240112164112.
[arguments]: Use G-expressions. Set version in setup.py. Do not attempt to
substitute files cwltool/schemas/v1.1/tests/env-tool1.cwl,
cwltool/schemas/v1.1/tests/env-tool2.cwl and
cwltool/schemas/v1.1/tests/imported-hint.cwl that have been removed in this
release. Do not remove "-n auto" from pytest flags.
[inputs]: Add python-cwl-utils and python-spython.
[native-inputs]: Add python-pytest-xdist.

Change-Id: If1be7cdf39b767c5ef2a1dbddfdc95f59ea98cb6

Change-Id: I2ba81e8569abcf43cd0fad33daab1e4f8689e77f
---
 gnu/packages/bioinformatics.scm | 89 ++++++++++++++++-----------------
 1 file changed, 43 insertions(+), 46 deletions(-)

diff --git a/gnu/packages/bioinformatics.scm b/gnu/packages/bioinformatics.scm
index 35c4d7d993..cc1a966b41 100644
--- a/gnu/packages/bioinformatics.scm
+++ b/gnu/packages/bioinformatics.scm
@@ -5235,7 +5235,7 @@ (define-public python-cwl-utils
 (define-public cwltool
   (package
     (name "cwltool")
-    (version "3.1.20220119140128")
+    (version "3.1.20240112164112")
     (source (origin
               (method git-fetch)
               (uri (git-reference
@@ -5244,56 +5244,51 @@ (define-public cwltool
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "1jmrm0qrqgka79avc1kq63fgh20gx6g07fc8p3iih4k85vhdyl3f"))))
+                "1fpc5kqgpbn48g5vlvy64p297x2wm3gfz8casgpk15ap593wwh33"))))
     (build-system pyproject-build-system)
     (arguments
-     `(#:phases
-       (modify-phases %standard-phases
-         (add-after 'unpack 'loosen-version-restrictions
-           (lambda _
-             (substitute* "setup.py"
-               (("== 1.5.1") ">=1.5.1")))) ; prov
-         (add-after 'unpack 'dont-use-git
-           (lambda _
-             (substitute* "gittaggers.py"
-               (("self.git_timestamp_tag\\(\\)")
-                (string-append "time.strftime('.%Y%m%d%H%M%S', time.gmtime(int("
-                               (string-drop ,version 4) ")))")))))
-         (add-after 'unpack 'modify-tests
-           (lambda _
-             ;; Tries to connect to the internet.
-             (delete-file "tests/test_content_type.py")
-             (delete-file "tests/test_udocker.py")
-             (delete-file "tests/test_http_input.py")
-             (substitute* "tests/test_load_tool.py"
-               (("def test_load_graph_fragment_from_packed")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_load_graph_fragment_from_packed")))
-             (substitute* "tests/test_examples.py"
-               (("def test_env_filtering")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_env_filtering")))
-             ;; Tries to use cwl-runners.
-             (substitute* "tests/test_examples.py"
-               (("def test_v1_0_arg_empty_prefix_separate_false")
-                (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
-                               "def test_v1_0_arg_empty_prefix_separate_false")))
-
-             (substitute* '("cwltool/schemas/v1.1/tests/env-tool1.cwl"
-                            "cwltool/schemas/v1.1/tests/env-tool2.cwl"
-                            "cwltool/schemas/v1.1/tests/imported-hint.cwl"
-                            "tests/subgraph/env-tool2.cwl"
-                            "tests/subgraph/env-tool2_req.cwl"
-                            "tests/subgraph/env-wf2_subwf-packed.cwl"
-                            "tests/subgraph/env-tool2_no_env.cwl")
-               (("\"/bin/sh\"") (string-append "\"" (which "sh") "\"")))
-             ;; Pytest doesn't know what to do with "-n auto"
-             (substitute* "tox.ini"
-               (("-n auto") "")))))))
+     (list
+      #:phases
+      #~(modify-phases %standard-phases
+          (add-after 'unpack 'loosen-version-restrictions
+            (lambda _
+              (substitute* "setup.py"
+                (("== 1.5.1") "> 1.5.1")))) ; prov
+          (add-after 'unpack 'set-version
+            (lambda _
+              ;; Set exact version.
+              (substitute* "setup.py"
+                (("use_scm_version=True")
+                 (string-append "version=\"" #$version "\"")))))
+          (add-after 'unpack 'modify-tests
+            (lambda _
+              ;; Tries to connect to the internet.
+              (delete-file "tests/test_content_type.py")
+              (delete-file "tests/test_udocker.py")
+              (delete-file "tests/test_http_input.py")
+              (substitute* "tests/test_load_tool.py"
+                (("def test_load_graph_fragment_from_packed")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_load_graph_fragment_from_packed")))
+              (substitute* "tests/test_examples.py"
+                (("def test_env_filtering")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_env_filtering")))
+              ;; Tries to use cwl-runners.
+              (substitute* "tests/test_examples.py"
+                (("def test_v1_0_arg_empty_prefix_separate_false")
+                 (string-append "@pytest.mark.skip(reason=\"Disabled by Guix\")\n"
+                                "def test_v1_0_arg_empty_prefix_separate_false")))
+              (substitute* '("tests/subgraph/env-tool2.cwl"
+                             "tests/subgraph/env-tool2_req.cwl"
+                             "tests/subgraph/env-wf2_subwf-packed.cwl"
+                             "tests/subgraph/env-tool2_no_env.cwl")
+                (("\"/bin/sh\"") (string-append "\"" (which "sh") "\""))))))))
     (inputs
      (list python-argcomplete
            python-bagit
            python-coloredlogs
+           python-cwl-utils
            python-mypy-extensions
            python-prov
            python-pydot
@@ -5303,6 +5298,7 @@ (define-public cwltool
            python-ruamel.yaml
            python-schema-salad
            python-shellescape
+           python-spython
            python-typing-extensions
            ;; Not listed as needed but still necessary:
            node))
@@ -5313,7 +5309,8 @@ (define-public cwltool
            python-pytest
            python-pytest-cov
            python-pytest-mock
-           python-pytest-runner))
+           python-pytest-runner
+           python-pytest-xdist))
     (home-page
      "https://github.com/common-workflow-language/common-workflow-language")
     (synopsis "Common Workflow Language reference implementation")
-- 
2.41.0

[bug#66690] [PATCH v2 0/8] Upgrade cwltool

[Ludovic Courtès]

Hi Arun!

Arun Isaac <arunisaac@systemreboot.net> skribis:

> In this patchset, I have updated cwltool once more to the latest version. And,
> I have split out ruamel-yaml into two separate package variants so that conda
> no longer breaks. I will push this patchset if it builds successfully on QA
> without breaking other packages.

I skimmed over the patches and they LGTM.

Thanks,
Ludo’.

bug#66690: [PATCH v2 0/8] Upgrade cwltool

[Arun Isaac]

Thanks, Ludo. I pushed this patchset after moving a couple of more
packages to the older python-ruamel-yaml-0.16.