unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#72766] [PATCH 0/2] slurm: Enable REST API.
@ 2024-08-22 23:30 Arun Isaac
  2024-08-22 23:36 ` [bug#72766] [PATCH 1/2] gnu: Add libjwt Arun Isaac
                   ` (2 more replies)
  0 siblings, 3 replies; 19+ messages in thread
From: Arun Isaac @ 2024-08-22 23:30 UTC (permalink / raw)
  To: 72766; +Cc: Arun Isaac

This short patchset enables the REST API support in our slurm package.

Arun Isaac (2):
  gnu: Add libjwt.
  gnu: slurm: Enable REST API.

 gnu/packages/parallel.scm | 26 ++++++++++++++++++++++----
 gnu/packages/web.scm      | 26 +++++++++++++++++++++++++-
 2 files changed, 47 insertions(+), 5 deletions(-)


base-commit: 6bec6c20b6b18dc173383a6828f1bb84314f986b
-- 
2.45.2





^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 1/2] gnu: Add libjwt.
  2024-08-22 23:30 [bug#72766] [PATCH 0/2] slurm: Enable REST API Arun Isaac
@ 2024-08-22 23:36 ` Arun Isaac
  2024-08-22 23:36 ` [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API Arun Isaac
  2024-09-09 15:24 ` [bug#72766] [PATCH v2 1/2] gnu: Add libjwt Arun Isaac
  2 siblings, 0 replies; 19+ messages in thread
From: Arun Isaac @ 2024-08-22 23:36 UTC (permalink / raw)
  To: 72766; +Cc: Arun Isaac

* gnu/packages/web.scm (libjwt): New variable.

Change-Id: Ied516fb41a90de316588622b999b27f0f7ee8de3
---
 gnu/packages/web.scm | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 34922911a7..dd1b9438a2 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -15,7 +15,7 @@
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2016, 2023 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is>
-;;; Copyright © 2016–2023 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2016–2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2016–2022 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2016 Bake Timmons <b3timmons@speedymail.org>
 ;;; Copyright © 2017 Thomas Danckaert <post@thomasdanckaert.be>
@@ -1604,6 +1604,30 @@ (define-public rapidjson
 style API.")
       (license license:expat))))
 
+(define-public libjwt
+  (package
+    (name "libjwt")
+    (version "1.17.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/benmcollins/libjwt/releases/download/v"
+                           version "/libjwt-"
+                           version ".tar.bz2"))
+       (sha256
+        (base32
+         "1bpfaa0y8bccz5hr677lkrprs07akx02k0qbf82z2c8syr24a77i"))))
+    (build-system gnu-build-system)
+    (inputs
+     (list jansson openssl))
+    (native-inputs
+     (list check pkg-config))
+    (home-page "https://github.com/benmcollins/libjwt")
+    (synopsis "C @acronym{JWT, JSON Web Token} library")
+    (description "@code{libjwt} is a @acronym{JWT, JSON Web Token} library for
+C.")
+    (license license:mpl2.0)))
+
 (define-public yajl
   (package
     (name "yajl")
-- 
2.45.2





^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-08-22 23:30 [bug#72766] [PATCH 0/2] slurm: Enable REST API Arun Isaac
  2024-08-22 23:36 ` [bug#72766] [PATCH 1/2] gnu: Add libjwt Arun Isaac
@ 2024-08-22 23:36 ` Arun Isaac
  2024-09-02  7:43   ` Ludovic Courtès
  2024-09-09 15:24 ` [bug#72766] [PATCH v2 1/2] gnu: Add libjwt Arun Isaac
  2 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-08-22 23:36 UTC (permalink / raw)
  To: 72766; +Cc: Arun Isaac

* gnu/packages/parallel.scm (slurm)[inputs]: Add dbus, http-parser, libjwt,
librdkafka, libtaml, linux-libre-headers and (mariadb "dev").
[arguments]: Add --enable-slurmrestd, --with-bpf, --with-http-parser and
--with-rdkafka to configure flags.

Change-Id: Icddaa8f883754129d975e82fccd732a47a467517
---
 gnu/packages/parallel.scm | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index e75e902a68..21c945382f 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2024 Zheng Junjie <873216071@qq.com>
 ;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
 ;;; Copyright © 2024 Romain Garbage <romain.garbage@inria.fr>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -49,13 +50,16 @@ (define-module (gnu packages parallel)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages databases)
   #:use-module (gnu packages documentation)
   #:use-module (gnu packages flex)
   #:use-module (gnu packages freeipmi)
+  #:use-module (gnu packages glib)
   #:use-module (gnu packages libevent)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages maths)
   #:use-module (gnu packages mpi)
+  #:use-module (gnu packages networking)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
@@ -63,6 +67,7 @@ (define-module (gnu packages parallel)
   #:use-module (gnu packages python-science)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages serialization)
   #:use-module (gnu packages tcl)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages web))
@@ -223,12 +228,18 @@ (define-public slurm
                   (mkdir "contribs")
                   (rename-file "tmp-pmi" "contribs/pmi")
                   (rename-file "tmp-pmi2" "contribs/pmi2")))))
-    ;; FIXME: More optional inputs could be added,
-    ;; in particular mysql and gtk+.
-    (inputs (list freeipmi
+    ;; FIXME: More optional inputs (for example, gtk+) could be added.
+    (inputs (list dbus
+                  freeipmi
+                  http-parser
                   `(,hwloc-2 "lib")
                   json-c
+                  libjwt
+                  librdkafka
+                  libyaml
+                  linux-libre-headers
                   linux-pam
+                  `(,mariadb "dev")
                   openpmix
                   munge
                   numactl
@@ -238,13 +249,20 @@ (define-public slurm
     (build-system gnu-build-system)
     (arguments
      (list #:configure-flags
-           #~(list "--enable-pam" "--sysconfdir=/etc/slurm"
+           #~(list "--enable-pam"
+                   "--enable-slurmrestd"
+                   "--sysconfdir=/etc/slurm"
                    "--disable-static"
+                   (string-append "--with-bpf="
+                                  #$(this-package-input "linux-libre-headers"))
                    (string-append "--with-freeipmi=" #$(this-package-input "freeipmi"))
+                   (string-append "--with-http-parser="
+                                  #$(this-package-input "http-parser"))
                    (string-append "--with-hwloc="
                                   (ungexp (this-package-input "hwloc") "lib"))
                    (string-append "--with-json=" #$(this-package-input "json-c"))
                    (string-append "--with-munge=" #$(this-package-input "munge"))
+                   (string-append "--with-rdkafka=" #$(this-package-input "librdkafka"))
 
                    ;; Use PMIx bundled with Open MPI (this is required for Open MPI 5.x).
                    ;; Note: Older versions that inherit from this package lack the
-- 
2.45.2





^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-08-22 23:36 ` [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API Arun Isaac
@ 2024-09-02  7:43   ` Ludovic Courtès
  2024-09-06 16:03     ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-02  7:43 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hi Arun,

Arun Isaac <arunisaac@systemreboot.net> skribis:

> * gnu/packages/parallel.scm (slurm)[inputs]: Add dbus, http-parser, libjwt,
> librdkafka, libtaml, linux-libre-headers and (mariadb "dev").
> [arguments]: Add --enable-slurmrestd, --with-bpf, --with-http-parser and
> --with-rdkafka to configure flags.
>
> Change-Id: Icddaa8f883754129d975e82fccd732a47a467517

I’m curious what the REST interface has to offer compared to DRMAA.

> +    (inputs (list dbus
> +                  freeipmi
> +                  http-parser
>                    `(,hwloc-2 "lib")
>                    json-c
> +                  libjwt
> +                  librdkafka
> +                  libyaml
> +                  linux-libre-headers
>                    linux-pam
> +                  `(,mariadb "dev")

That’s quite a few heavyweight dependencies (‘linux-libre-headers’ is
not necessary; it’s included by default).

It’s a bit of Inheritance Game in this file (ah ha!) so we should make
sure the other slurm variants still work fine.  qa.guix hasn’t tested it
yet.

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-09-02  7:43   ` Ludovic Courtès
@ 2024-09-06 16:03     ` Arun Isaac
  2024-09-06 16:06       ` Arun Isaac
  2024-09-10  8:32       ` Ludovic Courtès
  0 siblings, 2 replies; 19+ messages in thread
From: Arun Isaac @ 2024-09-06 16:03 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


Hi Ludo,

> I’m curious what the REST interface has to offer compared to DRMAA.

I don't know, to be honest.

I've started working on ravanan, a new implementation of the Common
Workflow Language that takes inspiration from Guix and integrates with
Guix for strong reproducibility.

https://git.systemreboot.net/ravanan/about/

ravanan uses the slurm REST API to talk to slurm, and that's why I need
this patch. I chose the slurm REST API over DRMAA rather arbitrarily. In
the future, I could support both in ravanan. So, it's still good to have
both.

>> +    (inputs (list dbus
>> +                  freeipmi
>> +                  http-parser
>>                    `(,hwloc-2 "lib")
>>                    json-c
>> +                  libjwt
>> +                  librdkafka
>> +                  libyaml
>> +                  linux-libre-headers
>>                    linux-pam
>> +                  `(,mariadb "dev")
>
> That’s quite a few heavyweight dependencies (‘linux-libre-headers’ is
> not necessary; it’s included by default).

Good catch! I'll remove it before pushing.

> It’s a bit of Inheritance Game in this file (ah ha!) so we should make
> sure the other slurm variants still work fine.  qa.guix hasn’t tested it
> yet.

I have built the other slurm variants, and they seem to build fine. I
also used these packages to migrate a cluster from ancient slurm 18.08
to modern slurm 23.02 in increments of two versions, and tested
functionality at each step. They all seem to work. So, I am relatively
confident these packages work.

Regards,
Arun




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-09-06 16:03     ` Arun Isaac
@ 2024-09-06 16:06       ` Arun Isaac
  2024-09-09  7:13         ` Ludovic Courtès
  2024-09-10  8:32       ` Ludovic Courtès
  1 sibling, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-06 16:06 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


>> That’s quite a few heavyweight dependencies (‘linux-libre-headers’ is
>> not necessary; it’s included by default).
>
> Good catch! I'll remove it before pushing.

But, how do I refer to an implicit input (linux-libre-headers, in this
case) using this-package-input? Is there some other way to do this?

Thanks!




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-09-06 16:06       ` Arun Isaac
@ 2024-09-09  7:13         ` Ludovic Courtès
  2024-09-09 14:59           ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-09  7:13 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hey Arun,

Arun Isaac <arunisaac@systemreboot.net> skribis:

>>> That’s quite a few heavyweight dependencies (‘linux-libre-headers’ is
>>> not necessary; it’s included by default).
>>
>> Good catch! I'll remove it before pushing.
>
> But, how do I refer to an implicit input (linux-libre-headers, in this
> case) using this-package-input? Is there some other way to do this?

It depends on the details, but perhaps you can use ‘search-input-file’?
‘this-package-input’ doesn’t let you access implicit inputs; it just
accesses the regular inputs fields.

HTH,
Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-09-09  7:13         ` Ludovic Courtès
@ 2024-09-09 14:59           ` Arun Isaac
  0 siblings, 0 replies; 19+ messages in thread
From: Arun Isaac @ 2024-09-09 14:59 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


>> But, how do I refer to an implicit input (linux-libre-headers, in this
>> case) using this-package-input? Is there some other way to do this?
>
> It depends on the details, but perhaps you can use ‘search-input-file’?
> ‘this-package-input’ doesn’t let you access implicit inputs; it just
> accesses the regular inputs fields.

Fixed with search-input-directory. Patchset v2 follows.

Thanks!




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 1/2] gnu: Add libjwt.
  2024-08-22 23:30 [bug#72766] [PATCH 0/2] slurm: Enable REST API Arun Isaac
  2024-08-22 23:36 ` [bug#72766] [PATCH 1/2] gnu: Add libjwt Arun Isaac
  2024-08-22 23:36 ` [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API Arun Isaac
@ 2024-09-09 15:24 ` Arun Isaac
  2024-09-09 15:24   ` [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API Arun Isaac
  2 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-09 15:24 UTC (permalink / raw)
  To: 72766; +Cc: Arun Isaac, Arun Isaac, Ludovic Courtès

* gnu/packages/web.scm (libjwt): New variable.

Change-Id: Ied516fb41a90de316588622b999b27f0f7ee8de3
---
 gnu/packages/web.scm | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index c768874a4a..735cc5ce98 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -15,7 +15,7 @@
 ;;; Copyright © 2016 Ben Woodcroft <donttrustben@gmail.com>
 ;;; Copyright © 2016, 2023 Clément Lassieur <clement@lassieur.org>
 ;;; Copyright © 2016, 2017 Nikita <nikita@n0.is>
-;;; Copyright © 2016–2023 Arun Isaac <arunisaac@systemreboot.net>
+;;; Copyright © 2016–2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;; Copyright © 2016–2022 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2016 Bake Timmons <b3timmons@speedymail.org>
 ;;; Copyright © 2017 Thomas Danckaert <post@thomasdanckaert.be>
@@ -1610,6 +1610,30 @@ (define-public rapidjson
 style API.")
       (license license:expat))))
 
+(define-public libjwt
+  (package
+    (name "libjwt")
+    (version "1.17.2")
+    (source
+     (origin
+       (method url-fetch)
+       (uri (string-append "https://github.com/benmcollins/libjwt/releases/download/v"
+                           version "/libjwt-"
+                           version ".tar.bz2"))
+       (sha256
+        (base32
+         "1bpfaa0y8bccz5hr677lkrprs07akx02k0qbf82z2c8syr24a77i"))))
+    (build-system gnu-build-system)
+    (inputs
+     (list jansson openssl))
+    (native-inputs
+     (list check pkg-config))
+    (home-page "https://github.com/benmcollins/libjwt")
+    (synopsis "C @acronym{JWT, JSON Web Token} library")
+    (description "@code{libjwt} is a @acronym{JWT, JSON Web Token} library for
+C.")
+    (license license:mpl2.0)))
+
 (define-public yajl
   (package
     (name "yajl")

base-commit: cc2619a7bd7542d834762de2430937a59d1f3a36
-- 
2.45.2





^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-09 15:24 ` [bug#72766] [PATCH v2 1/2] gnu: Add libjwt Arun Isaac
@ 2024-09-09 15:24   ` Arun Isaac
  2024-09-10  6:50     ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-09 15:24 UTC (permalink / raw)
  To: 72766; +Cc: Arun Isaac, Arun Isaac, Ludovic Courtès

* gnu/packages/parallel.scm (slurm)[inputs]: Add dbus, http-parser, libjwt,
librdkafka, libtaml and (mariadb "dev").
[arguments]: Add --enable-slurmrestd, --with-bpf, --with-http-parser,
--with-rdkafka and --with-yaml to configure flags.

Change-Id: Icddaa8f883754129d975e82fccd732a47a467517
---
 gnu/packages/parallel.scm | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm
index d5a55a8641..5d7544783a 100644
--- a/gnu/packages/parallel.scm
+++ b/gnu/packages/parallel.scm
@@ -15,6 +15,7 @@
 ;;; Copyright © 2024 Zheng Junjie <873216071@qq.com>
 ;;; Copyright © 2024 David Elsing <david.elsing@posteo.net>
 ;;; Copyright © 2024 Romain Garbage <romain.garbage@inria.fr>
+;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -49,13 +50,16 @@ (define-module (gnu packages parallel)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bash)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages databases)
   #:use-module (gnu packages documentation)
   #:use-module (gnu packages flex)
   #:use-module (gnu packages freeipmi)
+  #:use-module (gnu packages glib)
   #:use-module (gnu packages libevent)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages maths)
   #:use-module (gnu packages mpi)
+  #:use-module (gnu packages networking)
   #:use-module (gnu packages perl)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
@@ -63,6 +67,7 @@ (define-module (gnu packages parallel)
   #:use-module (gnu packages python-science)
   #:use-module (gnu packages python-xyz)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages serialization)
   #:use-module (gnu packages tcl)
   #:use-module (gnu packages tls)
   #:use-module (gnu packages web))
@@ -224,12 +229,17 @@ (define-public slurm
                   (mkdir "contribs")
                   (rename-file "tmp-pmi" "contribs/pmi")
                   (rename-file "tmp-pmi2" "contribs/pmi2")))))
-    ;; FIXME: More optional inputs could be added,
-    ;; in particular mysql and gtk+.
-    (inputs (list freeipmi
+    ;; FIXME: More optional inputs (for example, gtk+) could be added.
+    (inputs (list dbus
+                  freeipmi
+                  http-parser
                   `(,hwloc-2 "lib")
                   json-c
+                  libjwt
+                  librdkafka
+                  libyaml
                   linux-pam
+                  `(,mariadb "dev")
                   openpmix
                   munge
                   numactl
@@ -239,13 +249,22 @@ (define-public slurm
     (build-system gnu-build-system)
     (arguments
      (list #:configure-flags
-           #~(list "--enable-pam" "--sysconfdir=/etc/slurm"
+           #~(list "--enable-pam"
+                   "--enable-slurmrestd"
+                   "--sysconfdir=/etc/slurm"
                    "--disable-static"
+                   (string-append "--with-bpf="
+                                  (dirname (dirname (search-input-directory
+                                                     %build-inputs "include/linux"))))
                    (string-append "--with-freeipmi=" #$(this-package-input "freeipmi"))
+                   (string-append "--with-http-parser="
+                                  #$(this-package-input "http-parser"))
                    (string-append "--with-hwloc="
                                   (ungexp (this-package-input "hwloc") "lib"))
                    (string-append "--with-json=" #$(this-package-input "json-c"))
                    (string-append "--with-munge=" #$(this-package-input "munge"))
+                   (string-append "--with-rdkafka=" #$(this-package-input "librdkafka"))
+                   (string-append "--with-yaml=" #$(this-package-input "libyaml"))
 
                    ;; Use PMIx bundled with Open MPI (this is required for Open MPI 5.x).
                    ;; Note: Older versions that inherit from this package lack the
-- 
2.45.2





^ permalink raw reply related	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-09 15:24   ` [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API Arun Isaac
@ 2024-09-10  6:50     ` Ludovic Courtès
  2024-09-10 15:02       ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-10  6:50 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hello Arun!

Arun Isaac <arunisaac@systemreboot.net> skribis:

> * gnu/packages/parallel.scm (slurm)[inputs]: Add dbus, http-parser, libjwt,
> librdkafka, libtaml and (mariadb "dev").
> [arguments]: Add --enable-slurmrestd, --with-bpf, --with-http-parser,
> --with-rdkafka and --with-yaml to configure flags.
>
> Change-Id: Icddaa8f883754129d975e82fccd732a47a467517

[...]

> +    (inputs (list dbus
> +                  freeipmi
> +                  http-parser
>                    `(,hwloc-2 "lib")
>                    json-c
> +                  libjwt
> +                  librdkafka
> +                  libyaml
>                    linux-pam
> +                  `(,mariadb "dev")

Could you check how this impacts ‘guix size slurm’?

The reason I’m asking is that we at work often end up with slurm in
packs that we carry to supercomputers, so it would be best if it wasn’t
too big:

--8<---------------cut here---------------start------------->8---
$ guix size slurm |tail -1
total: 359.8 MiB
$ guix size slurm mariadb |tail -1
total: 663.4 MiB
--8<---------------cut here---------------end--------------->8---

I wonder if we should have a ‘slurm-minimal’ variant or something?

(That could probably come as a patch after this series.)

Apart from that, both patches look good to me!

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API.
  2024-09-06 16:03     ` Arun Isaac
  2024-09-06 16:06       ` Arun Isaac
@ 2024-09-10  8:32       ` Ludovic Courtès
  1 sibling, 0 replies; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-10  8:32 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hey,

Arun Isaac <arunisaac@systemreboot.net> skribis:

>> I’m curious what the REST interface has to offer compared to DRMAA.
>
> I don't know, to be honest.
>
> I've started working on ravanan, a new implementation of the Common
> Workflow Language that takes inspiration from Guix and integrates with
> Guix for strong reproducibility.
>
> https://git.systemreboot.net/ravanan/about/

Nice!

> ravanan uses the slurm REST API to talk to slurm, and that's why I need
> this patch. I chose the slurm REST API over DRMAA rather arbitrarily. In
> the future, I could support both in ravanan. So, it's still good to have
> both.

I’m mentioning DRMAA because Ricardo wrote guile-drmaa for similar
purposes, and I think DRMAA is implemented by several batch schedulers.
But I don’t know whether it would be a good fit for what you need.

Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-10  6:50     ` Ludovic Courtès
@ 2024-09-10 15:02       ` Arun Isaac
  2024-09-12  8:24         ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-10 15:02 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


Hi Ludo,

> Could you check how this impacts ‘guix size slurm’?

The size goes up from 141 MiB to 407 MiB.

> The reason I’m asking is that we at work often end up with slurm in
> packs that we carry to supercomputers, so it would be best if it wasn’t
> too big:
>
> --8<---------------cut here---------------start------------->8---
> $ guix size slurm |tail -1
> total: 359.8 MiB
> $ guix size slurm mariadb |tail -1
> total: 663.4 MiB
> --8<---------------cut here---------------end--------------->8---
>
> I wonder if we should have a ‘slurm-minimal’ variant or something?
>
> (That could probably come as a patch after this series.)

Yes, I could add a slurm-minimal. Would you like to have a slurm-minimal
for every version of slurm or just the latest?

Regards,
Arun




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-10 15:02       ` Arun Isaac
@ 2024-09-12  8:24         ` Ludovic Courtès
  2024-09-12 11:51           ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-12  8:24 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hello!

Arun Isaac <arunisaac@systemreboot.net> skribis:

>> Could you check how this impacts ‘guix size slurm’?
>
> The size goes up from 141 MiB to 407 MiB.

D’oh!

>> I wonder if we should have a ‘slurm-minimal’ variant or something?
>>
>> (That could probably come as a patch after this series.)
>
> Yes, I could add a slurm-minimal. Would you like to have a slurm-minimal
> for every version of slurm or just the latest?

Hmm!  The reason we keep different versions of slurm is because one
always needs a slurm client compatible with the version of slurmd
running on the target cluster.  And so yes, we’d need ‘slurm-minimal’
for each version.

But that doesn’t sound great so I’m not sure.

Another possibility would be to do the opposite: keep ‘slurm’ without
MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells
and whistles?

Thanks,
Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-12  8:24         ` Ludovic Courtès
@ 2024-09-12 11:51           ` Arun Isaac
  2024-09-12 15:31             ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-12 11:51 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


Hi Ludo,

> Another possibility would be to do the opposite: keep ‘slurm’ without
> MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells
> and whistles?

The problem with that approach is that I'm sure there will be more
optional features to add to slurm. Then, we would have a combinatorial
explosion of packages---slurm-rest, slurm-rest+feature1,
slurm-rest+feature2, slurm-feature1+feature2, etc. I am ok with having a
slurm-minimal for all versions. That's the lesser evil.

Regards,
Arun




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-12 11:51           ` Arun Isaac
@ 2024-09-12 15:31             ` Ludovic Courtès
  2024-09-16 21:55               ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-12 15:31 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Hi,

Arun Isaac <arunisaac@systemreboot.net> skribis:

>> Another possibility would be to do the opposite: keep ‘slurm’ without
>> MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells
>> and whistles?
>
> The problem with that approach is that I'm sure there will be more
> optional features to add to slurm. Then, we would have a combinatorial
> explosion of packages---slurm-rest, slurm-rest+feature1,
> slurm-rest+feature2, slurm-feature1+feature2, etc.

Not necessarily: that second variant could be maximal, where additional
features would be added.

> I am ok with having a slurm-minimal for all versions. That's the
> lesser evil.

OK.

There might be a third option: moving server-side code to a separate
output.  That way, perhaps client-side code (things like ‘openmpi’)
would not have MariaDB & co. in their closure?

Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-12 15:31             ` Ludovic Courtès
@ 2024-09-16 21:55               ` Arun Isaac
  2024-09-17  6:37                 ` Ludovic Courtès
  0 siblings, 1 reply; 19+ messages in thread
From: Arun Isaac @ 2024-09-16 21:55 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


>> I am ok with having a slurm-minimal for all versions. That's the
>> lesser evil.
>
> OK.

I'll send a v3 patchset with the minimal variants soon..

> There might be a third option: moving server-side code to a separate
> output.  That way, perhaps client-side code (things like ‘openmpi’)
> would not have MariaDB & co. in their closure?

I presume you might need to carry both the server-side and the
client-side code in a guix pack. So, how would this option help?




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-16 21:55               ` Arun Isaac
@ 2024-09-17  6:37                 ` Ludovic Courtès
  2024-09-17 14:54                   ` Arun Isaac
  0 siblings, 1 reply; 19+ messages in thread
From: Ludovic Courtès @ 2024-09-17  6:37 UTC (permalink / raw)
  To: Arun Isaac; +Cc: 72766

Morning!

Arun Isaac <arunisaac@systemreboot.net> skribis:

>> There might be a third option: moving server-side code to a separate
>> output.  That way, perhaps client-side code (things like ‘openmpi’)
>> would not have MariaDB & co. in their closure?
>
> I presume you might need to carry both the server-side and the
> client-side code in a guix pack. So, how would this option help?

No, to run code on a supercomputer, you only need MPI and things like
‘salloc’, ‘sbatch’, etc.  There’s definitely no need for ‘slurmd’ or
anything like that in that case because it’s already running on the
machine.

Ludo’.




^ permalink raw reply	[flat|nested] 19+ messages in thread

* [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API.
  2024-09-17  6:37                 ` Ludovic Courtès
@ 2024-09-17 14:54                   ` Arun Isaac
  0 siblings, 0 replies; 19+ messages in thread
From: Arun Isaac @ 2024-09-17 14:54 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: 72766


>> I presume you might need to carry both the server-side and the
>> client-side code in a guix pack. So, how would this option help?
>
> No, to run code on a supercomputer, you only need MPI and things like
> ‘salloc’, ‘sbatch’, etc.  There’s definitely no need for ‘slurmd’ or
> anything like that in that case because it’s already running on the
> machine.

Ah, ok, the server-side is maintained by a different admin team. I was
thinking you were maintaining the server-side as well. That makes sense,
thank you!

In that case, I'll split slurm into two outputs---a server and a
client-side. I'll send a patch in the coming days.




^ permalink raw reply	[flat|nested] 19+ messages in thread

end of thread, other threads:[~2024-09-17 14:56 UTC | newest]

Thread overview: 19+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-08-22 23:30 [bug#72766] [PATCH 0/2] slurm: Enable REST API Arun Isaac
2024-08-22 23:36 ` [bug#72766] [PATCH 1/2] gnu: Add libjwt Arun Isaac
2024-08-22 23:36 ` [bug#72766] [PATCH 2/2] gnu: slurm: Enable REST API Arun Isaac
2024-09-02  7:43   ` Ludovic Courtès
2024-09-06 16:03     ` Arun Isaac
2024-09-06 16:06       ` Arun Isaac
2024-09-09  7:13         ` Ludovic Courtès
2024-09-09 14:59           ` Arun Isaac
2024-09-10  8:32       ` Ludovic Courtès
2024-09-09 15:24 ` [bug#72766] [PATCH v2 1/2] gnu: Add libjwt Arun Isaac
2024-09-09 15:24   ` [bug#72766] [PATCH v2 2/2] gnu: slurm: Enable REST API Arun Isaac
2024-09-10  6:50     ` Ludovic Courtès
2024-09-10 15:02       ` Arun Isaac
2024-09-12  8:24         ` Ludovic Courtès
2024-09-12 11:51           ` Arun Isaac
2024-09-12 15:31             ` Ludovic Courtès
2024-09-16 21:55               ` Arun Isaac
2024-09-17  6:37                 ` Ludovic Courtès
2024-09-17 14:54                   ` Arun Isaac

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).