From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp2.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms13.migadu.com with LMTPS id uN36D3y4G2dNWAAAe85BDQ:P1 (envelope-from ) for ; Fri, 25 Oct 2024 15:25:48 +0000 Received: from aspmx1.migadu.com ([2001:41d0:403:58f0::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp2.migadu.com with LMTPS id uN36D3y4G2dNWAAAe85BDQ (envelope-from ) for ; Fri, 25 Oct 2024 17:25:48 +0200 X-Envelope-To: larch@yhetil.org Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=YBUaK2Ff; dkim=fail ("headers rsa verify failed") header.d=systemreboot.net header.s=default header.b=SgfPuy8x; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=systemreboot.net (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1729869948; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=pYlfdnECAH4sdA2o1jCLLm+Zmq+5cs2FjASPS/cqO90=; b=mKQkzMexSBZe+xqMkYm6Ngzrz/ZRSCZIjQ//uwfWfYHT83b6LX+mgEkKWqVJOpsu5+HtG2 +jjMZPDYdOIaAETQYX4Lr68xCp5LrwyP3SryoaoSagE0yqpVohXQTiuW98hDsRPjQGKs1q HpaJgPRNcFJlqfVltXlehGlMXKcFivI2TJb4t7qjs8GORQ79rp+kDHdx8Z8ppz/VIiueNC bDTr4rMAuGVqBJQ/fomQEtb8Bzd+DcbEi/cZwtxP0q6EoMShMOm1TffMZ7EV95l7sarFVQ 1Ubs3cTw7hAAPJEqCt4kuDX5QqOdzSMNuAvn6gi8AfupSODqEDZ31L9uNfFyoA== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=debbugs.gnu.org header.s=debbugs-gnu-org header.b=YBUaK2Ff; dkim=fail ("headers rsa verify failed") header.d=systemreboot.net header.s=default header.b=SgfPuy8x; spf=pass (aspmx1.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed), DKIM not aligned (relaxed)" header.from=systemreboot.net (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1729869948; a=rsa-sha256; cv=none; b=h4PTGX3K/h73WiS3wIn2SqT7oTI4tNiXHGN7+wep60A5n9xMELhjCiHUhjIP5RTzkvENXU YrXaX3z5kjKdLoPEuimiIpZoExQwi4hRuvpbOE1Fz/hghyYK3k7XKJy2Bi2nkRj49qEdGV LKXKAaxa2H4TjnHjBi9e2om7iI78WRmCdS2tjYVqsBt/MLM6OZOi+Hn2GNa0oeUx0nZddo m03GWtxoVdctYgjbKytrS3ogmT7KHxpQTu1/6DWrrxUtHXjl9x3m8XQH7aKl2bTsakLdk2 W8VmBTPBoGWflklXqPSdrhfGHC7ptUz/rakLJJWe0g4AEom3V/5xnw7u5y2XKA== Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 809F87E383 for ; Fri, 25 Oct 2024 17:25:46 +0200 (CEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1t4MC3-00058v-9V; Fri, 25 Oct 2024 11:25:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1t4MC1-00058m-L3 for guix-patches@gnu.org; Fri, 25 Oct 2024 11:25:37 -0400 Received: from debbugs.gnu.org ([2001:470:142:5::43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1t4MC1-00007u-CA; Fri, 25 Oct 2024 11:25:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=debbugs.gnu.org; s=debbugs-gnu-org; h=MIME-Version:Date:From:To:In-Reply-To:References:Subject; bh=pYlfdnECAH4sdA2o1jCLLm+Zmq+5cs2FjASPS/cqO90=; b=YBUaK2FfQBZ/kCUDyPRp1I/qjtjTBD8RgUc5wZEFEPSQPt52tWx74vh60PqBo1co0zSTWBkNF3x8XGc+TVen6h4PjaAsVjPvX0FL0wv24oqLtuWvCkHxMagFag/6+UbuA3VBx4b6pbQa3FhgLMVPkYwz0qYVo0tdeVw7KwSNf2kSEQxwVOLpvuB0Q0L+qav7hW7NOYVg/8nuXYInLlqZg+DAkeVcXF5bi6Wn7t+pKBmuTAqV6iXmUgPn8hJHUuBoZhuCIHugStkFOGTGpn/IHaqR0JMgBd/6LeC4VTfGVn/7jOzKOB6SHOMlT+klp9qbHvVTHIWzfWEOmr54buWY1w==; Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1t4MCQ-0001tE-HO; Fri, 25 Oct 2024 11:26:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#72766] [PATCH v4] gnu: slurm: Enable REST API. References: In-Reply-To: Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: arunisaac@systemreboot.net, ludovic.courtes@inria.fr, ludo@gnu.org, guix-patches@gnu.org Resent-Date: Fri, 25 Oct 2024 15:26:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 72766 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 72766@debbugs.gnu.org Cc: Arun Isaac , Arun Isaac , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Ludovic =?UTF-8?Q?Court=C3=A8s?= X-Debbugs-Original-Xcc: Arun Isaac , Ludovic =?UTF-8?Q?Court=C3=A8s?= , Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 72766-submit@debbugs.gnu.org id=B72766.17298699216845 (code B ref 72766); Fri, 25 Oct 2024 15:26:02 +0000 Received: (at 72766) by debbugs.gnu.org; 25 Oct 2024 15:25:21 +0000 Received: from localhost ([127.0.0.1]:39205 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t4MBk-0001mK-4S for submit@debbugs.gnu.org; Fri, 25 Oct 2024 11:25:20 -0400 Received: from mugam.systemreboot.net ([139.59.75.54]:60026) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1t4MBg-0001mA-Ei for 72766@debbugs.gnu.org; Fri, 25 Oct 2024 11:25:18 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Transfer-Encoding:Content-Type: MIME-Version:Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=pYlfdnECAH4sdA2o1jCLLm+Zmq+5cs2FjASPS/cqO90=; b=SgfPuy8xT2T7bnuujJuMOl96T2 vYGqitTwQhKLWW7AD+r4SNd2W32vO8HxYVLZEghMurNHJPbEOU/ELXeaAm78942CjDxzYtXwqjvRs Bhsy/dJFGIxJDnkfmkXW7POAftTt4sQUazhZChPJjq1uH6ppcw4dlPVLZFatKMNPv9WktEOV9jnJ7 RZs2c1D/0KWYKm9/iOToZzSAh8eXm05jqttPSZlH3bqQQjNFfPv4PWYnG8Kg9jvMbtm2Q19wjGz6F jfLnYZBLoPVh5/NNPCPzovB/V9yW9F80CHvHreo26N4vzTmcrIsGh2HRiTNbAPK3H22T1UT/ZfbBp ogyOnKeg==; Received: from [192.168.2.1] (port=41908 helo=localhost.localdomain) by systemreboot.net with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98) (envelope-from ) id 1t4M4y-000000003me-3mXf; Fri, 25 Oct 2024 16:18:21 +0100 From: Arun Isaac Date: Fri, 25 Oct 2024 16:24:30 +0100 Message-ID: <64fa0bf0f949a50512b2dc91ed436a5e96d71a77.1729869863.git.arunisaac@systemreboot.net> X-Mailer: git-send-email 2.45.2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Country: US X-Migadu-Flow: FLOW_IN X-Migadu-Scanner: mx11.migadu.com X-Migadu-Spam-Score: 4.31 X-Spam-Score: 4.31 X-Migadu-Queue-Id: 809F87E383 X-TUID: 8WNx6Nhnor7f Rename slurm and slurm-* to slurm-minimal and slurm-minimal-*. Add new slurm packages that inherit from slurm-minimal and have the REST API enabled. * gnu/packages/parallel.scm (make-slurm): New function. (slurm-minimal, slurm-minimal-23.02, slurm-minimal-22.05, slurm-minimal-21.08, slurm-minimal-20.11, slurm-minimal-20.02, slurm-minimal-19.05, slurm-minimal-18.08): New variables. (slurm, slurm-23.02, slurm-22.05, slurm-21.08, slurm-20.11, slurm-20.02, slurm-19.05, slurm-18.08): Inherit from slurm-minimal, slurm-minimal-23.02, slurm-minimal-22.05, slurm-minimal-21.08, slurm-minimal-20.11, slurm-minimal-20.02, slurm-minimal-19.05 and slurm-minimal-18.08 respectively. Change-Id: I6410ac385d8876f4b6a908ddb1272172475a87bb --- gnu/packages/parallel.scm | 98 ++++++++++++++++++++++++++++----------- 1 file changed, 72 insertions(+), 26 deletions(-) diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm index 89d82fb8da..66997951c3 100644 --- a/gnu/packages/parallel.scm +++ b/gnu/packages/parallel.scm @@ -15,6 +15,7 @@ ;;; Copyright © 2024 Zheng Junjie <873216071@qq.com> ;;; Copyright © 2024 David Elsing ;;; Copyright © 2024 Romain Garbage +;;; Copyright © 2024 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -49,14 +50,17 @@ (define-module (gnu packages parallel) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages check) + #:use-module (gnu packages databases) #:use-module (gnu packages documentation) #:use-module (gnu packages flex) #:use-module (gnu packages freeipmi) #:use-module (gnu packages gcc) + #:use-module (gnu packages glib) #:use-module (gnu packages libevent) #:use-module (gnu packages linux) #:use-module (gnu packages maths) #:use-module (gnu packages mpi) + #:use-module (gnu packages networking) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) @@ -64,6 +68,7 @@ (define-module (gnu packages parallel) #:use-module (gnu packages python-science) #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) + #:use-module (gnu packages serialization) #:use-module (gnu packages tcl) #:use-module (gnu packages tls) #:use-module (gnu packages web)) @@ -190,9 +195,9 @@ (define-public xjobs when jobs finish.") (license license:gpl2+))) -(define-public slurm +(define-public slurm-minimal (package - (name "slurm") + (name "slurm-minimal") (version "23.11.10") (source (origin (method url-fetch) @@ -223,8 +228,6 @@ (define-public slurm (mkdir "contribs") (rename-file "tmp-pmi" "contribs/pmi") (rename-file "tmp-pmi2" "contribs/pmi2"))))) - ;; FIXME: More optional inputs could be added, - ;; in particular mysql and gtk+. (inputs (list freeipmi `(,hwloc-2 "lib") json-c @@ -291,6 +294,35 @@ (define-public slurm "https://github.com/SchedMD/slurm/blob/master/contribs/pmi2/COPYRIGHT") license:gpl2+)))) ; the rest, often with OpenSSL exception +(define (make-slurm base-slurm) + "Make a slurm package with all optional features enabled. Base it off of the +minimal slurm package BASE-SLURM." + (package + (inherit base-slurm) + (name "slurm") + (arguments + (substitute-keyword-arguments (package-arguments base-slurm) + ((#:configure-flags flags #~'()) + #~(cons* "--enable-slurmrestd" + (string-append "--with-bpf=" + (dirname + (dirname (search-input-directory + %build-inputs "include/linux")))) + (string-append "--with-http-parser=" + #$(this-package-input "http-parser")) + (string-append "--with-rdkafka=" + #$(this-package-input "librdkafka")) + (string-append "--with-yaml=" + #$(this-package-input "libyaml")) + #$flags)))) + ;; FIXME: More optional inputs could be added. + (inputs + (modify-inputs (package-inputs base-slurm) + (prepend dbus freeipmi http-parser + libjwt librdkafka libyaml (list mariadb "dev")))))) + +(define-public slurm (make-slurm slurm-minimal)) + ;; The SLURM client/daemon protocol and file format changes from time to time ;; in incompatible ways, as noted in ;; . Thus, keep older @@ -298,9 +330,9 @@ (define-public slurm ;; As noted in the link, YY.MM is the release scheme, and the 'maintenance' ;; digit does not introduce incompatibilities. -(define-public slurm-23.02 +(define-public slurm-minimal-23.02 (package - (inherit slurm) + (inherit slurm-minimal) (version "23.02.6") (source (origin (inherit (package-source slurm)) @@ -314,12 +346,14 @@ (define-public slurm-23.02 (base32 "08rz3r1rlnb3pmfdnbh542gm44ja0fdy8rkj4vm4lclc48cvqp2a")))))) -(define-public slurm-22.05 +(define-public slurm-23.02 (make-slurm slurm-minimal-23.02)) + +(define-public slurm-minimal-22.05 (package - (inherit slurm-23.02) + (inherit slurm-minimal-23.02) (version "22.05.1") (source (origin - (inherit (package-source slurm-23.02)) + (inherit (package-source slurm-minimal-23.02)) (method url-fetch) (uri (string-append "https://download.schedmd.com/slurm/slurm-" @@ -329,12 +363,14 @@ (define-public slurm-22.05 (base32 "0f3hhlki8g7slllsnyj1qikbsvr62i0hig85lcdcfnmsagzlhbyi")))))) -(define-public slurm-21.08 +(define-public slurm-22.05 (make-slurm slurm-minimal-22.05)) + +(define-public slurm-minimal-21.08 (package - (inherit slurm-22.05) + (inherit slurm-minimal-22.05) (version "21.08.8") (source (origin - (inherit (package-source slurm-22.05)) + (inherit (package-source slurm-minimal-22.05)) (method url-fetch) (uri (string-append "https://download.schedmd.com/slurm/slurm-" @@ -346,15 +382,17 @@ (define-public slurm-21.08 ;; This and older versions of slurm have PMIx support but they seem to ;; require an older version of openpmix. Disable PMIx support. - (inputs (modify-inputs (package-inputs slurm-22.05) + (inputs (modify-inputs (package-inputs slurm-minimal-22.05) (delete "openpmix"))))) -(define-public slurm-20.11 +(define-public slurm-21.08 (make-slurm slurm-minimal-21.08)) + +(define-public slurm-minimal-20.11 (package - (inherit slurm-21.08) + (inherit slurm-minimal-21.08) (version "20.11.9") (source (origin - (inherit (package-source slurm-21.08)) + (inherit (package-source slurm-minimal-21.08)) (method url-fetch) (uri (string-append "https://download.schedmd.com/slurm/slurm-" @@ -364,12 +402,14 @@ (define-public slurm-20.11 (base32 "0xq2d6dm285y541dyg1h66z7svsisrq8c81ag0f601xz1cn3mq9m")))))) -(define-public slurm-20.02 +(define-public slurm-20.11 (make-slurm slurm-minimal-20.11)) + +(define-public slurm-minimal-20.02 (package - (inherit slurm-20.11) + (inherit slurm-minimal-20.11) (version "20.02.6-1") (source (origin - (inherit (package-source slurm-20.11)) + (inherit (package-source slurm-minimal-20.11)) (method url-fetch) (uri (string-append "https://download.schedmd.com/slurm/slurm-" @@ -379,17 +419,19 @@ (define-public slurm-20.02 (base32 "0qj4blfymrd2ry2qmb58l3jbr4jwygc3adcfw7my27rippcijlyc")))) (arguments - (substitute-keyword-arguments (package-arguments slurm-20.11) + (substitute-keyword-arguments (package-arguments slurm-minimal-20.11) ((#:configure-flags flags ''()) #~(append '("CFLAGS=-O2 -g -fcommon" "LDFLAGS=-fcommon") #$flags)))))) -(define-public slurm-19.05 +(define-public slurm-20.02 (make-slurm slurm-minimal-20.02)) + +(define-public slurm-minimal-19.05 (package - (inherit slurm-20.02) + (inherit slurm-minimal-20.02) (version "19.05.8") (source (origin - (inherit (package-source slurm-20.02)) + (inherit (package-source slurm-minimal-20.02)) (method url-fetch) (uri (string-append "https://download.schedmd.com/slurm/slurm-" @@ -399,14 +441,16 @@ (define-public slurm-19.05 (base32 "10c9j4a9a6d4ibpf75006mn03p8xgpaprc247x2idakysjf2fw43")))))) +(define-public slurm-19.05 (make-slurm slurm-minimal-19.05)) + ;; Same as Debian 10 -(define-public slurm-18.08 +(define-public slurm-minimal-18.08 (package - (inherit slurm-19.05) + (inherit slurm-minimal-19.05) (version "18.08.9") (source (origin - (inherit (package-source slurm-20.02)) + (inherit (package-source slurm-minimal-20.02)) (uri (string-append "https://download.schedmd.com/slurm/slurm-" version ".tar.bz2")) @@ -415,6 +459,8 @@ (define-public slurm-18.08 (base32 "1bgrpz75m7l4xhirsd0fvnkzlkrl8v2qpmjcz60barc5qm2kn457")))))) +(define-public slurm-18.08 (make-slurm slurm-minimal-18.08)) + (define-public slurm-drmaa (package (name "slurm-drmaa") base-commit: 2394a7f5fbf60dd6adc0a870366adb57166b6d8b -- 2.45.2