From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp12.migadu.com ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms5.migadu.com with LMTPS id eNaLDFr09WM0HgAAbAwnHQ (envelope-from ) for ; Wed, 22 Feb 2023 11:54:18 +0100 Received: from aspmx2.migadu.com ([2001:41d0:2:c151::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp12.migadu.com with LMTPS id QF5pDFr09WOGzQAAauVa8A (envelope-from ) for ; Wed, 22 Feb 2023 11:54:18 +0100 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx2.migadu.com (Postfix) with ESMTPS id D8D032B692 for ; Wed, 22 Feb 2023 11:54:17 +0100 (CET) Authentication-Results: aspmx2.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=D8U6HgGY; spf=pass (aspmx2.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Seal: i=1; s=key1; d=yhetil.org; t=1677063258; a=rsa-sha256; cv=none; b=MPk7ky2yevb/H82wX4DaKA9eqm9bVYuGlvPhQXKFoilR4TNqhHurQpHNErADvnlLKHtYLl Z5EE8HqMnPrTRZHB+NCTopos3odRr84DZQh6P4TRWJHRS5RISpSQRBPTLgaRD64hQMrmlE nUFZeAcU1RfvWs+ygq+2jCB7zM6ZxgiZ1HVJx6qUV3XevU82YNMFMqUHki2VBru+Dn64nX mTlQhVyvZrupuiw4mWAMAnFeYeU4N7lnuAVpKcf5d0DQdmE1xRXPgKcco9cMCTTVaOJUvM hzvosFYg7FPmJwGvSG3T6AEdJrCu/6QY+m5HSyC13+P8FXj/CrGQUddXFjxNAQ== ARC-Authentication-Results: i=1; aspmx2.migadu.com; dkim=fail ("headers rsa verify failed") header.d=gmail.com header.s=20210112 header.b=D8U6HgGY; spf=pass (aspmx2.migadu.com: domain of "guix-patches-bounces+larch=yhetil.org@gnu.org" designates 209.51.188.17 as permitted sender) smtp.mailfrom="guix-patches-bounces+larch=yhetil.org@gnu.org"; dmarc=fail reason="SPF not aligned (relaxed)" header.from=gmail.com (policy=none) ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1677063258; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:resent-cc: resent-from:resent-sender:resent-message-id:in-reply-to:in-reply-to: references:references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:dkim-signature; bh=vJk8FE+jvLwcAgq1J9sI7tEPIx7FX8zWJpbfGO+WpHM=; b=nZQwmNZwnmIqDIIrCplXMrVRFSLEhWygxyxnbwD6IP6EgBlWU3s7u+RM4rWYSJZP41SiWf 2OSNrL3yNlq2zfvSi3vRC4pjN1X5VrcxdFxIqLHzybya122rlGvZ4guopm7bYuO/CTismj srCNVRZxv6d3VZcEQsT4mx2fmL0czp54Z6Cyk1CQ9OJK2vrQGQw1PRkAZZUvZgqEijY4qF rUrZWnPbvFzztXSQmfRkKpxVrht51woVZnk8NKDHvMQU3Agm9adOAL+qZwzGFNzzzbJa82 ajPqnT22mWjJTVIe0vsZlIMkG9fsTdhUDvVa07p75X+OEOWXoolt1naedMj9lw== Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pUmlB-0008Qs-8E; Wed, 22 Feb 2023 05:54:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pUml9-0008QR-J0 for guix-patches@gnu.org; Wed, 22 Feb 2023 05:54:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pUml8-0001Ms-TS for guix-patches@gnu.org; Wed, 22 Feb 2023 05:54:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pUml8-0001tI-D5 for guix-patches@gnu.org; Wed, 22 Feb 2023 05:54:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#61701] [PATCH] doc: Propose new cookbook section for reproducible research. Resent-From: Simon Tournier Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 22 Feb 2023 10:54:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 61701 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: kyle , 61701@debbugs.gnu.org Cc: Kyle Andrews Received: via spool by 61701-submit@debbugs.gnu.org id=B61701.16770631977210 (code B ref 61701); Wed, 22 Feb 2023 10:54:02 +0000 Received: (at 61701) by debbugs.gnu.org; 22 Feb 2023 10:53:17 +0000 Received: from localhost ([127.0.0.1]:58111 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pUmkP-0001sD-7K for submit@debbugs.gnu.org; Wed, 22 Feb 2023 05:53:17 -0500 Received: from mail-wm1-f53.google.com ([209.85.128.53]:36857) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pUmkN-0001rz-2M for 61701@debbugs.gnu.org; Wed, 22 Feb 2023 05:53:15 -0500 Received: by mail-wm1-f53.google.com with SMTP id l2-20020a05600c1d0200b003e1f6dff952so5870572wms.1 for <61701@debbugs.gnu.org>; Wed, 22 Feb 2023 02:53:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1677063189; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:from:to:cc:subject:date:message-id :reply-to; bh=vJk8FE+jvLwcAgq1J9sI7tEPIx7FX8zWJpbfGO+WpHM=; b=D8U6HgGYy2lJWLA3iYAPymc3jK1815TjZ/2TFMDONQcqFC/fFK/udSq2SYpOuy5KIe Z0UbhlYs+q2qCao3cOuoCPwa84Ts1urr6rsy8IzdWkAHnAMNq/ruL+ejVPXZXxdkXXSu xQsoiQXIi+edWskRhM0giyrBlwTOuXWHhmyh1KENYxaaS3rVk/y1/Wkcq231ZGn1Kxzw uugDeQU1ImGHGp5hsWQsAzV+jCW0O/42iCp9GeZUhehsR5oI6EQFmbY4hP2EupOqwDs4 ukG1vKiL758JcVpqm4PKDZEQZluJ490Q+OQtmkUUUqNxMQVVsehI+0r87VDUXsoXoDs2 IREQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1677063189; h=content-transfer-encoding:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vJk8FE+jvLwcAgq1J9sI7tEPIx7FX8zWJpbfGO+WpHM=; b=wl7eftAVd5YKG7u1MgWgxpKdufep6BFYYUv0xA4VZB5owSvNp1m/et+Ou5N/PC8xqF WJoHCsrBJR0zBc2BDnKUUTPuZCp+lRaxHGyhmvh1nK3GuFB4a9RXFHVzbEp/VNvz56MM WaIxzyAiokVKC7zmbIX1GPkwe0LNQqNDp1U90+g6PzLb5gG03a3uYt6Rk7210NMHrzQH zkQzmLUTgtvmTJ8omtyMo06eznGC0FCXUkaVBSnZnyYAGKSM59hr37IxFqCKLjHB5UGA irzavNcBLTUW5ZXu09/NlUEAwpqmm5AW6ofFLBWkVqa6PYQLfF2LdPqsaSw5jCDNPpIk j/sA== X-Gm-Message-State: AO0yUKX8G5wAX0qHcHwzaOX0y59cn7JGvyfOd62WuL+UxQ6ogyfztp0C 2x8SatVCIxGjUuPSi8z0pio= X-Google-Smtp-Source: AK7set+ECNNv4zNr+NgRzNHduZPFtYBglQ70ivYLRkPYrZ7uQIeLoukt3RQh+XUblefHEyNUb5IdEg== X-Received: by 2002:a05:600c:1d1a:b0:3e0:b1:c12d with SMTP id l26-20020a05600c1d1a00b003e000b1c12dmr7415799wms.1.1677063188609; Wed, 22 Feb 2023 02:53:08 -0800 (PST) Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e]) by smtp.gmail.com with ESMTPSA id k16-20020a7bc410000000b003e21f959453sm2155612wmi.32.2023.02.22.02.53.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Feb 2023 02:53:08 -0800 (PST) From: Simon Tournier In-Reply-To: <3ffea5b37541a6f3409299f3e8e6200bc1c9aef6.1677043049.git.kyle@posteo.net> References: <3ffea5b37541a6f3409299f3e8e6200bc1c9aef6.1677043049.git.kyle@posteo.net> Date: Wed, 22 Feb 2023 11:52:43 +0100 Message-ID: <86h6vet1j8.fsf@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: X-Migadu-Queue-Id: D8D032B692 X-Spam-Score: -2.56 X-Migadu-Spam-Score: -2.56 X-Migadu-Scanner: scn0.migadu.com List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+larch=yhetil.org@gnu.org Sender: guix-patches-bounces+larch=yhetil.org@gnu.org X-Migadu-Flow: FLOW_IN X-Migadu-Country: US X-TUID: Mz+UxAi776H8 Hi, Cool! Nice! My suggestions could be dropped, I mean they are just suggestions. :-) On Wed, 22 Feb 2023 at 05:17, kyle wrote: > +Writing a manifest for packaging R code alone requires only minimal > +knowledge of the Guix infrastructure. This stub should work for most > +cases involving the R packages already in Guix. > + > +@example > +(use-modules > + (gnu packages cran) > + (gnu packages statistics)) > + > +(packages->manifest > + (list r r-tidyverse)) Missing @end example, no? > +R packages are defined predominantly inside of gnu/packages/cran.scm and > +gnu/packages/statistics.scm files under a guix source repository. When presenting Guix, usually I start with: guix shell r r-tidyverse then guix shell r r-tidyverse --export-manifest --8<---------------cut here---------------start------------->8--- ;; What follows is a "manifest" equivalent to the command line you gave. ;; You can store it in a file that you may then pass to any 'guix' command ;; that accepts a '--manifest' (or '-m') option. (specifications->manifest (list "r" "r-tidyverse")) --8<---------------cut here---------------end--------------->8--- which demystifies the =E2=80=9Cmanifest=E2=80=9D. Then, I go via: guix show r to point the location and explains why (gnu packages statistics). So then, people connect the dots between the command line invocation and the manifest: --8<---------------cut here---------------start------------->8--- (use-modules (gnu packages cran) (gnu packages statistics)) (packages->manifest (list r r-tidyverse)) --8<---------------cut here---------------end--------------->8--- Based on that, I introduce modules and why (guix licenses) for examples. > +This manifest can be run with the basic guix shell command: > + > +@example > +guix shell --manifest=3Dmanifest.scm --container > +@end example Do you need =E2=80=99container=E2=80=99? If yes, I should set the scenario= before: We want to run X and Y inside a container, because we want to be sure that we described all we need to run the analysis. Well, something like that. > +Please remember at the end to pin your channels so that others in the > +future know how to recover your exact Guix environment. > + > +@example > +guix describe --format=3Dchannels > channels.scm > +@end example > + > +This can be done with Guix time machine: > + > +@example > +guix time-machine --channels=3Dchannels.scm \ > + -- guix shell --manifest=3Dmanifest.scm --container > +@end example No, the correct is: guix time-machine --channels=3Dchannels.scm \ -- shell --manifest=3Dmanifest.scm --container without the extra =E2=80=99guix=E2=80=99. > +In contrast, the python scientific ecosystem is far less > +standardized. There is no effort made to integrate all python packages > +together. While there is a latest python version, it is less often less > +dominantly used for various reasons such as the fact that python tends > +to be employed with much larger teams than R is. This makes packaging up > +reproducible python environments much more difficult. Adding R together > +with python as a mixture complicates things still further. However, we > +have to be mindful of the goals of reproducible research. > + > +If reproducibility becomes an end in itself and not a catlyst towards > +faster discovery, then Guix will be a non-starter for scientists. Their > +goal is to develop useful understanding about particular aspects of the > +world. Cool paragraph! > +Thankfully, three common scenarios cover the vast majority of > +needs. These are: > + > +@itemize > +@item > +combining standard package definitions with custom package definitions > +@item > +combining package definitions from the current revision with other revis= ions > +@item > +combining package variants which need a modified build-system > +@end itemize > + > +In the rest of the tutorial we develop a manifest which tackles all > +three of these common issues. The hope is that if you see the hardest > +possible common situation as being readily solvable without writing > +thousands of lines of code, researchers will clearly see it as worth the > +effort which will not pose a significant detour from the main line of > +their research. > + > +@example > +(use-modules > + (guix packages) > + (guix download) > + (guix licenses) > + (guix profiles) Before jumping to that, I would add a paragraph after the first manifest explaining that other modules can be used, for instance the module (guix licenses) provides all the licenses known by Guix. Well, something like that. > + (gnu packages) > + (gnu packages cran) > + (guix inferior) > + (guix channels) > + (guix build-system python)) > + > +;; guix import pypi APTED I would add a sentence before this manifest point to the documentation of =E2=80=9Cguix import=E2=80=9D and explaining with one sentence that it a= llows to extract information from PyPI and generates a recipe for Guix. > +(define python-apted > + (package > + (name "python-apted") > + (version "1.0.3") > + (source (origin > + (method url-fetch) > + (uri (pypi-uri "apted" version)) > + (sha256 > + (base32 > + "1sawf6s5c64fgnliwy5w5yxliq2fc215m6alisl7yiflwa0m3ymy")))) > + (build-system python-build-system) > + (home-page "https://github.com/JoaoFelipe/apted") > + (synopsis "APTED algorithm for the Tree Edit Distance") > + (description "APTED algorithm for the Tree Edit Distance") > + (license expat))) > + > +(define last-guix-with-python-3.6 > + (list > + (channel > + (name 'guix) > + (url "https://git.savannah.gnu.org/git/guix.git") > + (commit > + "d66146073def03d1a3d61607bc6b77997284904b")))) > + > +(define connection-to-last-guix-with-python-3.6 > + (inferior-for-channels last-guix-with-python-3.6)) Why do you need an inferior? Is it to avoid the =E2=80=9Cguix time-machine= =E2=80=9D? Ah, no the answer below. :-) > +(define first car) > + > +(define python-3.6 > + (first > + (lookup-inferior-packages > + connection-to-last-guix-with-python-3.6 "python"))) > + > +(define python3.6-numpy > + (first > + (lookup-inferior-packages > + connection-to-last-guix-with-python-3.6 "python-numpy"))) > + > +(define included-packages > + (list r r-reticulate)) > +=20 > +(define inferior-packages > + (list python-3.6 python3.6-numpy)) > + > +(define package-with-python-3.6 > + (package-with-explicit-python python-3.6 > + "python-" "python3.6-" 'python3-variant)) > +=20 > +(define custom-variant-packages > + (list (package-with-python-3.6 python-apted))) > + > +(concatenate-manifest > + (map packages->manifest > + (list > + included-packages > + inferior-packages > + custom-variant-packages))) While this is cool, I would not recommend it as some practise. This kind of mix can lead to various annoyances, IMHO. First, it will scale poorly if you add more inferiors. Second, the probability that the resulting computational environment works well decreases. Anyway, the example is cool! :-) > +@end example > + > +This should produce a profile with the latest R and an older python > +3.6. These should be able to interoperate with code like: > + > +@example > +library(reticulate) > +use_python("python") > +apted =3D import("apted") > +t1 =3D '{a{b}{c}}' > +t2 =3D '{a{b{d}}}' > +metric =3D apted$APTED(t1, t2) > +distance =3D metric$compute_edit_distance() > +@end example This example is cool! > diff --git a/guix/build-system/python.scm b/guix/build-system/python.scm > index c8f04b2298..d4aaab906d 100644 > --- a/guix/build-system/python.scm > +++ b/guix/build-system/python.scm > @@ -36,6 +36,7 @@ (define-module (guix build-system python) > #:use-module (srfi srfi-1) > #:use-module (srfi srfi-26) > #:export (%python-build-system-modules > + package-with-explicit-python > package-with-python2 > strip-python2-variant > default-python Maybe this could be a separated patch. Cheers, simon