unofficial mirror of guix-patches@gnu.org 
 help / color / mirror / code / Atom feed
* [bug#69328] [PATCH 00/12] Better source code recovery from SWH
@ 2024-02-23 14:22 Ludovic Courtès
  2024-02-23 15:48 ` [bug#69328] [PATCH 01/12] lint: Switch to SRFI-71 Ludovic Courtès
                   ` (13 more replies)
  0 siblings, 14 replies; 33+ messages in thread
From: Ludovic Courtès @ 2024-02-23 14:22 UTC (permalink / raw)
  To: 69328
  Cc: Ludovic Courtès, Christopher Baines, Josselin Poiret,
	Ludovic Courtès, Mathieu Othacehe, Ricardo Wurmus,
	Simon Tournier, Tobias Geerinckx-Rice

Hello Guix!

This patch series improves source code recovery from SWH, as a followup
to <https://issues.guix.gnu.org/68741>.

It does several things:

  • ‘guix lint -c archival’ now emits save requests for VCSes other
    than Git.

  • Fix <https://issues.guix.gnu.org/69070>.

  • Allow content-addressed recovery of Mercurial and Subversion
    checkouts.

  • Allow Bazaar recovery using ‘download-nar’ (I didn’t bother with SWH).

  • Have all these things honor the ‘GUIX_DOWNLOAD_SEQUENCE’ environment
    variable.

You can try the various methods like this:

  GUIX_DOWNLOAD_SEQUENCE=nar ./pre-inst-env guix build -S apl --check
  GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-wisp --check
  GUIX_DOWNLOAD_SEQUENCE=swh ./pre-inst-env guix build -S guile-gcrypt --check

In the last case, note that you must be running guix-daemon for the checkout
since that uses “builtin:git-download”, which is implemented on the server
side.

There’s a few caveats:

  • Mercurial SWH fallback almost works, but not quite, due to this SWH bug:
    <https://gitlab.softwareheritage.org/swh/infra/sysadm-environment/-/issues/5256>.

  • Right now, no Subversion checkout has the nar-sha256 ExtID at SWH for
    unclear reasons, so retrieving the source of ‘apl’ (say) from SWH
    doesn’t work yet.

  • Multi-directory Subversion downloads (‘svn-multi-fetch’) is not supported
    yet.  For that we’ll need to arrange with our SWH friends so they
    compute nar-sha256 ExtIDs for combined directories (and we’ll have to
    include that info in ‘sources.json’).

Feedback welcome!

Ludo’.

Ludovic Courtès (12):
  lint: Switch to SRFI-71.
  lint: archival: Fix crash in non-Git case.
  lint: archival: Trigger “Save Code Now” for VCSes other than Git.
  swh: Add ‘type’ field to <visit>.
  swh: ‘origin-visits’ takes an optional ‘max’ parameter.
  swh: ‘lookup-origin-revision’ handles branches pointing to
    directories.
  hg-download: Use ‘swh-download-directory-by-nar-hash’.
  svn-download: Use ‘swh-download-directory-by-nar-hash’.
  bzr-download: Implement nar fallback.
  download-nar: Distinguish ‘output’ and ‘item’ parameter.
  perform-download: Allow use of ‘download-nar’ for ‘--check’ builds.
  download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable.

 guix/build/bzr.scm                |   3 +-
 guix/build/download-nar.scm       |  12 +--
 guix/build/download.scm           |  50 +++++++---
 guix/build/git.scm                |  27 ++++--
 guix/bzr-download.scm             |  57 ++++++++---
 guix/cvs-download.scm             |  24 +++--
 guix/download.scm                 |  53 ++++-------
 guix/git-download.scm             |  20 ++--
 guix/hg-download.scm              |  36 ++++---
 guix/lint.scm                     | 151 +++++++++++++++++++-----------
 guix/scripts/perform-download.scm |  65 +++++++------
 guix/svn-download.scm             |  84 +++++++++++------
 guix/swh.scm                      |  71 ++++++++------
 tests/lint.scm                    |  20 ++++
 tests/swh.scm                     |  74 +++++++++++++++
 15 files changed, 501 insertions(+), 246 deletions(-)


base-commit: ffcce77ec488e3c89401ad77fafa65fcd9e9f5be
-- 
2.41.0





^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2024-03-09 18:52 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-02-23 14:22 [bug#69328] [PATCH 00/12] Better source code recovery from SWH Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 01/12] lint: Switch to SRFI-71 Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 02/12] lint: archival: Fix crash in non-Git case Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 03/12] lint: archival: Trigger “Save Code Now” for VCSes other than Git Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 04/12] swh: Add ‘type’ field to <visit> Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 05/12] swh: ‘origin-visits’ takes an optional ‘max’ parameter Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 06/12] swh: ‘lookup-origin-revision’ handles branches pointing to directories Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 07/12] hg-download: Use ‘swh-download-directory-by-nar-hash’ Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 08/12] svn-download: " Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 09/12] bzr-download: Implement nar fallback Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 10/12] download-nar: Distinguish ‘output’ and ‘item’ parameter Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 11/12] perform-download: Allow use of ‘download-nar’ for ‘--check’ builds Ludovic Courtès
2024-02-23 15:48 ` [bug#69328] [PATCH 12/12] download: Honor ‘GUIX_DOWNLOAD_SEQUENCE’ environment variable Ludovic Courtès
2024-03-03  4:53   ` Timothy Sample
2024-03-05 10:26     ` Ludovic Courtès
2024-02-23 15:53 ` [bug#69328] [PATCH 00/12] Better source code recovery from SWH Ludovic Courtès
2024-03-03  4:54 ` Timothy Sample
2024-03-05 10:58   ` Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 " Ludovic Courtès
2024-03-07 18:38       ` Simon Tournier
2024-03-09 18:51         ` bug#69328: " Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 01/12] lint: Switch to SRFI-71 Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 02/12] lint: archival: Fix crash in non-Git case Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 03/12] lint: archival: Trigger “Save Code Now” for VCSes other than Git Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 04/12] swh: Add ‘type’ field to <visit> Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 05/12] swh: ‘origin-visits’ takes an optional ‘max’ parameter Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 06/12] swh: ‘lookup-origin-revision’ handles branches pointing to directories Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 07/12] hg-download: Use ‘swh-download-directory-by-nar-hash’ Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 08/12] svn-download: " Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 09/12] bzr-download: Implement nar fallback Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 10/12] download-nar: Distinguish ‘output’ and ‘item’ parameter Ludovic Courtès
2024-03-05 11:06     ` [bug#69328] [PATCH v2 11/12] perform-download: Allow use of ‘download-nar’ for ‘--check’ builds Ludovic Courtès
2024-03-05 11:07     ` [bug#69328] [PATCH v2 12/12] download: Honor ‘GUIX_DOWNLOAD_METHODS’ environment variable Ludovic Courtès

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).