[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[zimoun]

Dear,

The first message in [1] explains the concrete issue.  To avoid to pollute the
already long thread discussing long term Tarball Archiving (which will not be
ready before the down), here is sent patches fixing the concrete issue.

Aside, note that the tarballs should be now ingested by Software Heritage via:

https://archive.softwareheritage.org/browse/search/?q=https%3A%2F%2Fguix.gnu.org%2Fsources.json&with_visit=true&with_content=true

but the issue is to reach them; well see [2].


From [1], the non-archived packages (yet) are:

--8<---------------cut here---------------start------------->8---
  scheme@(guile-user)> ,pp (lset-difference eq? $7 $8)
  $11 = (
   #<package r-spams@2.6-2017-03-22 gnu/packages/statistics.scm:3931 7f632401a640>
   #<package mpfi@1.5.4 gnu/packages/multiprecision.scm:158 7f632ee3adc0>
   #<package gf2x@1.2 gnu/packages/algebra.scm:103 7f6323ea1280>
   #<package gmp-ecm@7.0.4 gnu/packages/algebra.scm:658 7f6323eb4960>
   #<package cmh@1.0 gnu/packages/algebra.scm:322 7f6323eb4dc0>)
--8<---------------cut here---------------end--------------->8---

However, some have migrated to gitlab:

 - r-spams
     <https://gitlab.inria.fr/thoth/spams-devel>
 - gf2x
     <https://gitlab.inria.fr/gf2x/gf2x>
 - cmh
    <https://prod-gitlab.inria.fr/cmh/cmh>

Note that repackage 'r-spams' from the Git checkout needs some love and is not
straightforward.  (Not tried yet the 2 others).


The aim of switching the 2 packages from 'url-fetch' to 'svn-fetch' is
twofolds, test case for improving:

 - "guix lint -c archival" (support 'svn' and 'hg')
 - the fallback to SWH (for non-git VCS source)

WDYT?


[1]: <http://issues.guix.gnu.org/issue/42162>
[2] <https://forge.softwareheritage.org/T2430>

All the best,
simon

PS:
I am not sure how to deal with <control@debbugs.gnu.org> to "clone" (split)
the bug #42162.  That's why this one. :-)


zimoun (2):
  gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.
  gnu: gmp-ecm: Replace 'url-fetch' by 'svn-fetch'.

 gnu/packages/algebra.scm        | 28 +++++++++++++++++++++-------
 gnu/packages/multiprecision.scm | 13 ++++++++-----
 2 files changed, 29 insertions(+), 12 deletions(-)


base-commit: f5163902d7c3cfed5a97033f38e92d7158b19fa8
-- 
2.28.0

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

Fixes <https://bugs.gnu.org/42162>.

* gnu/packages/multiprecision.scm (mpfi)[source]: Replace 'url-fetch' by
'svn-fetch'.
---
 gnu/packages/multiprecision.scm | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/gnu/packages/multiprecision.scm b/gnu/packages/multiprecision.scm
index b3a5ec5894..f1b152d9aa 100644
--- a/gnu/packages/multiprecision.scm
+++ b/gnu/packages/multiprecision.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
 ;;; Copyright © 2018, 2019 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -32,6 +33,7 @@
   #:use-module (gnu packages texinfo)
   #:use-module (guix packages)
   #:use-module (guix download)
+  #:use-module (guix svn-download)
   #:use-module (guix utils)
   #:use-module (guix build-system gnu))
 
@@ -157,14 +159,15 @@ precision and correctly rounds the results.")
 (define-public mpfi
   (package
     (name "mpfi")
-    (version "1.5.4")
+    (version "1.5.4-688")
     (source
      (origin
-       (method url-fetch)
-       (uri (string-append "https://gforge.inria.fr/frs/download.php"
-                           "/latestfile/181/mpfi-" version ".tgz"))
+       (method svn-fetch)
+       (uri (svn-reference
+             (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
+             (revision 688)))
        (sha256
-        (base32 "0mismr1ll3wp788dq2n22s5irm0dziy75byyfdwz22kjbmckhf9v"))))
+        (base32 "13pqkdvi8maj0mm76zh8p7jcc4c4569zh4kq336shh9cxlrh8qqx"))))
     (build-system gnu-build-system)
     (arguments
      `(#:tests? #f                      ;tests are broken in this release
-- 
2.28.0

[bug#43442] [PATCH 2/2] gnu: gmp-ecm: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

Fixes <https://bugs.gnu.org/42162>.

* gnu/packages/algebra.scm (gmp-ecm)[source]: Replace 'url-fetch' by
'svn-fetch'.
[natice-inputs]: Add requirements.
[arguments]: Adjust 'check phase to substitute correct path.
---
 gnu/packages/algebra.scm | 28 +++++++++++++++++++++-------
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 318d653618..1cf293f17a 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -13,6 +13,7 @@
 ;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;; Copyright © 2020 Vinicius Monego <monego@posteo.net>
+;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -66,6 +67,7 @@
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix hg-download)
+  #:use-module (guix svn-download)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
   #:use-module (guix utils))
@@ -670,15 +672,20 @@ geometry and singularity theory.")
 (define-public gmp-ecm
   (package
    (name "gmp-ecm")
-   (version "7.0.4")
+   (version "7.0.4-3088")
    (source (origin
-             (method url-fetch)
-             ;; Use the ‘Latest version’ link for a stable URI across releases.
-             (uri (string-append "https://gforge.inria.fr/frs/download.php/"
-                                 "latestfile/160/ecm-" version ".tar.gz"))
+             (method svn-fetch)
+             (uri
+              (svn-reference
+               (url "https://scm.gforge.inria.fr/anonscm/svn/ecm/trunk")
+               (revision 3088)))
              (sha256 (base32
-                      "0hxs24c2m3mh0nq1zz63z3sb7dhy1rilg2s1igwwcb26x3pb7xqc"))))
+                      "0g1jqgp9baqz4x9mzksch69i4kp3207d3wn56g9vnsmanrrb4qvb"))))
    (build-system gnu-build-system)
+   (native-inputs
+    `(("autoconf" ,autoconf)
+      ("automake" ,automake)
+      ("libtool" ,libtool)))
    (inputs
     `(("gmp" ,gmp)))
    (arguments
@@ -686,7 +693,14 @@ geometry and singularity theory.")
                           ;; Disable specific assembly routines, which depend
                           ;; on the subarchitecture of the build machine,
                           ;; and use gmp instead.
-                          "--disable-asm-redc")))
+                          "--disable-asm-redc")
+      #:phases (modify-phases %standard-phases
+                 (add-before 'check 'replace-/bin/rm
+                   (lambda _
+                     (substitute*
+                         (list "test.ecm")
+                       (("/bin/rm") (which "rm")))
+                     #t)))))
    (synopsis "Integer factorization library using the elliptic curve method")
    (description
     "GMP-ECM factors integers using the elliptic curve method (ECM) as well
-- 
2.28.0

[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[Ludovic Courtès]

Hello!

zimoun <zimon.toutoune@gmail.com> skribis:

> The aim of switching the 2 packages from 'url-fetch' to 'svn-fetch' is
> twofolds, test case for improving:
>
>  - "guix lint -c archival" (support 'svn' and 'hg')
>  - the fallback to SWH (for non-git VCS source)

Woow, it’s ambitious given that SWH/SVN integration is currently lacking
on our side.  :-)

It’s a good idea to switch to gitlab.inria.fr for those that have
migrated, though.

Thanks,
Ludo’.

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[Ludovic Courtès]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> Fixes <https://bugs.gnu.org/42162>.
>
> * gnu/packages/multiprecision.scm (mpfi)[source]: Replace 'url-fetch' by
> 'svn-fetch'.

[...]

> +       (method svn-fetch)
> +       (uri (svn-reference
> +             (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> +             (revision 688)))

Does this have any chance of working with SWH?  Does their HTTP API
support looking up svn revision for a given origin URL?

Also:

  scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
  $6 = #f

So the effect today would be the opposite of what we’re attempting: when
GForge goes down, we’d lose it completely.

Thoughts?

Ludo’.

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

Hi,

On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:

>> +       (method svn-fetch)
>> +       (uri (svn-reference
>> +             (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>> +             (revision 688)))
>
> Does this have any chance of working with SWH?  Does their HTTP API
> support looking up svn revision for a given origin URL?

Well, I have not yet did my homework on the topic, neither asked on
#swh-devel.

Probably not!

In any case, we should find a way because there are a lot of Subversion
sources. 


> Also:
>
>   scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>   $6 = #f

Since “guix lint -c archival” I should have forgotten to submit the
request via their Web interface.  Now, it is scheduled:

  <https://archive.softwareheritage.org/save/#requests>


> So the effect today would be the opposite of what we’re attempting: when
> GForge goes down, we’d lose it completely.

I agree.


Well, the shutdown of the INRIA’s GForge is a concrete deadline to add
the support of Subversion (and Mercurial) sources to:

 - fallback to SWH
 - guix lint -c archival

I am working on it. :-)


All the best,
simon

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:
>
>>> +       (method svn-fetch)
>>> +       (uri (svn-reference
>>> +             (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>>> +             (revision 688)))
>>
>> Does this have any chance of working with SWH?  Does their HTTP API
>> support looking up svn revision for a given origin URL?
>
> Well, I have not yet did my homework on the topic, neither asked on
> #swh-devel.
>
> Probably not!
>
> In any case, we should find a way because there are a lot of Subversion
> sources. 

Yes, we should implement it.  But until that is done, I’m not sure it’s
worth replacing tarballs with svn references.  WDYT?

>> Also:
>>
>>   scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>>   $6 = #f
>
> Since “guix lint -c archival” I should have forgotten to submit the
> request via their Web interface.  Now, it is scheduled:
>
>   <https://archive.softwareheritage.org/save/#requests>

‘lookup-origin’ still returns #f though.

>> So the effect today would be the opposite of what we’re attempting: when
>> GForge goes down, we’d lose it completely.
>
> I agree.
>
>
> Well, the shutdown of the INRIA’s GForge is a concrete deadline to add
> the support of Subversion (and Mercurial) sources to:
>
>  - fallback to SWH
>  - guix lint -c archival
>
> I am working on it. :-)

Awesome.

(Note that it seems possible that the shutdown of gforge.inria.fr will
be delayed a bit or at least be made less brutally that initially
announced.)

Thanks,
Ludo’.

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

On Wed, 23 Sep 2020 at 18:22, Ludovic Courtès <ludo@gnu.org> wrote:
> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:
> >
> >>> +       (method svn-fetch)
> >>> +       (uri (svn-reference
> >>> +             (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> >>> +             (revision 688)))
> >>
> >> Does this have any chance of working with SWH?  Does their HTTP API
> >> support looking up svn revision for a given origin URL?
> >
> > Well, I have not yet did my homework on the topic, neither asked on
> > #swh-devel.
> >
> > Probably not!
> >
> > In any case, we should find a way because there are a lot of Subversion
> > sources.
>
> Yes, we should implement it.  But until that is done, I’m not sure it’s
> worth replacing tarballs with svn references.  WDYT?

I do not have a strong opinion.

> >> Also:
> >>
> >>   scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> >>   $6 = #f
> >
> > Since “guix lint -c archival” I should have forgotten to submit the
> > request via their Web interface.  Now, it is scheduled:
> >
> >   <https://archive.softwareheritage.org/save/#requests>
>
> ‘lookup-origin’ still returns #f though.

Weird because SWH says it is in. Requests at 21/09/2020, 23:47:09 then
click. :-)

 - scheduled: 21/09/2020, 23:47:21
 - start: 21/09/2020, 23:47:24
 - end: 21/09/2020, 23:53:26
 - log: [2020-09-21 21:53:26,098: INFO/ForkPoolWorker-1] Task
swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[b13942f6-b415-4831-b25f-e68bd4515cf2]
succeeded in 361.4525023885071s: {'status': 'eventful'}

Well, the issue is:

- Not in: https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi
<https://archive.softwareheritage.org/browse/search/?q=https%3A%2F%2Fscm.gforge.inria.fr%2Fanonscm%2Fsvn%2Fmpfi%2Ftrunk%2Fmpfi&with_visit=true&with_content=true>

+ In: https://scm.gforge.inria.fr/anonscm/svn/mpfi
<https://archive.softwareheritage.org/browse/search/?q=https%3A%2F%2Fscm.gforge.inria.fr%2Fanonscm%2Fsvn%2Fmpfi&with_visit=true&with_content=true>

as you can see here:

<https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://scm.gforge.inria.fr/anonscm/svn/mpfi>


> (Note that it seems possible that the shutdown of gforge.inria.fr will
> be delayed a bit or at least be made less brutally that initially
> announced.)

Good to know. :-)

All the best,
simon

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[Ludovic Courtès]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> as you can see here:
>
> <https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://scm.gforge.inria.fr/anonscm/svn/mpfi>

Oh indeed:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix swh)
scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi")
$14 = #<<origin> visits-url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visits/" type: #f url: "https://scm.gforge.inria.fr/anonscm/svn/mpfi">
scheme@(guile-user)> (origin-visits $14)
$15 = (#<<visit> date: #<date nanosecond: 902765 second: 25 minute: 53 hour: 21 day: 21 month: 9 year: 2020 zone-offset: 0> origin: "https://scm.gforge.inria.fr/anonscm/svn/mpfi" url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/1/" snapshot-url: "https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/" status: full number: 1>)
scheme@(guile-user)> (visit-snapshot (car $15))
$16 = #<<snapshot> branches: (#<<branch> name: "HEAD" target-type: revision target-url: "https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/">)>
--8<---------------cut here---------------end--------------->8---

Note that the original svn revision number is discarded, AFAICS.

Ludo’.

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

Hi Ludo,

On Fri, 25 Sep 2020 at 10:56, Ludovic Courtès <ludo@gnu.org> wrote:

> Note that the original svn revision number is discarded, AFAICS.

No, it is not... but to find it one needs a bit of guidance by SWH folks. :-)

1. With the URL, fetch:
https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/latest/

2. From #1 one gets 'snapshot' and fetch:
https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/

3. From #3, one gets the 'target' and fetch:
https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/log/

And #3 lists all the revisions in SWH.

Well, this should be enough for the saving request.  WDYT?


However, it is not clear how to fetch cooking from the vault.  To be
continued...

Cheers,
simon

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[zimoun]

On Thu, 1 Oct 2020 at 22:26, zimoun <zimon.toutoune@gmail.com> wrote:

> 1. With the URL, fetch:
> https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/latest/
>
> 2. From #1 one gets 'snapshot' and fetch:
> https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/
>
> 3. From #3, one gets the 'target' and fetch:
> https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/log/
>
> And #3 lists all the revisions in SWH.
>
> Well, this should be enough for the saving request.  WDYT?
>

> However, it is not clear how to fetch cooking from the vault.  To be
> continued...

From #3, using 'directory' (4193e93b4aea0abeaff715b1a1274c4d7eb4a27d),
it seems possible to cook from the vault and get the tarball (I did
with the web interface).  And the integrity field (guix hash -r) is
the same as the one I put in the patch submission.

[bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.

[Ludovic Courtès]

Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

> On Fri, 25 Sep 2020 at 10:56, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Note that the original svn revision number is discarded, AFAICS.
>
> No, it is not... but to find it one needs a bit of guidance by SWH folks. :-)
>
> 1. With the URL, fetch:
> https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/latest/
>
> 2. From #1 one gets 'snapshot' and fetch:
> https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/
>
> 3. From #3, one gets the 'target' and fetch:
> https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/log/
>
> And #3 lists all the revisions in SWH.

Ah yes, under “extra_headers” there’s the SVN revision number.  (guix
swh) doesn’t expose “extra_headers” yet, but once it does, we can walk
snapshots until we find the SVN revision we’re looking for.

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/")
$3 = #<<origin> visits-url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visits/" type: #f url: "https://scm.gforge.inria.fr/anonscm/svn/mpfi">
scheme@(guile-user)> (origin-visits $3)
$4 = (#<<visit> date: #<date nanosecond: 902765 second: 25 minute: 53 hour: 21 day: 21 month: 9 year: 2020 zone-offset: 0> origin: "https://scm.gforge.inria.fr/anonscm/svn/mpfi" url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/1/" snapshot-url: "https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/" status: full number: 1>)
scheme@(guile-user)> (visit-snapshot (car $4))
$5 = #<<snapshot> branches: (#<<branch> name: "HEAD" target-type: revision target-url: "https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/">)>
--8<---------------cut here---------------end--------------->8---

So the next step is to augment (guix swh) with a
‘lookup-subversion-revision’ procedure.

Thanks for investigating!

Ludo’.

[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 3744 bytes --]

Hi!

Ludovic Courtès <ludo@gnu.org> skribis:

> Ah yes, under “extra_headers” there’s the SVN revision number.  (guix
> swh) doesn’t expose “extra_headers” yet, but once it does, we can walk
> snapshots until we find the SVN revision we’re looking for.
>
> scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/")
> $3 = #<<origin> visits-url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visits/" type: #f url: "https://scm.gforge.inria.fr/anonscm/svn/mpfi">
> scheme@(guile-user)> (origin-visits $3)
> $4 = (#<<visit> date: #<date nanosecond: 902765 second: 25 minute: 53 hour: 21 day: 21 month: 9 year: 2020 zone-offset: 0> origin: "https://scm.gforge.inria.fr/anonscm/svn/mpfi" url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/1/" snapshot-url: "https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/" status: full number: 1>)
> scheme@(guile-user)> (visit-snapshot (car $4))
> $5 = #<<snapshot> branches: (#<<branch> name: "HEAD" target-type: revision target-url: "https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/">)>
>
> So the next step is to augment (guix swh) with a
> ‘lookup-subversion-revision’ procedure.

The attached patch does that:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (lookup-subversion-revision "https://scm.gforge.inria.fr/anonscm/svn/mpfi" 680)
$12 = #<<revision> id: "72102de7605a2459ebcb016338ebbf1a997e8c8d" date: #<date nanosecond: 938388 second: 35 minute: 32 hour: 11 day: 6 month: 9 year: 2018 zone-offset: 0> directory: "5c89c025a4cd9d16befdfec12dfc23f7318d0d5b" directory-url: "https://archive.softwareheritage.org/api/1/directory/5c89c025a4cd9d16befdfec12dfc23f7318d0d5b/" parents-ids: ("16da41f1848d77a93aec565320b72b460c429b61") extra-headers: (("svn_repo_uuid" . "e2f78e0c-bb60-4709-9413-9660a31d4696") ("svn_revision" . "680"))>
scheme@(guile-user)> (lookup-subversion-revision "https://scm.gforge.inria.fr/anonscm/svn/mpfi" 666)
$13 = #<<revision> id: "148eb1e7206b111af4075c73c656e54c9efed6cb" date: #<date nanosecond: 654167 second: 2 minute: 52 hour: 12 day: 2 month: 8 year: 2018 zone-offset: 0> directory: "ed7b0bd7019fb85cd86d948a97c23b9d43aa8728" directory-url: "https://archive.softwareheritage.org/api/1/directory/ed7b0bd7019fb85cd86d948a97c23b9d43aa8728/" parents-ids: ("0ba2aa7e0d3fc0a1eb3ba72b32094515415ae47a") extra-headers: (("svn_repo_uuid" . "e2f78e0c-bb60-4709-9413-9660a31d4696") ("svn_revision" . "666"))>
--8<---------------cut here---------------end--------------->8---

The implementation is pretty bad though, because it walks the revision
history until it finds the right revision number—so you’re likely to
reach the bandwidth rate limit before you’ve found the revision you’re
looking for.

More importantly, most svn origins cannot be found, or at least not
by passing the URL as-is:

  https://sympa.inria.fr/sympa/arc/swh-devel/2023-03/msg00009.html

This whole hack looks like a dead end.

It would be ideal if SWH would compute nar hashes, as you proposed:

  https://gitlab.softwareheritage.org/swh/meta/-/issues/4538

As a stopgap, I wonder if we could use “double hashing” on our side, but
only for svn: we’d store both the nar sha256 as we currently do, plus
the swhid.  It still seems to me that it’d be hard to scale and to
maintain that over time, even if it’s limited to svn.  Plus, there’d
still be the problem of ‘svn-multi-fetch’, which is what most TeX Live
packages use.

Thoughts?

Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 4366 bytes --]

diff --git a/guix/swh.scm b/guix/swh.scm
index c7c1c873a2..a65635b1db 100644
--- a/guix/swh.scm
+++ b/guix/swh.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2018, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2018, 2019, 2020, 2021, 2023 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
 ;;; Copyright © 2021 Xinglu Chen <public@yoctocell.xyz>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
@@ -75,8 +75,10 @@ (define-module (guix swh)
             revision-id
             revision-date
             revision-directory
+            revision-parents
             lookup-revision
             lookup-origin-revision
+            lookup-subversion-revision
 
             content?
             content-checksums
@@ -207,6 +209,14 @@ (define string*
     ((? null?) #f)                                ;Guile-JSON 3.x
     ('null #f)))                                  ;Guile-JSON 4.x
 
+(define pair-vector->alist
+  (match-lambda
+    ('null '())
+    ((= vector->list lst)
+     (map (match-lambda
+            (#(key value) (cons key value)))
+          lst))))
+
 (define %allow-request?
   ;; Takes a URL and method (e.g., the 'http-get' procedure) and returns true
   ;; to keep going.  This can be used to disallow requests when
@@ -346,7 +356,14 @@ (define-json-mapping <revision> make-revision revision?
   (id            revision-id)
   (date          revision-date "date" (maybe-null string->date*))
   (directory     revision-directory)
-  (directory-url revision-directory-url "directory_url"))
+  (directory-url revision-directory-url "directory_url")
+  (parents-ids   revision-parent-ids "parents"
+                 (lambda (vector)
+                   (map (lambda (alist)
+                          (assoc-ref alist "id"))
+                        (vector->list vector))))
+  (extra-headers revision-extra-headers      ;alist--e.g., with "svn_revision"
+                 "extra_headers" pair-vector->alist))
 
 ;; <https://archive.softwareheritage.org/api/1/content/>
 (define-json-mapping <content> make-content content?
@@ -524,6 +541,50 @@ (define (lookup-origin-revision url tag)
         (()
          #f)))))
 
+(define (revision-parents revision)
+  "Return the parent revision(s) of REVISION."
+  (filter-map lookup-revision (revision-parent-ids revision)))
+
+(define (lookup-subversion-revision-in-history revision revision-number)
+  "Look for Subversion REVISION-NUMBER starting from REVISION and going back
+in history."
+  (let loop ((revision revision))
+    (let ((number (and=> (assoc-ref (revision-extra-headers revision)
+                                    "svn_revision")
+                         string->number)))
+      (and number
+           (cond ((= number revision-number)
+                  ;; Found it!
+                  revision)
+                 ((< number revision-number)
+                  ;; REVISION is ancestor of REVISION-NUMBER, so stop here.
+                  #f)
+                 (else
+                  ;; Check the parent(s) of REVISION.
+                  (any loop (revision-parents revision))))))))
+
+(define (lookup-subversion-revision url revision-number)
+  "Return either #f or the revision of the Subversion repository once
+available at URL with the given REVISION-NUMBER."
+  (match (lookup-origin url)
+    (#f #f)
+    (origin
+      (match (filter (lambda (visit)
+                       ;; Return #f if (visit-snapshot VISIT) would return #f.
+                       (and (visit-snapshot-url visit)
+                            (eq? 'full (visit-status visit))))
+                     (origin-visits origin))
+        (()
+         #f)
+        ((visit . _)
+         (any (lambda (branch)
+                (match (branch-target branch)
+                  ((? revision? revision)
+                   (lookup-subversion-revision-in-history revision
+                                                          revision-number))
+                  (_ #f)))
+              (snapshot-branches (visit-snapshot visit))))))))
+
 (define (release-target release)
   "Return the revision that is the target of RELEASE."
   (match (release-target-type release)

[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[Timothy Sample]

Hello,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> This whole hack looks like a dead end.
>
> It would be ideal if SWH would compute nar hashes, as you proposed:
>
>   https://gitlab.softwareheritage.org/swh/meta/-/issues/4538
>
> As a stopgap, I wonder if we could use “double hashing” on our side, but
> only for svn: we’d store both the nar sha256 as we currently do, plus
> the swhid.  It still seems to me that it’d be hard to scale and to
> maintain that over time, even if it’s limited to svn.  Plus, there’d
> still be the problem of ‘svn-multi-fetch’, which is what most TeX Live
> packages use.
>
> Thoughts?

Not too many, but I do have more bad news.  Apologies if this is already
known, but I’m just getting up to speed with how SWH handles Subversion
(for coverage checking) and thought this seemed pretty significant.

I was starting with doing a simple check for the “easy” Subversion
repositories.  That is, no externals (‘recursive?’) and no
‘svn-multi-fetch’ [1].  I immediately hit a problem.  Guix hashes the
export of the repository with the keywords processed, while SWH hashes
it with unprocessed keywords.

For example, take ‘libsmpeg’.  It has a file called “mkinstalldirs”,
which has a keyword in it: “$Id$”.  The SWH loader hashes this as

    $Id$

while we hash it as

    $Id: mkinstalldirs 9 1999-10-21 15:55:01Z hercules $

This is not a big issue in terms of coverage checking, but it will be an
issue for automatic recovery.  Even if you know the exact SWH directory
ID, you won’t get a directory that satisfies the daemon’s hash check.  I
have no idea how hard it is to process the keywords with only data from
SWH.  In this case, you would have to walk revisions to find the last
time “mkinstalldirs” was modified, and then format its metadata.
However, I assume the Subversion properties are gone, so there might be
edge cases like a file with “$Id$” (or whatever) that Subversion
wouldn’t processes.

Again, apologies if this is old news.  Actually apologies either way,
’cause this is a bit of a downer!


-- Tim

[1] More precisely, I was going to process recursive ‘svn-fetch’ origins
because a lot of them are needlessly marked as recursive.  In some
(many?) cases, the repositories don’t actually have external references,
so the flag does nothing.  I was only going to skip the ones where it
makes a difference.

[bug#43442] Subversion keyword substitution

[Ludovic Courtès]

[-- Attachment #1: Type: text/plain, Size: 3063 bytes --]

Hi Timothy,

Timothy Sample <samplet@ngyro.com> skribis:

> I was starting with doing a simple check for the “easy” Subversion
> repositories.  That is, no externals (‘recursive?’) and no
> ‘svn-multi-fetch’ [1].  I immediately hit a problem.  Guix hashes the
> export of the repository with the keywords processed, while SWH hashes
> it with unprocessed keywords.

Ouch.  I had forgotten keywords were also a thing in Subversion.  :-/

Can we tell Subversion to not expand them?  That could be the way
forward in Guix, though it won’t help for past revisions.

How frequent is the use of keywords though?  I tried the patch below to
turn off keyword substitution, and then ran

  guix build -S --check -v1 --keep-going

for the 407 packages identified by:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (define svn (fold-packages (lambda (p lst)
						  (if (and (not (hidden-package? p))
							   (not (package-superseded p))
							   (origin? (package-source p))
							   (memq (origin-method (package-source p))
								 (list svn-fetch svn-multi-fetch)))
						      (cons p lst)
						      lst))
						'()))
scheme@(guile-user)> (length svn)
$8 = 407
--8<---------------cut here---------------end--------------->8---

That led to:

--8<---------------cut here---------------start------------->8---
guix build: error: build of `/gnu/store/2byn59zmdbc4bz2wknnv0df4n67bdvgr-texlive-pdftex-59745-checkout.drv', `/gnu/store/2gj88z4plmwhraghxj5626zpiir1ck6k-libsmpeg-0.4.5-401-checkout.drv', `/gnu/store/2zygylsb2b333rzrvjyrh4qybw799hl3-ghmm-0.9-rc3-0.2341-checkout.drv', `/gnu/store/4a81qlka5w73rprapzi1w63xzb01n0r8-java-geronimo-xbean-reflect-4.5.drv', `/gnu/store/4mabgwil0ygwm3bkka3nzfbrwg1kk0wz-texlive-kpathsea-59745-checkout.drv', `/gnu/store/5ivk83abj22bs9ka10dk1v67kyczcd80-texlive-dvips-59745-checkout.drv', `/gnu/store/6zhnahylfr1zmpwzb8qzh8qp3yf9yl1p-texlive-tex-plain-59745-checkout.drv', `/gnu/store/f1sjmghs0f4v0y2pnljqaplifq52qbn2-texlive-cm-59745-checkout.drv', `/gnu/store/kd7kahaq71gi8j6zbabr0njqw60sjjxc-libsmpeg-0.4.5-399-checkout.drv', `/gnu/store/q3kip5bxkkdh14kx81afakbmpy7l4lr4-texlive-latexconfig-59745-checkout.drv', `/gnu/store/qskxc2c30fdclmrjp7nk35n1q2sl6sp8-texlive-tetex-59745-checkout.drv', `/gnu/store/wibsxy4kxlpq8lr76wvwgdyc2x5farr4-texlive-hyphen-base-59745-checkout.drv' failed
--8<---------------cut here---------------end--------------->8---

That’s 11 failures out of 407.

So, how about applying the ‘--ignore-keywords’ change and updating
hashes accordingly?

> [1] More precisely, I was going to process recursive ‘svn-fetch’ origins
> because a lot of them are needlessly marked as recursive.  In some
> (many?) cases, the repositories don’t actually have external references,
> so the flag does nothing.  I was only going to skip the ones where it
> makes a difference.

We should remove that recursive flag when it has no effect.  Perhaps we
could proceed similarly?

Thanks,
Ludo’.


[-- Attachment #2: Type: text/x-patch, Size: 1132 bytes --]

diff --git a/guix/build/svn.scm b/guix/build/svn.scm
index 2d960cb364..863c48e46d 100644
--- a/guix/build/svn.scm
+++ b/guix/build/svn.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2014, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2014, 2020, 2023 Ludovic Courtès <ludo@gnu.org>
 ;;; Copyright © 2014 Sree Harsha Totakura <sreeharsha@totakura.in>
 ;;; Copyright © 2018 Mark H Weaver <mhw@netris.org>
 ;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
@@ -47,6 +47,11 @@ (define* (svn-fetch url revision directory
            ;; verify the checksum later.  This can be removed when
            ;; ca-certificates package is added.
            "--trust-server-cert" "-r" (number->string revision)
+
+           ;; Disable keyword substitution (keywords are CVS-like strings
+           ;; like "$Date$", "$Id$", and so on).
+           "--ignore-keywords"
+
            `(,@(if (and user-name password)
                    (list (string-append "--username=" user-name)
                          (string-append "--password=" password))

[bug#43442] Subversion keyword substitution

[Timothy Sample]

Hey,

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> Hi Timothy,
>
> Timothy Sample <samplet@ngyro.com> skribis:
>
>> I was starting with doing a simple check for the “easy” Subversion
>> repositories.  That is, no externals (‘recursive?’) and no
>> ‘svn-multi-fetch’ [1].  I immediately hit a problem.  Guix hashes the
>> export of the repository with the keywords processed, while SWH hashes
>> it with unprocessed keywords.
>
> Ouch.  I had forgotten keywords were also a thing in Subversion.  :-/
>
> Can we tell Subversion to not expand them?  That could be the way
> forward in Guix, though it won’t help for past revisions.

Thinking entirely abstractly, the keywords should be expanded.  I’m not
really long enough in the tooth (old enough) to know how people use
keywords, but one might be tempted to do something like:

    printf ("This is foo version %s\n", "$Revision$");

If that ever happens, processing the keywords would be very important.

Thinking practically, we’ve never encountered anything like that (see
below), and compatibility with SWH is nice.

It’s not clear to me why SWH passes ‘--ignore-keywords’ to Subversion in
the first place.  I guess it saves storage, because having identical
files allows deduplication.

> How frequent is the use of keywords though?

Well, you found 11 in the current Guix, and I see 30 when I process
everything I have (from version 1.0 to a few weeks ago).  Furthermore,
the only usage pattern I see is “$Id” in a comment.

> So, how about applying the ‘--ignore-keywords’ change and updating
> hashes accordingly?

It’s probably the right default given the circumstances.

It seems like there’s a direct conflict between ease of packaging and
ease of time travel.  In the hypothetical case that a keyword mattered,
it would be a nasty surprise to the package author.  They would have to
(a) discover the problem and (b) manually do the keyword substitution in
Scheme (or work around it).

Similarly, for recursive checkouts (including Git), it would be easier
to do time travel if we explicitly listed each source and how to compose
them.  However, that’s a pain for package authors and maintainers.

Add to those two examples all the discussions about multihashing or
keeping track of the SWHID and something of a theme emerges....

Having said that, it’s nice that the work we are doing is letting us
think very clearly about this trade-off.

>> [1] More precisely, I was going to process recursive ‘svn-fetch’
>> origins because a lot of them are needlessly marked as recursive.  In
>> some (many?) cases, the repositories don’t actually have external
>> references, so the flag does nothing.  I was only going to skip the
>> ones where it makes a difference.
>
> We should remove that recursive flag when it has no effect.  Perhaps we
> could proceed similarly?

Huh.  My scripts tell me that we haven’t needed it at all in the last
three years.  That’s a suspicious enough result that I wonder if there’s
a bug in my scripts.  The results are looking good so far, but there are
a few things I still need to look over.

I ran your same experiment, passing all the packages through:

    (define (make-svn-package-non-recursive pkg)
      (package
       (inherit pkg)
       (source
        (origin
         (inherit (package-source pkg))
         (uri (match (origin-uri (package-source pkg))
                ((? svn-reference? ref)
                 (svn-reference
                  (inherit ref)
                  (recursive? #f)))
                ((? svn-multi-reference? ref)
                 (svn-multi-reference
                  (inherit ref)
                  (recursive? #f)))))))))

All of them were fine.  Maybe we just haven’t had a truly recursive
Subversion source in recent memory.


-- Tim

[bug#43442] Subversion keyword substitution

[Ludovic Courtès]

Hi,

Timothy Sample <samplet@ngyro.com> skribis:

> Thinking entirely abstractly, the keywords should be expanded.  I’m not
> really long enough in the tooth (old enough) to know how people use
> keywords, but one might be tempted to do something like:
>
>     printf ("This is foo version %s\n", "$Revision$");
>
> If that ever happens, processing the keywords would be very important.

“Very” might be an overstatement.  :-)

In practice, these were typically used in source file headers, so that
if you exported or copied files around (outside version control), they’d
have a timestamp of sorts at the top.

[...]

> It’s not clear to me why SWH passes ‘--ignore-keywords’ to Subversion in
> the first place.  I guess it saves storage, because having identical
> files allows deduplication.

I asked on #swh-devel and the fine folks there hinted at
non-reproducibility.  Looking at
<https://svnbook.red-bean.com/en/1.7/svn.advanced.props.special.keywords.html>,
one thing that’s definitely not reproducible is the “local time zone”
bit.  From that perspective it makes a lot of sense to disable keyword
substitution.

>> How frequent is the use of keywords though?
>
> Well, you found 11 in the current Guix, and I see 30 when I process
> everything I have (from version 1.0 to a few weeks ago).  Furthermore,
> the only usage pattern I see is “$Id” in a comment.

Interesting.

>> So, how about applying the ‘--ignore-keywords’ change and updating
>> hashes accordingly?
>
> It’s probably the right default given the circumstances.

OK.  I’ll submit a patch to that effect, unless you beat me at it.  :-)

> It seems like there’s a direct conflict between ease of packaging and
> ease of time travel.  In the hypothetical case that a keyword mattered,
> it would be a nasty surprise to the package author.  They would have to
> (a) discover the problem and (b) manually do the keyword substitution in
> Scheme (or work around it).

My intuition is that the worst “problem” we might have is ‘--version’
showing unexpanded keywords.

[...]

>> We should remove that recursive flag when it has no effect.  Perhaps we
>> could proceed similarly?
>
> Huh.  My scripts tell me that we haven’t needed it at all in the last
> three years.  That’s a suspicious enough result that I wonder if there’s
> a bug in my scripts.  The results are looking good so far, but there are
> a few things I still need to look over.

Looks like it might be easily addressed!

Thanks,
Ludo’.

[bug#43442] Subversion keyword substitution

[Simon Tournier]

Hi,

On lun., 27 mars 2023 at 11:04, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

>> It seems like there’s a direct conflict between ease of packaging and
>> ease of time travel.  In the hypothetical case that a keyword mattered,
>> it would be a nasty surprise to the package author.  They would have to
>> (a) discover the problem and (b) manually do the keyword substitution in
>> Scheme (or work around it).

That’s an application of the non-free lunch theorem, no? :-)


> My intuition is that the worst “problem” we might have is ‘--version’
> showing unexpanded keywords.

Somehow, I would prefer to pay the surprise by package author using
Subversion rather than not being fully reproducible over the time.


Cheers,
simon

[bug#43442] [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

[Simon Tournier]

Hi,

On lun., 20 mars 2023 at 15:09, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> As a stopgap, I wonder if we could use “double hashing” on our side, but
> only for svn: we’d store both the nar sha256 as we currently do, plus
> the swhid.  It still seems to me that it’d be hard to scale and to
> maintain that over time, even if it’s limited to svn.  Plus, there’d
> still be the problem of ‘svn-multi-fetch’, which is what most TeX Live
> packages use.

As proposed, somehow, in [1], I think we have nothing to loose with
optionally “double hashing” (or even triple or more).  I do not
understand your concerns in [1] and I will discuss overthere. :-)

About ’svn-multi-fetch’ and TeX, what is at our advantage is that most
TeX Live packages use ’svn-multi-fetch’ via ’simple-texlive-package’
relying on ’texlive-origin’.  And from official TeX documentation [2],
Subversion is not the recommended access, instead they suggest to rely
on ’rsync’.  Moreover, some Git mirror is available.

Therefore, about TeX we could ask if relying on Subversion is still the
“good” way and maybe we could switch to something else as url-fetch or
git-fetch.  Hum?!  Not convinced by my proposal here. :-)

Well, IMHO, there is 2 issues: one about using SWH as fallback for
packages using svn-fetch and another one about TeX Live packages.  It
could be nice to solve them both at once but maybe we should start now one
foot, now the other.

1: https://yhetil.org/guix/87jzzxd7z8.fsf@gmail.com
2: https://tug.org/texlive/svn/


Cheers,
simon

[bug#43442] Subversion keyword substitution

[Timothy Sample]

Hi!

Ludovic Courtès <ludovic.courtes@inria.fr> writes:

> OK.  I’ll submit a patch to that effect, unless you beat me at it.  :-)

I’m on it!  It might take me another day or two to actually submit the
patch.  The keyword expansion change affects ‘texlive-bin’, which means
a lot of rebuilds, so I guess we will need to coordinate a feature
branch or whatever (my understanding is that core-updates is essentially
frozen and deprecated at this point).

> Timothy Sample <samplet@ngyro.com> skribis:
>
>> Thinking entirely abstractly, the keywords should be expanded.  I’m not
>> really long enough in the tooth (old enough) to know how people use
>> keywords, but one might be tempted to do something like:
>>
>>     printf ("This is foo version %s\n", "$Revision$");
>>
>> If that ever happens, processing the keywords would be very important.
>
> “Very” might be an overstatement.  :-)
>
> In practice, these were typically used in source file headers, so that
> if you exported or copied files around (outside version control), they’d
> have a timestamp of sorts at the top.

I ended up finding 17 origins that make use of keyword expansion.  Two
of them indeed do so outside of comments.  (1) The “texlive-scripts”
source has some Perl scripts that do the Perl equivalent of my example
above.  (2) There’s some Java code (“geronimo”) that uses keywords in
Javadoc comments, which might show up in generated documentation.

So no, definitely not “very” important!  :)

>> Huh.  My scripts tell me that we haven’t needed it at all in the last
>> three years.  That’s a suspicious enough result that I wonder if there’s
>> a bug in my scripts.  The results are looking good so far, but there are
>> a few things I still need to look over.
>
> Looks like it might be easily addressed!

I’ll switch the ‘recursive?’ field to ‘#f’ by default as part of the
series I send in.


-- Tim

[bug#43442] Subversion keyword substitution

[Ludovic Courtès]

Hi!

Timothy Sample <samplet@ngyro.com> skribis:

>> So, how about applying the ‘--ignore-keywords’ change and updating
>> hashes accordingly?
>
> It’s probably the right default given the circumstances.

Patch submitted: <https://issues.guix.gnu.org/62712>.

Ludo’.

bug#43442: Code stored with Subversion (SVN) cannot be retrieved from SWH

[Ludovic Courtès]

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

> The attached patch does that:
>
> scheme@(guile-user)> (lookup-subversion-revision "https://scm.gforge.inria.fr/anonscm/svn/mpfi" 680)
> $12 = #<<revision> id: "72102de7605a2459ebcb016338ebbf1a997e8c8d" date: #<date nanosecond: 938388 second: 35 minute: 32 hour: 11 day: 6 month: 9 year: 2018 zone-offset: 0> directory: "5c89c025a4cd9d16befdfec12dfc23f7318d0d5b" directory-url: "https://archive.softwareheritage.org/api/1/directory/5c89c025a4cd9d16befdfec12dfc23f7318d0d5b/" parents-ids: ("16da41f1848d77a93aec565320b72b460c429b61") extra-headers: (("svn_repo_uuid" . "e2f78e0c-bb60-4709-9413-9660a31d4696") ("svn_revision" . "680"))>
> scheme@(guile-user)> (lookup-subversion-revision "https://scm.gforge.inria.fr/anonscm/svn/mpfi" 666)
> $13 = #<<revision> id: "148eb1e7206b111af4075c73c656e54c9efed6cb" date: #<date nanosecond: 654167 second: 2 minute: 52 hour: 12 day: 2 month: 8 year: 2018 zone-offset: 0> directory: "ed7b0bd7019fb85cd86d948a97c23b9d43aa8728" directory-url: "https://archive.softwareheritage.org/api/1/directory/ed7b0bd7019fb85cd86d948a97c23b9d43aa8728/" parents-ids: ("0ba2aa7e0d3fc0a1eb3ba72b32094515415ae47a") extra-headers: (("svn_repo_uuid" . "e2f78e0c-bb60-4709-9413-9660a31d4696") ("svn_revision" . "666"))>
>
> The implementation is pretty bad though, because it walks the revision
> history until it finds the right revision number—so you’re likely to
> reach the bandwidth rate limit before you’ve found the revision you’re
> looking for.
>
> More importantly, most svn origins cannot be found, or at least not
> by passing the URL as-is:
>
>   https://sympa.inria.fr/sympa/arc/swh-devel/2023-03/msg00009.html
>
> This whole hack looks like a dead end.
>
> It would be ideal if SWH would compute nar hashes, as you proposed:
>
>   https://gitlab.softwareheritage.org/swh/meta/-/issues/4538

Happy to report that this is solved with nar-sha256 ExtIDs at SWH, which
commit 0e73f933b291c7e154c7e019b6de1e2f3a97e4c1 uses to implement the
SWH fallback for Subversion checkouts.  Wo0t!

Ludo’.