bug#69070: ‘guix lint -c archival python-scikit-learn’ crashes

bug#69070: ‘guix lint -c archival python-scikit-learn’ crashes

[Ludovic Courtès]

Hi!

Starting from a couple of weeks ago, probably around the time SWH
deployed an updated version of their infrastructure, we get crashes like
this:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 290  Feb 11 2024 23:01:35    (current)
  guix c4372f7
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: c4372f7ebfac17236299346fe60c6ca125336b1f
$ guix lint -c archival python-scikit-learn
Backtrace:ython-scikit-learn@1.3.2 [archival]...
          12 (primitive-load "/home/ludo/.config/guix/current/bin/gu…")
In guix/ui.scm:
   2324:7 11 (run-guix . _)
  2287:10 10 (run-guix-command _ . _)
In ice-9/boot-9.scm:
  1752:10  9 (with-exception-handler _ _ #:unwind? _ # _)
In srfi/srfi-1.scm:
    634:9  8 (for-each #<procedure 7f45354e9400 at guix/scripts/lin…> …)
In guix/scripts/lint.scm:
     65:4  7 (run-checkers _ _ #:store _)
In srfi/srfi-1.scm:
    634:9  6 (for-each #<procedure 7f4535405d50 at guix/scripts/lin…> …)
In guix/scripts/lint.scm:
    74:21  5 (_ _)
In guix/lint.scm:
   1662:4  4 (check-archival _)
    877:2  3 (call-with-networking-fail-safe _ _ _)
In ice-9/boot-9.scm:
  1752:10  2 (with-exception-handler _ _ #:unwind? _ # _)
  1685:16  1 (raise-exception _ #:continuable? _)
  1683:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1683:16: In procedure raise-exception:
Throw to key `match-error' with args `("match" "no matching pattern" directory)'.
--8<---------------cut here---------------end--------------->8---

This can be reproduced by merely evaluating:

  (lookup-origin-revision "https://github.com/scikit-learn/scikit-learn" "1.3.2")

The crux of the problem is that ‘snapshot-branches’ used to return
either a “revision” or a “release”, but now it can also return a
“directory”:

--8<---------------cut here---------------start------------->8---
$ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
{
  "id": "c405490b80fb35cc58b34260d9ec6add8248857f",
  "branches": {
    "1.3.2": {
      "target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
      "target_type": "directory",
      "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
    },
    "HEAD": {
      "target": "1.3.2",
      "target_type": "alias",
      "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
    }
  },
  "next_branch": null
}
--8<---------------cut here---------------end--------------->8---

I’ll check with comrades at SWH whether this is intended.

(Note that this problem may vanish once
<https://issues.guix.gnu.org/68741> has been merged.)

Ludo’.

bug#69070: No releases/revisions in /snapshot endpoints?

[Ludovic Courtès]

Dear SWH,

In <https://issues.guix.gnu.org/69070> I stumbled upon what might be new
behavior from the /snapshot endpoint:

> The crux of the problem is that ‘snapshot-branches’ used to return
> either a “revision” or a “release”, but now it can also return a
> “directory”:
>
> $ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
> {
>   "id": "c405490b80fb35cc58b34260d9ec6add8248857f",
>   "branches": {
>     "1.3.2": {
>       "target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
>       "target_type": "directory",
>       "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>     },
>     "HEAD": {
>       "target": "1.3.2",
>       "target_type": "alias",
>       "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>     }
>   },
>   "next_branch": null
> }

Current documentation mentions the ‘target_types’ query parameter, which
I could use to restrict results to “revision” and “release”:

  https://archive.softwareheritage.org/api/1/snapshot/doc/

However, that’d be the empty list of the case above.

Other snapshots seem to be missing releases and revisions as well, such
as this one:

  https://archive.softwareheritage.org/api/1/snapshot/55bda9214f962550baad25026fb3ac3ad12e3eb9/

Did anything change here?  Am I holding it wrong?  :-)

Thanks in advance,
Ludo’.

bug#69070: [swh-devel] No releases/revisions in /snapshot endpoints?

[Antoine Lambert]

Hello Ludovic,

The SWH data model also allows for snapshot branches to target simple directories.

The new loaders we implemented to ingest a tarball content (visit type: tarball-directory)
or a directory targeted by a CVS tag or commit (visit types: git-checkout, hg-checkout)
produce such snapshot with a single non alias branch targeting a directory.

We do not produce revisions or releases with these loaders as their purpose is to
archive a single directory (with extid mapping) without any development history.

Antoine Lambert

On 12/02/2024 17:04, Ludovic Courtès wrote:
> Dear SWH,
>
> In <https://issues.guix.gnu.org/69070> I stumbled upon what might be new
> behavior from the /snapshot endpoint:
>
>> The crux of the problem is that ‘snapshot-branches’ used to return
>> either a “revision” or a “release”, but now it can also return a
>> “directory”:
>>
>> $ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
>> {
>>    "id": "c405490b80fb35cc58b34260d9ec6add8248857f",
>>    "branches": {
>>      "1.3.2": {
>>        "target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
>>        "target_type": "directory",
>>        "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>>      },
>>      "HEAD": {
>>        "target": "1.3.2",
>>        "target_type": "alias",
>>        "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>>      }
>>    },
>>    "next_branch": null
>> }
> Current documentation mentions the ‘target_types’ query parameter, which
> I could use to restrict results to “revision” and “release”:
>
>    https://archive.softwareheritage.org/api/1/snapshot/doc/
>
> However, that’d be the empty list of the case above.
>
> Other snapshots seem to be missing releases and revisions as well, such
> as this one:
>
>    https://archive.softwareheritage.org/api/1/snapshot/55bda9214f962550baad25026fb3ac3ad12e3eb9/
>
> Did anything change here?  Am I holding it wrong?  :-)
>
> Thanks in advance,
> Ludo’.

-- 
Antoine Lambert
Software engineer for Software Heritage
https://www.softwareheritage.org
Inria, Paris

bug#69070: ‘guix lint -c archival python-scikit-learn’ crashes

[Simon Tournier]

Hi Ludo,

On Mon, 12 Feb 2024 at 11:11, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> (Note that this problem may vanish once
> <https://issues.guix.gnu.org/68741> has been merged.)

This had been merged.  Now, it does not crash.  Is the problem hidden or
vanished?

Cheers,
simon

bug#69070: ‘guix lint -c archival python-scikit-learn’ crashes

[Ludovic Courtès]

Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

> On Mon, 12 Feb 2024 at 11:11, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>> (Note that this problem may vanish once
>> <https://issues.guix.gnu.org/68741> has been merged.)
>
> This had been merged.  Now, it does not crash.  Is the problem hidden or
> vanished?

I think the initial problem (that ‘snapshot-branches’ returns a
directory) is still there, although I cannot reproduce it right now
because SWH returns 500 currently:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(guix swh)
scheme@(guile-user)> (lookup-origin-revision "https://github.com/scikit-learn/scikit-learn" "1.3.2")
ice-9/boot-9.scm:1676:22: In procedure raise-exception:
Throw to key `swh-error' with args `("https://archive.softwareheritage.org/api/1/origin/https://github.com/scikit-learn/scikit-learn/visits/?per_page=30" #<procedure http-get* (uri . rest)> #<<response> version: (1 . 1) code: 500 reason-phrase: "Internal Server Error" headers: ((date . #<date nanosecond: 0 second: 48 minute: 28 hour: 8 day: 12 month: 9 year: 2024 zone-offset: 0>) (content-type application/json) (content-length . 130) (vary accept origin cookie) (allow HEAD OPTIONS GET OPTIONS) (x-ratelimit-limit . "120") (x-ratelimit-remaining . "118") (x-ratelimit-reset . "1726133328") (x-frame-options . "DENY") (x-content-type-options . "nosniff") (referrer-policy . "same-origin") (cross-origin-opener-policy . "same-origin") (x-varnish . "665128879") (age . 0) (via "1.1 varnish (Varnish/6.5)") (strict-transport-security . "max-age=15768000;") (connection close)) port: #<closed: string 7f0c8c5aea10>>)'.

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In guix/swh.scm:
   579:11  2 (lookup-origin-revision _ _)
   296:16  1 (call "https://archive.softwareheritage.org/api/1/origin/https://github.com/scikit-learn/scikit-learn/visits/?per_page=30" _ _ #:false-if-404? _)
In ice-9/boot-9.scm:
  1676:22  0 (raise-exception _ #:continuable? _)
--8<---------------cut here---------------end--------------->8---

That said, that code is much less useful now that
<https://issues.guix.gnu.org/68741> has been merged, so perhaps we could
close the issue?

Thanks,
Ludo’.