Hello,

> Is that because the changes you describe were done after the staging
> data was loaded or is it a bug?

Our staging instance inherits its append-only property from our main
archive. In the staging case (for "prototypes", soon-to-be-deployed new
feature or so), that makes it hard to see through the "old bug" noise.
It's old origins that were ingested initially with a first version of
the lister (which got iteratively fixed).

----

@anlambert made a pass this week in docker (from scratch) to check (thx ;)

> Excellent!  I believe this addresses a problem we recently reported
> regarding tarballs published with our own content-addressed URLs, which
> look like:
>
>   https://bordeaux.guix.gnu.org/file/BiocNeighbors_1.20.0.tar.gz/sha256/0a5wg099fgwjbzd6r3mr4l02rcmjqlkdcz1w97qzwx1mir41fmas

As a result, he actually enhanced the listing so the urls mentioned
earlier ^ is treated correctly out of the data in the url. (@me That
needs a bump in deployment [for next week])

Early on, I was referring to another heuristic using a HEAD query to
parse header informations [if any]. As that specific url does not
provide any, so it passed through.

----

Note: cc-ed julien@malka.sh instead of community@nixos.org (as asked in
the thread)

Cheers,
--
tony / Antoine R. Dumont (@ardumont)

-----------------------------------------------------------------
gpg fingerprint BF00 203D 741A C9D5 46A8 BE07 52E2 E984 0D10 C3B8


Timothy Sample <samplet@ngyro.com> writes:

> Hello,
>
> This is very exciting work, thanks everyone!
>
> "Antoine R. Dumont (@ardumont)" <ardumont@softwareheritage.org> writes:
>
>> FWIW, in the "new" lister [1] implementation, there are a bunch of extra
>> computations done [1] to try and resolve those situations. It's trying
>> to fetch more information from upstream server (e.g. crates urls which
>> ends in /download, ...) now. It's probably not exhaustive though.
>>
>> [1] https://gitlab.softwareheritage.org/swh/devel/swh-lister/-/blob/master/swh/lister/nixguix/lister.py?ref_type=heads
>
> I was just looking over some of the new results and noticed that crates
> are being treated as ‘content’ rather than ‘tarball-directory’.  E.g.:
>
> https://webapp.staging.swh.network/browse/content/sha1_git:e05b33b2d3b40254ceaaa5fe4c501d1b15c75ea6/?origin_url=https://crates.io/api/v1/crates/diff/0.1.12/download
>
> Is that because the changes you describe were done after the staging
> data was loaded or is it a bug?
>
>
> -- Tim