bug#35621: Simplify comparing guix challenge results

bug#35621: Simplify comparing guix challenge results

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

I'd like to see a simpler process for comparing challenge results
producing differences.

While the manual documents how to compare a failing challenge result

  $ wget -q -O - https://ci.guix.info/nar/…-git-2.5.0 \
     | guix archive -x /tmp/git
  $ diff -ur --no-dereference /gnu/store/…-git.2.5.0 /tmp/git

To check "git", you need to run "guix challenge git" and if the results
are inconclusive, manually cut-and-paste the correct URL(s) from the
challenge output and manually download it and unpack with guix archive,
and then run a comparison utility...

What about an argument to "guix challenge" that handles the downloading
and unpacking to a temporary directory, and possibly another that also
runs a comparison tool against the results. Maybe something like:

  $ guix challenge --download-differences
  /tmp/git-XXXXX/SUBSTITTE-X/...-git-2.5.0
  /tmp/git-XXXXX/SUBSTITUTE-Y/...-git-2.5.0
  /gnu/store/...-git-2.5.0

Downloads and unpacks the substitutes, and outputs the resulting
directories.


  $ guix challenge --download-differences --compare-with="diffoscope ..."

This would do the above, plus run diffoscope on the directories
... though I think diffoscope can only compare between two things at a
time... so that might get difficult if many substitutes return results.


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

bug#35621: Simplify comparing guix challenge results

[Ludovic Courtès]

Hi!

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

> What about an argument to "guix challenge" that handles the downloading
> and unpacking to a temporary directory, and possibly another that also
> runs a comparison tool against the results. Maybe something like:
>
>   $ guix challenge --download-differences
>   /tmp/git-XXXXX/SUBSTITTE-X/...-git-2.5.0
>   /tmp/git-XXXXX/SUBSTITUTE-Y/...-git-2.5.0
>   /gnu/store/...-git-2.5.0
>
> Downloads and unpacks the substitutes, and outputs the resulting
> directories.
>
>
>   $ guix challenge --download-differences --compare-with="diffoscope ..."
>
> This would do the above, plus run diffoscope on the directories
> ... though I think diffoscope can only compare between two things at a
> time... so that might get difficult if many substitutes return results.

Sounds like a good idea.  Perhaps ‘--compare-with’ is actually enough
(it would also download things), dunno.

Ludo’.

bug#35621: Simplify comparing guix challenge results

[Ricardo Wurmus]

Ludovic Courtès <ludo@gnu.org> writes:

> Hi!
>
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>
>> What about an argument to "guix challenge" that handles the downloading
>> and unpacking to a temporary directory, and possibly another that also
>> runs a comparison tool against the results. Maybe something like:
>>
>>   $ guix challenge --download-differences
>>   /tmp/git-XXXXX/SUBSTITTE-X/...-git-2.5.0
>>   /tmp/git-XXXXX/SUBSTITUTE-Y/...-git-2.5.0
>>   /gnu/store/...-git-2.5.0
>>
>> Downloads and unpacks the substitutes, and outputs the resulting
>> directories.
>>
>>
>>   $ guix challenge --download-differences --compare-with="diffoscope ..."
>>
>> This would do the above, plus run diffoscope on the directories
>> ... though I think diffoscope can only compare between two things at a
>> time... so that might get difficult if many substitutes return results.
>
> Sounds like a good idea.  Perhaps ‘--compare-with’ is actually enough
> (it would also download things), dunno.

I think it’s fine to have just “--compare-with”, which would print the
location of the downloaded things, so you could compare with “echo” if
all you wanted is to get the differences.

--
Ricardo

bug#35621: Simplify comparing guix challenge results

[Ludovic Courtès]

Hello!

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

> I'd like to see a simpler process for comparing challenge results
> producing differences.
>
> While the manual documents how to compare a failing challenge result
>
>   $ wget -q -O - https://ci.guix.info/nar/…-git-2.5.0 \
>      | guix archive -x /tmp/git
>   $ diff -ur --no-dereference /gnu/store/…-git.2.5.0 /tmp/git
>
> To check "git", you need to run "guix challenge git" and if the results
> are inconclusive, manually cut-and-paste the correct URL(s) from the
> challenge output and manually download it and unpack with guix archive,
> and then run a comparison utility...
>
> What about an argument to "guix challenge" that handles the downloading
> and unpacking to a temporary directory, and possibly another that also
> runs a comparison tool against the results. Maybe something like:
>
>   $ guix challenge --download-differences
>   /tmp/git-XXXXX/SUBSTITTE-X/...-git-2.5.0
>   /tmp/git-XXXXX/SUBSTITUTE-Y/...-git-2.5.0
>   /gnu/store/...-git-2.5.0
>
> Downloads and unpacks the substitutes, and outputs the resulting
> directories.
>
>
>   $ guix challenge --download-differences --compare-with="diffoscope ..."

I had forgotten about this bug report (thanks for the reminder on IRC!)
and came up with something similar to/different from it:

  https://issues.guix.gnu.org/issue/38518

Let’s see whether we should adapt it!

Ludo’.