Re: guix weather issue? (was Re: guix package builds, subsitutes and --no-build)

[Chris Marusich <cmmarusich@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 7715 bytes --]

Hi Giovanni,

I think I may have been confused when I sent my last email.  I saw Guix
downloading the source and mistakenly assumed that it was downloading a
substitute for Chromium.  I looked at the log more closely, and I think
now I was actually able to reproduce your issue.

Using the information you provided, I ran this command:

guix package --substitute-urls=https://ci.guix.info -p /tmp/test-profile -m /tmp/manifest.scm 2>&1 | tee /tmp/package-install.log

I've attached the gzipped log to this email, for your reference.  In it,
you can find messages like the following:

--8<---------------cut here---------------start------------->8---
$ grep ungoogled-chromium /tmp/package-install.log 
downloading from https://ci.guix.info/nar/gzip/g92nls9b9ggnhgmdd9ycmcws38z9aw43-ungoogled-chromium-a58db88-checkout...
 ungoogled-chromium-a58db88-checkout  672KiB 729KiB/s 00:01 [##################] 100.0%
building /gnu/store/w54r150nj8ksyznj6b8q92fgr1n0l20h-ungoogled-chromium-72.0.3626.109.tar.xz.drv...
building /gnu/store/4mvzzx2jmr4r4p2kx0hcvwr9s9lvx0gd-ungoogled-chromium-72.0.3626.109.drv...
   /gnu/store/r0jfc6lfakh0mg764kyxzzmlf21b3d7c-ungoogled-chromium-72.0.3626.109.drv
applying 8 grafts for /gnu/store/r0jfc6lfakh0mg764kyxzzmlf21b3d7c-ungoogled-chromium-72.0.3626.109.drv...
--8<---------------cut here---------------end--------------->8---

As you can see, it seems Guix actually did build ungoogled-chromium (the
line "building
/gnu/store/4mvzzx2jmr4r4p2kx0hcvwr9s9lvx0gd-ungoogled-chromium-72.0.3626.109.drv"
is followed by what appears to be many lines of compilation output).

Ultimately, what gets installed into the profile is:

  /gnu/store/za5b7s35jskvvvlv0favqy4as1cfk332-ungoogled-chromium-72.0.3626.109

which is the output of

  /gnu/store/r0jfc6lfakh0mg764kyxzzmlf21b3d7c-ungoogled-chromium-72.0.3626.109.drv

which itself is a graft derivation.  This graft derivation appears to
have the following derivation as an input:

  /gnu/store/4mvzzx2jmr4r4p2kx0hcvwr9s9lvx0gd-ungoogled-chromium-72.0.3626.109.drv

which is the derivation that builds the ungrafted ungoogled-chromium.
This makes sense, since the graft derivation needs to modify the
original.

A few minutes ago, I ran "guix weather" again.  It says 1 substitute
(which one, I wonder?) is available:

--8<---------------cut here---------------start------------->8---
$ guix weather -m /tmp/manifest.scm 
computing 1 package derivations for x86_64-linux...
looking for 1 store items on https://ci.guix.info...
https://ci.guix.info
  100.0% substitutes available (1 out of 1)
  99.3 MiB of nars (compressed)
  288.3 MiB on disk (uncompressed)
  0.002 seconds per request (0.0 seconds in total)
  612.7 requests per second
  'https://ci.guix.info/api/queue?nr=1000' returned 504 ("Gateway Time-out")
--8<---------------cut here---------------end--------------->8---

I then tried to download substitutes manually.  However, when I try to
download the grafted and ungrafted versions, I get a 404.  Here's a
summary of these derivations, based on what I observed:

This graft derivation for ungoogled-chromium...

  /gnu/store/r0jfc6lfakh0mg764kyxzzmlf21b3d7c-ungoogled-chromium-72.0.3626.109.drv

...produces...

  /gnu/store/za5b7s35jskvvvlv0favqy4as1cfk332-ungoogled-chromium-72.0.3626.109

When I try to fetch it via...

  curl -v -o /dev/null https://ci.guix.info/nar/gzip/za5b7s35jskvvvlv0favqy4as1cfk332-ungoogled-chromium-72.0.3626.109

...I get a 404 from ci.guix.info.

This non-graft derivation for ungoogled-chromium (which appears to be an
input to the graft derivation above)...

  /gnu/store/4mvzzx2jmr4r4p2kx0hcvwr9s9lvx0gd-ungoogled-chromium-72.0.3626.109.drv

...produces...

  /gnu/store/2xnx8nglb8dyjvkq0zra17baxkyic721-ungoogled-chromium-72.0.3626.109

When I try to fetch it via...

  curl -v -o /dev/null https://ci.guix.info/nar/gzip/2xnx8nglb8dyjvkq0zra17baxkyic721-ungoogled-chromium-72.0.3626.109

...I get a 404 from ci.guix.info.

I expected to get a 200 in at least one of those cases, since "guix
weather" said that 1 substitute was available.  I don't really know
what's going on here, but it seems strange.  I guess we need to
investigate more.

It would be interesting to know if "guix weather" honors the current
channel configuration, or if it always checks master.  If it always
checks master, then that could explain what we're seeing.  Anyway, it
seems like it might be time for a bug report.  Could you summarize the
issue and open one?

- 

Giovanni Biscuolo <g@xelera.eu> writes:

> Hi Chris,
>
> Chris Marusich <cmmarusich@gmail.com> writes:
>
>> Giovanni Biscuolo <g@xelera.eu> writes:
>>
>>> AFAIU the issue is "guix weather" reporting on the availability related
>>> to current master and not of user commit: am I wrong?
>>
>> I'm not sure.  That would explain the issue you saw.  I haven't checked
>> the code.  Maybe you could take a peek?  If "guix weather" is using
>> master branch and ignoring the current channel configuration, it seems
>> like it might be unintended behavior.
>
> I'll try to understand guix weather code (guix/guix/scripts/weather.scm)
> but I'm pretty new to Guile and Guix so it will be a long study for me
>
> ...so sorry, but for now do not count on me for a timely report on this
> :-)
>
>>> a little (digression
>>>
>>>   anyway even if that is not the issue, users should have some way to
>>>   check if a substitute is available for their current commit, so they
>>>   can decide if they are willing to locally build or not.
>
> [...]
>
>> I agree it would be nice if one could control the behavior more easily.
>> However, someone needs to put in the time to design and implement the
>> solution.  So far, I think people with time and energy have chosen
>> instead to focus on improving substitute availability, in the hopes that
>> it will prove more useful in the long term.
>
> yes I'm aware of this and I find this is not a priority for Guix,
> improving substitutes availability is
>
>> Would you be interesting in working on it?
>
> yes but for me "it's a long way to the top if you (me) wanna
> rock'n'roll" :-)
>
> [...]
>
>> I have attempted to reproduce the issue using that information.  When I
>> ran "guix pull" to use the same version of Guix you were using
>> (a4fc80254a53b46b33f138d1009ddd044b8cb6be) and then ran "guix weather",
>> I saw the same output as you (i.e., ci.guix.info reported that the
>> substitute was available).  However, when I ran...
>>
>>   guix package \
>>        --substitute-urls=https://ci.guix.info \
>>        -p /tmp/test-profile \
>>        -m /tmp/manifest.scm
>>
>> ...Guix began downloading chromium from ci.guix.info.  The contents of
>> /tmp/manifest.scm is the same manifest you provided.  So, unfortunately
>> this means I wasn't able to reproduce the issue you experienced.
>> Everything seems to be working correctly on my end.
>
> could you please report here the exact substitute derivation you got?
>
> on Monday Ricardo told us:
>
>>> --8<---------------cut here---------------start------------->8---
>>> $ guix package -i ungoogled-chromium
>>> substitute: updating substitutes from 'https://ci.guix.info'... 100.0%
>>> building
>>> /gnu/store/4mvzzx2jmr4r4p2kx0hcvwr9s9lvx0gd-ungoogled-chromium-72.0.3626.109.drv...
>>> \ 'set-paths' phase^C
>>> --8<---------------cut here---------------end--------------->8---
>>
>>There is no such derivation on the server at berlin.guixsd.org.
>
> It wold be interesting to see what derivation you got this time
>
> Thanks!
> Giovanni
>
> [...]

-- 
Chris

[-- Attachment #1.2: package-install.log.gz --]
[-- Type: application/octet-stream, Size: 498169 bytes --]

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]