bug#26608: Provide --only-substitutes flag to "guix package --upgrade"

bug#26608: Provide --only-substitutes flag to "guix package --upgrade"

[Christopher Allan Webber]

Sometimes I do an upgrade and I want to get the latest security updates,
but I also am not really available to burn through a lot of cpu time,
(especially on my x200).

I'd love it if thre were a flag so that I could specify "only bother to
upgrade the packages where we only upgrade if a substitute is already
available.

Note that I looked at the source to see if this would be an easy thing
to add; I figured that this would be handled in roughly the same place
as --keep-going or --fallback, but it looks to me like those are
actually just passed over to the build daemon, so I'm not so sure how
easy it would be to patch this in while we're still using Nix's C
daemon?  I'm not sure.

bug#26608: Provide --only-substitutes flag to "guix package --upgrade"

[Ludovic Courtès]

Christopher Allan Webber <cwebber@dustycloud.org> skribis:

> Sometimes I do an upgrade and I want to get the latest security updates,
> but I also am not really available to burn through a lot of cpu time,
> (especially on my x200).
>
> I'd love it if thre were a flag so that I could specify "only bother to
> upgrade the packages where we only upgrade if a substitute is already
> available.

+1

> Note that I looked at the source to see if this would be an easy thing
> to add; I figured that this would be handled in roughly the same place
> as --keep-going or --fallback, but it looks to me like those are
> actually just passed over to the build daemon, so I'm not so sure how
> easy it would be to patch this in while we're still using Nix's C
> daemon?  I'm not sure.

Clients could check, among the packages that are to be installed, which
ones are substitutable (with the ‘substitute-paths’ RPC or similar) and
filter out those that are not.  No C++ involved.

Cheers,
Ludo’.

bug#26608: channels.scm supporting substitutes

[Nome Grey]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

Ludovic posted some channels.scm code in September 2018 supporting using
more substitutes.  Unfortunately his code no longer functions due to an
upgrade of guile-json in guix.

I've tried to learn enough guile to upgrade the code to the newer json
structures, and posted my changes to github at
https://github.com/nomr72/guix-substitutes-channel .  It seems to run again
now.

Maybe I can learn enough to upgrade it to check the 'guix-master'
evaluations to find the latest build of key packages.  We'll see.

[-- Attachment #2: Type: text/html, Size: 734 bytes --]

Re: bug#26608: channels.scm supporting substitutes

[zimoun]

Hi Nome

Thank you for your message.


You sent your email to all 3 bugs: 26608@debbugs.gnu.org,
32022@debbugs.gnu.org, 22629@debbugs.gnu.org. It is not a good
practise. Instead, it appears to me more appropriate to send to
help-guix@gnu.org.


First note that these are closed:

 - 22629 on Sep 2nd 2018
 - 32022 on Dec 17th 2018

so what they were proposing now exist in master, I bet. ;-)


On Tue, 3 Dec 2019 at 21:40, Nome Grey <greynome72@gmail.com> wrote:

> Ludovic posted some channels.scm code in September 2018 supporting using more substitutes.

Second, it will help to point exactly where Ludo posted what you are
referencing to. Because 22629, 26608 and 32022 are big threads. :-)


> Unfortunately his code no longer functions due to an upgrade of guile-json in guix.

If you know more or less the date, you can find the commit. And *if*
this commit is after the big 0.15 overhaul, you can 1. Guix checkout
with "guix pull --commit=<hash>" then 2. Git checkout your guix clone
to this commit and 3. use "guix environment guix" to spawn an shell
with all the dependencies available. And the code you are referencing
should work.


> Maybe I can learn enough to upgrade it to check the 'guix-master' evaluations to find the latest build of key packages.  We'll see.

Third, please describe what you are trying to achieve.


All the best,
simon

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

Hi simon, help-guix,

Thanks for taking the time to handle my email.  I'm trying to get guix
going and upgrading on my resource-starved system without doing much
compilation, using substitutes instead.

The intent of my email was to add on to ludo's example channels file at
https://issues.guix.gnu.org/issue/26608#12 with changes that would let it
continue to work on more recent guix releases, which I'd shared at
https://github.com/nomr72/guix-substitutes-channel .

On Tue, Dec 3, 2019, 1:14 PM zimoun <zimon.toutoune@gmail.com> wrote:

> You sent your email to all 3 bugs: 26608@debbugs.gnu.org,
> 32022@debbugs.gnu.org, 22629@debbugs.gnu.org. It is not a good
> practise. Instead, it appears to me more appropriate to send to
> help-guix@gnu.org.
>

My intent was to help people finding the issue with google or who were
still following the bugs due to interest.  32022 was closed unimplemented.
Maybe including 22629 was an error; many of the relevant messages in 32022
and 26608 came from and were also sent to the 22629 thread, so I thought it
could be a path to find the topic from search engines, and I thought it
could be safe since it was closed.

I didn't know I would be bugging people unrelated to this topic -- is help
the right place for this?


> > Ludovic posted some channels.scm code in September 2018 supporting using
> more substitutes.
>
> Second, it will help to point exactly where Ludo posted what you are
> referencing to. Because 22629, 26608 and 32022 are big threads. :-)
>

Linked at top.  Sorry.

> Unfortunately his code no longer functions due to an upgrade of
> guile-json in guix.
>
> If you know more or less the date, you can find the commit. And *if*
> this commit is after the big 0.15 overhaul, you can 1. Guix checkout
> with "guix pull --commit=<hash>" then 2. Git checkout your guix clone
> to this commit and 3. use "guix environment guix" to spawn an shell
> with all the dependencies available. And the code you are referencing
> should work.
>

Unfortunately the code is for upgrading to the latest version of guix!  =)
Thank you though.  This might have helped a lot when I was bootstrapping.

> Maybe I can learn enough to upgrade it to check the 'guix-master'
> evaluations to find the latest build of key packages.  We'll see.
>
> Third, please describe what you are trying to achieve.
>

Here I am discussing trying to extend the channels.scm to handle an
arbitrary set of packages as ludo describes in
https://issues.guix.gnu.org/issue/26608#14 .  The hope is to produce the
latest guix commit hash where a set of packages have been successfully
built, using the cuirass api.

I think I can pull it off with some confused searching of the guile manual
and cuirass source code, but I'm not sure if it will meet my goal of having
a snappy-to-upgrade substitutes-based foreign guix.

Thanks for reading, processing, doing your work, and being you =)

nomr

Re: bug#26608: channels.scm supporting substitutes

[zimoun]

Dear,

On Wed, 4 Dec 2019 at 00:32, Nome Grey <greynome72@gmail.com> wrote:

> [...]  I'm trying to get guix going and upgrading on my resource-starved system without doing much compilation, using substitutes instead.

Have you already Guix installed on your system?
If yes, how did you install it? And what is the current version of
Guix that you are running?


All the best,
simon

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

Simon, thanks for your attention.

On Wed, Dec 4, 2019, 2:23 AM zimoun <zimon.toutoune@gmail.com> wrote:

> Dear,
>
> On Wed, 4 Dec 2019 at 00:32, Nome Grey <greynome72@gmail.com> wrote:
>
> > [...]  I'm trying to get guix going and upgrading on my resource-starved
> system without doing much compilation, using substitutes instead.
>
> Have you already Guix installed on your system?
> If yes, how did you install it? And what is the current version of
> Guix that you are running?
>

Yes, as mentioned in my previous email I am working with the channels.scm
file used by 'guix pull'.

I installed a foreign distribution of guix 1.0.1 i686 using the binary
installation steps from the manual.

I am currently running guix aca2bf5 but have a long-running process to
downgrade to 1b6c5e8 in order to find a substitute for ungoogled-chromium 78

>

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

Sorry, message sent before I finished composing.  My worn out fingers spaz
a lot.  A little more below.

On Wed, Dec 4, 2019, 4:25 AM Nome Grey <greynome72@gmail.com> wrote:

> Simon, thanks for your attention.
>
> On Wed, Dec 4, 2019, 2:23 AM zimoun <zimon.toutoune@gmail.com> wrote:
>
>> Dear,
>>
>> On Wed, 4 Dec 2019 at 00:32, Nome Grey <greynome72@gmail.com> wrote:
>>
>> > [...]  I'm trying to get guix going and upgrading on my
>> resource-starved system without doing much compilation, using substitutes
>> instead.
>>
>> Have you already Guix installed on your system?
>> If yes, how did you install it? And what is the current version of
>> Guix that you are running?
>>
>
> Yes, as mentioned in my previous email I am working with the channels.scm
> file used by 'guix pull'.
>
> I installed a foreign distribution of guix 1.0.1 i686 using the binary
> installation steps from the manual.
>
> I am currently running guix aca2bf5 but have a long-running process to
> downgrade to 1b6c5e8 in order to find a substitute for ungoogled-chromium 78
>

The system itself is a little ad hoc, because `guix upgrade` can take many
days to run, mostly building chromium, and I haven't been able to allow it
to complete yet.  Hence right now some packages are from mismatching
releases, but there are not very many installed overall, and I'm hoping I
can settle into 1b6c5e8 soon if the chromium substitution works out.

Downgrading back to 1.0.1 has also worked for me in the past to get this
system consistent.

The underlying system is fedora 18 running on an olpc xo 1.5 (all open
source hw&sw but uses device-specific custom bootloader, kernel, and some
daemons and setup utilities).  It seems too slow and specialized for guix
really, so I've been trying to put a lot of effort in myself to get it
working since my use case is so obscure.  Software updates haven't been
sustained for this system by olpc, and using guix has been super helpful in
bringing working recent software to it.

nomr

>

Re: bug#26608: channels.scm supporting substitutes

[zimoun]

On Wed, 4 Dec 2019 at 13:40, Nome Grey <greynome72@gmail.com> wrote:

>> Yes, as mentioned in my previous email I am working with the channels.scm file used by 'guix pull'.
>>
>> I installed a foreign distribution of guix 1.0.1 i686 using the binary installation steps from the manual.
>>

You do not have have to use all the code you found in the bug reports.
It should work out-of-box with your current version of Guix.

The channels are used with "guix pull -C /path/to/your/file.scm" and
'file.scm' respects the syntax presented here [1].

[1] https://guix.gnu.org/manual/en/html_node/Channels.html#Channels


>> I am currently running guix aca2bf5 but have a long-running process to downgrade to 1b6c5e8 in order to find a substitute for ungoogled-chromium 78

To see if the substitute is available or not, the command "guix
weather" fits your need [2].

[2] https://guix.gnu.org/manual/en/html_node/Invoking-guix-weather.html#Invoking-guix-weather


However, the package ungoogled-chromium 78 is not available for the
system i686. You probably need to build it yourself (ressource
consuming) with "guix build ungoogled-chromium".

I am not sure that channel will help to fetch the result of any build.
Channel helps to add more recipes about how to build packages. And I
am not aware of other public substitutes server than
https://ci.guix.gnu.org.


Hope that helps,
simon

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

Simon,

On Wed, Dec 4, 2019, 5:57 AM zimoun <zimon.toutoune@gmail.com> wrote:

> On Wed, 4 Dec 2019 at 13:40, Nome Grey <greynome72@gmail.com> wrote:
>
> >> Yes, as mentioned in my previous email I am working with the
> channels.scm file used by 'guix pull'.
> >>
> >> I installed a foreign distribution of guix 1.0.1 i686 using the binary
> installation steps from the manual.
> >>
>
> You do not have have to use all the code you found in the bug reports.
> It should work out-of-box with your current version of Guix.
>
> The channels are used with "guix pull -C /path/to/your/file.scm" and
> 'file.scm' respects the syntax presented here [1].
>
> [1] https://guix.gnu.org/manual/en/html_node/Channels.html#Channels


Thanks.  The problem is that old substitutes are not used on new guix
pulls, so upgrading takes unworkably long.  Pulling to the right commit, as
partly described in the bug reports, can make many more substitutes work,
and has helped me a lot.

>> I am currently running guix aca2bf5 but have a long-running process to
> downgrade to 1b6c5e8 in order to find a substitute for ungoogled-chromium 78
>
> To see if the substitute is available or not, the command "guix
> weather" fits your need [2].
>
> [2]
> https://guix.gnu.org/manual/en/html_node/Invoking-guix-weather.html#Invoking-guix-weather
>
>
> However, the package ungoogled-chromium 78 is not available for the
> system i686. You probably need to build it yourself (ressource
> consuming) with "guix build ungoogled-chromium".
>

I'm pretty sure I found just recently a successful build of
ungoogled-chromium-78 for i686-linux:
http://berlin.guixsd.org/build/1979882/details .  What do you think?

I misspoke with "downgrade": this build was for guix 1b6c5e8 which I am
trying to upgrade to in the hopes of using it.  I found it via the cuirass
api; it's evaluation 9047.

I am not sure that channel will help to fetch the result of any build.
> Channel helps to add more recipes about how to build packages. And I
> am not aware of other public substitutes server than
> https://ci.guix.gnu.org.
>

By only upgrading to releases with substitutes available, fewer builds are
required on the local system, which can still be occasionally brought up to
date.  The user channel filters the official channel, to hide releases
without substitutes built yet.

>

Re: bug#26608: channels.scm supporting substitutes

[zimoun]

On Wed, 4 Dec 2019 at 15:56, Nome Grey <greynome72@gmail.com> wrote:

> Thanks.  The problem is that old substitutes are not used on new guix pulls, so upgrading takes unworkably long.  Pulling to the right commit, as partly described in the bug reports, can make many more substitutes work, and has helped me a lot.

I am not sure it works the way you describe.

From my understanding, when you use "guix install foo", Guix computes
a derivation and query the substitutes. If the derivation exists on
the server, it downloads. It does not matter so much from which Guix
version you run "guix install". On the contrary, too recent version
(new "guix pull") should not be built by Cuirass yet and so served as
substitute.

What is the output of the command: "guix install ungoogled-chromium
--dry-run" on your machine?

If it does not do the job, you can try: "guix install
ungoogled-chromium --substitute-urls=http://berlin.guixsd.org". Does
it fail? What is the output of the command?


( I think that http://berlin.guixsd.org and https://ci.guix.gnu.org
point to the same build farm.)


>> >> I am currently running guix aca2bf5 but have a long-running process to downgrade to 1b6c5e8 in order to find a substitute for ungoogled-chromium 78
>>
>> To see if the substitute is available or not, the command "guix
>> weather" fits your need [2].
>>
>> [2] https://guix.gnu.org/manual/en/html_node/Invoking-guix-weather.html#Invoking-guix-weather
>>
>>
>> However, the package ungoogled-chromium 78 is not available for the
>> system i686. You probably need to build it yourself (ressource
>> consuming) with "guix build ungoogled-chromium".
>
>
> I'm pretty sure I found just recently a successful build of ungoogled-chromium-78 for i686-linux: http://berlin.guixsd.org/build/1979882/details .  What do you think?

You can also try: "guix build ungoogled-chromium --dry-run" to see
what Guix will download and what it will build locally.


I am not sure... but it is not because you find the webpage of a
successful build of the package ungoogled-chromium-78 for i686-linux
that it means the substitutes is available; even if this one is
available. ;-)

http://ci.guix.gnu.org/build/1979882/details


>> I am not sure that channel will help to fetch the result of any build.
>> Channel helps to add more recipes about how to build packages. And I
>> am not aware of other public substitutes server than
>> https://ci.guix.gnu.org.
>
>
> By only upgrading to releases with substitutes available, fewer builds are required on the local system, which can still be occasionally brought up to date.  The user channel filters the official channel, to hide releases without substitutes built yet.

I am not sure it works as you think.


I am checking why Guix claims the substitute is not available for i686
when it really is.


All the best,
simon

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

On Wed, Dec 4, 2019, 9:33 AM zimoun <zimon.toutoune@gmail.com> wrote:

> From my understanding, when you use "guix install foo", Guix computes
> a derivation and query the substitutes. If the derivation exists on
> the server, it downloads. It does not matter so much from which Guix
> version you run "guix install". On the contrary, too recent version
> (new "guix pull") should not be built by Cuirass yet and so served as
> substitute.
>

It seems the guix version you have is also the version of the set of
packages your system is aware of.  `guix pull` seems to update both.  So
which derivation is computed does depend on your guix version, and that is
why too new versions aren't matched yet on the server.

What is the output of the command: "guix install ungoogled-chromium
> --dry-run" on your machine?
>

So, I did this with the setup I described, but didn't e-mail you yet
because I wanted to try after my `guix pull` of commit 1b6c5e8 completed,
which finished today, but then my system crashed after I forgot to reboot
guix-daemon, and I lost the first output.  So the substitute is actually
now available for me -- selecting that commit worked -- but it wasn't
before.

I also noticed often the substitute is not chosen when the server is having
congestion issues.

$ guix --version
guix (GNU Guix) 1b6c5e803989210720de98862571a7ce1dd8e8dd
Copyright (C) 2019 the Guix authors
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ guix install ungoogled-chromium --dry-run
The following package would be upgraded:
   ungoogled-chromium 74.0.3729.131-0.9e33022 → 78.0.3904.108-0.8f06513
/gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513

The following derivation would be built:
   /gnu/store/akzyrbp4pirvxjss0s41fv3ggpy70a8q-profile.drv
82.1 MB would be downloaded:
   /gnu/store/9aiizpqhpiixds5x52w67q7p3wxsz1db-libcdio-paranoia-10.2+2.0.0
   /gnu/store/ghj6k72hpvilkv53hx2frz5a5qazvrk2-libcdio-2.1.0
   /gnu/store/mpsjsddxbzjncvrxqll59lmwad6yygl0-ffmpeg-4.2.1

 /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
The following profile hooks would be built:
   /gnu/store/71fl8l2n2l01yjd2ajqz1qs7a5f0g8ar-gtk-im-modules.drv
   /gnu/store/7nrwf66l0r9vdzcxyl56xzkrm3dj5bhl-gtk-icon-themes.drv
   /gnu/store/f3xpxb5a79xvxxaav7zvmg1lmsslxz8f-xdg-mime-database.drv
   /gnu/store/g1acs7i62984xkwkwshyirinqzvf20xl-glib-schemas.drv
   /gnu/store/k0xp9xvc8njzqzwy7ndj7w5p8kk4ljcv-ca-certificate-bundle.drv
   /gnu/store/n4zwqdspgy8dnfmz7y0sv425mc81mm87-fonts-dir.drv
   /gnu/store/qmb0w52ys35l7hdrbkx3vwbsd2b31nj3-manual-database.drv
   /gnu/store/syspl3xawhbpynvjn6a5n34fyap7awap-xdg-desktop-database.drv
   /gnu/store/vpy3llakmf6kfb2gf93309m8wdrkhjvc-info-dir.drv


( I think that http://berlin.guixsd.org and https://ci.guix.gnu.org
> point to the same build farm.)
>

Yeah, they both resolve to 141.80.181.40 over here.  I think berlin is
maybe left over from when hydra was in use, not sure.  I was kind of
assuming they were the same and may have confused things not clarifying
that.

> By only upgrading to releases with substitutes available, fewer builds
> are required on the local system, which can still be occasionally brought
> up to date.  The user channel filters the official channel, to hide
> releases without substitutes built yet.


I am not sure it works as you think.
>

That is how the specialized channels.scm they were discussing in the bugs
is supposed to work.  Following ludo's lead was how I managed to find this
way to use the substitute for ungoogled-chromium.

But yeah I'm not really sure where the package index is stored to directly
connect it to guix pull's behavior and such.  It's guesswork for me for
sure.

I am checking why Guix claims the substitute is not available for i686
> when it really is.
>

If it's not because the ci server was overloaded, maybe it is that one of
the inputs to the package has been updated, such that a new derivation than
the one built is used now?

Re: bug#26608: channels.scm supporting substitutes

[zimoun]

Hi,

On Wed, 4 Dec 2019 at 22:50, Nome Grey <greynome72@gmail.com> wrote:

> I also noticed often the substitute is not chosen when the server is having congestion issues.

I do not understand what this means.


The tool to know if the build farm has already built the package is
"guix weather". If it is not built yet, then you have only two
choices: 1. build it yourself or 2. wait. There are discussions to
share more in distributed fashion via IPFS -- which will partially
diminish the time to wait between when the package is included to the
tree and when the binary is available -- and a patch already exists in
the Bug Tracker; but it lacks tests and feedback. Note also that not
enough of us use "guix publish". ;-)


> $ guix install ungoogled-chromium --dry-run
> The following package would be upgraded:
>    ungoogled-chromium 74.0.3729.131-0.9e33022 → 78.0.3904.108-0.8f06513 /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
>
> The following derivation would be built:
>    /gnu/store/akzyrbp4pirvxjss0s41fv3ggpy70a8q-profile.drv
> 82.1 MB would be downloaded:
>    /gnu/store/9aiizpqhpiixds5x52w67q7p3wxsz1db-libcdio-paranoia-10.2+2.0.0
>    /gnu/store/ghj6k72hpvilkv53hx2frz5a5qazvrk2-libcdio-2.1.0
>    /gnu/store/mpsjsddxbzjncvrxqll59lmwad6yygl0-ffmpeg-4.2.1
>    /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
> The following profile hooks would be built:
>    /gnu/store/71fl8l2n2l01yjd2ajqz1qs7a5f0g8ar-gtk-im-modules.drv
>    /gnu/store/7nrwf66l0r9vdzcxyl56xzkrm3dj5bhl-gtk-icon-themes.drv
>    /gnu/store/f3xpxb5a79xvxxaav7zvmg1lmsslxz8f-xdg-mime-database.drv
>    /gnu/store/g1acs7i62984xkwkwshyirinqzvf20xl-glib-schemas.drv
>    /gnu/store/k0xp9xvc8njzqzwy7ndj7w5p8kk4ljcv-ca-certificate-bundle.drv
>    /gnu/store/n4zwqdspgy8dnfmz7y0sv425mc81mm87-fonts-dir.drv
>    /gnu/store/qmb0w52ys35l7hdrbkx3vwbsd2b31nj3-manual-database.drv
>    /gnu/store/syspl3xawhbpynvjn6a5n34fyap7awap-xdg-desktop-database.drv
>    /gnu/store/vpy3llakmf6kfb2gf93309m8wdrkhjvc-info-dir.drv


So this will download what you want. :-)
Everything is available, AFAICT.
No issue, no workaround.


> Yeah, they both resolve to 141.80.181.40 over here.  I think berlin is maybe left over from when hydra was in use, not sure.  I was kind of assuming they were the same and may have confused things not clarifying that.

AFAIU, it is a bit more complicated. It is the same build farm but
then some network configuration is different (CDN available or not,
etc.). In short: use ci.guix.gnu.org.


>> > By only upgrading to releases with substitutes available, fewer builds are required on the local system, which can still be occasionally brought up to date.  The user channel filters the official channel, to hide releases without substitutes built yet.
>>
>> I am not sure it works as you think.
>
> That is how the specialized channels.scm they were discussing in the bugs is supposed to work.  Following ludo's lead was how I managed to find this way to use the substitute for ungoogled-chromium.

I am not sure to understand. Maybe you should read (again?) the manual
about channels and derivations. It is one better source of information
than any bug report where hypothetical had been discussed. The manual
explains for sure how Guix works; bug reports not always, they are not
a reliable source of information, IMHO.


> But yeah I'm not really sure where the package index is stored to directly connect it to guix pull's behavior and such.  It's guesswork for me for sure.

AFAIU, the package index lives under ~/.config/guix/current. But it is
not important, IMO.


>> I am checking why Guix claims the substitute is not available for i686
>> when it really is.
>
> If it's not because the ci server was overloaded, maybe it is that one of the inputs to the package has been updated, such that a new derivation than the one built is used now?

No. I do not know if it comes from "-s i686-linux" because I running
on a x86 system, or other.
The point is: the derivation that my system computes is different of
the derivation that I expect.

--8<---------------cut here---------------start------------->8---
guix build ungoogled-chromium -s i686-linux -n -d
/gnu/store/45cfay69pp6prky2kw0r21nv5lp3xgs9-ungoogled-chromium-78.0.3904.108-0.8f06513.drv
--8<---------------cut here---------------start------------->8---

instead of /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513


All the best,
simon

Re: bug#26608: channels.scm supporting substitutes

[Nome Grey]

On Thu, Dec 5, 2019, 3:45 AM zimoun <zimon.toutoune@gmail.com> wrote:

> Hi,
>
> On Wed, 4 Dec 2019 at 22:50, Nome Grey <greynome72@gmail.com> wrote:
>
> > I also noticed often the substitute is not chosen when the server is
> having congestion issues.
>
> I do not understand what this means.
>
>
> The tool to know if the build farm has already built the package is
> "guix weather". If it is not built yet, then you have only two
> choices: 1. build it yourself or 2. wait. There are discussions to
> share more in distributed fashion via IPFS -- which will partially
> diminish the time to wait between when the package is included to the
> tree and when the binary is available -- and a patch already exists in
> the Bug Tracker; but it lacks tests and feedback. Note also that not
> enough of us use "guix publish". ;-)
>

I mean when I cannot access the pages of ci.guix.gnu.org, `guix weather`
could report missing substitutes.

>
> > $ guix install ungoogled-chromium --dry-run
> > The following package would be upgraded:
> >    ungoogled-chromium 74.0.3729.131-0.9e33022 → 78.0.3904.108-0.8f06513
> /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
> >
> > The following derivation would be built:
> >    /gnu/store/akzyrbp4pirvxjss0s41fv3ggpy70a8q-profile.drv
> > 82.1 MB would be downloaded:
> >
> /gnu/store/9aiizpqhpiixds5x52w67q7p3wxsz1db-libcdio-paranoia-10.2+2.0.0
> >    /gnu/store/ghj6k72hpvilkv53hx2frz5a5qazvrk2-libcdio-2.1.0
> >    /gnu/store/mpsjsddxbzjncvrxqll59lmwad6yygl0-ffmpeg-4.2.1
> >
> /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
> > The following profile hooks would be built:
> >    /gnu/store/71fl8l2n2l01yjd2ajqz1qs7a5f0g8ar-gtk-im-modules.drv
> >    /gnu/store/7nrwf66l0r9vdzcxyl56xzkrm3dj5bhl-gtk-icon-themes.drv
> >    /gnu/store/f3xpxb5a79xvxxaav7zvmg1lmsslxz8f-xdg-mime-database.drv
> >    /gnu/store/g1acs7i62984xkwkwshyirinqzvf20xl-glib-schemas.drv
> >    /gnu/store/k0xp9xvc8njzqzwy7ndj7w5p8kk4ljcv-ca-certificate-bundle.drv
> >    /gnu/store/n4zwqdspgy8dnfmz7y0sv425mc81mm87-fonts-dir.drv
> >    /gnu/store/qmb0w52ys35l7hdrbkx3vwbsd2b31nj3-manual-database.drv
> >    /gnu/store/syspl3xawhbpynvjn6a5n34fyap7awap-xdg-desktop-database.drv
> >    /gnu/store/vpy3llakmf6kfb2gf93309m8wdrkhjvc-info-dir.drv
>
>
> So this will download what you want. :-)
> Everything is available, AFAICT.
> No issue, no workaround.


I had to `guix pull --commit=1b6c5e8` to see the above, which took me a
couple days to finish because there weren't substitutes available for that
guix binary.

Would you try the command after a `guix pull --commit=1b6c5e8`, so you can
understand why I am working with this?

>> > By only upgrading to releases with substitutes available, fewer builds
> are required on the local system, which can still be occasionally brought
> up to date.  The user channel filters the official channel, to hide
> releases without substitutes built yet.
> >>
> >> I am not sure it works as you think.
> >
> > That is how the specialized channels.scm they were discussing in the
> bugs is supposed to work.  Following ludo's lead was how I managed to find
> this way to use the substitute for ungoogled-chromium.
>
> I am not sure to understand. Maybe you should read (again?) the manual
> about channels and derivations. It is one better source of information
> than any bug report where hypothetical had been discussed. The manual
> explains for sure how Guix works; bug reports not always, they are not
> a reliable source of information, IMHO.
>

Channels are usually used for selecting new sets of packages, but are also
flexible enough to provide for this filtering.  Try the command I mentioned
above to see the value.

> But yeah I'm not really sure where the package index is stored to
> directly connect it to guix pull's behavior and such.  It's guesswork for
> me for sure.
>
> AFAIU, the package index lives under ~/.config/guix/current. But it is
> not important, IMO.
>

Yeah somewhere in the profile holding the guix binary, as you say.

>> I am checking why Guix claims the substitute is not available for i686
> >> when it really is.
> >
> > If it's not because the ci server was overloaded, maybe it is that one
> of the inputs to the package has been updated, such that a new derivation
> than the one built is used now?
>
> No. I do not know if it comes from "-s i686-linux" because I running
> on a x86 system, or other.
> The point is: the derivation that my system computes is different of
> the derivation that I expect.
>
> --8<---------------cut here---------------start------------->8---
> guix build ungoogled-chromium -s i686-linux -n -d
>
> /gnu/store/45cfay69pp6prky2kw0r21nv5lp3xgs9-ungoogled-chromium-78.0.3904.108-0.8f06513.drv
> --8<---------------cut here---------------start------------->8---
>
> instead of
> /gnu/store/wcjr5n9ljvbw6ak3705pqs7y6n3fk934-ungoogled-chromium-78.0.3904.108-0.8f06513
>

Right.  I thought an output only changes if its inputs change, and the
derivations were outputs in this sense.  What version of guix computes that
output?

bug#32022: bug#26608: channels.scm supporting substitutes

[Ludovic Courtès]

Hi,

Nome Grey <greynome72@gmail.com> skribis:

> Ludovic posted some channels.scm code in September 2018 supporting using
> more substitutes.  Unfortunately his code no longer functions due to an
> upgrade of guile-json in guix.
>
> I've tried to learn enough guile to upgrade the code to the newer json
> structures, and posted my changes to github at
> https://github.com/nomr72/guix-substitutes-channel .  It seems to run again
> now.

Nice, thanks for sharing!

Ludo’.