Branch with only available substitutes ?

Branch with only available substitutes ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

Hi !

New user of guix for a few months, still getting used to everything.
Don't hesitate to advise me if I'm doing anything wrong ;)

I find myself often waiting for heavy packages (ungoogled-chromium,
cargo...) to be built as substitutes before updating my system and I
find it a bit long to my personal taste.

Would it be possible to add a branch in guix sources following only
available substitutes, so that each time I pull, I can immediatly
and quickly update. Or would that need to stay as someone's personal
project ?

Doesn't seem to complicated on principle, with guix weather and a
cron job.

Also thanks for everything folks, guix is awesome.

Cheers,

Nicolas Graves

Re: Branch with only available substitutes ?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1162 bytes --]

Hi,

This question comes up often on #guix.

Nicolas Graves via Development of GNU Guix and the GNU System
distribution. schreef op zo 09-01-2022 om 21:44 [+0100]:
> Hi !
> 
> New user of guix for a few months, still getting used to everything.
> Don't hesitate to advise me if I'm doing anything wrong ;)
> 
> I find myself often waiting for heavy packages (ungoogled-chromium,
> cargo...) to be built as substitutes before updating my system and I
> find it a bit long to my personal taste.

Potential problem: there are packages that fail to build.

> Would it be possible to add a branch in guix sources following only
> available substitutes, so that each time I pull, I can immediatly
> and quickly update. Or would that need to stay as someone's personal
> project ?
> 
> Doesn't seem to complicated on principle, with guix weather and a
> cron job.

Short answer: yes, but someone has to not just ask whether it's
possible or tell it ‘should be easy’, and actually implement all this.
Also, have a look at channel-with-substitutes-available.
It's not sufficiently general for your use case though.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Branch with only available substitutes ?

[Mathieu Othacehe]

Hello,

> Short answer: yes, but someone has to not just ask whether it's
> possible or tell it ‘should be easy’, and actually implement all this.
> Also, have a look at channel-with-substitutes-available.
> It's not sufficiently general for your use case though.

There's also a pending patch extending this procedure to support
manifests: https://issues.guix.gnu.org/47929.

It still needs some work but it could be conceptually close from what
Nicolas is asking.

Mathieu

Re: Branch with only available substitutes ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

Thanks all for your answer.

Of course Maxime, wasn't meant to be mean or anything.

I'm actually willing to help, although a bit limited in time in the
coming weeks.

What would be needed to help your pending patch @Mathieu ? Not sure if
my scheme is already good enough, but I can try and take a look.

Cheers,

Nicolas

Mathieu Othacehe <othacehe@gnu.org> writes:

> Hello,
>
>> Short answer: yes, but someone has to not just ask whether it's
>> possible or tell it ‘should be easy’, and actually implement all this.
>> Also, have a look at channel-with-substitutes-available.
>> It's not sufficiently general for your use case though.
>
> There's also a pending patch extending this procedure to support
> manifests: https://issues.guix.gnu.org/47929.
>
> It still needs some work but it could be conceptually close from what
> Nicolas is asking.
>
> Mathieu

Re: Branch with only available substitutes ?

[zimoun]

Hi,

In case you missed it:

<https://guix.gnu.org/en/manual/devel/en/guix.html#Channels-with-Substitutes>

which does not solve the issue but helps. :-)


On Sun, 9 Jan 2022 at 22:10, Nicolas Graves via Development of GNU
Guix and the GNU System distribution. <guix-devel@gnu.org> wrote:

> I find myself often waiting for heavy packages (ungoogled-chromium,
> cargo...) to be built as substitutes before updating my system and I
> find it a bit long to my personal taste.
>
> Would it be possible to add a branch in guix sources following only
> available substitutes, so that each time I pull, I can immediatly
> and quickly update. Or would that need to stay as someone's personal
> project ?

Adding what you are proposing would not change your "complaint". ;-)
Each time you would pull, you would stay on the same revision because
the substitutes would not be ready yet and so no quick update.  You
would have to wait and run again "guix pull" later, probably staying
on the same revision because the substitutes would not be ready.  In
the meantime, you would miss new features or new security fixes for
others of some packages you are using.

However, I agree that's annoying.  Some time ago (before the link
above), I used a script along these lines:

--8<---------------cut here---------------start------------->8---
#!/bin/bash

# url-cache-directory from guix/git.scm
# pjmkgl... = hash("https://git.savannah.gnu.org/git/guix.git")
CACHE=~/.cache/guix/checkouts
CHECKOUT=${CACHE}/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq

printf "Updating local checkout:\n'$CHECKOUT'..."
git -C $CHECKOUT fetch -q
echo " done."

guix pull --commit=$(git -C $CHECKOUT                                    \
                         log                                             \
                         --before=$(date --date='2 weeks ago' +%Y-%m-%d) \
                         --format="%h" | head -n1)                       \
     $@

echo "done."
exit 0
--8<---------------cut here---------------end--------------->8---

to lag by 2 week behind origin/master and so be almost sure to have
the substitutes -- if their build success-ed. ;-)

Well, yeah it's annoying to run "guix pull", check with "guix
weather", and depending on the result upgrade or not -- note that the
exit status cannot be used [1].  When all that could be automatized.
#47929, as Mathieu said, is grouping "guix pull" and "guix weather".
Maybe we could imagine an option to "guix upgrade" coupling "guix
weather"; somehow the converse of 'no-substitutes' or the converse of
'fallback', i.e., do not try to locally build if the substitutes is
not available.  I do not know.

1: <https://yhetil.org/guix/YOdhd7FfMOvKjTQe@jasmine.lan/>

Cheers,
simon

Re: Branch with only available substitutes ?

[Nicolas Graves via Development of GNU Guix and the GNU System distribution.]

Also thanks for your reply, I'll try it ;)


zimoun <zimon.toutoune@gmail.com> writes:

> Hi,
>
> In case you missed it:
>
> <https://guix.gnu.org/en/manual/devel/en/guix.html#Channels-with-Substitutes>
>
> which does not solve the issue but helps. :-)
>
>
> On Sun, 9 Jan 2022 at 22:10, Nicolas Graves via Development of GNU
> Guix and the GNU System distribution. <guix-devel@gnu.org> wrote:
>
>> I find myself often waiting for heavy packages (ungoogled-chromium,
>> cargo...) to be built as substitutes before updating my system and I
>> find it a bit long to my personal taste.
>>
>> Would it be possible to add a branch in guix sources following only
>> available substitutes, so that each time I pull, I can immediatly
>> and quickly update. Or would that need to stay as someone's personal
>> project ?
>
> Adding what you are proposing would not change your "complaint". ;-)
> Each time you would pull, you would stay on the same revision because
> the substitutes would not be ready yet and so no quick update.  You
> would have to wait and run again "guix pull" later, probably staying
> on the same revision because the substitutes would not be ready.  In
> the meantime, you would miss new features or new security fixes for
> others of some packages you are using.
>
> However, I agree that's annoying.  Some time ago (before the link
> above), I used a script along these lines:
>
> #!/bin/bash
>
> # url-cache-directory from guix/git.scm
> # pjmkgl... = hash("https://git.savannah.gnu.org/git/guix.git")
> CACHE=~/.cache/guix/checkouts
> CHECKOUT=${CACHE}/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq
>
> printf "Updating local checkout:\n'$CHECKOUT'..."
> git -C $CHECKOUT fetch -q
> echo " done."
>
> guix pull --commit=$(git -C $CHECKOUT                                    \
>                          log                                             \
>                          --before=$(date --date='2 weeks ago' +%Y-%m-%d) \
>                          --format="%h" | head -n1)                       \
>      $@
>
> echo "done."
> exit 0
>
> to lag by 2 week behind origin/master and so be almost sure to have
> the substitutes -- if their build success-ed. ;-)
>
> Well, yeah it's annoying to run "guix pull", check with "guix
> weather", and depending on the result upgrade or not -- note that the
> exit status cannot be used [1].  When all that could be automatized.
> #47929, as Mathieu said, is grouping "guix pull" and "guix weather".
> Maybe we could imagine an option to "guix upgrade" coupling "guix
> weather"; somehow the converse of 'no-substitutes' or the converse of
> 'fallback', i.e., do not try to locally build if the substitutes is
> not available.  I do not know.
>
> 1: <https://yhetil.org/guix/YOdhd7FfMOvKjTQe@jasmine.lan/>
>
> Cheers,
> simon