bug#57559: guix pull fails on http status code 503

bug#57559: guix pull fails on http status code 503

[André A. Gomes]

Hi Guix,

--8<---------------cut here---------------start------------->8---
$ guix pull
Updating channel 'nonguix' from Git repository at 'https://gitlab.com/nonguix/nonguix'...
guix pull: error: Git error: unexpected http status code: 503
--8<---------------cut here---------------end--------------->8---

It seems reasonable that if a certain channel isn't available, it should
be possible to ignore it and proceed.

Thanks.


-- 
André A. Gomes
"You cannot even find the ruins..."

bug#57559: guix pull fails on http status code 503

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1452 bytes --]

On 03-09-2022 13:21, André A. Gomes wrote:

> Hi Guix,
>
> --8<---------------cut here---------------start------------->8---
> $ guix pull
> Updating channel 'nonguix' from Git repository at 'https://gitlab.com/nonguix/nonguix'...
> guix pull: error: Git error: unexpected http status code: 503
> --8<---------------cut here---------------end--------------->8---
>
> It seems reasonable that if a certain channel isn't available, it should
> be possible to ignore it and proceed.
>
> Thanks.

More concretely:

  * if all channels are available --> ok, keep status quo
  * if no channels are available --> error, keep status quo
  * if some but not all channels are available, and there is at least
    one updated channel --> log the missing channels, and update the
    channels that _are_ available (but don't 'ignore' the missing
    channels by removing them!).
  * if some but not all channels are available, and none of the
    available channels were updated --> bail out, no progress was made
    and this might be due to lack of connectivity instead of lack of
    updates.

?

Technically, your message can be interpreted as: if I only have a single 
channel, the Guix channel, and it isn't available, it should be possible 
to ignore it and proceed. (Proceed with what? It's the only channel, 
what is there to proceed with.)  To actually implement this, we need to 
be precise.

Greetings,
Maxime


[-- Attachment #1.1.1.2: Type: text/html, Size: 2024 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57559: guix pull fails on http status code 503

[André A. Gomes]

Maxime Devos <maximedevos@telenet.be> writes:

> More concretely:
>
> * if all channels are available --> ok, keep status quo
> * if no channels are available --> error, keep status quo
> * if some but not all channels are available, and there is at least one
>   updated channel --> log the missing channels, and update the channels
>   that _are_ available (but don't 'ignore' the missing channels by
>   removing them!).
> * if some but not all channels are available, and none of the available
>   channels were updated --> bail out, no progress was made and this
>   might be due to lack of connectivity instead of lack of updates.
>
> ?
>
> Technically, your message can be interpreted as: if I only have a single
> channel, the Guix channel, and it isn't available, it should be possible
> to ignore it and proceed. (Proceed with what? It's the only channel,
> what is there to proceed with.)  To actually implement this, we need to
> be precise.

I'm just a user and unfortunately I don't have a deep understanding of
Guix's internals.

Your concrete description seems to resonate with my expectations.

Thanks.

-- 
André A. Gomes
"You cannot even find the ruins..."

bug#57559: guix pull fails on http status code 503

[zimoun]

Hi Maxime,

On sam., 03 sept. 2022 at 19:27, Maxime Devos <maximedevos@telenet.be> wrote:

>   * if some but not all channels are available, and there is at least
>     one updated channel --> log the missing channels, and update the
>     channels that _are_ available (but don't 'ignore' the missing
>     channels by removing them!).

What do you mean by «but don't 'ignore' the missing channels by removing
them!»?  Do you mean keep the last revision locally known for this
channel?


Cheers,
simon

bug#57559: guix pull fails on http status code 503

[Maxime Devos]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1725 bytes --]


On 05-09-2022 10:21, zimoun wrote:
> On sam., 03 sept. 2022 at 19:27, Maxime Devos<maximedevos@telenet.be>  wrote:
>
>>    * if some but not all channels are available, and there is at least
>>      one updated channel --> log the missing channels, and update the
>>      channels that_are_  available (but don't 'ignore' the missing
>>      channels by removing them!).
> What do you mean by «but don't 'ignore' the missing channels by removing
> them!»?  Do you mean keep the last revision locally known for this
> channel?

I'm not sure what you're asking, because of the negatives.

With "but don't ignore [...] by [...]", I meant that, yes.

If you meant "don't ignore [...] by [...]", then no, with 'ignoring by 
removing', I meant, literally removing them.  More concretely, a 
situation like this:

  * In the channels.scm, two channels are declared: guix and guix-foo.
  * User does "guix pull"; Guix downloads the source code of guix and
    guix-foo.
  * Downloading guix failed (503).  As such, Guix decided to ignore the
    guix channel, by removing it from the list of channels to build.
    (The in-memory list I mean, I don't mean modifying the channels.smc
    file)
  * Guix tries building the guix-foo channel and installing it, without
    the guix channel.
  * This cannot work (the guix-foo channel uses modules from guix for
    basic stuff like G-exps, packages, coreutils, ..., and it was the
    removed guix channel that had things build-aux/build-self.scm which
    is required for pulling).

(It's not unlike double negatives! I first thought of the latter 
interpretation but on second thought you might have meant the former.)

Greetings,
Maxime.


[-- Attachment #1.1.1.2: Type: text/html, Size: 2667 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57559: guix pull fails on http status code 503

[zimoun]

Hi Maxime,

On Mon, 05 Sep 2022 at 22:20, Maxime Devos <maximedevos@telenet.be> wrote:
> On 05-09-2022 10:21, zimoun wrote:
>> On sam., 03 sept. 2022 at 19:27, Maxime Devos<maximedevos@telenet.be>  wrote:
>>
>>>    * if some but not all channels are available, and there is at least
>>>      one updated channel --> log the missing channels, and update the
>>>      channels that_are_  available (but don't 'ignore' the missing
>>>      channels by removing them!).
>> What do you mean by «but don't 'ignore' the missing channels by removing
>> them!»?  Do you mean keep the last revision locally known for this
>> channel?
>
> I'm not sure what you're asking, because of the negatives.

[...]

> (It's not unlike double negatives! I first thought of the latter 
> interpretation but on second thought you might have meant the former.)

IIUC, you have hard time to parse the double-negative you wrote earlier
in this thread. :-)



> With "but don't ignore [...] by [...]", I meant that, yes.
>
> If you meant "don't ignore [...] by [...]", then no, with 'ignoring by 
> removing', I meant, literally removing them.  More concretely, a 
> situation like this:
>
>   * In the channels.scm, two channels are declared: guix and guix-foo.
>   * User does "guix pull"; Guix downloads the source code of guix and
>     guix-foo.
>   * Downloading guix failed (503).  As such, Guix decided to ignore the
>     guix channel, by removing it from the list of channels to build.
>     (The in-memory list I mean, I don't mean modifying the channels.smc
>     file)
>   * Guix tries building the guix-foo channel and installing it, without
>     the guix channel.
>   * This cannot work (the guix-foo channel uses modules from guix for
>     basic stuff like G-exps, packages, coreutils, ..., and it was the
>     removed guix channel that had things build-aux/build-self.scm which
>     is required for pulling).

The initial report by André is about:

--8<---------------cut here---------------start------------->8---
$ guix pull
Updating channel 'nonguix' from Git repository at 'https://gitlab.com/nonguix/nonguix'...
guix pull: error: Git error: unexpected http status code: 503
--8<---------------cut here---------------end--------------->8---

So the situation is more likely: Guix from Savannah is reachable and
guix-foo is not.

For sure, we can discuss the case when Guix is unavailable.  However,
the 'guix channel is special; see all conditionals using ’guix-channel?’.  

Well, %default-guix-channel is somehow a corner use-case when the
regular use-case seems about a failure of additional channels.


Cheers,
simon

bug#57559: guix pull fails on http status code 503

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1058 bytes --]


On 06-09-2022 15:09, zimoun wrote:
> The initial report by André is about:
>
> --8<---------------cut here---------------start------------->8---
> $ guix pull
> Updating channel 'nonguix' from Git repository at 'https://gitlab.com/nonguix/nonguix'...
> guix pull: error: Git error: unexpected http status code: 503
> --8<---------------cut here---------------end--------------->8---
>
> So the situation is more likely: Guix from Savannah is reachable and
> guix-foo is not.
>
> For sure, we can discuss the case when Guix is unavailable.  However,
> the 'guix channel is special; see all conditionals using ’guix-channel?’.
>
> Well, %default-guix-channel is somehow a corner use-case when the
> regular use-case seems about a failure of additional channels.

The ‘Guix is unavailable’ was just a simple example. For the example, 
you can replace it, with, say nonguix. Automatically removing the 
additional channels upon 503 is undesirable I'd think, though the error 
messages will be different.

Greetings,
Maixme.


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57559: guix pull fails on http status code 503

[zimoun]

Hi Maxime,

On Thu, 08 Sep 2022 at 02:55, Maxime Devos <maximedevos@telenet.be> wrote:

>> Well, %default-guix-channel is somehow a corner use-case when the
>> regular use-case seems about a failure of additional channels.
>
> The ‘Guix is unavailable’ was just a simple example. For the example, 
> you can replace it, with, say nonguix. Automatically removing the 
> additional channels upon 503 is undesirable I'd think, though the error 
> messages will be different.

My point is that ’Guix is unavailable’ is special and it appears to me
expected to raise an error.  Other said,

 1. if %default-guix-channel (or its mirrors) is unreachable,
    then stop the pull; as it is currently done.
 2. if an extra channel is unreachable,
    then keep the last local revision of this channel
    and try the others.

For sure, #2 can break the whole pull if something in the Guix channel
changed *and* the channel depends on this something.  But we cannot do
more than raise an error.


Cheers,
simon

bug#57559: guix pull fails on http status code 503

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 1061 bytes --]


On 08-09-2022 09:25, zimoun wrote:
> My point is that ’Guix is unavailable’ is special and it appears to me
> expected to raise an error.  Other said,
>
>   1. if %default-guix-channel (or its mirrors) is unreachable,
>      then stop the pull; as it is currently done.
>   2. if an extra channel is unreachable,
>      then keep the last local revision of this channel
>      and try the others.
>
> For sure, #2 can break the whole pull if something in the Guix channel
> changed*and*  the channel depends on this something.  But we cannot do
> more than raise an error.
Looking at 'latest-channel-instances', the code for deciding what the
latest version is, is the same for the guix channel and any other
channel.  As such, the 'Guix is unavailable' is, currently, not special.

I suppose we could special-case it in latest-channel-instances to raise
an error when Guix is unavailable, but I don't see why -- sometimes
Savannah is down, would be nice to then at least be able to update
the other channels.

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]

bug#57559: guix pull fails on http status code 503

[zimoun]

Hi Maxime,

On Thu, 08 Sep 2022 at 11:23, Maxime Devos <maximedevos@telenet.be> wrote:

> Looking at 'latest-channel-instances', the code for deciding what the
> latest version is, is the same for the guix channel and any other
> channel.  As such, the 'Guix is unavailable' is, currently, not special.

Well, ’latest-channel-instances’ relies on ’latest-channel-instance’
which tests ’guix-channel?’,

--8<---------------cut here---------------start------------->8---
(define (guix-channel? channel)
  "Return true if CHANNEL is the 'guix' channel."
  (eq? 'guix (channel-name channel)))
--8<---------------cut here---------------end--------------->8---

then, for instance, the occurrences are:

--8<---------------cut here---------------start------------->8---
12 candidates:
./guix/channels.scm:79:            guix-channel?
./guix/channels.scm:192:(define (guix-channel? channel)
./guix/channels.scm:199:  (if (and (guix-channel? chan)
./guix/channels.scm:424:            (when (guix-channel? channel)
./guix/channels.scm:437:    (when (guix-channel? channel)
./guix/channels.scm:778:            (guix-channel? (channel-instance-channel instance)))
./guix/channels.scm:903:        (item (if (guix-channel? channel)
./guix/scripts/pull.scm:775:        (match (find guix-channel? channels)
./guix/scripts/pull.scm:788:                   (remove guix-channel? channels))))
./gnu/packages/package-management.scm:125:                                guix-channel?
./gnu/packages/package-management.scm:618:   (delay (match (or (find guix-channel? (current-channels))
./gnu/ci.scm:478:    (find guix-channel? channels))
--8<---------------cut here---------------end--------------->8---

And my point is that the Guix channel is special when pulling.
Therefore, we could split the cases when the Guix channel is unreachable
and when other extra-channels are unreachable; because the impact are
different, IMHO.

As I pointed earlier in the thread, the main issue is when these extra
channels are unreachable; as André have initially reported.


Cheers,
simon

bug#57559: guix pull fails on http status code 503

[Maxime Devos]

[-- Attachment #1.1.1: Type: text/plain, Size: 3415 bytes --]


On 08-09-2022 13:11, zimoun wrote:
> Hi Maxime,
>
> On Thu, 08 Sep 2022 at 11:23, Maxime Devos <maximedevos@telenet.be> wrote:
>
>> Looking at 'latest-channel-instances', the code for deciding what the
>> latest version is, is the same for the guix channel and any other
>> channel.  As such, the 'Guix is unavailable' is, currently, not special.
> Well, ’latest-channel-instances’ relies on ’latest-channel-instance’
> which tests ’guix-channel?’,
>
> --8<---------------cut here---------------start------------->8---
> (define (guix-channel? channel)
>    "Return true if CHANNEL is the 'guix' channel."
>    (eq? 'guix (channel-name channel)))
> --8<---------------cut here---------------end--------------->8---
That's only for channel authentication and patches, not relevant for 
this bug#57559.
> then, for instance, the occurrences are:
>
> --8<---------------cut here---------------start------------->8---
> 12 candidates:
> ./guix/channels.scm:79:            guix-channel?
> ./guix/channels.scm:192:(define (guix-channel? channel)
> ./guix/channels.scm:199:  (if (and (guix-channel? chan)
> ./guix/channels.scm:424:            (when (guix-channel? channel)
> ./guix/channels.scm:437:    (when (guix-channel? channel)
> ./guix/channels.scm:778:            (guix-channel? (channel-instance-channel instance)))
> ./guix/channels.scm:903:        (item (if (guix-channel? channel)
> ./guix/scripts/pull.scm:775:        (match (find guix-channel? channels)
> ./guix/scripts/pull.scm:788:                   (remove guix-channel? channels))))
> ./gnu/packages/package-management.scm:125:                                guix-channel?
> ./gnu/packages/package-management.scm:618:   (delay (match (or (find guix-channel? (current-channels))
> ./gnu/ci.scm:478:    (find guix-channel? channels))
> --8<---------------cut here---------------end--------------->8---
>
> And my point is that the Guix channel is special when pulling.
It is special for some things, yes, but it isn't special for deciding
what is the latest version. These occurrences are for channel 
authentication,
patching, building, Cuirass and the 'guix' package, not for deciding
what is to be considered the latest available version of the channel
> Therefore, we could split the cases when the Guix channel is unreachable
> and when other extra-channels are unreachable; because the impact are
> different, IMHO.
In principle, we could, but I don't see the point. Solving things for 
the case
where the unavailable channel is Guix is no more complicated than for other
channels.
> As I pointed earlier in the thread, the main issue is when these extra
> channels are unreachable; as André have initially reported.
This might be the main issue, yes, but I still don't see the point of 
treating
'guix' specially for the puposes of #57559.  It would make the semantics of
"guix pull" and its implementation more complex, for no apparent purpose
AFAICT (solving the general issue would solve the main issue, right?).

About the complexity of the implementation: as mentioned previously,
latest-channel-instances doesn't treat the Guix channel specially
(the authentication check and patches isn't relevant here), so adding an
additional (if (guix-channel?) ...) would add complexity (and hence, require
additional testing and maybe documentation).

Greetings,
Maxime.

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 929 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 236 bytes --]