Guix mirrors

Guix mirrors

[Begley Brothers Inc]

Hi,
Over the last 24 hours I've experienced `guix pull` etc being
unavailable (HTTP 504's then 502's) more than available.

Is there a reason why a post receive hook can't be added to the guix
repo to push to github, gitlab, etc. and in that way at least give
users some protection against these outages?

There is a mirror[1] possibly (unofficial?) but it looks like it is
driven by some chron task.

The required post receive hook is well documented[2], and not
un-common amoung reputable OS projects:

- Android
- The Apache Software Foundation
- The Chromium Project
- The Eclipse Foundation
- The FreeBSD Project
- The Glasgow Haskell Compiler
- GNOME
- The Linux kernel source tree
- Qt

[1]: https://github.com/guix-mirror/guix
[2]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

-- 
Kind Regards

Begley Brothers Inc.

The content of this email is confidential and intended for the
recipient specified in message only. It is strictly forbidden to share
any part of this message with any third party, without a written
consent of the sender. If you received this message by mistake, please
reply to this message and follow with its deletion, so that we can
ensure such a mistake does not occur in the future.
This message has been sent as a part of discussion between Begley
Brothers Inc. and the addressee whose name is specified above. Should
you receive this message by mistake, we would be most grateful if you
informed us that the message has been sent to you. In this case, we
also ask that you delete this message from your mailbox, and do not
forward it or any part of it to anyone else. Thank you for your
cooperation and understanding.
Begley Brothers Inc. puts the security of the client at a high
priority. Therefore, we have put efforts into ensuring that the
message is error and virus-free. Unfortunately, full security of the
email cannot be ensured as, despite our efforts, the data included in
emails could be infected, intercepted, or corrupted. Therefore, the
recipient should check the email for threats with proper software, as
the sender does not accept liability for any damage inflicted by
viewing the content of this email.
The views and opinions included in this email belong to their author
and do not necessarily mirror the views and opinions of the company.
Our employees are obliged not to make any defamatory clauses,
infringe, or authorize infringement of any legal right. Therefore, the
company will not take any liability for such statements included in
emails. In case of any damages or other liabilities arising, employees
are fully responsible for the content of their emails.

Re: Guix mirrors

[Nikita Gillmann]

Begley Brothers Inc transcribed 2.6K bytes:
> Hi,
> Over the last 24 hours I've experienced `guix pull` etc being
> unavailable (HTTP 504's then 502's) more than available.
> 
> Is there a reason why a post receive hook can't be added to the guix
> repo to push to github, gitlab, etc. and in that way at least give
> users some protection against these outages?

I suspect this is mostly project guidelines which disocurage this (see
GNU Free Systems Distribution Guidelines). I don't know if I'm correct
about this, so one of the project maintainers will likely comment.

> There is a mirror[1] possibly (unofficial?) but it looks like it is
> driven by some chron task.
> 
> The required post receive hook is well documented[2], and not
> un-common amoung reputable OS projects:
> 
> - Android
> - The Apache Software Foundation
> - The Chromium Project
> - The Eclipse Foundation
> - The FreeBSD Project
> - The Glasgow Haskell Compiler
> - GNOME
> - The Linux kernel source tree
> - Qt
> 
> [1]: https://github.com/guix-mirror/guix
> [2]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
> 
> -- 
> Kind Regards
> 
> Begley Brothers Inc.
> 
> The content of this email is confidential and intended for the
> recipient specified in message only. It is strictly forbidden to share
> any part of this message with any third party, without a written
> consent of the sender. If you received this message by mistake, please
> reply to this message and follow with its deletion, so that we can
> ensure such a mistake does not occur in the future.
> This message has been sent as a part of discussion between Begley
> Brothers Inc. and the addressee whose name is specified above. Should
> you receive this message by mistake, we would be most grateful if you
> informed us that the message has been sent to you. In this case, we
> also ask that you delete this message from your mailbox, and do not
> forward it or any part of it to anyone else. Thank you for your
> cooperation and understanding.
> Begley Brothers Inc. puts the security of the client at a high
> priority. Therefore, we have put efforts into ensuring that the
> message is error and virus-free. Unfortunately, full security of the
> email cannot be ensured as, despite our efforts, the data included in
> emails could be infected, intercepted, or corrupted. Therefore, the
> recipient should check the email for threats with proper software, as
> the sender does not accept liability for any damage inflicted by
> viewing the content of this email.
> The views and opinions included in this email belong to their author
> and do not necessarily mirror the views and opinions of the company.
> Our employees are obliged not to make any defamatory clauses,
> infringe, or authorize infringement of any legal right. Therefore, the
> company will not take any liability for such statements included in
> emails. In case of any damages or other liabilities arising, employees
> are fully responsible for the content of their emails.
> 

Re: Guix mirrors

[Ricardo Wurmus]

Begley Brothers Inc <begleybrothers@gmail.com> writes:

> Over the last 24 hours I've experienced `guix pull` etc being
> unavailable (HTTP 504's then 502's) more than available.

Yeah, it seems that there have been some problems with Savannah.  We did
host a mirror on ci.guix.gnu.org, which is hosted independtly of GNU
project infrastructure, but it was never actually enabled or advertised.

I expect that this will change soon after some more discussion of the
details.

-- 
Ricardo

Re: Guix mirrors

[zimoun]

Hi Ricardo,

On Wed, 20 May 2020 at 13:02, Ricardo Wurmus <rekado@elephly.net> wrote:

> I expect that this will change soon after some more discussion of the
> details.

Last time we discussed that was on December [1].  What is missing is: discuss if

  a) an "official" mirror is hard-coded in '%default-channels'
or
  b) an "official" mirror is documented in the  manual and it is up to
the user to setup it via ~/.config/guix/channels.scm.

The a) annoys people who do not want the mirror, so they have to tweak
the list by themself and in the same a) is more newcomer-friendly
because it just works.
The b) annoys newcomers because it is another step and in the same
time b) is more customizable.

Obviously, there is option c): guix-install.sh asks a question and
then do the b) dance.


And discuss if a channel is described as:

 (channel
   (name 'guix)
   (url "https://savannah…")
   (mirror (list "https://mirror-one…"
                    "https://mirror-two…")))

[ where mirror is optional ]

or something else?


Well, Ludo suggested [2] to address "Trustable guix pull" [3]; which
should be really cool!
Now, I am not sure to understand if there is no mirror mechanism
because no one took the time to implement it or if any other reasons
related to "trustable guix pull".


[1] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00150.html
[2] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00276.html
[3] http://issues.guix.gnu.org/issue/22883


Cheers,
simon

Re: Guix mirrors

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

> On Wed, 20 May 2020 at 13:02, Ricardo Wurmus <rekado@elephly.net> wrote:
>
>> I expect that this will change soon after some more discussion of the
>> details.
>
> Last time we discussed that was on December [1].

There’s a very recent discussion among the Guix sysadmins and I hope it
leads to results soon.

-- 
Ricardo

Re: Guix mirrors

[zimoun]

On Wed, 20 May 2020 at 14:15, Ricardo Wurmus <rekado@elephly.net> wrote:

> There’s a very recent discussion among the Guix sysadmins and I hope it
> leads to results soon.

Thank you for reporting.
Since it is a private mailing list, I cannot be aware.

BTW, why such discussion happens on Guix sysadmins mailing list and
not on guix-devel?

Re: Guix mirrors

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 253 bytes --]

zimoun 写道：
> BTW, why such discussion happens on Guix sysadmins mailing list 
> and
> not on guix-devel?

Because that's where I sent a quick & dirty patch yonks ago. 
There's also really not that much to discuss.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Guix mirrors

[Begley Brothers Inc]

Hi,
Thanks for taking the time to respond.  Sounds like good news overall....

On Wed, May 20, 2020 at 6:57 AM zimoun <zimon.toutoune@gmail.com> wrote:
>
> Hi Ricardo,
>
> On Wed, 20 May 2020 at 13:02, Ricardo Wurmus <rekado@elephly.net> wrote:
>
> > I expect that this will change soon after some more discussion of the
> > details.
>
> Last time we discussed that was on December [1].  What is missing is: discuss if
>
>   a) an "official" mirror is hard-coded in '%default-channels'
> or
>   b) an "official" mirror is documented in the  manual and it is up to
> the user to setup it via ~/.config/guix/channels.scm.
>
> The a) annoys people who do not want the mirror, so they have to tweak
> the list by themself and in the same a) is more newcomer-friendly
> because it just works.
> The b) annoys newcomers because it is another step and in the same
> time b) is more customizable.
>
> Obviously, there is option c): guix-install.sh asks a question and
> then do the b) dance.
>
>
> And discuss if a channel is described as:
>
>  (channel
>    (name 'guix)
>    (url "https://savannah…")
>    (mirror (list "https://mirror-one…"
>                     "https://mirror-two…")))
>
> [ where mirror is optional ]
>
> or something else?

There should be a built in (hard-coded) failover list - ideally three
- without the user being told or asked to do anything during setup.
Ideally the user could alter that list - but that seems a stage 2
feature given the curent state.
Stage 3 might alllow the default list to vary by country code - driven
by and env var set to two letter ISO code?

I was surprised guix doesn't seem to be mirrored to gnu.io - or is
that the same machine(s)?

On reflection in this context 'mirror' is not descriptive in the
channel config - 'failover' might better communicate the intent and
context

  (channel
    (name 'guix)
    (url "https://savannah…")
    (failover (list (url "https://mirror-one…")
                       (url "https://mirror-two…"))))

where **overriding** the failover list is optional, but you cannot
unset the default list

HTH

> Well, Ludo suggested [2] to address "Trustable guix pull" [3]; which
> should be really cool!
> Now, I am not sure to understand if there is no mirror mechanism
> because no one took the time to implement it or if any other reasons
> related to "trustable guix pull".
>
>
> [1] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00150.html
> [2] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00276.html
> [3] http://issues.guix.gnu.org/issue/22883
>
>
> Cheers,
> simon



-- 
Kind Regards

Begley Brothers Inc.

The content of this email is confidential and intended for the
recipient specified in message only. It is strictly forbidden to share
any part of this message with any third party, without a written
consent of the sender. If you received this message by mistake, please
reply to this message and follow with its deletion, so that we can
ensure such a mistake does not occur in the future.
This message has been sent as a part of discussion between Begley
Brothers Inc. and the addressee whose name is specified above. Should
you receive this message by mistake, we would be most grateful if you
informed us that the message has been sent to you. In this case, we
also ask that you delete this message from your mailbox, and do not
forward it or any part of it to anyone else. Thank you for your
cooperation and understanding.
Begley Brothers Inc. puts the security of the client at a high
priority. Therefore, we have put efforts into ensuring that the
message is error and virus-free. Unfortunately, full security of the
email cannot be ensured as, despite our efforts, the data included in
emails could be infected, intercepted, or corrupted. Therefore, the
recipient should check the email for threats with proper software, as
the sender does not accept liability for any damage inflicted by
viewing the content of this email.
The views and opinions included in this email belong to their author
and do not necessarily mirror the views and opinions of the company.
Our employees are obliged not to make any defamatory clauses,
infringe, or authorize infringement of any legal right. Therefore, the
company will not take any liability for such statements included in
emails. In case of any damages or other liabilities arising, employees
are fully responsible for the content of their emails.

Re: Guix mirrors

[zimoun]

Hi Tobias,

On Wed, 20 May 2020 at 14:53, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> Because that's where I sent a quick & dirty patch yonks ago.

Cool! A patch!


> There's also really not that much to discuss.

As we quickly talked about it on Monday on IRC, I have the feeling
that some details are still vague -- I have tried to summarize above
in this thread.

Even if Savannah is down each syzygy, it is always frustrating when it happens.


Cheers,
simon

Re: Guix mirrors

[zimoun]

Dear,

On Wed, 20 May 2020 at 15:18, Begley Brothers Inc
<begleybrothers@gmail.com> wrote:

> I was surprised guix doesn't seem to be mirrored to gnu.io - or is
> that the same machine(s)?

What do you mean?
The site gnu.io is about GNU FM and GNU Social as the "About" explains.

BTW, I do not know what is the link between gnu.io and gnu.org; other
said, is gnu.io officially supported by gnu.org?


All the best,
simon

Re: Guix mirrors

[Begley Brothers Inc]

Thanks to @kozodev on gitlab.com, a possible workaround, available
immediately, is to document the following in the web pages and alt-F2
help as appropriate:

3.6.2 Proceeding with the Installation

8.1 Using the Configuration System

with (note the inital root init channel path needs to be inserted)

```bash
...
(use-package-modules screen)

;; Your `guix system init /etc/conf.scm` or
;; susequent `guix pull` and `guix system reconfigure` may
;; fail because the Guix server is unavailable - you will see
;; HTTPS errors 404, 504, 502.
;; In this situtaion, please use one of:
;;   - "https://mirror1.com/x/y/z/guix.git"
;;   - "https://mirror2.com/a/guix.git"
;;   - "https://mirror2.com/d/e/guix.git"
;; in the `with-output-to-file` code below. Which you should uncomment,
;; then re-run:
;;
;; # guix system init /etc/conf.scm
;;
;; NOTE: Air-Gapped Facility Users.
;; If you are using guix in an air-gapped facility you will
;; need to add your Guix repository and uncomment this code
;; *before* running:
;;
;; # guix system init /etc/conf.scm
;;
;; There is curently no other way to repoint the init phase
;; to your air-gapped repository.

;;(with-output-to-file "/path/to/roots/first/init/channels.scm"
;;  (lambda ()
;;    (display "(cons* (channel (name 'guix) (url
\"https://internal.net/x/y/z/guix.git\")) %default-channels)")))

(operating-system
...
```

HTH?


On Tue, May 19, 2020 at 2:32 AM Begley Brothers Inc
<begleybrothers@gmail.com> wrote:
>
> Hi,
> Over the last 24 hours I've experienced `guix pull` etc being
> unavailable (HTTP 504's then 502's) more than available.
>
> Is there a reason why a post receive hook can't be added to the guix
> repo to push to github, gitlab, etc. and in that way at least give
> users some protection against these outages?
>
> There is a mirror[1] possibly (unofficial?) but it looks like it is
> driven by some chron task.
>
> The required post receive hook is well documented[2], and not
> un-common amoung reputable OS projects:
>
> - Android
> - The Apache Software Foundation
> - The Chromium Project
> - The Eclipse Foundation
> - The FreeBSD Project
> - The Glasgow Haskell Compiler
> - GNOME
> - The Linux kernel source tree
> - Qt
>
> [1]: https://github.com/guix-mirror/guix
> [2]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
>
> --
> Kind Regards
>
> Begley Brothers Inc.
>
> The content of this email is confidential and intended for the
> recipient specified in message only. It is strictly forbidden to share
> any part of this message with any third party, without a written
> consent of the sender. If you received this message by mistake, please
> reply to this message and follow with its deletion, so that we can
> ensure such a mistake does not occur in the future.
> This message has been sent as a part of discussion between Begley
> Brothers Inc. and the addressee whose name is specified above. Should
> you receive this message by mistake, we would be most grateful if you
> informed us that the message has been sent to you. In this case, we
> also ask that you delete this message from your mailbox, and do not
> forward it or any part of it to anyone else. Thank you for your
> cooperation and understanding.
> Begley Brothers Inc. puts the security of the client at a high
> priority. Therefore, we have put efforts into ensuring that the
> message is error and virus-free. Unfortunately, full security of the
> email cannot be ensured as, despite our efforts, the data included in
> emails could be infected, intercepted, or corrupted. Therefore, the
> recipient should check the email for threats with proper software, as
> the sender does not accept liability for any damage inflicted by
> viewing the content of this email.
> The views and opinions included in this email belong to their author
> and do not necessarily mirror the views and opinions of the company.
> Our employees are obliged not to make any defamatory clauses,
> infringe, or authorize infringement of any legal right. Therefore, the
> company will not take any liability for such statements included in
> emails. In case of any damages or other liabilities arising, employees
> are fully responsible for the content of their emails.



--
Kind Regards

Begley Brothers Inc.

The content of this email is confidential and intended for the
recipient specified in message only. It is strictly forbidden to share
any part of this message with any third party, without a written
consent of the sender. If you received this message by mistake, please
reply to this message and follow with its deletion, so that we can
ensure such a mistake does not occur in the future.
This message has been sent as a part of discussion between Begley
Brothers Inc. and the addressee whose name is specified above. Should
you receive this message by mistake, we would be most grateful if you
informed us that the message has been sent to you. In this case, we
also ask that you delete this message from your mailbox, and do not
forward it or any part of it to anyone else. Thank you for your
cooperation and understanding.
Begley Brothers Inc. puts the security of the client at a high
priority. Therefore, we have put efforts into ensuring that the
message is error and virus-free. Unfortunately, full security of the
email cannot be ensured as, despite our efforts, the data included in
emails could be infected, intercepted, or corrupted. Therefore, the
recipient should check the email for threats with proper software, as
the sender does not accept liability for any damage inflicted by
viewing the content of this email.
The views and opinions included in this email belong to their author
and do not necessarily mirror the views and opinions of the company.
Our employees are obliged not to make any defamatory clauses,
infringe, or authorize infringement of any legal right. Therefore, the
company will not take any liability for such statements included in
emails. In case of any damages or other liabilities arising, employees
are fully responsible for the content of their emails.

Re: Guix mirrors

[Begley Brothers Inc]

On Wed, May 20, 2020 at 7:45 PM Begley Brothers Inc
<begleybrothers@gmail.com> wrote:
>
> Thanks to @kozodev on gitlab.com, a possible workaround, available
> immediately, is to document the following in the web pages and alt-F2
> help as appropriate:
>
> 3.6.2 Proceeding with the Installation
>
> 8.1 Using the Configuration System
>
> with (note the inital root init channel path needs to be inserted)
>
> ```bash
> ...
> (use-package-modules screen)
>
> ;; Your `guix system init /etc/conf.scm` or
> ;; susequent `guix pull` and `guix system reconfigure` may
> ;; fail because the Guix server is unavailable - you will see
> ;; HTTPS errors 404, 504, 502.
> ;; In this situtaion, please use one of:
> ;;   - "https://mirror1.com/x/y/z/guix.git"
> ;;   - "https://mirror2.com/a/guix.git"
> ;;   - "https://mirror2.com/d/e/guix.git"
> ;; in the `with-output-to-file` code below. Which you should uncomment,
> ;; then re-run:
> ;;
> ;; # guix system init /etc/conf.scm
> ;;
> ;; NOTE: Air-Gapped Facility Users.
> ;; If you are using guix in an air-gapped facility you will
> ;; need to add your Guix repository and uncomment this code
> ;; *before* running:
> ;;
> ;; # guix system init /etc/conf.scm
> ;;
> ;; There is curently no other way to repoint the init phase
> ;; to your air-gapped repository.
>
> ;;(with-output-to-file "/path/to/roots/first/init/channels.scm"
> ;;  (lambda ()
> ;;    (display "(cons* (channel (name 'guix) (url
> \"https://internal.net/x/y/z/guix.git\")) %default-channels)")))
>
> (operating-system
> ...
> ```
>
> HTH?
>
>
> On Tue, May 19, 2020 at 2:32 AM Begley Brothers Inc
> <begleybrothers@gmail.com> wrote:
> >
> > Hi,
> > Over the last 24 hours I've experienced `guix pull` etc being
> > unavailable (HTTP 504's then 502's) more than available.
> >
> > Is there a reason why a post receive hook can't be added to the guix
> > repo to push to github, gitlab, etc. and in that way at least give
> > users some protection against these outages?
> >
> > There is a mirror[1] possibly (unofficial?) but it looks like it is
> > driven by some chron task.
> >
> > The required post receive hook is well documented[2], and not
> > un-common amoung reputable OS projects:
> >
> > - Android
> > - The Apache Software Foundation
> > - The Chromium Project
> > - The Eclipse Foundation
> > - The FreeBSD Project
> > - The Glasgow Haskell Compiler
> > - GNOME
> > - The Linux kernel source tree
> > - Qt
> >
> > [1]: https://github.com/guix-mirror/guix
> > [2]: https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks
> >
> > --
> > Kind Regards
> >
> > Begley Brothers Inc.

PS

I'm not sure if you want to add this to the docs, but there does not
seem to be dedicated functionality to securely deal with secret data
in Guix.
One way to work a around that is to host your own package, and add
that to the channels before you run (as root):

```bash
guix system init /etc/conf.csm
```

These two issues (configurable repo and secret management) are what
I've encountered that blocked "reproducible operating systems" being
true.
Since that is a claim made on the fornt page (https://guix.gnu.org/) I
think adding something like the prior suggestion to the docs mean you
can plausibly claim the statement is not misleading.
Otherwise maybe change the statement to "partially reproducible
systems" and at the "declarative system configuration" link to section
8.1 Using the Configuration System, just add two bullet points that
still in devlopment are 1) configurable init repositories 2) secret
management.

Its debatable whether orchestration is a necessary function for the
"reproducible operating systems" claim to be considered (reasonably)
true. Hashicorp/Terraform and  Packet/Tinkerbell would disagree.
I should note I haven't mentioned the FSF/GNU/Guix sponor and their
orchestration product mainly because it is Apache Airflow adapted to a
cloud use case. Apache Airflow themselves say they are more comparable
to Oozie and Azkaban, so you have to do a lot of heavy lifting to get
your sponsor's product - as best I know there is not OSS project
upstream to your sponsors product.  Happy to stand corrected.

-- 
Kind Regards

Begley Brothers Inc.

The content of this email is confidential and intended for the
recipient specified in message only. It is strictly forbidden to share
any part of this message with any third party, without a written
consent of the sender. If you received this message by mistake, please
reply to this message and follow with its deletion, so that we can
ensure such a mistake does not occur in the future.
This message has been sent as a part of discussion between Begley
Brothers Inc. and the addressee whose name is specified above. Should
you receive this message by mistake, we would be most grateful if you
informed us that the message has been sent to you. In this case, we
also ask that you delete this message from your mailbox, and do not
forward it or any part of it to anyone else. Thank you for your
cooperation and understanding.
Begley Brothers Inc. puts the security of the client at a high
priority. Therefore, we have put efforts into ensuring that the
message is error and virus-free. Unfortunately, full security of the
email cannot be ensured as, despite our efforts, the data included in
emails could be infected, intercepted, or corrupted. Therefore, the
recipient should check the email for threats with proper software, as
the sender does not accept liability for any damage inflicted by
viewing the content of this email.
The views and opinions included in this email belong to their author
and do not necessarily mirror the views and opinions of the company.
Our employees are obliged not to make any defamatory clauses,
infringe, or authorize infringement of any legal right. Therefore, the
company will not take any liability for such statements included in
emails. In case of any damages or other liabilities arising, employees
are fully responsible for the content of their emails.

Re: Guix mirrors

[zimoun]

Dear Tobias,

On Wed, 20 May 2020 at 14:53, Tobias Geerinckx-Rice <me@tobias.gr> wrote:
> zimoun 写道：

> > BTW, why such discussion happens on Guix sysadmins mailing list
> > and
> > not on guix-devel?
>
> Because that's where I sent a quick & dirty patch yonks ago.
> There's also really not that much to discuss.

What is the status of this "quick & dirty" patch?  Could you send it
on guix-devel or guix-patches?
Other said, what is the current status of "Guix mirror"?

All the best,
simon