Guix mirror: sourceware discussion report

Guix mirror: sourceware discussion report

[zimoun]

Dear,

Currently "guix pull" from Savannah and issues can arise. As we
recently experimented. Tobias and Ricardo recently discussed how to
mirror the repo. IMHO, it is a good idea to mirror but not a good idea
to locate it on Ricardo infrastructure, again. :-)

Well, I propose to see if we can mirror on sourceware.org which are
GNU friends. ;-)

I took the liberty to contact them on IRC. See below.

What I see is: we still "guix pull" from Savannah as usual.
Then if Savannah is down, we catch the error and we try (transparently
for the user) the sourceware url.

As a first contact, they agree on the principle. Even they ask numbers
about traffic etc. :-)


What do you think?


All the best,
simon

--





[Fri Dec 13 2019]
<zimoun> I am working with Guix and we discussed mirroring the GNU Guix repo
         on sourceware.  [17:56]
<zimoun> The best before doing plan is to ask if it should be possible, on the
         principle.  [17:57]
<fche> is this a source code repo or somethign else?
<fche> (sorry, not familiar)
<cgf> Also, what would be the reason for a mirror on sourceware if you already
      have a repo?  [17:58]
<zimoun> To be more concrete, and if you are not familiar with Guix, there is
         command named "guix pull" which basically "git pull". Currently, the
         main repo is on Savannah.
<zimoun> And everything is fine with Savannah. :-)
<zimoun> However, when Savannah is down; which happens, really rarely, but
         happens, then all Guix is kind-of broken.  [17:59]
<fche> so how big is this repo, what form is it, ?  [18:00]
<zimoun> Broken means that the users cannot update their system. And it is not
         related to development but related on normal usage. Well, "guix pull"
         is kind-of equivalent to "apt-get update".
<fche> accessed via git or what?
<fche> we're not set up to host large binaries e.g.  [18:01]
<zimoun> Yes, via Git. And it is only for source.
<cgf> Maybe hold off on this until new sourceware is online?
<zimoun> The mirror of https://git.savannah.gnu.org/cgit/guix.git/
<zimoun> And if it is too big, maybe even only the master branch  [18:02]
<fche> hm, I can't git-clone that bad boy here
<fche> ah never mind starting  [18:03]
<zimoun> What I have in mind, and I speak for myself and not on the behalf of
         the Guix project.
<fche> not that big apparently, some 170MB of content, that's not a problem
<zimoun> sorry, my browser returns me the wrong URL  [18:04]
<zimoun> https://git.savannah.gnu.org/cgit/guix.git/
<zimoun> https://git.savannah.gnu.org/git/guix.git
<zimoun>
<fche> yeah
<fche> do you have a personal sourceware presence already?
<zimoun> Just to be on the same wavelength, currently, each time an user run
         "guix pull" then they "git pull" from
         https://git.savannah.gnu.org/git/guix.git. However, when Savannah is
         down which rarely happens, everybody is annoyed. So I would like that
         "guix pull" catch the error and then try to use a mirror. Which
         means, that sourceware.org/your-convention-guix will be the default
         mirror. So the traffic pull will be very low (when Savannah is
                                                                        [18:11]
<zimoun> down or unreachable) and the traffic push a bit more.
<mjw> guix is awesome! But savannah often is overloaded :{
<zimoun> No I do not have a personnal sourceware presence already.  [18:12]
* fche has no objection
<fche> how do you plan to make the local git repo a fresh enough mirror?
<zimoun> Then, I can ask to Savannah admin to have a hook to automatically
         mirror to sourceware
<fche> or you can run a cron job on sourceware to auto git-pull periodically
                                                                        [18:13]
<zimoun> yes, I have not think too much about the details. Because I would
         like first if it was even possible. :-)
<fche> do you expect the repo to seriously grow over time ?  [18:14]
<zimoun> No. But I can do some stats with Git to answer you more
         precisely. Currently, Guix is around 13000 commits per
         years. However, I do not know how many addition lines it represents
                                                                        [18:16]
<mjw> It guix.git is ~124MB currently. It normally sees multiple commits a
      day, but most of them are fairly small (a couple of lines to a handful
      of files).  [18:18]
<fche> I mean it's not going to grow from 120MB to 1GB or 10G in the
       forseeable future?
<fche> those are the levels at which we might get concerned
<zimoun> Who knows ;-) But I do not think so. And in case, you can say: stop,
         it is too much for our infrastructure.  [18:20]
<zimoun> Well, if we have a first agreement on an hostage (mirror), can I save
         this conversation and post it on the mailing list guix-devel. Collect
         feedback; especially from the maintainers. Then come back to you with
         more concrete details.  [18:21]
<fche> 'forseeable' doesn't ask for infinite prophecy powers :)
<zimoun> :-)
<fche> yeah, I think we can do it.
<zimoun> thank you for your help.
<segher> but how much load on sourceware will this cause?
<zimoun> do you want to be CC of the email?
<mjw> guix is very much pure GNU, and very much sources should produce
      reproducible builds. You can expect it to stay text based forever and
      never see any binary blobs.
<fche> segher, it's a git repo similar in size to valgrind  [18:22]
<zimoun> Almost no load on sourceware. As I tried to explain above. It would
         be a feedback when Savannah is down -- which happens rarely from my
         experience.
<segher> fche: no...  how many pulls per seconds are asked from it  [18:23]
<segher> zimoun: that doesn't change the worst case at all
<zimoun> segher: I have no idea. I think it is hard to count.
<fche> segher, I expect nothing noticeable, but zimoun will let us know if
       there's reason to worry  [18:24]
<fche> ie a few downloads a day is nothing; a hundred git updates an hour is
       nothing
<segher> and a simple gcc pull is almost 20m
<mjw> zimoun, I can also provide a backup mirror. Not as fancy as sourceware,
      but if you need something to fallback on.  [18:25]
<zimoun> segher: yes, it is just a first contact to see if we can agree. I
         totally understand your limitation.
<segher> a fresh clone, i mean
<fche> the gcc git repo is weird/svny and much larger
<segher> yes
<segher> my point is that we already are overloaded  [18:26]
<fche> not with git clones
<segher> the server, in general
<mjw> zimoun, it would go under https://code.wildebeest.org/git/mirror/ (I
      cannot guarantee an SLA though, but I think neither can sourceware)
<fche> and definitely not with git updates
<segher> if it is actually light weight, then who cares ;-)  [18:27]
<cgf> overloaded sourceware is why I suggested waiting until the new server
      comes online.
<segher> right
<cgf> There's also the issue of outdated git.  [18:28]
<segher> it already is super annoying (to GCC) that half of the time (or
         whatever; it feels very often) the problem is caused by cygwin, not
         by us
<cgf> Probably more like 70% of the time.  :-)  [18:29]
<zimoun> mjw: thank you. And thank you for the hard work you did on the gnu
         list moderation.
<segher> cgf: it feels that way, yes
<mjw> zimoun, haha, yeah about that... o boy.
<cgf> I may be wrong but I don't think there are any other projects like
      cygwin on sourceware.  cygwin is a source code repository (lightweight)
      and a binary distribution mechanism (imo, heavy weight)  [18:30]
<cgf> but I digress...
<zimoun> mjw: as you are also on #guix maybe you can suggest the idea of
         wildebeest. Did you subscribe to the guix-devel mailing list?  [18:31]
<segher> cgf: right, and sharing resources with a project with different usage
         patterns of those resources is a bit problematic
<fche> segher, let us know of problems when they appear, and we'll keep
       tweaking things  [18:32]
<fche> there are a lot of knobs to turn
<segher> fche: i can tell you things were bad yesterday  [18:33]
<segher> not too useful, eh :-)
<zimoun> My point coming here and asking is: establish a first contact to see
         if we can go further together. Or if it is already a no go. Because I
         understand you need numbers and details. And I do not have them,
         right now. But I would like to open the discussion on this topic of
         mirroring.
<fche> segher, exactly :)
<mjw> And on Monday gcc will switch to git too!  [18:34]
<fche> zimoun, we can put something like that up in a matter of minutes
<segher> (gcc bugzilla gave ISEs most of the time)
<mjw> well as a mirror and then Dec 31! finally git all the way!
<fche> mjw, that should help our loads, svn is much less efficient than git
<zimoun> fche: thank you for the offer but I am not maintainer and I speak on
         my own voice. I would like to hear the civodul/Ludo's opinion :-)
                                                                        [18:35]
<mjw> civodul did suggest sourceware too, I did tell him to join here. But
      does he listen to me...  [18:36]
<zimoun> mjw: yeah! I know... that's why I am doing ;-)
<zimoun> but you know, french diplomacy :-)  [18:37]

Re: Guix mirror: sourceware discussion report

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 2008 bytes --]

Zimoun,

I've replied on IRC as well, apologies if I repeat myself.  :-)

zimoun 写道：
> Currently "guix pull" from Savannah and issues can arise. As we
> recently experimented. Tobias and Ricardo recently discussed how 
> to
> mirror the repo. IMHO, it is a good idea to mirror but not a 
> good idea
> to locate it on Ricardo infrastructure, again. :-)

I disagree.

I think berlin is the best place to create a simple ‘main’ mirror 
at this point in time.  It's a simple git one-liner to keep it up 
to date.  In the extremely unlikely event that it breaks, many 
people can fix it.  Ricardo works hard but isn't responsible for 
all of Berlin on his own.

> Well, I propose to see if we can mirror on sourceware.org which 
> are
> GNU friends. ;-)
>
> I took the liberty to contact them on IRC. See below.
>
> What I see is: we still "guix pull" from Savannah as usual.
> Then if Savannah is down, we catch the error and we try 
> (transparently
> for the user) the sourceware url.

By ‘we’, you mean the ‘guix pull’ command itself, correct?

Something akin to

  (channel
    (name 'guix)
    (url (list "https://savannah…"
               "https://sourceware…"))
    …)

?

I'd like to see that added to the channels.scm format for selfish 
reasons.

However, I'd still like the first or second URL to start with 
‘guix.gnu.org’.  I have nothing against Sourceware, but 
guix.gnu.org has been pretty solid (AFAIK) and, more importantly, 
does not require users to trust yet another party by default.

[‘Trustable Guix pull’ rears its head again :-) …]

> As a first contact, they agree on the principle. Even they ask 
> numbers
> about traffic etc. :-)

I know that about 1,000 unique IPs requested substitutes on Berlin 
during the month of November.

You'll have to ask Savannah for the git stats (roughly how many 
IPs ran ‘guix pull’), which may wildly differ (but I doubt that).

Thanks!

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: Guix mirror: sourceware discussion report

[zimoun]

Hi Tobias,

On Fri, 13 Dec 2019 at 19:39, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> I've replied on IRC as well, apologies if I repeat myself.  :-)

It is hard to follow everything that is said on IRC. ;-)
And times to times, IMO, it is necessary to report back on guix-devel
what was said: the community is then aware and it is recorded (so
searchable in the future if needed).


> zimoun 写道：
> > Currently "guix pull" from Savannah and issues can arise. As we
> > recently experimented. Tobias and Ricardo recently discussed how
> > to
> > mirror the repo. IMHO, it is a good idea to mirror but not a
> > good idea
> > to locate it on Ricardo infrastructure, again. :-)
>
> I disagree.
>
> I think berlin is the best place to create a simple ‘main’ mirror
> at this point in time.  It's a simple git one-liner to keep it up
> to date.  In the extremely unlikely event that it breaks, many
> people can fix it.  Ricardo works hard but isn't responsible for
> all of Berlin on his own.

I agree on these lines. And we are talking about a service (mirror)
easy to deploy elsewhere.
But... obviously a but. :-)

Correct me if I am wrong, currently, Guix mainly depends on:
 - the FSF guys for the main repo (Savannah) located in US.
 - the MDC institute located in Germany.
    - the build farm (Cuirass instance): ci.guix.gnu.org
    - the storage of substitutes (even it is mitigated with CDN)
    - the Mumi front-end of Debbugs: issues.guix.gnu.org
 - Boards here and there (Julien and now Pierre, Andreas, etc.)

Well, how many people have a physical access to the MDC infrastructure?
How many people are currently working for the MDC institute?
To my knowledge, only Ricardo.
So what will happen if Ricardo "leaves" (long vacation, change of job,
bus attack, etc.).


Guix should not put all the eggs in the same basket, IMHO.

Therefore, this tiny service (mirror) should done elsewhere than on
the machines located in Berlin. As a simple first step.


> > Well, I propose to see if we can mirror on sourceware.org which
> > are
> > GNU friends. ;-)
> >
> > I took the liberty to contact them on IRC. See below.
> >
> > What I see is: we still "guix pull" from Savannah as usual.
> > Then if Savannah is down, we catch the error and we try
> > (transparently
> > for the user) the sourceware url.
>
> By ‘we’, you mean the ‘guix pull’ command itself, correct?

Yes.
Sorry. Too passionate. ;-)


> Something akin to
>
>   (channel
>     (name 'guix)
>     (url (list "https://savannah…"
>                "https://sourceware…"))
>     …)
>
> ?

Yes.
Or built-in. Whatever.

Even, talking about that, a official list of Git mirrors should be
maintained and included in the manual or cookbook or website or
you-name-it.


> I'd like to see that added to the channels.scm format for selfish
> reasons.

I understand and I do not have any strong opinion. :-)
Even, the channels.scm seems better than hard-coded.


> However, I'd still like the first or second URL to start with
> ‘guix.gnu.org’.  I have nothing against Sourceware, but
> guix.gnu.org has been pretty solid (AFAIK) and, more importantly,
> does not require users to trust yet another party by default.

I agree.

I am not expert but can mirror.guix.gnu.org point to a machine located
elsewhere than Berlin?


> [‘Trustable Guix pull’ rears its head again :-) …]

+1


> > As a first contact, they agree on the principle. Even they ask
> > numbers
> > about traffic etc. :-)
>
> I know that about 1,000 unique IPs requested substitutes on Berlin
> during the month of November.

Thank you for this number.
Interesting.


> You'll have to ask Savannah for the git stats (roughly how many
> IPs ran ‘guix pull’), which may wildly differ (but I doubt that).

I will. :-)



Thank you.

All the best,
simon

Re: Guix mirror: sourceware discussion report

[Ricardo Wurmus]

zimoun <zimon.toutoune@gmail.com> writes:

> Well, how many people have a physical access to the MDC infrastructure?
> How many people are currently working for the MDC institute?
> To my knowledge, only Ricardo.
> So what will happen if Ricardo "leaves" (long vacation, change of job,
> bus attack, etc.).

Mădălin also has physical access.

-- 
Ricardo

Re: Guix mirror: sourceware discussion report

[zimoun]

Hi,

On Sat, 14 Dec 2019 at 23:44, Ricardo Wurmus <rekado@elephly.net> wrote:

> > Well, how many people have a physical access to the MDC infrastructure?
> > How many people are currently working for the MDC institute?
> > To my knowledge, only Ricardo.
> > So what will happen if Ricardo "leaves" (long vacation, change of job,
> > bus attack, etc.).
>
> Mădălin also has physical access.

Nice!
I withdraw my proposal.

Something as mirror.guix.gnu.org sounds sweet! :-)

All the best,
simon

ps:
I hope that nobody has taken my tiny proposal as a criticism of all
the tough work done in Berlin. On the contrary, it was an attempt to
help.

Re: Guix mirror: sourceware discussion report

[Ludovic Courtès]

Hi!

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> Something akin to
>
>  (channel
>    (name 'guix)
>    (url (list "https://savannah…"
>               "https://sourceware…"))
>    …)
>
> ?
>
> I'd like to see that added to the channels.scm format for selfish
> reasons.

Yup, that’d be nice.

> However, I'd still like the first or second URL to start with
> ‘guix.gnu.org’.  I have nothing against Sourceware, but guix.gnu.org
> has been pretty solid (AFAIK) and, more importantly, does not require
> users to trust yet another party by default.
>
> [‘Trustable Guix pull’ rears its head again :-) …]

Yeah…  I discussed this a bit with an in-toto/TUF person at the R-B
Summit and I’m wrapping my head again around it now, because clearly,
this is the main showstopper here.

Once this is addressed, at least to some level, we can happily mirror on
all the hosting sites we can think of.

Anyway, thank you zimoun for contacting the Sourceware folks, and thanks
Tobias & Ricardo for setting up the mirror on berlin!

Ludo’.