About guix mirror build script

About guix mirror build script

[tumashu]

In China , guix  substitute  download  seem to be  tooooooooooooo slow,  nearly unuseable.
this week, I have found that  NIX channels has been built China mirror successful with the help of
4 files:

1. https://github.com/tuna/tunasync-scripts/blob/master/nix.py
2. https://github.com/tuna/tunasync-scripts/blob/master/nix-channels.py
3. https://github.com/tuna/tunasync-scripts/blob/master/nixos-images.py
4. https://github.com/tuna/tunasync-scripts/blob/master/dockerfiles/nix-channels/Dockerfile

guix is very similar nix, so I want to know how to port 4 files to work with guix?
is it possible add similar tools to guix?

Re: About guix mirror build script

[zimoun]

Dear,

On Tue, 10 Mar 2020 at 03:48, tumashu <tumashu@163.com> wrote:

> In China , guix  substitute  download  seem to be  tooooooooooooo slow,  nearly unuseable.

About which kind of substitutes are you talking?
 1. from where Guix pulls? (guix pull)
 2. from where Guix fetches the binaries? (guix install)

About 1. we had a discussion [1] on December about mirroring the
official Git repo on Savannah. But AFAIK, nothing relevant has be done
in this area. Feel free to propose a plan. :-)


About 2. currently the binaries are mainly served by ci.guix.gnu.org
located in Berlin.
And an attempt to use CDN had been done a couple of month ago. You
could be interested by this thread [2] and as you reported [3] the CDN
experience was improving the situation in China.

Well, the best is perhaps to time the issue, see this message [4].

[1] https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00148.html
[2] https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00135.html
[3] https://lists.gnu.org/archive/html/guix-devel/2019-03/msg00222.html
[4] https://lists.gnu.org/archive/html/guix-devel/2018-12/msg00192.html


On the long term, a patch [5] using IPFS needs some love (review, tests, etc.).

[5] https://issues.guix.info/issue/33899


> this week, I have found that  NIX channels has been built China mirror successful with the help of
> 4 files:
>
> 1. https://github.com/tuna/tunasync-scripts/blob/master/nix.py
> 2. https://github.com/tuna/tunasync-scripts/blob/master/nix-channels.py
> 3. https://github.com/tuna/tunasync-scripts/blob/master/nixos-images.py
> 4. https://github.com/tuna/tunasync-scripts/blob/master/dockerfiles/nix-channels/Dockerfile
>
> guix is very similar nix, so I want to know how to port 4 files to work with guix?
> is it possible add similar tools to guix?

Well, I have not dove into all these Python scripts to know exactly
what they  are doing.
And I do not know what are the storage resources one needs to mirror
ci.guix.gnu.org.
But yes, if you have the resources (storage) to mirror the binaries
susbtitute, you can try to adapt these scripts by replacing the URLs.


All the best,
simon

Re: About guix mirror build script

[Ricardo Wurmus]

tumashu <tumashu@163.com> writes:

> In China , guix  substitute  download  seem to be  tooooooooooooo slow,  nearly unuseable.

I have enabled an rsync daemon on ci.guix.gnu.org to share the “guix
publish” cache.  There are a few things missing:

* not all files in /gnu/cache/guix can be read due to overly restrictive
  permissions generated by “guix publish”

* having a copy of the cache alone is not enough.  You’ll need to serve
  them like “guix publish” would.  I haven’t looked closely enough, but
  it should be possible to serve this all statically.

The first useful step, I believe, is to mirror the “guix publish” cache.

--
Ricardo