No 'guix pull' substitutes for foreign distros?

No 'guix pull' substitutes for foreign distros?

[Nigko Yerden]

Hello Guix!

I have two systems: first (A) is a full Guix system and second (B) is a 
foreign distribution with Guix package manager installed onto. I have 
noticed when I'm performing 'guix pull' system B always build 'guix 
pull' derivations by itself while system A most often just download 
substitutes for them. The latter, of course, is much faster.

Here is how it looks on system B (commit 
014875b29e68da6357a5323e6dd1eaa74a05b753):

$ guix pull
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to 08360a2 (781 new 
commits)...
Building from these channels:
   guix      https://git.savannah.gnu.org/git/guix.git	08360a2
substitute: updating substitutes from 
'https://mirrors.sjtug.sjtu.edu.cn/guix'... 100.0%
26,4 MB will be downloaded
....
The following derivations will be built:
   /gnu/store/q9b8yr9452im7mmah1cfk2dpy44y755v-profile.drv
   /gnu/store/87avc5s6q4xq1rdk26cnlviai7hfjzi0-inferior-script.scm.drv
   /gnu/store/saji8f542jbrmj6315lxrgmgjdgrzlx4-guix-08360a2cd.drv
   /gnu/store/3j2xd5l3q0s1jf9798fr4s3k3ky5fw07-guix-manual.drv
   /gnu/store/42bwpkmcv6jsdzhsxc0pwc13k1fgxh25-guix-08360a2cd-modules.drv
 
/gnu/store/2cl0dr63d900v6406hs10qgjsp7iaysi-guix-packages-base-modules.drv
   /gnu/store/j8av01zjcx9kimjm0yrdrkhmiba7wwmj-guix-packages-base.drv
   /gnu/store/a7w5hbq6kppl90838b3iwpgn4v93jv75-guix-extra.drv
   /gnu/store/gd8lqzj70wx5qgx39lnh3lh4g5slln5g-guix-core-source.drv
   /gnu/store/2f0z0x4nqpwgwmrk5qs7ka0p3y2a2gny-config.scm.drv
   /gnu/store/6r0k3wg5bcjnjdjlx8rvxr2q3izyhgvh-guix-system-tests-modules.drv
   /gnu/store/7pmrsz3mlnhiggm3pz99wiyrddahl9mb-guix-system-tests.drv
   /gnu/store/3jwjfb0rjxsvv4xy0m65db0nmbg56nx7-guix-cli.drv
   /gnu/store/87yna60fdd0gv70nb4svqvxq8br4cbm6-guix-cli-core.drv
   /gnu/store/b28ayyn4jqfz5dbgcshh5gmayzms65wj-guix-system.drv
   /gnu/store/x5x97w98q16myzx3rfik1gs06n1vbccr-guix-packages.drv
   /gnu/store/sxhm9vii5lwig55vjpc4jz639r9ygdyx-guix-home.drv
   /gnu/store/7i2z4b2imjxc0qd5kgis9c045ybq0wwy-guix-system-modules.drv
   /gnu/store/f3qxxs77klhk0q8bvsxdly1xr147wb9s-guix-cli-modules.drv
   /gnu/store/fl6m42fdnykhd5drv42s1zn5x18vbiy9-guix-extra-modules.drv
   /gnu/store/h7ynn7s9p8ywd5s2s4g43l6rgkqdidqd-guix-home-modules.drv
   /gnu/store/i0qnblii1as7yk2pz3vvvfd9mh28fdzv-guix-config-modules.drv
   /gnu/store/m02vi15w4chc27kgrf2x55riacsl20gc-guix-config-source.drv
   /gnu/store/6z80pzmm18nmbmblk9mh2vjl1hvdzm02-config.scm.drv
   /gnu/store/jilxlihmy07h9xhr1pbxw5i01b4l1js1-guix-cli-core-modules.drv
   /gnu/store/ng76j3d6m66w7jfk0gds9204zp4l727p-guix-packages-modules.drv
   /gnu/store/znffqlp69qjbvzpnjvvfmpc4fq6q9b8w-guix-core-modules.drv
   /gnu/store/qxaqgq36bg5mi210yflrg31hz3nxjfb3-guix-command.drv
   /gnu/store/as05nvh2dq4s0pb0lxkmcyf407c7wjcm-guix-module-union.drv
   /gnu/store/qxsz62r8giy0hf8qqzzv8f35g6470kcj-guix-daemon.drv
   /gnu/store/za0rf8ypn7v1a9a7cc7fmnnh7pig5580-guix-misc.drv
   /gnu/store/pf1h4mhpp6qhvixzr4172v81wqy1wj1x-inferior-script.scm.drv
   /gnu/store/3aci63l4yq7i9jh4v55f0lyk3hji8z1w-inferior-script.scm.drv
   /gnu/store/3kjkjx8fayv97p48rgdmi51mlwx6lb7c-profile.drv

3,0 MB will be downloaded
  module-import-compiled  128KiB        57KiB/s 00:02 
▕██████████████████▏ 100.0%
  guix-config  4KiB                    647KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-core  1.8MiB                    418KiB/s 00:04 
▕██████████████████▏ 100.0%
  guix-translated-texinfo  1.7MiB      971KiB/s 00:02 
▕██████████████████▏ 100.0%
  module-import-compiled  8KiB          13KiB/s 00:01 
▕██████████████████▏ 100.0%
  module-import-compiled  47KiB         88KiB/s 00:01 
▕██████████████████▏ 100.0%
  module-import-compiled  66KiB         20KiB/s 00:03 
▕██████████████████▏ 100.0%
  module-import-compiled  97KiB          6KiB/s 00:17 
▕██████████████████▏ 100.0%
  module-import-compiled  54KiB          8KiB/s 00:07 
▕██████████████████▏ 100.0%
building /gnu/store/2f0z0x4nqpwgwmrk5qs7ka0p3y2a2gny-config.scm.drv...
building /gnu/store/6z80pzmm18nmbmblk9mh2vjl1hvdzm02-config.scm.drv...
applying 9 grafts for gd-2.3.2 ...
applying 5 grafts for gts-0.7.6 ...
building 
/gnu/store/m02vi15w4chc27kgrf2x55riacsl20gc-guix-config-source.drv...
applying 19 grafts for graphviz-minimal-7.0.1 ...
building 
/gnu/store/i0qnblii1as7yk2pz3vvvfd9mh28fdzv-guix-config-modules.drv...
building /gnu/store/gd8lqzj70wx5qgx39lnh3lh4g5slln5g-guix-core-source.drv...
building /gnu/store/za0rf8ypn7v1a9a7cc7fmnnh7pig5580-guix-misc.drv...
building 
/gnu/store/znffqlp69qjbvzpnjvvfmpc4fq6q9b8w-guix-core-modules.drv...
building /gnu/store/a7w5hbq6kppl90838b3iwpgn4v93jv75-guix-extra.drv...
building 
/gnu/store/87avc5s6q4xq1rdk26cnlviai7hfjzi0-inferior-script.scm.drv...
building 
/gnu/store/fl6m42fdnykhd5drv42s1zn5x18vbiy9-guix-extra-modules.drv...
building 
/gnu/store/j8av01zjcx9kimjm0yrdrkhmiba7wwmj-guix-packages-base.drv...
building 
/gnu/store/pf1h4mhpp6qhvixzr4172v81wqy1wj1x-inferior-script.scm.drv...
building /gnu/store/87yna60fdd0gv70nb4svqvxq8br4cbm6-guix-cli-core.drv...
building 
/gnu/store/2cl0dr63d900v6406hs10qgjsp7iaysi-guix-packages-base-modules.drv...
building 
/gnu/store/jilxlihmy07h9xhr1pbxw5i01b4l1js1-guix-cli-core-modules.drv...
building /gnu/store/x5x97w98q16myzx3rfik1gs06n1vbccr-guix-packages.drv...
applying 3 grafts for libfontenc-1.1.4 ...
building 
/gnu/store/ng76j3d6m66w7jfk0gds9204zp4l727p-guix-packages-modules.drv...
building /gnu/store/b28ayyn4jqfz5dbgcshh5gmayzms65wj-guix-system.drv...
applying 8 grafts for mkfontscale-1.2.2 ...
building /gnu/store/sxhm9vii5lwig55vjpc4jz639r9ygdyx-guix-home.drv...
building 
/gnu/store/7i2z4b2imjxc0qd5kgis9c045ybq0wwy-guix-system-modules.drv...
building /gnu/store/3jwjfb0rjxsvv4xy0m65db0nmbg56nx7-guix-cli.drv...
building 
/gnu/store/h7ynn7s9p8ywd5s2s4g43l6rgkqdidqd-guix-home-modules.drv...
building /gnu/store/f3qxxs77klhk0q8bvsxdly1xr147wb9s-guix-cli-modules.drv...
building 
/gnu/store/7pmrsz3mlnhiggm3pz99wiyrddahl9mb-guix-system-tests.drv...
applying 2 grafts for mkfontdir-1.0.7 ...
building 
/gnu/store/6r0k3wg5bcjnjdjlx8rvxr2q3izyhgvh-guix-system-tests-modules.drv...
applying 6 grafts for texinfo-6.8 ...
building 
/gnu/store/42bwpkmcv6jsdzhsxc0pwc13k1fgxh25-guix-08360a2cd-modules.drv...
building /gnu/store/3j2xd5l3q0s1jf9798fr4s3k3ky5fw07-guix-manual.drv...
building 
/gnu/store/as05nvh2dq4s0pb0lxkmcyf407c7wjcm-guix-module-union.drv...
building /gnu/store/qxaqgq36bg5mi210yflrg31hz3nxjfb3-guix-command.drv...
building /gnu/store/qxsz62r8giy0hf8qqzzv8f35g6470kcj-guix-daemon.drv...
building /gnu/store/saji8f542jbrmj6315lxrgmgjdgrzlx4-guix-08360a2cd.drv...
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
...


And there how it looks on system A (the same commit 
014875b29e68da6357a5323e6dd1eaa74a05b753):
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
Authenticating channel 'guix', commits 9edb3f6 to 08360a2 (781 new 
commits)...
Building from these channels:
   guix      https://git.savannah.gnu.org/git/guix.git	08360a2
26,4 MB will be downloaded
...
substitute: updating substitutes from 
'https://mirrors.sjtug.sjtu.edu.cn/guix'... 100.0%
...
The following derivations will be built:
   /gnu/store/pfs471chzsnwyl3zdkz3wkzl5gpsqchm-profile.drv
   /gnu/store/87avc5s6q4xq1rdk26cnlviai7hfjzi0-inferior-script.scm.drv
 
/gnu/store/1mwhp5crimgvqd4jmlfsj3017yk8vxsv-guix-packages-base-modules.drv
   /gnu/store/4bay72mqflnhx3382gy5lmcjiv0k673s-guix-system-tests-modules.drv
   /gnu/store/5dsx6hxbrsy8iljmz6sjn08f1c9q5zi9-guix-system-modules.drv
   /gnu/store/j33qgaybzml6fbfldw5nljgq4haxd34f-guix-cli-core-modules.drv
   /gnu/store/rzwb58zr3mnz5pff4v0581nr8bhh4bj5-guix-cli-modules.drv
   /gnu/store/yhv9fr3rzlnk8dmgiwz73rjz6xalip94-guix-extra-modules.drv
   /gnu/store/hmcds0yp3bxl8y8gaf6a42jfipvqgsii-guix-daemon.drv
   /gnu/store/x91k63vwwj6mahn804hjg5n9xzsjj4sn-guix-command.drv
   /gnu/store/pf1h4mhpp6qhvixzr4172v81wqy1wj1x-inferior-script.scm.drv
   /gnu/store/2x3s1sx169y9lk9wdnz8p21c88jr8jkd-profile.drv
   /gnu/store/mkq51lalsi9vy94afskmi92854sx1cxm-inferior-script.scm.drv

70,0 MB will be downloaded
substitute: updating substitutes from 
'https://mirrors.sjtug.sjtu.edu.cn/guix'... 100.0%
...
  module-import-compiled  128KiB      43.7MiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-cli  315KiB                      27KiB/s 00:12 
▕██████████████████▏ 100.0%
  guix-cli-core  922KiB                 21KiB/s 00:44 
▕██████████████████▏ 100.0%
  guix-cli-modules  260B                56KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-cli-core-modules  261B           46KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-extra  2.8MiB                    10KiB/s 04:45 
▕██████████████████▏ 100.0%
  guix-manual  5.2MiB                   13KiB/s 06:58 
▕██████████████████▏ 100.0%
  guix-extra-modules  260B              61KiB/s 00:00 
▕██████████████████▏ 100.0%
  module-import-compiled  66KiB         34KiB/s 00:02 
▕██████████████████▏ 100.0%
  guix-packages-base  17.4MiB          777KiB/s 00:23 
▕██████████████████▏ 100.0%
  guix-system  7.9MiB                  702KiB/s 00:12 
▕██████████████████▏ 100.0%
  guix-packages-base-modules  268B     140KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-system-modules  261B            101KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-system-tests  921KiB            368KiB/s 00:03 
▕██████████████████▏ 100.0%
  module-import-compiled  54KiB         69KiB/s 00:01 
▕██████████████████▏ 100.0%
  guix-system-tests-modules  263B       57KiB/s 00:00 
▕██████████████████▏ 100.0%
building 
/gnu/store/87avc5s6q4xq1rdk26cnlviai7hfjzi0-inferior-script.scm.drv...
applying 8 grafts for mkfontscale-1.2.2 ...
  guix-08360a2cd-modules  31.2MiB      1.9MiB/s 00:17 
▕██████████████████▏ 100.0%
building 
/gnu/store/pf1h4mhpp6qhvixzr4172v81wqy1wj1x-inferior-script.scm.drv...
  guix-module-union  2KiB              149KiB/s 00:00 
▕██████████████████▏ 100.0%
applying 2 grafts for mkfontdir-1.0.7 ...
  guix-command  653B                   436KiB/s 00:00 
▕██████████████████▏ 100.0%
applying 6 grafts for texinfo-6.8 ...
  guix-daemon  402B                    105KiB/s 00:00 
▕██████████████████▏ 100.0%
  guix-08360a2cd  800B                  94KiB/s 00:00 
▕██████████████████▏ 100.0%
building CA certificate bundle...
listing Emacs sub-directories...
building fonts directory...
building directory of Info manuals...
...


Shouldn't 'guix pull' substitutes also work on foreign distros?
If they should, why don't sytem B get them?

Regards,
Nigko

Re: No 'guix pull' substitutes for foreign distros?

[André A. Gomes]

Nigko Yerden <nigko.yerden@gmail.com> writes:

> Hello Guix!
> Shouldn't 'guix pull' substitutes also work on foreign distros?

I'd like to re-iterate the question.  I have been using the Guix system
for years but lately I'm also using it as a package manager.  It is
rather off-putting that guix pull will take more than 20 minutes in this
setting.  Newcomers will probably start by using it a package manager
and only then commit to the system.  

Thanks.


-- 
André A. Gomes
"You cannot even find the ruins..."

Re: No 'guix pull' substitutes for foreign distros?

[Andreas Enge]

Am Sun, May 19, 2024 at 10:38:28PM +0300 schrieb André A. Gomes:
> Nigko Yerden <nigko.yerden@gmail.com> writes:
> > Hello Guix!
> > Shouldn't 'guix pull' substitutes also work on foreign distros?
> I'd like to re-iterate the question.  I have been using the Guix system
> for years but lately I'm also using it as a package manager.  It is
> rather off-putting that guix pull will take more than 20 minutes in this
> setting.  Newcomers will probably start by using it a package manager
> and only then commit to the system.  

I have no answer, but can confirm the problem and state that it is not
related to using Guix as a package manager or as a full system. In both
cases "guix pull" should do the same thing.

However, I am also surprised by the following behaviour: I have three users
on my Guix system, and always do three "guix pull --commit=..." on the same
commit for all of them. It happens that the second or third user compiles
derivations that I would expect to be in the store already. I do not know
what the differences are. Someone suggested to me it could be related to
channels. Apparently, using a different set of channels leads to different
derivations. But last week it has also happened for my two users with only
the main channel.

Andreas

Re: No 'guix pull' substitutes for foreign distros?

[pelzflorian (Florian Pelz)]

Hello all, the missing substitutes in Debian can be worked around; Nigko
and I discussed in bug#70954 by editing
/etc/systemd/services/multi-user.target.wants/guix-daemon.service this way:

Nigko Yerden <nigko.yerden@gmail.com> writes:
<https://lists.gnu.org/archive/html/bug-guix/2024-05/msg00147.html>:
> Hello Florian,
>
> These are great news! Thanks a lot for investigating this!
>
> Indeed, foreign distros still run guix-daemon from Guix v1.4,
> which is very old now. It makes sense that old guix-daemon + recent
> remaining parts of Guix might have produced modified derivations.
>
> For foreign distros using systemd, the following workaround may be
> useful to replace old guix-daemon by guix-daemon from USER profile:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~guix-daemon.service
> ...
> [Service]
> ExecStart=/home/USER/.config/guix/current/bin/guix-daemon \
>     --build-users-group=_guixbuild --discover=no \
>     --substitute-urls='.....'
>
> Environment='GUIX_LOCPATH=/home/USER/.guix-profile/lib/locale'
> ...
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Regards,
> Nigko

Theoretically, the problem could be resolved in the Debian package by
including as a patch the Guix commit

commit 95f21231352b66f92c466cb30f2258291e854f2f
Author: Ludovic Courtès <ludo@gnu.org>
Date:   Mon Sep 11 14:22:44 2023 +0200

    daemon: Add “git-download” built-in builder.
    
    The new builder makes it possible to break cycles that occurs when the
    fixed-output derivation for the source of a dependency of ‘git’ would
    itself depend on ‘git’.


(should be the right commit, I think), but Debian policy is to only
deliver important bug fixes in their stable distribution.  Is this an
important bug fix?  Pull works but is just slow without.  My intuition
is we better wait for a Guix 1.5.0 release.  Cc to Vagrant who maintains
the Debian package.

Similarly for other foreign distros’ packages.

Regards,
Florian

Re: No 'guix pull' substitutes for foreign distros?

[André A. Gomes]

"pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> writes:

> Hello all, the missing substitutes in Debian can be worked around; Nigko
> and I discussed in bug#70954 by editing
> /etc/systemd/services/multi-user.target.wants/guix-daemon.service this way:

Thanks Florian, it does fix it!

I understand the arguments behind waiting for 1.5 and the conservative
nature of Debian distributions.  But I wonder why would anyone want to
run a daemon that is not the one being fetched from guix pull.  Clearly,
the exception would be on the first run (guix installation as a foreign
distro), where a tagged guix-daemon must be used.  That's my naive take,
but I may be missing some important details.


-- 
André A. Gomes
"You cannot even find the ruins..."