bug#45173: ‘guix search’ traverses all the modules when extra channels are used

bug#45173: ‘guix search’ traverses all the modules when extra channels are used

[Ludovic Courtès]

When a single channel is in use, ‘guix search’ understandable traverses
nothing but gnu/packages/*.scm:

--8<---------------cut here---------------start------------->8---
$ strace -o /tmp/,,s guix search sdfsdf
$ grep gnu/services /tmp/,,s
$ echo $?
1
--8<---------------cut here---------------end--------------->8---

But with multiple channels, all of the guile-module-union directory is
traversed, and that includes *everything*:

--8<---------------cut here---------------start------------->8---
$ /tmp/past/bin/guix describe
Generacio 1     Dec 11 2020 10:26:58    (nuna)
  guix-past 829923f
    repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
    branch: master
    commit: 829923f01f894f1e687735627025ada26230832f
  guix f765577
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: f765577dec0340d035021d030f17b54a3a5fd6b1
$ strace -o /tmp/,,s /tmp/past/bin/guix search sdfsdf
$ grep gnu/services /tmp/,,s
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services.scm", {st_mode=S_IFREG|0444, st_size=41045, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services.go", {st_mode=S_IFREG|0444, st_size=420021, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services.go", O_RDONLY|O_CLOEXEC) = 14
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/shepherd.scm", {st_mode=S_IFREG|0444, st_size=23156, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/shepherd.go", {st_mode=S_IFREG|0444, st_size=405997, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/shepherd.go", O_RDONLY|O_CLOEXEC) = 14
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/herd.scm", {st_mode=S_IFREG|0444, st_size=10387, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/herd.go", {st_mode=S_IFREG|0444, st_size=188509, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/herd.go", O_RDONLY|O_CLOEXEC) = 14
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/base.scm", {st_mode=S_IFREG|0444, st_size=110305, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/base.go", {st_mode=S_IFREG|0444, st_size=2203613, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/base.go", O_RDONLY|O_CLOEXEC) = 14
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/admin.scm", {st_mode=S_IFREG|0444, st_size=12494, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/admin.go", {st_mode=S_IFREG|0444, st_size=416133, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/admin.go", O_RDONLY|O_CLOEXEC) = 14
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/mcron.scm", {st_mode=S_IFREG|0444, st_size=7667, ...}) = 0
stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/mcron.go", {st_mode=S_IFREG|0444, st_size=187549, ...}) = 0
openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services/mcron.go", O_RDONLY|O_CLOEXEC) = 14
--8<---------------cut here---------------end--------------->8---

This was reported by bdju on #guix, who were getting the new SSH
password warning while running ‘guix search’.

Ludo’.

bug#45173: ‘guix search’ traverses all the modules when extra channels are used

[zimoun]

Hi Ludo,

On Fri, 11 Dec 2020 at 10:29, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> --8<---------------cut here---------------start------------->8---
> $ strace -o /tmp/,,s guix search sdfsdf
> $ grep gnu/services /tmp/,,s
> $ echo $?
> 1
> --8<---------------cut here---------------end--------------->8---

I do not know what is the point of the strace command, but this file
contains a lot:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 50	Dec 01 2020 23:31:01	(current)
  guix f4450e8
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: f4450e8ca909530ee3432710b82f97e23acc7fef
$ strace -o /tmp/,,s guix search sdfsdf    
$ cat /tmp/,,s | grep ice-9 | wc -l
448
$ cat /tmp/,,s | grep srfi | wc -l
330
$ cat /tmp/,,s | grep 'guix/build' | wc -l
129
$ cat /tmp/,,s | grep 'guix/scripts' | wc -l
12
$cat /tmp/,,s | grep 'guix/import' | wc -l
9
--8<---------------cut here---------------end--------------->8---

But no ’services’, indeed.  The 3 first are expected since they used
somehow by files in ’gnu/packages/’.  I am surprised by ’guix/scripts’
or ’guix/import’, the few I tried.


And I note a variation with the option ’-L’.

--8<---------------cut here---------------start------------->8---
$ mkdir -p /tmp/foo
$ ls /tmp/foo/
$ strace -o /tmp/,,l guix search sdfsdf -L /tmp/foo
$ wc -l /tmp/,,l /tmp/,,s
  10059 /tmp/,,l
   8745 /tmp/,,s
  18804 total
$ grep services /tmp/,,l
$ echo $?
1  
--8<---------------cut here---------------end--------------->8---

Well, I am not sure to understand…  And last, the bug:

--8<---------------cut here---------------start------------->8---
$ strace -o /tmp/,,g guix search sdfsdf -L ~/src/guix/guix-past/modules
$ cat /tmp/,,g | grep 'gnu/services' | wc -l
30
$ ag services ~/src/guix/guix-past/
$ echo $?
1
$ ls -1 ~/src/guix/guix/gnu/services/*.scm | wc -l
50
--8<---------------cut here---------------end--------------->8---


All the best,
simon

bug#45173: ‘guix search’ traverses all the modules when extra channels are used

[Ludovic Courtès]

Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

> On Fri, 11 Dec 2020 at 10:29, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>> --8<---------------cut here---------------start------------->8---
>> $ strace -o /tmp/,,s guix search sdfsdf
>> $ grep gnu/services /tmp/,,s
>> $ echo $?
>> 1
>> --8<---------------cut here---------------end--------------->8---
>
> I do not know what is the point of the strace command, but this file
> contains a lot:

The point :-) is to show that things that should definitely not be
loaded, such as gnu/services/*, are getting loaded when multiple
channels are in use.

Ludo’.

bug#45173: ‘guix search’ traverses all the modules when extra channels are used

[zimoun]

Hey,

On Fri, 11 Dec 2020 at 16:56, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

> The point :-) is to show that things that should definitely not be
> loaded, such as gnu/services/*, are getting loaded when multiple
> channels are in use.

What I have tried to show is that even with the default, «things that
should definitely not be loaded, such as guix/scripts/* or
guix/import/*» are getting loaded.  And worse, adding an empty
“--load-path” adds burden.  Multiple channels seems only one part of the
story. :-)

Cheers,
simon

bug#45173: ‘guix search’ traverses all the modules when extra channels are used

[Ludovic Courtès]

Ludovic Courtès <ludovic.courtes@inria.fr> skribis:

> But with multiple channels, all of the guile-module-union directory is
> traversed, and that includes *everything*:
>
> $ /tmp/past/bin/guix describe
> Generacio 1     Dec 11 2020 10:26:58    (nuna)
>   guix-past 829923f
>     repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
>     branch: master
>     commit: 829923f01f894f1e687735627025ada26230832f
>   guix f765577
>     repository URL: https://git.savannah.gnu.org/git/guix.git
>     branch: master
>     commit: f765577dec0340d035021d030f17b54a3a5fd6b1
> $ strace -o /tmp/,,s /tmp/past/bin/guix search sdfsdf
> $ grep gnu/services /tmp/,,s
> stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services.scm", {st_mode=S_IFREG|0444, st_size=41045, ...}) = 0
> stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services.go", {st_mode=S_IFREG|0444, st_size=420021, ...}) = 0
> openat(AT_FDCWD, "/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/lib/guile/3.0/site-ccache/gnu/services.go", O_RDONLY|O_CLOEXEC) = 14
> stat("/gnu/store/3jnlvi2akawnxd1jq6m58kxc7nj07phc-guix-module-union/share/guile/site/3.0/gnu/services/shepherd.scm", {st_mode=S_IFREG|0444, st_size=23156, ...}) = 0

False alarm!  This was caused by a #:use-module (gnu) statement in one
of the Guix-Past modules, fixed here:

  https://gitlab.inria.fr/guix-hpc/guix-past/-/commit/4f5856f85578a04991dc646a06279d51f85973bf

Ludo’.