Emacs not seeing newer packages after pull

Emacs not seeing newer packages after pull

[Ivan Vilata i Balaguer]

Hi everyone!  Recently it caught my attention that emacs-guix seems to always
be on the *previous* generation of Guix, i.e. the one before the latest pull.
That is, when I do a `guix pull` and check for new packages and versions in
`guix pull --news`, when I go to the emacs-guix packages list, these packages
and versions are not there; the ones from the previous Guix generation are
instead.

Actually, if I upgrade or install such new packages from the command line,
some packages under emacs-guix appear as "from the future".

Restaring Emacs or pulling from it doesn't seem to fix things.  I also came
across the `guix-repl-use-latest` variable, but it's already `t` (in fact,
`ps` shows the right `~/.config/guix/current` paths being passed to `guile`).

I'm running Guix on top of Debian, and I also have package `guix` installed in
my profile (if I remove it, it gets even worse: the Emacs installed package
list shows most packages as "unknown").

Is this behavior expected?  Does it also happen to you?  (Hint: do you see
many blue packages "from the future" in the Emacs list.)  Maybe I'm doing
something wrong?

Thanks in advance for your help!

-- 
Ivan Vilata i Balaguer -- https://elvil.net/

Re: Emacs not seeing newer packages after pull

[Konrad Hinsen]

Hi Ivan,

> Hi everyone!  Recently it caught my attention that emacs-guix seems to
> always be on the *previous* generation of Guix, i.e. the one before
> the latest pull.  That is, when I do a `guix pull` and check for new
...

> Is this behavior expected?  Does it also happen to you?  

No, and yes. Meaning that I most definitely didn't expect it, but I
confirm your observations. And I am running Guix System, so it's not
related to your Guix-under-Debian installation.

In fact, when I saw your mail, it reminded me of similar issues I had
noticed in the past but never taken the time to explore in more depth.

When trying to analyze this in more detail, I noticed that emacs-guix
runs guile 2.2, whereas guix itself used guile 3.0. That shouldn't make
a difference (my understanding is that the current Guix source code
works with both Guile releases), but it's a bit surprising as well.

Cheers,
  Konrad.

Re: Emacs not seeing newer packages after pull

[Konrad Hinsen]

Konrad Hinsen <konrad.hinsen@fastmail.net> writes:

> No, and yes. Meaning that I most definitely didn't expect it, but I
> confirm your observations. And I am running Guix System, so it's not
> related to your Guix-under-Debian installation.

I did some experiments in the Guix REPL created by emacs-guix, but
I don't understand what's happening. Here I am exploring
emacs-caps-lock, a package that I got with today's pull:

scheme@(emacs-guix)> (%search-load-path "gnu/packages/emacs-xyz")
$4 = "/home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm"

Looks OK. Let's look at the module and its filename:

scheme@(emacs-guix)> (resolve-module '(gnu packages emacs-xyz))
$5 = #<directory (gnu packages emacs-xyz) 7fb01ffb3280>
scheme@(emacs-guix)> (module-filename (resolve-module '(gnu packages emacs-xyz)))
$6 = "gnu/packages/emacs-xyz.scm"

A relative filename, but relative to what? Why isn't this an absolute
one? As for the package:

scheme@(emacs-guix)> (module-variable (resolve-module '(gnu packages emacs-xyz)) 'emacs-caps-lock)
$7 = #f

It's not in there (but it is in the source code). And now for the real
surprise:

scheme@(emacs-guix)> (reload-module (resolve-module '(gnu packages emacs-xyz)))
$8 = #<directory (gnu packages emacs-xyz) 7fb01ffb3280>
scheme@(emacs-guix)> (module-filename (resolve-module '(gnu packages emacs-xyz)))
$9 = "/home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm"
scheme@(emacs-guix)> (module-variable (resolve-module '(gnu packages emacs-xyz)) 'emacs-caps-lock)
$10 = #<variable 7fb010e61210 value: #<package emacs-caps-lock@1.0 /home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm:1745 7fb01398ef20>>

What the heck has Guile been loading the first time???

Cheers,
  Konrad

Re: Emacs not seeing newer packages after pull

[Ivan Vilata i Balaguer]

Konrad Hinsen (2020-03-31 20:25:02 +0200) wrote:

> I did some experiments in the Guix REPL created by emacs-guix, but
> I don't understand what's happening. Here I am exploring
> emacs-caps-lock, a package that I got with today's pull:
> 
> scheme@(emacs-guix)> (%search-load-path "gnu/packages/emacs-xyz")
> $4 = "/home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm"
> 
> Looks OK. Let's look at the module and its filename:
> 
> scheme@(emacs-guix)> (resolve-module '(gnu packages emacs-xyz))
> $5 = #<directory (gnu packages emacs-xyz) 7fb01ffb3280>
> scheme@(emacs-guix)> (module-filename (resolve-module '(gnu packages emacs-xyz)))
> $6 = "gnu/packages/emacs-xyz.scm"
> 
> A relative filename, but relative to what? Why isn't this an absolute
> one? As for the package:
> 
> scheme@(emacs-guix)> (module-variable (resolve-module '(gnu packages emacs-xyz)) 'emacs-caps-lock)
> $7 = #f
> 
> It's not in there (but it is in the source code). And now for the real
> surprise:
> 
> scheme@(emacs-guix)> (reload-module (resolve-module '(gnu packages emacs-xyz)))
> $8 = #<directory (gnu packages emacs-xyz) 7fb01ffb3280>
> scheme@(emacs-guix)> (module-filename (resolve-module '(gnu packages emacs-xyz)))
> $9 = "/home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm"
> scheme@(emacs-guix)> (module-variable (resolve-module '(gnu packages emacs-xyz)) 'emacs-caps-lock)
> $10 = #<variable 7fb010e61210 value: #<package emacs-caps-lock@1.0 /home/hinsen/.config/guix/current/share/guile/site/3.0/gnu/packages/emacs-xyz.scm:1745 7fb01398ef20>>
> 
> What the heck has Guile been loading the first time???

Thanks Konrad for investigating this!  After seeing your tests, I came back to
the `guix-repl-use-latest` variable.  According to info:

> Set this variable to nil, if you don’t want to use the latest Guix code
> received with ‘guix pull’ command.

My variable is `t` as defined at emacs-guix's `guix-repl.el`.  It is used in
the `guix-repl-guile-args` to set the command line args for Guile.  According
to `ps` in my system, I see these arguments:

    /gnu/store/…-guile-2.2.6/bin/guile --no-auto-compile \
      -L /gnu/store/…-emacs-guix-0.5.2/share/guile/site/2.2 \
      -C /gnu/store/…-emacs-guix-0.5.2/lib/guile/2.2/site-ccache \
      -L /home/ivan/.config/guix/current/share/guile/site/3.0 \
      -C /home/ivan/.config/guix/current/lib/guile/3.0/site-ccache \
      -L /gnu/store/…-guix-1.0.1-14.c2f9ea2/share/guile/site/2.2 \
      -C /gnu/store/…-guix-1.0.1-14.c2f9ea2/lib/guile/2.2/site-ccache \
      --listen=/tmp/emacs-guix-…/repl-socket -q \
      -L /gnu/store/…-emacs-geiser-0.11.2/share/geiser/guile/

Please note the `-L …-guix-1.0.1-14… -C …-guix-1.0.1-14…`, this is added in
the `guix-repl-guile-args` function from the value of the variable
`guix-config-guix-scheme-directory`, defined in emacs-guix's
`guix-build-config.el` as
`/gnu/store/…-guix-1.0.1-14.c2f9ea2/share/guile/site/2.2`.

If I run `guix package -s guix`, the version of the `guix` package (which I
assume is the same that I got after the last `guix pull`, but I may be wrong)
is `1.0.1-15…`.

I guess that the Guile binary and `…-guix-1.0.1-14…` coming from emacs-guix
are unavoidable, the issue may be whether the `.config/guix/current` paths are
really taking precedence over emacs-guix's hardwired ones.

Now, if I do `guix package -u guix emacs-guix emacs-magit OTHER_DEPS…`, run
Emacs and invoke the package list, `ps` shows:

    /gnu/store/…-guile-2.2.6/bin/guile --no-auto-compile \
        -L /gnu/store/…-emacs-guix-0.5.2/share/guile/site/2.2 \
        -C /gnu/store/…-emacs-guix-0.5.2/lib/guile/2.2/site-ccache \
        -L /home/ivan/.config/guix/current/share/guile/site/3.0 \
        -C /home/ivan/.config/guix/current/lib/guile/3.0/site-ccache \
        --listen=/tmp/emacs-guix-…/repl-socket -q \
        -L /gnu/store/…-emacs-geiser-0.11.2/share/geiser/guile/

Please note that the hardwired Guix paths are no longer there.  Now the
package list only shows `guix` and `emacs-magit` as "from the future" (why?),
and some "obsolete" packages, which is fine.

Maybe I should just `guix package -u` this time and check if the problem
repeats with the next pull…

Thanks again!

-- 
Ivan Vilata i Balaguer -- https://elvil.net/

Re: Emacs not seeing newer packages after pull

[Konrad Hinsen]

Hi Ivan,

> My variable is `t` as defined at emacs-guix's `guix-repl.el`.  It is used in
> the `guix-repl-guile-args` to set the command line args for Guile.  According
> to `ps` in my system, I see these arguments:
...

Interesting. That looks very different for me on Guix System! But then,
I don't have "guix" in my profile. Not sure it's a good idea.

> If I run `guix package -s guix`, the version of the `guix` package (which I
> assume is the same that I got after the last `guix pull`, but I may be wrong)
> is `1.0.1-15…`.

If you have "guix" in your profile, you'd have to update your profile
after a pull to run the latest version.

> Maybe I should just `guix package -u` this time and check if the problem
> repeats with the next pull…

That sounds like a good experiment to do.

Cheers,
  Konrad

Re: Emacs not seeing newer packages after pull

[Ivan Vilata i Balaguer]

Konrad Hinsen (2020-04-02 09:20:58 +0200) wrote:

> Hi Ivan,
> 
> […] Interesting. That looks very different for me on Guix System! But then,
> I don't have "guix" in my profile. Not sure it's a good idea.
> 
> > If I run `guix package -s guix`, the version of the `guix` package (which I
> > assume is the same that I got after the last `guix pull`, but I may be wrong)
> > is `1.0.1-15…`.
> 
> If you have "guix" in your profile, you'd have to update your profile
> after a pull to run the latest version.
> 
> > Maybe I should just `guix package -u` this time and check if the problem
> > repeats with the next pull…
> 
> That sounds like a good experiment to do.

So I did `guix package -u` and `emacs-guix` continued to show the list of
packages and references to package definitions from the previous pull.  I then
confirmed that definitions belonged to the `guix` package *in my profile*
while, for instance, `guix package -s whatever` showed references to files *in
my pulled Guix*.

Anyway, that kinda confirmed what a bad idea keeping `guix` in my profile was
(it landed there in some of my first generations).  So I removed it, pulled
and upgraded packages again.

However (as I had tested before), `emacs-guix` now shows a lot of "unknown"
pink packages in my installed list.  Some of the few green ones can't be
opened because of unbound variables, others can be opened but have empty
descriptions with just a link to their store directory… but some do have
descriptions and a valid reference (absolute path) to their definition and, lo
and behold, the files are under my pulled Guix.  Yay!  However, `guix-pull` in
Emacs still fails with a bunch of "incompatible bytecode kind" errors then
"guix: pull: command not found".

So I guess the proper way to go now is to report why `emacs-guix` on a foreign
distro doesn't really work.  I'll look at the archives and open a new thread
if needed.

Thanks anyway Konrad for your help!

-- 
Ivan Vilata i Balaguer -- https://elvil.net/

Re: Emacs not seeing newer packages after pull

[Konrad Hinsen]

Hi Ivan,

> and behold, the files are under my pulled Guix.  Yay!  However, `guix-pull` in
> Emacs still fails with a bunch of "incompatible bytecode kind" errors then
> "guix: pull: command not found".

The "incompatible byte code" messages are due to emacs-guix using Guile
2.2 whereas Guix is using Guile 3. They will disappear in the ongoing
transition of everything to Guile 3. And they don't signal any real
problem. It's the last message that is critical.

> So I guess the proper way to go now is to report why `emacs-guix` on a foreign
> distro doesn't really work.  I'll look at the archives and open a new thread
> if needed.

Emacs-guix is a project separate from Guix, so it's best to open an
issue at https://gitlab.com/emacs-guix.

Cheers,
  Konrad

Re: Emacs not seeing newer packages after pull

[Ivan Vilata i Balaguer]

Konrad Hinsen (2020-04-07 10:16:28 +0200) wrote:

> > and behold, the files are under my pulled Guix.  Yay!  However, `guix-pull` in
> > Emacs still fails with a bunch of "incompatible bytecode kind" errors then
> > "guix: pull: command not found".
> 
> The "incompatible byte code" messages are due to emacs-guix using Guile
> 2.2 whereas Guix is using Guile 3. They will disappear in the ongoing
> transition of everything to Guile 3. And they don't signal any real
> problem. It's the last message that is critical.

Indeed.  I see that someone already reported Guile 3 incompatibilities in
<https://github.com/alezost/guix.el/issues/37>.  I'll keep an eye on that.

> > So I guess the proper way to go now is to report why `emacs-guix` on a foreign
> > distro doesn't really work.  I'll look at the archives and open a new thread
> > if needed.
> 
> Emacs-guix is a project separate from Guix, so it's best to open an
> issue at https://gitlab.com/emacs-guix.

Thanks for the pointer!  I see that after GitHub issues #28 and #29, Alex
added an explicit warning for foreign distro users here:
<https://github.com/alezost/guix.el#important-note-for-non-guixsd-users>.
I'll report that they result in `emacs-guix` seeing packages from the previous
pull instead.

Thanks again!

-- 
Ivan Vilata i Balaguer -- https://elvil.net/

Re: Emacs not seeing newer packages after pull

[Ivan Vilata i Balaguer]

Ivan Vilata i Balaguer (2020-04-07 15:44:05 -0400) wrote:

> > > So I guess the proper way to go now is to report why `emacs-guix` on a
> > > foreign distro doesn't really work.  I'll look at the archives and open
> > > a new thread if needed.
> > 
> > Emacs-guix is a project separate from Guix, so it's best to open an
> > issue at https://gitlab.com/emacs-guix.
> 
> Thanks for the pointer!  I see that after GitHub issues #28 and #29, Alex
> added an explicit warning for foreign distro users here:
> <https://github.com/alezost/guix.el#important-note-for-non-guixsd-users>.
> I'll report that they result in `emacs-guix` seeing packages from the
> previous pull instead.

FYI: <https://github.com/alezost/guix.el/issues/38>

Cheers!

-- 
Ivan Vilata i Balaguer -- https://elvil.net/