bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

emacs-org-roam is installed in my default profile and all the other
emacs packages work with the emacs-next-pgtk package in the same
profile.
guix shell emacs-org-roam emacs-next-pgtk does not work, guix shell
emacs-org-roam emacs does.

Possibly related: when I had flatpak installed it was not showing up in
my profile's bin directory, so I had to open it through guix shell.

Maybe this is related to the bug where shell and build produce different
derivations?

current guix describe output (but this has been going on for at least a
month or so)
Generation 186	Feb 28 2023 02:57:13	(current)
  guix ff5fbcc
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: ff5fbcc19bce6e94ead0cc79b27ae8ed0307463d
  raingloom c75fd77
    repository URL: https://git.sr.ht/~raingloom/guix-packages
    branch: master
    commit: c75fd77980c7c5b45c91b0d921a25d8558266f41
  guix-science 5fc5e3e
    repository URL: https://github.com/guix-science/guix-science.git
    branch: master
    commit: 5fc5e3e855c298a692c62129e9edd0fcc7c6949f
  nonguix 3a1e9f0
    repository URL: https://gitlab.com/nonguix/nonguix.git
    branch: master
    commit: 3a1e9f0507d99403d1ba0f5557ee868b95b61eef

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Liliana Marie Prikler]

Am Mittwoch, dem 01.03.2023 um 02:58 +0000 schrieb Csepp:
> emacs-org-roam is installed in my default profile and all the other
> emacs packages work with the emacs-next-pgtk package in the same
> profile.
> guix shell emacs-org-roam emacs-next-pgtk does not work, guix shell
> emacs-org-roam emacs does.
> 
> Possibly related: when I had flatpak installed it was not showing up
> in my profile's bin directory, so I had to open it through guix
> shell.
> 
> Maybe this is related to the bug where shell and build produce
> differen derivations?
You probably messed up some of your paths.  As a hint,
  guix shell --pure -E DISPLAY emacs-next-pgtk emacs-org-roam -- emacs
should run without any issues (apart from your init.el not working as
intended – in that case, supply -Q).

Try adding --check to your invocation of guix shell to see whether your
rc files are borked.

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> writes:

> Am Mittwoch, dem 01.03.2023 um 02:58 +0000 schrieb Csepp:
>> emacs-org-roam is installed in my default profile and all the other
>> emacs packages work with the emacs-next-pgtk package in the same
>> profile.
>> guix shell emacs-org-roam emacs-next-pgtk does not work, guix shell
>> emacs-org-roam emacs does.
>> 
>> Possibly related: when I had flatpak installed it was not showing up
>> in my profile's bin directory, so I had to open it through guix
>> shell.
>> 
>> Maybe this is related to the bug where shell and build produce
>> differen derivations?
> You probably messed up some of your paths.  As a hint,
>   guix shell --pure -E DISPLAY emacs-next-pgtk emacs-org-roam -- emacs
> should run without any issues (apart from your init.el not working as
> intended – in that case, supply -Q).
>
> Try adding --check to your invocation of guix shell to see whether your
> rc files are borked.
How the hell would my paths affect what's in the bin folder?  Like, the
flatpak binary is literally not present in the profile, that's why it's
not showing up in $PATH.
Well, I'll check next time I'm on that machine, right now I'm writing
from the netbook and I'm in a bit of a hurry and don't want to wait for
derivation computations.
But I'm like 99% sure it's not due to my paths, because all other emacs
packages work.

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[bokr]

Hi,

On +2023-03-01 12:16:56 +0100, Csepp wrote:
[...]
> How the hell would my paths affect what's in the bin folder?  Like, the
> flatpak binary is literally not present in the profile, that's why it's
> not showing up in $PATH.

Could something in one of your path directories
accidentally have gotten a name starting with '-' ?
(or full name '-')?

Surprising things can happen depending on how an
app rejects an unexpected option, or tries to use it :)

BTW: If you can't delete a file named '-'
     try emacs dirmode. IIRC emacs seems to see
     anything and be able to delete it.

Do you have any scripts that are both sourced and executed?
If so, are they doing return or exit respectively or
something trickier as intended?

--
Regards,
Bengt Richter

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

bokr@bokr.com writes:

> Hi,
>
> On +2023-03-01 12:16:56 +0100, Csepp wrote:
> [...]
>> How the hell would my paths affect what's in the bin folder?  Like, the
>> flatpak binary is literally not present in the profile, that's why it's
>> not showing up in $PATH.
>
> Could something in one of your path directories
> accidentally have gotten a name starting with '-' ?
> (or full name '-')?
>
> Surprising things can happen depending on how an
> app rejects an unexpected option, or tries to use it :)
>
> BTW: If you can't delete a file named '-'
>      try emacs dirmode. IIRC emacs seems to see
>      anything and be able to delete it.
>
> Do you have any scripts that are both sourced and executed?
> If so, are they doing return or exit respectively or
> something trickier as intended?

So, first things first:
% guix package -I | grep -E '(flatpak|roam)'
emacs-org-roam      	2.2.2-0.74422df   	out       	/gnu/store/bxxjy8ydm62fr0bckxfrj27xnlvqbfmy-emacs-org-roam-2.2.2-0.74422df
flatpak             	1.14.1            	out       	/gnu/store/mf0k987xvpgk79l74lmdjv9jz8gy8cdf-flatpak-1.14.1

Both are installed.

ls $(guix build flatpak)/bin/
flatpak  flatpak-bisect  flatpak-coredumpctl

The store paths also match.

This path also exists:
$(guix build emacs-org-roam)/share/emacs/site-lisp/org-roam-2.2.2-0.74422df

I don't know the details of how Emacs loads things, but org-roam has an
org-roam-autoloads.el while lsp-mode (a package that does work) does
not.

Some relevant paths:
EMACSLOADPATH=/home/raingloom/.guix-profile/share/emacs/site-lisp
PATH=/home/raingloom/.local/bin:/run/setuid-programs:/home/raingloom/.config/guix/current/bin:/home/raingloom/.guix-profile/bin:/home/raingloom/.guix-profile/sbin:/run/current-system/profile/bin:/run/current-system/profile/sbin

The only additional item is on PATH and at worst it would shadow an
existing flatpak binary.  But it doesn't, there is just a single utility
script there that I should probably get rid of.

Also:
guix shell --check -p .guix-profile
This seems to hang.  Or at least it's taking an awful long time for
something so simple.

I'm running Guix as my system distro and I'm not in the habit of adding
random junk to my dotfiles, so I would be rather surprised if this
turned out to be a path issue, especially when it's obvious the packages
are literally not even showing up in the profile they are supposed to
show up in.

% ls ~/.guix-profile/share/emacs/site-lisp/

This does not print any org-roam directory.

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Liliana Marie Prikler]

Am Mittwoch, dem 01.03.2023 um 17:11 +0100 schrieb Csepp:
> 
> bokr@bokr.com writes:
> 
> > Hi,
> > 
> > On +2023-03-01 12:16:56 +0100, Csepp wrote:
> > [...]
> > > How the hell would my paths affect what's in the bin folder? 
> > > Like, the flatpak binary is literally not present in the profile,
> > > that's why it's not showing up in $PATH.
> > 
> > Could something in one of your path directories
> > accidentally have gotten a name starting with '-' ?
> > (or full name '-')?
> > 
> > Surprising things can happen depending on how an
> > app rejects an unexpected option, or tries to use it :)
> > 
> > BTW: If you can't delete a file named '-'
> >      try emacs dirmode. IIRC emacs seems to see
> >      anything and be able to delete it.
> > 
> > Do you have any scripts that are both sourced and executed?
> > If so, are they doing return or exit respectively or
> > something trickier as intended?
> 
> So, first things first:
> % guix package -I | grep -E '(flatpak|roam)'
> emacs-org-roam          2.2.2-
> 0.74422df         out             /gnu/store/bxxjy8ydm62fr0bckxfrj27x
> nlvqbfmy-emacs-org-roam-2.2.2-0.74422df
> flatpak                 1.14.1                  out             /gnu/
> store/mf0k987xvpgk79l74lmdjv9jz8gy8cdf-flatpak-1.14.1
> 
> Both are installed.
> 
> ls $(guix build flatpak)/bin/
> flatpak  flatpak-bisect  flatpak-coredumpctl
> 
> The store paths also match.
> 
> This path also exists:
> $(guix build emacs-org-roam)/share/emacs/site-lisp/org-roam-2.2.2-
> 0.74422df
You're comparing apples to oranges here.  `guix build' need not
reproduce the contents of your profile, especially if you pulled a new
version of guix in between.  Instead, try listing the contents of the
reported store paths.

As for `guix build' and `guix shell' producing different results within
a single generation, the only instance I know of which has this
requires the presence of grafts (and IIRC might already be fixed?)

> I don't know the details of how Emacs loads things, but org-roam has
> an org-roam-autoloads.el while lsp-mode (a package that does work)
> does not.
There ought to be a subdirs.el in your $GUIX_PROFILE/share/emacs/site-
lisp, which explicitly mentions the directories to add to your load
path.  These are store paths.  Thus, even in the off chance that some
symlink in your profile is gone (which I'd find highly alerting in the
first place), it should correctly see the package to add.

> Some relevant paths:
> EMACSLOADPATH=/home/raingloom/.guix-profile/share/emacs/site-lisp
> PATH=/home/raingloom/.local/bin:/run/setuid-
> programs:/home/raingloom/.config/guix/current/bin:/home/raingloom/.gu
> ix-profile/bin:/home/raingloom/.guix-profile/sbin:/run/current-
> system/profile/bin:/run/current-system/profile/sbin
> 
> The only additional item is on PATH and at worst it would shadow an
> existing flatpak binary.  But it doesn't, there is just a single
> utility
> script there that I should probably get rid of.
> 
> Also:
> guix shell --check -p .guix-profile
> This seems to hang.  Or at least it's taking an awful long time for
> something so simple.
You could try something easier like the aforementioned `guix shell
emacs-next-pgtk emacs-org-roam --pure --check'.

> I'm running Guix as my system distro and I'm not in the habit of
> adding random junk to my dotfiles, so I would be rather surprised if
> this turned out to be a path issue
Good to know.

> especially when it's obvious the packages are literally not even
> showing up in the profile they are supposed to show up in.
This was not obvious from your previous report, which see
> emacs-org-roam is installed in my default profile and all the other
> emacs packages work with the emacs-next-pgtk package in the same
> profile.
> guix shell emacs-org-roam emacs-next-pgtk does not work, guix shell
> emacs-org-roam emacs does.
Basing my response on this rather than the otherwise inconsequential
bit about flatpak, it would appear as though you are reporting a bug
specific to emacs-next-pgtk rather than emacs-org-roam.

> % ls ~/.guix-profile/share/emacs/site-lisp/
> 
> This does not print any org-roam directory.
Which leads me to believe that
$ ls /gnu/store/bxxjy8ydm62fr0bckxfrj27xnlvqbfmy-emacs-org-roam-2.2.2-
0.74422df/share/emacs/site-lisp
does not report any such directory either.

If that is indeed the case, try `guix build --repair'-ing it.

This still does not explain the different behaviour of emacs vs. emacs-
next-pgtk in your shell, which has further debugging complexities due
to the lack of isolation.

Cheers

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

Liliana Marie Prikler <liliana.prikler@ist.tugraz.at> writes:

>> % ls ~/.guix-profile/share/emacs/site-lisp/
>> 
>> This does not print any org-roam directory.
> Which leads me to believe that
> $ ls /gnu/store/bxxjy8ydm62fr0bckxfrj27xnlvqbfmy-emacs-org-roam-2.2.2-
> 0.74422df/share/emacs/site-lisp
> does not report any such directory either.

It definitely reports it, I just didn't paste the result to save space.

I'll try running a repair when I get home, since FS corruption could
lead to any kind of weird behaviour, but I haven't had any corruption on
that machine ever, BTRFS has been pretty reliable for me.

Sorry for being so grumpy, I guess I should have highlighted the profile
corruption thing over the emacs-next-pgtk specific bits.

bug#61882: profile is frozen / packages can't be installed

[Csepp]

(Jump forward a bit, this is a bit stream-of-consciousness-y, I wrote
things down as I was debugging things.  TLDR: profile got corrupted
somehow, it's not related to the packages themselves.)

Certain packages like flatpak do not get installed in my main user
profile for some unknown reason.
So far they are:
* gallery-dl
* flatpak
* emacs-org-roam

It has been happening for at least a month across several pulls.

If I export a manifest and load it in either guix shell -m or guix
package -m to a different profile, bin/flatpak exists, if I do guix
package -m without a profile argument or with the profile set to the
default user profile, bin/flatpak is missing.

The packages that are broken are always the same.

If I create a manifest with only the broken packages, guix package -I
reports exactly those packages, but when I look in ~/.guix-profile/bin
it still has a bunch of other packages in it.

So it seems the profile is frozen?  I have no idea how this could
happen.

guix package --list-generations reports three generations, but there is
only one generation in my home directory, called .guix-profile-1-link.

There is also a .guix-profile.lock, maybe that's related?
I see no lock for the other profile with the same manifest.

Removed the lock, tried guix package -m again, still don't have flatpak,
still only one generation symlink.

Deleted all the symlinks, ran guix package -m, now it works.

I'm gonna hold off on running the GC for a few days, if anyone has an
idea of where the bug is and wants me to upload some files, I can do it
until then.
For my own reference, this is the store item of the broken profile:
/gnu/store/0w3jxl95cchxn14zph3lmnqwmijf8971-profile

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Maxim Cournoyer]

tags 61882 +notabug
quit

Hello,

Csepp <raingloom@riseup.net> writes:

> (Jump forward a bit, this is a bit stream-of-consciousness-y, I wrote
> things down as I was debugging things.  TLDR: profile got corrupted
> somehow, it's not related to the packages themselves.)
>
> Certain packages like flatpak do not get installed in my main user
> profile for some unknown reason.
> So far they are:
> * gallery-dl
> * flatpak
> * emacs-org-roam
>
> It has been happening for at least a month across several pulls.
>
> If I export a manifest and load it in either guix shell -m or guix
> package -m to a different profile, bin/flatpak exists, if I do guix
> package -m without a profile argument or with the profile set to the
> default user profile, bin/flatpak is missing.
>
> The packages that are broken are always the same.
>
> If I create a manifest with only the broken packages, guix package -I
> reports exactly those packages, but when I look in ~/.guix-profile/bin
> it still has a bunch of other packages in it.
>
> So it seems the profile is frozen?  I have no idea how this could
> happen.
>
> guix package --list-generations reports three generations, but there is
> only one generation in my home directory, called .guix-profile-1-link.
>
> There is also a .guix-profile.lock, maybe that's related?
> I see no lock for the other profile with the same manifest.
>
> Removed the lock, tried guix package -m again, still don't have flatpak,
> still only one generation symlink.
>
> Deleted all the symlinks, ran guix package -m, now it works.
>
> I'm gonna hold off on running the GC for a few days, if anyone has an
> idea of where the bug is and wants me to upload some files, I can do it
> until then.
> For my own reference, this is the store item of the broken profile:
> /gnu/store/0w3jxl95cchxn14zph3lmnqwmijf8971-profile

Did you have any corruption at the FS level or something equally bad?
Were you using a different Guix in between the 'guix package -m'
attempts?  If the profile was in the cache, it wouldn't rebuild it,
and you'd be stuck with your broken version.

I'm closing this, as I we don't have much to push the
investigation further, but if you have a more precise idea of what
happened and ways to reproduce that do reopen.

-- 
Thanks,
Maxim

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> tags 61882 +notabug
> quit

I don't think notabug applies until we actually know the root cause.

> Hello,
>
> Csepp <raingloom@riseup.net> writes:
>
>> (Jump forward a bit, this is a bit stream-of-consciousness-y, I
>> wrote
>> things down as I was debugging things. TLDR: profile got corrupted
>> somehow, it's not related to the packages themselves.)
>>
>> Certain packages like flatpak do not get installed in my main user
>> profile for some unknown reason.
>> So far they are:
>> * gallery-dl
>> * flatpak
>> * emacs-org-roam
>>
>> It has been happening for at least a month across several pulls.
>>
>> If I export a manifest and load it in either guix shell -m or guix
>> package -m to a different profile, bin/flatpak exists, if I do guix
>> package -m without a profile argument or with the profile set to the
>> default user profile, bin/flatpak is missing.
>>
>> The packages that are broken are always the same.
>>
>> If I create a manifest with only the broken packages, guix package
>> -I
>> reports exactly those packages, but when I look in
>> ~/.guix-profile/bin
>> it still has a bunch of other packages in it.
>>
>> So it seems the profile is frozen?  I have no idea how this could
>> happen.
>>
>> guix package --list-generations reports three generations, but there
>> is
>> only one generation in my home directory, called
>> .guix-profile-1-link.
>>
>> There is also a .guix-profile.lock, maybe that's related?
>> I see no lock for the other profile with the same manifest.
>>
>> Removed the lock, tried guix package -m again, still don't have
>> flatpak,
>> still only one generation symlink.
>>
>> Deleted all the symlinks, ran guix package -m, now it works.
>>
>> I'm gonna hold off on running the GC for a few days, if anyone has
>> an
>> idea of where the bug is and wants me to upload some files, I can do
>> it
>> until then.
>> For my own reference, this is the store item of the broken profile:
>> /gnu/store/0w3jxl95cchxn14zph3lmnqwmijf8971-profile
>
> Did you have any corruption at the FS level or something equally bad?
> Were you using a different Guix in between the 'guix package -m'
> attempts?  If the profile was in the cache, it wouldn't rebuild it,
> and you'd be stuck with your broken version.
>
> I'm closing this, as I we don't have much to push the
> investigation further, but if you have a more precise idea of what
> happened and ways to reproduce that do reopen.

I'm pretty sure I already went through these, but once again, to be sure:
- no there was no file system corruption as far as I could tell
- store was checked for corruption multiple times
- the issue persisted through multiple guix pulls
- only one profile was affected

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Maxim Cournoyer]

Hi,

Csepp <raingloom@riseup.net> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> tags 61882 +notabug
>> quit
>
> I don't think notabug applies until we actually know the root cause.

Sadly I don't think there's anything actionable here until you can
reproduce the problem and share the recipe with us, so I wanted to close
the issue without it being marked as "resolved".

-- 
Thanks,
Maxim

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hi,
>
> Csepp <raingloom@riseup.net> writes:
>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>
>>> tags 61882 +notabug
>>> quit
>>
>> I don't think notabug applies until we actually know the root cause.
>
> Sadly I don't think there's anything actionable here until you can
> reproduce the problem and share the recipe with us, so I wanted to close
> the issue without it being marked as "resolved".

Neither "resolved" nor "notabug" are applicable.  If stalled incident
reports / issues are a problem, they should probably be marked as
stalled, or needinfo, for easy filtering.  Marking it as notabug is just
going to make the job of the next person harder when they search for
issues related to these symptoms.

I appreciate all the work going into closing old issues, but I don't
think chasing a low open issue count should be a goal unto itself
See https://fvsch.com/stale-bots .

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Maxim Cournoyer]

tags 61882 = moreinfo unreproducible
quit

Hi,

Csepp <raingloom@riseup.net> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>> Hi,
>>
>> Csepp <raingloom@riseup.net> writes:
>>
>>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>>
>>>> tags 61882 +notabug
>>>> quit
>>>
>>> I don't think notabug applies until we actually know the root cause.
>>
>> Sadly I don't think there's anything actionable here until you can
>> reproduce the problem and share the recipe with us, so I wanted to close
>> the issue without it being marked as "resolved".
>
> Neither "resolved" nor "notabug" are applicable.  If stalled incident
> reports / issues are a problem, they should probably be marked as
> stalled, or needinfo, for easy filtering.  Marking it as notabug is just
> going to make the job of the next person harder when they search for
> issues related to these symptoms.

I don't think a bug as particular as 'my profile got corrupted' without
any way to recreate it has much value; it's also the first time I've
heard of such a report.  That's why I'd prefer to treat it as an oddity
and close it; if it reproduces (by you or others) let's reopen it, with
fresh and clear information.

> I appreciate all the work going into closing old issues, but I don't
> think chasing a low open issue count should be a goal unto itself
> See https://fvsch.com/stale-bots .

To be clear, I wholly agree.  I've now tagged it as moreinfo and
unreproducible.

-- 
Thanks,
Maxim

bug#61882: emacs-next-pgtk does not find emacs-org-roam, other path issues

[Csepp]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> tags 61882 = moreinfo unreproducible
> quit
>
> Hi,
>
> Csepp <raingloom@riseup.net> writes:
>
>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>
>>> Hi,
>>>
>>> Csepp <raingloom@riseup.net> writes:
>>>
>>>> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>>>>
>>>>> tags 61882 +notabug
>>>>> quit
>>>>
>>>> I don't think notabug applies until we actually know the root
>>>> cause.
>>>
>>> Sadly I don't think there's anything actionable here until you can
>>> reproduce the problem and share the recipe with us, so I wanted to
>>> close
>>> the issue without it being marked as "resolved".
>>
>> Neither "resolved" nor "notabug" are applicable. If stalled incident
>> reports / issues are a problem, they should probably be marked as
>> stalled, or needinfo, for easy filtering. Marking it as notabug is
>> just
>> going to make the job of the next person harder when they search for
>> issues related to these symptoms.
>
> I don't think a bug as particular as 'my profile got corrupted'
> without
> any way to recreate it has much value; it's also the first time I've
> heard of such a report. That's why I'd prefer to treat it as an oddity
> and close it; if it reproduces (by you or others) let's reopen it,
> with
> fresh and clear information.
>
>> I appreciate all the work going into closing old issues, but I don't
>> think chasing a low open issue count should be a goal unto itself
>> See https://fvsch.com/stale-bots .
>
> To be clear, I wholly agree.  I've now tagged it as moreinfo and
> unreproducible.

It could be a file system bug, but while there are reports of BTRFS
being unstable in certain RAID modes, I would be very surprised if there
was a data corruption issue in the default single device mode.

My hunch is that Guix's profile update logic is not actually as atomic
as it's advertised and I interrupted it at the wrong moment.