GNOME in Guix

GNOME in Guix

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 3162 bytes --]

Hi,

On Mon, 02 Nov 2020 08:44:29 +0100
Pierre Neidhardt <mail@ambrevar.xyz> wrote:

> Danny Milosavljevic <dannym@scratchpost.org> writes:
> > Not much more works yet because I've hit this (design) bug in Guix and/or GNOME:
> >
> > * https://github.com/spk121/guile-gi/issues/96  
> 
> Have you tried g-golf?  The Nomad browser uses it, it may not suffer
> from the aforementioned issue.
> Thoughts on that?

It really depends on what you mean.  If it also uses gobject-introspection, it
will have the same problem.

Even gobject-introspection links to glib internally, then registers a class in
that, THEN opens (potentially another) glib at runtime using dlopen in the same
address same.
I'm pretty sure that GNOME is not designed for that.  If you then use the
dlopened glib to use the first-mentioned class, good luck with that.

The only reason it didn't fall onto the head of users of other distributions
is because those don't support internal dependencies of packages at all--so
you always have to update all in lockstep.  So the following can only happen
in Guix:

If a library A has an input Q, and library B has the same input Q, and you
use A and B in your program, and then you (guix-) update just B to use Q'
(for example a newer version of Q) and recompile your program, then you have
both Q and Q' in your process address space!

Q is an internal dependency of A, and by coincidence it's an internal
dependency of B, too. Just because the internal dependency of B changed
shouldn't change the internal depenency of A--that's what "internal"
means.

Otherwise the modules aren't really modular (because they then don't isolate
from each other).

I think that the gobject type registry does not consider that it can happen
to have to classes with the same class name, registered in respective
internal dependencies.  The objects of the internal dependency shouldn't
be available to the outside program in the first place.

That means actually we should propagate a lot of GNOME inputs, which means
in practice what a regular user has in his profile will be very much like
in other distributions, and internal dependencies will become a weird corner
feature nobody uses.

Does anyone know how to make dlopen fail on duplicate global variables?

I tried to make it fail but I can't get it to work:

I tried:

File d1.c contains:
int x;

File d2.c contains:
int x;

File a.c contains:

#include <stdio.h>
#include <dlfcn.h>

// https://stackoverflow.com/questions/43582165/handling-global-variables-in-shared-object

int main() {
        if (!dlopen("./d1.so", RTLD_NOW | RTLD_GLOBAL /*| RTLD_DEEPBIND*/)) {
                fprintf(stderr, "dlerror: %s\n", dlerror());
                return 1;
        }
        if (!dlopen("./d2.so", RTLD_NOW | RTLD_GLOBAL /*| RTLD_DEEPBIND*/)) {
                fprintf(stderr, "dlerror: %s\n", dlerror());
                return 2;
        }
        return 0;
}

Then I do:

$ gcc -fPIC -shared -Wl,--export-dynamic -o d1.so d1.c
$ gcc -fPIC -shared -Wl,--export-dynamic -o d2.so d2.c
$ gcc a.c -ldl
$ ./a.out
$ echo $?
0

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

GNOME in Guix

[Leo Prikler]

Hi Danny,

Am Montag, den 02.11.2020, 11:17 +0100 schrieb Danny Milosavljevic:
> Hi,
> 
> On Mon, 02 Nov 2020 08:44:29 +0100
> Pierre Neidhardt <mail@ambrevar.xyz> wrote:
> 
> > Danny Milosavljevic <dannym@scratchpost.org> writes:
> > > Not much more works yet because I've hit this (design) bug in
> Guix and/or 
> > > GNOME:
> > >
> > > * https://github.com/spk121/guile-gi/issues/96  
> > 
> > Have you tried g-golf?  The Nomad browser uses it, it may not
> suffer
> > from the aforementioned issue.
> > Thoughts on that?
> 
> It really depends on what you mean.  If it also uses gobject-
> introspection, it
> will have the same problem.
As much as I rant about nomad in IRC now and then, it is still a rather
sophisticated program, and as a user I have not yet encountered any
problem similar in effect to what is described here.  
The bugs I do encounter whenever I try it out for my own amusement are
much rather due to g-golf and emacsy being in rather early stages of
development and therefore not feature-complete.

I am not sure, whether the Nomad devs have encountered a problem
similar to Guile-GI#96 (I myself am not one), but the Guile-GI devs
speculate, that the problem is somewhat specific to their
implementation.
They also mention, that you should still be able to prototype your
application by using "--no-grafts" for the development environment.  Do
you still encounter Guile-GI#96 after packaging?  If so, you might want
to explain your situation upstream, so that they can better help you.

> Even gobject-introspection links to glib internally, then registers a
> class in
> that, THEN opens (potentially another) glib at runtime using dlopen
> in the same
> address same.
> I'm pretty sure that GNOME is not designed for that.  If you then use
> the
> dlopened glib to use the first-mentioned class, good luck with that.
> 
> The only reason it didn't fall onto the head of users of other
> distributions
> is because those don't support internal dependencies of packages at
> all--so
> you always have to update all in lockstep.  So the following can only
> happen
> in Guix:
> 
> If a library A has an input Q, and library B has the same input Q,
> and you
> use A and B in your program, and then you (guix-) update just B to
> use Q'
> (for example a newer version of Q) and recompile your program, then
> you have
> both Q and Q' in your process address space!
> 
> Q is an internal dependency of A, and by coincidence it's an internal
> dependency of B, too. Just because the internal dependency of B
> changed
> shouldn't change the internal depenency of A--that's what "internal"
> means.
Sure, but that's only if either of them ends up using a different
version.  That should not be the case for Guix' GNOME stack apart from
some bootstrap shenanigans, that don't really matter at the point of
application packages.

In order to make use of any GI in Guix, you need
- the GI bindings (one of python-gobject, g-golf, guile-gi, gjs, etc.),
- *and* the packages of the libraries themselves
as inputs to your package.  So the resulting graph for your package
will always be complete and should only contain one relevant package
per library.

And yes, you can totally do that on other distros as well.  (One way to
do so would be to use Guix on them ;D)  It's just that users typically
shy away from installing conflicting versions of the same package,
whereas Guix will happily allow them to coexist and go out of its way
to deal with potential issues arising from that.

> I think that the gobject type registry does not consider that it can
> happen
> to have to classes with the same class name, registered in respective
> internal dependencies.  The objects of the internal dependency
> shouldn't
> be available to the outside program in the first place.

Yes it does, or at least you can check, whether a type of that name is
already registered and bail.  You could even do so in Guile-GI as you
are instantiating the types, but I doubt upstream devs want to bail
when they encounter a reserved type.

> That means actually we should propagate a lot of GNOME inputs, which
> means
> in practice what a regular user has in his profile will be very much
> like
> in other distributions, and internal dependencies will become a weird
> corner
> feature nobody uses.

No, it does not.  It perhaps means, that we might want to take care of
GI_TYPELIB_PATH in glib-or-gtk-build-system.  It at least would make
sense to add logic dealing with gobject-introspection to glib-or-gtk-
wrap, which could then be reused in other build systems as well.

Packages, be they GNOME or not, should be dynamically linked, so that
Guix can correctly graft them.  Assuming correct grafting, the library
loaded from GI_TYPELIB_PATH should be the same as the library loaded
internally with the build setup I discussed earlier. 

> Does anyone know how to make dlopen fail on duplicate global
> variables?
No, but I don't think that is too relevant here.  See my earlier point
on how it *is* possible to handle this with the methods GLib provides.

Am Donnerstag, den 29.10.2020, 20:34 +0100 schrieb Danny Milosavljevic:
> Or I could just use Gtk in C and use popen("guix ...").
You're probably joking, but you should know, that I had moderate
success passing around just data from Guile to Gtk while not using g-
golf or guile-gi.  You could either do all your conversions in C like I
did back then or declare your own data models as C structs and then
fill them from guile using e.g. guile-bytestructures.
Obviously, it would be nicer if one could program the GUI itself in
Guile, but you are free to make the choices you feel are fitting.

A slightly more elaborate approach would be to try and expose Guix over
DBUS (this would require guile bindings to dbus, that afaik don't exist
outside of projects that want to happily marry Guile and GNOME).  Once
that is written, you could again write your GUI in any language.  Using
DBUS would probably also enable you to get fine-grained control over
when to grant administrative privileges for `guix system`.

Regards, Leo

Re: GNOME in Guix

[Danny Milosavljevic]

[-- Attachment #1.1: Type: text/plain, Size: 10724 bytes --]

Hi Leo,

On Mon, 02 Nov 2020 14:51:42 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

> As much as I rant about nomad in IRC now and then, it is still a rather
> sophisticated program, and as a user I have not yet encountered any
> problem similar in effect to what is described here.  

Thanks.  That is good to know!

I've now gotten guile-gi to work okay for me, too.

For me, there were many reasons why it didn't work before--some of them follow:

(1) I originally built guile-gi from source using

    guix environment --pure guile-gi --ad-hoc guile autoconf automake gettext texinfo

(2) I originally used the following to run guix-gui

    guix environment -l ${HOME}/src/guile-gi/guile-gi-dannym/guile-gi/guix.scm --preserve=XAUTHORITY --preserve=DISPLAY --preserve=G_MESSAGES_DEBUG --ad-hoc guile gdk-pixbuf adwaita-icon-theme shared-mime-info guix gtk+ guile-readline -- ${HOME}/src/guile-gi/guile-gi-dannym/guile-gi/tools/uninstalled-env  guix repl -L . "$@"

(3) gobject-introspection refers to cairo-gobject, but does not embed the store path to it (it would cause a cycle).  There was (an old) cairo-gobject in my user profile, which was picked up

(4) Additionally (in parallel), gtk+ in guix environment refers to a newer gtk than I had in my profile.  Furthermore, guix environment entries depend on glib-with-documentation and similar.

Adding "--pure" to my "guix environment" invocation DID NOT fix the problem.

In any case, using

   guix environment --pure -l guix.scm --preserve=XAUTHORITY --preserve=DISPLAY --preserve=G_MESSAGES_DEBUG --ad-hoc guile adwaita-icon-theme shared-mime-info guix guile-readline -- guix repl -L . "$@"

  (note: "-l guix.scm")

seems to have fixed most of the problems.
(There is no automated diagnostic--so who knows whether it did fix them for real?)

I rate the development experience on Guix in this case at about -15 of 10 points :P

The following bad diagnostics conspired:

(1) guix environment --pure picked up TWO different glib.  Even though both
packages are named the same ("glib") and one is hidden and one is not hidden and
both were used in the environment anyway, no warning was emitted.
That is very bad!
(2) Even though GNOME does not like having two different gobjects loaded, it did
so anyway and caused difficult-to-diagnose problems.
gobject-introspection is not built such that it dlopens libraries in a way
that if there are duplicate global symbols, it fails.  Therefore, there was not
even a failure caused at dlopen time.  Additionally, there was no high-level
failure being caused at any point.  It just did the wrong things and continued
running until eventually crashing because of a low-level failure (memory
allocation, runtime C glib type checking etc).

Before all this, I know that weird things like that could happen so I tried
using something like

  guix environment --ad-hoc pkg1 pkg2 -e '(@ (gnu packages foo) pkg3)' pkg4

in the very beginning--and that didn't work either because this way of using
"-e" is not supported.

> I am not sure, whether the Nomad devs have encountered a problem
> similar to Guile-GI#96 (I myself am not one), but the Guile-GI devs
> speculate, that the problem is somewhat specific to their
> implementation.

guile-gi seems to think that it's their fault--but I disagree.  Changing it
in guile-gi would fix nothing, because gobject-introspection does the exact
same thing.  In fact, let me tell them again.

This was both my fault, and guix's fault for being VERY obtuse.

> They also mention, that you should still be able to prototype your
> application by using "--no-grafts" for the development environment.

If I pass --no-grafts, I cannot read any letters anymore (they are boxes).
See attachment.  So I cannot use that.  Also, it is unsafe to do that.  Grafts
are there for a reason, namely security updates.

> Do you still encounter Guile-GI#96 after packaging?

I've not finished packaging it yet (I tried--but that's pretty difficult,
too!  Help wanted--see README and guix.scm in the guix-gui repo).

> Sure, but that's only if either of them ends up using a different
> version.

It's easy for this to happen.  I did not go out of my way to make it
happen--quite the opposite.

> > If a library A has an input Q, and library B has the same input Q,
> > and you
> > use A and B in your program, and then you (guix-) update just B to
> > use Q'
> > (for example a newer version of Q) and recompile your program, then
> > you have
> > both Q and Q' in your process address space!
> > 
> > Q is an internal dependency of A, and by coincidence it's an internal
> > dependency of B, too. Just because the internal dependency of B
> > changed
> > shouldn't change the internal depenency of A--that's what "internal"
> > means.  
> Sure, but that's only if either of them ends up using a different
> version.  That should not be the case for Guix' GNOME stack apart from
> some bootstrap shenanigans, that don't really matter at the point of
> application packages.
> 
> In order to make use of any GI in Guix, you need
> - the GI bindings (one of python-gobject, g-golf, guile-gi, gjs, etc.),
> - *and* the packages of the libraries themselves
> as inputs to your package.  So the resulting graph for your package
> will always be complete and should only contain one relevant package
> per library.

It is good to know that guix guarantees to only have one libgobject
(i.e. "glib" package derivation) in this case.
I therefore withdraw the propagated-input proposal.

> [having two different libgobject in the same process] should not be the case
> for Guix' GNOME stack apart from [bootstrapping]

When using guix environment, it is easy for this to happen in Guix's
GNOME.

Grafting, different glib-with-or-without-documentation packages and propagated
inputs in profiles propagating cairo-gobject into my user profile (not to
mention stuff in the SYSTEM profile) and guix environment being
obtuse not only make this possible, but make it a regular occurence
when using guix environment.  I think the first step in fixing this
is actually automatically diagnosing this problem, which Guix (and
gobject-introspection) do not do right now.

As it is now, often there are two different libgobjects being loaded at the same
time, and those register their types in the respective type registry.  Depending
on who talks to which registry, they get different actual implementers back.

Which registry you talk to also determines whether you will get "the type
doesn't exist" back or not (which is how it should be (!), and fine if people
talked to the correct respective registry--which they don't).

So the current (and maybe perpetual) state is that GNOME does not support this
(and they really don't need to support it), so somebody (possibly
gobject-introspection) should be catching it, if necessary at runtime.
They don't.

> > Does anyone know how to make dlopen fail on duplicate global
> > variables?  
> No, but I don't think that is too relevant here.  See my earlier point
> on how it *is* possible to handle this with the methods GLib provides.

(1) This would make it extremely easy to find problems like this now.

So it's relevant in the sense of it could have spared both me and the guile-gi
devs a LOT of time.

(2) It would make it possible to find problems of this kind in the future.
I do not want to keep fixing the same thing over and over again.  I want there
to be an automated check in place so if there are problems, they are
automatically detected.
In this case I want dlopen to fail in this case and not be able to load a
second libgobject in the first place--neither directly nor indirectly.

(Evidently, programs and *GNOME's own libraries* are not designed for the
alternative of two libgobject libraries loaded in the same process--so
gobject-introspection should not allow it in the first place)

> Am Donnerstag, den 29.10.2020, 20:34 +0100 schrieb Danny Milosavljevic:
> > Or I could just use Gtk in C and use popen("guix ...").  
> You're probably joking, but you should know, that I had moderate
> success passing around just data from Guile to Gtk while not using g-
> golf or guile-gi.  You could either do all your conversions in C like I
> did back then or declare your own data models as C structs and then
> fill them from guile using e.g. guile-bytestructures.
> Obviously, it would be nicer if one could program the GUI itself in
> Guile, but you are free to make the choices you feel are fitting.

I don't want to maintain data structure marshallers that add no value to
anything.  One of the touted advantages of using guix as a package manager
is that you don't need to do that anymore--you just use guile as a
general-purpose programming language and not keep manually translating back
and forth between different incompatible environments, with all the
impedance mismatch, slowness, different semantics, protocol versioning and
bugs that that entails.

Using gtk on guile also already makes me sometimes force a square peg into a
round hole--that should be enough.

> [expose guix via dbus]

You should keep in mind for this approach: impedance mismatch, slowness,
different semantics, protocol versioning, and bugs in dbus (the latter
of which I also regularily experience in guix--dbus bus hangs etc).

The advantages you listed of better and easier privilege checking are
valid, though.

(In addition, I don't want a hard dependency to gtk+ in the guix package manager.
But that can be avoided in other ways, too).

I should note that I'm on regular guix master with no custom patches.
If I do patches, it's always in a ./pre-inst-env, never on the system
I use.

The following is historical and not current anymore:

$ guix environment --pure -l ${HOME}/src/guile-gi/guile-gi/guix.scm --preserve=XAUTHORITY --preserve=DISPLAY --preserve=G_MESSAGES_DEBUG --ad-hoc guile gdk-pixbuf adwaita-icon-theme shared-mime-info guix gtk+ guile-readline -- ${HOME}/src/guile-gi/guile-gi/tools/uninstalled-env guix repl -L . guix-gui.in

(see "run" in repository below)
(note: I do not preserve any GI environment variables)

using https://gitlab.com/daym/guix-gui/-/tree/002e931232c818f9100b5ea622a52bd6b20b9a0e 
and https://github.com/spk121/guile-gi/commit/b454a99b65f927e947faab17d25bd3499829c1b4 as guile-gi
I incessantly encounter this:
Frequent runtime problems at different points in time while
the program is running, that has been traced (in an automated way) to two
different libgobject being used at the same time.

[-- Attachment #1.2: boxes.png --]
[-- Type: image/png, Size: 79896 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNOME in Guix

[Leo Prikler]

Hi Danny,

Am Dienstag, den 03.11.2020, 10:14 +0100 schrieb Danny Milosavljevic:
> I've now gotten guile-gi to work okay for me, too.
That is great to hear.

> For me, there were many reasons why it didn't work before--some of
> them follow:
> 
> (1) I originally built guile-gi from source using
> 
>     guix environment --pure guile-gi --ad-hoc guile autoconf automake
> gettext texinfo
> 
> (2) I originally used the following to run guix-gui
> 
>     guix environment -l ${HOME}/src/guile-gi/guile-gi-dannym/guile-
> gi/guix.scm --preserve=XAUTHORITY --preserve=DISPLAY --
> preserve=G_MESSAGES_DEBUG --ad-hoc guile gdk-pixbuf adwaita-icon-
> theme shared-mime-info guix gtk+ guile-readline -- ${HOME}/src/guile-
> gi/guile-gi-dannym/guile-gi/tools/uninstalled-env  guix repl -L .
> "$@"
> 
> (3) gobject-introspection refers to cairo-gobject, but does not embed
> the store path to it (it would cause a cycle).  There was (an old)
> cairo-gobject in my user profile, which was picked up
> 
> (4) Additionally (in parallel), gtk+ in guix environment refers to a
> newer gtk than I had in my profile.  Furthermore, guix environment
> entries depend on glib-with-documentation and similar.
> 
> Adding "--pure" to my "guix environment" invocation DID NOT fix the
> problem.
Apologize my rudeness, but that is a very arcane incantation ;)
From what I can see (2) on its own should trigger Guile-GI#96, as you
execute it from a build environment without --no-grafts.  Not
saying (3) and (4) had no influence, but pure environments should
mitigate them.

> In any case, using
> 
>    guix environment --pure -l guix.scm --preserve=XAUTHORITY --
> preserve=DISPLAY --preserve=G_MESSAGES_DEBUG --ad-hoc guile adwaita-
> icon-theme shared-mime-info guix guile-readline -- guix repl -L .
> "$@"
> 
>   (note: "-l guix.scm")
> 
> seems to have fixed most of the problems.
> (There is no automated diagnostic--so who knows whether it did fix
> them for real?)
What diagnostic would you want here?

> I rate the development experience on Guix in this case at about -15
> of 10 points :P
> 
> The following bad diagnostics conspired:
> 
> (1) guix environment --pure picked up TWO different glib.  Even
> though both
> packages are named the same ("glib") and one is hidden and one is not
> hidden and
> both were used in the environment anyway, no warning was emitted.
> That is very bad!
Is that really the problem at hand? I don't see glib explicitly
mentioned here, so you should not build glib-with-documentation here. 
Adding glib to one of your --ad-hoc "chains" should yield a different
profile.

> (2) Even though GNOME does not like having two different gobjects
> loaded, it did
> so anyway and caused difficult-to-diagnose problems.
> gobject-introspection is not built such that it dlopens libraries in
> a way
> that if there are duplicate global symbols, it fails.  Therefore,
> there was not
> even a failure caused at dlopen time.  Additionally, there was no
> high-level
> failure being caused at any point.  It just did the wrong things and
> continued
> running until eventually crashing because of a low-level failure
> (memory
> allocation, runtime C glib type checking etc).
What is the meaning of "doesn't like" here?  Is it explicitly
discouraged, poorly supported, work in progress, ...?

> Before all this, I know that weird things like that could happen so I
> tried
> using something like
> 
>   guix environment --ad-hoc pkg1 pkg2 -e '(@ (gnu packages foo)
> pkg3)' pkg4
> 
> in the very beginning--and that didn't work either because this way
> of using
> "-e" is not supported.
Is this perhaps an ordering problem? 
$ guix environment --ad-hoc -e '(@ (gnu packages glib) glib)'
seems to be building something.

> > I am not sure, whether the Nomad devs have encountered a problem
> > similar to Guile-GI#96 (I myself am not one), but the Guile-GI devs
> > speculate, that the problem is somewhat specific to their
> > implementation.
> 
> guile-gi seems to think that it's their fault--but I
> disagree.  Changing it
> in guile-gi would fix nothing, because gobject-introspection does the
> exact
> same thing.  In fact, let me tell them again.
> 
> This was both my fault, and guix's fault for being VERY obtuse.
I don't think looking for someone to blame is particularly useful.  It
also looks as if you are discussing different things there.  Your issue
there and here is using Guile-GI as a dependency, but theirs is the
development of Guile-GI itself on one hand and which use cases they
want to support on the other.  On your side it appears as though your
use case is already well-supported by the library (because it is?) and
only Guix is interfering by mixing up inputs.  On their side, there
exists the question of whether they can (and want) to support more than
that.

> > They also mention, that you should still be able to prototype your
> > application by using "--no-grafts" for the development environment.
> 
> If I pass --no-grafts, I cannot read any letters anymore (they are
> boxes).
> See attachment.  So I cannot use that.  Also, it is unsafe to do
> that.  Grafts
> are there for a reason, namely security updates.
Not having full font support sounds like an issue, I can understand
that.  Regarding the safety aspect, I don't think that's what they're
going for.  They might be thinking, that "--pure --no-grafts" is close
enough to what you'd get inside `guix build`, so that you don't have to
use `guix build` for debugging.

> > Do you still encounter Guile-GI#96 after packaging?
> 
> I've not finished packaging it yet (I tried--but that's pretty
> difficult,
> too!  Help wanted--see README and guix.scm in the guix-gui repo).
I'll see what I can do to help, but the only thing I can see are
"FIXME" strings in guix.scm.  The README even suggests to run `guix
build`.

> > Sure, but that's only if either of them ends up using a different
> > version.
> 
> It's easy for this to happen.  I did not go out of my way to make it
> happen--quite the opposite.
One could disagree about that based on the workflow you've posted here
;)
Jokes aside, the Guile-GI devs seem to be hitting a similar issue, but
as they write, it appears to mainly affect their own development
environments (i.e. exactly the thing that hurts you the most as a
developer).  There is potentially a discrepancy between `guix build`
and `guix environment`.

> > > If a library A has an input Q, and library B has the same input
> > > Q,
> > > and you
> > > use A and B in your program, and then you (guix-) update just B
> > > to
> > > use Q'
> > > (for example a newer version of Q) and recompile your program,
> > > then
> > > you have
> > > both Q and Q' in your process address space!
> > > 
> > > Q is an internal dependency of A, and by coincidence it's an
> > > internal
> > > dependency of B, too. Just because the internal dependency of B
> > > changed
> > > shouldn't change the internal depenency of A--that's what
> > > "internal"
> > > means.  
> > Sure, but that's only if either of them ends up using a different
> > version.  That should not be the case for Guix' GNOME stack apart
> > from
> > some bootstrap shenanigans, that don't really matter at the point
> > of
> > application packages.
> > 
> > In order to make use of any GI in Guix, you need
> > - the GI bindings (one of python-gobject, g-golf, guile-gi, gjs,
> > etc.),
> > - *and* the packages of the libraries themselves
> > as inputs to your package.  So the resulting graph for your package
> > will always be complete and should only contain one relevant
> > package
> > per library.
> 
> It is good to know that guix guarantees to only have one libgobject
> (i.e. "glib" package derivation) in this case.
> I therefore withdraw the propagated-input proposal.
I'm not sure, how hard this guarantee is, but the same grafting
operation should be applied to all packages.
Having a look at Guile-GI specifically, ldd `guix build guile-gi` has
the following:
libgobject-2.0.so.0 => /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-
glib-2.62.6/lib/libgobject-2.0.so.0

You should recognize that hash by now ;)

> > [having two different libgobject in the same process] should not be
> > the case
> > for Guix' GNOME stack apart from [bootstrapping]
> 
> When using guix environment, it is easy for this to happen in Guix's
> GNOME.
> 
> Grafting, different glib-with-or-without-documentation packages and
> propagated
> inputs in profiles propagating cairo-gobject into my user profile
> (not to
> mention stuff in the SYSTEM profile) and guix environment being
> obtuse not only make this possible, but make it a regular occurence
> when using guix environment.  I think the first step in fixing this
> is actually automatically diagnosing this problem, which Guix (and
> gobject-introspection) do not do right now.
You can eliminate a huge array of those issues by using pure
environments.  What remains is the gobject-introspection glib, that is
differently grafted from plain glib for no apparent reason.  (This
might actually be a bug in Guix).  With or without documentation is an
issue if you specify glib at the command line, but not when you build
an environment from guix.scm.  Perhaps glib should get the gcc-
toolchain treatment.

> As it is now, often there are two different libgobjects being loaded
> at the same
> time, and those register their types in the respective type
> registry.  Depending
> on who talks to which registry, they get different actual
> implementers back.
> 
> Which registry you talk to also determines whether you will get "the
> type
> doesn't exist" back or not (which is how it should be (!), and fine
> if people
> talked to the correct respective registry--which they don't).
> 
> So the current (and maybe perpetual) state is that GNOME does not
> support this
> (and they really don't need to support it), so somebody (possibly
> gobject-introspection) should be catching it, if necessary at
> runtime.
> They don't.
I'm not sure whether they must specifically enforce one and only one
type registry or whether those can be made to coexist.  You should
probably talk to gobject-introspection about this.

> > > Does anyone know how to make dlopen fail on duplicate global
> > > variables?  
> > No, but I don't think that is too relevant here.  See my earlier
> > point
> > on how it *is* possible to handle this with the methods GLib
> > provides.
> 
> (1) This would make it extremely easy to find problems like this now.
> 
> So it's relevant in the sense of it could have spared both me and the
> guile-gi
> devs a LOT of time.
It would also make it extremely easy to shoot yourself in the foot,
thereby costing you even more time.  It's like shooting flies with a
cannon.  Sure, you might kill some bugs, but you can just as well tear
down castle walls with that.

> (2) It would make it possible to find problems of this kind in the
> future.
> I do not want to keep fixing the same thing over and over again.  I
> want there
> to be an automated check in place so if there are problems, they are
> automatically detected.
> In this case I want dlopen to fail in this case and not be able to
> load a
> second libgobject in the first place--neither directly nor
> indirectly.
I think this is something, that should be solved at the application
level, not lower.

> (Evidently, programs and *GNOME's own libraries* are not designed for
> the
> alternative of two libgobject libraries loaded in the same process
> --so
> gobject-introspection should not allow it in the first place)
I'm not sure how sane loading another version of GObject is in other GI
implementations.

> > Am Donnerstag, den 29.10.2020, 20:34 +0100 schrieb Danny
> > Milosavljevic:
> > > Or I could just use Gtk in C and use popen("guix ...").  
> > You're probably joking, but you should know, that I had moderate
> > success passing around just data from Guile to Gtk while not using
> > g-
> > golf or guile-gi.  You could either do all your conversions in C
> > like I
> > did back then or declare your own data models as C structs and then
> > fill them from guile using e.g. guile-bytestructures.
> > Obviously, it would be nicer if one could program the GUI itself in
> > Guile, but you are free to make the choices you feel are fitting.
> 
> I don't want to maintain data structure marshallers that add no value
> to
> anything.  One of the touted advantages of using guix as a package
> manager
> is that you don't need to do that anymore--you just use guile as a
> general-purpose programming language and not keep manually
> translating back
> and forth between different incompatible environments, with all the
> impedance mismatch, slowness, different semantics, protocol
> versioning and
> bugs that that entails.
Fair enough.

> Using gtk on guile also already makes me sometimes force a square peg
> into a
> round hole--that should be enough.
I know what you mean, GObject is not very functional in its programming
style.

> > [expose guix via dbus]
> 
> You should keep in mind for this approach: impedance mismatch,
> slowness,
> different semantics, protocol versioning, and bugs in dbus (the
> latter
> of which I also regularily experience in guix--dbus bus hangs etc).
> 
> The advantages you listed of better and easier privilege checking are
> valid, though.
> 
> (In addition, I don't want a hard dependency to gtk+ in the guix
> package manager.
> But that can be avoided in other ways, too).
> 
> I should note that I'm on regular guix master with no custom patches.
> If I do patches, it's always in a ./pre-inst-env, never on the system
> I use.
My suggestion was not to change Guix in-place, but to write a piece of
software, that interacts with Guix on the one hand (via REPL/Scheme
bindings) and DBus on the other.  But yeah, given that that's not
really simpler than writing your Gtk frontend in Scheme, let's stick to
that for now.

> The following is historical and not current anymore:
> 
> $ guix environment --pure -l ${HOME}/src/guile-gi/guile-gi/guix.scm
> --preserve=XAUTHORITY --preserve=DISPLAY --preserve=G_MESSAGES_DEBUG
> --ad-hoc guile gdk-pixbuf adwaita-icon-theme shared-mime-info guix
> gtk+ guile-readline -- ${HOME}/src/guile-gi/guile-
> gi/tools/uninstalled-env guix repl -L . guix-gui.in
> 
> (see "run" in repository below)
> (note: I do not preserve any GI environment variables)
> 
> using 
> https://gitlab.com/daym/guix-gui/-/tree/002e931232c818f9100b5ea622a52bd6b20b9a0e
>  
> and 
> https://github.com/spk121/guile-gi/commit/b454a99b65f927e947faab17d25bd3499829c1b4
> as guile-gi
> I incessantly encounter this:
> Frequent runtime problems at different points in time while
> the program is running, that has been traced (in an automated way) to
> two
> different libgobject being used at the same time.
I don't think I can help you with that here, but if you have something
usable for debugging, like a stack trace, it might be worth to ask
around Guile-GI (assuming it is not just a repetition of their #96).

Regards, Leo

Re: GNOME in Guix

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 11370 bytes --]

Hi Leo,

On Tue, 03 Nov 2020 14:41:31 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

> >   (note: "-l guix.scm")
> > 
> > seems to have fixed most of the problems.
> > (There is no automated diagnostic--so who knows whether it did fix
> > them for real?)  
> What diagnostic would you want here?

Whether there exist packages with the same name but different derivations in
the environment (especially if mixing hidden and non-hidden packages with
the same name).

Also, there's probably some kind of environment build file list printing in guix.
Using it, it could list libgobjects are in the environment without me having
to use a LD_PRELOAD to find which they are...

Does it exist?  If so, how to use it?

> Is that really the problem at hand? I don't see glib explicitly
> mentioned here, so you should not build glib-with-documentation here. 
> Adding glib to one of your --ad-hoc "chains" should yield a different
> profile.

It's not that direct--but yes, something like that is indeed the problem at hand.

If you want more detail, the guile-gi bug report is pretty detailed on it.

Also, better, the reason I put the "historical" section at the end of my
previous post is that it is reproducible later, so we can find out what exactly
happened here.

Following that you will get exactly the same situation--and using (only) the
LD_PRELOAD discussed on guile-gi, you will find the culprit, and you will see
that it is two different libgobjects being loaded.

We are now in a better situation because I do all my development using version
control--even throwaway stuff.  guile-gi devs had this happen before--but their
reproducers got lost to time.  Now we have reproducers that cause it every
single time (both the historical version of guix-gui mentioned and also
file test/insanity.scm in guile-gi by now).

> What is the meaning of "doesn't like" here?  Is it explicitly
> discouraged, poorly supported, work in progress, ...?

It means it does not work, and given what they are doing in the source code
GNOME evidently did not think about this case at all.

And at runtime using guix environment, this does (pretty much always) cause
problems without any special things to be done to cause it.

You can read a lot of details on the bug report on the guile-gi website.

> > This was both my fault, and guix's fault for being VERY obtuse.  

> I don't think looking for someone to blame is particularly useful.

We are engineers--so finding out what is to blame for a problem and
fixing that place is indeed what we do all day (that includes deciding
which projects should have the change, and which shouldn't.  If the
answer is "none" the situation will not improve.  So that's right out).

If you mean blaming people, I don't do that--that would be silly.

"Blaming" programs I will do all day every day (including mine).  Problems like
this need to be seen, otherwise us guix devs don't know that it's like this.

I know it from my own stuff I put into Guix master.  *I* did get those
to work by configuring stuff.  But is it usable for a regular user?

For this guix environment stuff, the answer is a definite no.  (there's
another thread on the guix-devel list about improving that--so it's not
like I'm the only one--it's just a bad interface.  It happens.)

>Regarding the safety aspect, I don't think that's what they're
> going for.  They might be thinking, that "--pure --no-grafts" is close
> enough to what you'd get inside `guix build`, so that you don't have to
> use `guix build` for debugging.

If grafts are broken (which seems likely according to guile-gi devs), grafting
should be fixed now, before a security problem happens and we miss it (...but
state that we fixed it).

I know that guile-gi argues for doing development with --no-grafts, but

(1) I don't have any readable characters when I do that
(2) If I did, it would still not help the end user because it is not the right fix

> > > Do you still encounter Guile-GI#96 after packaging?  
> > 
> > I've not finished packaging it yet (I tried--but that's pretty
> > difficult,
> > too!  Help wanted--see README and guix.scm in the guix-gui repo).  

> I'll see what I can do to help, but the only thing I can see are
> "FIXME" strings in guix.scm.  The README even suggests to run `guix
> build`.

Thanks!

If you do the guix build and then run the final script, then
you will see that it does not start up because it cannot find guile
modules.  I am not knowledgeable how to deploy large guile programs,
so I'd appreciate help in doing that.

(I can totally find a hacky way to make it work--but I'd rather use the
canonical way to find external guile modules that surely exists,
especially in guix (... which is written in guile and is a large program
with external guile modules))

> One could disagree about that based on the workflow you've posted here
> ;)

I can assure you this is what anyone would try, until he gets burned like
this and then does some of

(1) gives up and deletes guix
(2) posts a bad review to the media
(3) posts bug reports detailing what happened
(4) uses another way than "guix environment" and it finally works (but does
that really do the right thing?  Who knows...)

I prefer people do (3) and (4).  Ideally, getting information for (3) would be
automatic (at least more automatic than "<GValue> is not compatible with
<GValue>".  Oh yeah?  It's not compatible?  Good to know ;)

Also, guile-gi devs always used "--pure -l guix.scm" and it still happens to
them!

> Jokes aside, the Guile-GI devs seem to be hitting a similar issue, but
> as they write, it appears to mainly affect their own development
> environments (i.e. exactly the thing that hurts you the most as a
> developer).  There is potentially a discrepancy between `guix build`
> and `guix environment`.

Apparently so.

Note that with the current state of guix (no gui...) we mainly can target only
developers--so if the story for developers is bad, well, that's bad for guix.

> Having a look at Guile-GI specifically, ldd `guix build guile-gi` has
> the following:
> libgobject-2.0.so.0 => /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-
> glib-2.62.6/lib/libgobject-2.0.so.0
> 
> You should recognize that hash by now ;)

Definitely.  With how often in different contexts I saw gobject hashes by now,
I'll probably dream about those ;)

> You can eliminate a huge array of those issues by using pure
> environments.

As I said, "guix environment --pure" did not fix all (or even many) of these
problems.  I added "--pure" very soon--and WITH the "--pure" it still loads
two different libgobjects.  Using "-l" helps tremendously.  "--pure"? Nope.

> I'm not sure whether they must specifically enforce one and only one
> type registry or whether those can be made to coexist.  You should
> probably talk to gobject-introspection about this.

Yeah.  But I will see what guile-gi devs have to say about it first.

> It would also make it extremely easy to shoot yourself in the foot,
> thereby costing you even more time.  It's like shooting flies with a
> cannon.  Sure, you might kill some bugs, but you can just as well tear
> down castle walls with that.

I don't see what is even controversal with it.  The same duplicate symbol
checking *always* happens with regular linking.  So if you now dlopen stuff
that usually is regularily linked into C programs, the latter C program
developers already ensured that the libraries don't have duplicate symbols
(in a certain sense), otherwise the libraries wouldn't link with the C
programs in the first place.

I'm only suggesting for dlopen to to do the same thing in diagnostics mode.

Also, I'm not suggesting to always enable the feature--but it should *exist*.

But you are right, an easier fix would be guix environment just warning about
packages that have the same name but are different packages in the same
environment.  That way you wouldn't have to care about linker stuff at all.

Bad debugging features are the achilles heel of guix--and it's difficult
for me to justify spending time on guix when every time there's something
to debug it is so bad.

Examples:

(1) There is no debug info by default.  Ludo fixed that in that now one can
use package transformations to build certain packages with debug info without
having to recompile a guix checkout.

Nice.  (previously, you had to check out guix, manually edit packages (first
find out how), then compile it (which is not easy) and then use ./pre-inst-env
to rebuild whatever you wanted to debug in the first place so you get line
number information when one of the programs segfaults and you are in gdb.
Think joe regular dev would do that when collecting info for reporting bugs?
I don't)

(2) guix guile backtraces always just say eval.scm and not the actual
source file or line something happened in.  That is still the case.
I basically use divide and conquer "write"s in order to find out where in
my program the actual line it is talking about is.

(3) guix environment and the actual guix build daemon set up slightly
different enviroments.  That way, it can happen that a bug that
happens inside the guix build container doesn't happen when you search
for it using guix environment.

> I'm not sure how sane loading another version of GObject is in other GI
> implementations.

Not sane at all.

That duplicate loading of libraries with the same basename is not supposed to
be done.

I mean it's nice if someone gets it to work anyway--but I suspect that that
won't fly with gobject's runtime type checking.

> I don't think I can help you with that here, but if you have something
> usable for debugging, like a stack trace, it might be worth to ask
> around Guile-GI (assuming it is not just a repetition of their #96).

The problem is that the software stack does not fail correctly,
and somewhere the software stack needs to be adapted so that it fails
instead of dragging the error state all the way through the entire profile
without it being detected.  There are only usable stacktraces when you
go out of your way using LD_PRELOAD.  That is not a tenable way.

I want to be clear that guile-gi is not at fault, by which I mean that
this part of the software stack should not be changed.  If you did,
gobject-introspection would still do it "wrong".  If you changed that, the
next GNOME library over there would STILL do it "wrong".  That is not the
way to fix it.

Also, what is "fixing it"?  Is it preventing duplicate loading for good
or is it changing 29 libraries upstream to support duplicate loading?
Or is it warning if it happens?  All of those in different points of time?

With this, I hope to at least have raised awareness and started the
discussion so progress can be made.

(For example, it makes not much sense to bother guile-gi devs with a problem
that guile-gi did not cause.  gobject-introspection has the exact same
design (and is a dependency of guile-gi), so everyone who uses
gobject-introspection inherits that limitation.  The guile-gi people
are just nice enough to look at it anyway--even though it's not something
they can unilaterally reasonably fix anyway)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNOME in Guix

[Leo Prikler]

Hi Danny,

Am Dienstag, den 03.11.2020, 20:26 +0100 schrieb Danny Milosavljevic:
> Hi Leo,
> 
> On Tue, 03 Nov 2020 14:41:31 +0100
> Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> 
> > >   (note: "-l guix.scm")
> > > 
> > > seems to have fixed most of the problems.
> > > (There is no automated diagnostic--so who knows whether it did
> > > fix
> > > them for real?)  
> > What diagnostic would you want here?
> 
> Whether there exist packages with the same name but different
> derivations in
> the environment (especially if mixing hidden and non-hidden packages
> with
> the same name).
If conflicting versions of the same package are pulled in through
propagated inputs, things would fail.  If you get two different files
into the same profile otherwise, Guix silently resolves that conflict,
but you can get a warning about that at higher verbosity.  I'm not
exactly sure, where this falls into, however.  I have the weird
feeling, that you don't get two different packages into the
environment.  Not directly at least.  And at that point making
meaningful predictions based on (transitive) inputs is going to be
complicated.

> Also, there's probably some kind of environment build file list
> printing in guix.
> Using it, it could list libgobjects are in the environment without me
> having
> to use a LD_PRELOAD to find which they are...
> 
> Does it exist?  If so, how to use it?
I'm not sure if I interpret this correctly, but you're talking about
the derivation (.drv) files, right?  If so, Guix prints them on first
build and you can grep for the path of $GUIX_ENVIRONMENT in /gnu/store
too in order to find them.
Reading them on the other hand…

> > Is that really the problem at hand? I don't see glib explicitly
> > mentioned here, so you should not build glib-with-documentation
> > here. 
> > Adding glib to one of your --ad-hoc "chains" should yield a
> > different
> > profile.
> 
> It's not that direct--but yes, something like that is indeed the
> problem at hand.
> 
> If you want more detail, the guile-gi bug report is pretty detailed
> on it.
I think they mention glib-with-documentation as something the user
might want to put in their development environments for some other
purpose (or inadvertently pull in anyway), not as the thing that gets
put there in this specific instance.  Either way, you'd need to install
glib-with-documentation (= user-facing glib) either globally or in your
development environment for that to happen.

> Also, better, the reason I put the "historical" section at the end of
> my
> previous post is that it is reproducible later, so we can find out
> what exactly
> happened here.
> 
> Following that you will get exactly the same situation--and using
> (only) the
> LD_PRELOAD discussed on guile-gi, you will find the culprit, and you
> will see
> that it is two different libgobjects being loaded.
> 
> We are now in a better situation because I do all my development
> using version
> control--even throwaway stuff.  guile-gi devs had this happen before-
> -but their
> reproducers got lost to time.  Now we have reproducers that cause it
> every
> single time (both the historical version of guix-gui mentioned and
> also
> file test/insanity.scm in guile-gi by now).
Having reproducible reproducers is a wonderful thing, but you need to
understand them too.  I guess in the context of guile-gi,
test/insanity.scm catches the case, where the same structure has two
GTypes due to it being loaded from different places.

> > What is the meaning of "doesn't like" here?  Is it explicitly
> > discouraged, poorly supported, work in progress, ...?
> 
> It means it does not work, and given what they are doing in the
> source code
> GNOME evidently did not think about this case at all.
> 
> And at runtime using guix environment, this does (pretty much always)
> cause
> problems without any special things to be done to cause it.
> 
> You can read a lot of details on the bug report on the guile-gi
> website.
I am following this bug somewhat closely, but to me there doesn't seem
to be this hard evidence you are talking about.  If I read this
correctly, upstream thinks it might be possible to work around such
issues on their end.  What do GI maintainers say about loading a
different version of GObject?

> > > This was both my fault, and guix's fault for being VERY obtuse.  
> > I don't think looking for someone to blame is particularly useful.
> 
> We are engineers--so finding out what is to blame for a problem and
> fixing that place is indeed what we do all day (that includes
> deciding
> which projects should have the change, and which shouldn't.  If the
> answer is "none" the situation will not improve.  So that's right
> out).
Emphasis on "someone".  You can (and should of course) search for
issues in code and architecture, but not in people :)

> If you mean blaming people, I don't do that--that would be silly.
Fuck, you've anticipated my response.

> "Blaming" programs I will do all day every day (including
> mine).  Problems like
> this need to be seen, otherwise us guix devs don't know that it's
> like this.
> 
> I know it from my own stuff I put into Guix master.  *I* did get
> those
> to work by configuring stuff.  But is it usable for a regular user?
I think in this case having a pinned issue in Guile-GI saying "sorry
folks, `guix environment guile-gi` is broken" is a step forward.  The
question is how Guix or Guile-GI devs can "unbreak" this environment.

The crux seem to be two different versions of glib spawning from what
should be one.  Of the following:
(1) /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-glib-
2.62.6/lib/libgobject-2.0.so.0
(2) /gnu/store/xkfc1275h55ynpgfr3wwmzy9707nblwc-glib-
2.62.6/lib/libgobject-2.0.so.0
only (1) should be built in my opinion, but I have nothing to back up
my reasoning about that except for the fact that it's the one GI wants.

> For this guix environment stuff, the answer is a definite
> no.  (there's
> another thread on the guix-devel list about improving that--so it's
> not
> like I'm the only one--it's just a bad interface.  It happens.)
Which other thread are you referring to?  The most relevant thing I
found was you encountering something similar with mrustc 2 years ago.

> > Regarding the safety aspect, I don't think that's what they're
> > going for.  They might be thinking, that "--pure --no-grafts" is
> > close
> > enough to what you'd get inside `guix build`, so that you don't
> > have to
> > use `guix build` for debugging.
> 
> If grafts are broken (which seems likely according to guile-gi devs),
> grafting
> should be fixed now, before a security problem happens and we miss it
> (...but
> state that we fixed it).
I don't think grafting itself is broken.  What might be broken is `guix
environment --yes-grafts`, as it departs from the expectation that the
environment you get is "just like `guix build`".

> I know that guile-gi argues for doing development with --no-grafts,
> but
> 
> (1) I don't have any readable characters when I do that
> (2) If I did, it would still not help the end user because it is not
> the right fix
I don't think the end user will care too much about some development
hiccup.  What matters, is that the resulting package is functional and
you should get that by enforcing GI_TYPELIB_PATH = (getenv
"GI_TYPELIB_PATH").

> > > > Do you still encounter Guile-GI#96 after packaging?  
> > > 
> > > I've not finished packaging it yet (I tried--but that's pretty
> > > difficult,
> > > too!  Help wanted--see README and guix.scm in the guix-gui
> > > repo).  
> > I'll see what I can do to help, but the only thing I can see are
> > "FIXME" strings in guix.scm.  The README even suggests to run `guix
> > build`.
> 
> Thanks!
> 
> If you do the guix build and then run the final script, then
> you will see that it does not start up because it cannot find guile
> modules.  I am not knowledgeable how to deploy large guile programs,
> so I'd appreciate help in doing that.
> 
> (I can totally find a hacky way to make it work--but I'd rather use
> the
> canonical way to find external guile modules that surely exists,
> especially in guix (... which is written in guile and is a large
> program
> with external guile modules))
Given that your program is an application, you would wrap
GUILE_LOAD_PATH and GUILE_LOAD_COMPILED_PATH into the launcher using
wrap-program.  If it was a library, you'd request the user to install
guile in the same environment and propagate any Guile modules you
depend on.

> > One could disagree about that based on the workflow you've posted
> > here
> > ;)
> 
> I can assure you this is what anyone would try, until he gets burned
> like
> this and then does some of
> 
> (1) gives up and deletes guix
> (2) posts a bad review to the media
> (3) posts bug reports detailing what happened
> (4) uses another way than "guix environment" and it finally works
> (but does
> that really do the right thing?  Who knows...)
Building increasingly arcane commands is the exact opposite of what you
should do in Guix.  If it looks wrong and feels wrong it's most likely
wrong ;)
I know that npm and others educate their users to pack even more
brittle constructs onto already brittle constructs, but that's the
state of software that we want to get rid of.

> I prefer people do (3) and (4).  Ideally, getting information for (3)
> would be
> automatic (at least more automatic than "<GValue> is not compatible
> with
> <GValue>".  Oh yeah?  It's not compatible?  Good to know ;)
I think Guile-GI devs are at least going to have a laugh if you post
that over there.  Sadly I get stuff like "<GObject> imported from both
(x) and (y)", which is a slightly better clue that things are going
wrong.

> Also, guile-gi devs always used "--pure -l guix.scm" and it still
> happens to them!
Is this perhaps related to their request to use --no-grafts?

> > Jokes aside, the Guile-GI devs seem to be hitting a similar issue,
> > but
> > as they write, it appears to mainly affect their own development
> > environments (i.e. exactly the thing that hurts you the most as a
> > developer).  There is potentially a discrepancy between `guix
> > build`
> > and `guix environment`.
> 
> Apparently so.
> 
> Note that with the current state of guix (no gui...) we mainly can
> target only
> developers--so if the story for developers is bad, well, that's bad
> for guix.
Okay, but what is Guix to do?  "--no-graft" environments by default to
make them closer to `guix build`?  There is already a movement saying
they want "--ad-hoc" by default, which goes into the completely other
direction.  "Graft more carefully"?  If so, how?

> > Having a look at Guile-GI specifically, ldd `guix build guile-gi`
> > has
> > the following:
> > libgobject-2.0.so.0 => /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-
> > glib-2.62.6/lib/libgobject-2.0.so.0
> > 
> > You should recognize that hash by now ;)
> 
> Definitely.  With how often in different contexts I saw gobject
> hashes by now,
> I'll probably dream about those ;)
xa1vf...

> > You can eliminate a huge array of those issues by using pure
> > environments.
> 
> As I said, "guix environment --pure" did not fix all (or even many)
> of these
> problems.  I added "--pure" very soon--and WITH the "--pure" it still
> loads
> two different libgobjects.  Using "-l" helps tremendously.  "--pure"?
> Nope.
I'm quite certain Guile-GI devs use "--pure -l".  Where is your God
now?

> > It would also make it extremely easy to shoot yourself in the foot,
> > thereby costing you even more time.  It's like shooting flies with
> > a
> > cannon.  Sure, you might kill some bugs, but you can just as well
> > tear
> > down castle walls with that.
> 
> I don't see what is even controversal with it.  The same duplicate
> symbol
> checking *always* happens with regular linking.  So if you now dlopen
> stuff
> that usually is regularily linked into C programs, the latter C
> program
> developers already ensured that the libraries don't have duplicate
> symbols
> (in a certain sense), otherwise the libraries wouldn't link with the
> C
> programs in the first place.
I wouldn't be too certain about that.  You are excluding very benign
behaviour here.  Going back to your earlier examples with A, B, Q and
Q', I'm not sure how you'd get that to link statically, whereas
dynamically linking them together should be of medium trickery.

> I'm only suggesting for dlopen to to do the same thing in diagnostics
> mode.
> 
> Also, I'm not suggesting to always enable the feature--but it should
> *exist*.
Anyway, we are departing a bit much from the original topic, are we
not?  I also feel as if this question is something better posed
separately, perhaps even as a general libc question on stackoverflow or
similar?

> But you are right, an easier fix would be guix environment just
> warning about
> packages that have the same name but are different packages in the
> same
> environment.  That way you wouldn't have to care about linker stuff
> at all.
I still wonder what exactly you mean by that, see my earlier point
regarding meaningful predictions.

> Bad debugging features are the achilles heel of guix--and it's
> difficult
> for me to justify spending time on guix when every time there's
> something
> to debug it is so bad.

To be honest, I have already gazed into the deepest abyss of debugging
there is and it gazed back, so nothing you will say after this line
will faze me.  It also means to take my counter-points with a grain of
salt.

> (1) There is no debug info by default.  Ludo fixed that in that now
> one can
> use package transformations to build certain packages with debug info
> without
> having to recompile a guix checkout.
> 
> Nice.  (previously, you had to check out guix, manually edit packages
> (first
> find out how), then compile it (which is not easy) and then use
> ./pre-inst-env
> to rebuild whatever you wanted to debug in the first place so you get
> line
> number information when one of the programs segfaults and you are in
> gdb.
> Think joe regular dev would do that when collecting info for
> reporting bugs?
> I don't)
I don't find it nice to bring already solved problems here, but Guix
has a huge number of ways to write custom packages (even more with guix
transformations) and of those only one requires checkout and
recompilation.
It also has a pretty decent manual.

> (2) guix guile backtraces always just say eval.scm and not the actual
> source file or line something happened in.  That is still the case.
> I basically use divide and conquer "write"s in order to find out
> where in
> my program the actual line it is talking about is.
Can't say I've encountered too many of those, but I think it might have
to do with (lack of) compilation.  Not really sure what I do wrong
here, I've been searching for less meaningful backtraces far and wide.

> (3) guix environment and the actual guix build daemon set up slightly
> different enviroments.  That way, it can happen that a bug that
> happens inside the guix build container doesn't happen when you
> search
> for it using guix environment.
Back to that one aren't we?  I feel like this is the main driver as to
why Guile-GI devs suggest --no-grafts.
That being said, I don't feel like there is a consensus on what `guix
environment` should be, so it ends up filling different niches all at
once.  I also don't know, whether those niches can neatly be separated
into different commands and if so how to name them.

> > I'm not sure how sane loading another version of GObject is in
> > other GI
> > implementations.
> 
> Not sane at all.
> 
> That duplicate loading of libraries with the same basename is not
> supposed to
> be done.
> 
> I mean it's nice if someone gets it to work anyway--but I suspect
> that that
> won't fly with gobject's runtime type checking.
That's nice as far as theory goes, but have you tried?
I'm joking of course.  There is no other version of GObject you could
load but one that is slightly differently grafted.

> > I don't think I can help you with that here, but if you have
> > something
> > usable for debugging, like a stack trace, it might be worth to ask
> > around Guile-GI (assuming it is not just a repetition of their
> > #96).
> 
> The problem is that the software stack does not fail correctly,
> and somewhere the software stack needs to be adapted so that it fails
> instead of dragging the error state all the way through the entire
> profile
> without it being detected.  There are only usable stacktraces when
> you
> go out of your way using LD_PRELOAD.  That is not a tenable way.
What kind of stack traces do you get from that?  Traces of when
conflicting libraries are loaded, I assume.

> I want to be clear that guile-gi is not at fault, by which I mean
> that
> this part of the software stack should not be changed.  If you did,
> gobject-introspection would still do it "wrong".  If you changed
> that, the
> next GNOME library over there would STILL do it "wrong".  That is not
> the
> way to fix it.
> 
> Also, what is "fixing it"?  Is it preventing duplicate loading for
> good
> or is it changing 29 libraries upstream to support duplicate loading?
> Or is it warning if it happens?  All of those in different points of
> time?
I feel like you've read me wrong here.  The point was, that the Guile-
GI devs can aid you in debugging the situation if it's not something
you already are familiar with.  They are (presumably) the ones who know
their codebase best, so if you raise an issue along the lines of "if I
do this and that, then this segfault happens with that warning printed"
or "if I spawn a Gtk Window and wait for three seconds, my application
crashes", they will likely help you out.

Of course, if it's a direct result of #96, you're not adding more than
a data point unless you go out of your way to prove something about it,
but I was speaking in slightly broader terms.

Whether or not it is their issue or someone else's is up for them to
decide and perhaps negotiate.  Looking at it from another perspective,
even if GI changed in the way you wanted, Guile-GI could still misuse
it in a way, that breaks things.  Perhaps that way one could interpret
#96 as them trying their best not to be the mood killer.

> With this, I hope to at least have raised awareness and started the
> discussion so progress can be made.
> 
> (For example, it makes not much sense to bother guile-gi devs with a
> problem
> that guile-gi did not cause.  gobject-introspection has the exact
> same
> design (and is a dependency of guile-gi), so everyone who uses
> gobject-introspection inherits that limitation.  The guile-gi people
> are just nice enough to look at it anyway--even though it's not
> something
> they can unilaterally reasonably fix anyway)
I understand the sentiment, but at the same time I don't think you
should tell them not to bother with that issue.  Look at it this way:
If they somehow do manage against all the odds to more or less
meaningfully load a different GObject through GI, would that not
encourage GI to adapt so as to make that easier?

Regards, Leo

Re: GNOME in Guix

[Danny Milosavljevic]

[-- Attachment #1.1: Type: text/plain, Size: 2958 bytes --]

Hi,

I've checked guile-gi test/insanity.scm again to find "hard" evidence.

For that, I've just checked out guile-gi anew, then ran test/insanity.scm.

Steps:

(1) git clone https://github.com/spk121/guile-gi.git guile-gi
(2) cd guile-gi
(3) git checkout b454a99b65f927e947faab17d25bd3499829c1b4 # the current one
(4) guix environment --pure -l guix.scm --ad-hoc gtk+ guile
(5) ./bootstrap && ./configure && make
(6) tools/run-guile test/insanity.scm
%%%% Starting test insanity.scm  (Writing full log to "insanity.scm.log")
/home/dannym/src/guile-gi/guile-gi/test/insanity.scm:19: FAIL GObject.Value
/home/dannym/src/guile-gi/guile-gi/test/insanity.scm:23: FAIL GObject.Closure
# of expected passes      1
# of unexpected failures  2
WARNING: (guile-user): imported module (gi) overrides core binding `quit'
(7) cat insanity.scm.log
%%%% Starting test insanity.scm
Group begin: insanity.scm
Test begin:
  test-name: "GObject.Object"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 15
  source-form: (test-equal "GObject.Object" <GObject> (module-ref m (quote <GObject>)))
Test end:
  result-kind: pass
  actual-value: #<<class> <GObject> 7f2d4e2ca180>
  expected-value: #<<class> <GObject> 7f2d4e2ca180>
Test begin:
  test-name: "GObject.Value"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 19
  source-form: (test-equal "GObject.Value" <GValue> (module-ref m (quote <GValue>)))
Test end:
  result-kind: fail
  actual-value: #<<class> <GValue> 7f2d4e2ea680>
  expected-value: #<<applicable-struct-with-setter-class> <GValue> 7f2d4e2e0200>
Test begin:
  test-name: "GObject.Closure"
  source-file: "/home/dannym/src/guile-gi/guile-gi/test/insanity.scm"
  source-line: 23
  source-form: (test-equal "GObject.Closure" <GClosure> (module-ref m (quote <GClosure>)))
Test end:
  result-kind: fail
  actual-value: #<<class> <GClosure> 7f2d4e2ea900>
  expected-value: #<<applicable-struct-class> <GClosure> 7f2d4e2eae80>
Group end: insanity.scm
# of expected passes      1
# of unexpected failures  2

That's it.  It fails.

(8) Using attached dlopen logger (gcc -fPIC -shared -o block-open.so block-open.c, then edit tools/uninstalled-env bottom before the exec to export LD_PRELOAD=$PWD/block-open.so), I get:
dlopen libguile-gi
dlopen /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen libguile-gi
dlopen /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-glib-2.62.6/lib/libgobject-2.0.so.0

There it is again.

(9) ldd /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5 |grep gobject
[...]
        libgobject-2.0.so.0 => /gnu/store/4jl613j0d5y6icbxxmwij75fd0i7qpwn-profile/lib/libgobject-2.0.so.0 (0x00007fe2a0677000)

And there is the other one.

Now how do I get more info, using Guix tools?

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.2: block-open.c --]
[-- Type: text/x-c++src, Size: 719 bytes --]

#define _GNU_SOURCE
#include <dlfcn.h>
#include <stdlib.h>
#include <string.h>
#include <stdio.h>

typedef void *(*dlopen_t)(const char *filename, int flags);

void *dlopen(const char *filename, int flags) {
	void* result;
	dlopen_t dlopen = dlsym(RTLD_NEXT, "dlopen");
	fprintf(stderr, "dlopen %s\n", filename);
	if (filename && strstr(filename, "/") == NULL && strstr(filename, "libcairo-gobject.so")) {
		fprintf(stderr, "dlopen blocked %s\n", filename);
		result = dlopen(filename, flags);
		if (result) {
			fprintf(stderr, "and would have been found! Aborting!\n");
			/*int* x = 0;
			*x = 5;
			abort();*/
		}
		//return NULL;
	}
	result = dlopen(filename, flags);
	return result;
}


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNOME in Guix

[Leo Prikler]

Hi,
Am Mittwoch, den 04.11.2020, 09:08 +0100 schrieb Danny Milosavljevic:
> Hi,
> 
> I've checked guile-gi test/insanity.scm again to find "hard"
> evidence.
> 
> For that, I've just checked out guile-gi anew, then ran
> test/insanity.scm.
> 
> Steps:
> [...]
> That's it.  It fails.
Okay, but it already states something along similar lines in the
description of the issue.
"It is pretty clear to me, that the main culprit here is a different
version of GLib being linked to Guile-GI than the one that should be
loaded through Guile-GI."

> (8) Using attached dlopen logger (gcc -fPIC -shared -o block-open.so
> block-open.c, then edit tools/uninstalled-env bottom before the exec
> to export LD_PRELOAD=$PWD/block-open.so), I get:
> dlopen libguile-gi
> dlopen /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5
> dlopen libguile-gi
> dlopen libguile-gi
> dlopen libguile-gi
> dlopen libguile-gi
> dlopen libguile-gi
> dlopen libguile-gi
> dlopen /gnu/store/xa1vfhfc42x655hi7vxqmbyvwldnz7r0-glib-
> 2.62.6/lib/libgobject-2.0.so.0
> 
> There it is again.
This is not meaningful at all.  Nothing here is "there again" except
for the libguile-gi, which if I understand it correctly is set up
multiple times with different init calls.  In particular, the internal
loading of libgobject by libguile-gi is obscured, hence why you need
(9) to figure out, what actually goes on.

> (9) ldd /home/dannym/src/guile-gi/guile-gi/./.libs/libguile-gi.so.5
> |grep gobject
> [...]
>         libgobject-2.0.so.0 =>
> /gnu/store/4jl613j0d5y6icbxxmwij75fd0i7qpwn-profile/lib/libgobject-
> 2.0.so.0 (0x00007fe2a0677000)
> 
> And there is the other one.
It is not yet clear, that this is "the other one".  You would first
need to readlink it, but indeed, as is pointed out in Guile-GI#96, they
are different.  

But we already know all this from our earlier discussion.  #96 clearly
states, that those two don't line up in `guix environment`, while they
do line up in `guix environment --no-graphs` and `guix build`.  This is
certainly "evidence" and it might even be "hard" depending on how you
view it, but the questions is how to interpret it.

> Now how do I get more info, using Guix tools?
What kind of information do you even want to gather?  To be honest, you
lose me every time you end up establishing knowledge, that already
exists between this thread and the mentioned Guile-GI issue.
I suppose if you want to look at how environments are built, building
them with higher verbosity would be a start, no?
For the purpose of this, it would probably suffice to look at something
simpler than guile-gi in its totality, perhaps just gobject-
introspection + glib (note, that you'd need Scheme code to access the
latter).

Regards, Leo

Re: GNOME in Guix

[Danny Milosavljevic]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

Hi,

On Wed, 04 Nov 2020 10:45:06 +0100
Leo Prikler <leo.prikler@student.tugraz.at> wrote:

> But we already know all this from our earlier discussion.

I *know* you already know that--but "we" don't.  I want someone to actually
proceed further, because I cannot.  Hence I posted this on guix-devel
(for the first time with an easy short recipe to follow).

> > Now how do I get more info, using Guix tools?  
> What kind of information do you even want to gather?  To be honest, you
> lose me every time you end up establishing knowledge, that already
> exists between this thread and the mentioned Guile-GI issue.

I am hereby asking someone who knows how this part of guix works to find out
what happens here.

> I suppose if you want to look at how environments are built, building
> them with higher verbosity would be a start, no?

Thanks.  That is what I wanted to know.

I tried guix environment -v 99 -d 1 and it adds nothing useful, especially not
how the other libgobject got in there.

So I still want to know.

> For the purpose of this, it would probably suffice to look at something
> simpler than guile-gi in its totality, perhaps just gobject-
> introspection + glib (note, that you'd need Scheme code to access the
> latter).

Sure, sounds good.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: GNOME in Guix

[Leo Prikler]

Hi,

Am Mittwoch, den 04.11.2020, 14:43 +0100 schrieb Danny Milosavljevic:
> Hi,
> 
> On Wed, 04 Nov 2020 10:45:06 +0100
> Leo Prikler <leo.prikler@student.tugraz.at> wrote:
> 
> > But we already know all this from our earlier discussion.
> 
> I *know* you already know that--but "we" don't.  I want someone to
> actually
> proceed further, because I cannot.  Hence I posted this on guix-devel
> (for the first time with an easy short recipe to follow).
Ah, okay.  I'm still a bit confused, given how verbose that recipe was,
but I get where you're coming from.

> Thanks.  That is what I wanted to know.
> 
> I tried guix environment -v 99 -d 1 and it adds nothing useful,
> especially not
> how the other libgobject got in there.
> 
> So I still want to know.
I think some important information gets lost here, because you already
built most of the packages.  You could try garbage-collecting them and
then start anew, but then you'd be building the entire profile at full
debug info.  Interestingly, nothing shows up for the generation of
profile.drv, which might indicate, that there are not actually two
packages in there.  It's at least a bit suspicious.

Having received the path of the profile.drv, you might now want to look
at what packages it references (recursively), specifically all mentions
of glib.  (Use emacs-guix for quick navigation through .drv files.)

For the record, in case you forgot to note the .drv, my profile
containing glib-with-documentation and gobject-introspection is

/gnu/store/cl1wz5wsgjy208n330z4p9f5kljzixp2-profile.drv

Regards, Leo

Re: GNOME in Guix

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 303 bytes --]

Hi Danny,

So if I got it right from the GitHub issue and the discussion with Leo,
this issue here mostly targets developers.

But if the guix-gui (or Nomad) package has the right inputs, then there
should be no grafting issue.

Is this correct?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

Re: GNOME in Guix

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1708 bytes --]

Danny Milosavljevic <dannym@scratchpost.org> writes:

>> [expose guix via dbus]
>
> You should keep in mind for this approach: impedance mismatch, slowness,
> different semantics, protocol versioning, and bugs in dbus (the latter
> of which I also regularily experience in guix--dbus bus hangs etc).
>
> The advantages you listed of better and easier privilege checking are
> valid, though.
>
> (In addition, I don't want a hard dependency to gtk+ in the guix package manager.
> But that can be avoided in other ways, too).

To add on top of what Danny said, I have had experience in writing GUI
over D-Bus with Nyxt: up to version 1.5 (when it was called Next), we
used to have a C server that would communicate over D-Bus to the Common
Lisp client.

We faced many issues.

- Performance was OK but could be critically bad on low-end hardware.

- Code is cumbersome to write (well, it's C :p), but maybe it's less of
  a problem if the server is Guix.

- Extensibility takes a huge toll because any modification to the server
  means the  client must be updated and vice-versa.

- What did it for us was input support: we had to encode and decode
  input events and share them over D-Bus.  This was very brittle, had
  tons of bugs and killed input method support.  I don't think it's
  doable to be honest.

When it comes to Guix, if the server does not deal with keybindings and
if the UI responsiveness only depends on the client (meaning that no
user input result depends on a result returned by the server), then I
suppose that D-Bus could work.  But I would go for GI first before going
down this road.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]