Required packages

Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

A question related to my previous posting, but also of independent 
justification: Should we maybe implement somthing similar to the "depends" 
field of Debian packages?

For instance, programming with mpc also requires mpfr and gmp; in fact, the 
first few lines of mpc.h read
  #include "gmp.h"
  #include "mpfr.h"

So it would make sense to automatically pull in gmp and mpfr when a user 
installs mpc. As dicussed with Ludovic, this could be handled by the 
"propagated-inputs" field of the package definition.

But it also would make sense to not use such an automatism. For instance, 
thanks to libtool, the mpc shared library is perfectly usable without 
installing mpfr into the user profile, as libmpc.la contains a pointer to 
the mpfr package:
  dependency_libs=' /nix/store/l0999b93cw0by4hcv6z5ykzwz0gw358x-
mpfr-3.1.1/lib/libmpfr.la /nix/store/ydxa85j3i21ac74dv0vbc6cxjjqpsfsv-
gmp-5.1.0/lib/libgmp.la -lm'
So a user who only wants to use a library and not develop with it may not 
be interested in getting all the dependent headers in the user profile.

An obvious approach would be to split into library and devel packages, as 
does debian. But I do not think we should go this road, it is quite 
convenient to get all of mpc in only one package.

What do you think?

Andreas

[-- Attachment #2: Type: text/html, Size: 4283 bytes --]

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> A question related to my previous posting, but also of independent 
> justification: Should we maybe implement somthing similar to the "depends" 
> field of Debian packages?

What’s this?

> For instance, programming with mpc also requires mpfr and gmp; in fact, the 
> first few lines of mpc.h read
>   #include "gmp.h"
>   #include "mpfr.h"
>
> So it would make sense to automatically pull in gmp and mpfr when a user 
> installs mpc. As dicussed with Ludovic, this could be handled by the 
> "propagated-inputs" field of the package definition.

s/could/should/  :-)

“Propagated inputs” was intended precisely for this use case.  I don’t
know why I didn’t use it but it’s a mistake since it requires packages
using MPC to know about MPC’s dependencies, and to hard-code them.

> But it also would make sense to not use such an automatism. For instance, 
> thanks to libtool, the mpc shared library is perfectly usable without 
> installing mpfr into the user profile, as libmpc.la contains a pointer to 
> the mpfr package:
>   dependency_libs=' /nix/store/l0999b93cw0by4hcv6z5ykzwz0gw358x-
> mpfr-3.1.1/lib/libmpfr.la /nix/store/ydxa85j3i21ac74dv0vbc6cxjjqpsfsv-
> gmp-5.1.0/lib/libgmp.la -lm'
> So a user who only wants to use a library and not develop with it may not 
> be interested in getting all the dependent headers in the user profile.

Yes, but mpfr.h and gmp.h still need to be in the user’s CPATH, which
contains ~/.guix-profile/include.  So putting them in the user’s profile
seems unavoidable.

I think you’re concerned about cluttering the user’s profile, right?

One thing that could be done to mitigate “cluttering” is to distinguish
between packages that were pulled as dependencies of what the user
installed, from packages the user explicitly installed (like apt does.)

WDYT?

> An obvious approach would be to split into library and devel packages, as 
> does debian. But I do not think we should go this road, it is quite 
> convenient to get all of mpc in only one package.

Yeah, I think it depends on the package.

The main criterion for determining whether to split in several outputs
would be the size of the closure of applications that use them.

Example: glibc is 37 MiB in total, plus 5 MiB for linux-libre-headers,
but lib/ is only 20 MiB.  So any application that uses glibc adds all
that to its closure.

GLib has 20 MiB of doc–good to avoid pulling that if all you want is to
run PulseAudio.  ;-)

Libtool’s “bin” output refers to the shell, which noticeably increases
the size of its closure.  Good to avoid that if all you want is
libltdl.so (like Guile does.)

But there are many more counterexamples, typically libraries of a few
hundred KiBs in total for which splitting is an inconvenience that
doesn’t provide any significant benefit.

Thoughts?

Ludo’.

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 2480 bytes --]

Am Montag, 4. Februar 2013 schrieb Ludovic Courtès:
> Andreas Enge <andreas@enge.fr> skribis:
> > A question related to my previous posting, but also of independent
> > justification: Should we maybe implement somthing similar to the
> > "depends" field of Debian packages?
> What’s this?

When A "depends on" B in debian jargon, then installing A automatically 
also installs B. In our case, a user issuing "guix-package -i mpc" would 
end up with gmp, mpfr and mpc in the profile.

> > But it also would make sense to not use such an automatism. For
> > instance, thanks to libtool, the mpc shared library is perfectly
> > usable without installing mpfr into the user profile, as libmpc.la
> > contains a pointer to
> > the mpfr package:
> >   dependency_libs=' /nix/store/l0999b93cw0by4hcv6z5ykzwz0gw358x-
> > mpfr-3.1.1/lib/libmpfr.la /nix/store/ydxa85j3i21ac74dv0vbc6cxjjqpsfsv-
> > gmp-5.1.0/lib/libgmp.la -lm'
> > So a user who only wants to use a library and not develop with it may
> > not be interested in getting all the dependent headers in the user
> > profile.
> 
> Yes, but mpfr.h and gmp.h still need to be in the user’s CPATH, which
> contains ~/.guix-profile/include.  So putting them in the user’s profile
> seems unavoidable.

Probably so; come to think of it, the cases where one wants to link against 
a dynamic library, but not write source code using the headers, should be 
really rare (linking a compiled program against a newer version of the 
library would be a possible case, which is somewhat contrary to the guix 
philosophy).

> I think you’re concerned about cluttering the user’s profile, right?

Not really. I just wonder what we should do, and what a user would expect. 
For instance, in debian, gcc depends on binutils and the glibc. So when you 
install gcc, you also get the other two packages. Is this what we want or 
not?

> One thing that could be done to mitigate “cluttering” is to distinguish
> between packages that were pulled as dependencies of what the user
> installed, from packages the user explicitly installed (like apt does.)

This would be the next question, only relevant if packages are indeed 
pulled in automatically. Then should they be uninstalled with the package 
that pulled them in? That is a bit complicated: If A and B need C and the 
user installs A and B (and thus also obtains C), C can only be uninstalled 
when A _and_ B are gone.

Andreas

[-- Attachment #2: Type: text/html, Size: 8562 bytes --]

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Am Montag, 4. Februar 2013 schrieb Ludovic Courtès:
>> Andreas Enge <andreas@enge.fr> skribis:
>> > A question related to my previous posting, but also of independent
>> > justification: Should we maybe implement somthing similar to the
>> > "depends" field of Debian packages?
>> What’s this?
>
> When A "depends on" B in debian jargon, then installing A automatically 
> also installs B. In our case, a user issuing "guix-package -i mpc" would 
> end up with gmp, mpfr and mpc in the profile.

OK.  That’s “user-environment-propagated-inputs”.  We could achieve that
by just changing guix-profile to install propagated inputs.  That’s
probably the right thing to do anyway.

[...]

>> Yes, but mpfr.h and gmp.h still need to be in the user’s CPATH, which
>> contains ~/.guix-profile/include.  So putting them in the user’s profile
>> seems unavoidable.
>
> Probably so; come to think of it, the cases where one wants to link against 
> a dynamic library, but not write source code using the headers, should be 
> really rare (linking a compiled program against a newer version of the 
> library would be a possible case, which is somewhat contrary to the guix 
> philosophy).

Yes.

>> I think you’re concerned about cluttering the user’s profile, right?
>
> Not really. I just wonder what we should do, and what a user would expect. 
> For instance, in debian, gcc depends on binutils and the glibc. So when you 
> install gcc, you also get the other two packages. Is this what we want or 
> not?

In the general case, I think so, yes.

Installing MPC should definitely install MPFR and GMP.

Similarly for Guile 2.0, whose public headers refer to libgc’s.

For GCC, the situation is more complicated, because GCC has a loose tie
to Binutils and libc.  For instance, it happily takes whichever ‘ld’
program is in $PATH.  It’s more closely tied to libc, though, because we
explicitly patch its spec strings to refer to it; so at least libc
should be a propagated input of GCC.

WDYT?

Thanks,
Ludo’.

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 428 bytes --]

Am Dienstag, 5. Februar 2013 schrieb Ludovic Courtès:
> For GCC, the situation is more complicated, because GCC has a loose tie
> to Binutils and libc.  For instance, it happily takes whichever ‘ld’
> program is in $PATH.  It’s more closely tied to libc, though, because we
> explicitly patch its spec strings to refer to it; so at least libc
> should be a propagated input of GCC.

Sounds good to me.

Andreas

[-- Attachment #2: Type: text/html, Size: 2369 bytes --]

Re: Required packages

[Ludovic Courtès]

Hi!

ludo@gnu.org (Ludovic Courtès) skribis:

> Andreas Enge <andreas@enge.fr> skribis:
>
>> Am Montag, 4. Februar 2013 schrieb Ludovic Courtès:
>>> Andreas Enge <andreas@enge.fr> skribis:
>>> > A question related to my previous posting, but also of independent
>>> > justification: Should we maybe implement somthing similar to the
>>> > "depends" field of Debian packages?
>>> What’s this?
>>
>> When A "depends on" B in debian jargon, then installing A automatically 
>> also installs B. In our case, a user issuing "guix-package -i mpc" would 
>> end up with gmp, mpfr and mpc in the profile.
>
> OK.  That’s “user-environment-propagated-inputs”.  We could achieve that
> by just changing guix-profile to install propagated inputs.  That’s
> probably the right thing to do anyway.

Commit 4dede02 implements that.

So now, when one installs MPC, MPFR and GMP automatically end up in the
profile as well.  When removing MPC, all three vanish.

Of course, if you had installed MPC and MPFR (say), and then remove MPC,
MPFR and GMP remain in the profile.  (IOW, it does the right thing.)

This required a slight change in the format of profile manifests, so
that the list of propagated inputs of each installed package is also
memorized.

Please test and report any issues!

Thanks,
Ludo’.

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1037 bytes --]

Am Mittwoch, 6. Februar 2013 schrieb Ludovic Courtès:
\> > OK.  That’s “user-environment-propagated-inputs”.  We could achieve
> > that by just changing guix-profile to install propagated inputs. 
> > That’s probably the right thing to do anyway.
> 
> Commit 4dede02 implements that.

Good initiative!

In my attempt to install guix from guix, I came upon the problem that the 
libgcrypt headers require the libgpg-error headers. This is a propagated 
input, so should in the future be installed automatically.

However, the next problem is:
In file included from /home/privat/.guix-profile/include/sys/socket.h:39:0,
                 from /home/privat/.guix-profile/include/gcrypt.h:42,
                 from conftest.c:65:
/home/privat/.guix-profile/include/bits/socket.h:342:24: fatal error: 
asm/socket.h: No such file or directory

The file is in linux-libre-headers; so I suppose this should also be a 
propagated input of libgcrypt? If you agree, I will modify the package 
accordingly.

Andreas

[-- Attachment #2: Type: text/html, Size: 4429 bytes --]

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 910 bytes --]

Concerning more checking in configure.ac:
The files included from nix require sqlite, it would be good to add a check 
for that.

Now I get the following error:
Backtrace:
In ice-9/boot-9.scm:
 157: 5 [catch #t #<catch-closure ca1780> ...]
In unknown file:
   ?: 4 [apply-smob/1 #<catch-closure ca1780>]
In ice-9/boot-9.scm:
  63: 3 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 421: 2 [eval # #]
In /home/privat/.guix-profile/bin/guild:
  54: 1 [main ("/home/privat/.guix-profile/bin/guild" "compile" "-L" ...)]
In unknown file:
   ?: 0 [setlocale 6 ""]

ERROR: In procedure setlocale:
ERROR: In procedure setlocale: Invalid argument
make[2]: *** [guix/base32.go] Fehler 1

The lines in guild around the error are:
  (if (defined? 'setlocale)
      (setlocale LC_ALL ""))
If I replace this by
  (if (defined? 'setlocale)
      (setlocale LC_ALL "C"))
compilation succeeds. A bug in guile/guild?

Andreas

[-- Attachment #2: Type: text/html, Size: 5892 bytes --]

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Now I get the following error:
> Backtrace:
> In ice-9/boot-9.scm:
>  157: 5 [catch #t #<catch-closure ca1780> ...]
> In unknown file:
>    ?: 4 [apply-smob/1 #<catch-closure ca1780>]
> In ice-9/boot-9.scm:
>   63: 3 [call-with-prompt prompt0 ...]
> In ice-9/eval.scm:
>  421: 2 [eval # #]
> In /home/privat/.guix-profile/bin/guild:
>   54: 1 [main ("/home/privat/.guix-profile/bin/guild" "compile" "-L" ...)]
> In unknown file:
>    ?: 0 [setlocale 6 ""]
>
> ERROR: In procedure setlocale:
> ERROR: In procedure setlocale: Invalid argument
> make[2]: *** [guix/base32.go] Fehler 1
>
> The lines in guild around the error are:
>   (if (defined? 'setlocale)
>       (setlocale LC_ALL ""))
> If I replace this by
>   (if (defined? 'setlocale)
>       (setlocale LC_ALL "C"))
> compilation succeeds. A bug in guile/guild?

Yes.  Fixed at
<http://git.savannah.gnu.org/cgit/guile.git/commit/?h=stable-2.0&id=e2c6bf3866d1186c60bacfbd4fe5037087ee5e3f>.

The core problem is that our glibc doesn’t install locale data (nor
timezone data).  Something to be fixed in the next core-updates.

Ludo’.

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> In my attempt to install guix from guix, I came upon the problem that the 
> libgcrypt headers require the libgpg-error headers. This is a propagated 
> input, so should in the future be installed automatically.

The future is actually yesterday.  :-)

> However, the next problem is:
> In file included from /home/privat/.guix-profile/include/sys/socket.h:39:0,
>                  from /home/privat/.guix-profile/include/gcrypt.h:42,
>                  from conftest.c:65:
> /home/privat/.guix-profile/include/bits/socket.h:342:24: fatal error: 
> asm/socket.h: No such file or directory
>
> The file is in linux-libre-headers; so I suppose this should also be a 
> propagated input of libgcrypt? If you agree, I will modify the package 
> accordingly.

It’s a bit trickier.  linux-libre-headers is propagated by glibc, so if
you install glibc, you get both.

However, glibc is an /implicit/ input of everything that uses
‘gnu-build-system’, such as libgcrypt.  So it doesn’t get installed when
you install libgcrypt.

I’m not sure whether glibc should be automatically installed as if it
were a propagated input.  If the answer were “yes”, I’m not sure how to
achieve it elegantly, because implicit inputs are hidden, by definition.

Thoughts?

Ludo’.

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 1480 bytes --]

Am Donnerstag, 7. Februar 2013 schrieb Ludovic Courtès:
> Andreas Enge <andreas@enge.fr> skribis:
> > In my attempt to install guix from guix, I came upon the problem that
> > the libgcrypt headers require the libgpg-error headers. This is a
> > propagated input, so should in the future be installed automatically.
> The future is actually yesterday.  :-)

Well, I first needed to compile yesterday's version, so my future was 
indeed in the future (although now it is in the past, a fate every future 
has to suffer). ;-)

> It’s a bit trickier.  linux-libre-headers is propagated by glibc, so if
> you install glibc, you get both.
> 
> However, glibc is an /implicit/ input of everything that uses
> ‘gnu-build-system’, such as libgcrypt.  So it doesn’t get installed when
> you install libgcrypt.
> 
> I’m not sure whether glibc should be automatically installed as if it
> were a propagated input.  If the answer were “yes”, I’m not sure how to
> achieve it elegantly, because implicit inputs are hidden, by definition.

Is this not too dangerous? It would mean that as soon as you install 
anything, you get the glibc. And in a system that has guix and a native 
glibc, things might stop working from there on...

Since I installed the guix glibc before you implemented the automatic 
installation of propagated inputs, I hit a problem that should have 
disappeared in the meantime. So I would suggest to do nothing.

Andreas

[-- Attachment #2: Type: text/html, Size: 5611 bytes --]

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Am Donnerstag, 7. Februar 2013 schrieb Ludovic Courtès:
>> Andreas Enge <andreas@enge.fr> skribis:
>> > In my attempt to install guix from guix, I came upon the problem that
>> > the libgcrypt headers require the libgpg-error headers. This is a
>> > propagated input, so should in the future be installed automatically.
>> The future is actually yesterday.  :-)
>
> Well, I first needed to compile yesterday's version, so my future was 
> indeed in the future (although now it is in the past, a fate every future 
> has to suffer). ;-)

I see.  Hopefully you don’t have to roll back to the future.  :-)

>> It’s a bit trickier.  linux-libre-headers is propagated by glibc, so if
>> you install glibc, you get both.
>> 
>> However, glibc is an /implicit/ input of everything that uses
>> ‘gnu-build-system’, such as libgcrypt.  So it doesn’t get installed when
>> you install libgcrypt.
>> 
>> I’m not sure whether glibc should be automatically installed as if it
>> were a propagated input.  If the answer were “yes”, I’m not sure how to
>> achieve it elegantly, because implicit inputs are hidden, by definition.
>
> Is this not too dangerous? It would mean that as soon as you install 
> anything, you get the glibc. And in a system that has guix and a native 
> glibc, things might stop working from there on...
>
> Since I installed the guix glibc before you implemented the automatic 
> installation of propagated inputs, I hit a problem that should have 
> disappeared in the meantime. So I would suggest to do nothing.

Yeah, I prefer it this way.

Thanks for the feedback,
Ludo’.

Re: Required packages

[Andreas Enge]

[-- Attachment #1: Type: text/plain, Size: 755 bytes --]

Am Mittwoch, 6. Februar 2013 schrieb Ludovic Courtès:
> > OK.  That’s “user-environment-propagated-inputs”.  We could achieve
> > that by just changing guix-profile to install propagated inputs. 
> > That’s probably the right thing to do anyway.
> Commit 4dede02 implements that.

It apparently works. The propagated input packages are hidden when issuing
   guix-package --list-installed ,
which seems a good idea so that the user is not confused by losing several 
packages at a time when just uninstalling one.

(Well, there is always source for confusion: When I install mpc and compile 
code needing gmp, everything will work; then I uninstall mpc, and things 
stop working, although the code does not require mpc.)

Andreas

[-- Attachment #2: Type: text/html, Size: 3133 bytes --]

Re: Required packages

[Ludovic Courtès]

Andreas Enge <andreas@enge.fr> skribis:

> Am Mittwoch, 6. Februar 2013 schrieb Ludovic Courtès:
>> > OK.  That’s “user-environment-propagated-inputs”.  We could achieve
>> > that by just changing guix-profile to install propagated inputs. 
>> > That’s probably the right thing to do anyway.
>> Commit 4dede02 implements that.
>
> It apparently works. The propagated input packages are hidden when issuing
>    guix-package --list-installed ,
> which seems a good idea so that the user is not confused by losing several 
> packages at a time when just uninstalling one.

Yes.

> (Well, there is always source for confusion: When I install mpc and compile 
> code needing gmp, everything will work; then I uninstall mpc, and things 
> stop working, although the code does not require mpc.)

Right, but that’s life.  ;-)

I’ll update the manual to document this behavior.

Thanks for testing!

Ludo’.