unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* pkg-config "Requires" fields and propagated inputs
@ 2015-03-19 20:48 Taylan Ulrich Bayırlı/Kammer
  2015-03-24 20:57 ` Ludovic Courtès
  2015-03-26 22:03 ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 2 replies; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-03-19 20:48 UTC (permalink / raw)
  To: guix-devel

I would appreciate it if someone helped me wrap my head around this.
Ideally I already got it and the following summary is correct; otherwise
please correct.

Pkg-config files (e.g. lib/pkgconfig/foo.pc for libfoo) may have a
"Requires" and a "Requires.private" field.

If I'm reading pkg-config documentation[0] right, Requires are full
run-time dependencies, meaning if a program uses libfoo which Requires
libbar, the program will also want/have to use libbar's interface; OTOH
Requires.private are "link-time" dependencies, i.e. libfoo dynamically
links to libbar, so libbar needs to exist on the system at run-time, but
a program using libfoo needn't use libbar's interface.

This would mean that Requires should be propagated inputs, and
Requires.private only normal inputs; they needn't be in the profile of
the user who installs libfoo; it's enough that libfoo refers directly to
their path in the store.

However, pkg-config isn't aware of compile-time/run-time dependency
differences; it's considered an error if a Requires.private of libfoo
isn't found, because as far as pkg-config is concerned, it means libfoo
is dysfunctional.  So we *do* need to propagate Requires.private, for
the sake of pkg-config.  (The problem mainly manifests in the form of
./configure scripts claiming libfoo isn't found when it's only libbar
that's missing, because in that case pkg-config returns an error to the
./configure script when inquired about libfoo.)

[0] http://people.freedesktop.org/~dbn/pkg-config-guide.html

P.S.: I'll see if I can write a tool that compares the union of the
Requires[.private] fields of all .pc files in a package to the package's
propagated inputs, so we can detect mismatches automatically.

Taylan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-19 20:48 pkg-config "Requires" fields and propagated inputs Taylan Ulrich Bayırlı/Kammer
@ 2015-03-24 20:57 ` Ludovic Courtès
  2015-03-24 22:52   ` Taylan Ulrich Bayırlı/Kammer
  2015-03-26 22:03 ` Taylan Ulrich Bayırlı/Kammer
  1 sibling, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2015-03-24 20:57 UTC (permalink / raw)
  To: Taylan Ulrich "Bayırlı/Kammer"; +Cc: guix-devel

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> If I'm reading pkg-config documentation[0] right, Requires are full
> run-time dependencies, meaning if a program uses libfoo which Requires
> libbar, the program will also want/have to use libbar's interface; OTOH
> Requires.private are "link-time" dependencies, i.e. libfoo dynamically
> links to libbar, so libbar needs to exist on the system at run-time, but
> a program using libfoo needn't use libbar's interface.

My reading of pkg-config(1) is that Requires.private is taken into
account only for ‘pkg-config foo --static --cflags/--libs’:

   Requires.private:
          A list of packages required by this package. The difference
          from Requires is that the packages listed under
          Requires.private are not taken into account when a flag list
          is computed for dynamically linked executable (i.e., when
          --static was not specified).

This is analogous to Libs.private in that regard.

In practice, we usually only care about shared libraries, thus...

> This would mean that Requires should be propagated inputs, and
> Requires.private only normal inputs;

... is mostly correct.  “Mostly,” because it might be that, unless
--static is passed, things in Requires.private don’t even need to be
present.

Do you have an example of a package that uses Requires.private?

> However, pkg-config isn't aware of compile-time/run-time dependency
> differences; it's considered an error if a Requires.private of libfoo
> isn't found,

Really, even without --static?

> because as far as pkg-config is concerned, it means libfoo is
> dysfunctional.  So we *do* need to propagate Requires.private, for the
> sake of pkg-config.  (The problem mainly manifests in the form of
> ./configure scripts claiming libfoo isn't found when it's only libbar
> that's missing, because in that case pkg-config returns an error to
> the ./configure script when inquired about libfoo.)

Do you have an example of that?

> P.S.: I'll see if I can write a tool that compares the union of the
> Requires[.private] fields of all .pc files in a package to the package's
> propagated inputs, so we can detect mismatches automatically.

Would be neat!

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-24 20:57 ` Ludovic Courtès
@ 2015-03-24 22:52   ` Taylan Ulrich Bayırlı/Kammer
  2015-03-25 16:56     ` Ludovic Courtès
  0 siblings, 1 reply; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-03-24 22:52 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

ludo@gnu.org (Ludovic Courtès) writes:

> Do you have an example of a package that uses Requires.private?

libxrandr is a good test case.  Indeed I patched it a few days ago
because some libraries in its Requires.private weren't propagated, which
fixed my issue.

>> However, pkg-config isn't aware of compile-time/run-time dependency
>> differences; it's considered an error if a Requires.private of libfoo
>> isn't found,
>
> Really, even without --static?
>
>> because as far as pkg-config is concerned, it means libfoo is
>> dysfunctional.  So we *do* need to propagate Requires.private, for the
>> sake of pkg-config.  (The problem mainly manifests in the form of
>> ./configure scripts claiming libfoo isn't found when it's only libbar
>> that's missing, because in that case pkg-config returns an error to
>> the ./configure script when inquired about libfoo.)
>
> Do you have an example of that?

I hit the issue with libxrandr while packaging mpv.  It has x11 under
its Requires.private, and:

$ ./pre-inst-env guix environment --pure -E guile guile-json mpv
GNU Guile 2.0.11
Copyright (C) 1995-2014 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guile-user)> ,use (srfi srfi-1)
scheme@(guile-user)> (define path (string-split (getenv "PKG_CONFIG_PATH") #\:))
scheme@(guile-user)> (find (lambda (s) (string-match "x11" s)) path)
$1 = "/gnu/store/99bgd1yjf51y0mrh2p40g499ida8l1qr-libx11-1.6.2/lib/pkgconfig"
scheme@(guile-user)> (system* "pkg-config" "--exists" "xrandr")
$2 = 0
scheme@(guile-user)> (system* "pkg-config" "--print-errors" "xrandr")
$3 = 0
scheme@(guile-user)> (system* "pkg-config" "--libs" "xrandr")
-L/gnu/store/05a5lxii4qvgf9jnxq85jvmbpalljr9a-libxrandr-1.4.2/lib -lXrandr
$4 = 0
scheme@(guile-user)> (define path* (filter (lambda (s) (not (string-match "x11" s))) path))
scheme@(guile-user)> (setenv "PKG_CONFIG_PATH" (string-join path* ":"))
scheme@(guile-user)> (system* "pkg-config" "--exists" "xrandr")
$5 = 256
scheme@(guile-user)> (system* "pkg-config" "--print-errors" "xrandr")
Package x11 was not found in the pkg-config search path.
Perhaps you should add the directory containing `x11.pc'
to the PKG_CONFIG_PATH environment variable
Package 'x11', required by 'xrandr', not found
$6 = 256
scheme@(guile-user)> (system* "pkg-config" "--libs" "xrandr")
-L/gnu/store/05a5lxii4qvgf9jnxq85jvmbpalljr9a-libxrandr-1.4.2/lib -lXrandr
$7 = 0
scheme@(guile-user)>

So while e.g. --libs actually works, --exists reports failure, and
--print-errors explains why.

One could say this is partially a bug, because if libxrandr statically
linked against libx11, then libx11 needn't exist at build nor run time.


It might be nice to have build-only propagated inputs.  On the other
hand, someone installing libxrandr in their profile will expect
pkg-config on libxrandr to work without errors.  More generally, they
will expect any build operation to work; i.e. a user might want to use
Guix-installed libraries for Guix-external development, and build-only
propagated inputs would mean annoying such users because they now have
to install them manually.  Unless we provide a special "install for
development" operation that is.  But the complexity grows. :-)

Taylan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-24 22:52   ` Taylan Ulrich Bayırlı/Kammer
@ 2015-03-25 16:56     ` Ludovic Courtès
  2015-03-25 19:48       ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2015-03-25 16:56 UTC (permalink / raw)
  To: Taylan Ulrich "Bayırlı/Kammer"; +Cc: guix-devel

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Do you have an example of a package that uses Requires.private?
>
> libxrandr is a good test case.  Indeed I patched it a few days ago
> because some libraries in its Requires.private weren't propagated, which
> fixed my issue.
>
>>> However, pkg-config isn't aware of compile-time/run-time dependency
>>> differences; it's considered an error if a Requires.private of libfoo
>>> isn't found,
>>
>> Really, even without --static?
>>
>>> because as far as pkg-config is concerned, it means libfoo is
>>> dysfunctional.  So we *do* need to propagate Requires.private, for the
>>> sake of pkg-config.  (The problem mainly manifests in the form of
>>> ./configure scripts claiming libfoo isn't found when it's only libbar
>>> that's missing, because in that case pkg-config returns an error to
>>> the ./configure script when inquired about libfoo.)
>>
>> Do you have an example of that?
>
> I hit the issue with libxrandr while packaging mpv.  It has x11 under
> its Requires.private, and:

[...]

> So while e.g. --libs actually works, --exists reports failure, and
> --print-errors explains why.
>
> One could say this is partially a bug, because if libxrandr statically
> linked against libx11, then libx11 needn't exist at build nor run time.

OK.  Thanks for digging.

> It might be nice to have build-only propagated inputs.  On the other
> hand, someone installing libxrandr in their profile will expect
> pkg-config on libxrandr to work without errors.  More generally, they
> will expect any build operation to work; i.e. a user might want to use
> Guix-installed libraries for Guix-external development, and build-only
> propagated inputs would mean annoying such users because they now have
> to install them manually.  Unless we provide a special "install for
> development" operation that is.  But the complexity grows. :-)

Heh.  Another option would be to have per-output propagated inputs.
That is, if a package has “out” and “lib” outputs, you’d want the
propagated inputs to apply only to “lib” since this is where the .pc is.
(This has been in ‘TODO’ for a couple of years.)

Thoughts?

Thanks,
Ludo’.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-25 16:56     ` Ludovic Courtès
@ 2015-03-25 19:48       ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 0 replies; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-03-25 19:48 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

ludo@gnu.org (Ludovic Courtès) writes:

> taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
>
>> It might be nice to have build-only propagated inputs.  On the other
>> hand, someone installing libxrandr in their profile will expect
>> pkg-config on libxrandr to work without errors.  More generally, they
>> will expect any build operation to work; i.e. a user might want to use
>> Guix-installed libraries for Guix-external development, and build-only
>> propagated inputs would mean annoying such users because they now have
>> to install them manually.  Unless we provide a special "install for
>> development" operation that is.  But the complexity grows. :-)
>
> Heh.  Another option would be to have per-output propagated inputs.
> That is, if a package has “out” and “lib” outputs, you’d want the
> propagated inputs to apply only to “lib” since this is where the .pc is.
> (This has been in ‘TODO’ for a couple of years.)
>
> Thoughts?

Sounds like an obvious feature to add, and in my (limited) experience
these library deps are the most common propagated inputs...

Taylan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-19 20:48 pkg-config "Requires" fields and propagated inputs Taylan Ulrich Bayırlı/Kammer
  2015-03-24 20:57 ` Ludovic Courtès
@ 2015-03-26 22:03 ` Taylan Ulrich Bayırlı/Kammer
  2015-03-31 12:14   ` Ludovic Courtès
  1 sibling, 1 reply; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-03-26 22:03 UTC (permalink / raw)
  To: guix-devel

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:

> P.S.: I'll see if I can write a tool that compares the union of the
> Requires[.private] fields of all .pc files in a package to the package's
> propagated inputs, so we can detect mismatches automatically.

So here's a first version.

https://notabug.org/taylanub/guix/src/master/missing-propagated-inputs.scm

Sample output from my system is below; evidently it's open to some
improvement.

Maybe I should try to move to a strategy that calls pkg-config instead
of parsing the files in Guile, but it might be nontrivial to get the
PKG_CONFIG_PATH right for each call, and doing many execs might be too
slow, so I'll first see how much further I can get with this strategy.

$ ./pre-inst-env guile -c '((@ (guix missing-propagated-inputs) print-missing-propagated-inputs))'
"fftw": ("-lfftw3" "-lm")
"fftwf": ("-lfftw3f" "-lm")
"apr": ("-ldl" "-lpthread" "-lcrypt" "-lrt" "-lapr-${APR_MAJOR_VERSION}")
"apr-util": ("apr-1" "-laprutil-${APRUTIL_MAJOR_VERSION}")
"portaudio": ("-lpthread" "-lm" "-lasound")
"libbs2b": ("-lm")
"jack2": ("-ljack")
"soxr": ("-lsoxr-lsr")
"avahi": ("glib-2.0" "avahi-client" "avahi-glib" "gobject-2.0" "-lavahi-client" "-lavahi-common" "-lavahi-core" "-lavahi-glib" "-lavahi-gobject")
"libatomic-ops": ("-latomic_ops")
"libcdio-paranoia": ("-lm" "-lcdio_cdda" "-lcdio_paranoia")
"libcdio": ("-lm" "-lcdio++" "-liso9660" "-liso9660++" "-ludf")
"guile": ("-lgc" "-lffi")
"xz": ("-llzma")
"curl": ("-lz" "-lldap" "-llber" "-lgss" "-lgnutls" "-lnettle" "-lssh2" "-lidn")
"cyrus-sasl": ("-lsasl2" "-lresolv" "-ldl")
"sqlite": ("-lsqlite3" "-lpthread" "-ldl")
"freetype": ("-lbz2")
"fontconfig": ("freetype2")
"gd": ("-lfreetype" "-lpng15" "-lm" "-lz")
"lcms": ("-llcms2" "-lm")
"glu": ("gl" "-lm")
"ftgl": ("freetype2" "-lm" "-lGL" "-lGLU")
"s2tc": ("-ltxc_dxtn")
"mesa": ("xcb-dri3" "xcb-dri2" "libudev" "-lEGL" "-ldl" "-lpthread" "-lm" "-lgbm" "-lGL" "-lOSMesa" "-lxatracker")
"dbus": ("-lpthread")
"glib": ("gobject-2.0" "gmodule-no-export-2.0" "gobject-2.0,gio-2.0" "-lgio-2.0" "-lresolv" "-lz" "-lgmodule-2.0" "-ldl" "-lgobject-2.0" "-lffi" "-lgthread-2.0")
"gnutls": ("libtasn1," "hogweed," "nettle," "-lgmp")
"nspr": ("-lnspr4" "-lplc4" "-lplds4")
"nss": ("-lnssutil3" "-lnss3" "-lsmime3" "-lssl3")
"pango": ("gobject-2.0" "glib-2.0" "gmodule-no-export-2.0" "fontconfig" "freetype2" "xft" "pangoft2" "-lpangocairo-1.0" "-lpangoft2-1.0" "-lpangoxft-1.0")
"cairo": ("xcb-shm" "xcb-render" "cairo-xlib" "freetype2" "gobject-2.0" "-lcairo-gobject" "-lz")
"harfbuzz": ("icu-uc" "-lharfbuzz-icu")
"guile": ("-lgc" "-lffi")
"icu4c": ("icu-uc" "icu-i18n" "icu-le" "-licui18n" "-licuio" "-licule" "-liculx" "-licudata" "-licuuc")
"leptonica": ("-llept")
"openjpeg": ("-lopenjp2" "-lm")
"libpng": ("-lpng15" "-lm")
"libtiff": ("-lz" "-ljpeg" "-llzma")
"libwebp": ("-lm" "-lwebpdecoder" "-lwebpdemux" "-lwebpmux")
"libunwind": ("libunwind-generic" "-lunwind-coredump" "-lunwind-generic" "-lunwind-ptrace" "-lunwind-setjmp")
"libusb": ("-lusb-1.0" "-lrt")
"util-linux": ("uuid" "blkid" "-lblkid" "-lmount" "-lsmartcols" "-luuid")
"alsa-lib": ("-lasound" "-lrt" "-lpthread" "-ldl" "-lm")
"eudev": ("gobject-2.0" "glib-2.0," "-lgudev-1.0" "-ludev")
"kmod": ("-lz" "-llzma")
"luajit": ("-l${libname}" "-ldl" "-lm")
"nettle": ("-lhogweed")
"tesseract-ocr": ("lept" "-lpthread" "-llept" "-ltesseract")
"openssl": ("libcrypto" "libssl" "-lcrypto" "-ldl" "-lssl")
"pciutils": ("-lpci")
"pcre": ("libpcre" "-lpcrecpp" "-lpcreposix")
"libsndfile": ("-lvorbisenc" "-lFLAC")
"pulseaudio": ("glib-2.0" "libpulse" "-lpulse-mainloop-glib" "-lpulsecommon-6.0" "-lpulse-simple" "-lpulse")
"python": ("-lutil" "-ldl" "-lpthread" "-lpython2.7")
"python": ("-lutil" "-ldl" "-lpthread" "-lpython3.3m")
"libxkbcommon": ("xcb-xkb" "xcb" "-lxkbcommon-x11")
"tcl": ("-lm" "-lieee" "-lpthread" "-ldl")
"enca": ("-lm")
"libdvdnav": ("dvdread" "-lpthread")
"vapoursynth": ("python-3.3" "libavcodec" "libavutil," "libswscale," "-lvapoursynth-script" "-lpython3.3m" "-ldl" "-lswscale" "-lavutil")
"libvpx": ("-lm" "-lpthread")
"ffmpeg": ("libavutil" "libswresample" "libavcodec" "libavformat" "libavresample" "libpostproc" "libswscale" "libavfilter" "-lavcodec" "-lz" "-lbz2" "-llzma" "-lm" "-lass" "-lbluray" "-lcaca" "-lfreetype" "-lfontconfig" "-lmp3lame" "-lopus" "-lpulse" "-lquvi" "-lsoxr" "-lspeex" "-logg" "-ltheoradec" "-ltheoraenc" "-ltwolame" "-lvorbis" "-lvorbisenc" "-lvpx" "-lxvidcore" "-lopenal" "-lcdio" "-lcdio_cdda" "-lcdio_paranoia" "-lavdevice" "-lavfilter" "-lavformat" "-lavresample" "-lavutil" "-lpostproc" "-lswresample" "-lswscale")
"libcaca": ("-lcaca++" "-lz")
"libva": ("-lva-${display}" "-lva-tpi")
"libbluray": ("-lfreetype" "-lfontconfig" "-lxml2" "-ldl")
"libdvdnav": ("dvdread" "-lpthread")
"libass": ("freetype2" "enca," "-lm")
"libdrm": ("-ldrm_intel" "-ldrm_nouveau" "-ldrm_radeon" "-lkms")
"libvorbis": ("-lm" "-lvorbisenc" "-lvorbisfile")
"libtheora": ("theoradec," "-ltheoradec" "-ltheoraenc")
"speex": ("-lm" "-lspeexdsp")
"ao": ("-ldl" "-lpthread")
"flac": ("-lFLAC++" "-lm")
"opus": ("-lm")
"libxml2": ("-lm")
"libxslt": ("libxml-2.0" "-lgpg-error" "-lgcrypt" "-lm" "-ldl" "-lz" "-lxml2" "-lexslt")
"libxrender": ("x11" "xproto")
"libsm": ("xproto")
"libxfixes": ("xproto" "x11")
"libxscrnsaver": ("xext" "x11" "-lXss")
"xcb-util-renderutil": ("xcb-render" "-lxcb-render-util")
"libxv": ("xproto" "xext" "x11")
"xcb-util-image": ("xcb-shm" "-lxcb-image")
"libxxf86vm": ("x11")
"libxcb": ("xcb-xfixes" "xcb-sync" "xcb-randr" "xcb-render" "xcb-shape" "xcb-shm" "xcb-xv" "-lxcb-composite" "-lxcb-damage" "-lxcb-dpms" "-lxcb-dri2" "-lxcb-dri3" "-lxcb-glx" "-lxcb-present" "-lxcb-randr" "-lxcb-record" "-lxcb-render" "-lxcb-res" "-lxcb-screensaver" "-lxcb-shape" "-lxcb-shm" "-lxcb-sync" "-lxcb-xevie" "-lxcb-xf86dri" "-lxcb-xfixes" "-lxcb-xinerama" "-lxcb-xkb" "-lxcb-xprint" "-lxcb-xtest" "-lxcb-xv" "-lxcb-xvmc")
"libx11": ("xproto" "-lX11-xcb" "-lpthread")
"xcb-util-wm": ("-lxcb-ewmh" "-lxcb-icccm")
"libxtst": ("xextproto" "xext" "x11")
"libxdmcp": ("xproto")
"xcb-util-keysyms": ("-lxcb-keysyms")
"libxinerama": ("xext" "x11")
"libxext": ("x11")
"libxft": ("xproto" "freetype2" "fontconfig," "xrender,")

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-26 22:03 ` Taylan Ulrich Bayırlı/Kammer
@ 2015-03-31 12:14   ` Ludovic Courtès
  2015-04-29 20:17     ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2015-03-31 12:14 UTC (permalink / raw)
  To: Taylan Ulrich "Bayırlı/Kammer"; +Cc: guix-devel

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:

> taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") writes:
>
>> P.S.: I'll see if I can write a tool that compares the union of the
>> Requires[.private] fields of all .pc files in a package to the package's
>> propagated inputs, so we can detect mismatches automatically.
>
> So here's a first version.
>
> https://notabug.org/taylanub/guix/src/master/missing-propagated-inputs.scm
>
> Sample output from my system is below; evidently it's open to some
> improvement.

Nice!

> Maybe I should try to move to a strategy that calls pkg-config instead
> of parsing the files in Guile, but it might be nontrivial to get the
> PKG_CONFIG_PATH right for each call, and doing many execs might be too
> slow, so I'll first see how much further I can get with this strategy.

Yeah forking would be slow; parsing sounds good.  However, the ‘satisfy’
predicates could be based on actual package contents (when available):
looking for actual .pc files and libraries.

> $ ./pre-inst-env guile -c '((@ (guix missing-propagated-inputs) print-missing-propagated-inputs))'
> "fftw": ("-lfftw3" "-lm")
> "fftwf": ("-lfftw3f" "-lm")

This is a false positives: -lfftw3* is actually provided by the package
itself, and -lm is part of libc, so it’s always found (libc libraries
will have to be special-cased.)

> "apr": ("-ldl" "-lpthread" "-lcrypt" "-lrt" "-lapr-${APR_MAJOR_VERSION}")

Again libc libraries, and the library of APR itself.  Problem is that
this one requires almost a full reimplementation of pkg-config, to be
able to do variable substitution (probably not insurmountable though.)

> "guile": ("-lgc" "-lffi")

guile-2.0.pc has this:

  Libs: -L${libdir} -lguile-2.0 -L/gnu/store/dxfv00pgmnhxkj5jh3yxyp0q94l6i2ch-libgc-7.4.2/lib -lgc 

So here -lgc is a false positive: libgc is found by virtue of the -L
flag.

‘-lffi’ only appears in ‘Libs.private’ (static linking) so I would just
ignore it.


It looks like there are 3 classes of false positives here that would
need to be addressed to make the report more directly readable.

WDYT?

Thank you!

Ludo’.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-03-31 12:14   ` Ludovic Courtès
@ 2015-04-29 20:17     ` Taylan Ulrich Bayırlı/Kammer
  2015-05-03 19:48       ` Ludovic Courtès
  0 siblings, 1 reply; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-04-29 20:17 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

Picking this topic back up after some things got in between:

ludo@gnu.org (Ludovic Courtès) writes:

> taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:
>
>> Maybe I should try to move to a strategy that calls pkg-config instead
>> of parsing the files in Guile, but it might be nontrivial to get the
>> PKG_CONFIG_PATH right for each call, and doing many execs might be too
>> slow, so I'll first see how much further I can get with this strategy.
>
> Yeah forking would be slow; parsing sounds good.  However, the ‘satisfy’
> predicates could be based on actual package contents (when available):
> looking for actual .pc files and libraries.

I overhauled the strategy somewhat, and indeed look for actual .pc files
(under $prefix/lib/pkgconfig directories) and libfoo.so files (under
$prefix/lib directories) now.  For .pc files, the extension is stripped,
for libfoo.so files, the foo part is taken.  These then serve as exact
pkg-config package names and library names respectively, making
everything more robust.  Thanks for the idea!

>> $ ./pre-inst-env guile -c '((@ (guix missing-propagated-inputs) print-missing-propagated-inputs))'
>> "fftw": ("-lfftw3" "-lm")
>> "fftwf": ("-lfftw3f" "-lm")
>
> This is a false positives: -lfftw3* is actually provided by the package
> itself, and -lm is part of libc, so it’s always found (libc libraries
> will have to be special-cased.)

Libc components are now handled.  The fftw3 stuff is handled fine under
the new strategy.

>> "apr": ("-ldl" "-lpthread" "-lcrypt" "-lrt" "-lapr-${APR_MAJOR_VERSION}")
>
> Again libc libraries, and the library of APR itself.  Problem is that
> this one requires almost a full reimplementation of pkg-config, to be
> able to do variable substitution (probably not insurmountable though.)

Implementing variable substitution was fairly straightforward, so these
are fine now.

>> "guile": ("-lgc" "-lffi")
>
> guile-2.0.pc has this:
>
>   Libs: -L${libdir} -lguile-2.0 -L/gnu/store/dxfv00pgmnhxkj5jh3yxyp0q94l6i2ch-libgc-7.4.2/lib -lgc
>
> So here -lgc is a false positive: libgc is found by virtue of the -L
> flag.

The -L flags are handled too now; the directory they point at is handled
equivalently to $prefix/lib directories under the new strategy.
(I.e. they're scanned for libfoo.so files.)

> ‘-lffi’ only appears in ‘Libs.private’ (static linking) so I would just
> ignore it.

You're right; unlike Requires.private, Libs.private has no effect on
e.g. pkg-config --exists, so it's really only for static linking; I
removed the parsing of Libs.private fields.

> It looks like there are 3 classes of false positives here that would
> need to be addressed to make the report more directly readable.

I think all these false positives are fixed now!

New sample output from my machine:

portaudio: pkg-config: () libs: ("asound")
avahi: pkg-config: ("glib-2.0" "gobject-2.0" "glib-2.0") libs: ()
libatomic-ops: pkg-config: () libs: ("atomic_ops")
ijs: pkg-config: () libs: ("ijs")
mesa: pkg-config: ("xcb-dri3" "xcb-dri2" "libudev" "xcb-dri3" "xcb-dri2") libs: ()
dbus-glib: pkg-config: ("gobject-2.0" "glib-2.0" "dbus-1") libs: ()
libcroco: pkg-config: ("libxml-2.0" "glib-2.0") libs: ()
gst-plugins-base: pkg-config: ("gio-2.0" "glib-2.0" "gio-2.0") libs: ()
harfbuzz: pkg-config: ("icu-uc") libs: ()
gtk+: pkg-config: ("gio-2.0" "cairo" "gio-2.0" "cairo" "gio-2.0" "cairo") libs: ()
pango: pkg-config: ("gobject-2.0" "glib-2.0" "gmodule-no-export-2.0" "fontconfig" "freetype2" "xft") libs: ()
cairo: pkg-config: ("xcb-shm" "xcb-render" "xcb-render") libs: ("z" "z")
eudev: pkg-config: ("gobject-2.0" "glib-2.0") libs: ()
qpdf: pkg-config: ("zlib") libs: ()
poppler: pkg-config: ("cairo" "cairo" "gio-2.0" "gobject-2.0" "glib-2.0") libs: ()
pulseaudio: pkg-config: ("glib-2.0") libs: ()
libdvdnav: pkg-config: ("dvdread") libs: ()
libxslt: pkg-config: ("libxml-2.0" "libxml-2.0") libs: ("z" "z")
libxv: pkg-config: ("xproto" "xext" "x11") libs: ()
libx11: pkg-config: ("xproto") libs: ()
libxmu: pkg-config: ("xt" "x11" "xproto" "xext" "xt" "x11" "x11" "xproto" "x11") libs: ()
libxinerama: pkg-config: ("xext" "x11") libs: ()
libxft: pkg-config: ("xproto") libs: ()
libxext: pkg-config: ("x11") libs: ()
libxaw: pkg-config: ("xproto" "x11" "xproto" "x11") libs: ()
libxdmcp: pkg-config: ("xproto") libs: ()
libxpm: pkg-config: ("x11" "x11") libs: ()
libxkbfile: pkg-config: ("kbproto" "x11") libs: ()
xkbcomp: pkg-config: ("x11") libs: ()
libxxf86vm: pkg-config: ("x11") libs: ()
libxrender: pkg-config: ("x11" "xproto" "x11") libs: ()
libxfixes: pkg-config: ("xproto" "x11") libs: ()
libxt: pkg-config: ("xproto") libs: ()
libsm: pkg-config: ("xproto") libs: ()

Taylan

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-04-29 20:17     ` Taylan Ulrich Bayırlı/Kammer
@ 2015-05-03 19:48       ` Ludovic Courtès
  2015-05-03 20:57         ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 1 reply; 10+ messages in thread
From: Ludovic Courtès @ 2015-05-03 19:48 UTC (permalink / raw)
  To: Taylan Ulrich "Bayırlı/Kammer"; +Cc: guix-devel

taylanbayirli@gmail.com (Taylan Ulrich "Bayırlı/Kammer") skribis:


[...]

>> It looks like there are 3 classes of false positives here that would
>> need to be addressed to make the report more directly readable.
>
> I think all these false positives are fixed now!

Impressive!

> New sample output from my machine:
>
> portaudio: pkg-config: () libs: ("asound")
> avahi: pkg-config: ("glib-2.0" "gobject-2.0" "glib-2.0") libs: ()
> libatomic-ops: pkg-config: () libs: ("atomic_ops")

So ‘libs’ means that the .pc has “-latomic_ops”, right?

> ijs: pkg-config: () libs: ("ijs")
> mesa: pkg-config: ("xcb-dri3" "xcb-dri2" "libudev" "xcb-dri3" "xcb-dri2") libs: ()
> dbus-glib: pkg-config: ("gobject-2.0" "glib-2.0" "dbus-1") libs: ()
> libcroco: pkg-config: ("libxml-2.0" "glib-2.0") libs: ()
> gst-plugins-base: pkg-config: ("gio-2.0" "glib-2.0" "gio-2.0") libs: ()
> harfbuzz: pkg-config: ("icu-uc") libs: ()
> gtk+: pkg-config: ("gio-2.0" "cairo" "gio-2.0" "cairo" "gio-2.0" "cairo") libs: ()
> pango: pkg-config: ("gobject-2.0" "glib-2.0" "gmodule-no-export-2.0" "fontconfig" "freetype2" "xft") libs: ()
> cairo: pkg-config: ("xcb-shm" "xcb-render" "xcb-render") libs: ("z" "z")

delete-duplicates?  :-)

That looks nice.  So where’s the code?  ;-)

What are your thoughts on how to integrate it?  Should we make a ‘lint’
pass, or is there something that can be turned into a build phase?

Thanks!

Ludo’.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: pkg-config "Requires" fields and propagated inputs
  2015-05-03 19:48       ` Ludovic Courtès
@ 2015-05-03 20:57         ` Taylan Ulrich Bayırlı/Kammer
  0 siblings, 0 replies; 10+ messages in thread
From: Taylan Ulrich Bayırlı/Kammer @ 2015-05-03 20:57 UTC (permalink / raw)
  To: Ludovic Courtès; +Cc: guix-devel

ludo@gnu.org (Ludovic Courtès) writes:

>> portaudio: pkg-config: () libs: ("asound")
>> avahi: pkg-config: ("glib-2.0" "gobject-2.0" "glib-2.0") libs: ()
>> libatomic-ops: pkg-config: () libs: ("atomic_ops")
>
> So ‘libs’ means that the .pc has “-latomic_ops”, right?

Indeed.

There are libatomic_ops.a and .la files in the $prefix/lib of that
package, but no .so files.  I'm not sure what this means.

>> ijs: pkg-config: () libs: ("ijs")
>> mesa: pkg-config: ("xcb-dri3" "xcb-dri2" "libudev" "xcb-dri3" "xcb-dri2") libs: ()
>> dbus-glib: pkg-config: ("gobject-2.0" "glib-2.0" "dbus-1") libs: ()
>> libcroco: pkg-config: ("libxml-2.0" "glib-2.0") libs: ()
>> gst-plugins-base: pkg-config: ("gio-2.0" "glib-2.0" "gio-2.0") libs: ()
>> harfbuzz: pkg-config: ("icu-uc") libs: ()
>> gtk+: pkg-config: ("gio-2.0" "cairo" "gio-2.0" "cairo" "gio-2.0" "cairo") libs: ()
>> pango: pkg-config: ("gobject-2.0" "glib-2.0" "gmodule-no-export-2.0" "fontconfig" "freetype2" "xft") libs: ()
>> cairo: pkg-config: ("xcb-shm" "xcb-render" "xcb-render") libs: ("z" "z")
>
> delete-duplicates?  :-)

Indeed.  Done.

> That looks nice.  So where’s the code?  ;-)

Here's the repo:

https://notabug.org/taylanub/guix

> What are your thoughts on how to integrate it?  Should we make a ‘lint’
> pass, or is there something that can be turned into a build phase?

It should be possible to make a build phase of it, using
'requires-and-libs-for-path' on the output directories, then using
'missing-packages' and 'missing-libraries' to find any pkg-config
packages, and libraries, which wouldn't be satisfied during use of the
package/output given its propagated inputs.

I'll try implementing this.

Taylan

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2015-05-03 20:57 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-19 20:48 pkg-config "Requires" fields and propagated inputs Taylan Ulrich Bayırlı/Kammer
2015-03-24 20:57 ` Ludovic Courtès
2015-03-24 22:52   ` Taylan Ulrich Bayırlı/Kammer
2015-03-25 16:56     ` Ludovic Courtès
2015-03-25 19:48       ` Taylan Ulrich Bayırlı/Kammer
2015-03-26 22:03 ` Taylan Ulrich Bayırlı/Kammer
2015-03-31 12:14   ` Ludovic Courtès
2015-04-29 20:17     ` Taylan Ulrich Bayırlı/Kammer
2015-05-03 19:48       ` Ludovic Courtès
2015-05-03 20:57         ` Taylan Ulrich Bayırlı/Kammer

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).