Rethinking propagated inputs?

Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi Guix,

some while ago we made the decision to propagate inputs, that are
mentioned in pkg-config files, the rationale being that those
propagated inputs will be needed in packages in order to compile.  This
has saved us some typing, but at a cost.  For instance, it is now no
longer possible to upgrade "zile" and "icecat" independently, because
both propagate glib.  "libreoffice" and "telegram-desktop", two
packages that have failed us loudly before, are also in that list.  To
see what else is in here, you might want to use the following snippet
in a guix repl:

--8<---------------cut here---------------start------------->8---
(use-modules (guix packages)
             (gnu packages)
             (guix discovery)
             (srfi srfi-1)
             (srfi srfi-26))

(display
 (fold-packages
 (lambda (p s)
   (if (any (compose (lambda (input)
                       (and (package? input)
                            (string=? "glib" (package-name input))))
                     cadr)
            (package-transitive-inputs p))
       (cons (package-name p) s)
       s))
 '()
 (all-modules %default-package-module-path)))
--8<---------------cut here---------------end--------------->8---

It returns more than 1400 packages – a bit less than 10% of Guix. 
Needless to say, that's a bad thing and I think we should do something
about it, particularly when it comes to leaf packages, that users are
likely to install.

Does anyone have an idea how we should handle propagations for the sake
of pkg-config?  Perhaps we could add "linked-inputs", which are added
when building packages and environments when not using --ad-hoc, but
not when union-building profiles.  WDYT?

Re: Rethinking propagated inputs?

[Sarah Morgensen]

Hi Liliana,

(Efraim, I've Cc'd you since you're working on re-doing Rust inputs.)

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Does anyone have an idea how we should handle propagations for the sake
> of pkg-config?  Perhaps we could add "linked-inputs", which are added
> when building packages and environments when not using --ad-hoc, but
> not when union-building profiles.  WDYT?

I know nothing about pkg-config, but such an input would help
simplify things for Go (and I think for Rust) since many inputs need to
be propagated only at build-time.

What do you think of "build-propagated-inputs"?

(A quick search of the ML turned up one previous discussion [0]; does
anyone know of others?)

[0] https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00362.html

--
Sarah

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Am Samstag, den 04.09.2021, 17:50 -0700 schrieb Sarah Morgensen:
> Hi Liliana,
> 
> (Efraim, I've Cc'd you since you're working on re-doing Rust inputs.)
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> 
> > Does anyone have an idea how we should handle propagations for the
> > sake of pkg-config?  Perhaps we could add "linked-inputs", which
> > are added when building packages and environments when not using --
> > ad-hoc, but not when union-building profiles.  WDYT?
> 
> I know nothing about pkg-config, but such an input would help
> simplify things for Go (and I think for Rust) since many inputs need
> to be propagated only at build-time.
To be fair, I wasn't not thinking about Go and Rust, which at least on
the surface appear to have similar propagation semantics.  I do however
not know whether all currently propagated inputs from those two could
be reclassified as linked-inputs.  FWIW I don't think (most) Emacs,
Python or Guile packages work that way, but I do know of at least one
that would profit from having linked-inputs.

> What do you think of "build-propagated-inputs"?
We don't call things build-inputs here in Guix land, that's a no-no :P

> (A quick search of the ML turned up one previous discussion [0]; does
> anyone know of others?)
> 
> [0] 
> https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00362.html
W.r.t. native-inputs, I think native-inputs should propagate
propagated-inputs, but not linked-inputs.  Makes sense, doesn't it?

Re: Rethinking propagated inputs?

[Bengt Richter]

Hi,

On +2021-09-05 09:36:30 +0200, Liliana Marie Prikler wrote:
> Am Samstag, den 04.09.2021, 17:50 -0700 schrieb Sarah Morgensen:
> > Hi Liliana,
> > 
> > (Efraim, I've Cc'd you since you're working on re-doing Rust inputs.)
> > 
> > Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> > 
> > > Does anyone have an idea how we should handle propagations for the
> > > sake of pkg-config?  Perhaps we could add "linked-inputs", which
> > > are added when building packages and environments when not using --
> > > ad-hoc, but not when union-building profiles.  WDYT?
> > 
> > I know nothing about pkg-config, but such an input would help
> > simplify things for Go (and I think for Rust) since many inputs need
> > to be propagated only at build-time.
> To be fair, I wasn't not thinking about Go and Rust, which at least on
> the surface appear to have similar propagation semantics.  I do however
> not know whether all currently propagated inputs from those two could
> be reclassified as linked-inputs.  FWIW I don't think (most) Emacs,
> Python or Guile packages work that way, but I do know of at least one
> that would profit from having linked-inputs.
> 
> > What do you think of "build-propagated-inputs"?
> We don't call things build-inputs here in Guix land, that's a no-no :P
>

Is there an official guix  jargon file or glossary file or texi file or
wikimedia/wiktionary/wikipedia clone on gnu.org that non-cognoscenti
could use to get a clue?

Is there a thread that on that topic making any progress on making it
happen?

when someone in a thread like this offers a candidate official definition,
(off-topic sort of, but meta-on-topic for relevant documentation)
could it be snip-quoted for easy search and aggregation for maintainers
of official definitions and translations? E.g.
(or actually borrow some rfc0842 or descendant so an attached file
generates a usuable section in mail archives that can be snarfed automatically?)

--8<---------------cut here---------------start------------->8---
X-Content-type: Cadidate-guix-jargon-definition
Ad lib comment and metacomment ended by blank line ...
"> We don't call things build-inputs here in Guix land, that's a no-no :P"

build-propagated-inputs:
	<please fill in :) >
--8<---------------cut here---------------end--------------->8---

> > (A quick search of the ML turned up one previous discussion [0]; does
> > anyone know of others?)
> > 
> > [0] 
> > https://lists.gnu.org/archive/html/guix-devel/2017-02/msg00362.html
> W.r.t. native-inputs, I think native-inputs should propagate
> propagated-inputs, but not linked-inputs.  Makes sense, doesn't it?
> 
> 

-- 
Regards,
Bengt Richter

Guix Jargon File (WAS: Rethinking propagated inputs?)

[Liliana Marie Prikler]

Hi,

Am Sonntag, den 05.09.2021, 11:50 +0200 schrieb Bengt Richter:
> > We don't call things build-inputs here in Guix land, that's a no-no 
> > :P
> 
> Is there an official guix jargon file or glossary file or texi file
> or wikimedia/wiktionary/wikipedia clone on gnu.org that non-
> cognoscenti could use to get a clue?
AFAIK no, you more or less have to go by what the manual tells you.  As
for why we have native-inputs and not build-inputs like other distros,
it's because people often misclassify "build" inputs, so the definition
actually does more harm than good.  Guix knows which files are actually
"just used for build" by what ends up in the store, with some
exceptions like UTF-32 encoded strings.

> Is there a thread that on that topic making any progress on making it
> happen?
AFAIK no.

> when someone in a thread like this offers a candidate official
> definition, (off-topic sort of, but meta-on-topic for relevant
> documentation) could it be snip-quoted for easy search and
> aggregation for maintainers of official definitions and translations?
> E.g.
> (or actually borrow some rfc0842 or descendant so an attached file
> generates a usuable section in mail archives that can be snarfed
> automatically?)
> 
> --8<---------------cut here---------------start------------->8---
> X-Content-type: Cadidate-guix-jargon-definition
> Ad lib comment and metacomment ended by blank line ...
> "> We don't call things build-inputs here in Guix land, that's a no-
> no :P"
> 
> build-propagated-inputs:
> 	<please fill in :) >
> --8<---------------cut here---------------end--------------->8---
When you quote someone like that out-of-context, you run a risk of
misrepresenting what is actually stated.  What I mean, is that a
package field called something along the lines of "build-inputs" is
likely to confuse Guix veterans and newcomers alike, as evidenced by
the following reply:

Am Sonntag, den 05.09.2021, 10:06 +0000 schrieb Attila Lendvai:
> potentially worthless two cents from a newcomer's perspective:
> 'build-time' and 'run-time' are well established concepts in the
> wider community.
And one, that is well misunderstood.  

> if i were reading 'linked-inputs' in a package definition, i wouldn't
> associate it to being the set of build-time dependencies.
That's not what linked-inputs are, though.  Take the following
paragraph from propagated-inputs:

> For example this is necessary when packaging a C/C++ library
> that needs headers of another library to compile, or when a
> pkg-config file refers to another one via its ‘Requires’
> field.
This use case of propagated inputs explains why they need to be
propagated when given as a (propagated-)input to a package, but not
when given as a native input or merely existing in a profile.

The – required if we go by other systems – use case of installing
libraries system- or user-wide is already discouraged by Guix, for it
is not needed.  As long as we can spawn an environment, in which we can
compile these things, it should be enough.

Note, that this is not equivalent to being a "build-time" dependency. 
Going by Gentoo's definition "Build dependencies are used to specify
any dependencies that are required to unpack, patch, compile, test or
install the package", GCC is a build dependency of nearly any C program
(and a native one at that, i.e. BDEPEND in Gentoo), but it's not a
linked-dependency, because there are numerous ways in which other
programs could use it without ever needing to invoke GCC.  Guix, of
course, includes GCC as an implicit native input anyway, but that's a
different topic.

Regards

Re: Guix Jargon File (WAS: Rethinking propagated inputs?)

[Bengt Richter]

Hi Liliana,

Thank you for starting this renamed thread (as I should have done).

I think a people who are just looking at _maybe_ installing guix
should have an easy way to look up terms they haven't seen before.

But really I am more interested in promoting the idea of a snippet-quoting
convention modeled on a subset of mime email standards.

Very simple, but capable of containing and transferring anything unambiguously
(if not always with efficient transmission encodings).

We can of course already do that with signed and attached files, and we can
archive them and retrieve them, but I am interested in retrieving little pieces
and making it easy to mark things in arbitratry contexts (like this email or
a cannibal-friendly program source) so that simple snarfing utilities will be
able to extract snippet-quote info based on tags and identifiers or anything
in the headers or content per search options much like for any search engine.

This is to create a simple contribution mechanism as well as a format
for retrieval.

I have seen many code snippets from developers that are tutorial material
as well as practical how-tos for debugging and browsing guix.

Wouldn't it be nice if they were snip-quoted so that we could extract them
from mail archives in a better way than searching the raw archives, or having
to browse though treads and extract nuggets by hand?

simply:
--8<---------------cut here---------------start------------->8---
header part, ending with blank line

optional content part, encoded and delimited or referenced per header info
--8<---------------cut here---------------end--------------->8---


The header part could start with just prefixing GX- like the optional custom
header X- prefix described in mime rfcs, and we could borrow whatever was useful.

Tbc.. So called "real life" demands I postpone making a decent real example
'til later, sorry ;/

Please excuse the big top-post. I had intended to comment and edit in line ;/

BTW, I know "info guix|grep -i whatever" often gives clues about whatever, for pursuing
"C-s whatever" once inside "info guix whatever", but though concept and api indices
are great, they are not a Jargon File, and not as handy for an outsider :)

On +2021-09-05 12:50:56 +0200, Liliana Marie Prikler wrote:
> Hi,
> 
> Am Sonntag, den 05.09.2021, 11:50 +0200 schrieb Bengt Richter:
> > > We don't call things build-inputs here in Guix land, that's a no-no 
> > > :P
> > 
> > Is there an official guix jargon file or glossary file or texi file
> > or wikimedia/wiktionary/wikipedia clone on gnu.org that non-
> > cognoscenti could use to get a clue?
> AFAIK no, you more or less have to go by what the manual tells you.  As
> for why we have native-inputs and not build-inputs like other distros,
> it's because people often misclassify "build" inputs, so the definition
> actually does more harm than good.  Guix knows which files are actually
> "just used for build" by what ends up in the store, with some
> exceptions like UTF-32 encoded strings.
> 
> > Is there a thread that on that topic making any progress on making it
> > happen?
> AFAIK no.
> 
> > when someone in a thread like this offers a candidate official
> > definition, (off-topic sort of, but meta-on-topic for relevant
> > documentation) could it be snip-quoted for easy search and
> > aggregation for maintainers of official definitions and translations?
> > E.g.
> > (or actually borrow some rfc0842 or descendant so an attached file
> > generates a usuable section in mail archives that can be snarfed
> > automatically?)
> > 
> > --8<---------------cut here---------------start------------->8---
> > X-Content-type: Cadidate-guix-jargon-definition
> > Ad lib comment and metacomment ended by blank line ...
> > "> We don't call things build-inputs here in Guix land, that's a no-
> > no :P"
> > 
> > build-propagated-inputs:
> > 	<please fill in :) >
> > --8<---------------cut here---------------end--------------->8---
> When you quote someone like that out-of-context, you run a risk of
> misrepresenting what is actually stated.  What I mean, is that a
> package field called something along the lines of "build-inputs" is
> likely to confuse Guix veterans and newcomers alike, as evidenced by
> the following reply:
> 
> Am Sonntag, den 05.09.2021, 10:06 +0000 schrieb Attila Lendvai:
> > potentially worthless two cents from a newcomer's perspective:
> > 'build-time' and 'run-time' are well established concepts in the
> > wider community.
> And one, that is well misunderstood.  
> 
> > if i were reading 'linked-inputs' in a package definition, i wouldn't
> > associate it to being the set of build-time dependencies.
> That's not what linked-inputs are, though.  Take the following
> paragraph from propagated-inputs:
> 
> > For example this is necessary when packaging a C/C++ library
> > that needs headers of another library to compile, or when a
> > pkg-config file refers to another one via its ‘Requires’
> > field.
> This use case of propagated inputs explains why they need to be
> propagated when given as a (propagated-)input to a package, but not
> when given as a native input or merely existing in a profile.
> 
> The – required if we go by other systems – use case of installing
> libraries system- or user-wide is already discouraged by Guix, for it
> is not needed.  As long as we can spawn an environment, in which we can
> compile these things, it should be enough.
> 
> Note, that this is not equivalent to being a "build-time" dependency. 
> Going by Gentoo's definition "Build dependencies are used to specify
> any dependencies that are required to unpack, patch, compile, test or
> install the package", GCC is a build dependency of nearly any C program
> (and a native one at that, i.e. BDEPEND in Gentoo), but it's not a
> linked-dependency, because there are numerous ways in which other
> programs could use it without ever needing to invoke GCC.  Guix, of
> course, includes GCC as an implicit native input anyway, but that's a
> different topic.
> 
> Regards
> 

-- 
Regards,
Bengt Richter

Re: Guix Jargon File (WAS: Rethinking propagated inputs?)

[Liliana Marie Prikler]

Hi,

Am Sonntag, den 05.09.2021, 16:54 +0200 schrieb Bengt Richter:
> Hi Liliana,
> 
> Thank you for starting this renamed thread (as I should have done).
> 
> I think a people who are just looking at _maybe_ installing guix
> should have an easy way to look up terms they haven't seen before.
Personally speaking, the manual and cookbook lend themselves easily to
grepping (and it turns out you acknowledge that later on yourself), but
of course they only reference what Guix *has*, i.e. inputs and native-
inputs, not what it doesn't have, e.g. the Gentoo distinction between
DEPEND and BDEPEND.

> But really I am more interested in promoting the idea of a snippet-
> quoting convention modeled on a subset of mime email standards.
> 
> Very simple, but capable of containing and transferring anything
> unambiguously (if not always with efficient transmission encodings).
> 
> We can of course already do that with signed and attached files, and
> we can archive them and retrieve them, but I am interested in
> retrieving little pieces and making it easy to mark things in
> arbitratry contexts (like this email or a cannibal-friendly program
> source) so that simple snarfing utilities will be able to extract
> snippet-quote info based on tags and identifiers or anything
> in the headers or content per search options much like for any search
> engine.
Both the mailing lists and IRC channel are already archived, so what
you would need to do is simply to collect the funniest quotes and
publish them somewhere, perhaps with links back for context.  As for
tag-based lookup mechanisms, those already exist in the wild as well,
you don't particularly need to reinvent the wheel.

What I don't really understand from all this is for one, what purpose
this Jargon file would serve, and for another, who you would want to
collaborate it.  For all its explanatory power, I do think a manual
would be better suited most of the time if we're talking about serious
definitions of important concepts.

> This is to create a simple contribution mechanism as well as a format
> for retrieval.
> 
> I have seen many code snippets from developers that are tutorial
> material as well as practical how-tos for debugging and browsing
> guix.
> 
> Wouldn't it be nice if they were snip-quoted so that we could extract
> them from mail archives in a better way than searching the raw
> archives, or having to browse though treads and extract nuggets by
> hand?
Possibly, though your particular suggestion is probably more fun if
you're used to writing mail headers by hand than using a GUI client
which mostly hides such things for you (I personally haven't switched
to using Emacs for my mail yet, and I think integrating it with EDS or
even just GOA might take some while).

> simply:
> --8<---------------cut here---------------start------------->8---
> header part, ending with blank line
> 
> optional content part, encoded and delimited or referenced per header
> info
> --8<---------------cut here---------------end--------------->8---
> 
> 
> The header part could start with just prefixing GX- like the optional
> custom header X- prefix described in mime rfcs, and we could borrow
> whatever was useful.
X- exists for a reason, so you should make that X-SOMEID-KEY, where
SOMEID uniquely labels your scheme.  Or who knows, maybe some scheme
already exists out there and you don't need to create a new one.

> BTW, I know "info guix|grep -i whatever" often gives clues about
> whatever, for pursuing "C-s whatever" once inside "info guix
> whatever", but though concept and api indices are great, they are not
> a Jargon File, and not as handy for an outsider :)
I always thought jargon files to be insider humor?  Did I miss
something?  Like, obviously you can read them as an outsider, but you
won't have a clear understanding of what is actually said by having a
bunch more definitions in the back of your head.  To me that sounds
like learning a language with nothing but a dictionary, or worse a
thesaurus.

Re: Guix Jargon File (WAS: Rethinking propagated inputs?)

[Jonathan McHugh]

Hi Bengt,

I believe that a collection of regular expressions for recognising starting block and closing block for differing formats. It would for instance become political making a choice between (say):
* -a-dangerous-pair-of-scissors--8<--ouch- ;
* an Orgmode output; a GemText block;
* somebody who uses '£' as a delimiter,
* et al.

That way people can maintain their workflows while still having a better idea of where other peoples blocks/citations are.

FYI, I am looking into parsing manpages and 'cheat' style command tools to generate Parsing Expression Grammars - so as to permit people to identify coding, if not perform actions. Hopefully one can then identify inline content, as well as inferences regarding when coding blocks start and stop (let alone additional knowledge)

I was planning on using the Lisp TXR to do so - if somebody thinks this is a mistake please say so! If somebody would like to propose what a Guile PEG environment should look like I can make an additional grammar. If I get enough positive feedback I can prioritise it more for a project Im working on.

Kind regards,


Jonathan McHugh
indieterminacy@libre.brussels

September 5, 2021 4:54 PM, "Bengt Richter" <bokr@bokr.com> wrote:

> Hi Liliana,
> 
> Thank you for starting this renamed thread (as I should have done).
> 
> I think a people who are just looking at _maybe_ installing guix
> should have an easy way to look up terms they haven't seen before.
> 
> But really I am more interested in promoting the idea of a snippet-quoting
> convention modeled on a subset of mime email standards.
> 
> Very simple, but capable of containing and transferring anything unambiguously
> (if not always with efficient transmission encodings).
> 
> We can of course already do that with signed and attached files, and we can
> archive them and retrieve them, but I am interested in retrieving little pieces
> and making it easy to mark things in arbitratry contexts (like this email or
> a cannibal-friendly program source) so that simple snarfing utilities will be
> able to extract snippet-quote info based on tags and identifiers or anything
> in the headers or content per search options much like for any search engine.
> 
> This is to create a simple contribution mechanism as well as a format
> for retrieval.
> 
> I have seen many code snippets from developers that are tutorial material
> as well as practical how-tos for debugging and browsing guix.
> 
> Wouldn't it be nice if they were snip-quoted so that we could extract them
> from mail archives in a better way than searching the raw archives, or having
> to browse though treads and extract nuggets by hand?
> 
> simply:
> --8<---------------cut here---------------start------------->8---
> header part, ending with blank line
> 
> optional content part, encoded and delimited or referenced per header info
> --8<---------------cut here---------------end--------------->8---
> 
> The header part could start with just prefixing GX- like the optional custom
> header X- prefix described in mime rfcs, and we could borrow whatever was useful.
> 
> Tbc.. So called "real life" demands I postpone making a decent real example
> 'til later, sorry ;/
> 
> Please excuse the big top-post. I had intended to comment and edit in line ;/
> 
> BTW, I know "info guix|grep -i whatever" often gives clues about whatever, for pursuing
> "C-s whatever" once inside "info guix whatever", but though concept and api indices
> are great, they are not a Jargon File, and not as handy for an outsider :)
> 
> On +2021-09-05 12:50:56 +0200, Liliana Marie Prikler wrote:
> 
>> Hi,
>> 
>> Am Sonntag, den 05.09.2021, 11:50 +0200 schrieb Bengt Richter:
>>> We don't call things build-inputs here in Guix land, that's a no-no
>>> :P
>> 
>> Is there an official guix jargon file or glossary file or texi file
>> or wikimedia/wiktionary/wikipedia clone on gnu.org that non-
>> cognoscenti could use to get a clue?
>> AFAIK no, you more or less have to go by what the manual tells you. As
>> for why we have native-inputs and not build-inputs like other distros,
>> it's because people often misclassify "build" inputs, so the definition
>> actually does more harm than good. Guix knows which files are actually
>> "just used for build" by what ends up in the store, with some
>> exceptions like UTF-32 encoded strings.
>> 
>> Is there a thread that on that topic making any progress on making it
>> happen?
>> AFAIK no.
>> 
>> when someone in a thread like this offers a candidate official
>> definition, (off-topic sort of, but meta-on-topic for relevant
>> documentation) could it be snip-quoted for easy search and
>> aggregation for maintainers of official definitions and translations?
>> E.g.
>> (or actually borrow some rfc0842 or descendant so an attached file
>> generates a usuable section in mail archives that can be snarfed
>> automatically?)
>> 
>> --8<---------------cut here---------------start------------->8---
>> X-Content-type: Cadidate-guix-jargon-definition
>> Ad lib comment and metacomment ended by blank line ...
>> "> We don't call things build-inputs here in Guix land, that's a no-
>> no :P"
>> 
>> build-propagated-inputs:
>> <please fill in :) >
>> --8<---------------cut here---------------end--------------->8---
>> When you quote someone like that out-of-context, you run a risk of
>> misrepresenting what is actually stated. What I mean, is that a
>> package field called something along the lines of "build-inputs" is
>> likely to confuse Guix veterans and newcomers alike, as evidenced by
>> the following reply:
>> 
>> Am Sonntag, den 05.09.2021, 10:06 +0000 schrieb Attila Lendvai:
>> potentially worthless two cents from a newcomer's perspective:
>> 'build-time' and 'run-time' are well established concepts in the
>> wider community.
>> And one, that is well misunderstood.
>> 
>> if i were reading 'linked-inputs' in a package definition, i wouldn't
>> associate it to being the set of build-time dependencies.
>> That's not what linked-inputs are, though. Take the following
>> paragraph from propagated-inputs:
>> 
>> For example this is necessary when packaging a C/C++ library
>> that needs headers of another library to compile, or when a
>> pkg-config file refers to another one via its ‘Requires’
>> field.
>> This use case of propagated inputs explains why they need to be
>> propagated when given as a (propagated-)input to a package, but not
>> when given as a native input or merely existing in a profile.
>> 
>> The – required if we go by other systems – use case of installing
>> libraries system- or user-wide is already discouraged by Guix, for it
>> is not needed. As long as we can spawn an environment, in which we can
>> compile these things, it should be enough.
>> 
>> Note, that this is not equivalent to being a "build-time" dependency.
>> Going by Gentoo's definition "Build dependencies are used to specify
>> any dependencies that are required to unpack, patch, compile, test or
>> install the package", GCC is a build dependency of nearly any C program
>> (and a native one at that, i.e. BDEPEND in Gentoo), but it's not a
>> linked-dependency, because there are numerous ways in which other
>> programs could use it without ever needing to invoke GCC. Guix, of
>> course, includes GCC as an implicit native input anyway, but that's a
>> different topic.
>> 
>> Regards
> 
> --
> Regards,
> Bengt Richter

Re: Guix Jargon File (WAS: Rethinking propagated inputs?)

[Bengt Richter]

Hi Jonathan,

Thanks for the TXR reference, I will have to look into it.

Just on the basis of the author name "Kaz Kylheku" I would check it out.
I have encountered his posts 'way in the past, and they were always
intelligent and interesting. (If he is older than me, I'd like to know more
about his diet and habits ;-)

Anyway, if you are interested in PEGs in any guix connection, I think
you should look at Guile's PEG implementation and syntaxes (scheme and string/regex styles).

Type "info guile" (not guix), and then "Ctrl-s Peg Parsing"
which will take you to an index, and either use that link (hit Return) or
hit Ctrl-s again and you should be at the beginning of some interesting reading ;-)

Note that guix is built on top of guile and its infrastructure of libaries
(mostly C still where they aren't scheme, UIAM, but that may be changing
to include more, I'm not sure -- I am not insider enough to follow ;-)

Maybe you can make a guile-txr.scm package for a txr wrapper module
using foreign function or your own C extension loadable by guile scheme.

Have fun!
Maybe my best strategy is to see what you come up with before re-inventing
things you may already be way ahead in :)

Oh, if you haven't already, also check into guile's regular expressions, e.g.
"info guile Ret Ctrl-s regular expressions Ret Ret"

And also try guile's repl by just typing "guile Ret" and then ",help"

Mvh, Bengt Richter

On +2021-09-05 15:53:34 +0000, Jonathan McHugh wrote:
> Hi Bengt,
> 
> I believe that a collection of regular expressions for recognising starting block and closing block for differing formats. It would for instance become political making a choice between (say):
> * -a-dangerous-pair-of-scissors--8<--ouch- ;
> * an Orgmode output; a GemText block;
> * somebody who uses '£' as a delimiter,
> * et al.
> 
> That way people can maintain their workflows while still having a better idea of where other peoples blocks/citations are.
> 
> FYI, I am looking into parsing manpages and 'cheat' style command tools to generate Parsing Expression Grammars - so as to permit people to identify coding, if not perform actions. Hopefully one can then identify inline content, as well as inferences regarding when coding blocks start and stop (let alone additional knowledge)
> 
> I was planning on using the Lisp TXR to do so - if somebody thinks this is a mistake please say so! If somebody would like to propose what a Guile PEG environment should look like I can make an additional grammar. If I get enough positive feedback I can prioritise it more for a project Im working on.
> 
> Kind regards,
> 
> 
> Jonathan McHugh
> indieterminacy@libre.brussels
> 
> September 5, 2021 4:54 PM, "Bengt Richter" <bokr@bokr.com> wrote:
> 
> > Hi Liliana,
> > 
> > Thank you for starting this renamed thread (as I should have done).
> > 
> > I think a people who are just looking at _maybe_ installing guix
> > should have an easy way to look up terms they haven't seen before.
> > 
> > But really I am more interested in promoting the idea of a snippet-quoting
> > convention modeled on a subset of mime email standards.
> > 
> > Very simple, but capable of containing and transferring anything unambiguously
> > (if not always with efficient transmission encodings).
> > 
> > We can of course already do that with signed and attached files, and we can
> > archive them and retrieve them, but I am interested in retrieving little pieces
> > and making it easy to mark things in arbitratry contexts (like this email or
> > a cannibal-friendly program source) so that simple snarfing utilities will be
> > able to extract snippet-quote info based on tags and identifiers or anything
> > in the headers or content per search options much like for any search engine.
> > 
> > This is to create a simple contribution mechanism as well as a format
> > for retrieval.
> > 
> > I have seen many code snippets from developers that are tutorial material
> > as well as practical how-tos for debugging and browsing guix.
> > 
> > Wouldn't it be nice if they were snip-quoted so that we could extract them
> > from mail archives in a better way than searching the raw archives, or having
> > to browse though treads and extract nuggets by hand?
> > 
> > simply:
> > --8<---------------cut here---------------start------------->8---
> > header part, ending with blank line
> > 
> > optional content part, encoded and delimited or referenced per header info
> > --8<---------------cut here---------------end--------------->8---
> > 
> > The header part could start with just prefixing GX- like the optional custom
> > header X- prefix described in mime rfcs, and we could borrow whatever was useful.
> > 
> > Tbc.. So called "real life" demands I postpone making a decent real example
> > 'til later, sorry ;/
> > 
> > Please excuse the big top-post. I had intended to comment and edit in line ;/
> > 
> > BTW, I know "info guix|grep -i whatever" often gives clues about whatever, for pursuing
> > "C-s whatever" once inside "info guix whatever", but though concept and api indices
> > are great, they are not a Jargon File, and not as handy for an outsider :)
> > 
> > On +2021-09-05 12:50:56 +0200, Liliana Marie Prikler wrote:
> > 
> >> Hi,
> >> 
> >> Am Sonntag, den 05.09.2021, 11:50 +0200 schrieb Bengt Richter:
> >>> We don't call things build-inputs here in Guix land, that's a no-no
> >>> :P
> >> 
> >> Is there an official guix jargon file or glossary file or texi file
> >> or wikimedia/wiktionary/wikipedia clone on gnu.org that non-
> >> cognoscenti could use to get a clue?
> >> AFAIK no, you more or less have to go by what the manual tells you. As
> >> for why we have native-inputs and not build-inputs like other distros,
> >> it's because people often misclassify "build" inputs, so the definition
> >> actually does more harm than good. Guix knows which files are actually
> >> "just used for build" by what ends up in the store, with some
> >> exceptions like UTF-32 encoded strings.
> >> 
> >> Is there a thread that on that topic making any progress on making it
> >> happen?
> >> AFAIK no.
> >> 
> >> when someone in a thread like this offers a candidate official
> >> definition, (off-topic sort of, but meta-on-topic for relevant
> >> documentation) could it be snip-quoted for easy search and
> >> aggregation for maintainers of official definitions and translations?
> >> E.g.
> >> (or actually borrow some rfc0842 or descendant so an attached file
> >> generates a usuable section in mail archives that can be snarfed
> >> automatically?)
> >> 
> >> --8<---------------cut here---------------start------------->8---
> >> X-Content-type: Cadidate-guix-jargon-definition
> >> Ad lib comment and metacomment ended by blank line ...
> >> "> We don't call things build-inputs here in Guix land, that's a no-
> >> no :P"
> >> 
> >> build-propagated-inputs:
> >> <please fill in :) >
> >> --8<---------------cut here---------------end--------------->8---
> >> When you quote someone like that out-of-context, you run a risk of
> >> misrepresenting what is actually stated. What I mean, is that a
> >> package field called something along the lines of "build-inputs" is
> >> likely to confuse Guix veterans and newcomers alike, as evidenced by
> >> the following reply:
> >> 
> >> Am Sonntag, den 05.09.2021, 10:06 +0000 schrieb Attila Lendvai:
> >> potentially worthless two cents from a newcomer's perspective:
> >> 'build-time' and 'run-time' are well established concepts in the
> >> wider community.
> >> And one, that is well misunderstood.
> >> 
> >> if i were reading 'linked-inputs' in a package definition, i wouldn't
> >> associate it to being the set of build-time dependencies.
> >> That's not what linked-inputs are, though. Take the following
> >> paragraph from propagated-inputs:
> >> 
> >> For example this is necessary when packaging a C/C++ library
> >> that needs headers of another library to compile, or when a
> >> pkg-config file refers to another one via its ‘Requires’
> >> field.
> >> This use case of propagated inputs explains why they need to be
> >> propagated when given as a (propagated-)input to a package, but not
> >> when given as a native input or merely existing in a profile.
> >> 
> >> The – required if we go by other systems – use case of installing
> >> libraries system- or user-wide is already discouraged by Guix, for it
> >> is not needed. As long as we can spawn an environment, in which we can
> >> compile these things, it should be enough.
> >> 
> >> Note, that this is not equivalent to being a "build-time" dependency.
> >> Going by Gentoo's definition "Build dependencies are used to specify
> >> any dependencies that are required to unpack, patch, compile, test or
> >> install the package", GCC is a build dependency of nearly any C program
> >> (and a native one at that, i.e. BDEPEND in Gentoo), but it's not a
> >> linked-dependency, because there are numerous ways in which other
> >> programs could use it without ever needing to invoke GCC. Guix, of
> >> course, includes GCC as an implicit native input anyway, but that's a
> >> different topic.
> >> 
> >> Regards
> > 
> > --
> > Regards,
> > Bengt Richter

Re: Rethinking propagated inputs?

[Attila Lendvai]

> > What do you think of "build-propagated-inputs"?
>
> We don't call things build-inputs here in Guix land, that's a no-no :P

potentially worthless two cents from a newcomer's perspective:
'build-time' and 'run-time' are well established concepts in the wider
community.

if i were reading 'linked-inputs' in a package definition, i wouldn't
associate it to being the set of build-time dependencies.

the best name, from my admittedly uninformed perspective, would be
`build-time-inputs`.

- attila
PGP: 5D5F 45C7 DFCD 0A39

Re: Rethinking propagated inputs?

[Julien Lepiller]

[-- Attachment #1: Type: text/plain, Size: 1263 bytes --]

All inputs of a package are build-time, whether normal, native or propagated. The result may reference some of them, and these references are the run-time dependencies.

native-inputs are usually buill-time only, because they are usually programs that are run during the build (hence the need for them to be native in a cross-build).

propagated-inputs breaks this nice model by making them run-time dependencies despite the abscence of a reference.

I think focusing on the name is distracting, but maybe "dependent-inputs", since this type of input would only affect the inputs of its dependents?

Le 5 septembre 2021 06:06:02 GMT-04:00, Attila Lendvai <attila@lendvai.name> a écrit :
>> > What do you think of "build-propagated-inputs"?
>>
>> We don't call things build-inputs here in Guix land, that's a no-no :P
>
>potentially worthless two cents from a newcomer's perspective:
>'build-time' and 'run-time' are well established concepts in the wider
>community.
>
>if i were reading 'linked-inputs' in a package definition, i wouldn't
>associate it to being the set of build-time dependencies.
>
>the best name, from my admittedly uninformed perspective, would be
>`build-time-inputs`.
>
>- attila
>PGP: 5D5F 45C7 DFCD 0A39
>
>

[-- Attachment #2: Type: text/html, Size: 1812 bytes --]

Re: Rethinking propagated inputs?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 3588 bytes --]

Liliana Marie Prikler schreef op za 04-09-2021 om 20:24 [+0200]:
> Hi Guix,
> 
> some while ago we made the decision to propagate inputs, that are
> mentioned in pkg-config files, the rationale being that those
> propagated inputs will be needed in packages in order to compile.  This
> has saved us some typing, but at a cost.  For instance, it is now no
> longer possible to upgrade "zile"

Zile doesn't propagate glib: it's in inputs, not propagated-inputs:
https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zile.scm#n84.

> " and "icecat" independently, because
> both propagate glib.  "libreoffice" and "telegram-desktop", two
> packages that have failed us loudly before, are also in that list.

libreoffice doesn't propagate anything.  Neither does icecat.

>   To
> see what else is in here, you might want to use the following snippet
> in a guix repl:
> 
> --8<---------------cut here---------------start------------->8---
> (use-modules (guix packages)
>              (gnu packages)
>              (guix discovery)
>              (srfi srfi-1)
>              (srfi srfi-26))
> 
> (display
>  (fold-packages
>  (lambda (p s)
>    (if (any (compose (lambda (input)
>                        (and (package? input)
>                             (string=? "glib" (package-name input))))
>                      cadr)
>             (package-transitive-inputs p))
>        (cons (package-name p) s)
>        s))
>  '()
>  (all-modules %default-package-module-path)))
> --8<---------------cut here---------------end--------------->8---
> 
> It returns more than 1400 packages – a bit less than 10% of Guix. 
> Needless to say, that's a bad thing and I think we should do something
> about it, particularly when it comes to leaf packages, that users are
> likely to install.
> 
> Does anyone have an idea how we should handle propagations for the sake
> of pkg-config?  Perhaps we could add "linked-inputs", which are added
> when building packages and environments when not using --ad-hoc, but
> not when union-building profiles.  WDYT?

For packages using pkg-config, I had the following scheme in mind:

A package that has .pc files puts them into a separate "build" output.
The code handling 'propagated-inputs' is modified such that an entry in
'propagated-inputs' can have an additional 'propagated-for-output' component.

(Feel free to suggest a more concise name.)

I.e., something like:

  (define glib
    (package
      (name "glib")
      (outputs '("out" ; everything
                 "bin" ; glib-mkenums ... depends on Python
                 "build")) ; glib-2.0.pc
      (propagated-inputs
        `(("pcre" ,pcre #:propagate-for-output "build") ; in the Requires.private field of glib-2.0.pc
          ("libffi" ,libffi #:propagate-for-output "build") ; ditto, for gobject-2.0.pc
          ;; etc.
          ))
      (native-inputs '()) ; not relevant to this e-ail
      ...))

Now, imagine the "build" output of "zile" had glib:build in propagated-inputs,
using the scheme described above.  Then, if the "out" output of zile is installed
in a profile, that doesn't cause glib to appear in the profile as well, because glib
is only propagated for the "build" output of zile, and not for "out" output of zile.

However, if "build" is installed in the profile (e.g. because someone runs
"guix environment --ad-hoc zile:build various compilation tools" to create
an application using the zile library), then the .pc becomes available in the profile. 

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi

Am Sonntag, den 05.09.2021, 18:17 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op za 04-09-2021 om 20:24 [+0200]:
> > Hi Guix,
> > 
> > some while ago we made the decision to propagate inputs, that are
> > mentioned in pkg-config files, the rationale being that those
> > propagated inputs will be needed in packages in order to
> > compile.  This
> > has saved us some typing, but at a cost.  For instance, it is now
> > no
> > longer possible to upgrade "zile"
> 
> Zile doesn't propagate glib: it's in inputs, not propagated-inputs:
> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zile.scm#n84.
Oops, what a blunder.  It turns out my mistake was
> > " and "icecat" independently, because
> > both propagate glib.  "libreoffice" and "telegram-desktop", two
> > packages that have failed us loudly before, are also in that list.
> 
> libreoffice doesn't propagate anything.  Neither does icecat.
> 
> >   To
> > see what else is in here, you might want to use the following
> > snippet
> > in a guix repl:
> > 
> > --8<---------------cut here---------------start------------->8---
> > (use-modules (guix packages)
> >              (gnu packages)
> >              (guix discovery)
> >              (srfi srfi-1)
> >              (srfi srfi-26))
> > 
> > (display
> >  (fold-packages
> >  (lambda (p s)
> >    (if (any (compose (lambda (input)
> >                        (and (package? input)
> >                             (string=? "glib" (package-name
> > input))))
> >                      cadr)
> >             (package-transitive-inputs p))
> >        (cons (package-name p) s)
> >        s))
> >  '()
> >  (all-modules %default-package-module-path)))
> > --8<---------------cut here---------------end--------------->8---
(package-transitive-inputs p) should be (package-transitive-propagated-
inputs p).  With that, telegram-desktop is still in the list, as is
gst-plugins-base (why?), rhythmbox, polari, and some others.

> > It returns more than 1400 packages – a bit less than 10% of Guix. 
> > Needless to say, that's a bad thing and I think we should do
> > something about it, particularly when it comes to leaf packages,
> > that users are likely to install.
> > 
> > Does anyone have an idea how we should handle propagations for the
> > sake of pkg-config?  Perhaps we could add "linked-inputs", which
> > are added when building packages and environments when not using --
> > ad-hoc, but not when union-building profiles.  WDYT?
> 
> For packages using pkg-config, I had the following scheme in mind:
> 
> A package that has .pc files puts them into a separate "build"
> output.  The code handling 'propagated-inputs' is modified such that
> an entry in 'propagated-inputs' can have an additional 'propagated-
> for-output' component.
> 
> (Feel free to suggest a more concise name.)
Since "propagated" is already given by propagated-inputs, what about
simply having #:by?

E.g. something like:
  (define glib
    (package
      (name "glib")
      (outputs '("out" ; everything
                 "bin")) ; glib-mkenums ... depends on Python
      (propagated-inputs
        `(("pcre" ,pcre #:by ("out"))     ; future-proof #:by  
          ("libffi" ,libffi #:by ("out")) ; have it be a list
          [...]))

I'm not sure whether we need to necessarily add a new "build" output
for pkg-config, that'd lead to a lot of doubly-propagated includes. 
(Imagine both libffi and libffi:build etc.)  I think the more
meaningful division here would be to add "bin" or "lib" outputs to
those packages, where we don't want propagation to continue
indefinitely.  This does cause problems with language bindings though,
e.g. pygobject, as those also propagate the package in question and
can't be neatly separated.

> Now, imagine the "build" output of "zile" had glib:build in
> propagated-inputs, using the scheme described above.  Then, if the
> "out" output of zile is installed in a profile, that doesn't cause
> glib to appear in the profile as well, because glib
> is only propagated for the "build" output of zile, and not for "out"
> output of zile.
> 
> However, if "build" is installed in the profile (e.g. because someone
> runs "guix environment --ad-hoc zile:build various compilation tools"
> to create an application using the zile library), then the .pc
> becomes available in the profile. 
I must admit that this solution appears to have some surface elegance,
but what exactly would go in the "build" output of a package?  You
mentioned pkg-config files (obviously), but those don't suffice to
actually build a package, do they?  Would we need an extra syntax to
e.g. propagate the "out" output by "build" (and in some cases the "lib"
output instead)?

Greetings

Re: Rethinking propagated inputs?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 3065 bytes --]

Liliana Marie Prikler schreef op zo 05-09-2021 om 18:50 [+0200]:
> 
[...]
> > (Feel free to suggest a more concise name.)
> Since "propagated" is already given by propagated-inputs, what about
> simply having #:by?
> E.g. something like: [...]
>   (define glib
>     (package
>       (name "glib")
>       (outputs '("out" ; everything
>                  "bin")) ; glib-mkenums ... depends on Python
>       (propagated-inputs
>         `(("pcre" ,pcre #:by ("out"))     ; future-proof #:by  
>           ("libffi" ,libffi #:by ("out")) ; have it be a list
>           [...]))

That looks nice, thanks.

> I'm not sure whether we need to necessarily add a new "build" output
> for pkg-config, that'd lead to a lot of doubly-propagated includes. 
> (Imagine both libffi and libffi:build etc.)  I think the more
> meaningful division here would be to add "bin" or "lib" outputs to
> those packages, where we don't want propagation to continue
> indefinitely.

Putting pkg-config files into the "lib" or "out" output (together with shared libraries)
seems reasonable to me ("lib" if the package is usually used as a binary,
but also has libaries with pkg-config files, and "out" if it is usually
used as a library).

>   This does cause problems with language bindings though,
> e.g. pygobject, as those also propagate the package in question and
> can't be neatly separated.

python-pygobject can just keep everything in the output "out",
and let glib and libffi be propagated by "out", no?  I don't see
how this would cause trouble for language bindings.

> > Now, imagine the "build" output of "zile" had glib:build in
> > propagated-inputs, using the scheme described above.  Then, if the
> > "out" output of zile is installed in a profile, that doesn't cause
> > glib to appear in the profile as well, because glib
> > is only propagated for the "build" output of zile, and not for "out"
> > output of zile.
> > 
> > However, if "build" is installed in the profile (e.g. because someone
> > runs "guix environment --ad-hoc zile:build various compilation tools"
> > to create an application using the zile library), then the .pc
> > becomes available in the profile. 
> I must admit that this solution appears to have some surface elegance,
> but what exactly would go in the "build" output of a package?  You
> mentioned pkg-config files (obviously), but those don't suffice to
> actually build a package, do they?

Sometimes they do suffice.  The .pc files contain the "-L/.../LIB",
"-I/.../include" and "-lstuff" flags needed for compilation.  If the
build system of the package uses pkg-config, it will use those flags,
so the compiler will find the library in that case.

Not sure if they always do suffice.

>   Would we need an extra syntax to
> e.g. propagate the "out" output by "build" (and in some cases the "lib"
> output instead)?

Not if .pc files are put in "out" (or "lib" in some cases) instead of the
originally proposed "build", but otherwise, possibly?

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Am Sonntag, den 05.09.2021, 21:18 +0200 schrieb Maxime Devos:
> [...]
> [Liliana Marie Prikler schreef op zo 05-09-2021 om 18:50 [+0200]:]
> > This does cause problems with language bindings though,
> > e.g. pygobject, as those also propagate the package in question and
> > can't be neatly separated.
> 
> python-pygobject can just keep everything in the output "out",
> and let glib and libffi be propagated by "out", no?  I don't see
> how this would cause trouble for language bindings.
It doesn't immediately cause problems with the language bindings
themselves, you are correct about that, but since packages using such
bindings must by virtue of being python packages already propagate all
their inputs we are back at square zero, so to speak.  However, perhaps
we can solve that by putting launchers in the "bin" output?

> > > Now, imagine the "build" output of "zile" had glib:build in
> > > propagated-inputs, using the scheme described above.  Then, if
> > > the "out" output of zile is installed in a profile, that doesn't
> > > cause glib to appear in the profile as well, because glib is only
> > > propagated for the "build" output of zile, and not for "out"
> > > output of zile.
> > > 
> > > However, if "build" is installed in the profile (e.g. because
> > > someone runs "guix environment --ad-hoc zile:build various
> > > compilation tools" to create an application using the zile
> > > library), then the .pc becomes available in the profile. 
> > I must admit that this solution appears to have some surface
> > elegance, but what exactly would go in the "build" output of a
> > package?  You mentioned pkg-config files (obviously), but those
> > don't suffice to actually build a package, do they?
> 
> Sometimes they do suffice.  The .pc files contain the "-L/.../LIB",
> "-I/.../include" and "-lstuff" flags needed for compilation.  If the
> build system of the package uses pkg-config, it will use those flags,
> so the compiler will find the library in that case.
> 
> Not sure if they always do suffice.
Is that so?  I would think the build process needs to see stuff outside
of its inputs for that to work, e.g. the actual header it wants to
include, which isn't part of "build".  Am I misunderstanding our
sandbox requirements?

> > Would we need an extra syntax to e.g. propagate the "out" output by
> > "build" (and in some cases the "lib" output instead)?
> 
> Not if .pc files are put in "out" (or "lib" in some cases) instead of
> the originally proposed "build", but otherwise, possibly?
Okay, let's talk about the other things then until we can put a certain
(as in "sure to be correct") answer to this question.

Greetings

Re: Rethinking propagated inputs?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 1727 bytes --]

Liliana Marie Prikler schreef op zo 05-09-2021 om 21:37 [+0200]:
> > > I must admit that this solution appears to have some surface
> > > elegance, but what exactly would go in the "build" output of a
> > > package?  You mentioned pkg-config files (obviously), but those
> > > don't suffice to actually build a package, do they?
> > 
> > Sometimes they do suffice.  The .pc files contain the "-L/.../LIB",
> > "-I/.../include" and "-lstuff" flags needed for compilation.  If the
> > build system of the package uses pkg-config, it will use those flags,
> > so the compiler will find the library in that case.
> > 
> > Not sure if they always do suffice.
>
> Is that so?  I would think the build process needs to see stuff outside
> of its inputs for that to work, e.g. the actual header it wants to
> include, which isn't part of "build".  Am I misunderstanding our
> sandbox requirements?

The .pc file includes the absolute path to the library and include directories,
so the output "build" with the .pc file has a reference to the output "out"
with the libraries and include headers.  More concretely, take the .pc from
the glib package:

prefix=/gnu/store/98hgv3i6hdqgiq98ldy7rkpdwhah8iq2-glib-2.62.6
libdir=${prefix}/lib
includedir=${prefix}/include
[more stuff]
Requires.private: libpcre >=  8.31
Libs: -L${libdir} -lglib-2.0
Libs.private: -pthread
Cflags: -I${includedir}/glib-2.0 -I${libdir}/glib-2.0/include

The (transitive) references of all inputs to the build process are included
in the sandbox.  In this case, if the package has the hypothetical glib:build
among its inputs, the daemon will automatically make glib:out available as well
in the sandbox.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Am Sonntag, den 05.09.2021, 22:27 +0200 schrieb Maxime Devos:
> Liliana Marie Prikler schreef op zo 05-09-2021 om 21:37 [+0200]:
> > > > I must admit that this solution appears to have some surface
> > > > elegance, but what exactly would go in the "build" output of a
> > > > package?  You mentioned pkg-config files (obviously), but those
> > > > don't suffice to actually build a package, do they?
> > > 
> > > Sometimes they do suffice.  The .pc files contain the "-
> > > L/.../LIB", "-I/.../include" and "-lstuff" flags needed for
> > > compilation.  If the build system of the package uses pkg-config, 
> > > it will use those flags, so the compiler will find the library in
> > > that case.
> > > 
> > > Not sure if they always do suffice.
> > 
> > Is that so?  I would think the build process needs to see stuff
> > outside of its inputs for that to work, e.g. the actual header it
> > wants to include, which isn't part of "build".  Am I
> > misunderstanding our sandbox requirements?
> 
> The .pc file includes the absolute path to the library and include
> directories, so the output "build" with the .pc file has a reference
> to the output "out" with the libraries and include headers.  More
> concretely, take the .pc from the glib package:
> 
> prefix=/gnu/store/98hgv3i6hdqgiq98ldy7rkpdwhah8iq2-glib-2.62.6
> libdir=${prefix}/lib
> includedir=${prefix}/include
> [more stuff]
> Requires.private: libpcre >=  8.31
> Libs: -L${libdir} -lglib-2.0
> Libs.private: -pthread
> Cflags: -I${includedir}/glib-2.0 -I${libdir}/glib-2.0/include
> 
> The (transitive) references of all inputs to the build process are
> included in the sandbox.  In this case, if the package has the
> hypothetical glib:build among its inputs, the daemon will
> automatically make glib:out available as well in the sandbox.
And IIUC if glib had a "lib" output, glib:lib would be available in the
sandbox instead of glib:out due to the reference by glib:build?  If
that's the case using #:by and "build" outputs might be a preferable
solution, if not for all packages then at least for some.

At this point the question then becomes what to name this "build"
output and what to put into it besides pkg-config stuff.  Particularly
in the land of glib, there also exist typelibs, which can be used as
"build" inputs in the case of Vala or as runtime inputs in the case of
pygobject and other language bindings.  Perhaps this is early
bikeshedding when we'd actually need to code up #:by, but what would be
the better approach here?  Separate "build" into "pkg-config", "cmake"
(for CMake-based package discovery), "typelib" etc. or have one more or
less anonymous blob called "build"?

Greetings

Re: Rethinking propagated inputs?

[Maxime Devos]

[-- Attachment #1: Type: text/plain, Size: 3220 bytes --]

Liliana Marie Prikler schreef op zo 05-09-2021 om 23:10 [+0200]:
> Am Sonntag, den 05.09.2021, 22:27 +0200 schrieb Maxime Devos:
> > Liliana Marie Prikler schreef op zo 05-09-2021 om 21:37 [+0200]:
> > > > > I must admit that this solution appears to have some surface
> > > > > elegance, but what exactly would go in the "build" output of a
> > > > > package?  You mentioned pkg-config files (obviously), but those
> > > > > don't suffice to actually build a package, do they?
> > > > 
> > > > Sometimes they do suffice.  The .pc files contain the "-
> > > > L/.../LIB", "-I/.../include" and "-lstuff" flags needed for
> > > > compilation.  If the build system of the package uses pkg-config, 
> > > > it will use those flags, so the compiler will find the library in
> > > > that case.
> > > > 
> > > > Not sure if they always do suffice.
> > > 
> > > Is that so?  I would think the build process needs to see stuff
> > > outside of its inputs for that to work, e.g. the actual header it
> > > wants to include, which isn't part of "build".  Am I
> > > misunderstanding our sandbox requirements?
> > 
> > The .pc file includes the absolute path to the library and include
> > directories, so the output "build" with the .pc file has a reference
> > to the output "out" with the libraries and include headers.  More
> > concretely, take the .pc from the glib package:
> > 
> > prefix=/gnu/store/98hgv3i6hdqgiq98ldy7rkpdwhah8iq2-glib-2.62.6
> > libdir=${prefix}/lib
> > includedir=${prefix}/include
> > [more stuff]
> > Requires.private: libpcre >=  8.31
> > Libs: -L${libdir} -lglib-2.0
> > Libs.private: -pthread
> > Cflags: -I${includedir}/glib-2.0 -I${libdir}/glib-2.0/include
> > 
> > The (transitive) references of all inputs to the build process are
> > included in the sandbox.  In this case, if the package has the
> > hypothetical glib:build among its inputs, the daemon will
> > automatically make glib:out available as well in the sandbox.
> And IIUC if glib had a "lib" output, glib:lib would be available in the
> sandbox instead of glib:out due to the reference by glib:build?

If a package has the proposed 'glib:build' in its (propagated-)inputs,
and if 'glib' had a 'lib' output, then in the build sandbox of the package,
'glib:lib' will be available.

>   If
> that's the case using #:by and "build" outputs might be a preferable
> solution, if not for all packages then at least for some.

> At this point the question then becomes what to name this "build"
> output and what to put into it besides pkg-config stuff.  Particularly
> in the land of glib, there also exist typelibs, which can be used as
> "build" inputs in the case of Vala or as runtime inputs in the case of
> pygobject and other language bindings.  Perhaps this is early
> bikeshedding when we'd actually need to code up #:by, but what would be
> the better approach here?  Separate "build" into "pkg-config", "cmake"
> (for CMake-based package discovery), "typelib" etc. or have one more or
> less anonymous blob called "build"?

I don't know (-:.  Maybe let's just start with pkg-config and #:by?
for now?  The name can always be changed later.

Greetings,
Maxime.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

Re: Rethinking propagated inputs?

[宋文武]

Hello, good plan!


Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> At this point the question then becomes what to name this "build"
> output and what to put into it besides pkg-config stuff.
In Debian, it's a "dev" package, includes .h (C headers), .pc, .a
(static libraries).


> Particularly in the land of glib, there also exist typelibs, which can
> be used as "build" inputs in the case of Vala or as runtime inputs in
> the case of pygobject and other language bindings.
It's "gir1.2-glib-2.0" in debian, we can add a "gir" output.


> Perhaps this is
> early bikeshedding when we'd actually need to code up #:by, but what
> would be the better approach here?  Separate "build" into
> "pkg-config", "cmake" (for CMake-based package discovery), "typelib"
> etc. or have one more or less anonymous blob called "build"?
I think we should have "build" (or "dev") and "typelib" (or "gir"), but
not "pkg-config" and "cmake" in most cases.

With a "dev" output, we don't need 'propagated-inputs' in other outputs
most cases, so reduce the mess for profiles.

With a "typelib" output, we can reduce the runtime closure size for
packages which don't depends on them.

The build time clojure size reduced by "pkg-config" and "cmake" outputs is
small.

Re: Rethinking propagated inputs?

[Maxim Cournoyer]

Hello,

Liliana Marie Prikler <liliana.prikler@gmail.com> writes:

> Hi
>
> Am Sonntag, den 05.09.2021, 18:17 +0200 schrieb Maxime Devos:
>> Liliana Marie Prikler schreef op za 04-09-2021 om 20:24 [+0200]:
>> > Hi Guix,
>> > 
>> > some while ago we made the decision to propagate inputs, that are
>> > mentioned in pkg-config files, the rationale being that those
>> > propagated inputs will be needed in packages in order to
>> > compile.  This
>> > has saved us some typing, but at a cost.  For instance, it is now
>> > no
>> > longer possible to upgrade "zile"
>> 
>> Zile doesn't propagate glib: it's in inputs, not propagated-inputs:
>> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zile.scm#n84.
> Oops, what a blunder.  It turns out my mistake was
>> > " and "icecat" independently, because
>> > both propagate glib.  "libreoffice" and "telegram-desktop", two
>> > packages that have failed us loudly before, are also in that list.
>> 
>> libreoffice doesn't propagate anything.  Neither does icecat.

It seems the original issue pointed at was somewhat misguided; is there
still something to be fixed about propagated inputs?  It seems the
discussion has moved toward handling propagated inputs for the use of
pkg-config.

What are the current problems with it, and what would the advantages be
to move away from the status quo?  If there aren't clear benefits, I'd
prefer the status quo, abstaining from the added complexity.

Thanks,

Maxim

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi,

Am Montag, den 06.09.2021, 14:07 -0400 schrieb Maxim Cournoyer:
> Hello,
> 
> Liliana Marie Prikler <liliana.prikler@gmail.com> writes:
> 
> > Hi
> > 
> > Am Sonntag, den 05.09.2021, 18:17 +0200 schrieb Maxime Devos:
> > > Liliana Marie Prikler schreef op za 04-09-2021 om 20:24 [+0200]:
> > > > Hi Guix,
> > > > 
> > > > some while ago we made the decision to propagate inputs, that
> > > > are
> > > > mentioned in pkg-config files, the rationale being that those
> > > > propagated inputs will be needed in packages in order to
> > > > compile.  This
> > > > has saved us some typing, but at a cost.  For instance, it is
> > > > now
> > > > no
> > > > longer possible to upgrade "zile"
> > > 
> > > Zile doesn't propagate glib: it's in inputs, not propagated-
> > > inputs:
> > > https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/zile.scm#n84.
> > Oops, what a blunder.  It turns out my mistake was
> > > > " and "icecat" independently, because
> > > > both propagate glib.  "libreoffice" and "telegram-desktop", two
> > > > packages that have failed us loudly before, are also in that
> > > > list.
> > > 
> > > libreoffice doesn't propagate anything.  Neither does icecat.
> 
> It seems the original issue pointed at was somewhat misguided; is
> there still something to be fixed about propagated inputs?  It seems
> the discussion has moved toward handling propagated inputs for the
> use of pkg-config.
While those two examples were picked from a set, that might arguably be
too large, the point still stands that far too many packages are
propagating glib through some way or another.  Rhythmbox for instance
propagates it through dconf, Polari through telepathy-glib, and so on.

The question I've been poking at has always been that of propagating
packages mentioned by pkg-config, because that's the main reason glib
gets propagated by a bunch of related libraries.  There are some
fundamental flaws with certain software like GTKSourceView, that can't
function unless propagated by the package using it, but those are more
limited in the number of packages they affect.  There's still 355
packages somehow propagating glib (about a third of all packages
dependant on glib).

Sometimes those packages can be fixed without propagation.  For
instance, a meld bug that still exists on master would probably
disappear by applying <http://issues.guix.gnu.org/48445>, which
mentions that installing or propagating some package would also fix
this for some.

Perhaps instead of propagating packages mentioned by pkg-config, we
could symlink the required .pc files in an additional phase.  Then, as
per what Maxime Devos mentioned for "build" outputs, we would always
have the packages present in the build environment even if they're not
propagated.  WDYT?

> What are the current problems with it, and what would the advantages
> be to move away from the status quo?  If there aren't clear benefits,
> I'd prefer the status quo, abstaining from the added complexity.
The problem with propagated packages is that they devolve Guix into a
traditional distro in which you can not upgrade parts of your system
without also upgrading other parts.  This is all fine and dandy when it
affects related systems, e.g. only Emacs packages, but if two random
packages can crash because they depend on some low-level library,
that's very, very bad.  I think we should try to minimize the
occurrences of that.

Thanks

Re: Rethinking propagated inputs?

[Sarah Morgensen]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> It seems the original issue pointed at was somewhat misguided; is there
> still something to be fixed about propagated inputs?  It seems the
> discussion has moved toward handling propagated inputs for the use of
> pkg-config.
>
> What are the current problems with it, and what would the advantages be
> to move away from the status quo?  If there aren't clear benefits, I'd
> prefer the status quo, abstaining from the added complexity.

As I mentioned up-thread:

Sarah Morgensen <iskarian@mgsn.dev> writes:

>> Does anyone have an idea how we should handle propagations for the sake
>> of pkg-config?  Perhaps we could add "linked-inputs", which are added
>> when building packages and environments when not using --ad-hoc, but
>> not when union-building profiles.  WDYT?
>
> I know nothing about pkg-config, but such an input would help
> simplify things for Go (and I think for Rust) since many inputs need to
> be propagated only at build-time.

In Go(/Rust), the dependency packages' output contains only source, and
for a package to build it must have the source of all its transitive
inputs available.

However, IMO it doesn't make sense to automatically install these in a
profile when installing that package.  If you're installing a
source-only package, you likely don't want all the transitive
dependencies automatically installed--what if you're just inspecting the
source, or want to try building it with different dependencies?  If you
want all the build dependencies as well, you would just do

  guix environment go-foo --ad-hoc go-foo

like you would do for other Guix packages.

In the context of this discussion, I can think of three types of
solution:

1. Add a modifier like #:profile? (defaulting to #f) to
propagated-inputs entries
  (propagated-inputs
    `(("foo" #:profile? #t)))

2. Rename "propagated-inputs" to e.g. "profile-propagated-inputs", and
then introduce a new "propagated-inputs", which only propagates into the
inputs of dependents, and not into built profiles.

3. Introduce a way to prevent an input from propagating anything into
the current package/profile
  (inputs
    `(("foo" #:propagate? #f)))
and then provide CLI options (transformations) to control this.

I think the first two are clear and make the packager consider exactly
what they're going to propagate into user profiles.  Of course, both
also rely on the packager always knowing what should be propagated to a
user's profile or not.  The third option allows more control over that
(and could be combined with one of the first two), but feels a bit rough
to use...

WDYT?

--
Sarah

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi,

Am Dienstag, den 07.09.2021, 12:01 -0700 schrieb Sarah Morgensen:
> [...]
> 
> However, IMO it doesn't make sense to automatically install these in
> a profile when installing that package.  If you're installing a
> source-only package, you likely don't want all the transitive
> dependencies automatically installed--what if you're just inspecting
> the source, or want to try building it with different
> dependencies?  If you want all the build dependencies as well, you
> would just do
> 
>   guix environment go-foo --ad-hoc go-foo
> 
> like you would do for other Guix packages.
I think if you wanted to build go-foo locally, you should go with just
`guix environment go-foo' anyway.  For the source code, you might be
interested in `guix build --source'.

> In the context of this discussion, I can think of three types of
> solution:
> 
> 1. Add a modifier like #:profile? (defaulting to #f) to
> propagated-inputs entries
>   (propagated-inputs
>     `(("foo" #:profile? #t)))
This is basically a weaker linked-inputs, because linked-inputs could
be sanitized while these propagated inputs could not.

> 2. Rename "propagated-inputs" to e.g. "profile-propagated-inputs",
> and then introduce a new "propagated-inputs", which only propagates
> into the inputs of dependents, and not into built profiles.
Again same as introducing a new field, but also changing semantics for
everything in the tree.  I think having "propagated-inputs" mean what
it means is fine.  If we do find that we're lacking a field with
certain semantics, we should add that or change existing fields in
backwards-compatible ways.

> 3. Introduce a way to prevent an input from propagating anything into
> the current package/profile
>   (inputs
>     `(("foo" #:propagate? #f)))
> and then provide CLI options (transformations) to control this.
Your example should have propagated-inputs.  I'm not sure what benefit
there is to this solution that isn't given by adding a field, except
that it can't be sanitized.  Now that I think about it, this seems to
be a different syntax to 1.

> I think the first two are clear and make the packager consider
> exactly what they're going to propagate into user profiles.  Of
> course, both also rely on the packager always knowing what should be
> propagated to a user's profile or not.  The third option allows more
> control over that (and could be combined with one of the first two),
> but feels a bit rough to use...
W.r.t. relying on the packager to know that, we could document such use
cases in the manual (similar to how they are documented currently) and
we still have a review process to fall back on.

Regards

Re: Rethinking propagated inputs?

[iskarian]

Hi,

September 8, 2021 12:18 AM, "Liliana Marie Prikler" <liliana.prikler@gmail.com> wrote:

> Hi,
> 
> Am Dienstag, den 07.09.2021, 12:01 -0700 schrieb Sarah Morgensen:
> 
>> [...]
>> 
>> However, IMO it doesn't make sense to automatically install these in
>> a profile when installing that package. If you're installing a
>> source-only package, you likely don't want all the transitive
>> dependencies automatically installed--what if you're just inspecting
>> the source, or want to try building it with different
>> dependencies? If you want all the build dependencies as well, you
>> would just do
>> 
>> guix environment go-foo --ad-hoc go-foo
>> 
>> like you would do for other Guix packages.
> 
> I think if you wanted to build go-foo locally, you should go with just
> `guix environment go-foo' anyway. For the source code, you might be
> interested in `guix build --source'.

guix build --source doesn't capture any modifications made in the dependency package, such as
rewriting paths or other patching.

You're right, though, in that (for now) it's a rare use-case, and probably doesn't warrant the
added complexity and the extra field. It just feels... messy as it is now.

>> In the context of this discussion, I can think of three types of
>> solution:
>> 
>> 1. Add a modifier like #:profile? (defaulting to #f) to
>> propagated-inputs entries
>> (propagated-inputs
>> `(("foo" #:profile? #t)))
> 
> This is basically a weaker linked-inputs, because linked-inputs could
> be sanitized while these propagated inputs could not.

Good point. I included it because there seemed to be pushback on adding a field.

>> 2. Rename "propagated-inputs" to e.g. "profile-propagated-inputs",
>> and then introduce a new "propagated-inputs", which only propagates
>> into the inputs of dependents, and not into built profiles.
> 
> Again same as introducing a new field, but also changing semantics for
> everything in the tree. I think having "propagated-inputs" mean what
> it means is fine. If we do find that we're lacking a field with
> certain semantics, we should add that or change existing fields in
> backwards-compatible ways.

I agree; I should have stated this differently ("Add a new field"). The name is just bikeshedding.

>> 3. Introduce a way to prevent an input from propagating anything into
>> the current package/profile
>> (inputs
>> `(("foo" #:propagate? #f)))
>> and then provide CLI options (transformations) to control this.
> 
> Your example should have propagated-inputs. I'm not sure what benefit
> there is to this solution that isn't given by adding a field, except
> that it can't be sanitized. Now that I think about it, this seems to
> be a different syntax to 1.

Ah, I forgot to state that in this example, "foo" is the package with a propagated-inputs field,
and this is a dependent package (or a profile) which does not want "foo"'s propagated-inputs to be
propagated to it.

This way a packager has an "escape hatch" if something unwanted got propagated. Thinking about it
some more, though, in such a case it's more likely to want to disallow a specific package than a
whole set of propagated inputs. If such a feature would actually be useful (questionable), it's
probably better implemented differently than this (disallowed-inputs, perhaps, to mirror
disallowed-references)?

I hope I made some more sense this time!

--
Sarah

Re: Rethinking propagated inputs?

[Ludovic Courtès]

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> For packages using pkg-config, I had the following scheme in mind:

As a reminder to everyone, pkg-config is just one of the reasons for
propagating inputs, probably not the main one numerically speaking.

> A package that has .pc files puts them into a separate "build" output.
> The code handling 'propagated-inputs' is modified such that an entry in
> 'propagated-inputs' can have an additional 'propagated-for-output' component.
>
> (Feel free to suggest a more concise name.)
>
> I.e., something like:
>
>   (define glib
>     (package
>       (name "glib")
>       (outputs '("out" ; everything
>                  "bin" ; glib-mkenums ... depends on Python
>                  "build")) ; glib-2.0.pc
>       (propagated-inputs
>         `(("pcre" ,pcre #:propagate-for-output "build") ; in the Requires.private field of glib-2.0.pc
>           ("libffi" ,libffi #:propagate-for-output "build") ; ditto, for gobject-2.0.pc
>           ;; etc.
>           ))
>       (native-inputs '()) ; not relevant to this e-ail
>       ...))

Good’ol ‘ROADMAP’ file has a half-baked proposal along these lines.
It’s interesting, but I’m a bit concerned about the maintenance
cost/benefit ratio of this approach.  (It also goes against work on
simplifying the surface syntax of inputs.)

It would be great if the ‘Requires’ field of .pc files could specify
absolute file name; it would no longer be necessary to set
PKG_CONFIG_PATH, and thus propagation wouldn’t be needed in this case.

Regarding outputs, Nixpkgs introduced a “dev” output a while back that
lumps together our “lib” and “include” outputs, roughly.  I think that’s
a good idea (for other reasons too).

Ludo’.

Re: Rethinking propagated inputs?

[zimoun]

On Thu, 09 Sep 2021 at 00:12, Ludovic Courtès <ludo@gnu.org> wrote:
> Maxime Devos <maximedevos@telenet.be> skribis:

>>   (define glib
>>     (package
>>       (name "glib")
>>       (outputs '("out" ; everything
>>                  "bin" ; glib-mkenums ... depends on Python
>>                  "build")) ; glib-2.0.pc
>>       (propagated-inputs
>>         `(("pcre" ,pcre #:propagate-for-output "build") ; in the Requires.private field of glib-2.0.pc
>>           ("libffi" ,libffi #:propagate-for-output "build") ; ditto, for gobject-2.0.pc
>>           ;; etc.
>>           ))
>>       (native-inputs '()) ; not relevant to this e-ail
>>       ...))
>
> Good’ol ‘ROADMAP’ file has a half-baked proposal along these lines.

I think you mean the ’TODO’ file:

<https://git.savannah.gnu.org/cgit/guix.git/tree/TODO#n37>

Cheers,
simon

Re: Rethinking propagated inputs?

[Liliana Marie Prikler]

Hi,

Am Donnerstag, den 09.09.2021, 00:12 +0200 schrieb Ludovic Courtès:
> Hi,
> 
> Maxime Devos <maximedevos@telenet.be> skribis:
> 
> > For packages using pkg-config, I had the following scheme in mind:
> 
> As a reminder to everyone, pkg-config is just one of the reasons for
> propagating inputs, probably not the main one numerically speaking.
That is true, pkg-config propagations are merely closest to the "core"
of Guix and thus (have the potential affect to) larger parts, whereas other forms are typically – though perhaps not necessarily – limited to some language ecosystem or similar.

> [...]
> It would be great if the ‘Requires’ field of .pc files could specify
> absolute file name; it would no longer be necessary to set
> PKG_CONFIG_PATH, and thus propagation wouldn’t be needed in this
> case.
In that regard, would symlinking other pkg-config files as proposed
somewhere in passing in [1] also work?  We would have to add a phase,
that parses the Requires field from installed pkg-config files, and
then symlinks the respective files from the inputs to gnu-build-system, 
but since they would then exist in PKG_CONFIG_PATH and point to the
library in question, theoretically we could get around this limitation
of pkg-config without requiring propagation.

> Regarding outputs, Nixpkgs introduced a “dev” output a while back
> that lumps together our “lib” and “include” outputs, roughly.  I
> think that’s a good idea (for other reasons too).
Naming-wise I'd still prefer the more concrete “lib” as “dev” to me is
a weird umbrella term that doesn't really tell me what I'm getting
(also it probably contributes to the distinction of users and devs, and
as some distros like Debian have demonstrated often becomes
pointless[2] the minute you introduce language bindings which require
dev inputs anyway).  

As far as the lib/include split in Guix is concerned, there appear to
be few packages that split them, so “lib” effectively is “dev” in Guix
when “out” isn't.  So apart from those outliers, is there something our
“lib” outputs are missing that Nix' “dev” outputs include?

Regards

[1] 
https://yhetil.org/guix-devel/997473f47b882a670e4b9bf6fe3fc66e66ba6911.camel@gmail.com/
[2] Pointless in the sense, that “users” need to install those “dev”
packages anyway if an application requires them.  Consider for example
a Guile application built on guile-git, which therefore requires
libgit2-dev to be installed. 

Re: Rethinking propagated inputs?

[Ludovic Courtès]

Hi Liliana,

Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:

> Am Donnerstag, den 09.09.2021, 00:12 +0200 schrieb Ludovic Courtès:

[...]

>> It would be great if the ‘Requires’ field of .pc files could specify
>> absolute file name; it would no longer be necessary to set
>> PKG_CONFIG_PATH, and thus propagation wouldn’t be needed in this
>> case.
> In that regard, would symlinking other pkg-config files as proposed
> somewhere in passing in [1] also work?  We would have to add a phase,
> that parses the Requires field from installed pkg-config files, and
> then symlinks the respective files from the inputs to gnu-build-system, 
> but since they would then exist in PKG_CONFIG_PATH and point to the
> library in question, theoretically we could get around this limitation
> of pkg-config without requiring propagation.

Why not, that could work.

>> Regarding outputs, Nixpkgs introduced a “dev” output a while back
>> that lumps together our “lib” and “include” outputs, roughly.  I
>> think that’s a good idea (for other reasons too).
> Naming-wise I'd still prefer the more concrete “lib” as “dev” to me is
> a weird umbrella term that doesn't really tell me what I'm getting
> (also it probably contributes to the distinction of users and devs, and
> as some distros like Debian have demonstrated often becomes
> pointless[2] the minute you introduce language bindings which require
> dev inputs anyway).  
>
> As far as the lib/include split in Guix is concerned, there appear to
> be few packages that split them, so “lib” effectively is “dev” in Guix
> when “out” isn't.  So apart from those outliers, is there something our
> “lib” outputs are missing that Nix' “dev” outputs include?

We’d have to check how they do it in Nix.  Basically, it’s about
reversing things: currently, everything goes to “out”, and a tiny bit
goes to “lib”.  Here, it’d be the other way around: “out” would contain
only binaries and doc (unless there’s a “doc” output), and “dev” would
contain everything else (headers, pkg-config files, cmake files, section
3 man pages, etc.)

It needs more thought and it may be hard to get right, but hopefully we
can learn from Nixpkgs here.

Now, IIRC, the Nixpkgs folks found that it didn’t have a significant
impact on closure size, contrary to what they had hoped for.  That too
needs to be taken into account.

Ludo’.

Re: Rethinking propagated inputs?

[Hartmut Goebel]

Am 09.09.21 um 11:48 schrieb Ludovic Courtès:
> We’d have to check how they do it in Nix.  Basically, it’s about
> reversing things: currently, everything goes to “out”, and a tiny bit
> goes to “lib”.  Here, it’d be the other way around: “out” would contain
> only binaries and doc (unless there’s a “doc” output), and “dev” would
> contain everything else (headers, pkg-config files, cmake files, section
> 3 man pages, etc.)

+1

Additionally any static lib (.a) should go into  "lib".

-- 
Regards
Hartmut Goebel

| Hartmut Goebel          | h.goebel@crazy-compilers.com               |
| www.crazy-compilers.com | compilers which you thought are impossible |

Re: Rethinking propagated inputs?

[zimoun]

Hi,

On Sat, 04 Sep 2021 at 20:24, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:

> Does anyone have an idea how we should handle propagations for the sake
> of pkg-config?  Perhaps we could add "linked-inputs", which are added
> when building packages and environments when not using --ad-hoc, but
> not when union-building profiles.  WDYT?

The TODO file contains:

--8<---------------cut here---------------start------------->8---
** extend ‘propagated-build-inputs’ with support for multiple outputs

#+BEGIN_SRC scheme
  (outputs '("out" "include"))
  (propagated-build-inputs
    `(((("i1" ,p1 "o1")
        ("i2" ,p2))
       => "include")
      ("i3" ,p3)))
#+END_SRC
--8<---------------cut here---------------end--------------->8---

<https://git.savannah.gnu.org/cgit/guix.git/tree/TODO#n37>

which seems a direction the discussion currently takes.  Maybe it could
be worth to dig the history and resume this previous discussion.

Cheers,
simon