native or not

native or not

[Vincent Legoll]

Hello,

I've been having a first look at guix data service.

And looking at the guix lint page, I saw the "should
probably be native input" warnings, and gave it a
try on sudo, naively thinking it would benefit
something (maybe a container or vm image size).

Is there a way to see any benefit from these changes
(without building a vm or container image each time) ?

Are those changes useful to do on their own ?

For example this:

diff --git {a,b}/gnu/packages/admin.scm
index 2f661f5e81..457dc1e3dc 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -1268,9 +1268,10 @@ system administrator.")
        #:tests? #f))
+    (native-inputs
+     `(("groff" ,groff)))
     (inputs
-     `(("groff" ,groff)
-       ("linux-pam" ,linux-pam)
+     `(("linux-pam" ,linux-pam)
        ("zlib" ,zlib)


-- 
Vincent Legoll

Re: native or not

[Mathieu Othacehe]

Hello Vincent,

> Is there a way to see any benefit from these changes
> (without building a vm or container image each time) ?
>
> Are those changes useful to do on their own ?

Well yes it may reduce the closure size of the package (run `guix size
sudo`) to get it.

It can also fix cross-compilation. Because when cross-compiling, if
groff needs to be run at build-time, it needs to be for the native
architecture and not the target one.

You can check it by running `guix build --target=aarch64-linux-gnu sudo`
for instance.

Thanks,

Mathieu

Re: native or not

[Vincent Legoll]

Hello Mathieu,

On Mon, Mar 30, 2020 at 8:57 AM Mathieu Othacehe <m.othacehe@gmail.com> wrote:
> > Are those changes useful to do on their own ?
>
> Well yes it may reduce the closure size of the package (run `guix size
> sudo`) to get it.

I'm not seeing any size difference, but groff is not in the output:

on master, groff in inputs:

$ ./pre-inst-env guix size sudo
store item                                                       total    self
/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29
37.4    35.8  36.9%
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib
70.0    32.6  33.7%
/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31
90.0    16.5  17.0%
/gnu/store/vsvba1ilj2zj536pvsil6r0mf5rnjj45-sudo-1.8.31p1
96.9     3.5   3.6%
/gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2
72.6     2.6   2.7%
/gnu/store/vkj5rdiavl87m21d9i0k69rfw79p13gj-linux-pam-1.3.1
73.2     2.1   2.2%
/gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7
1.6     1.6   1.7%
/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7
38.4     1.0   1.1%
/gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53
70.7     0.5   0.5%
/gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48
70.3     0.3   0.3%
/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11
70.2     0.2   0.2%
/gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27
70.2     0.2   0.2%
total: 96.9 MiB

on a branch with groff in native-inputs:

$ ./pre-inst-env guix size sudo
store item                                                       total    self
/gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29
37.4    35.8  36.9%
/gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib
70.0    32.6  33.7%
/gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31
90.0    16.5  17.0%
/gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1
96.9     3.5   3.6%
/gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2
72.6     2.6   2.7%
/gnu/store/vkj5rdiavl87m21d9i0k69rfw79p13gj-linux-pam-1.3.1
73.2     2.1   2.2%
/gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7
1.6     1.6   1.7%
/gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7
38.4     1.0   1.1%
/gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53
70.7     0.5   0.5%
/gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48
70.3     0.3   0.3%
/gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11
70.2     0.2   0.2%
/gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27
70.2     0.2   0.2%
total: 96.9 MiB

> It can also fix cross-compilation. Because when cross-compiling, if
> groff needs to be run at build-time, it needs to be for the native
> architecture and not the target one.
>
> You can check it by running `guix build --target=aarch64-linux-gnu sudo`
> for instance.

That fails on master (libpaper) whereas with the patch it works,
so I guess the patch is useful on that front.

The patch for sudo will be in the following emails.

Is there anything else to check / test ?

-- 
Vincent Legoll

Re: native or not

[Vincent Legoll]

[-- Attachment #1: Type: text/plain, Size: 3413 bytes --]

Here is a set of patches, for starting discussion...

I only build-tested them natively on/for x86_64 (and
cross built for aarch64 for the sudo one)

On Mon, Mar 30, 2020 at 11:25 PM Vincent Legoll
<vincent.legoll@gmail.com> wrote:
>
> Hello Mathieu,
>
> On Mon, Mar 30, 2020 at 8:57 AM Mathieu Othacehe <m.othacehe@gmail.com> wrote:
> > > Are those changes useful to do on their own ?
> >
> > Well yes it may reduce the closure size of the package (run `guix size
> > sudo`) to get it.
>
> I'm not seeing any size difference, but groff is not in the output:
>
> on master, groff in inputs:
>
> $ ./pre-inst-env guix size sudo
> store item                                                       total    self
> /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29
> 37.4    35.8  36.9%
> /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib
> 70.0    32.6  33.7%
> /gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31
> 90.0    16.5  17.0%
> /gnu/store/vsvba1ilj2zj536pvsil6r0mf5rnjj45-sudo-1.8.31p1
> 96.9     3.5   3.6%
> /gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2
> 72.6     2.6   2.7%
> /gnu/store/vkj5rdiavl87m21d9i0k69rfw79p13gj-linux-pam-1.3.1
> 73.2     2.1   2.2%
> /gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7
> 1.6     1.6   1.7%
> /gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7
> 38.4     1.0   1.1%
> /gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53
> 70.7     0.5   0.5%
> /gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48
> 70.3     0.3   0.3%
> /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11
> 70.2     0.2   0.2%
> /gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27
> 70.2     0.2   0.2%
> total: 96.9 MiB
>
> on a branch with groff in native-inputs:
>
> $ ./pre-inst-env guix size sudo
> store item                                                       total    self
> /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29
> 37.4    35.8  36.9%
> /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib
> 70.0    32.6  33.7%
> /gnu/store/9kzrrccpzl6i1sfwb0drb00gi2gwk0x0-coreutils-8.31
> 90.0    16.5  17.0%
> /gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1
> 96.9     3.5   3.6%
> /gnu/store/dvs3acxwfnwgc7yma6h3y937ri2li47y-gmp-6.1.2
> 72.6     2.6   2.7%
> /gnu/store/vkj5rdiavl87m21d9i0k69rfw79p13gj-linux-pam-1.3.1
> 73.2     2.1   2.2%
> /gnu/store/cp72ncw4prnsga65n3pzll07hpsg524f-bash-static-5.0.7
> 1.6     1.6   1.7%
> /gnu/store/29jhbbg1hf557x8j53f9sxd9imlmf02a-bash-minimal-5.0.7
> 38.4     1.0   1.1%
> /gnu/store/nffbgghxyvrj29lcgxs5fpmi3sx9zzql-acl-2.2.53
> 70.7     0.5   0.5%
> /gnu/store/in1738m2zvhgpz78n2yqa972sdzc42ss-attr-2.4.48
> 70.3     0.3   0.3%
> /gnu/store/qx7p7hiq90mi7r78hcr9cyskccy2j4bg-zlib-1.2.11
> 70.2     0.2   0.2%
> /gnu/store/waw5ci4lazbf2a1x9v6gw1v274nk0gny-libcap-2.27
> 70.2     0.2   0.2%
> total: 96.9 MiB
>
> > It can also fix cross-compilation. Because when cross-compiling, if
> > groff needs to be run at build-time, it needs to be for the native
> > architecture and not the target one.
> >
> > You can check it by running `guix build --target=aarch64-linux-gnu sudo`
> > for instance.
>
> That fails on master (libpaper) whereas with the patch it works,
> so I guess the patch is useful on that front.
>
> The patch for sudo will be in the following emails.
>
> Is there anything else to check / test ?
>
> --
> Vincent Legoll



-- 
Vincent Legoll

[-- Attachment #2: 0001-gnu-privoxy-Make-some-inputs-native.patch --]
[-- Type: text/x-patch, Size: 1003 bytes --]

From a782816fe096e786673c03dca3b2866e62933fff Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Mon, 30 Mar 2020 22:21:08 +0200
Subject: [PATCH 1/4] gnu: privoxy: Make some inputs native.

* gnu/packages/tor.scm (privoxy)[native-inputs]: New field.
[inputs]: Move autoconf & automake to native-inputs.
---
 gnu/packages/tor.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/tor.scm b/gnu/packages/tor.scm
index 61e52ba22c..841158871e 100644
--- a/gnu/packages/tor.scm
+++ b/gnu/packages/tor.scm
@@ -141,8 +141,9 @@ rejects UDP traffic from the application you're using.")
     (inputs
      `(("w3m" ,w3m)
        ("pcre" ,pcre)
-       ("zlib" ,zlib)
-       ("autoconf" ,autoconf)
+       ("zlib" ,zlib)))
+    (native-inputs
+     `(("autoconf" ,autoconf)
        ("automake" ,automake)))
     (home-page "https://www.privoxy.org")
     (synopsis "Web proxy with advanced filtering capabilities for enhancing privacy")
-- 
2.25.2


[-- Attachment #3: 0004-gnu-procenv-Make-some-inputs-native-make-multiline.patch --]
[-- Type: text/x-patch, Size: 1414 bytes --]

From 3887538d37256424d5654da5fce4c94c924a4e67 Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Mon, 30 Mar 2020 22:42:28 +0200
Subject: [PATCH 4/4] gnu: procenv: Make some inputs native & make multiline.

* gnu/packages/linux.scm (procenv)[native-inputs]: New field.
[inputs]: Move groff to native-inputs, move each remaining item on its own line.
---
 gnu/packages/linux.scm | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index a45847cbe5..945c15d972 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -5867,9 +5867,11 @@ the MTP device as a file system.")
       (base32 "1javw97yw0qvjmj14js8vw6nsfyf2xc0kfiyq5f2hsp0553w2cdq"))))
    (build-system gnu-build-system)
    (arguments `(#:configure-flags '("--disable-silent-rules")))
-   (native-inputs `(("pkg-config" ,pkg-config)))
-   (inputs `(("expat" ,expat) ("libcap" ,libcap) ("check" ,check)
-             ("groff" ,groff)           ; for tests
+   (native-inputs `(("groff" ,groff) ; for tests
+                    ("pkg-config" ,pkg-config)))
+   (inputs `(("check" ,check)
+             ("expat" ,expat)
+             ("libcap" ,libcap)
              ("libselinux" ,libselinux)))
    (synopsis "Utility to show process environment")
    (description "Procenv is a command-line tool that displays as much detail about
-- 
2.25.2


[-- Attachment #4: 0003-gnu-nftables-Make-some-inputs-native.patch --]
[-- Type: text/x-patch, Size: 1363 bytes --]

From 60e400cc78a6f2caf991800a13622906b6e6dabb Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Mon, 30 Mar 2020 22:38:44 +0200
Subject: [PATCH 3/4] gnu: nftables: Make some inputs native.

* gnu/packages/linux.scm (procenv)[native-inputs]: New field.
[inputs]: Move bison & flex to native-inputs.
---
 gnu/packages/linux.scm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 701df764cd..a45847cbe5 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -5647,13 +5647,13 @@ used by nftables.")
     (build-system gnu-build-system)
     (arguments `(#:configure-flags
                  '("--disable-man-doc"))) ; FIXME: Needs docbook2x.
-    (inputs `(("bison" ,bison)
-              ("flex" ,flex)
-              ("gmp" ,gmp)
+    (inputs `(("gmp" ,gmp)
               ("libmnl" ,libmnl)
               ("libnftnl" ,libnftnl)
               ("readline" ,readline)))
-    (native-inputs `(("pkg-config" ,pkg-config)))
+    (native-inputs `(("pkg-config" ,pkg-config)
+                     ("bison" ,bison)
+                     ("flex" ,flex)))
     (home-page "https://www.nftables.org")
     (synopsis "Userspace utility for Linux packet filtering")
     (description "nftables is the project that aims to replace the existing
-- 
2.25.2


[-- Attachment #5: 0002-gnu-sudo-Make-some-inputs-native.patch --]
[-- Type: text/x-patch, Size: 1040 bytes --]

From bd2f9cac9229bb31cc64f5d647b6ef8085f46db4 Mon Sep 17 00:00:00 2001
From: Vincent Legoll <vincent.legoll@gmail.com>
Date: Mon, 30 Mar 2020 22:28:55 +0200
Subject: [PATCH 2/4] gnu: sudo: Make some inputs native.

* gnu/packages/admin.scm (sudo)[native-inputs]: New field.
[inputs]: Move groff to native-inputs.
---
 gnu/packages/admin.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/admin.scm b/gnu/packages/admin.scm
index 2f661f5e81..457dc1e3dc 100644
--- a/gnu/packages/admin.scm
+++ b/gnu/packages/admin.scm
@@ -1268,9 +1268,10 @@ system administrator.")
        ;; XXX: The 'testsudoers' test series expects user 'root' to exist, but
        ;; the chroot's /etc/passwd doesn't have it.  Turn off the tests.
        #:tests? #f))
+    (native-inputs
+     `(("groff" ,groff)))
     (inputs
-     `(("groff" ,groff)
-       ("linux-pam" ,linux-pam)
+     `(("linux-pam" ,linux-pam)
        ("zlib" ,zlib)
        ("coreutils" ,coreutils)))
     (home-page "https://www.sudo.ws/")
-- 
2.25.2

Re: native or not

[Mathieu Othacehe]

Hey,

> I'm not seeing any size difference, but groff is not in the output:

OK, then its normal.

> That fails on master (libpaper) whereas with the patch it works,
> so I guess the patch is useful on that front.
>
> The patch for sudo will be in the following emails.
>
> Is there anything else to check / test ?

Well you can also check for deterministic compilation, and all the other
items listed here:
https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html

Also be careful because there were quite a lot of cross-compilation
related patches pushed on core-updates. You may want to check that you
are not duplicating this work.

Anyway, most of the packages that should cross-compile fail to do so
because of inputs/native-inputs mix-up, so your patches are really
welcome :)

Thanks,

Mathieu

Re: native or not

[Mathieu Othacehe]

> I only build-tested them natively on/for x86_64 (and
> cross built for aarch64 for the sudo one)

This LGTM, I added your copyright on the two first patches and pushed!

Thanks,

Mathieu

Re: native or not

[Vincent Legoll]

Hello,

On Tue, Mar 31, 2020 at 9:45 AM Mathieu Othacehe <m.othacehe@gmail.com> wrote:
> This LGTM, I added your copyright on the two first patches and pushed!

Thanks for handling that.

Stay tuned for the next batch.

Christopher, looks like your work on data.guix.gnu.org already triggered
something useful ! ;-)

--
Vincent Legoll

Re: native or not

[Vincent Legoll]

> Christopher, looks like your work on data.guix.gnu.org
> already triggered something useful ! ;-)

And the view for comparing with previous run is just showing
me what I wanted to see, perfect !

Question: would this view show any `guix size` difference if any ?

Keep up the good work !

-- 
Vincent Legoll

Re: native or not

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

Vincent,

Vincent Legoll 写道：
> I'm not seeing any size difference, but groff is not in the 
> output:

There's some deeper confusion here: why do you expect the size to 
change, at all?

Making ‘groff’ native was the right thing to do (and please, keep 
fixing bugs like this! :-) but it has nothing to do with making 
packages smaller.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: native or not

[Vincent Legoll]

Hello,

On Tue, Mar 31, 2020 at 11:44 AM Tobias Geerinckx-Rice <me@tobias.gr> wrote:
> Vincent Legoll 写道：
> > I'm not seeing any size difference, but groff is not in the
> > output:
>
> There's some deeper confusion here: why do you expect the size to
> change, at all?

Because I've been told so...

On Mon, Mar 30, 2020 at 8:57 AM Mathieu Othacehe wrote:
> Well yes it may reduce the closure size of the package (run `guix size
> sudo`) to get it.

But I don't expect that to happen every time (I've seen the "it may")...

> Making ‘groff’ native was the right thing to do (and please, keep
> fixing bugs like this! :-) but it has nothing to do with making
> packages smaller.

Never will ?

I'm not expecting package size to shrink, but package closure
(is that the right word) size to sometimes shrink.

And if I've understood that may be visible for ISO, VM, container
image sizes.

Am I mistaken ?
Still learning...

-- 
Vincent Legoll

Re: native or not

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 506 bytes --]

Vincent Legoll <vincent.legoll@gmail.com> writes:

>> Making ‘groff’ native was the right thing to do (and please, keep
>> fixing bugs like this! :-) but it has nothing to do with making
>> packages smaller.
>
> Never will ?
>
> I'm not expecting package size to shrink, but package closure
> (is that the right word) size to sometimes shrink.

Unless you are cross-compiling (i.e. guix build --target=foo),
native-inputs and inputs behave exactly the same; they are simply
concatenated.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: native or not

[Christopher Baines]

[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]


Vincent Legoll <vincent.legoll@gmail.com> writes:

>> Christopher, looks like your work on data.guix.gnu.org
>> already triggered something useful ! ;-)
>
> And the view for comparing with previous run is just showing
> me what I wanted to see, perfect !

Nice, I'm glad that you're enjoying using the Guix Data Service, and
finding it useful :)

> Question: would this view show any `guix size` difference if any ?

So, `guix size` as I understand it looks at the size of the store item +
the size of all store items referenced both directly and indirectly.

The `guix size` script can use your local store, as well the data in
narinfo files from substitute files to determine the size of store
items, and there references.

The Guix Data Service gathers up narinfo files from substitute servers,
so the database should contain the necessary data on sizes and
references to provide equivalent information to the `guix size` script.

I like the idea of showing size changes on the comparison page, but I
think a good first step to take towards this would be to show the "size"
for a single item in the store.

This [1] is the page for the store output for the sudo package, with
your changes. It already shows the size from the narinfo files provided
by bayfront and berlin (3633056 bytes).

1: http://data.guix.gnu.org/gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1

It would be nice to have a URL like [2] (which obviously doesn't exist
yet) which would show similar information to running the `guix size`
script [3].

2: http://data.guix.gnu.org/gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1/size
3: guix size /gnu/store/l320ig872ny66d1yi6v7n4zb93iz50dx-sudo-1.8.31p1

Would this be something I'd be able to convince and support you to do
Vincent? I'd be more than happy to help you to implement this.

Chris

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 962 bytes --]

Re: native or not

[Vincent Legoll]

Hello,

On Tue, Mar 31, 2020 at 3:25 PM Marius Bakke <mbakke@fastmail.com> wrote:
> Unless you are cross-compiling (i.e. guix build --target=foo),
> native-inputs and inputs behave exactly the same; they are simply
> concatenated.

OK, thanks for the clarification.

Christopher, I'm not a web guy, I may take a shot at it, when
my todo list gets emptyish, but don't hold your breath, that
may not happen in this life...

Following this will come the next batch of native-inputs.
Built on x86_64, tried to also target aarch64, but kind of
unconclusive (one stopped at perl, another at mit-krb5...)

I've searched core-updates to avoid duplicating work,
but only after having done it... Got lucky to have only
lost a few minutes on openldap...

The graphviz patch may not be for master as it may
force 2918 packages to be rebuilt. The other ones
should be OK.

Question: should I stay on guix-devel, should I add
guix-patches or should I move to guix-patches for
the following patche series ?

-- 
Vincent Legoll

[PATCH 1/6] gnu: cgit: Make some inputs native.

[Vincent Legoll]

* gnu/packages/version-control.scm (cgit)[native-inputs]: New field.
[inputs]: Move groff & python-docutils to native-inputs.
---
 gnu/packages/version-control.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index 8af54c6e35..0aff346e0a 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -28,6 +28,7 @@
 ;;; Copyright © 2020 Roel Janssen <roel@gnu.org>
 ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>
 ;;; Copyright © 2020 John D. Boy <jboy@bius.moe>
+;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -857,6 +858,8 @@ collaboration using typical untrusted file hosts or services.")
      `(("asciidoc" ,asciidoc)
        ("gzip" ,gzip)
        ("bzip2" ,bzip2)
+       ("groff" ,groff)
+       ("python-docutils" ,python-docutils)
        ("xz" ,xz)))
     (inputs
      `(;; Building cgit requires a Git source tree.
@@ -869,9 +872,7 @@ collaboration using typical untrusted file hosts or services.")
            (sha256
             (base32 "09lzwa183nblr6l8ib35g2xrjf9wm9yhk3szfvyzkwivdv69c9r2"))))
        ("openssl" ,openssl)
-       ("groff" ,groff)
        ("python" ,python)
-       ("python-docutils" ,python-docutils)
        ("python-markdown" ,python-markdown)
        ("python-pygments" ,python-pygments)
        ("zlib" ,zlib)))
-- 
2.25.2

[PATCH 2/6] gnu: darktable: Make some inputs native.

[Vincent Legoll]

* gnu/packages/photo.scm (darktable)[native-inputs]: New field.
[inputs]: Move intltool & pkg-config to native-inputs.
---
 gnu/packages/photo.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/photo.scm b/gnu/packages/photo.scm
index 585289daf1..9f6e4a3031 100644
--- a/gnu/packages/photo.scm
+++ b/gnu/packages/photo.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2018 Leo Famulari <leo@famulari.name>
 ;;; Copyright © 2020 Sebastian Schott <sschott@mailbox.org>
+;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -483,6 +484,9 @@ photographic equipment.")
                      (string-append (assoc-ref inputs "ilmbase")
                                     "/include/OpenEXR:" (or (getenv "CPATH") "")))
              #t)))))
+    (native-inputs
+     `(("intltool" ,intltool)
+       ("pkg-config" ,pkg-config)))
     (inputs
      `(("libxslt" ,libxslt)
        ("libxml2" ,libxml2)
@@ -502,9 +506,7 @@ photographic equipment.")
        ("ilmbase" ,ilmbase)
        ("libsoup" ,libsoup)
        ("python-jsonschema" ,python-jsonschema)
-       ("intltool" ,intltool)
        ("perl" ,perl)
-       ("pkg-config" ,pkg-config)
        ("libwebp" ,libwebp)
        ("lensfun" ,lensfun)
        ("librsvg" ,librsvg)
-- 
2.25.2

[PATCH 3/6] gnu: graphviz: Make some inputs native.

[Vincent Legoll]

* gnu/packages/graphviz.scm (graphviz)[native-inputs]: New field.
[inputs]: Move swig to native-inputs.
---
 gnu/packages/graphviz.scm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/graphviz.scm b/gnu/packages/graphviz.scm
index 2d2bb11130..051089881a 100644
--- a/gnu/packages/graphviz.scm
+++ b/gnu/packages/graphviz.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2017 Gábor Boskovits <boskovits@gmail.com>
 ;;; Copyright © 2018 Mathieu Lirzin <mthl@gnu.org>
 ;;; Copyright © 2020 Marius Bakke <mbakke@fastmail.com>
+;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -97,7 +98,6 @@
        ("gts" ,gts)
        ("gd" ,gd)                                 ; FIXME: Our GD is too old
        ("guile" ,guile-2.0)                       ;Guile bindings
-       ("swig" ,swig)
        ("pango" ,pango)
        ("fontconfig" ,fontconfig)
        ("freetype" ,freetype)
@@ -108,6 +108,7 @@
        ("libpng" ,libpng)))
     (native-inputs
      `(("bison" ,bison)
+       ("swig" ,swig)
        ("pkg-config" ,pkg-config)))
     (outputs '("out" "doc"))                      ; 5 MiB of html + pdfs
     (home-page "http://www.graphviz.org/")
-- 
2.25.2

[PATCH 4/6] gnu: iwd: Make some inputs native.

[Vincent Legoll]

* gnu/packages/networking.scm (iwd)[native-inputs]: New field.
[inputs]: Move libtool to native-inputs.
---
 gnu/packages/networking.scm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index ec2f0b64bd..79b07e23f0 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -2712,13 +2712,13 @@ protocol daemons for BGP, IS-IS, LDP, OSPF, PIM, and RIP. ")
     (build-system gnu-build-system)
     (inputs
      `(("dbus" ,dbus)
-       ("libtool" ,libtool)
        ("ell" ,ell)
        ("readline" ,readline)))
     (native-inputs
      `(("asciidoc" ,asciidoc)
        ("autoconf" ,autoconf)
        ("automake" ,automake)
+       ("libtool" ,libtool)
        ("pkgconfig" ,pkg-config)
        ("python" ,python)
        ("openssl" ,openssl)))
-- 
2.25.2

[PATCH 5/6] gnu: mailutils: Make some inputs native.

[Vincent Legoll]

* gnu/packages/mail.scm (mailutils)[native-inputs]: New field.
[inputs]: Move dejagnu & texinfo to native-inputs.
---
 gnu/packages/mail.scm | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index 36b94f409f..458d82ecce 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -29,6 +29,7 @@
 ;;; Copyright © 2018 Gábor Boskovits <boskovits@gmail.com>
 ;;; Copyright © 2018, 2019, 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org>
+;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -218,11 +219,12 @@
 
        #:parallel-tests? #f))
     (native-inputs
-     `(("perl" ,perl)))                           ;for 'gylwrap'
+     `(("perl" ,perl)
+       ("texinfo" ,texinfo)
+       ("dejagnu" ,dejagnu)))                           ;for 'gylwrap'
     (inputs
-     `(("dejagnu" ,dejagnu)
+     `(
        ("m4" ,m4)
-       ("texinfo" ,texinfo)
        ("guile" ,guile-2.2)
        ("gsasl" ,gsasl)
        ("gnutls" ,gnutls)
-- 
2.25.2

[PATCH 6/6] gnu: nethack: Make some inputs native.

[Vincent Legoll]

* gnu/packages/games.scm (nethack)[native-inputs]: New field.
[inputs]: Move flex & bison to native-inputs.
---
 gnu/packages/games.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/games.scm b/gnu/packages/games.scm
index 3284459021..addadebbba 100644
--- a/gnu/packages/games.scm
+++ b/gnu/packages/games.scm
@@ -48,6 +48,7 @@
 ;;; Copyright © 2017, 2019 Hartmut Goebel <h.goebel@crazy-compilers.com>
 ;;; Copyright © 2020 Alberto Eleuterio Flores Guerrero <barbanegra+guix@posteo.mx>
 ;;; Copyright © 2020 Naga Malleswari <nagamalli@riseup.net>
+;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -1250,10 +1251,11 @@ watch your CPU playing while enjoying a cup of tea!")
                         (string-join (string-split version #\.) "") "-src.tgz"))
         (sha256
           (base32 "1liyckjp34j354qnxc1zn9730lh1p2dabrg1hap24z6xnqx0rpng"))))
+    (native-inputs
+      `(("bison" ,bison)
+        ("flex" ,flex)))
     (inputs
       `(("ncurses" ,ncurses)
-        ("bison" ,bison)
-        ("flex" ,flex)
         ("less" ,less)))
     (build-system gnu-build-system)
     (arguments
-- 
2.25.2

Re: [PATCH 1/6] gnu: cgit: Make some inputs native.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 2163 bytes --]

Vincent,

Thank you!

This list is for general Guix discussion.  Although that can 
sometimes include rough patches (heh), please send any future 
contributions to our patch tracker at guix-patches@gnu.org — after 
carefully reading the ‘Submitting Patches’ section of the Guix 
manual.

Vincent Legoll 写道：
> * gnu/packages/version-control.scm (cgit)[native-inputs]: New 
> field.

This is no new field.  Neither are those of mailutils, iwd, or 
graphviz!

Please rewrite all commit messages as either

  * gnu/packages/version-control.scm (cgit)[inputs]: Move groff &
  python-docutils from here…
  [native-inputs]: …to this new field.

or

  * gnu/packages/version-control.scm (cgit)[inputs]: Move groff &
  python-docutils from here…
  [native-inputs]: …to here.

although to be honest, I only ever use the latter format unless 
I'm feeling particularly pedantic.

> [inputs]: Move groff & python-docutils to native-inputs.

If we run

  $ guix gc --references `guix build cgit`

we can see that both groff and python-docutils are part of cgit's 
closure.  This means they must to be able to run… at run time :-o 
Oh dear.

python-docutils's ‘rst2html.py’ is required by cgit's 
/lib/cgit/filters/html-converters/rst2html, and groff itself by 
cgit's /lib/cgit/filters/html-converters/man2html.  Both of these 
uses look legitimate to me: they weren't accidentally captured by 
a stray environment variable or overzealous wrapper.

Now, it's still possible that either or both of these dependents 
must *also* run at build time, and for that they would need to be 
native indeed.  Unfortunately:

  $ guix build cgit --target=mips64el-linux-gnu
  guix build: error: gnu/packages/python-xyz.scm:2950:2:
  python-docutils@0.16: build system `python' does not support 
  cross
  builds

I could ignore that and pretend that all Python packages are 
safely architecture-independent (they're not) to continue 
investigating, but I'm out of time.

I don't think this particular patch adds any value at this time, 
so let's drop it.

The rest LGTM!

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [PATCH 1/6] gnu: cgit: Make some inputs native.

[Mathieu Othacehe]

Hello Vincent,

> +       ("groff" ,groff)
> +       ("python-docutils" ,python-docutils)
>         ("xz" ,xz)))

python-docutils provides rst2html.py that seem to be used at run-time
(see patch-absolute-file-names phase).

So I think it should be an input not a native-input.

Thanks,

Mathieu

Re: [PATCH 2/6] gnu: darktable: Make some inputs native.

[Mathieu Othacehe]

> +    (native-inputs
> +     `(("intltool" ,intltool)
> +       ("pkg-config" ,pkg-config)))
>      (inputs
>       `(("libxslt" ,libxslt)
>         ("libxml2" ,libxml2)
> @@ -502,9 +506,7 @@ photographic equipment.")
>         ("ilmbase" ,ilmbase)
>         ("libsoup" ,libsoup)
>         ("python-jsonschema" ,python-jsonschema)
> -       ("intltool" ,intltool)
>         ("perl" ,perl)
> -       ("pkg-config" ,pkg-config)
>         ("libwebp" ,libwebp)
>         ("lensfun" ,lensfun)
>         ("librsvg" ,librsvg)

I think that "perl" might also be a native-input. Could you check it?

Thanks,

Mathieu

Re: [PATCH 5/6] gnu: mailutils: Make some inputs native.

[Mathieu Othacehe]

> +     `(
>         ("m4" ,m4)

You can join those two lines.

Thanks,

Mathieu

Re: [PATCH 6/6] gnu: nethack: Make some inputs native.

[Mathieu Othacehe]

> +    (native-inputs
> +      `(("bison" ,bison)
> +        ("flex" ,flex)))
>      (inputs
>        `(("ncurses" ,ncurses)
> -        ("bison" ,bison)
> -        ("flex" ,flex)
>          ("less" ,less)))
>      (build-system gnu-build-system)
>      (arguments

This one LGTM but it doesn't apply on master. Could you please send an
updated serie?

Thanks for your work!

Mathieu

Re: [PATCH 1/6] gnu: cgit: Make some inputs native.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 615 bytes --]

Mathieu,

Did you not get my reply[0] yesterday?  Please let me know if it 
didn't get delivered.

Mathieu Othacehe 写道：
>> +       ("groff" ,groff)
>> +       ("python-docutils" ,python-docutils)
>>         ("xz" ,xz)))
>
> python-docutils provides rst2html.py that seem to be used at 
> run-time
> (see patch-absolute-file-names phase).
>
> So I think it should be an input not a native-input.

Both are used at run-time to make the pretty HTMLs, so I think 
this patch can be dropped.

Kind regards,

T G-R

[0]: 
https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00509.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [PATCH 2/6] gnu: darktable: Make some inputs native.

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 162 bytes --]

Mathieu Othacehe 写道：
> I think that "perl" might also be a native-input. Could you 
> check it?

Good catch!  Indeed, it is.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: [PATCH 1/6] gnu: cgit: Make some inputs native.

[Mathieu Othacehe]

Hey,

> Did you not get my reply[0] yesterday?  Please let me know if it didn't get
> delivered.

Nope, didn't get it. Too bad because you did a more complete review :)

Mathieu

Re: [PATCH 1/6] gnu: cgit: Make some inputs native.

[Vincent Legoll]

I'll send a revised patchset, taking all the comments
into account.

Thanks

-- 
Vincent Legoll

Re: native or not

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 5437 bytes --]

Vincent,

Vincent Legoll 写道：
> On Tue, Mar 31, 2020 at 11:44 AM Tobias Geerinckx-Rice 
> <me@tobias.gr> wrote:
>> There's some deeper confusion here: why do you expect the size 
>> to
>> change, at all?
>
> Because I've been told so...

Hm.  I don't think it's correct.  Perhaps this was in the context 
of one specific case where making something native had this 
pleasant side effect?

It's certainly not the reason native-inputs and inputs exist.

>> Making ‘groff’ native was the right thing to do (and please, 
>> keep
>> fixing bugs like this! :-) but it has nothing to do with making
>> packages smaller.
>
> Never will ?

If not never, as a happy side effect at best.

Doing anything for its side effect while being unaware of their 
potentially significant main effects is a recipe for ruin.

> I'm not expecting package size to shrink, but package closure
> (is that the right word) size to sometimes shrink.

So yes, I think you're using the right word.  I was certainly 
talking about closure myself, so that doesn't change my points 
above.

> And if I've understood that may be visible for ISO, VM, 
> container
> image sizes.
>
> Am I mistaken ?

You're correct that the size of an ISO (or VM, container, …) is 
nothing more than the ‘final closure’ over all the store items 
that it contains.

> Still learning...

Of course!  This is something that confuses *many* people 
initially, until it finally clicks.

Below this mail, I'm attaching a draft of the response I'd written 
to your first message in this thread.  I abandoned it due to time 
constraints.  It's… rough, and I've explained this much better on 
IRC at least once, but maybe it can help.

Kind regards,

T G-R

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Vincent Legoll 写道：
> And looking at the guix lint page, I saw the "should
> probably be native input" warnings, and gave it a
> try on sudo, naively thinking it would benefit
> something (maybe a container or vm image size).

The linter doesn't mention size at all, so there's a hidden (but
common) assumption above: that

- ‘inputs’ is just a fancy word for ‘run-time dependency’, and
- ‘native-inputs’ is just an obscure way to say ‘build-time 
  dependency’.

It's very important that you understand that neither of those is 
true.

It might help to think of Guix as two completely separate things:

1. a functional package builder that takes a package recipe and
   its various inputs (mainly source & {native-,}inputs), builds
   it, and saves the output to /gnu/store/<hash>-package.

   While it's true that build-time-only dependencies are often
   native-inputs, and run-time dependencies should probably be
   regular inputs, the two aren't equivalent

2. a functional package manager that only deals with the items
   (build artefacts) that it finds in /gnu/store.  It's important
   to note that this part does *not* deal with package recipes at
   all!

   The notion of a run-time dependency in Guix is simple and
   brilliant[0]: does the string ‘/gnu/store/<hash>-foo’ appear
   anywhere in ‘/gnu/store/<hash>-bar’?  If so, then foo is a
   dependency of bar because bar ‘refers’ to foo.  That's it.

   This side of Guix's personality doesn't care about package
   recipes *at all*.  Which is good, since it needs to deal
   with store items whose package recipes have long vanished.
   Therefore, it can't know or care which ‘/gnu/store/…’ strings
   it finds were once native, and which weren't.  Nativeness

This is a vast oversimplification (no mention of .drvs and of 
course
Guix can't be simply chopped in half Solomon-stylee), but it's 
been
helpful before.  I've also ignored propagated-inputs, which is 
fine.

> Is there a way to see any benefit from these changes
> (without building a vm or container image each time) ?

The benefit is that, if your changes are correct, it will (help) 
fix cross-compilation.

You can always try that yourself with

  $ guix build --target=mips64el-linux-gnu sudo

Guix will build/install a cross-toolchain the first time, which 
will take a while, but after that you'll be building just regular 
packages at roughly the same speed.

When not cross-compiling, there's no difference at all:

- With groff as a regular input:

  $ guix build sudo
  …
  environment variable `PATH' set to 
  `…:/gnu/store/hhwzz…-groff-1.22.4/bin:…'
  environment variable `LIBRARY_PATH' set to 
  `…:/gnu/store/hhwzz…-groff-1.22.4/lib:…'
  …

- Now let's make groff as a native-input: surely this will have an
  effect on, say, which libraries we're allowed to link against?

  $ guix build sudo
  …
  environment variable `PATH' set to 
  `…:/gnu/store/hhwzz…-groff-1.22.4/bin:…'
  environment variable `LIBRARY_PATH' set to 
  `…:/gnu/store/hhwzz…-groff-1.22.4/lib:…'
  …

Nope.  That's right: when not cross-compiling, all your precious 
native-vs.-not distinctions are tossed in the bin before the build 
has even started.

> Are those changes useful to do on their own ?

Absolutely!  Making groff native here is the right thing to do. 
You just got distracted by something as irrelevant as size ;-)

Kind regards,

T G-R

[0]: This is something we inherited from Nix.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: native or not

[Vincent Legoll]

Thanks a lot, Tobias, for this explanation, I certainly had
some of those assumptions. It may not completely enlighten
me at the first reading, but it certainly helps knowing that "it's
more complicated" than what I assumed.