Including sources in guix archive --export

Including sources in guix archive --export

[Georges Dupéron]

[-- Attachment #1: Type: text/plain, Size: 1025 bytes --]

Hi!

Is it possible to export a .nar including the sources necessary to 
rebuild that package?

I tried running these commands on the host:

sudo guix archive --generate-key
guix archive --export --recursive hello > hello.nar

And these commands inside a fresh, minimal GUIX environment without 
network access:

guix archive --authorize < signing-key.pub
guix archive --import < hello.nar
$(guix build hello)/bin/hello # works
guix build --check hello # fails

The above fails because it tries to download the sources:

Starting download of /gnu/store/…-hello-2.10.tar.gz
 From https://gtpmirror.gnu.or/gnu/hello/hello-2.10.tar.gz...
In procedure getaddringo: Name or service not known

I have created at 
https://github.com/jsmaniac/guix-reproducible-system/tree/guix-question 
a small automated environment which creates the archive, creates a QEMU 
VM with a minimalistic Guix configuration, copies the .nar archive 
inside the VM and tries the commands above.

Thanks,
Georges Dupéron

[-- Attachment #2: Type: text/html, Size: 1498 bytes --]

Re: Including sources in guix archive --export

[Ludovic Courtès]

Hi Georges,

Georges Dupéron <jahvascriptmaniac@gmail.com> skribis:

> Is it possible to export a .nar including the sources necessary to
> rebuild that package?

Not directly, but you could do something like:

  guix archive --export -r hello $(guix build -S hello) > hello.nar

(Or even “guix build --sources=transitive hello”.)

HTH!

Ludo’.

Re: Including sources in guix archive --export

[Dupéron Georges]

[-- Attachment #1: Type: text/plain, Size: 1256 bytes --]

Thanks a lot Ludovic, this really helped :) .

The following command nearly worked:

guix archive --export --recursive hello $(guix build --source
--sources=transitive hello $MORE_SOURCES)

but I had to explicitly add more sources, namely /gnu/store/*-$i.drv for
these files:

binutils-2.23.2.tar.xz
bison-3.0.4.tar.xz
gc-7.6.4.tar.gz
gcc-4.8.2.tar.xz
glibc-2.18.tar.xz
guile-2.0.9.tar.xz
gcc-4.9.4.tar.xz
gettext-0.19.8.1.tar.gz
gmp-6.1.2.tar.xz
guile-2.2.3.tar.xz
libatomic_ops-7.6.4.tar.gz
libffi-3.2.1.tar.gz
libtool-2.4.6.tar.xz
libunistring-0.9.9.tar.xz
m4-1.4.18.tar.xz
perl-5.26.1.tar.gz
pkg-config-0.29.2.tar.gz
static-binaries.tar.xz
texinfo-6.5.tar.xz
zlib-1.2.11.tar.gz

It seems that these are used to bootstrap guile and GCC. Is there a way to
get this list programmatically, without hardcoding version numbers?

I have updated at
https://github.com/jsmaniac/guix-reproducible-system/tree/guix-answer a
small automated script which creates the archive, creates a QEMU VM with a
minimalistic Guix configuration, copies the .nar archive inside the VM,
imports it and tries guix build --check hello (some Guile test seems to
fall in a deadlock, but that's for another topic).

Cheers,
Georges Dupéron

[-- Attachment #2: Type: text/html, Size: 1645 bytes --]

Re: Including sources in guix archive --export

[Ludovic Courtès]

Hi Georges,

Dupéron Georges <jahvascriptmaniac@gmail.com> skribis:

> Thanks a lot Ludovic, this really helped :) .
>
> The following command nearly worked:
>
> guix archive --export --recursive hello $(guix build --source
> --sources=transitive hello $MORE_SOURCES)
>
> but I had to explicitly add more sources, namely /gnu/store/*-$i.drv for
> these files:
>
> binutils-2.23.2.tar.xz
> bison-3.0.4.tar.xz
> gc-7.6.4.tar.gz
> gcc-4.8.2.tar.xz
> glibc-2.18.tar.xz
> guile-2.0.9.tar.xz
> gcc-4.9.4.tar.xz
> gettext-0.19.8.1.tar.gz
> gmp-6.1.2.tar.xz
> guile-2.2.3.tar.xz
> libatomic_ops-7.6.4.tar.gz
> libffi-3.2.1.tar.gz
> libtool-2.4.6.tar.xz
> libunistring-0.9.9.tar.xz
> m4-1.4.18.tar.xz
> perl-5.26.1.tar.gz
> pkg-config-0.29.2.tar.gz
> static-binaries.tar.xz
> texinfo-6.5.tar.xz
> zlib-1.2.11.tar.gz
>
> It seems that these are used to bootstrap guile and GCC. Is there a way to
> get this list programmatically, without hardcoding version numbers?

Does “guix build --sources=all hello” help?

HTH,
Ludo’.

Re: Including sources in guix archive --export

[Dupéron Georges]

[-- Attachment #1: Type: text/plain, Size: 1970 bytes --]

Hi Ludovic,

On my system,

$ guix build --sources=all hello
/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
$ guix build --source --sources=all hello
/gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz

whereas guix build --source --sources=transitive hello does give a
(partial) list of dependencies. Did I miss something?

I managed to get a list with partial overlap using the following¹:
$ guix build --source --sources=transitive glibc-bootstrap gcc-bootstrap
bootstrap-binaries bootstrap-tarballs

Thanks!
Georges Dupéron

¹ It's possible that since I was including stuff using wildcards; I
accidentally dragged in more dependencies than was strictly needed.

Le lun. 24 sept. 2018 à 17:29, Ludovic Courtès <ludo@gnu.org> a écrit :

> Hi Georges,
>
> Dupéron Georges <jahvascriptmaniac@gmail.com> skribis:
>
> > Thanks a lot Ludovic, this really helped :) .
> >
> > The following command nearly worked:
> >
> > guix archive --export --recursive hello $(guix build --source
> > --sources=transitive hello $MORE_SOURCES)
> >
> > but I had to explicitly add more sources, namely /gnu/store/*-$i.drv for
> > these files:
> >
> > binutils-2.23.2.tar.xz
> > bison-3.0.4.tar.xz
> > gc-7.6.4.tar.gz
> > gcc-4.8.2.tar.xz
> > glibc-2.18.tar.xz
> > guile-2.0.9.tar.xz
> > gcc-4.9.4.tar.xz
> > gettext-0.19.8.1.tar.gz
> > gmp-6.1.2.tar.xz
> > guile-2.2.3.tar.xz
> > libatomic_ops-7.6.4.tar.gz
> > libffi-3.2.1.tar.gz
> > libtool-2.4.6.tar.xz
> > libunistring-0.9.9.tar.xz
> > m4-1.4.18.tar.xz
> > perl-5.26.1.tar.gz
> > pkg-config-0.29.2.tar.gz
> > static-binaries.tar.xz
> > texinfo-6.5.tar.xz
> > zlib-1.2.11.tar.gz
> >
> > It seems that these are used to bootstrap guile and GCC. Is there a way
> to
> > get this list programmatically, without hardcoding version numbers?
>
> Does “guix build --sources=all hello” help?
>
> HTH,
> Ludo’.
>


-- 
Georges Dupéron

[-- Attachment #2: Type: text/html, Size: 2841 bytes --]

Re: Including sources in guix archive --export

[Ludovic Courtès]

Hi Georges,

Dupéron Georges <jahvascriptmaniac@gmail.com> skribis:

> On my system,
>
> $ guix build --sources=all hello
> /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
> $ guix build --source --sources=all hello
> /gnu/store/hbdalsf5lpf01x4dcknwx6xbn6n5km6k-hello-2.10.tar.gz
>
> whereas guix build --source --sources=transitive hello does give a
> (partial) list of dependencies. Did I miss something?

Hmm no, ‘--sources=transitive’ is supposed to give you the complete list
of source tarballs.  Any idea which ones are missing?

Thanks,
Ludo’.

Re: Including sources in guix archive --export

[Georges Dupéron]

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

On mar., oct. 2, 2018 at 11:57 , Ludovic Courtès <ludo@gnu.org> wrote:
> Hmm no, ‘--sources=transitive’ is supposed to give you the 
> complete list
> of source tarballs.  Any idea which ones are missing

The ones missing seem to be the sources used to bootstrap GCC and Guile 
(there might be a couple of false positives in that list, because: I 
added these one by one to the list after "hello" by using the wildcard 
/gnu/store/*-$i.drv, until guix build inside the VM stopped indicating 
missing dependencies):

binutils-2.23.2.tar.xz
bison-3.0.4.tar.xz
gc-7.6.4.tar.gz
gcc-4.8.2.tar.xz
glibc-2.18.tar.xz
guile-2.0.9.tar.xz
gcc-4.9.4.tar.xz
gettext-0.19.8.1.tar.gz
gmp-6.1.2.tar.xz
guile-2.2.3.tar.xz
libatomic_ops-7.6.4.tar.gz
libffi-3.2.1.tar.gz
libtool-2.4.6.tar.xz
libunistring-0.9.9.tar.xz
m4-1.4.18.tar.xz
perl-5.26.1.tar.gz
pkg-config-0.29.2.tar.gz
static-binaries.tar.xz
texinfo-6.5.tar.xz
zlib-1.2.11.tar.gz

Note that when I run guix build --check hello inside a VM created from 
a minimal configuration, it tries to build much more than just hello 
(e.g. GCC, Guile, …), whereas if I run it on my Guix host, it simply 
rebuilds hello and nothing else.

Thanks :)
Georges

[-- Attachment #2: Type: text/html, Size: 1678 bytes --]

Re: Including sources in guix archive --export

[Ludovic Courtès]

Georges Dupéron <jahvascriptmaniac@gmail.com> skribis:

> On mar., oct. 2, 2018 at 11:57 , Ludovic Courtès <ludo@gnu.org> wrote:
>> Hmm no, ‘--sources=transitive’ is supposed to give you the complete
>> list
>> of source tarballs.  Any idea which ones are missing
>
> The ones missing seem to be the sources used to bootstrap GCC and
> Guile (there might be a couple of false positives in that list,
> because: I added these one by one to the list after "hello" by using
> the wildcard /gnu/store/*-$i.drv, until guix build inside the VM
> stopped indicating missing dependencies):
>
> binutils-2.23.2.tar.xz
> bison-3.0.4.tar.xz
> gc-7.6.4.tar.gz
> gcc-4.8.2.tar.xz
> glibc-2.18.tar.xz
> guile-2.0.9.tar.xz
> gcc-4.9.4.tar.xz
> gettext-0.19.8.1.tar.gz
> gmp-6.1.2.tar.xz
> guile-2.2.3.tar.xz
> libatomic_ops-7.6.4.tar.gz
> libffi-3.2.1.tar.gz
> libtool-2.4.6.tar.xz
> libunistring-0.9.9.tar.xz
> m4-1.4.18.tar.xz
> perl-5.26.1.tar.gz
> pkg-config-0.29.2.tar.gz
> static-binaries.tar.xz
> texinfo-6.5.tar.xz
> zlib-1.2.11.tar.gz

Hmm, I see.  Perhaps we have a bug here.

Thanks,
Ludo’.