Setting up guix source tree for testing a package

Setting up guix source tree for testing a package

[Zelphir Kaltstahl]

Hello Guix Users,

I am trying to set up GNU Guix for package testing, as I want to test a package
before sending in a patch to add it to GNU Guix.

I'll describe the steps I am taking and where I am seeing them in the docs in
the following.

All is happening inside a VM, which runs a Debian 10. The VM is almost fresh,
except for guest additions and git and vim and such things.


I start at https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html
<https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html>, wherein I
find a link to https://guix.gnu.org/manual/en/html_node/Defining-Packages.html
<https://guix.gnu.org/manual/en/html_node/Defining-Packages.html>.

Next the packaging guidelines tell me, that I may run for example:

~~~~
./pre-inst-env guix build gnew --keep-failed
~~~~

inside the Guix build tree.

I clone Guix, so that I have a source tree of Guix.

I define my package inside `guix/gnu/packages/guile.scm`.

In packaging guidelines, there is a link to
https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
<https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>.
I am supposed to run the test inside a fresh Guix, to make sure, that the
package will work fine for others, if I understand correctly.

On
https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
<https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>
I am told, that I need to generate a pre-installation script to run guix without
installing it (running stuff in the Guix build tree, as far as I understand). To
generate it, I need Guix again. The docs direct me to
https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
<https://guix.gnu.org/manual/en/html_node/Building-from-Git.html>.

On https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
<https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> the dilemma is
finally uncovered, that I need Guix to build the pre-installation script, to use
that, to build a package in the Guix source tree.

> As you can see, there is a chicken-and-egg problem: you first need to have
Guix installed. Typically you would install Guix System (see System
Installation) or Guix on top of another distro (see Binary Installation); in
either case, you would verify the OpenPGP signature on the installation medium.
This “bootstraps” the trust chain.

I install it.

Then I run:

~~~~
guix environment guix --pure
~~~~

The docs tell me I have to run:

~~~~
./bootstrap
./configure --localstatedir=some_directory
~~~~

But here I am facing a problem. I have no clue what to set the
`--localstatedir=directory` to. The docs tell me it is important, but I see no
instructions on how to choose a directory for the local state, when I have
already a Guix installed (binary installation on foreign distro). The linked
docs about the https://guix.gnu.org/manual/en/html_node/The-Store.html
<https://guix.gnu.org/manual/en/html_node/The-Store.html> do not help me
choosing it either, because that page is about the store itself, not about my
specific scenario of doing things from Guix source tree. Since I intend to build
a package inside the Guix source tree, I think it should exactly _not_ be /var,
as that would conflict with the Guix installation, that I had to do to create
the pre-inst-env script.

What is the idea for the local state directory? Is it the location of a
temporary store, which will contain anything, that I produce using the
pre-inst-env script? That is what I am guessing. In that case I think
`localstoredir` might be a good alias for `localstatedir`.

I am trying with an empty directory, which I create just for that purpose, on
the same level as the cloned Guix repository:

~~~~
$ tree -L 2
.
├── guix
│   ├── ABOUT-NLS
│   ├── aclocal.m4
│   ├── AUTHORS
│   ├── autom4te.cache
│   ├── bootstrap
│   ├── build-aux
│   ├── ChangeLog
│   ├── CODE-OF-CONDUCT
│   ├── config-daemon.ac
│   ├── config.log
│   ├── config.status
│   ├── configure
│   ├── configure.ac
│   ├── COPYING
│   ├── doc
│   ├── etc
│   ├── gnu
│   ├── gnu.scm
│   ├── guix
│   ├── guix.scm
│   ├── HACKING
│   ├── INSTALL
│   ├── m4
│   ├── Makefile
│   ├── Makefile.am
│   ├── Makefile.in
│   ├── NEWS
│   ├── nix
│   ├── po
│   ├── pre-inst-env
│   ├── README
│   ├── ROADMAP
│   ├── scripts
│   ├── test-env
│   ├── tests
│   ├── THANKS
│   └── TODO
└── guix-store
~~~~

So I run:

~~~~
./configure --localstatedir=/home/user/dev/guix-store
~~~~

Next is running:

~~~~
make check
~~~~

Some warnings appear:

~~~~
Your input po file po/doc/guix-manual.de.po seems outdated (The amount of entries differ between files: 10012 is not 334
). Please consider running po4a-updatepo to refresh it.
~~~~

But I am guessing, that these can be ignored for my purposes, as I am not doing
anything with translations or po files.

However, there are other errors, like not finding code for some modules (the
output of make check is very long and probably I should rather put the log
somewhere and link to it). The check seems to fail ultimately with the following
result:

~~~~
make[4]: *** [Makefile:5563: tests/store.log] Error 1
make[4]: Leaving directory '/home/user/dev/guix'
make[3]: *** [Makefile:5545: check-TESTS] Error 2
make[3]: Leaving directory '/home/user/dev/guix'
make[2]: *** [Makefile:5794: check-am] Error 2
make[2]: Leaving directory '/home/user/dev/guix'
make[1]: *** [Makefile:5322: check-recursive] Error 1
make[1]: Leaving directory '/home/user/dev/guix'
make: *** [Makefile:5796: check] Error 2
~~~~

The only red colored fail I can see is above from that, first line of the following:

~~~~
FAIL: tests/go.scm
PASS: tests/grafts.scm
PASS: tests/graph.scm
PASS: tests/gremlin.scm
SKIP: tests/hackage.scm
PASS: tests/import-utils.scm
PASS: tests/inferior.scm
PASS: tests/lint.scm
PASS: tests/modules.scm
PASS: tests/monads.scm
PASS: tests/nar.scm
PASS: tests/networking.scm
PASS: tests/opam.scm
PASS: tests/openpgp.scm
PASS: tests/packages.scm
SKIP: tests/pack.scm
PASS: tests/pki.scm
PASS: tests/print.scm
PASS: tests/processes.scm
PASS: tests/profiles.scm
SKIP: tests/publish.scm
SKIP: tests/pypi.scm
PASS: tests/records.scm
PASS: tests/scripts.scm
PASS: tests/search-paths.scm
PASS: tests/services.scm
PASS: tests/services/file-sharing.scm
PASS: tests/services/linux.scm
PASS: tests/sets.scm
PASS: tests/size.scm
SKIP: tests/snix.scm
PASS: tests/status.scm
PASS: tests/store-database.scm
PASS: tests/store-deduplication.scm
PASS: tests/store-roots.scm
make[4]: *** [Makefile:5563: tests/store.log] Error 1
make[4]: Leaving directory '/home/user/dev/guix'
make[3]: *** [Makefile:5545: check-TESTS] Error 2
make[3]: Leaving directory '/home/user/dev/guix'
make[2]: *** [Makefile:5794: check-am] Error 2
make[2]: Leaving directory '/home/user/dev/guix'
make[1]: *** [Makefile:5322: check-recursive] Error 1
make[1]: Leaving directory '/home/user/dev/guix'
make: *** [Makefile:5796: check] Error 2
~~~~

I am not sure, whether this means, that I have an erroneous Guix or whether the
localstatedir was wrong or what else it means. However, I continue.

The docs on https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
<https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> continue:

> From there on, you can authenticate all the commits included in your checkout
by running: make authenticate

It is not clear to me, whether that means, that I need to run it in my scenario
or whether I should not run it.

A look into the Makefile shows, that it is actually calling a guix command `guix
git authenticate`. The description is "verify commit signatures and
authorizations". "verify" or "verify-signatures" seems a more appropriate name,
because it indicates checking signatures, instead of authenticating commits.
However, this might be my lacking understanding of what `make authenticate` does
or aims to do.

When I try running:

~~~~
make authenticate
~~~~

I get an error:

~~~~
$ make authenticate 
Authenticating Git checkout...
/bin/bash: line 1: guix: command not found
make: *** [Makefile:6290: authenticate] Error 127
~~~~

I am confused as to why it does not find guix. I am inside the environment
created by `guix environment guix --pure`.

I continue nevertheless and try to run:

~~~~
./pre-inst-env guix build guile-fslib
~~~~

However, my whole setup now seems wo be messed up. The `guix` command can simply
no longer be found:

~~~~
$ pwd
/home/user/dev
$ guix environment guix --pure
$ pushd guix
~/dev/guix ~/dev
$ ./pre-inst-env guix build guile-fslib
guix build: error: failed to connect to `/home/user/dev/guix-store/guix/daemon-socket/socket': No such file or directory
~~~~

A logout and login does not change the error.

Where did I go wrong?

Regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Setting up guix source tree for testing a package

[Fredrik Salomonsson]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> Hello Guix Users,
>
> I am trying to set up GNU Guix for package testing, as I want to test a package
> before sending in a patch to add it to GNU Guix.
>
> I'll describe the steps I am taking and where I am seeing them in the docs in
> the following.
>
> All is happening inside a VM, which runs a Debian 10. The VM is almost fresh,
> except for guest additions and git and vim and such things.
>
>
> I start at https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html
> <https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html>, wherein I
> find a link to https://guix.gnu.org/manual/en/html_node/Defining-Packages.html
> <https://guix.gnu.org/manual/en/html_node/Defining-Packages.html>.
>
> Next the packaging guidelines tell me, that I may run for example:
>
> ~~~~
> ./pre-inst-env guix build gnew --keep-failed
> ~~~~
>
> inside the Guix build tree.
>
> I clone Guix, so that I have a source tree of Guix.
>
> I define my package inside `guix/gnu/packages/guile.scm`.
>
> In packaging guidelines, there is a link to
> https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
> <https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>.
> I am supposed to run the test inside a fresh Guix, to make sure, that the
> package will work fine for others, if I understand correctly.
>
> On
> https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
> <https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>
> I am told, that I need to generate a pre-installation script to run guix without
> installing it (running stuff in the Guix build tree, as far as I understand). To
> generate it, I need Guix again. The docs direct me to
> https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html>.
>
> On https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> the dilemma is
> finally uncovered, that I need Guix to build the pre-installation script, to use
> that, to build a package in the Guix source tree.
>
> As you can see, there is a chicken-and-egg problem: you first need to have
> Guix installed. Typically you would install Guix System (see System
> Installation) or Guix on top of another distro (see Binary Installation); in
> either case, you would verify the OpenPGP signature on the installation medium.
> This “bootstraps” the trust chain.
>
> I install it.

You don't need guix to build guix, it just makes it a lot easier to
setup the environment to build guix as you don't need to hunt for its
dependencies yourself.

> Then I run:
>
> ~~~~
> guix environment guix --pure
> ~~~~
>
> The docs tell me I have to run:
>
> ~~~~
> ./bootstrap
> ./configure --localstatedir=some_directory
> ~~~~
>
> But here I am facing a problem. I have no clue what to set the
> `--localstatedir=directory` to. The docs tell me it is important, but I see no
> instructions on how to choose a directory for the local state, when I have
> already a Guix installed (binary installation on foreign distro). The linked
> docs about the https://guix.gnu.org/manual/en/html_node/The-Store.html
> <https://guix.gnu.org/manual/en/html_node/The-Store.html> do not help me
> choosing it either, because that page is about the store itself, not about my
> specific scenario of doing things from Guix source tree. Since I intend to build
> a package inside the Guix source tree, I think it should exactly _not_ be /var,
> as that would conflict with the Guix installation, that I had to do to create
> the pre-inst-env script.
>
> What is the idea for the local state directory? Is it the location of a
> temporary store, which will contain anything, that I produce using the
> pre-inst-env script? That is what I am guessing. In that case I think
> `localstoredir` might be a good alias for `localstatedir`.
>
>
> I am trying with an empty directory, which I create just for that purpose, on
> the same level as the cloned Guix repository:
>
> ~~~~
> $ tree -L 2
> .
> ├── guix
> │   ├── ABOUT-NLS
> │   ├── aclocal.m4
> │   ├── AUTHORS
> │   ├── autom4te.cache
> │   ├── bootstrap
> │   ├── build-aux
> │   ├── ChangeLog
> │   ├── CODE-OF-CONDUCT
> │   ├── config-daemon.ac
> │   ├── config.log
> │   ├── config.status
> │   ├── configure
> │   ├── configure.ac
> │   ├── COPYING
> │   ├── doc
> │   ├── etc
> │   ├── gnu
> │   ├── gnu.scm
> │   ├── guix
> │   ├── guix.scm
> │   ├── HACKING
> │   ├── INSTALL
> │   ├── m4
> │   ├── Makefile
> │   ├── Makefile.am
> │   ├── Makefile.in
> │   ├── NEWS
> │   ├── nix
> │   ├── po
> │   ├── pre-inst-env
> │   ├── README
> │   ├── ROADMAP
> │   ├── scripts
> │   ├── test-env
> │   ├── tests
> │   ├── THANKS
> │   └── TODO
> └── guix-store
> ~~~~
>
> So I run:
>
> ~~~~
> ./configure --localstatedir=/home/user/dev/guix-store
> ~~~~
>
> Next is running:
>
> ~~~~
> make check
> ~~~~
>
> Some warnings appear:
>
> ~~~~
> Your input po file po/doc/guix-manual.de.po seems outdated (The amount of entries differ between files: 10012 is not 334
> ). Please consider running po4a-updatepo to refresh it.
> ~~~~
>
> But I am guessing, that these can be ignored for my purposes, as I am not doing
> anything with translations or po files.
>
> However, there are other errors, like not finding code for some modules (the
> output of make check is very long and probably I should rather put the log
> somewhere and link to it). The check seems to fail ultimately with the following
> result:
>
> ~~~~
> make[4]: *** [Makefile:5563: tests/store.log] Error 1
> make[4]: Leaving directory '/home/user/dev/guix'
> make[3]: *** [Makefile:5545: check-TESTS] Error 2
> make[3]: Leaving directory '/home/user/dev/guix'
> make[2]: *** [Makefile:5794: check-am] Error 2
> make[2]: Leaving directory '/home/user/dev/guix'
> make[1]: *** [Makefile:5322: check-recursive] Error 1
> make[1]: Leaving directory '/home/user/dev/guix'
> make: *** [Makefile:5796: check] Error 2
> ~~~~
>
> The only red colored fail I can see is above from that, first line of the following:
>
> ~~~~
> FAIL: tests/go.scm
> PASS: tests/grafts.scm
> PASS: tests/graph.scm
> PASS: tests/gremlin.scm
> SKIP: tests/hackage.scm
> PASS: tests/import-utils.scm
> PASS: tests/inferior.scm
> PASS: tests/lint.scm
> PASS: tests/modules.scm
> PASS: tests/monads.scm
> PASS: tests/nar.scm
> PASS: tests/networking.scm
> PASS: tests/opam.scm
> PASS: tests/openpgp.scm
> PASS: tests/packages.scm
> SKIP: tests/pack.scm
> PASS: tests/pki.scm
> PASS: tests/print.scm
> PASS: tests/processes.scm
> PASS: tests/profiles.scm
> SKIP: tests/publish.scm
> SKIP: tests/pypi.scm
> PASS: tests/records.scm
> PASS: tests/scripts.scm
> PASS: tests/search-paths.scm
> PASS: tests/services.scm
> PASS: tests/services/file-sharing.scm
> PASS: tests/services/linux.scm
> PASS: tests/sets.scm
> PASS: tests/size.scm
> SKIP: tests/snix.scm
> PASS: tests/status.scm
> PASS: tests/store-database.scm
> PASS: tests/store-deduplication.scm
> PASS: tests/store-roots.scm
> make[4]: *** [Makefile:5563: tests/store.log] Error 1
> make[4]: Leaving directory '/home/user/dev/guix'
> make[3]: *** [Makefile:5545: check-TESTS] Error 2
> make[3]: Leaving directory '/home/user/dev/guix'
> make[2]: *** [Makefile:5794: check-am] Error 2
> make[2]: Leaving directory '/home/user/dev/guix'
> make[1]: *** [Makefile:5322: check-recursive] Error 1
> make[1]: Leaving directory '/home/user/dev/guix'
> make: *** [Makefile:5796: check] Error 2
> ~~~~
>
> I am not sure, whether this means, that I have an erroneous Guix or whether the
> localstatedir was wrong or what else it means.

I get the same error with localstatedir=/var and using
`guix environment guix --pure`. I don't get this error when I drop the
--pure argument. Might be some environment variables that needs to be
setup for it to communicate properly with the guix daemon.

> However, I continue.
>
> The docs on https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> continue:
>
>> From there on, you can authenticate all the commits included in your checkout
> by running: make authenticate
>
> It is not clear to me, whether that means, that I need to run it in my scenario
> or whether I should not run it.
>
> A look into the Makefile shows, that it is actually calling a guix command `guix
> git authenticate`. The description is "verify commit signatures and
> authorizations". "verify" or "verify-signatures" seems a more appropriate name,
> because it indicates checking signatures, instead of authenticating commits.
> However, this might be my lacking understanding of what `make authenticate` does
> or aims to do.
>
> When I try running:
>
> ~~~~
> make authenticate
> ~~~~
>
> I get an error:
>
> ~~~~
> $ make authenticate 
> Authenticating Git checkout...
> /bin/bash: line 1: guix: command not found
> make: *** [Makefile:6290: authenticate] Error 127
> ~~~~
>
> I am confused as to why it does not find guix. I am inside the environment
> created by `guix environment guix --pure`.
>
> I continue nevertheless and try to run:
>
> ~~~~
> ./pre-inst-env guix build guile-fslib
> ~~~~
>
> However, my whole setup now seems wo be messed up. The `guix` command can simply
> no longer be found:
>
> ~~~~
> $ pwd
> /home/user/dev
> $ guix environment guix --pure
> $ pushd guix
> ~/dev/guix ~/dev
> $ ./pre-inst-env guix build guile-fslib
> guix build: error: failed to connect to `/home/user/dev/guix-store/guix/daemon-socket/socket': No such file or directory
> ~~~~
>
> A logout and login does not change the error.
>
> Where did I go wrong?

The issue, I believe, is that there is no guix-daemon running that is
listening on that store, hence the error about missing socket file for
it.

You can try in a different shell:
  guix environment guix --pure
  ./pre-inst-env guix-daemon

And then call `guix build` in the other shell and see if that works.

Some one correct me if I'm wrong. But just contributing a package, using
/var should be enough. Then you can use the build daemon that is already
running. As the manual points out:

"(this assumes ‘guix-daemon’ is already running on your system; it’s OK
if it’s a different version)" [0]

[0] https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html#Running-Guix-Before-It-Is-Installed

I hope that helps.

-- 
s/Fred[re]+i[ck]+/Fredrik/g

Re: Setting up guix source tree for testing a package

[Joshua Branson]

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> Hello Guix Users,
>
> I am trying to set up GNU Guix for package testing, as I want to test a package
> before sending in a patch to add it to GNU Guix.
>
> I'll describe the steps I am taking and where I am seeing them in the docs in
> the following.
>
> All is happening inside a VM, which runs a Debian 10. The VM is almost fresh,
> except for guest additions and git and vim and such things.

This sounds a little complicated...trying to develop things for guix
inside a virtual machine that's running Debian...I mean that sure makes
the testing REALLY potentially reproducible...But wouldn't it be
easier/simpler to do development for guix whilst using guix (or guix
system) on bare metal?

If you really prefer to develop for guix in a vm, you can download guix
virtual machine images here: http://guix.gnu.org/en/download/

Oh, having read the building from git section in the guix manual...they
do make it should like you should start from a virtual machine...What
operating system are you running on bare metal?  That's what I would use
to develop for guix.

>> As you can see, there is a chicken-and-egg problem: you first need to have
> Guix installed. Typically you would install Guix System (see System
> Installation) or Guix on top of another distro (see Binary Installation); in
> either case, you would verify the OpenPGP signature on the installation medium.
> This “bootstraps” the trust chain.

May I suggest that this chicken and egg problem exists because you chose
to use a debian vm?  This is not a criticism.  :)

> But here I am facing a problem. I have no clue what to set the
> `--localstatedir=directory` to. The docs tell me it is important, but I see no
> instructions on how to choose a directory for the local state, when I have
> already a Guix installed (binary installation on foreign distro). The linked
> docs about the https://guix.gnu.org/manual/en/html_node/The-Store.html
> <https://guix.gnu.org/manual/en/html_node/The-Store.html> do not help me
> choosing it either, because that page is about the store itself, not about my
> specific scenario of doing things from Guix source tree. Since I intend to build
> a package inside the Guix source tree, I think it should exactly _not_ be /var,
> as that would conflict with the Guix installation, that I had to do to create
> the pre-inst-env script.

I'm not certain how to answer this.  Since I develop for GNU Guix on
guix system, I always run "./configure --localstatedir=/var".  This
works for me!

>
> What is the idea for the local state directory? Is it the location of a
> temporary store, which will contain anything, that I produce using the
> pre-inst-env script? That is what I am guessing. In that case I think
> `localstoredir` might be a good alias for `localstatedir`.

I think localstatedir is the location of guix specific things?  My
/var/guix contains gcroots, profiles, substitute, and daemon-socket
directories.

I applaud your attention to detail and trying to submit a "perfect"
patch to GNU guix.  :) That really does help the project have good
standards.  Trust me not all of the guix patches are perfect.  Somehow
some of mine were merged.  :)

>
> When I try running:
>
> ~~~~
> make authenticate
> ~~~~
>
> I get an error:
>
> ~~~~
> $ make authenticate
> Authenticating Git checkout...
> /bin/bash: line 1: guix: command not found
> make: *** [Makefile:6290: authenticate] Error 127
> ~~~~

I've run into this issue before too...hmmm.  How did I fix it?  I think
that you need to download Ludo's gpg keyring...

https://wiki.archlinux.org/index.php/GnuPG#Import_a_public_key

gpg --import public.key

Then you should be able to run:

     git fetch origin keyring:keyring
     guix git authenticate 9edb3f66fd807b096b48283debdcddccfea34bad \
       "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"

> Where did I go wrong?
> Regards,
> Zelphir

I think you just tried to start developing in a "perfect" but difficult
way.  :)  If you already use GNU/Linux as your main OS, install guix on
it, and develop from that OS.  If you are running guix system, develop
from that.  Best of luck!

P.S.  You might find my online videos where I try to contribute to guix
helpful too:
https://video.hardlimit.com/accounts/joshua_branson/video-channels They
may be a little boring, and perhaps not the best way to develop, but the
videos show me developing in some kind of haphazard way.  My newer
videos involving endlessh are probably close to the "correct" way.

--
Joshua Branson (joshuaBPMan in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar

Re: Setting up guix source tree for testing a package

[Leo Famulari]

On Fri, Mar 12, 2021 at 12:31:36AM +0100, Zelphir Kaltstahl wrote:
> ~~~~
> ./bootstrap
> ./configure --localstatedir=some_directory
> ~~~~
> 
> But here I am facing a problem. I have no clue what to set the
> `--localstatedir=directory` to. The docs tell me it is important, but I see no
> instructions on how to choose a directory for the local state, when I have
> already a Guix installed (binary installation on foreign distro).

The manual section Building From Git says this:

"Then, run ./configure as usual. Make sure to pass
--localstatedir=directory where directory is the localstatedir value
used by your current installation (see The Store, for information about
this), usually /var."

Did you try using '/var'?

Re: Setting up guix source tree for testing a package

[Zelphir Kaltstahl]

Hello Frederik!

On 3/12/21 2:01 AM, Fredrik Salomonsson wrote:
> Hi Zelphir,
>
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> Hello Guix Users,
>>
>> I am trying to set up GNU Guix for package testing, as I want to test a package
>> before sending in a patch to add it to GNU Guix.
>>
>> I'll describe the steps I am taking and where I am seeing them in the docs in
>> the following.
>>
>> All is happening inside a VM, which runs a Debian 10. The VM is almost fresh,
>> except for guest additions and git and vim and such things.
>>
>>
>> I start at https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html
>> <https://guix.gnu.org/manual/en/html_node/Packaging-Guidelines.html>, wherein I
>> find a link to https://guix.gnu.org/manual/en/html_node/Defining-Packages.html
>> <https://guix.gnu.org/manual/en/html_node/Defining-Packages.html>.
>>
>> Next the packaging guidelines tell me, that I may run for example:
>>
>> ~~~~
>> ./pre-inst-env guix build gnew --keep-failed
>> ~~~~
>>
>> inside the Guix build tree.
>>
>> I clone Guix, so that I have a source tree of Guix.
>>
>> I define my package inside `guix/gnu/packages/guile.scm`.
>>
>> In packaging guidelines, there is a link to
>> https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
>> <https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>.
>> I am supposed to run the test inside a fresh Guix, to make sure, that the
>> package will work fine for others, if I understand correctly.
>>
>> On
>> https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html
>> <https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html>
>> I am told, that I need to generate a pre-installation script to run guix without
>> installing it (running stuff in the Guix build tree, as far as I understand). To
>> generate it, I need Guix again. The docs direct me to
>> https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
>> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html>.
>>
>> On https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
>> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> the dilemma is
>> finally uncovered, that I need Guix to build the pre-installation script, to use
>> that, to build a package in the Guix source tree.
>>
>> As you can see, there is a chicken-and-egg problem: you first need to have
>> Guix installed. Typically you would install Guix System (see System
>> Installation) or Guix on top of another distro (see Binary Installation); in
>> either case, you would verify the OpenPGP signature on the installation medium.
>> This “bootstraps” the trust chain.
>>
>> I install it.
> You don't need guix to build guix, it just makes it a lot easier to
> setup the environment to build guix as you don't need to hunt for its
> dependencies yourself.
>
>> Then I run:
>>
>> ~~~~
>> guix environment guix --pure
>> ~~~~
>>
>> The docs tell me I have to run:
>>
>> ~~~~
>> ./bootstrap
>> ./configure --localstatedir=some_directory
>> ~~~~
>>
>> But here I am facing a problem. I have no clue what to set the
>> `--localstatedir=directory` to. The docs tell me it is important, but I see no
>> instructions on how to choose a directory for the local state, when I have
>> already a Guix installed (binary installation on foreign distro). The linked
>> docs about the https://guix.gnu.org/manual/en/html_node/The-Store.html
>> <https://guix.gnu.org/manual/en/html_node/The-Store.html> do not help me
>> choosing it either, because that page is about the store itself, not about my
>> specific scenario of doing things from Guix source tree. Since I intend to build
>> a package inside the Guix source tree, I think it should exactly _not_ be /var,
>> as that would conflict with the Guix installation, that I had to do to create
>> the pre-inst-env script.
>>
>> What is the idea for the local state directory? Is it the location of a
>> temporary store, which will contain anything, that I produce using the
>> pre-inst-env script? That is what I am guessing. In that case I think
>> `localstoredir` might be a good alias for `localstatedir`.
>>
>>
>> I am trying with an empty directory, which I create just for that purpose, on
>> the same level as the cloned Guix repository:
>>
>> ~~~~
>> $ tree -L 2
>> .
>> ├── guix
>> │   ├── ABOUT-NLS
>> │   ├── aclocal.m4
>> │   ├── AUTHORS
>> │   ├── autom4te.cache
>> │   ├── bootstrap
>> │   ├── build-aux
>> │   ├── ChangeLog
>> │   ├── CODE-OF-CONDUCT
>> │   ├── config-daemon.ac
>> │   ├── config.log
>> │   ├── config.status
>> │   ├── configure
>> │   ├── configure.ac
>> │   ├── COPYING
>> │   ├── doc
>> │   ├── etc
>> │   ├── gnu
>> │   ├── gnu.scm
>> │   ├── guix
>> │   ├── guix.scm
>> │   ├── HACKING
>> │   ├── INSTALL
>> │   ├── m4
>> │   ├── Makefile
>> │   ├── Makefile.am
>> │   ├── Makefile.in
>> │   ├── NEWS
>> │   ├── nix
>> │   ├── po
>> │   ├── pre-inst-env
>> │   ├── README
>> │   ├── ROADMAP
>> │   ├── scripts
>> │   ├── test-env
>> │   ├── tests
>> │   ├── THANKS
>> │   └── TODO
>> └── guix-store
>> ~~~~
>>
>> So I run:
>>
>> ~~~~
>> ./configure --localstatedir=/home/user/dev/guix-store
>> ~~~~
>>
>> Next is running:
>>
>> ~~~~
>> make check
>> ~~~~
>>
>> Some warnings appear:
>>
>> ~~~~
>> Your input po file po/doc/guix-manual.de.po seems outdated (The amount of entries differ between files: 10012 is not 334
>> ). Please consider running po4a-updatepo to refresh it.
>> ~~~~
>>
>> But I am guessing, that these can be ignored for my purposes, as I am not doing
>> anything with translations or po files.
>>
>> However, there are other errors, like not finding code for some modules (the
>> output of make check is very long and probably I should rather put the log
>> somewhere and link to it). The check seems to fail ultimately with the following
>> result:
>>
>> ~~~~
>> make[4]: *** [Makefile:5563: tests/store.log] Error 1
>> make[4]: Leaving directory '/home/user/dev/guix'
>> make[3]: *** [Makefile:5545: check-TESTS] Error 2
>> make[3]: Leaving directory '/home/user/dev/guix'
>> make[2]: *** [Makefile:5794: check-am] Error 2
>> make[2]: Leaving directory '/home/user/dev/guix'
>> make[1]: *** [Makefile:5322: check-recursive] Error 1
>> make[1]: Leaving directory '/home/user/dev/guix'
>> make: *** [Makefile:5796: check] Error 2
>> ~~~~
>>
>> The only red colored fail I can see is above from that, first line of the following:
>>
>> ~~~~
>> FAIL: tests/go.scm
>> PASS: tests/grafts.scm
>> PASS: tests/graph.scm
>> PASS: tests/gremlin.scm
>> SKIP: tests/hackage.scm
>> PASS: tests/import-utils.scm
>> PASS: tests/inferior.scm
>> PASS: tests/lint.scm
>> PASS: tests/modules.scm
>> PASS: tests/monads.scm
>> PASS: tests/nar.scm
>> PASS: tests/networking.scm
>> PASS: tests/opam.scm
>> PASS: tests/openpgp.scm
>> PASS: tests/packages.scm
>> SKIP: tests/pack.scm
>> PASS: tests/pki.scm
>> PASS: tests/print.scm
>> PASS: tests/processes.scm
>> PASS: tests/profiles.scm
>> SKIP: tests/publish.scm
>> SKIP: tests/pypi.scm
>> PASS: tests/records.scm
>> PASS: tests/scripts.scm
>> PASS: tests/search-paths.scm
>> PASS: tests/services.scm
>> PASS: tests/services/file-sharing.scm
>> PASS: tests/services/linux.scm
>> PASS: tests/sets.scm
>> PASS: tests/size.scm
>> SKIP: tests/snix.scm
>> PASS: tests/status.scm
>> PASS: tests/store-database.scm
>> PASS: tests/store-deduplication.scm
>> PASS: tests/store-roots.scm
>> make[4]: *** [Makefile:5563: tests/store.log] Error 1
>> make[4]: Leaving directory '/home/user/dev/guix'
>> make[3]: *** [Makefile:5545: check-TESTS] Error 2
>> make[3]: Leaving directory '/home/user/dev/guix'
>> make[2]: *** [Makefile:5794: check-am] Error 2
>> make[2]: Leaving directory '/home/user/dev/guix'
>> make[1]: *** [Makefile:5322: check-recursive] Error 1
>> make[1]: Leaving directory '/home/user/dev/guix'
>> make: *** [Makefile:5796: check] Error 2
>> ~~~~
>>
>> I am not sure, whether this means, that I have an erroneous Guix or whether the
>> localstatedir was wrong or what else it means.
> I get the same error with localstatedir=/var and using
> `guix environment guix --pure`. I don't get this error when I drop the
> --pure argument. Might be some environment variables that needs to be
> setup for it to communicate properly with the guix daemon.
>
>> However, I continue.
>>
>> The docs on https://guix.gnu.org/manual/en/html_node/Building-from-Git.html
>> <https://guix.gnu.org/manual/en/html_node/Building-from-Git.html> continue:
>>
>>> From there on, you can authenticate all the commits included in your checkout
>> by running: make authenticate
>>
>> It is not clear to me, whether that means, that I need to run it in my scenario
>> or whether I should not run it.
>>
>> A look into the Makefile shows, that it is actually calling a guix command `guix
>> git authenticate`. The description is "verify commit signatures and
>> authorizations". "verify" or "verify-signatures" seems a more appropriate name,
>> because it indicates checking signatures, instead of authenticating commits.
>> However, this might be my lacking understanding of what `make authenticate` does
>> or aims to do.
>>
>> When I try running:
>>
>> ~~~~
>> make authenticate
>> ~~~~
>>
>> I get an error:
>>
>> ~~~~
>> $ make authenticate 
>> Authenticating Git checkout...
>> /bin/bash: line 1: guix: command not found
>> make: *** [Makefile:6290: authenticate] Error 127
>> ~~~~
>>
>> I am confused as to why it does not find guix. I am inside the environment
>> created by `guix environment guix --pure`.
>>
>> I continue nevertheless and try to run:
>>
>> ~~~~
>> ./pre-inst-env guix build guile-fslib
>> ~~~~
>>
>> However, my whole setup now seems wo be messed up. The `guix` command can simply
>> no longer be found:
>>
>> ~~~~
>> $ pwd
>> /home/user/dev
>> $ guix environment guix --pure
>> $ pushd guix
>> ~/dev/guix ~/dev
>> $ ./pre-inst-env guix build guile-fslib
>> guix build: error: failed to connect to `/home/user/dev/guix-store/guix/daemon-socket/socket': No such file or directory
>> ~~~~
>>
>> A logout and login does not change the error.
>>
>> Where did I go wrong?
> The issue, I believe, is that there is no guix-daemon running that is
> listening on that store, hence the error about missing socket file for
> it.
>
> You can try in a different shell:
>   guix environment guix --pure
>   ./pre-inst-env guix-daemon
>
> And then call `guix build` in the other shell and see if that works.
>
> Some one correct me if I'm wrong. But just contributing a package, using
> /var should be enough. Then you can use the build daemon that is already
> running. As the manual points out:
>
> "(this assumes ‘guix-daemon’ is already running on your system; it’s OK
> if it’s a different version)" [0]
>
> [0] https://guix.gnu.org/manual/en/html_node/Running-Guix-Before-It-Is-Installed.html#Running-Guix-Before-It-Is-Installed
>
> I hope that helps.
>
This definitely is helpful!

I'm sorry for the delay in responding. Thank you for your efforts!

I hit other errors when leaving away the --pure and I have the suspicion, that
gnu-build-system requires me to have a configure script in the git tree of my
repository, which I did not know before I saw it failing with a 127 error on
calling a configure script, which is/was not commited in my repository. I will
probably have to revise a section somewhere in my guile-hall packaging guide, to
note, that one should add the configure script, even though it is generated by hall.

Best regards,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Setting up guix source tree for testing a package

[Zelphir Kaltstahl]

Hi Joshua!

On 3/12/21 8:59 PM, Joshua Branson wrote:
> Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:
>
>> Hello Guix Users,
>>
>> I am trying to set up GNU Guix for package testing, as I want to test a package
>> before sending in a patch to add it to GNU Guix.
>>
>> I'll describe the steps I am taking and where I am seeing them in the docs in
>> the following.
>>
>> All is happening inside a VM, which runs a Debian 10. The VM is almost fresh,
>> except for guest additions and git and vim and such things.
> This sounds a little complicated...trying to develop things for guix
> inside a virtual machine that's running Debian...I mean that sure makes
> the testing REALLY potentially reproducible...But wouldn't it be
> easier/simpler to do development for guix whilst using guix (or guix
> system) on bare metal?
>
> If you really prefer to develop for guix in a vm, you can download guix
> virtual machine images here: http://guix.gnu.org/en/download/
>
> Oh, having read the building from git section in the guix manual...they
> do make it should like you should start from a virtual machine...What
> operating system are you running on bare metal?  That's what I would use
> to develop for guix.
>
>>> As you can see, there is a chicken-and-egg problem: you first need to have
>> Guix installed. Typically you would install Guix System (see System
>> Installation) or Guix on top of another distro (see Binary Installation); in
>> either case, you would verify the OpenPGP signature on the installation medium.
>> This “bootstraps” the trust chain.
> May I suggest that this chicken and egg problem exists because you chose
> to use a debian vm?  This is not a criticism.  :)
>
>> But here I am facing a problem. I have no clue what to set the
>> `--localstatedir=directory` to. The docs tell me it is important, but I see no
>> instructions on how to choose a directory for the local state, when I have
>> already a Guix installed (binary installation on foreign distro). The linked
>> docs about the https://guix.gnu.org/manual/en/html_node/The-Store.html
>> <https://guix.gnu.org/manual/en/html_node/The-Store.html> do not help me
>> choosing it either, because that page is about the store itself, not about my
>> specific scenario of doing things from Guix source tree. Since I intend to build
>> a package inside the Guix source tree, I think it should exactly _not_ be /var,
>> as that would conflict with the Guix installation, that I had to do to create
>> the pre-inst-env script.
> I'm not certain how to answer this.  Since I develop for GNU Guix on
> guix system, I always run "./configure --localstatedir=/var".  This
> works for me!
>
>> What is the idea for the local state directory? Is it the location of a
>> temporary store, which will contain anything, that I produce using the
>> pre-inst-env script? That is what I am guessing. In that case I think
>> `localstoredir` might be a good alias for `localstatedir`.
> I think localstatedir is the location of guix specific things?  My
> /var/guix contains gcroots, profiles, substitute, and daemon-socket
> directories.
>
> I applaud your attention to detail and trying to submit a "perfect"
> patch to GNU guix.  :) That really does help the project have good
> standards.  Trust me not all of the guix patches are perfect.  Somehow
> some of mine were merged.  :)
>
>> When I try running:
>>
>> ~~~~
>> make authenticate
>> ~~~~
>>
>> I get an error:
>>
>> ~~~~
>> $ make authenticate
>> Authenticating Git checkout...
>> /bin/bash: line 1: guix: command not found
>> make: *** [Makefile:6290: authenticate] Error 127
>> ~~~~
> I've run into this issue before too...hmmm.  How did I fix it?  I think
> that you need to download Ludo's gpg keyring...
>
> https://wiki.archlinux.org/index.php/GnuPG#Import_a_public_key
>
> gpg --import public.key
>
> Then you should be able to run:
>
>      git fetch origin keyring:keyring
>      guix git authenticate 9edb3f66fd807b096b48283debdcddccfea34bad \
>        "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"
>
>> Where did I go wrong?
>> Regards,
>> Zelphir
> I think you just tried to start developing in a "perfect" but difficult
> way.  :)  If you already use GNU/Linux as your main OS, install guix on
> it, and develop from that OS.  If you are running guix system, develop
> from that.  Best of luck!
>
> P.S.  You might find my online videos where I try to contribute to guix
> helpful too:
> https://video.hardlimit.com/accounts/joshua_branson/video-channels They
> may be a little boring, and perhaps not the best way to develop, but the
> videos show me developing in some kind of haphazard way.  My newer
> videos involving endlessh are probably close to the "correct" way.
>
> --
> Joshua Branson (joshuaBPMan in #guix)
> Sent from Emacs and Gnus
>   https://gnucode.me
>   https://video.hardlimit.com/accounts/joshua_branson/video-channels
>   https://propernaming.org
>   "You can have whatever you want, as long as you help
> enough other people get what they want." - Zig Ziglar

Thanks Joshua! I'll have a look at your resources once I get to it ; )

(I think I added them to the guile awesome list already, just did not yet watch
them all.)

Best wishes,
Zelphir

-- 
repositories: https://notabug.org/ZelphirKaltstahl

Re: Setting up guix source tree for testing a package

[Fredrik Salomonsson]

Hi Zelphir,

Zelphir Kaltstahl <zelphirkaltstahl@posteo.de> writes:

> This definitely is helpful!

Glad I could be of some help :).

> I'm sorry for the delay in responding. Thank you for your efforts!

No worries.

> I hit other errors when leaving away the --pure and I have the suspicion, that
> gnu-build-system requires me to have a configure script in the git tree of my
> repository, which I did not know before I saw it failing with a 127 error on
> calling a configure script, which is/was not commited in my repository. I will
> probably have to revise a section somewhere in my guile-hall packaging guide, to
> note, that one should add the configure script, even though it is generated by hall.

Hmm, the bootstrap phase in gnu-build-system should invoke
`autoreconf -vif` if the package contains either a `configure.ac` or
`configure.in` and does not contain a `bootstrap`, `bootstrap.sh` or
`autogen.sh`. At least that what I get from reading the source:
gnu-build-system.scm:203 [0]

[0] https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build/gnu-build-system.scm

I also ran it on one of my projects that are using guile-hall, which
does not have a configure script and it build it just fine.

Maybe you have one of those bootstrap scripts that does not generate a
configure script?

-- 
s/Fred[re]+i[ck]+/Fredrik/g