Re: Question about compile packages

[Mark H Weaver <mhw@netris.org>]

Hi,

Charles Direg <carloshujara@gmail.com> asked:

> How can I modify the flags that any program is compiled with within guix? That
> is, I can allow in the gnu-build-system to modify it globally so that I can
> add the build flags to any package, for example, add the flags -O2
> -march=native -mtune=native so global as I already mentioned, so that these
> are added to each package at the time of compilation, this would not be
> within the guix development environment, because what I want is that this
> compilation is natively for my pc.

Julien Lepiller <julien@lepiller.eu> replied:

> Not sure about your first question.  Maybe create your own fork and
> regularly rebase it?  Guix will not be very happy with that I think,
> but it should work.

I've been doing this all along, since the early days of Guix.  No one
seems to be unhappy with me about it, and I'm not sure why they would.

I build everything locally, never use substitutes, and never use "guix
pull".  I always run guix via ./pre-inst-env from a git checkout of my
personal private branch of Guix.  I use a little script in ~/bin/guix
that lets me simply type "guix ..." like everyone else:

--8<---------------cut here---------------start------------->8---
#!/bin/sh

source /var/guix/gcroots/per-user/mhw/environment-guix/etc/profile
exec /home/mhw/guix/pre-inst-env guix "$@"
--8<---------------cut here---------------end--------------->8---

I periodically rebase my private branch onto 'master', because I want to
keep my private modifications in the form of a small number of patches,
but if I didn't care about that, periodically merging master into my
branch would also work.

This approach gives *enormous* flexibility -- you can easily change just
about any aspect of Guix, while still benefitting from our upstream
work, and with nearly automatic merges thanks to git -- but there are
some rough edges.

One is that if you're not careful to keep a suitable build environment
for building Guix, and your git checkout gets into a broken state, you
can get stuck.  Also, sometimes Guix adds additional build requirements,
and so you can get into a state where you've updated your git checkout
to a version with new requirements, but you cannot build those new
requirements because your git checkout (from which you run guix) is not
yet built.

These problems are manageable, although it's slightly tedious.  First,
I'm very careful to always keep a GC root to a known working build
environment: a profile created by "guix environment guix", as referenced
by $GUIX_ENVIRONMENT, which I make the target of a symlink
/var/guix/gcroots/per-user/mhw/environment-guix.  When I update this GC
root, I always keep the older GC root around until I'm absolutely
certain that the new one works.

With this GC root in hand, I can always load this build environment,
without running guix, by simply sourcing its etc/profile file (using the
first command of the script I quoted above).

If Guix adds a new requirement, and I fail to notice before updating my
git checkout, I can always roll back to an earlier checkout, rebuild it,
and then use that to run "guix environment guix --ad-hoc
<NEW-REQUIREMENTS>".

Also, beware that when you build everything locally, which takes a
*long* time (a couple of weeks of continuous building for a GNOME system
on a Thinkpad X200), it's important to pass "--gc-keep-derivations=yes"
and "--gc-keep-outputs=yes" to the Guix daemon.

I have something close to this in the 'services' field of my OS config:

  (modify-services %desktop-services
    (guix-service-type config =>
                       (guix-configuration
                         (inherit config)
                         (use-substitutes? #f)
                         (authorize-key?   #f)
                         (authorized-keys '())
                         (substitute-urls '())
                         (extra-options '("--gc-keep-derivations=yes"
                                          "--gc-keep-outputs=yes")))))

I can say from long experience that with the methods outlined here, you
can make arbitrary local changes to your Guix that affects *every*
package built in Guix.  I've gone through long periods of running Guix
with additional patches deep in the early bootstrap.

If more people are interested in using Guix in this way, the experience
could probably be made much nicer.

* * *

In theory, the "--url", "--branch", and "--commit" to "guix pull" should
allow this level of flexibility without the gotchas listed above, at
least for those who are willing to put their branch of Guix on a public
Git repo.

However, I'm not sure that "guix pull" will actually work correctly if
you make "rebuild-the-world" changes like the ones that Charles is
asking about.  I suspect this because of the bug I reported here:
<https://bugs.gnu.org/38203>.  Since I seem to be the only one using
Guix in this way, the bug report didn't get attention.  That bug was
about "make as-derivation", but I guess it's likely that "guix pull"
would be affected as well.

        Mark