unofficial mirror of guix-devel@gnu.org 
 help / color / mirror / code / Atom feed
* Question about compile packages
@ 2021-03-29 19:05 Charles Direg
  2021-03-30 11:12 ` Julien Lepiller
  0 siblings, 1 reply; 5+ messages in thread
From: Charles Direg @ 2021-03-29 19:05 UTC (permalink / raw)
  To: guix-devel

[-- Attachment #1: Type: text/plain, Size: 772 bytes --]

 Dear,

How can I modify the flags that any program is compiled with within guix? That
is, I can allow in the gnu-build-system to modify it globally so that I can
add the build flags to any package, for example, add the flags -O2
-march=native -mtune=native so global as I already mentioned, so that these
are added to each package at the time of compilation, this would not be
within the guix development environment, because what I want is that this
compilation is natively for my pc.

As a second question, how could I set the --no-susbtitutes option when
installing the guix system from ISO, since I would like all installed
packages to be compiled natively first?

I really appreciate your kind time and I look forward to your responses.

Sincerely,
~ Abraham Huerta

[-- Attachment #2: Type: text/html, Size: 2421 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Question about compile packages
  2021-03-29 19:05 Question about compile packages Charles Direg
@ 2021-03-30 11:12 ` Julien Lepiller
  2021-03-30 16:12   ` raingloom
  2021-03-31  0:28   ` Mark H Weaver
  0 siblings, 2 replies; 5+ messages in thread
From: Julien Lepiller @ 2021-03-30 11:12 UTC (permalink / raw)
  To: guix-devel, Charles Direg

[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]

Not sure about your first question. Maybe create your own fork and regularly rebase it? Guix will not be very happy with that I think, but it should work.

To pass --no-substitutes, you can pass it to guix-daemon, or remove the authorized keys from /etc/guix/acl. The second option lets you download substitutes for fixed output derivations (basically source tarballs), so it's easier in case some of those tarballs disappear from their original location.

I think the first option is selectable from the installer, but I might be wrong. Otherwise, stop guix-daemon from herd and run it manually with your options.

Le 29 mars 2021 15:05:08 GMT-04:00, Charles Direg <carloshujara@gmail.com> a écrit :
> Dear,
>
>How can I modify the flags that any program is compiled with within
>guix? That
>is, I can allow in the gnu-build-system to modify it globally so that I
>can
>add the build flags to any package, for example, add the flags -O2
>-march=native -mtune=native so global as I already mentioned, so that
>these
>are added to each package at the time of compilation, this would not be
>within the guix development environment, because what I want is that
>this
>compilation is natively for my pc.
>
>As a second question, how could I set the --no-susbtitutes option when
>installing the guix system from ISO, since I would like all installed
>packages to be compiled natively first?
>
>I really appreciate your kind time and I look forward to your
>responses.
>
>Sincerely,
>~ Abraham Huerta

[-- Attachment #2: Type: text/html, Size: 3360 bytes --]

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Question about compile packages
  2021-03-30 11:12 ` Julien Lepiller
@ 2021-03-30 16:12   ` raingloom
  2021-03-31  0:28   ` Mark H Weaver
  1 sibling, 0 replies; 5+ messages in thread
From: raingloom @ 2021-03-30 16:12 UTC (permalink / raw)
  To: Julien Lepiller; +Cc: guix-devel, Charles Direg

On Tue, 30 Mar 2021 07:12:27 -0400
Julien Lepiller <julien@lepiller.eu> wrote:

> Not sure about your first question. Maybe create your own fork and
> regularly rebase it? Guix will not be very happy with that I think,
> but it should work.
> 
> To pass --no-substitutes, you can pass it to guix-daemon, or remove
> the authorized keys from /etc/guix/acl. The second option lets you
> download substitutes for fixed output derivations (basically source
> tarballs), so it's easier in case some of those tarballs disappear
> from their original location.
> 
> I think the first option is selectable from the installer, but I
> might be wrong. Otherwise, stop guix-daemon from herd and run it
> manually with your options.
> 
> Le 29 mars 2021 15:05:08 GMT-04:00, Charles Direg
> <carloshujara@gmail.com> a écrit :
> > Dear,
> >
> >How can I modify the flags that any program is compiled with within
> >guix? That
> >is, I can allow in the gnu-build-system to modify it globally so
> >that I can
> >add the build flags to any package, for example, add the flags -O2
> >-march=native -mtune=native so global as I already mentioned, so that
> >these
> >are added to each package at the time of compilation, this would not
> >be within the guix development environment, because what I want is
> >that this
> >compilation is natively for my pc.
> >
> >As a second question, how could I set the --no-susbtitutes option
> >when installing the guix system from ISO, since I would like all
> >installed packages to be compiled natively first?
> >
> >I really appreciate your kind time and I look forward to your
> >responses.
> >
> >Sincerely,
> >~ Abraham Huerta  

I'm mostly guessing here, but wrapping gcc with a script that sets the
options you want might work. Could try creating a gcc-toolchain that
is set up the way you want it (don't even have to recompile, like i
said, just create a wrapper script that calls the original toolchain)
and then run guix build --with-toolchain=gcc-impure-native-toolchain

At least that's how I'd get started.

You could also just modify the toolchain package, but then you'd have
to recompile everything.

Anyways, if you want -march=native optimisations, there is some recent
discussion around some new GCC feature that detects CPU features at
runtime. The high performance computing (HPC) related developers are
very interested in using it in Guix.


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Question about compile packages
  2021-03-30 11:12 ` Julien Lepiller
  2021-03-30 16:12   ` raingloom
@ 2021-03-31  0:28   ` Mark H Weaver
  2021-03-31 17:36     ` Vincent Legoll
  1 sibling, 1 reply; 5+ messages in thread
From: Mark H Weaver @ 2021-03-31  0:28 UTC (permalink / raw)
  To: Julien Lepiller, guix-devel, Charles Direg

Hi,

Charles Direg <carloshujara@gmail.com> asked:

> How can I modify the flags that any program is compiled with within guix? That
> is, I can allow in the gnu-build-system to modify it globally so that I can
> add the build flags to any package, for example, add the flags -O2
> -march=native -mtune=native so global as I already mentioned, so that these
> are added to each package at the time of compilation, this would not be
> within the guix development environment, because what I want is that this
> compilation is natively for my pc.

Julien Lepiller <julien@lepiller.eu> replied:

> Not sure about your first question.  Maybe create your own fork and
> regularly rebase it?  Guix will not be very happy with that I think,
> but it should work.

I've been doing this all along, since the early days of Guix.  No one
seems to be unhappy with me about it, and I'm not sure why they would.

I build everything locally, never use substitutes, and never use "guix
pull".  I always run guix via ./pre-inst-env from a git checkout of my
personal private branch of Guix.  I use a little script in ~/bin/guix
that lets me simply type "guix ..." like everyone else:

--8<---------------cut here---------------start------------->8---
#!/bin/sh

source /var/guix/gcroots/per-user/mhw/environment-guix/etc/profile
exec /home/mhw/guix/pre-inst-env guix "$@"
--8<---------------cut here---------------end--------------->8---

I periodically rebase my private branch onto 'master', because I want to
keep my private modifications in the form of a small number of patches,
but if I didn't care about that, periodically merging master into my
branch would also work.

This approach gives *enormous* flexibility -- you can easily change just
about any aspect of Guix, while still benefitting from our upstream
work, and with nearly automatic merges thanks to git -- but there are
some rough edges.

One is that if you're not careful to keep a suitable build environment
for building Guix, and your git checkout gets into a broken state, you
can get stuck.  Also, sometimes Guix adds additional build requirements,
and so you can get into a state where you've updated your git checkout
to a version with new requirements, but you cannot build those new
requirements because your git checkout (from which you run guix) is not
yet built.

These problems are manageable, although it's slightly tedious.  First,
I'm very careful to always keep a GC root to a known working build
environment: a profile created by "guix environment guix", as referenced
by $GUIX_ENVIRONMENT, which I make the target of a symlink
/var/guix/gcroots/per-user/mhw/environment-guix.  When I update this GC
root, I always keep the older GC root around until I'm absolutely
certain that the new one works.

With this GC root in hand, I can always load this build environment,
without running guix, by simply sourcing its etc/profile file (using the
first command of the script I quoted above).

If Guix adds a new requirement, and I fail to notice before updating my
git checkout, I can always roll back to an earlier checkout, rebuild it,
and then use that to run "guix environment guix --ad-hoc
<NEW-REQUIREMENTS>".

Also, beware that when you build everything locally, which takes a
*long* time (a couple of weeks of continuous building for a GNOME system
on a Thinkpad X200), it's important to pass "--gc-keep-derivations=yes"
and "--gc-keep-outputs=yes" to the Guix daemon.

I have something close to this in the 'services' field of my OS config:

  (modify-services %desktop-services
    (guix-service-type config =>
                       (guix-configuration
                         (inherit config)
                         (use-substitutes? #f)
                         (authorize-key?   #f)
                         (authorized-keys '())
                         (substitute-urls '())
                         (extra-options '("--gc-keep-derivations=yes"
                                          "--gc-keep-outputs=yes")))))

I can say from long experience that with the methods outlined here, you
can make arbitrary local changes to your Guix that affects *every*
package built in Guix.  I've gone through long periods of running Guix
with additional patches deep in the early bootstrap.

If more people are interested in using Guix in this way, the experience
could probably be made much nicer.

* * *

In theory, the "--url", "--branch", and "--commit" to "guix pull" should
allow this level of flexibility without the gotchas listed above, at
least for those who are willing to put their branch of Guix on a public
Git repo.

However, I'm not sure that "guix pull" will actually work correctly if
you make "rebuild-the-world" changes like the ones that Charles is
asking about.  I suspect this because of the bug I reported here:
<https://bugs.gnu.org/38203>.  Since I seem to be the only one using
Guix in this way, the bug report didn't get attention.  That bug was
about "make as-derivation", but I guess it's likely that "guix pull"
would be affected as well.

        Mark


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Question about compile packages
  2021-03-31  0:28   ` Mark H Weaver
@ 2021-03-31 17:36     ` Vincent Legoll
  0 siblings, 0 replies; 5+ messages in thread
From: Vincent Legoll @ 2021-03-31 17:36 UTC (permalink / raw)
  To: Mark H Weaver; +Cc: guix-devel, Charles Direg

Hello Mark,

On Wed, Mar 31, 2021 at 2:30 AM Mark H Weaver <mhw@netris.org> wrote:
> If more people are interested in using Guix in this way, the experience
> could probably be made much nicer.

I think this would interest quite a few people, a guixtoo cookbook chapter ?

I've wondered about such subject myself since my early tries with guix,
and just today, I got to know it is doable, and there are people just doing
it, and some hints about what & how to try it myself...

So thank you, and Raingloom.

-- 
Vincent Legoll


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2021-03-31 17:37 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-29 19:05 Question about compile packages Charles Direg
2021-03-30 11:12 ` Julien Lepiller
2021-03-30 16:12   ` raingloom
2021-03-31  0:28   ` Mark H Weaver
2021-03-31 17:36     ` Vincent Legoll

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/guix.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).