bug#55657: libgccjit is unusable

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Hi Guix,

with the release of Emacs 28.1, there has been some demand to enable
native-compilation.  While trying to set that up, I've come to realize
that no matter how I slice it, I can't make libgccjit usable.

My test:

$ cd /tmp
$ # fetch the "Hello World" [1] and insert it into gccjit-test.c
$ guix shell --pure gcc-toolchain@9 libgccjit@9 -- \
  gcc gccjit-test.c -o gccjit-test -lgccjit
$ ./gccjit-test
x86_64-unknown-linux-gnu-gcc-9.4.0: fatal error: cannot execute 'as':
execvp: No such file or directory
compilation terminated.

Welp, okay, so maybe testing gccjit outside of its installation does
not work.  What if we try it inside the shell (we can always propagate
libgccjit, no?)

guix shell gcc-toolchain@9 libgccjit@9 -- ./gccjit-test            
ld: cannot find crtbeginS.o: Datei oder Verzeichnis nicht gefunden
ld: cannot find -lgcc
ld: cannot find -lgcc
collect2: Fehler: ld gab 1 als Ende-Status zurück
libgccjit.so: error: error invoking gcc driver
NULL result

For the record, Emacs 28 fails with a different error during
configuration, though interestingly they use a smaller program and
appear to omit -lgccjit.
./conftest: error while loading shared libraries: libgccjit.so.0:
cannot open shared object file: No such file or directory

I'm at a loss.  Is there any way to make libgccjit actually usable?

[1] https://gcc.gnu.org/onlinedocs/jit/intro/tutorial01.html

bug#55657: libgccjit is unusable

[Remco van 't Veer]

2022/05/26 15:07, Liliana Marie Prikler:

> Hi Guix,
>
> with the release of Emacs 28.1, there has been some demand to enable
> native-compilation.  While trying to set that up, I've come to realize
> that no matter how I slice it, I can't make libgccjit usable.

The guixrus channel has an emacs which seems to support gccjit.  I have
not tried it myself.

  https://git.sr.ht/~whereiseveryone/guixrus/

Especially:

  https://git.sr.ht/~whereiseveryone/guixrus/tree/master/item/guixrus/packages/emacs.scm
  https://git.sr.ht/~whereiseveryone/guixrus/tree/master/item/guixrus/packages/gcc.scm

Maybe you can borrow some knowledge there? ;-)

Cheers,
Remco

bug#55657: libgccjit is unusable

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

Lily,

I think a lot of people are using emacs-native-comp from here: https://github.com/flatwhatson/guix-channel/blob/master/flat/packages/emacs.scm

I don't know if I've updated mine since your failed experiments, but it has worked forever...

Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

bug#55657: libgccjit is unusable

[Tobias Geerinckx-Rice via Bug reports for GNU Guix]

Meant to include link to https://github.com/flatwhatson/guix-channel/blob/657da22f0229b978b7bf4e4d476f59f17f6a175f/flat/packages/gcc.scm#L27 which is where the majic happens

Kind regards,

T G-R

Sent on the go.  Excuse or enjoy my brevity.

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Samstag, dem 04.06.2022 um 14:25 +0000 schrieb Tobias Geerinckx-
Rice:
> Meant to include link to
> https://github.com/flatwhatson/guix-channel/blob/657da22f0229b978b7bf4e4d476f59f17f6a175f/flat/packages/gcc.scm#L27
>  which is where the majic happens
I'm not personally using that channel, but I noticed this issue by
trying to adapt that recipe towards master.  Now there's a chance that
I've made an error in my copypasta game, but I'm pretty sure the
current state of libgccjit is not a good one regardless.

Cheers

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hello,

I currently use the flatwhatson channel (cc'ed on this email, hope that is okay Andrew!) and it has worked well for Emacs, though never tried anything else with libgccjit. It would be great to get this upstreamed in Guix, especially since libgccjit is a beast to build (more than emacs if I remember correctly).

Happy to help test and work on this, let me know anything I can do.

Thanks,
John

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Hi,

Am Dienstag, dem 28.06.2022 um 00:53 +0000 schrieb John Kehayias:
> Hello,
> 
> I currently use the flatwhatson channel (cc'ed on this email, hope
> that is okay Andrew!) and it has worked well for Emacs, though never
> tried anything else with libgccjit. It would be great to get this
> upstreamed in Guix, especially since libgccjit is a beast to build
> (more than emacs if I remember correctly).
Keyword here is "has worked for emacs".  I've tried porting the logic
from flatwhatson's channel over, but regardless of what I do, it
already fails in the configure step of Emacs (in a manner that's
reproducible outside as well).  Thus, I think this is a bug in
libgccjit (or perhaps our packaging of it) that simply happened to be
ignored during development of Emacs 28, but no longer in the release.

Cheers

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hi,

------- Original Message -------
On Tuesday, June 28th, 2022 at 12:17 AM, Liliana Marie Prikler wrote:

> Keyword here is "has worked for emacs". I've tried porting the logic
> from flatwhatson's channel over, but regardless of what I do, it
> already fails in the configure step of Emacs (in a manner that's
> reproducible outside as well). Thus, I think this is a bug in
> libgccjit (or perhaps our packaging of it) that simply happened to be
> ignored during development of Emacs 28, but no longer in the release.
>

Sorry, I should be extra clear that I mean has in the past and continues to work for Emacs. I've been using emacs-pgtk-native-comp through the flatwhatson channel from well before v28 was released. Currently I'm using emacs-pgtk-native-comp-28.1.50-223.3ddccb5. Everything has built, installed, and run fine for as long as I have been using it. Just in case that was in question, and as a point of reference.

Anyway, I'll try to reproduce when I can (tomorrow likely) what you reported in the first message using this setup, if that is of use.

Appreciate the efforts from everyone working on this!
John

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hi everyone,

Found out some useful info and a work around for the original reported issue of the simple "hello world" of gccjit not working.


------- Original Message -------
On Tuesday, June 28th, 2022 at 1:16 AM, John Kehayias wrote:

> Hi,
>
> ------- Original Message -------
> On Tuesday, June 28th, 2022 at 12:17 AM, Liliana Marie Prikler wrote:
>
> > Keyword here is "has worked for emacs". I've tried porting the logic
> > from flatwhatson's channel over, but regardless of what I do, it
> > already fails in the configure step of Emacs (in a manner that's
> > reproducible outside as well). Thus, I think this is a bug in
> > libgccjit (or perhaps our packaging of it) that simply happened to be
> > ignored during development of Emacs 28, but no longer in the release.
>
>
> Sorry, I should be extra clear that I mean has in the past and continues to work for Emacs. I've been using emacs-pgtk-native-comp through the flatwhatson channel from well before v28 was released. Currently I'm using emacs-pgtk-native-comp-28.1.50-223.3ddccb5. Everything has built, installed, and run fine for as long as I have been using it. Just in case that was in question, and as a point of reference.
>
> Anyway, I'll try to reproduce when I can (tomorrow likely) what you reported in the first message using this setup, if that is of use.
>

I was able to reproduce the original error, though I used the libgccjit package from the flatwhatson channel, at v11.3.0 (along with GCC at that version). For good measure, I also used the tutorial at that version, just in case https://gcc.gnu.org/onlinedocs/gcc-11.3.0/jit/intro/tutorial01.html  I chose this version since that is what emacs-native-comp from that channel is built with.

Searching for these error messages of missing libraries/files, I found

https://ref.strikr.io/jit/internals/index.html#environment-variables

and a bug report at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87808

I didn't dive into the details and I'm not expert here, but it gave me the clues to work around it. Seems that where gccjit looks for things has some assumptions (bugs?) which we can fix at runtime with:

LIBRARY_PATH=$GUIX_ENVIRONMENT/lib/gcc/x86_64-unknown-linux-gnu/11.3.0:$LIBRARY_PATH ./gccjittest

The errors reported before were solved with this LIBRARY_PATH addition of the lib/gcc subdirectory. So, the test program runs in

guix shell gcc-toolchain@11 libgccjit@11 --pure

where I compiled to gccjittest following the tutorial directions (no change to LIBRARY_PATH).

So, looking at the emacs-native-comp definition in flatwhatson, we can see that a phase is used to set LIBRARY_PATH before configure just as I did here: https://github.com/flatwhatson/guix-channel/blob/master/flat/packages/emacs.scm#L65

Hope this is helpful and unblocks libgccjit and emacs-native-comp for Guix!

John

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Hi John,

Am Mittwoch, dem 03.08.2022 um 21:13 +0000 schrieb John Kehayias:
> I didn't dive into the details and I'm not expert here, but it gave
> me the clues to work around it. Seems that where gccjit looks for
> things has some assumptions (bugs?) which we can fix at runtime with:
> 
> LIBRARY_PATH=$GUIX_ENVIRONMENT/lib/gcc/x86_64-unknown-linux-
> gnu/11.3.0:$LIBRARY_PATH ./gccjittest
> 
> The errors reported before were solved with this LIBRARY_PATH
> addition of the lib/gcc subdirectory. So, the test program runs in
> 
> guix shell gcc-toolchain@11 libgccjit@11 --pure
> 
> where I compiled to gccjittest following the tutorial directions (no
> change to LIBRARY_PATH).
while this does help insofar as I now know which snippet I forgot to
copy, I do still think that this leaves us with two unreasonable
options if we want to use emacs to compile other packages:

1. Propagate gcc-toolchain from emacs.
2. Patch LIBRARY_PATH not just before configuration, but also via a
wrapper.

At the very least I don't see how Emacs would be able to compile other
packages to native code without either of the above.

WDYT?

bug#55657: libgccjit is unusable

[Andrew Whatson]

Hi John, Liliana,

Sorry I haven't jumped in before now, I appreciate your efforts to
bring emacs native-comp to guix!

On Thu, 4 Aug 2022 at 04:26, Liliana Marie Prikler
<liliana.prikler@gmail.com> wrote:
>
> while this does help insofar as I now know which snippet I forgot to
> copy, I do still think that this leaves us with two unreasonable
> options if we want to use emacs to compile other packages:
>
> 1. Propagate gcc-toolchain from emacs.
> 2. Patch LIBRARY_PATH not just before configuration, but also via a
> wrapper.
>
> At the very least I don't see how Emacs would be able to compile other
> packages to native code without either of the above.
>
> WDYT?

The solution used in the package-definition in my channel is to patch
`comp.el` to directly reference the necessary gcc/glibc paths instead
of relying on the environment.  This occurs in the
"patch-driver-options" step immediately after the "set-libgccjit-path"
step mentioned earlier.  This makes gcc-toolchain part of emacs
closure, without requiring it to be propagated into the profile.

If I understand the problem correctly, that should suffice?

Cheers,
Andrew

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Donnerstag, dem 04.08.2022 um 04:48 +0000 schrieb Andrew Whatson:
> Hi John, Liliana,
> 
> Sorry I haven't jumped in before now, I appreciate your efforts to
> bring emacs native-comp to guix!
> 
> On Thu, 4 Aug 2022 at 04:26, Liliana Marie Prikler
> <liliana.prikler@gmail.com> wrote:
> > 
> > while this does help insofar as I now know which snippet I forgot
> > to
> > copy, I do still think that this leaves us with two unreasonable
> > options if we want to use emacs to compile other packages:
> > 
> > 1. Propagate gcc-toolchain from emacs.
> > 2. Patch LIBRARY_PATH not just before configuration, but also via a
> > wrapper.
> > 
> > At the very least I don't see how Emacs would be able to compile
> > other
> > packages to native code without either of the above.
> > 
> > WDYT?
> 
> The solution used in the package-definition in my channel is to patch
> `comp.el` to directly reference the necessary gcc/glibc paths instead
> of relying on the environment.  This occurs in the
> "patch-driver-options" step immediately after the "set-libgccjit-
> path" step mentioned earlier.  This makes gcc-toolchain part of emacs
> closure, without requiring it to be propagated into the profile.
> 
> If I understand the problem correctly, that should suffice?
But if I read your recipe correctly, you're not resolving %host-type in
those options.  Does that really suffice?

bug#55657: libgccjit is unusable

[Andrew Whatson]

On Thu, 4 Aug 2022 at 16:52, Liliana Marie Prikler
<liliana.prikler@gmail.com> wrote:
>
> > The solution used in the package-definition in my channel is to patch
> > `comp.el` to directly reference the necessary gcc/glibc paths instead
> > of relying on the environment.  This occurs in the
> > "patch-driver-options" step immediately after the "set-libgccjit-
> > path" step mentioned earlier.  This makes gcc-toolchain part of emacs
> > closure, without requiring it to be propagated into the profile.
> >
> > If I understand the problem correctly, that should suffice?
>
> But if I read your recipe correctly, you're not resolving %host-type in
> those options.  Does that really suffice?

Ah, yes that is a little confusing.  This is a quirk of the different
behaviour of the LIBRARY_PATH environment variable and the -B flag to
gcc.

I recommend reading about "-Bprefix" in `man gcc`, but in short it
tries those paths with and without "machine/version" appended for the
target machine and compiler version.  We *could* hard-code those, but
it isn't necessary, and it seemed like that might cause problems if
someone's brave enough to attempt cross-compilation of native-comp
emacs.

A major benefit of patching "comp.el" directly is that it avoids
leaking gcc into the user's environment.  It's possible that someone
is running emacs in a profile with a specific version of gcc, maybe
without libgccjit support (eg. while hacking on some legacy code), so
having emacs insist on a libgccjit-compatible gcc present in the
environment at runtime would cause lots of problems.

The other important bit is the libgccjit package.  The one in guix
mainline works fine, it should be possible to get a working libgccjit
as-is.  I updated the package definition on my channel for the
following reasons:

a) Support newer gcc versions

While developing native-comp support for emacs, Andrea found and fixed
some libgccjit bugs which made their way into subsequent releases of
gcc.  The native-comp library includes work-arounds for these
problems, but produces faster/smaller code with a fresher libgccjit.

b) Reduce compilation time

The definition in guix is basically the standard gcc build, but with
libgccjit also enabled.  This results in an arduous double-bootstrap
and building a bunch of compilers and libraries that are completely
unnecessary and unused by libgccjit.  I've disabled all the
unnecessary stuff and depend on the main gcc package to build
libgccjit, relying on the fact that it's already properly
bootstrapped.  It's much quicker to build, which is important if you
don't have substitutes.

Hope this helps,
Andrew

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Freitag, dem 05.08.2022 um 00:59 +0000 schrieb Andrew Whatson:
> On Thu, 4 Aug 2022 at 16:52, Liliana Marie Prikler
> <liliana.prikler@gmail.com> wrote
> > But if I read your recipe correctly, you're not resolving %host-
> > type in those options.  Does that really suffice?
> 
> Ah, yes that is a little confusing.  This is a quirk of the different
> behaviour of the LIBRARY_PATH environment variable and the -B flag to
> gcc.
> 
> I recommend reading about "-Bprefix" in `man gcc`, but in short it
> tries those paths with and without "machine/version" appended for the
> target machine and compiler version.  We *could* hard-code those, but
> it isn't necessary, and it seemed like that might cause problems if
> someone's brave enough to attempt cross-compilation of native-comp
> emacs.
Fair enough.  The compiler driver does not appear to be the failing
part anyway, though given the status quo I can't exactly claim it is
not failing.

> The other important bit is the libgccjit package.  The one in guix
> mainline works fine, it should be possible to get a working libgccjit
> as-is.  
From my own experience, I doubt that.  Read also the thread title.

> I updated the package definition on my channel for the following
> reasons:
> 
> a) Support newer gcc versions
> 
> While developing native-comp support for emacs, Andrea found and
> fixed some libgccjit bugs which made their way into subsequent
> releases of gcc.  The native-comp library includes work-arounds for
> these problems, but produces faster/smaller code with a fresher
> libgccjit.
> 
> b) Reduce compilation time
> 
> The definition in guix is basically the standard gcc build, but with
> libgccjit also enabled.  This results in an arduous double-bootstrap
> and building a bunch of compilers and libraries that are completely
> unnecessary and unused by libgccjit.  I've disabled all the
> unnecessary stuff and depend on the main gcc package to build
> libgccjit, relying on the fact that it's already properly
> bootstrapped.  It's much quicker to build, which is important if you
> don't have substitutes.
> 
> Hope this helps,
> Andrew
I actually am at a loss, so instead of referring to your code, I shall
share mine.  Note that, style aside, it should actually perform the
same steps as yours, but fails to do so.  I also tried adding the
unversioned lib directory to LIBRARY_PATH and setting LD_LIBRARY_PATH
to little fanfare.  Whatever I do, it seems configure wants to smoke a
different joint.

Cheers

;; gcc.scm

(define-public (make-libgccjit gcc)
  (package
    (inherit gcc)
    (name "libgccjit")
    (outputs (delete "lib" (package-outputs gcc)))
    (properties (alist-delete 'hidden? (package-properties gcc)))
    (arguments
     (substitute-keyword-arguments (package-arguments gcc)
       ((#:modules _ '())
        '((guix build gnu-build-system)
          (guix build utils)
          (ice-9 regex)
          (srfi srfi-1)
          (srfi srfi-26)))
       ((#:configure-flags flags)
        #~(cons* "--disable-bootstrap"
                 "--disable-libatomic"
                 "--disable-libgomp"
                 "--disable-libquadmath"
                 "--disable-libssp"
                 "--enable-host-shared"
                 "--enable-languages=jit"
                 (remove (cut string-match "--enable-languages.*" <>)
                         #$flags)))
       ((#:phases phases)
        #~(modify-phases #$phases
            (add-after 'install 'remove-broken-or-conflicting-files
              (lambda* (#:key outputs #:allow-other-keys)
                (for-each delete-file
                          (find-files
                           (string-append (assoc-ref outputs "out")
"/bin")
                           ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-
.*)"))))))))
    (inputs (modify-inputs (package-inputs gcc)
              (delete "libstdc++")))
    (native-inputs (modify-inputs (package-native-inputs gcc)
                     (prepend gcc)))
    (synopsis "GCC library generating machine code on-the-fly at
runtime")
    (description
     "This package is part of the GNU Compiler Collection and provides
an
embeddable library for generating machine code on-the-fly at runtime. 
This
shared library can then be dynamically-linked into bytecode
interpreters and
other such programs that want to generate machine code on-the-fly at
run-time.
It can also be used for ahead-of-time code generation for building
standalone
compilers.  The just-in-time (jit) part of the name is now something of
a
misnomer.")))

(define-public libgccjit-9 (make-libgccjit gcc-9))
(define-public libgccjit-10 (make-libgccjit gcc-10))
(define-public libgccjit-11 (make-libgccjit gcc-11))
(define-public libgccjit-12 (make-libgccjit gcc-12))

(define-public libgccjit libgccjit-10)

;; emacs.scm

(define-public emacs
  (package
    (name "emacs")
    (version "28.1")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnu/emacs/emacs-"
                                  version ".tar.xz"))
              (sha256
               (base32
               
"1qbmmmhnjhn4lvzsnyk7l5ganbi6wzbm38jc1a7hhyh3k78b7c98"))
              (patches (search-patches "emacs-exec-path.patch"
                                       "emacs-fix-scheme-indent-
function.patch"
                                       "emacs-source-date-
epoch.patch"))
              (modules '((guix build utils)))
              (snippet
               '(with-directory-excursion "lisp"
                  ;; Delete the bundled byte-compiled elisp files and
generated
                  ;; autoloads.
                  (for-each delete-file
                            (append (find-files "." "\\.elc$")
                                    (find-files "." "loaddefs\\.el$")
                                    (find-files "eshell" "^esh-
groups\\.el$")))

                  ;; Make sure Tramp looks for binaries in the right
places on
                  ;; remote Guix System machines, where 'getconf PATH'
returns
                  ;; something bogus.
                  (substitute* "net/tramp.el"
                    ;; Patch the line after "(defcustom tramp-remote-
path".
                    (("\\(tramp-default-remote-path")
                     (format #f "(tramp-default-remote-path ~s ~s ~s ~s
"
                             "~/.guix-profile/bin" "~/.guix-
profile/sbin"
                             "/run/current-system/profile/bin"
                             "/run/current-system/profile/sbin")))

                  ;; Make sure Man looks for C header files in the
right
                  ;; places.
                  (substitute* "man.el"
                    (("\"/usr/local/include\"" line)
                     (string-join
                      (list line
                            "\"~/.guix-profile/include\""
                           
"\"/var/guix/profiles/system/profile/include\"")
                      " ")))))))
    (build-system glib-or-gtk-build-system)
    (arguments
     (list
      #:tests? #f                      ; no check target
      #:modules `((guix build glib-or-gtk-build-system)
                  (guix build utils)
                  (srfi srfi-1)
                  (ice-9 ftw))
      #:configure-flags #~(list "--with-modules"
                                "--with-cairo"
                                "--with-native-compilation"
                                "--disable-build-details")
      #:make-flags #~(list "NATIVE_FULL_AOT=1")
      #:phases
      #~(modify-phases %standard-phases
          (add-after 'set-paths 'set-libgccjit-path
            (lambda* (#:key inputs #:allow-other-keys)
              (define (first-subdirectory/absolute directory)
                (let ((files (scandir
                              directory
                              (lambda (file)
                                (and (not (member file '("." "..")))
                                     (file-is-directory? (string-append
                                                          directory "/"
                                                          file)))))))
                  (and (not (null? files))
                       (string-append directory "/" (car files)))))

              (let* ((libgccjit-libdir
                      (first-subdirectory/absolute ;; version
                       (first-subdirectory/absolute ;; host type
                        (search-input-directory inputs "lib/gcc")))))
                (setenv "LIBRARY_PATH"
                        (string-append libgccjit-libdir ":"
                                       (getenv "LIBRARY_PATH"))))))
          (add-after 'unpack 'enable-elogind
            (lambda _
              (substitute* "configure.ac"
                (("libsystemd") "libelogind"))
              (when (file-exists? "configure")
                (delete-file "configure"))))
          (add-after 'unpack 'patch-program-file-names
            (lambda* (#:key inputs #:allow-other-keys)
              (substitute* '("src/callproc.c"
                             "lisp/term.el"
                             "lisp/htmlfontify.el"
                             "lisp/textmodes/artist.el"
                             "lisp/progmodes/sh-script.el")
                (("\"/bin/sh\"")
                 (format #f "~s" (search-input-file inputs
"/bin/sh"))))
              (substitute* "lisp/doc-view.el"
                (("\"(gs|dvipdf|ps2pdf|pdftotext)\"" all what)
                 (let ((replacement (false-if-exception
                                     (search-input-file
                                      inputs
                                      (string-append "/bin/" what)))))
                   (if replacement
                       (string-append "\"" replacement "\"")
                       all))))
              ;; match ".gvfs-fuse-daemon-real" and ".gvfsd-fuse-real"
              ;; respectively when looking for GVFS processes.
              (substitute* "lisp/net/tramp-gvfs.el"
                (("\\(tramp-compat-process-running-p \"(.*)\"\\)" all
process)
                 (format #f "(or ~a (tramp-compat-process-running-p
~s))"
                         all (string-append "." process "-real"))))))
          (add-after 'unpack 'patch-compilation-driver
            (lambda _
              (substitute* "lisp/emacs-lisp/comp.el"
                (("\\(defcustom native-comp-driver-options nil")
                 (format
                  #f "(defcustom native-comp-driver-options '(~s ~s ~s
~s)"
                  (string-append
                   "-B" #$(this-package-input "binutils") "/bin/")
                  (string-append
                   "-B" #$(this-package-input "glibc") "/lib/")
                  (string-append
                   "-B" #$(this-package-input "libgccjit") "/lib/")
                  (string-append
                   "-B" #$(this-package-input "libgccjit")
"/lib/gcc/"))))))
          (add-before 'configure 'fix-/bin/pwd
            (lambda _
              ;; Use `pwd', not `/bin/pwd'.
              (substitute* (find-files "." "^Makefile\\.in$")
                (("/bin/pwd")
                 "pwd"))))
          (add-after 'install 'install-site-start
            ;; Use 'guix-emacs' in "site-start.el", which is used
autoload the
            ;; Elisp packages found in EMACSLOADPATH.
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((out      (assoc-ref outputs "out"))
                     (lisp-dir (string-append out "/share/emacs/site-
lisp"))
                     (emacs    (string-append out "/bin/emacs")))

                ;; This is duplicated from emacs-utils to prevent
coupling.
                (define* (emacs-byte-compile-directory dir)
                  (let ((expr `(progn
                                (setq byte-compile-debug t)
                                (byte-recompile-directory
                                 (file-name-as-directory ,dir) 0 1))))
                    (invoke emacs "--quick" "--batch"
                            (format #f "--eval=~s" expr))))

                (copy-file #$(local-file
                              (search-auxiliary-file "emacs/guix-
emacs.el"))
                           (string-append lisp-dir "/guix-emacs.el"))
                (with-output-to-file (string-append lisp-dir "/site-
start.el")
                  (lambda ()
                    (display
                     (string-append
                      "(when (require 'guix-emacs nil t)\n"
                      "  (guix-emacs-autoload-packages)\n"
                      "  (advice-add 'package-load-all-descriptors"
                      " :after #'guix-emacs-load-package-
descriptors))"))))
                ;; Remove the extraneous subdirs.el file, as it causes
Emacs to
                ;; add recursively all the the sub-directories of a
profile's
                ;; share/emacs/site-lisp union when added to
EMACSLOADPATH,
                ;; which leads to conflicts.
                (delete-file (string-append lisp-dir "/subdirs.el"))
                ;; Byte compile the site-start files.
                (emacs-byte-compile-directory lisp-dir))))
          (add-after 'glib-or-gtk-wrap 'restore-emacs-pdmp
            ;; restore the dump file that Emacs installs somewhere in
            ;; libexec/ to its original state
            (lambda* (#:key outputs target #:allow-other-keys)
              (let* ((libexec (string-append (assoc-ref outputs "out")
                                             "/libexec"))
                     ;; each of these ought to only match a single
file,
                     ;; but even if not (find-files) sorts by string<,
                     ;; so the Nth element in one maps to the Nth
element of
                     ;; the other
                     (pdmp (find-files libexec "\\.pdmp$"))
                     (pdmp-real (find-files libexec "\\.pdmp-real$")))
                (for-each rename-file pdmp-real pdmp))))
          (add-after 'glib-or-gtk-wrap 'strip-double-wrap
            (lambda* (#:key outputs #:allow-other-keys)
              ;; Directly copy emacs-X.Y to emacs, so that it is not
wrapped
              ;; twice.  This also fixes a minor issue, where WMs would
not be
              ;; able to track emacs back to emacs.desktop.
              (with-directory-excursion (assoc-ref outputs "out")
                (copy-file
                 (car (find-files "bin" "^emacs-([0-9]+\\.)+[0-9]+$"))
                 "bin/emacs"))))
          (add-after 'strip-double-wrap 'wrap-emacs-paths
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((out (assoc-ref outputs "out"))
                     (lisp-dirs (find-files (string-append out
"/share/emacs")
                                            "^lisp$"
                                            #:directories? #t)))
                (for-each
                 (lambda (prog)
                   (wrap-program prog
                     ;; emacs-next and variants rely on uname being in
PATH for
                     ;; Tramp.  Tramp paths can't be hardcoded, because
they
                     ;; need to be portable.
                     `("PATH" suffix
                       ,(map dirname
                             (list (search-input-file inputs
"/bin/gzip")
                                   ;; for coreutils
                                   (search-input-file inputs
"/bin/yes"))))
                     `("EMACSLOADPATH" suffix ,lisp-dirs)))
                 (find-files (string-append out "/bin")
                             ;; Matches versioned and unversioned emacs
binaries.
                             ;; We don't patch emacsclient, because it
takes its
                             ;; environment variables from emacs.
                             ;; Likewise, we don't need to patch helper
binaries
                             ;; like etags, ctags or ebrowse.
                             "^emacs(-[0-9]+(\\.[0-9]+)*)?$"))))))))
    (inputs
     (list gnutls
           ncurses

           ;; For native compilation
           binutils
           glibc
           libgccjit

           ;; Required for "core" functionality, such as dired and
compression.
           coreutils
           gzip

           ;; Avoid Emacs's limited movemail substitute that retrieves
POP3
           ;; email only via insecure channels.
           ;; This is not needed for (modern) IMAP.
           mailutils

           gpm
           libx11
           gtk+
           cairo
           pango
           harfbuzz
           libxft
           libtiff
           giflib
           lcms
           libjpeg-turbo
           libselinux
           acl
           jansson
           gmp
           ghostscript
           poppler
           elogind

           ;; When looking for libpng `configure' links with `-lpng -
lz', so we
           ;; must also provide zlib as an input.
           libpng
           zlib
           (if (target-x86-64?)
               librsvg-bootstrap
               librsvg-2.40)
           libxpm
           libxml2
           libice
           libsm
           alsa-lib
           dbus

           ;; multilingualization support
           libotf
           m17n-lib))
    (native-inputs
     (list autoconf pkg-config texinfo))
    (native-search-paths
     (list (search-path-specification
            (variable "EMACSLOADPATH")
            (files '("share/emacs/site-lisp")))
           (search-path-specification
            (variable "INFOPATH")
            (files '("share/info")))))

    (home-page "https://www.gnu.org/software/emacs/")
    (synopsis "The extensible, customizable, self-documenting text
editor")
    (description
     "GNU Emacs is an extensible and highly customizable text editor. 
It is
based on an Emacs Lisp interpreter with extensions for text editing. 
Emacs
has been extended in essentially all areas of computing, giving rise to
a
vast array of packages supporting, e.g., email, IRC and XMPP messaging,
spreadsheets, remote server editing, and much more.  Emacs includes
extensive
documentation on all aspects of the system, from basic editing to
writing
large Lisp programs.  It has full Unicode support for nearly all human
languages.")
    (license license:gpl3+)))

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hi,

Looks like the code you sent got line wrapped and I couldn't easily untangle it. The difference that pops out to me is how you are locating the lib/gcc directory, is it maybe picking up from the build system rather than libgccjit? (though I think gcc:lib should be the same here, but not gcc-toolchain)

In any event, I just want to reiterate that the libgccjit and emacs-native-comp from Andrew's channel does indeed work: it compiles, runs, and does native compilations successfully. So hopefully we've narrowed down now that there is some difference in the code you tried and from Andrew's that is leading to different behavior.

John

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

[-- Attachment #1: Type: text/plain, Size: 1230 bytes --]

Am Freitag, dem 05.08.2022 um 20:01 +0000 schrieb John Kehayias:
> Hi,
> 
> Looks like the code you sent got line wrapped and I couldn't easily
> untangle it. The difference that pops out to me is how you are
> locating the lib/gcc directory, is it maybe picking up from the build
> system rather than libgccjit? (though I think gcc:lib should be the
> same here, but not gcc-toolchain)
No: /gnu/store/640hfpr069hvqv9gcs0ayq0is21zfii6-libgccjit-
10.3.0/lib/gcc/x86_64-unknown-linux-gnu/10.3.0

> In any event, I just want to reiterate that the libgccjit and emacs-
> native-comp from Andrew's channel does indeed work: it compiles,
> runs, and does native compilations successfully. So hopefully we've
> narrowed down now that there is some difference in the code you tried
> and from Andrew's that is leading to different behavior.
The style changes are not the only thing at play here, though.  In
particular, I am trying to build Emacs 28.1, whereas Andrew builds
other versions, though notably the newest one ought to include the
smoke test.  You could also try other combinations for libgccjit and
gcc to see if those make a difference – last time I tried it did not.

Attached the definitions as a file this time.

Cheers

[-- Attachment #2: emacs-native-comp-merged.scm --]
[-- Type: text/x-scheme, Size: 15795 bytes --]

;; gcc.scm

(define-public (make-libgccjit gcc)
  (package
    (inherit gcc)
    (name "libgccjit")
    (outputs (delete "lib" (package-outputs gcc)))
    (properties (alist-delete 'hidden? (package-properties gcc)))
    (arguments
     (substitute-keyword-arguments (package-arguments gcc)
       ((#:modules _ '())
        '((guix build gnu-build-system)
          (guix build utils)
          (ice-9 regex)
          (srfi srfi-1)
          (srfi srfi-26)))
       ((#:configure-flags flags)
        #~(cons* "--disable-bootstrap"
                 "--disable-libatomic"
                 "--disable-libgomp"
                 "--disable-libquadmath"
                 "--disable-libssp"
                 "--enable-host-shared"
                 "--enable-languages=jit"
                 (remove (cut string-match "--enable-languages.*" <>)
                         #$flags)))
       ((#:phases phases)
        #~(modify-phases #$phases
            (add-after 'install 'remove-broken-or-conflicting-files
              (lambda* (#:key outputs #:allow-other-keys)
                (for-each delete-file
                          (find-files
                           (string-append (assoc-ref outputs "out") "/bin")
                           ".*(c\\+\\+|cpp|g\\+\\+|gcov|gcc|gcc-.*)"))))))))
    (inputs (modify-inputs (package-inputs gcc)
              (delete "libstdc++")))
    (native-inputs (modify-inputs (package-native-inputs gcc)
                     (prepend gcc)))
    (synopsis "GCC library generating machine code on-the-fly at runtime")
    (description
     "This package is part of the GNU Compiler Collection and provides an
embeddable library for generating machine code on-the-fly at runtime.  This
shared library can then be dynamically-linked into bytecode interpreters and
other such programs that want to generate machine code on-the-fly at run-time.
It can also be used for ahead-of-time code generation for building standalone
compilers.  The just-in-time (jit) part of the name is now something of a
misnomer.")))

(define-public libgccjit-9 (make-libgccjit gcc-9))
(define-public libgccjit-10 (make-libgccjit gcc-10))
(define-public libgccjit-11 (make-libgccjit gcc-11))
(define-public libgccjit-12 (make-libgccjit gcc-12))

(define-public libgccjit libgccjit-10)

;; emacs.scm

(define-public emacs
  (package
    (name "emacs")
    (version "28.1.90")
    (source (origin
              (method url-fetch)
              (uri (string-append "mirror://gnu/emacs/emacs-"
                                  version ".tar.xz"))
              (sha256
               (base32
                "1qbmmmhnjhn4lvzsnyk7l5ganbi6wzbm38jc1a7hhyh3k78b7c98"))
              (patches (search-patches "emacs-exec-path.patch"
                                       "emacs-fix-scheme-indent-function.patch"
                                       "emacs-source-date-epoch.patch"))
              (modules '((guix build utils)))
              (snippet
               '(with-directory-excursion "lisp"
                  ;; Delete the bundled byte-compiled elisp files and generated
                  ;; autoloads.
                  (for-each delete-file
                            (append (find-files "." "\\.elc$")
                                    (find-files "." "loaddefs\\.el$")
                                    (find-files "eshell" "^esh-groups\\.el$")))

                  ;; Make sure Tramp looks for binaries in the right places on
                  ;; remote Guix System machines, where 'getconf PATH' returns
                  ;; something bogus.
                  (substitute* "net/tramp.el"
                    ;; Patch the line after "(defcustom tramp-remote-path".
                    (("\\(tramp-default-remote-path")
                     (format #f "(tramp-default-remote-path ~s ~s ~s ~s "
                             "~/.guix-profile/bin" "~/.guix-profile/sbin"
                             "/run/current-system/profile/bin"
                             "/run/current-system/profile/sbin")))

                  ;; Make sure Man looks for C header files in the right
                  ;; places.
                  (substitute* "man.el"
                    (("\"/usr/local/include\"" line)
                     (string-join
                      (list line
                            "\"~/.guix-profile/include\""
                            "\"/var/guix/profiles/system/profile/include\"")
                      " ")))))))
    (build-system glib-or-gtk-build-system)
    (arguments
     (list
      #:tests? #f                      ; no check target
      #:modules `((guix build glib-or-gtk-build-system)
                  (guix build utils)
                  (srfi srfi-1)
                  (ice-9 ftw))
      #:configure-flags #~(list "--with-modules"
                                "--with-cairo"
                                "--with-native-compilation"
                                "--disable-build-details")
      #:make-flags #~(list "NATIVE_FULL_AOT=1")
      #:phases
      #~(modify-phases %standard-phases
          (add-after 'set-paths 'set-libgccjit-path
            (lambda* (#:key inputs #:allow-other-keys)
              (define (first-subdirectory/absolute directory)
                (let ((files (scandir
                              directory
                              (lambda (file)
                                (and (not (member file '("." "..")))
                                     (file-is-directory? (string-append
                                                          directory "/"
                                                          file)))))))
                  (and (not (null? files))
                       (string-append directory "/" (car files)))))

              (let* ((libgccjit-libdir
                      (first-subdirectory/absolute ;; version
                       (first-subdirectory/absolute ;; host type
                        (search-input-directory inputs "lib/gcc")))))
                (setenv "LIBRARY_PATH"
                        (string-append libgccjit-libdir ":"
                                       (getenv "LIBRARY_PATH"))))))
          (add-after 'unpack 'enable-elogind
            (lambda _
              (substitute* "configure.ac"
                (("libsystemd") "libelogind"))
              (when (file-exists? "configure")
                (delete-file "configure"))))
          (add-after 'unpack 'patch-program-file-names
            (lambda* (#:key inputs #:allow-other-keys)
              (substitute* '("src/callproc.c"
                             "lisp/term.el"
                             "lisp/htmlfontify.el"
                             "lisp/textmodes/artist.el"
                             "lisp/progmodes/sh-script.el")
                (("\"/bin/sh\"")
                 (format #f "~s" (search-input-file inputs "/bin/sh"))))
              (substitute* "lisp/doc-view.el"
                (("\"(gs|dvipdf|ps2pdf|pdftotext)\"" all what)
                 (let ((replacement (false-if-exception
                                     (search-input-file
                                      inputs
                                      (string-append "/bin/" what)))))
                   (if replacement
                       (string-append "\"" replacement "\"")
                       all))))
              ;; match ".gvfs-fuse-daemon-real" and ".gvfsd-fuse-real"
              ;; respectively when looking for GVFS processes.
              (substitute* "lisp/net/tramp-gvfs.el"
                (("\\(tramp-compat-process-running-p \"(.*)\"\\)" all process)
                 (format #f "(or ~a (tramp-compat-process-running-p ~s))"
                         all (string-append "." process "-real"))))))
          (add-after 'unpack 'patch-compilation-driver
            (lambda _
              (substitute* "lisp/emacs-lisp/comp.el"
                (("\\(defcustom native-comp-driver-options nil")
                 (format
                  #f "(defcustom native-comp-driver-options '(~s ~s ~s ~s)"
                  (string-append
                   "-B" #$(this-package-input "binutils") "/bin/")
                  (string-append
                   "-B" #$(this-package-input "glibc") "/lib/")
                  (string-append
                   "-B" #$(this-package-input "libgccjit") "/lib/")
                  (string-append
                   "-B" #$(this-package-input "libgccjit") "/lib/gcc/"))))))
          (add-before 'configure 'fix-/bin/pwd
            (lambda _
              ;; Use `pwd', not `/bin/pwd'.
              (substitute* (find-files "." "^Makefile\\.in$")
                (("/bin/pwd")
                 "pwd"))))
          (add-after 'install 'install-site-start
            ;; Use 'guix-emacs' in "site-start.el", which is used autoload the
            ;; Elisp packages found in EMACSLOADPATH.
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((out      (assoc-ref outputs "out"))
                     (lisp-dir (string-append out "/share/emacs/site-lisp"))
                     (emacs    (string-append out "/bin/emacs")))

                ;; This is duplicated from emacs-utils to prevent coupling.
                (define* (emacs-byte-compile-directory dir)
                  (let ((expr `(progn
                                (setq byte-compile-debug t)
                                (byte-recompile-directory
                                 (file-name-as-directory ,dir) 0 1))))
                    (invoke emacs "--quick" "--batch"
                            (format #f "--eval=~s" expr))))

                (copy-file #$(local-file
                              (search-auxiliary-file "emacs/guix-emacs.el"))
                           (string-append lisp-dir "/guix-emacs.el"))
                (with-output-to-file (string-append lisp-dir "/site-start.el")
                  (lambda ()
                    (display
                     (string-append
                      "(when (require 'guix-emacs nil t)\n"
                      "  (guix-emacs-autoload-packages)\n"
                      "  (advice-add 'package-load-all-descriptors"
                      " :after #'guix-emacs-load-package-descriptors))"))))
                ;; Remove the extraneous subdirs.el file, as it causes Emacs to
                ;; add recursively all the the sub-directories of a profile's
                ;; share/emacs/site-lisp union when added to EMACSLOADPATH,
                ;; which leads to conflicts.
                (delete-file (string-append lisp-dir "/subdirs.el"))
                ;; Byte compile the site-start files.
                (emacs-byte-compile-directory lisp-dir))))
          (add-after 'glib-or-gtk-wrap 'restore-emacs-pdmp
            ;; restore the dump file that Emacs installs somewhere in
            ;; libexec/ to its original state
            (lambda* (#:key outputs target #:allow-other-keys)
              (let* ((libexec (string-append (assoc-ref outputs "out")
                                             "/libexec"))
                     ;; each of these ought to only match a single file,
                     ;; but even if not (find-files) sorts by string<,
                     ;; so the Nth element in one maps to the Nth element of
                     ;; the other
                     (pdmp (find-files libexec "\\.pdmp$"))
                     (pdmp-real (find-files libexec "\\.pdmp-real$")))
                (for-each rename-file pdmp-real pdmp))))
          (add-after 'glib-or-gtk-wrap 'strip-double-wrap
            (lambda* (#:key outputs #:allow-other-keys)
              ;; Directly copy emacs-X.Y to emacs, so that it is not wrapped
              ;; twice.  This also fixes a minor issue, where WMs would not be
              ;; able to track emacs back to emacs.desktop.
              (with-directory-excursion (assoc-ref outputs "out")
                (copy-file
                 (car (find-files "bin" "^emacs-([0-9]+\\.)+[0-9]+$"))
                 "bin/emacs"))))
          (add-after 'strip-double-wrap 'wrap-emacs-paths
            (lambda* (#:key inputs outputs #:allow-other-keys)
              (let* ((out (assoc-ref outputs "out"))
                     (lisp-dirs (find-files (string-append out "/share/emacs")
                                            "^lisp$"
                                            #:directories? #t)))
                (for-each
                 (lambda (prog)
                   (wrap-program prog
                     ;; emacs-next and variants rely on uname being in PATH for
                     ;; Tramp.  Tramp paths can't be hardcoded, because they
                     ;; need to be portable.
                     `("PATH" suffix
                       ,(map dirname
                             (list (search-input-file inputs "/bin/gzip")
                                   ;; for coreutils
                                   (search-input-file inputs "/bin/yes"))))
                     `("EMACSLOADPATH" suffix ,lisp-dirs)))
                 (find-files (string-append out "/bin")
                             ;; Matches versioned and unversioned emacs binaries.
                             ;; We don't patch emacsclient, because it takes its
                             ;; environment variables from emacs.
                             ;; Likewise, we don't need to patch helper binaries
                             ;; like etags, ctags or ebrowse.
                             "^emacs(-[0-9]+(\\.[0-9]+)*)?$"))))))))
    (inputs
     (list gnutls
           ncurses

           ;; For native compilation
           binutils
           glibc
           libgccjit

           ;; Required for "core" functionality, such as dired and compression.
           coreutils
           gzip

           ;; Avoid Emacs's limited movemail substitute that retrieves POP3
           ;; email only via insecure channels.
           ;; This is not needed for (modern) IMAP.
           mailutils

           gpm
           libx11
           gtk+
           cairo
           pango
           harfbuzz
           libxft
           libtiff
           giflib
           lcms
           libjpeg-turbo
           libselinux
           acl
           jansson
           gmp
           ghostscript
           poppler
           elogind

           ;; When looking for libpng `configure' links with `-lpng -lz', so we
           ;; must also provide zlib as an input.
           libpng
           zlib
           (if (target-x86-64?)
               librsvg-bootstrap
               librsvg-2.40)
           libxpm
           libxml2
           libice
           libsm
           alsa-lib
           dbus

           ;; multilingualization support
           libotf
           m17n-lib))
    (native-inputs
     (list autoconf pkg-config texinfo))
    (native-search-paths
     (list (search-path-specification
            (variable "EMACSLOADPATH")
            (files '("share/emacs/site-lisp")))
           (search-path-specification
            (variable "INFOPATH")
            (files '("share/info")))))

    (home-page "https://www.gnu.org/software/emacs/")
    (synopsis "The extensible, customizable, self-documenting text editor")
    (description
     "GNU Emacs is an extensible and highly customizable text editor.  It is
based on an Emacs Lisp interpreter with extensions for text editing.  Emacs
has been extended in essentially all areas of computing, giving rise to a
vast array of packages supporting, e.g., email, IRC and XMPP messaging,
spreadsheets, remote server editing, and much more.  Emacs includes extensive
documentation on all aspects of the system, from basic editing to writing
large Lisp programs.  It has full Unicode support for nearly all human
languages.")
    (license license:gpl3+)))

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hello,

------- Original Message -------
On Friday, August 5th, 2022 at 5:31 PM, Liliana Marie Prikler wrote:

> The style changes are not the only thing at play here, though. In
> particular, I am trying to build Emacs 28.1, whereas Andrew builds
> other versions, though notably the newest one ought to include the
> smoke test. You could also try other combinations for libgccjit and
> gcc to see if those make a difference – last time I tried it did not.
>

I'm seeing the same version, 28.1.90, in yours and from Andrew's channel. And just checking that indeed it is 28.1.90 that I've been running locally from that channel, with libgccjit@11.3.0. I have not tried with v29.

> Attached the definitions as a file this time.
>

Thanks, will play around with it this weekend to see if I have anything useful to add.

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Samstag, dem 06.08.2022 um 05:37 +0000 schrieb John Kehayias:
> Hello,
> 
> ------- Original Message -------
> On Friday, August 5th, 2022 at 5:31 PM, Liliana Marie Prikler wrote:
> 
> > The style changes are not the only thing at play here, though. In
> > particular, I am trying to build Emacs 28.1, whereas Andrew builds
> > other versions, though notably the newest one ought to include the
> > smoke test. You could also try other combinations for libgccjit and
> > gcc to see if those make a difference – last time I tried it did
> > not.
> > 
> 
> I'm seeing the same version, 28.1.90, in yours and from Andrew's
> channel. And just checking that indeed it is 28.1.90 that I've been
> running locally from that channel, with libgccjit@11.3.0. I have not
> tried with v29.
Pardon that, it's an artifact from trying to fetch 28.1.90 via url-
fetch and failing.  Note that the hash is the one for 28.1.

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Hi,

------- Original Message -------
On Saturday, August 6th, 2022 at 1:53 AM, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:

> Pardon that, it's an artifact from trying to fetch 28.1.90 via url-
> fetch and failing. Note that the hash is the one for 28.1.

On Andrew's channel I can change the version/commit/hash to 28.1 and it builds fine. Though this is using a git checkout at the 28.1 tag commit, but I don't think that matters here? (Emacs runs too, but only tried in guix shell and just to see it opens, didn't see if it compiles anything.)

And yet with your code I cannot. I tried using the phase from Andrew instead of your code, tried adding gcc to the native-inputs to match Andrew, used gcc-11 with libgccjit-11, ...still the same configure failure on the libgccjit test program: can't find libgccjit.so

Weird.

Adding just the libgccjit library path to LD_LIBRARY_PATH makes configure find libgccjit, but then fails to find libx11 in that same libgccjit test program. Instead, making LD_LIBRARY_PATH=LIBRARY_PATH fixes the configure and it finally fails on the runpath validation. Maybe cause of messing with LD_LIBRARY_PATH or could use the patch Andrew has? Anyway, wasn't looking into this.

I thought I had a similar problem with the test program you had started with, but in the end just the LIBRARY_PATH tweak was needed. I wish I could remember the combination of things I tried that also had it failing to find libgccjit. Or perhaps it was something libgccjit was linked to, but I hadn't explored?

So. I tried to see what was different between your and Andrew's code and I'm not seeing what it could be. There must be some subtle difference we're missing? Seems we have all the ingredients and a known working example.

John

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Sonntag, dem 07.08.2022 um 03:19 +0000 schrieb John Kehayias:
> So. I tried to see what was different between your and Andrew's code
> and I'm not seeing what it could be. There must be some subtle
> difference we're missing? Seems we have all the ingredients and a
> known working example.
It's a really stupid one.  Basically, the tests and really any
executable you try to build fails to execute without LD_LIBRARY_PATH
set.  Now obviously, that's an issue with ld and you know which package
has ld?  That's right, it's binutils!

In Andrew's recipe, he sneakily snarfed out binutils from inputs using
assoc-ref, but I'm using the new package style with (this-package-
input) to achieve the same thing, so I had to add binutils.  But this
now shadows ld-wrapper.  So we have to add ld-wrapper as well.  Now
this makes me question whether the driver options are actually sane or
whether we'd have to prepend ld-wrapper to those as well.

WDYT, Andrew?

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

------- Original Message -------
On Sunday, August 7th, 2022 at 9:59 AM, Liliana Marie Prikler wrote:

> It's a really stupid one. Basically, the tests and really any
> executable you try to build fails to execute without LD_LIBRARY_PATH
> set. Now obviously, that's an issue with ld and you know which package
> has ld? That's right, it's binutils!
>

Ah! That LD_LIBRARY_PATH was needed was the clue.

> In Andrew's recipe, he sneakily snarfed out binutils from inputs using
> assoc-ref, but I'm using the new package style with (this-package-
> input) to achieve the same thing, so I had to add binutils. But this
> now shadows ld-wrapper. So we have to add ld-wrapper as well. Now
> this makes me question whether the driver options are actually sane or
> whether we'd have to prepend ld-wrapper to those as well.
>

Is the assoc-ref for binutils (implicit input?) equivalent to just using #$binutils directly? e.g. (string-append "-B" #$binutils "/bin/") with no added binutils to the inputs. Is that not a good idea? As a test, that does indeed work, everything configures, builds, and runs (only tested opening with no configuration).

> WDYT, Andrew?

bug#55657: libgccjit is unusable

[Liliana Marie Prikler]

Am Sonntag, dem 07.08.2022 um 15:09 +0000 schrieb John Kehayias:
> Is the assoc-ref for binutils (implicit input?) equivalent to just
> using #$binutils directly? e.g. (string-append "-B" #$binutils
> "/bin/") with no added binutils to the inputs. Is that not a good
> idea? As a test, that does indeed work, everything configures,
> builds, and runs (only tested opening with no configuration).
Not quite, because this binutils is actually binutils-final from
commencement.scm, which it doesn't seem we can import the normal way. 
Also doing this would make it so that binutils isn't configurable,
which with an input you get.  The trick is just to also use make-ld-
wrapper from base.scm

Cheers

bug#55657: [PATCH 0/6] Add native compilation to Emacs

[Liliana Marie Prikler]

Hi Guix,

at long last the following patch set should enable native compilation
for both Emacs and emacs-build-system.  I tested emacs-dash and at the
very least native code is generated, though I haven't yet checked
whether it is also loaded.

As with any shiny new Emacs feature, please verify that the Emacs
portion of your manifests/home configurations build and report any
related errors *before* I push this and curse your configuration.

Cheers

Liliana Marie Prikler (6):
  gnu: Parameterize libgccjit.
  gnu: libgccjit: Build with bootstrapped gcc.
  gnu: libgccjit: Build multiple versions.
  gnu: emacs: Build with native compilation.
  guix: emacs-utils: Add emacs-compile-directory.
  build-system: emacs: Use native compilation.

 gnu/packages/emacs.scm            | 64 ++++++++++++++++++++++++++++++-
 gnu/packages/gcc.scm              | 53 +++++++++++++++++--------
 guix/build/emacs-build-system.scm |  5 ++-
 guix/build/emacs-utils.scm        | 26 +++++++++++++
 4 files changed, 128 insertions(+), 20 deletions(-)

-- 
2.37.0

bug#55657: libgccjit is unusable

[John Kehayias via Bug reports for GNU Guix]

Thanks for the new patchset Lily, very excited to have emacs with native compiliation in upstream Guix! (At the very least so I don't have to compile libgccjit and emacs)

For everyone following along at home, please see https://issues.guix.gnu.org/57086

As for the original issue here, I guess the LIBRARY_PATH and ld shadowing is the workaround. I don't know if that is something that can/should be incorporated into the libgccjit package definition or should just be handled by any package using it. Currently, that will just be emacs anyway and hopefully the discussion here is useful to anyone trying to use libgccjit.

Thanks again Lily and Andrew for your work here!

bug#55657: libgccjit is unusable

[Andrew Whatson]

John Kehayias wrote:
>
> As for the original issue here, I guess the LIBRARY_PATH and ld shadowing is the workaround. I don't know if that is something that can/should be incorporated into the libgccjit package definition or should just be handled by any package using it. Currently, that will just be emacs anyway and hopefully the discussion here is useful to anyone trying to use libgccjit.

This might be possible by adding LIBRARY_PATH to native-search-paths
in the libgccjit package definition?

> Thanks again Lily and Andrew for your work here!

Thanks John & Lily for persisting with getting this submitted, I'm
very grateful for your efforts.

Cheers,
Andrew