Re: On elisp running native

[Andrea Corallo <akrl@sdf.org>]

Adam Porter <adam@alphapapa.net> writes:
>
> #+RESULTS:
> | 5.819892666 | 1 | 0.6101534569999956 |  In Emacs 26.3
> | 2.317804428 | 3 | 0.6223487619999997 |  In Emacs 28 with native-comp
>
> Then I tested this more complex query and observed an even more
> impressive speedup of about 3.4x!
>
>   '(and (not (done))
>         (or (habit)
>             (deadline auto)
>             (scheduled :to today)
>             (ts-active :on today)))
>
> #+RESULTS:
> | 8.377843199 | 1 | 0.6277314330000081 |  In Emacs 26.3
> | 2.436048375 | 2 | 0.4278436059999997 |  In Emacs 28 with native-comp
>
> This is very exciting!  The difference between 8.4 seconds and 2.4
> seconds is a huge usability improvement.

Wow these are very good results, it's interesting to collect feedback
from real world scenarios, thanks.

> I did encounter a minor issue related to macro-expansion and loading
> that may be my fault.  When I tried to (require 'org-ql) or call a
> function that is autoloaded from it, I got an error saying that the .eln
> file did not provide the org-ql feature.  I surmised that meant that the
> file hadn't been compiled properly, so I looked at the async compilation
> log and saw an error saying that the variable org-ql-predicates was not
> defined.  At this URL you can see a macro definition,
> org-ql--def-plain-query-fn, and the call to it that's wrapped in
> cl-eval-when to ensure it works properly with the byte-compiler:
>
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L1671
>
> The macro uses the value of org-ql-predicates, which is defined here:
>
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L126
>
> But when the org-ql--defpred macro:
>
> https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L140
>
> ...is called, it adds to that variable.  So all of that works with the
> byte-compiler, but apparently not with the native compilation.
>
> To work around it, I wrapped (defvar org-ql-predicates...) in
> eval-and-compile, and then wrapped all of the calls to org-ql--defpred
> in eval-and-compile as well.  Then I deleted the org-ql.eln file and
> recompiled it, restarted Emacs, and then everything worked fine.
>
> I'm guessing that this issue is "native" to the native-compilation and
> might need to be documented in some way, but I wouldn't expect it to
> affect many packages, only ones that do tricky things with variables and
> macro-expansion like this.

This is strange.  The semantic of the native compiled code should be
exactly the same of the byte-compiled one.  I may look at this in the
weekend if I manage to.  Thanks for reporting.

> So, a few ideas for fixes or improvements:
>
> 1.  I guess the .eln file should not have been produced or left behind
>     in the directory, since there was a compilation error.

Agree, in case of error no eln should have been produced.  I thought it
was the case but if it's not that's a bug.

> 2.  The compilation error should probably be shown as a warning or
>     something, because it's easily lost in the async compilation log
>     buffer.

Are you suggesting to use the minibuffer to report these?

> 3.  It would probably be good if the loader fell back to .elc and .el
>     files if the .eln file doesn't seem to work, and showed a warning
>     when doing so.  The library would remain usable in its byte-compiled
>     form, and in the case of a package like this, the user could then
>     report the native-compilation error to the developer.
>
> Other than that minor issue, everything works and feels very fast!  This
> is great!
>
>>   ~eln~ files are compiled in specific sub-folders taking in account
>>   host architecture and current Emacs configuration to disambiguate
>>   possible incompatibilities.
>
>>  As example a file ~.../foo.el~ will compile into something like
>>  ~.../x86_64-pc-linux-gnu-12383a81d4f33a87/foo.eln~.
>
> FYI, I did not see the .eln files being put in subdirectories like that
> in the Docker image.  Maybe it didn't receive that update yet.

Ooops, should be ok by now.

> This is all very exciting.  I'm imagining an update to package.el that
> would byte-compile packages immediately, as usual, then start
> native-compile-async'ing them in the background, loading each file's
> native version as it completes, with the packages being immediately
> usable and seeming to get faster as each file is compiled and loaded.

Yes that's the idea but I'd like to make it not package.el specific but
for every .elc being loaded in Emacs.

https://lists.gnu.org/archive/html/emacs-devel/2020-02/msg00225.html

Thanks for your mail is great to have feedback to work on.

  Andrea

--
akrl@sdf.org