From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Andrea Corallo Newsgroups: gmane.emacs.devel Subject: Re: On elisp running native Date: Thu, 05 Mar 2020 13:19:31 +0000 Message-ID: References: <83tv5mp48l.fsf@gnu.org> <83sgl0lchm.fsf@gnu.org> <83imlwl9vm.fsf@gnu.org> <83o8uegykm.fsf@gnu.org> <74dd94a9-28cb-a5fd-dbc7-ab21009834ad@cs.ucla.edu> <87mu8vjcmm.fsf@alphapapa.net> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="99043"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) Cc: emacs-devel@gnu.org To: Adam Porter Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Mar 05 14:20:12 2020 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j9qQ8-000Pei-1t for ged-emacs-devel@m.gmane-mx.org; Thu, 05 Mar 2020 14:20:12 +0100 Original-Received: from localhost ([::1]:49044 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j9qQ7-00046t-0f for ged-emacs-devel@m.gmane-mx.org; Thu, 05 Mar 2020 08:20:11 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:48800) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j9qPb-0003hv-Iq for emacs-devel@gnu.org; Thu, 05 Mar 2020 08:19:40 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j9qPZ-0004KP-PX for emacs-devel@gnu.org; Thu, 05 Mar 2020 08:19:39 -0500 Original-Received: from mx.sdf.org ([205.166.94.20]:50939) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1j9qPZ-0004JL-GN for emacs-devel@gnu.org; Thu, 05 Mar 2020 08:19:37 -0500 Original-Received: from sdf.org (ma.sdf.org [205.166.94.33]) by mx.sdf.org (8.15.2/8.14.5) with ESMTPS id 025DJWbk005698 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256 bits) verified NO); Thu, 5 Mar 2020 13:19:32 GMT Original-Received: (from akrl@localhost) by sdf.org (8.15.2/8.12.8/Submit) id 025DJVrW031507; Thu, 5 Mar 2020 13:19:31 GMT In-Reply-To: <87mu8vjcmm.fsf@alphapapa.net> (Adam Porter's message of "Thu, 05 Mar 2020 04:54:09 -0600") X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 205.166.94.20 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.io gmane.emacs.devel:245261 Archived-At: Adam Porter writes: > > #+RESULTS: > | 5.819892666 | 1 | 0.6101534569999956 | In Emacs 26.3 > | 2.317804428 | 3 | 0.6223487619999997 | In Emacs 28 with native-comp > > Then I tested this more complex query and observed an even more > impressive speedup of about 3.4x! > > '(and (not (done)) > (or (habit) > (deadline auto) > (scheduled :to today) > (ts-active :on today))) > > #+RESULTS: > | 8.377843199 | 1 | 0.6277314330000081 | In Emacs 26.3 > | 2.436048375 | 2 | 0.4278436059999997 | In Emacs 28 with native-comp > > This is very exciting! The difference between 8.4 seconds and 2.4 > seconds is a huge usability improvement. Wow these are very good results, it's interesting to collect feedback from real world scenarios, thanks. > I did encounter a minor issue related to macro-expansion and loading > that may be my fault. When I tried to (require 'org-ql) or call a > function that is autoloaded from it, I got an error saying that the .eln > file did not provide the org-ql feature. I surmised that meant that the > file hadn't been compiled properly, so I looked at the async compilation > log and saw an error saying that the variable org-ql-predicates was not > defined. At this URL you can see a macro definition, > org-ql--def-plain-query-fn, and the call to it that's wrapped in > cl-eval-when to ensure it works properly with the byte-compiler: > > https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L1671 > > The macro uses the value of org-ql-predicates, which is defined here: > > https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L126 > > But when the org-ql--defpred macro: > > https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L140 > > ...is called, it adds to that variable. So all of that works with the > byte-compiler, but apparently not with the native compilation. > > To work around it, I wrapped (defvar org-ql-predicates...) in > eval-and-compile, and then wrapped all of the calls to org-ql--defpred > in eval-and-compile as well. Then I deleted the org-ql.eln file and > recompiled it, restarted Emacs, and then everything worked fine. > > I'm guessing that this issue is "native" to the native-compilation and > might need to be documented in some way, but I wouldn't expect it to > affect many packages, only ones that do tricky things with variables and > macro-expansion like this. This is strange. The semantic of the native compiled code should be exactly the same of the byte-compiled one. I may look at this in the weekend if I manage to. Thanks for reporting. > So, a few ideas for fixes or improvements: > > 1. I guess the .eln file should not have been produced or left behind > in the directory, since there was a compilation error. Agree, in case of error no eln should have been produced. I thought it was the case but if it's not that's a bug. > 2. The compilation error should probably be shown as a warning or > something, because it's easily lost in the async compilation log > buffer. Are you suggesting to use the minibuffer to report these? > 3. It would probably be good if the loader fell back to .elc and .el > files if the .eln file doesn't seem to work, and showed a warning > when doing so. The library would remain usable in its byte-compiled > form, and in the case of a package like this, the user could then > report the native-compilation error to the developer. > > Other than that minor issue, everything works and feels very fast! This > is great! > >> ~eln~ files are compiled in specific sub-folders taking in account >> host architecture and current Emacs configuration to disambiguate >> possible incompatibilities. > >> As example a file ~.../foo.el~ will compile into something like >> ~.../x86_64-pc-linux-gnu-12383a81d4f33a87/foo.eln~. > > FYI, I did not see the .eln files being put in subdirectories like that > in the Docker image. Maybe it didn't receive that update yet. Ooops, should be ok by now. > This is all very exciting. I'm imagining an update to package.el that > would byte-compile packages immediately, as usual, then start > native-compile-async'ing them in the background, loading each file's > native version as it completes, with the packages being immediately > usable and seeming to get faster as each file is compiled and loaded. Yes that's the idea but I'd like to make it not package.el specific but for every .elc being loaded in Emacs. https://lists.gnu.org/archive/html/emacs-devel/2020-02/msg00225.html Thanks for your mail is great to have feedback to work on. Andrea -- akrl@sdf.org