bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Robert Boyer]

[-- Attachment #1.1: Type: text/plain, Size: 6063 bytes --]

I show beyond all doubt how much Emacs Lisp needs, for its great
'native-compile',
'declare' and 'the' for fixnums and arrays.

Why? 8X!

If every time you aref a vector or increment a fixnum you have to check
things, you are going to waste a lot of cpu time. A programmer NEEDS to be
able to TELL the system things that should not be checked constantly.
Imagine
how slow one would be if he/she had to check that he/she was alive every
heart beat.

I have attached two files, eratosthenese.el and eratosthenese.lisp, the
first coded in Emacs Lisp and the second in Common Lisp.

In all that follows, I am running on a $100 Lenovo Chromebook.

The Sieve of Eratosthenese is the second oldest algorithm I am aware of, the
oldest being gcd.

Consider this form:

(progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt
10 8)) 1))

First of all, 'benchmark' has an obvious bug because it reports a time of
.000005 seconds.

So I used the timer on my trusty $130 moto g pure phone.

After finding the file eratosthenese.el, the evaluation of the form above
takes 69 seconds in Emacs.

After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt 10
8))
takes 8 seconds.

Q.E.D.

Bob

I attach stuff that I do not understand but that you may like.

From: bob <bob@penguin>
To: bug-gnu-emacs@gnu.org
Subject: 28.2; Why Emacs Lisp needs declare and the for fixnums and arrays.
--text follows this line--




In GNU Emacs 28.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.37,
cairo version 1.16.0)
 of 2023-05-13, modified by Debian built on x86-ubc-01
Windowing system distributor 'The X.Org Foundation', version 11.0.12014000
System Description: Debian GNU/Linux 12 (bookworm)

Configured using:
 'configure --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --build x86_64-linux-gnu --prefix=/usr
 --sharedstatedir=/var/lib --libexecdir=/usr/libexec
 --localstatedir=/var/lib --infodir=/usr/share/info
 --mandir=/usr/share/man --with-libsystemd --with-pop=yes
 --enable-locallisppath=/etc/emacs:/usr/local/share/emacs/28.2/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/28.2/site-lisp:/usr/share/emacs/site-lisp
 --with-sound=alsa --without-gconf --with-mailutils
 --with-native-compilation --with-cairo --with-x=yes
 --with-x-toolkit=gtk3 --with-toolkit-scroll-bars 'CFLAGS=-g -O2
 -ffile-prefix-map=/build/emacs-mPr7Vr/emacs-28.2+1=.
 -fstack-protector-strong -Wformat -Werror=format-security -Wall'
 'CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2' LDFLAGS=-Wl,-z,relro'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF
TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail thingatpt
dabbrev misearch multi-isearch help-fns radix-tree cl-print debug
backtrace find-func time-date compile text-property-search comp
comp-cstr warnings rx cl-extra help-mode cus-start etags fileloop
generator xref project dired-aux cus-edit pp cus-load wid-edit trace
sh-script smie executable dired dired-loaddefs cal-menu calendar
cal-loaddefs ange-ftp shell pcomplete comint ansi-color ring benchmark
time rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums mm-util mail-prsvr
mail-utils face-remap finder-inf package browse-url url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util mailcap url-handlers url-parse auth-source cl-seq eieio
eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
url-vars seq byte-opt gv bytecomp byte-compile cconv cl-loaddefs cl-lib
iso-transl tooltip eldoc paren electric uniquify ediff-hook vc-hooks
lisp-float-type elisp-mode mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite emoji-zwj charscript
charprop case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice button loaddefs faces cus-face macroexp files
window text-properties overlay sha1 md5 base64 format env code-pages
mule custom widget hashtable-print-readable backquote threads dbusbind
inotify lcms2 dynamic-setting system-font-setting font-render-setting
cairo move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 178787 77633)
 (symbols 48 13676 66)
 (strings 32 42888 8138)
 (string-bytes 1 1386823)
 (vectors 16 26477)
 (vector-slots 8 100493063 154471)
 (floats 8 62 377)
 (intervals 56 2423 424)
 (buffers 992 31))

[-- Attachment #1.2: Type: text/html, Size: 6864 bytes --]

[-- Attachment #2: eratosthenes.el --]
[-- Type: application/octet-stream, Size: 2373 bytes --]

;;; This file eratosthenese.el was coded by Robert Boyer,
;;; roberstephenboyer@gmail.com, in 2024, and is public domain.

;;;                           A Sieve of Eratosthenese

;;; Born c. 276 B. C. in Libya, Eratosthenese calculated the radius of the Earth
;;; and was the librarian of Alexandria.

;;; The build a sieve of Eratosthenese of length n, call (build-sieve n),
;;; after which i is prime if and only if (is-a-prime i).

;;; (build-sieve (expt 10 8)) takes 1600 seconds to execute.

(defvar test-form '(progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt 10 8)) 1)))

(cl-declaim (optimize (safety 0) (speed 3) (debug 0) (space 0)))

(defvar *sieve-length* 0)

(setq *sieve-length* 0)

(defvar *sieve* (make-vector 1 0))

(setq *sieve* (make-vector 1 0))

(cl-defun build-sieve (n)
  "After executing (build-sieve n), if j is a nonnegative integer less than
   *sieve-length*, then j is prime <-> (is-a-prime j)."
  (setq *sieve* (make-vector n 0))
  (setq *sieve-length* n)
  ;; Neither 0 nor 1 is prime.
  (setf (aref *sieve* 0) 1)
  (setf (aref *sieve* 1) 1)
  ;; Loop through a from 0 below l. If you encounter a 0 at file position
  ;; i, then '1 out' all the subsequent multiples of i.
  (cl-loop for i from 2 below *sieve-length* do
           (cond ((eql 0 (aref *sieve* i))
                  (cl-loop for j from (ash i 1) below *sieve-length* by i
                           do (setf (aref *sieve* j) 1))))))

(cl-defun test-sieve (n)
  (cond ((not (< n *sieve-length*)) (error "n is too big.")))
  (cl-loop for i from 2 to n do
           (let ((number-of-divisors (cl-loop for j from 2 to i when (eql 0 (% i j)) count 1)))
             (cond ((eql (aref *sieve* i) 1)
                    (cond ((not (< 1 number-of-divisors))
                           (error "bad at %s." i))))
                   ((eql (aref *sieve* i) 0)
                    (cond ((not (eql 1 number-of-divisors))
                           (error "bad at %s." i))))
                   (t (error "ridiculous")))))
  'ok)

(defun is-a-prime (n)
  (cond ((eql 0 (aref *sieve* 0))
         (error "It looks like (build-sieve) has not been run.")))
  (cond ((< n *sieve-length*)
         (eql 0 (aref *sieve* n)))
        (t (error "need a bigger sieve"))))

\f
;;; Local Variables: ;;;
;;; mode: Emacs-Lisp ;;;
;;; coding: utf-8    ;;;
;;; End:             ;;;

[-- Attachment #3: eratosthenes.lisp --]
[-- Type: application/x-lisp, Size: 2857 bytes --]

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Robert Boyer <robertstephenboyer@gmail.com>
> Date: Thu, 29 Feb 2024 13:40:14 -0600
> 
> Consider this form:
> 
> (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt 10 8)) 1))
> 
> First of all, 'benchmark' has an obvious bug because it reports a time of
> .000005 seconds.

You use benchmark incorrectly.  And you should use benchmark-run
instead, anyway.

> After finding the file eratosthenese.el, the evaluation of the form above
> takes 69 seconds in Emacs.
> 
> After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt 10 8))
> takes 8 seconds.

It takes 16.7 sec on my system.

I see no bug here, native-compiled Emacs Lisp will never be able to
produce performance similar to SBCL, not the way native compilation is
implemented in Emacs.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

> You use benchmark incorrectly.

Huh?  Here is the documentation.

benchmark is an interactive native-compiled Lisp function in ‘benchmark.el’.

(benchmark REPETITIONS FORM)

Print the time taken for REPETITIONS executions of FORM.
Interactively, REPETITIONS is taken from the prefix arg, and
the command prompts for the form to benchmark.
For non-interactive use see also ‘benchmark-run’ and
‘benchmark-run-compiled’.
FORM can also be a function in which case we measure the time it takes
to call it without any argument.

  Probably introduced at or before Emacs version 22.1.

 > And you should use benchmark-run instead, anyway.

Thanks for that.  I'll give it a try.

>  It takes 16.7 sec on my system.

All of my times come from using a $100 Lenovo Chromebook.  How much did
your machine
cost?

> Emacs Lisp will never be able to produce performance similar to SBCL, not
the way
> native compilation implemented in Emacs.

Wanna bet?  There are too many geniuses working on Emacs to let this chance
for 8X pass.

Thanks for your reply.  My message was inspired by something someone said
when I sent
in a much simpler demo of the same 8x point, saying that one needed to show
the difference  on
real code.  So I have done so, I hope everyone agrees. Who is more famous
than Eratosthenese?

Bob


On Thu, Feb 29, 2024 at 2:11 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Robert Boyer <robertstephenboyer@gmail.com>
> > Date: Thu, 29 Feb 2024 13:40:14 -0600
> >
> > Consider this form:
> >
> > (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve
> (expt 10 8)) 1))
> >
> > First of all, 'benchmark' has an obvious bug because it reports a time of
> > .000005 seconds.
>
> You use benchmark incorrectly.  And you should use benchmark-run
> instead, anyway.
>
> > After finding the file eratosthenese.el, the evaluation of the form above
> > takes 69 seconds in Emacs.
> >
> > After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt
> 10 8))
> > takes 8 seconds.
>
> It takes 16.7 sec on my system.
>
> I see no bug here, native-compiled Emacs Lisp will never be able to
> produce performance similar to SBCL, not the way native compilation is
> implemented in Emacs.
>

[-- Attachment #2: Type: text/html, Size: 3085 bytes --]

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Robert Boyer]

[-- Attachment #1: Type: text/plain, Size: 1148 bytes --]

benchmark-run-compiled reported the same sorry time to *Messages* on:

(progn (emacs-lisp-native-compile-and-load) (benchmark-run-compiled
(build-sieve (expt 10 8))))

(68.42455689 1 1.1884078129999995)

On Thu, Feb 29, 2024 at 2:11 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Robert Boyer <robertstephenboyer@gmail.com>
> > Date: Thu, 29 Feb 2024 13:40:14 -0600
> >
> > Consider this form:
> >
> > (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve
> (expt 10 8)) 1))
> >
> > First of all, 'benchmark' has an obvious bug because it reports a time of
> > .000005 seconds.
>
> You use benchmark incorrectly.  And you should use benchmark-run
> instead, anyway.
>
> > After finding the file eratosthenese.el, the evaluation of the form above
> > takes 69 seconds in Emacs.
> >
> > After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt
> 10 8))
> > takes 8 seconds.
>
> It takes 16.7 sec on my system.
>
> I see no bug here, native-compiled Emacs Lisp will never be able to
> produce performance similar to SBCL, not the way native compilation is
> implemented in Emacs.
>

[-- Attachment #2: Type: text/html, Size: 1633 bytes --]

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Michael Heerdegen via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Robert Boyer <robertstephenboyer@gmail.com> writes:

> Huh?  Here is the documentation.
>
> benchmark is an interactive native-compiled Lisp function in
> ‘benchmark.el’.
>
> (benchmark REPETITIONS FORM)
                         ^^^^

It's a _function_, Robert.  So you are passing the evaluation result,
not a form.

Michael.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Robert Boyer <robertstephenboyer@gmail.com>
> Date: Thu, 29 Feb 2024 14:54:46 -0600
> Cc: 69480@debbugs.gnu.org
> 
> > You use benchmark incorrectly. 
> 
> Huh?  Here is the documentation.
> 
> benchmark is an interactive native-compiled Lisp function in ‘benchmark.el’.
> 
> (benchmark REPETITIONS FORM)
> 
> Print the time taken for REPETITIONS executions of FORM.
> Interactively, REPETITIONS is taken from the prefix arg, and
> the command prompts for the form to benchmark.
> For non-interactive use see also ‘benchmark-run’ and
> ‘benchmark-run-compiled’.
> FORM can also be a function in which case we measure the time it takes
> to call it without any argument.

Yes, and in your recipe you reversed the order of the arguments:

  (benchmark (build-sieve (expt 10 8)) 1)

> >  It takes 16.7 sec on my system.
> 
> All of my times come from using a $100 Lenovo Chromebook.  How much did your machine
> cost?

I gave my timing as another data point.  I have no way of measuring
the timing on your machine, obviously.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Thu, 29 Feb 2024 22:10:27 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Robert Boyer <robertstephenboyer@gmail.com>
>> Date: Thu, 29 Feb 2024 13:40:14 -0600
>>
>> Consider this form:
>>
>> (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt 10
>> 8)) 1))
>>
>> First of all, 'benchmark' has an obvious bug because it reports a time of
>> .000005 seconds.
>
> You use benchmark incorrectly.  And you should use benchmark-run
> instead, anyway.
>
>> After finding the file eratosthenese.el, the evaluation of the form above
>> takes 69 seconds in Emacs.
>>
>> After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt 10 8))
>> takes 8 seconds.
>
> It takes 16.7 sec on my system.

I was curious to see how long it takes on my system, compared to the
byte-compiled and uncompiled files, and the results surprised me.
First, I visited the file eratosthenes.el, ran `eval-buffer' and then
`M-: (benchmark-run nil (build-sieve (expt 10 8)))', with this result:

(143.326808051 1 0.344846223)

Then I ran `M-: (progn (emacs-lisp-native-compile-and-load)
(benchmark-run nil (build-sieve (expt 10 8))))', with this result:

(37.457440511 1 0.36922945500000004)

The native compilation also produced a byte-compiled file
eratosthenes.elc, so I then loaded that file and again ran `M-:
(benchmark-run nil (build-sieve (expt 10 8)))', with this result:

(21.854069551000002 1 0.3595161699999999)

I was surprised that this was much faster than the run with native
compilation, but I thought perhaps this was due to the time spent
producing both the native and byte-compiled files (though more than 15
seconds for that seemed unlikely), so I ran `M-: (progn
(emacs-lisp-native-compile-and-load) (benchmark-run nil (build-sieve
(expt 10 8))))' again, which just loaded the already native-compiled
file (and updated its timestamp), but the result was practically the
same as the first time:

(37.095767574 1 0.36986937500000017)

Why is the timing with native compilation so much slower than with byte
compilation?

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Andrea Corallo]

Stephen Berman via "Bug reports for GNU Emacs, the Swiss army knife of
text editors" <bug-gnu-emacs@gnu.org> writes:

> On Thu, 29 Feb 2024 22:10:27 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>
>>> From: Robert Boyer <robertstephenboyer@gmail.com>
>>> Date: Thu, 29 Feb 2024 13:40:14 -0600
>>>
>>> Consider this form:
>>>
>>> (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt 10
>>> 8)) 1))
>>>
>>> First of all, 'benchmark' has an obvious bug because it reports a time of
>>> .000005 seconds.
>>
>> You use benchmark incorrectly.  And you should use benchmark-run
>> instead, anyway.
>>
>>> After finding the file eratosthenese.el, the evaluation of the form above
>>> takes 69 seconds in Emacs.
>>>
>>> After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt 10 8))
>>> takes 8 seconds.
>>
>> It takes 16.7 sec on my system.
>
> I was curious to see how long it takes on my system, compared to the
> byte-compiled and uncompiled files, and the results surprised me.
> First, I visited the file eratosthenes.el, ran `eval-buffer' and then
> `M-: (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
>
> (143.326808051 1 0.344846223)
>
> Then I ran `M-: (progn (emacs-lisp-native-compile-and-load)
> (benchmark-run nil (build-sieve (expt 10 8))))', with this result:
>
> (37.457440511 1 0.36922945500000004)
>
> The native compilation also produced a byte-compiled file
> eratosthenes.elc, so I then loaded that file and again ran `M-:
> (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
>
> (21.854069551000002 1 0.3595161699999999)
>
> I was surprised that this was much faster than the run with native
> compilation, but I thought perhaps this was due to the time spent
> producing both the native and byte-compiled files (though more than 15
> seconds for that seemed unlikely), so I ran `M-: (progn
> (emacs-lisp-native-compile-and-load) (benchmark-run nil (build-sieve
> (expt 10 8))))' again, which just loaded the already native-compiled
> file (and updated its timestamp), but the result was practically the
> same as the first time:
>
> (37.095767574 1 0.36986937500000017)
>
> Why is the timing with native compilation so much slower than with byte
> compilation?

Because the file is not lexically scoped so we are not running
optimizations.

On my laptop I see:

11 secs byte compiled
18 secs native compiled

7 secs byte compiled with lexical scope
5 secs  native compiled with lexical scope.

Bests

  Andrea

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 07:18:29 -0500 Andrea Corallo <acorallo@gnu.org> wrote:

> Stephen Berman via "Bug reports for GNU Emacs, the Swiss army knife of
> text editors" <bug-gnu-emacs@gnu.org> writes:
>
>> On Thu, 29 Feb 2024 22:10:27 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>
>>>> From: Robert Boyer <robertstephenboyer@gmail.com>
>>>> Date: Thu, 29 Feb 2024 13:40:14 -0600
>>>>
>>>> Consider this form:
>>>>
>>>> (progn (emacs-lisp-native-compile-and-load) (benchmark (build-sieve (expt 10
>>>> 8)) 1))
>>>>
>>>> First of all, 'benchmark' has an obvious bug because it reports a time of
>>>> .000005 seconds.
>>>
>>> You use benchmark incorrectly.  And you should use benchmark-run
>>> instead, anyway.
>>>
>>>> After finding the file eratosthenese.el, the evaluation of the form above
>>>> takes 69 seconds in Emacs.
>>>>
>>>> After entering SBCL and loading eratosthenese.lisp, (build-sieve (expt 10 8))
>>>> takes 8 seconds.
>>>
>>> It takes 16.7 sec on my system.
>>
>> I was curious to see how long it takes on my system, compared to the
>> byte-compiled and uncompiled files, and the results surprised me.
>> First, I visited the file eratosthenes.el, ran `eval-buffer' and then
>> `M-: (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
>>
>> (143.326808051 1 0.344846223)
>>
>> Then I ran `M-: (progn (emacs-lisp-native-compile-and-load)
>> (benchmark-run nil (build-sieve (expt 10 8))))', with this result:
>>
>> (37.457440511 1 0.36922945500000004)
>>
>> The native compilation also produced a byte-compiled file
>> eratosthenes.elc, so I then loaded that file and again ran `M-:
>> (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
>>
>> (21.854069551000002 1 0.3595161699999999)
>>
>> I was surprised that this was much faster than the run with native
>> compilation, but I thought perhaps this was due to the time spent
>> producing both the native and byte-compiled files (though more than 15
>> seconds for that seemed unlikely), so I ran `M-: (progn
>> (emacs-lisp-native-compile-and-load) (benchmark-run nil (build-sieve
>> (expt 10 8))))' again, which just loaded the already native-compiled
>> file (and updated its timestamp), but the result was practically the
>> same as the first time:
>>
>> (37.095767574 1 0.36986937500000017)
>>
>> Why is the timing with native compilation so much slower than with byte
>> compilation?
>
> Because the file is not lexically scoped so we are not running
> optimizations.

Ah, ok.

> On my laptop I see:
>
> 11 secs byte compiled
> 18 secs native compiled
>
> 7 secs byte compiled with lexical scope
> 5 secs  native compiled with lexical scope.

I got these timings with lexical scope:

native compiled: (12.717023709000001 1 0.34653286699999997)
  byte compiled: (9.597325325 1 0.38987931)

It's still slower with native compilation than with byte compilation on
my machine, unlike on yours.  Do you have any idea why?

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: Robert Boyer <robertstephenboyer@gmail.com>,  69480@debbugs.gnu.org
> Date: Fri, 01 Mar 2024 12:28:30 +0100
> 
> On Thu, 29 Feb 2024 22:10:27 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
> 
> > It takes 16.7 sec on my system.
> 
> I was curious to see how long it takes on my system, compared to the
> byte-compiled and uncompiled files, and the results surprised me.
> First, I visited the file eratosthenes.el, ran `eval-buffer' and then
> `M-: (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
> 
> (143.326808051 1 0.344846223)
> 
> Then I ran `M-: (progn (emacs-lisp-native-compile-and-load)
> (benchmark-run nil (build-sieve (expt 10 8))))', with this result:
> 
> (37.457440511 1 0.36922945500000004)
> 
> The native compilation also produced a byte-compiled file
> eratosthenes.elc, so I then loaded that file and again ran `M-:
> (benchmark-run nil (build-sieve (expt 10 8)))', with this result:
> 
> (21.854069551000002 1 0.3595161699999999)
> 
> I was surprised that this was much faster than the run with native
> compilation, but I thought perhaps this was due to the time spent
> producing both the native and byte-compiled files (though more than 15
> seconds for that seemed unlikely), so I ran `M-: (progn
> (emacs-lisp-native-compile-and-load) (benchmark-run nil (build-sieve
> (expt 10 8))))' again, which just loaded the already native-compiled
> file (and updated its timestamp), but the result was practically the
> same as the first time:
> 
> (37.095767574 1 0.36986937500000017)
> 
> Why is the timing with native compilation so much slower than with byte
> compilation?

That's not what I see here (in an optimized build of Emacs 29.2).  On
my system, the byte-compiled version and the native-compiled version
yield almost the same times: 16.12 vs 16.5 sec.  The way native
compilation is currently implemented, whether it produces a
significant speedup depends on the actual Lisp code, and having the
same or even slightly slower performance is not unheard of.  So I
don't see anything unexpected on my system, at least in Emacs 29.

Adding Andrea, in case he has comments to this.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Andrea Corallo <acorallo@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Stephen Berman <stephen.berman@gmx.net>,
>   Robert Boyer <robertstephenboyer@gmail.com>,  69480@debbugs.gnu.org
> Date: Fri, 01 Mar 2024 07:18:29 -0500
> 
> > Why is the timing with native compilation so much slower than with byte
> > compilation?
> 
> Because the file is not lexically scoped so we are not running
> optimizations.
> 
> On my laptop I see:
> 
> 11 secs byte compiled
> 18 secs native compiled
> 
> 7 secs byte compiled with lexical scope
> 5 secs  native compiled with lexical scope.

Right, thanks.

My timings are 6.35 and 6.28 sec accordingly (down from ~16.5).

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: Stephen Berman via "Bug reports for GNU Emacs, the Swiss army knife of
>  text editors" <bug-gnu-emacs@gnu.org>,  Eli Zaretskii <eliz@gnu.org>,
>   Robert Boyer <robertstephenboyer@gmail.com>,  69480@debbugs.gnu.org
> Date: Fri, 01 Mar 2024 13:33:56 +0100
> 
> On Fri, 01 Mar 2024 07:18:29 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
> 
> >> Why is the timing with native compilation so much slower than with byte
> >> compilation?
> >
> > Because the file is not lexically scoped so we are not running
> > optimizations.
> 
> Ah, ok.
> 
> > On my laptop I see:
> >
> > 11 secs byte compiled
> > 18 secs native compiled
> >
> > 7 secs byte compiled with lexical scope
> > 5 secs  native compiled with lexical scope.
> 
> I got these timings with lexical scope:
> 
> native compiled: (12.717023709000001 1 0.34653286699999997)
>   byte compiled: (9.597325325 1 0.38987931)
> 
> It's still slower with native compilation than with byte compilation on
> my machine, unlike on yours.  Do you have any idea why?

Maybe your system takes time to allocate memory?  This program
allocates almost 850MB of memory on my system, so maybe yours starts
paging?

Anyway, profiling could give some hints.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 14:45:39 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: Stephen Berman via "Bug reports for GNU Emacs, the Swiss army knife of
>>  text editors" <bug-gnu-emacs@gnu.org>,  Eli Zaretskii <eliz@gnu.org>,
>>   Robert Boyer <robertstephenboyer@gmail.com>,  69480@debbugs.gnu.org
>> Date: Fri, 01 Mar 2024 13:33:56 +0100
>>
>> On Fri, 01 Mar 2024 07:18:29 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
>>
>> >> Why is the timing with native compilation so much slower than with byte
>> >> compilation?
>> >
>> > Because the file is not lexically scoped so we are not running
>> > optimizations.
>>
>> Ah, ok.
>>
>> > On my laptop I see:
>> >
>> > 11 secs byte compiled
>> > 18 secs native compiled
>> >
>> > 7 secs byte compiled with lexical scope
>> > 5 secs  native compiled with lexical scope.
>>
>> I got these timings with lexical scope:
>>
>> native compiled: (12.717023709000001 1 0.34653286699999997)
>>   byte compiled: (9.597325325 1 0.38987931)
>>
>> It's still slower with native compilation than with byte compilation on
>> my machine, unlike on yours.  Do you have any idea why?
>
> Maybe your system takes time to allocate memory?  This program
> allocates almost 850MB of memory on my system, so maybe yours starts
> paging?

I see a similar allocation, but I have 16 GB of RAM (~12 of which were
free when I ran the benchmark), so I think that shouldn't be an issue.

> Anyway, profiling could give some hints.

That says almost all memory and CPU are consumed by `build-sieve':

    807,918,497  99% - command-execute
    806,047,846  99%  - funcall-interactively
    804,440,179  99%   - eval-expression
    804,440,179  99%    - #<compiled -0x5db3e1955cb81d1>
    804,440,179  99%     - #<compiled -0x88d7268f58840fe>
    804,440,179  99%      - eval
    804,440,179  99%       - progn
    800,000,184  99%        - benchmark-call
    800,000,104  99%         - #<lambda 0x8b1d56ce50812>
    800,000,104  99%          - build-sieve
             40   0%             expt
             80   0%           time-since
      4,439,995   0%        + emacs-lisp-native-compile-and-load
      1,607,667   0%   + execute-extended-command
      1,870,651   0%  + byte-code
         99,848   0% + redisplay_internal (C function)
          1,729   0%   Automatic GC
          1,416   0% + timer-event-handler
          1,180   0% + ...

But I don't see how that bears on the difference between native and byte
compilation in my timings versus yours and Andrea's.

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: acorallo@gnu.org,  bug-gnu-emacs@gnu.org,  robertstephenboyer@gmail.com,
>   69480@debbugs.gnu.org
> Date: Fri, 01 Mar 2024 14:07:50 +0100
> 
> On Fri, 01 Mar 2024 14:45:39 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
> 
> >> It's still slower with native compilation than with byte compilation on
> >> my machine, unlike on yours.  Do you have any idea why?
> >
> > Maybe your system takes time to allocate memory?  This program
> > allocates almost 850MB of memory on my system, so maybe yours starts
> > paging?
> 
> I see a similar allocation, but I have 16 GB of RAM (~12 of which were
> free when I ran the benchmark), so I think that shouldn't be an issue.
> 
> > Anyway, profiling could give some hints.
> 
> That says almost all memory and CPU are consumed by `build-sieve':

This is skewed, use cpu profiling instead of "memory" profiling.

> But I don't see how that bears on the difference between native and byte
> compilation in my timings versus yours and Andrea's.

My suggestion was to compare profiles in the byte-compiled and
native-compiled cases.

Btw, are you running both cases in the same session?  If so, don't:
restart Emacs and run the other case instead.

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: acorallo@gnu.org,  bug-gnu-emacs@gnu.org,  robertstephenboyer@gmail.com,
>>   69480@debbugs.gnu.org
>> Date: Fri, 01 Mar 2024 14:07:50 +0100
>>
>> On Fri, 01 Mar 2024 14:45:39 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>
>> >> It's still slower with native compilation than with byte compilation on
>> >> my machine, unlike on yours.  Do you have any idea why?
>> >
>> > Maybe your system takes time to allocate memory?  This program
>> > allocates almost 850MB of memory on my system, so maybe yours starts
>> > paging?
>>
>> I see a similar allocation, but I have 16 GB of RAM (~12 of which were
>> free when I ran the benchmark), so I think that shouldn't be an issue.
>>
>> > Anyway, profiling could give some hints.
>>
>> That says almost all memory and CPU are consumed by `build-sieve':
>
> This is skewed, use cpu profiling instead of "memory" profiling.

Ah, ok.

>> But I don't see how that bears on the difference between native and byte
>> compilation in my timings versus yours and Andrea's.
>
> My suggestion was to compare profiles in the byte-compiled and
> native-compiled cases.
>
> Btw, are you running both cases in the same session?  If so, don't:
> restart Emacs and run the other case instead.

Ok, I've now done that.  Here's the report for the run with native
compilation:

       12599  95% - command-execute
       12487  95%  - funcall-interactively
       12486  95%   - eval-expression
       12485  95%    - #<compiled -0x5db3e1955cb81d1>
       12485  95%     - #<compiled -0x8a5cf032951a0fe>
       12480  95%      - eval
       12480  95%       - progn
       12367  94%        - benchmark-call
       12367  94%         - #<lambda 0x8c97b8cb7bd82>
       12367  94%            build-sieve
         113   0%        - emacs-lisp-native-compile-and-load
         113   0%         - emacs-lisp-native-compile
         113   0%          - native-compile
          79   0%           - comp--native-compile
          37   0%            - comp--fwprop
          25   0%             - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_109>
          25   0%              - comp--fwprop*
          25   0%               - maphash
          25   0%                - #<compiled 0xf5cda0529cb6e6>
          22   0%                 - comp--fwprop-insn
          20   0%                  - comp-cstr-intersection
          19   0%                   - comp-cstr-intersection-no-mem
          19   0%                    - apply
          19   0%                     - comp-cstr-intersection-homogeneous
          19   0%                      - apply
          19   0%                       - comp-intersect-typesets
          19   0%                        - comp-normalize-typeset
          19   0%                         - comp--normalize-typeset0
          19   0%                          - comp--direct-subtypes
          18   0%                           - comp--direct-supertypes
          18   0%                            - comp-supertypes
           4   0%                               comp--intersection
           1   0%                           - cl-sort
           1   0%                              #<subr comp--sym-lessp>
           1   0%                     comp-cstr-copy
           2   0%                  - comp-cstr-union-no-range
           2   0%                   - comp-cstr-union-1
           2   0%                    - comp-cstr-union-1-no-mem
           2   0%                     - apply
           2   0%                      - comp-cstr-union-homogeneous
           2   0%                       - comp-cstr-union-homogeneous-no-range
           1   0%                        - comp-union-typesets
           1   0%                         - comp-normalize-typeset
           1   0%                          - comp--normalize-typeset0
           1   0%                           - comp--direct-subtypes
           1   0%                            - comp--direct-supertypes
           1   0%                               comp-supertypes
           1   0%                        - comp-normalize-valset
           1   0%                           cl-sort
           1   0%                 - comp--copy-insn
           1   0%                  - comp--copy-insn
           1   0%                     copy-comp-mvar
          10   0%             - comp--ssa
          10   0%              - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_100>
           7   0%               - comp--ssa-rename
           7   0%                - #<compiled 0x1cb53415437137ed>
           7   0%                 - #<compiled 0x1cb53415437137ed>
           4   0%                  - #<compiled 0x1cb55f52f37137ed>
           4   0%                   - #<compiled 0x1cb55f52f37137ed>
           4   0%                    - #<compiled 0x1cb55f52f37137ed>
           4   0%                     - #<compiled 0x1cb55f52f37137ed>
           4   0%                      - #<compiled 0x1cb55f52f37137ed>
           2   0%                       - comp--ssa-rename-insn
           2   0%                        - cl-nsubst-if
           2   0%                         - cl-nsublis
           2   0%                          - cl--nsublis-rec
           2   0%                           - cl--nsublis-rec
           1   0%                              #<compiled 0x293a3dbb887bc18>
           1   0%                              cl--nsublis-rec
           2   0%                       - #<compiled 0x1cb508a8837137ed>
           2   0%                        - #<compiled 0x1cb508a8837137ed>
           2   0%                         - #<compiled 0x1cb508a8837137ed>
           1   0%                          - #<compiled 0x1cb508a8837137ed>
           1   0%                           - comp--ssa-rename-insn
           1   0%                              comp--assign-op-p
           1   0%                          - comp--ssa-rename-insn
           1   0%                           - cl-nsubst-if
           1   0%                            - cl-nsublis
           1   0%                             - cl--nsublis-rec
           1   0%                                #<compiled 0x293a3d93887bc18>
           3   0%                  - comp--ssa-rename-insn
           3   0%                   - cl-nsubst-if
           3   0%                    - cl-nsublis
           3   0%                     - cl--nsublis-rec
           2   0%                      - cl--nsublis-rec
           1   0%                         #<compiled 0x293a3dbb887bc18>
           1   0%                       - cl--nsublis-rec
           1   0%                        - #<compiled 0x293a3dbb887bc18>
           1   0%                           type-of
           1   0%                        #<compiled 0x293a3dbb887bc18>
           1   0%                 comp--compute-dominator-tree
           1   0%               - comp--compute-edges
           1   0%                  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_80>
           1   0%               - comp--log-block-info
           1   0%                  #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_86>
           2   0%             - comp--dead-code
           2   0%              - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_115>
           2   0%                 comp--dead-assignments-func
          22   0%            - comp--spill-lap
          16   0%             - comp--spill-lap-function
          16   0%              - apply
          16   0%               - #<compiled -0x11c2c6042193c0b4>
          16   0%                - byte-compile-file
          15   0%                 - byte-compile-from-buffer
          15   0%                  - bytecomp--displaying-warnings
           9   0%                   - byte-compile-log-file
           9   0%                    - emacs-lisp-compilation-mode
           9   0%                     - compilation-mode
           5   0%                      - byte-code
           5   0%                       - require
           2   0%                        - byte-code
           2   0%                           require
           1   0%                        - custom-declare-face
           1   0%                           quote
           6   0%                   - #<compiled -0x14c9cececd5ff012>
           6   0%                    - byte-compile-toplevel-file-form
           6   0%                     - byte-compile-recurse-toplevel
           3   0%                      - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_50>
           3   0%                       - byte-compile-file-form
           2   0%                        - byte-compile-file-form-defvar
           2   0%                         - byte-compile-defvar
           2   0%                          - byte-compile-top-level
           2   0%                             byte-optimize-one-form
           1   0%                        - byte-compile-file-form-defalias
           1   0%                         - #<compiled -0x134743fab8beb490>
           1   0%                          - byte-compile-file-form-defmumble
           1   0%                             set-message-functions
           3   0%                      - macroexp-macroexpand
           3   0%                         macroexpand-1
           1   0%                 - normal-mode
           1   0%                  - set-auto-mode
           1   0%                     inhibit-local-variables-p
           6   0%             - comp-cstr-ctxt-update-type-slots
           3   0%              - comp--compute-typeof-types
           3   0%               - comp--all-classes
           3   0%                - #<compiled -0xc298594251b9291>
           1   0%                   cl-find-class
           3   0%              - comp--compute--pred-type-h
           3   0%               - comp--all-classes
           2   0%                - #<compiled -0xc298594251b9291>
           1   0%                   cl-find-class
          15   0%            - comp--final
          12   0%             - redisplay_internal (C function)
           1   0%              - jit-lock-function
           1   0%               - jit-lock-fontify-now
           1   0%                - jit-lock--run-functions
           1   0%                 - #<compiled 0x1baf465008c447bf>
           1   0%                  - font-lock-fontify-region
           1   0%                   - font-lock-default-fontify-region
           1   0%                    - font-lock-fontify-keywords-region
           1   0%                       lisp--match-hidden-arg
           1   0%              - redisplay--pre-redisplay-functions
           1   0%                 window-buffer
           2   0%               #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_130>
           3   0%            - make-comp-ctxt
           2   0%             - comp--compute-typeof-types
           2   0%                comp--all-classes
           1   0%             - comp--compute--pred-type-h
           1   0%              - comp--all-classes
           1   0%                 #<compiled -0xc298594251b9291>
           1   0%              time-since
           1   0%            - comp--limplify
           1   0%             - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_58>
           1   0%              - comp--limplify-function
           1   0%               - comp--limplify-block
           1   0%                  comp--limplify-lap-inst
          16   0%           - byte-code
          16   0%            - require
           4   0%             - byte-code
           4   0%              - require
           2   0%               - byte-code
           2   0%                - require
           1   0%                 - defvar
           1   0%                  - byte-code
           1   0%                     tool-bar-local-item-from-menu
          12   0%           - defconst
          12   0%            - byte-code
           6   0%             - make-comp-cstr-ctxt
           4   0%              - comp--compute-typeof-types
           4   0%               - comp--all-classes
           3   0%                - #<compiled -0xc298594251b9291>
           3   0%                   cl-find-class
           2   0%              - comp--compute--pred-type-h
           2   0%               - comp--all-classes
           2   0%                - #<compiled -0xc298594251b9291>
           1   0%                   cl-find-class
           6   0%             - comp-type-spec-to-cstr
           6   0%              - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_54>
           6   0%               - comp-type-spec-to-cstr
           5   0%                - comp-cstr-union-make
           5   0%                 - comp-cstr-union
           5   0%                  - comp-cstr-union-1
           5   0%                   - comp-cstr-union-1-no-mem
           5   0%                    - apply
           5   0%                     - comp-cstr-union-homogeneous
           5   0%                      - comp-cstr-union-homogeneous-no-range
           5   0%                       - comp-union-typesets
           5   0%                        - comp-normalize-typeset
           3   0%                         - comp--normalize-typeset0
           3   0%                          - comp--direct-subtypes
           3   0%                           - comp--direct-supertypes
           3   0%                              comp-supertypes
           2   0%                           cl-remove-duplicates
           1   0%                - comp-type-spec-to-cstr
           1   0%                 - comp-cstr-union-make
           1   0%                  - comp-cstr-union
           1   0%                   - comp-cstr-union-1
           1   0%                    - comp-cstr-union-1-no-mem
           1   0%                     - apply
           1   0%                      - comp-cstr-union-homogeneous
           1   0%                       - comp-cstr-union-homogeneous-no-range
           1   0%                        - comp-union-typesets
           1   0%                         - comp-normalize-typeset
           1   0%                          - comp--normalize-typeset0
           1   0%                           - comp--direct-subtypes
           1   0%                            - comp--direct-supertypes
           1   0%                             - comp-supertypes
           1   0%                                comp--intersection
           5   0%      - macroexpand-all
           5   0%       - macroexp--expand-all
           5   0%        - macroexp--all-forms
           5   0%         - macroexp--expand-all
           5   0%          - macroexp-macroexpand
           5   0%             macroexpand-1
           1   0%     execute-extended-command
         112   0%  - byte-code
          66   0%   - read--expression
          38   0%    - redisplay_internal (C function)
           1   0%       redisplay--pre-redisplay-functions
           5   0%    - command-execute
           5   0%     - funcall-interactively
           2   0%      - yank
           2   0%       - current-kill
           2   0%        - gui-selection-value
           2   0%         - gui--selection-value-internal
           2   0%          - gui-get-selection
           2   0%           - gui-backend-get-selection
           2   0%            - apply
           2   0%             - #<compiled 0xb78d7eebc4d3cfc>
           2   0%                x-get-selection-internal
           2   0%        delete-indentation
           1   0%      - read--expression-try-read
           1   0%         read
           2   0%    - timer-event-handler
           2   0%     - apply
           2   0%      - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_12>
           2   0%       - eldoc-print-current-symbol-info
           2   0%        - eldoc--invoke-strategy
           2   0%         - eldoc-documentation-default
           2   0%          - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_17>
           2   0%           - elisp-eldoc-funcall
           2   0%            - elisp--fnsym-in-current-sexp
           2   0%             - syntax-ppss
           2   0%                parse-partial-sexp
          46   0%   - read-extended-command
          46   0%    - read-extended-command-1
          46   0%     - completing-read-default
          26   0%      - redisplay_internal (C function)
           1   0%       - redisplay--pre-redisplay-functions
           1   0%        - run-hook-with-args
           1   0%           redisplay--update-region-highlight
           2   0%      - timer-event-handler
           2   0%       - apply
           2   0%        - #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_9>
           2   0%           jit-lock-context-fontify
         489   3%   Automatic GC
          15   0% + timer-event-handler
          14   0% + redisplay_internal (C function)
          13   0% + ...

And here's the report for the run with byte compilation:

        9055  95% - command-execute
        8994  95%  - funcall-interactively
        8993  95%   - eval-expression
        8993  95%    - #<compiled -0x5db3e1955cb81d1>
        8993  95%     - #<compiled 0x1be2f2b973e98d46>
        8988  95%      - eval
        8988  95%       - benchmark-call
        8988  95%        - #<lambda 0x8f52207467a22>
        8988  95%         - build-sieve
         206   2%            ash
         169   1%            make-vector
           5   0%      - macroexpand-all
           5   0%       - macroexp--expand-all
           5   0%        - macroexp-macroexpand
           5   0%           macroexpand-1
           1   0%     execute-extended-command
          61   0%  - byte-code
          37   0%   - read-extended-command
          37   0%    - read-extended-command-1
          37   0%     - completing-read-default
          18   0%        redisplay_internal (C function)
           1   0%        clear-minibuffer-message
           1   0%      - timer-event-handler
           1   0%       - apply
           1   0%        - show-paren-function
           1   0%         - show-paren--default
           1   0%            show-paren--locate-near-paren
          24   0%   - read--expression
          12   0%      redisplay_internal (C function)
           1   0%    - minibuffer-mode
           1   0%     - run-mode-hooks
           1   0%      - run-hooks
           1   0%       - global-font-lock-mode-enable-in-buffers
           1   0%        - turn-on-font-lock-if-desired
           1   0%         - turn-on-font-lock
           1   0%          - font-lock-mode
           1   0%           - called-interactively-p
           1   0%              backtrace-frame
           1   0%    - command-execute
           1   0%     - funcall-interactively
           1   0%      - yank
           1   0%       - current-kill
           1   0%        - gui-selection-value
           1   0%         - gui--set-last-clipboard-selection
           1   0%          - gui-backend-get-selection
           1   0%           - apply
           1   0%            - #<compiled 0xb78ea5818013cfc>
           1   0%               x-get-selection-internal
         345   3%   Automatic GC
          23   0% + timer-event-handler
          12   0% + redisplay_internal (C function)
          10   0% + ...

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Andrea Corallo]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>
>>> From: Stephen Berman <stephen.berman@gmx.net>
>>> Cc: acorallo@gnu.org,  bug-gnu-emacs@gnu.org,  robertstephenboyer@gmail.com,
>>>   69480@debbugs.gnu.org
>>> Date: Fri, 01 Mar 2024 14:07:50 +0100
>>>
>>> On Fri, 01 Mar 2024 14:45:39 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>>
>>> >> It's still slower with native compilation than with byte compilation on
>>> >> my machine, unlike on yours.  Do you have any idea why?
>>> >
>>> > Maybe your system takes time to allocate memory?  This program
>>> > allocates almost 850MB of memory on my system, so maybe yours starts
>>> > paging?
>>>
>>> I see a similar allocation, but I have 16 GB of RAM (~12 of which were
>>> free when I ran the benchmark), so I think that shouldn't be an issue.
>>>
>>> > Anyway, profiling could give some hints.
>>>
>>> That says almost all memory and CPU are consumed by `build-sieve':
>>
>> This is skewed, use cpu profiling instead of "memory" profiling.
>
> Ah, ok.
>
>>> But I don't see how that bears on the difference between native and byte
>>> compilation in my timings versus yours and Andrea's.
>>
>> My suggestion was to compare profiles in the byte-compiled and
>> native-compiled cases.
>>
>> Btw, are you running both cases in the same session?  If so, don't:
>> restart Emacs and run the other case instead.
>
> Ok, I've now done that.  Here's the report for the run with native
> compilation:
>
>        12599  95% - command-execute
>        12487  95%  - funcall-interactively
>        12486  95%   - eval-expression
>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>        12480  95%      - eval
>        12480  95%       - progn
>        12367  94%        - benchmark-call
>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>        12367  94%            build-sieve
>          113   0%        - emacs-lisp-native-compile-and-load
>          113   0%         - emacs-lisp-native-compile
>          113   0%          - native-compile

IIUC this is profiling the native compilation itself.

BTW I'd suggest the profile is done with perf (and running batch).

Given you see on your machine similar times for native and byte compiled
the expected outcome should be tha tthe time is spent in some C routine
of our core.

Best Regards

  Andrea

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 09:07:54 -0500 Andrea Corallo <acorallo@gnu.org> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>
[...]
>>> My suggestion was to compare profiles in the byte-compiled and
>>> native-compiled cases.
>>>
>>> Btw, are you running both cases in the same session?  If so, don't:
>>> restart Emacs and run the other case instead.
>>
>> Ok, I've now done that.  Here's the report for the run with native
>> compilation:
>>
>>        12599  95% - command-execute
>>        12487  95%  - funcall-interactively
>>        12486  95%   - eval-expression
>>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>>        12480  95%      - eval
>>        12480  95%       - progn
>>        12367  94%        - benchmark-call
>>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>>        12367  94%            build-sieve
>>          113   0%        - emacs-lisp-native-compile-and-load
>>          113   0%         - emacs-lisp-native-compile
>>          113   0%          - native-compile
>
> IIUC this is profiling the native compilation itself.
>
> BTW I'd suggest the profile is done with perf (and running batch).

I don't have perf installed, but I build the kernel from source, so I
guess I could build and install perf, but...

> Given you see on your machine similar times for native and byte compiled
> the expected outcome should be tha tthe time is spent in some C routine
> of our core.

If you consider ~12.7 (native-compiled) and ~9.6 (byte-compiled),
similar for this benchmark, and since Eli said it's expected that
native-compiled elisp can be slower than byte-compiled elisp for some
programs, then I guess I can just accept that for this case, the
difference between my timings is within a reasonable margin of error and
not due to some problem with my libgccjit (which I also built and
installed myself).

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Andrea Corallo]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Fri, 01 Mar 2024 09:07:54 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
>
>> Stephen Berman <stephen.berman@gmx.net> writes:
>>
>>> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>>
> [...]
>>>> My suggestion was to compare profiles in the byte-compiled and
>>>> native-compiled cases.
>>>>
>>>> Btw, are you running both cases in the same session?  If so, don't:
>>>> restart Emacs and run the other case instead.
>>>
>>> Ok, I've now done that.  Here's the report for the run with native
>>> compilation:
>>>
>>>        12599  95% - command-execute
>>>        12487  95%  - funcall-interactively
>>>        12486  95%   - eval-expression
>>>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>>>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>>>        12480  95%      - eval
>>>        12480  95%       - progn
>>>        12367  94%        - benchmark-call
>>>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>>>        12367  94%            build-sieve
>>>          113   0%        - emacs-lisp-native-compile-and-load
>>>          113   0%         - emacs-lisp-native-compile
>>>          113   0%          - native-compile
>>
>> IIUC this is profiling the native compilation itself.
>>
>> BTW I'd suggest the profile is done with perf (and running batch).
>
> I don't have perf installed, but I build the kernel from source, so I
> guess I could build and install perf, but...
>
>> Given you see on your machine similar times for native and byte compiled
>> the expected outcome should be tha tthe time is spent in some C routine
>> of our core.
>
> If you consider ~12.7 (native-compiled) and ~9.6 (byte-compiled),
> similar for this benchmark, and since Eli said it's expected that
> native-compiled elisp can be slower than byte-compiled elisp for some
> programs, then I guess I can just accept that for this case, the
> difference between my timings is within a reasonable margin of error and
> not due to some problem with my libgccjit (which I also built and
> installed myself).

I doubt the correctness of your measure.  Some points:

1- Your benchmark results shows you are measuring the compilation
process as well.

2- As Eli mentioned you should always start from on a freshly started
session (probably running batch).

3- You should also do several measures of the same test to estimate the
noise and, as consequence, the accuracy of your measure.  The reason is
that there are many sources of noise on a running system (OS, paging,
CPU throttle due to thermal conditions etc...).  These sources of noise
can have a big impact.

Measuring performance in reliable way is more tricky than what most
people think :)

Thanks

  Andrea

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Eli Zaretskii]

> From: Stephen Berman <stephen.berman@gmx.net>
> Cc: acorallo@gnu.org,  robertstephenboyer@gmail.com,  69480@debbugs.gnu.org
> Date: Fri, 01 Mar 2024 14:53:18 +0100
> 
> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
> 
> >> That says almost all memory and CPU are consumed by `build-sieve':
> >
> > This is skewed, use cpu profiling instead of "memory" profiling.
> 
> Ah, ok.
> 
> >> But I don't see how that bears on the difference between native and byte
> >> compilation in my timings versus yours and Andrea's.
> >
> > My suggestion was to compare profiles in the byte-compiled and
> > native-compiled cases.
> >
> > Btw, are you running both cases in the same session?  If so, don't:
> > restart Emacs and run the other case instead.
> 
> Ok, I've now done that.  Here's the report for the run with native
> compilation:
> 
>        12599  95% - command-execute
>        12487  95%  - funcall-interactively
>        12486  95%   - eval-expression
>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>        12480  95%      - eval
>        12480  95%       - progn
>        12367  94%        - benchmark-call
>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>        12367  94%            build-sieve
>          113   0%        - emacs-lisp-native-compile-and-load
>          113   0%         - emacs-lisp-native-compile
>          113   0%          - native-compile
>           79   0%           - comp--native-compile
>           37   0%            - comp--fwprop

Why are subroutines of native-compilation in the profile?

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 18:34:30 +0200 Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Stephen Berman <stephen.berman@gmx.net>
>> Cc: acorallo@gnu.org,  robertstephenboyer@gmail.com,  69480@debbugs.gnu.org
>> Date: Fri, 01 Mar 2024 14:53:18 +0100
>> 
>> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>> >> That says almost all memory and CPU are consumed by `build-sieve':
>> >
>> > This is skewed, use cpu profiling instead of "memory" profiling.
>> 
>> Ah, ok.
>> 
>> >> But I don't see how that bears on the difference between native and byte
>> >> compilation in my timings versus yours and Andrea's.
>> >
>> > My suggestion was to compare profiles in the byte-compiled and
>> > native-compiled cases.
>> >
>> > Btw, are you running both cases in the same session?  If so, don't:
>> > restart Emacs and run the other case instead.
>> 
>> Ok, I've now done that.  Here's the report for the run with native
>> compilation:
>> 
>>        12599  95% - command-execute
>>        12487  95%  - funcall-interactively
>>        12486  95%   - eval-expression
>>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>>        12480  95%      - eval
>>        12480  95%       - progn
>>        12367  94%        - benchmark-call
>>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>>        12367  94%            build-sieve
>>          113   0%        - emacs-lisp-native-compile-and-load
>>          113   0%         - emacs-lisp-native-compile
>>          113   0%          - native-compile
>>           79   0%           - comp--native-compile
>>           37   0%            - comp--fwprop
>
> Why are subroutines of native-compilation in the profile?

I mistakenly used emacs-lisp-native-compile-and-load because its doc
string says "Native-compile the current buffer’s file (if it has
changed), then load it", and since it hadn't changed, I concluded
(apparently wrongly) that it would just be loaded.  Also, I didn't
realize you can load an .eln file just like you can an .el or .elc file.
Now I've done the profile after loading the .eln file and running
(benchmark-run nil (build-sieve (expt 10 8))).  The timing was
nevertheless practical the same as before:

(12.527129893 1 0.340515292)

Here's the profile report:

       12253  96% - command-execute
       12195  96%  - funcall-interactively
       12193  96%   - eval-expression
       12193  96%    - #<compiled -0x5db3e1955cb81d1>
       12193  96%     - #<compiled 0xfed2db7b0724d71>
       12187  96%      - eval
       12187  96%       - benchmark-call
       12184  96%        - #<lambda 0x833a720aa0822>
       12184  96%           build-sieve
           3   0%        - time-since
           1   0%         - byte-code
           1   0%            require
           6   0%      - macroexpand-all
           6   0%       - macroexp--expand-all
           6   0%        - macroexp-macroexpand
           6   0%           macroexpand-1
           2   0%     execute-extended-command
          58   0%  - byte-code
          35   0%   - read-extended-command
          35   0%    - read-extended-command-1
          35   0%     - completing-read-default
          12   0%        redisplay_internal (C function)
           1   0%        minibuffer-mode
          23   0%   - read--expression
           8   0%      redisplay_internal (C function)
           2   0%    - command-execute
           2   0%     - funcall-interactively
           2   0%      - yank
           2   0%       - current-kill
           2   0%        - gui-selection-value
           1   0%         - gui--selection-value-internal
           1   0%          - gui-get-selection
           1   0%           - gui-backend-get-selection
           1   0%            - apply
           1   0%             - #<compiled 0xb78982bfc8d3cfc>
           1   0%                x-get-selection-internal
           1   0%         - gui--set-last-clipboard-selection
           1   0%          - gui-backend-get-selection
           1   0%           - apply
           1   0%            - #<compiled 0xb78982bfc8d3cfc>
           1   0%               x-get-selection-internal
           1   0%    - timer-event-handler
           1   0%     - apply
           1   0%      - show-paren-function
           1   0%         show-paren--default
         351   2%   Automatic GC
          26   0%   redisplay_internal (C function)
          21   0% + timer-event-handler
          11   0% + ...

Steve Berman

bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Stephen Berman via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Fri, 01 Mar 2024 11:10:21 -0500 Andrea Corallo <acorallo@gnu.org> wrote:

> Stephen Berman <stephen.berman@gmx.net> writes:
>
>> On Fri, 01 Mar 2024 09:07:54 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
>>
>>> Stephen Berman <stephen.berman@gmx.net> writes:
>>>
>>>> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>>>
>> [...]
>>>>> My suggestion was to compare profiles in the byte-compiled and
>>>>> native-compiled cases.
>>>>>
>>>>> Btw, are you running both cases in the same session?  If so, don't:
>>>>> restart Emacs and run the other case instead.
>>>>
>>>> Ok, I've now done that.  Here's the report for the run with native
>>>> compilation:
>>>>
>>>>        12599  95% - command-execute
>>>>        12487  95%  - funcall-interactively
>>>>        12486  95%   - eval-expression
>>>>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>>>>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>>>>        12480  95%      - eval
>>>>        12480  95%       - progn
>>>>        12367  94%        - benchmark-call
>>>>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>>>>        12367  94%            build-sieve
>>>>          113   0%        - emacs-lisp-native-compile-and-load
>>>>          113   0%         - emacs-lisp-native-compile
>>>>          113   0%          - native-compile
>>>
>>> IIUC this is profiling the native compilation itself.
>>>
>>> BTW I'd suggest the profile is done with perf (and running batch).
>>
>> I don't have perf installed, but I build the kernel from source, so I
>> guess I could build and install perf, but...
>>
>>> Given you see on your machine similar times for native and byte compiled
>>> the expected outcome should be tha tthe time is spent in some C routine
>>> of our core.
>>
>> If you consider ~12.7 (native-compiled) and ~9.6 (byte-compiled),
>> similar for this benchmark, and since Eli said it's expected that
>> native-compiled elisp can be slower than byte-compiled elisp for some
>> programs, then I guess I can just accept that for this case, the
>> difference between my timings is within a reasonable margin of error and
>> not due to some problem with my libgccjit (which I also built and
>> installed myself).
>
> I doubt the correctness of your measure.  Some points:
>
> 1- Your benchmark results shows you are measuring the compilation
> process as well.

Yes, this was due to my misunderstanding of the doc string of
emacs-lisp-native-compile-and-load.

> 2- As Eli mentioned you should always start from on a freshly started
> session (probably running batch).

I did start a fresh session in the above, though not a batch run.

> 3- You should also do several measures of the same test to estimate the
> noise and, as consequence, the accuracy of your measure.  The reason is
> that there are many sources of noise on a running system (OS, paging,
> CPU throttle due to thermal conditions etc...).  These sources of noise
> can have a big impact.

I know, though I have now done several runs using
emacs-lisp-native-compile-and-load, all resulting in nearly the same
timing, and my corrected run after simply loading the natively compiled
file also resulted in virtually the same time (see my reply to Eli).
Maybe it's still not an accurate measurement, but can it be so
inaccurate as to result in a 1/3 slower time than the byte-compiled run?

Steve Berman