bug#69480: Emacs Lisp needs, for its great 'native-compile', 'declare' and 'the' for fixnums and arrays.

[Andrea Corallo <acorallo@gnu.org>]

Stephen Berman <stephen.berman@gmx.net> writes:

> On Fri, 01 Mar 2024 09:07:54 -0500 Andrea Corallo <acorallo@gnu.org> wrote:
>
>> Stephen Berman <stephen.berman@gmx.net> writes:
>>
>>> On Fri, 01 Mar 2024 15:41:24 +0200 Eli Zaretskii <eliz@gnu.org> wrote:
>>>
> [...]
>>>> My suggestion was to compare profiles in the byte-compiled and
>>>> native-compiled cases.
>>>>
>>>> Btw, are you running both cases in the same session?  If so, don't:
>>>> restart Emacs and run the other case instead.
>>>
>>> Ok, I've now done that.  Here's the report for the run with native
>>> compilation:
>>>
>>>        12599  95% - command-execute
>>>        12487  95%  - funcall-interactively
>>>        12486  95%   - eval-expression
>>>        12485  95%    - #<compiled -0x5db3e1955cb81d1>
>>>        12485  95%     - #<compiled -0x8a5cf032951a0fe>
>>>        12480  95%      - eval
>>>        12480  95%       - progn
>>>        12367  94%        - benchmark-call
>>>        12367  94%         - #<lambda 0x8c97b8cb7bd82>
>>>        12367  94%            build-sieve
>>>          113   0%        - emacs-lisp-native-compile-and-load
>>>          113   0%         - emacs-lisp-native-compile
>>>          113   0%          - native-compile
>>
>> IIUC this is profiling the native compilation itself.
>>
>> BTW I'd suggest the profile is done with perf (and running batch).
>
> I don't have perf installed, but I build the kernel from source, so I
> guess I could build and install perf, but...
>
>> Given you see on your machine similar times for native and byte compiled
>> the expected outcome should be tha tthe time is spent in some C routine
>> of our core.
>
> If you consider ~12.7 (native-compiled) and ~9.6 (byte-compiled),
> similar for this benchmark, and since Eli said it's expected that
> native-compiled elisp can be slower than byte-compiled elisp for some
> programs, then I guess I can just accept that for this case, the
> difference between my timings is within a reasonable margin of error and
> not due to some problem with my libgccjit (which I also built and
> installed myself).

I doubt the correctness of your measure.  Some points:

1- Your benchmark results shows you are measuring the compilation
process as well.

2- As Eli mentioned you should always start from on a freshly started
session (probably running batch).

3- You should also do several measures of the same test to estimate the
noise and, as consequence, the accuracy of your measure.  The reason is
that there are many sources of noise on a running system (OS, paging,
CPU throttle due to thermal conditions etc...).  These sources of noise
can have a big impact.

Measuring performance in reliable way is more tricky than what most
people think :)

Thanks

  Andrea