Lexical binding doesn't seem to be faster?

Lexical binding doesn't seem to be faster?

[Hi-Angel]

I've stumbled upon a discussion about converting Emacs code to a
faster lexical-binding and that help needed. I noted that cc-mode.el
I'm regularly use doesn't have one, and figured I could help.

So I inserted to cc-mode.el in emacs-git revision I have:

    ;;; -*- lexical-binding: t -*-

The first bad sign happened to be sizes of cc-mode.elc:

    ;;; -*- lexical-binding: t -*-   | 217581
    ;;; -*- lexical-binding: nil -*- | 212542

Lexical-binding bloats byte-code by 5KB. Odd. I tried comparing
assembly, but without experience in reading lisp assembly I haven't
come up with anything. So I thought "Okay, must be heavy inlining;
let's just compare the speed".

So, I took someone's function `with-timer` for benchmarking¹; and then
for both t and nil lexical-binding I did following steps:

1. `byte-compile` `cc-mode.el`
2. `load-file`    `cc-mode.elc`
3. open a 5k lines C file²
4. Execute multiple times `(with-timer (c-font-lock-fontify-region 0
(point-max)))`
5. Kill the buffer.

These are results I got:

    ;;; -*- lexical-binding: t -*-   | 1.174s, 1.144s, 1.177s, 1.125s, 1.177s
    ;;; -*- lexical-binding: nil -*- | 1.037s, 1.061s, 1.037s, 1.037s, 0.991s


1: https://emacs.stackexchange.com/a/541/2671#how-do-i-measure-performance-of-elisp-code
2: https://gitlab.freedesktop.org/libinput/libinput/blob/9a2d6f55b1276da11dd9b2c4c8e22a405576dfea/src/libinput.h

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> I've stumbled upon a discussion about converting Emacs code to a
> faster lexical-binding and that help needed.

While the semantics of lexical-binding does offer more opportunities for
optimization, the current Elisp implementation does not make much effort
to take advantage of it, so compiling with lexical-binding may generate
code that goes marginally faster sometimes and marginally slower at
other times simply because the code generated is a bit different, but in
my experience there's rarely much difference either way.

> The first bad sign happened to be sizes of cc-mode.elc:
>
>     ;;; -*- lexical-binding: t -*-   | 217581
>     ;;; -*- lexical-binding: nil -*- | 212542
>
> Lexical-binding bloats byte-code by 5KB. Odd.

Indeed, in my experience lexical-binding tends to generate slightly
larger .elc files.  Not sure why, actually: I never bothered
to investigate.  Intuitively, I can't think of a good reason why that
should be the case, so it may be a simple performance bug.

[ BTW, a performance problem with lexical-binding was found during
  Emacs-24 which got fixed in Emacs-25, so I assume you're using
  a recent enough version which is not affected by this problem.  ]

> So, I took someone's function `with-timer` for benchmarking¹;

Any reason why you didn't use `benchmark(-run(-compiled))`?

>     ;;; -*- lexical-binding: t -*-   | 1.174s, 1.144s, 1.177s, 1.125s, 1.177s
>     ;;; -*- lexical-binding: nil -*- | 1.037s, 1.061s, 1.037s, 1.037s, 0.991s

Hmmm... ~13% slowdown isn't terribly bad, but it's indeed
rather disappointing.  It'd be nice to try and compare the profiles
in those two cases to try and see if the slowdown is just evenly spread
(which would suck) or is neatly limited to a specific spot (which would
be great).

FWIW, the usual case where lexical-binding leads to bigger&slower code
is when you pass a lambda-expression to another function and that
lambda-expression has free variables (in the lexical-binding case, it
requires building a closure which is rather costly currently, whereas in
the dynamic-binding case it relies on the dynamic-scoping instead).

So, for example, calls to `mapcar` where you pass a (lambda ...) that
refers to variables from the context tend to be a bit more expensive
(tho each iteration may be very slightly faster because the references
to those variables can be slightly cheaper, so the overall impact may
depend on the number of elements in the list).


        Stefan

Re: Lexical binding doesn't seem to be faster?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 654 bytes --]

On Fri, Mar 08, 2019 at 12:39:09AM -0500, Stefan Monnier wrote:

[...]

> FWIW, the usual case where lexical-binding leads to bigger&slower code
> is when you pass a lambda-expression to another function and that
> lambda-expression has free variables (in the lexical-binding case, it
> requires building a closure which is rather costly currently, whereas in
> the dynamic-binding case it relies on the dynamic-scoping instead).

Let's see if I got that right: in those cases, the code would (most
probably!) behave differently depending on the binding regime (lexical
vs dynamic), so it would be doing quite different things in each case?

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

>> FWIW, the usual case where lexical-binding leads to bigger&slower code
>> is when you pass a lambda-expression to another function and that
>> lambda-expression has free variables (in the lexical-binding case, it
>> requires building a closure which is rather costly currently, whereas in
>> the dynamic-binding case it relies on the dynamic-scoping instead).
> Let's see if I got that right: in those cases, the code would (most
> probably!) behave differently depending on the binding regime (lexical
> vs dynamic), so it would be doing quite different things in each case?

Actually, in most such cases the code behaves identically in the end,
but it gets there in a different way (so yes, in some cases it does
behave differently).

E.g.

    (let ((buf (current-buffer)))
      [...]
      (with-temp-buffer
        (mapcar (lambda (x)
                  (with-current-buffer buf ...))
                ...))
      [...])
      
The end result will most likely be the same regardless which binding
style is used, but the way to find the value of `buf` from within the
lambda is different in the two cases.

The difference becomes apparent if you do

    (advice-add 'mapcar :around
                (lambda (&rest orig-call)
                  (let ((buf 42)) (apply orig-call))))
                  
since in this case dynamic-binding will cause your `with-current-buffer`
to try and use buffer 42 and signal an error (because of the name
conflict, aka "variable capture"), whereas lexical-binding
will be unaffected.


        Stefan

Re: Lexical binding doesn't seem to be faster?

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

On Fri, Mar 08, 2019 at 08:53:32AM -0500, Stefan Monnier wrote:

[free variables, lexical vs. dynamic bindings and differences in
 behaviour]

> Actually, in most such cases the code behaves identically in the end,
> but it gets there in a different way (so yes, in some cases it does
> behave differently).
> 
> E.g.
> 
>     (let ((buf (current-buffer)))
>       [...]
>       (with-temp-buffer
>         (mapcar (lambda (x)
>                   (with-current-buffer buf ...))
>                 ...))
>       [...])
>       
> The end result will most likely be the same regardless which binding
> style is used, but the way to find the value of `buf` from within the
> lambda is different in the two cases.
> 
> The difference becomes apparent if you do
> 
>     (advice-add 'mapcar :around
>                 (lambda (&rest orig-call)
>                   (let ((buf 42)) (apply orig-call))))
>                   
> since in this case dynamic-binding will cause your `with-current-buffer`
> to try and use buffer 42 and signal an error (because of the name
> conflict, aka "variable capture"), whereas lexical-binding
> will be unaffected.

Thanks for the nice example. In a way it is somewhat surprising how
"similarly" lexical and dynamic binding tend to behave in general,
but that is, I think, due to how we tend to structure code, avoiding
variables whose "resolution" lies "far away".

And your example illustrates that the differences manifest themselves
most with constructs which upset the "dynamic flow" of the code, i.e.
some kind of callback (which, in a very handwavy way, advice is a kind
of).

I.e. I'm "here" and tell the code "there" to run my own snippet "here"
in its context "there". Or something.

Cheers
-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

On Fri, 8 Mar 2019 at 08:39, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > I've stumbled upon a discussion about converting Emacs code to a
> > faster lexical-binding and that help needed.
>
> While the semantics of lexical-binding does offer more opportunities for
> optimization, the current Elisp implementation does not make much effort
> to take advantage of it, so compiling with lexical-binding may generate
> code that goes marginally faster sometimes and marginally slower at
> other times simply because the code generated is a bit different, but in
> my experience there's rarely much difference either way.
>
> > The first bad sign happened to be sizes of cc-mode.elc:
> >
> >     ;;; -*- lexical-binding: t -*-   | 217581
> >     ;;; -*- lexical-binding: nil -*- | 212542
> >
> > Lexical-binding bloats byte-code by 5KB. Odd.
>
> Indeed, in my experience lexical-binding tends to generate slightly
> larger .elc files.  Not sure why, actually: I never bothered
> to investigate.  Intuitively, I can't think of a good reason why that
> should be the case, so it may be a simple performance bug.
>
> [ BTW, a performance problem with lexical-binding was found during
>   Emacs-24 which got fixed in Emacs-25, so I assume you're using
>   a recent enough version which is not affected by this problem.  ]

Yeah, it's a ≈month old version from git, so it's pretty recent.

> > So, I took someone's function `with-timer` for benchmarking¹
>
> Any reason why you didn't use `benchmark(-run(-compiled))`?

Ah, thanks! Previously I only found `benchmark` function, but running

     (benchmark 10 (c-font-lock-fontify-region 0 (point-max)))

results in error "void-function jit-lock-bounds".

However the versions `benchmark-run` and `benchmark-run-compiled` works for me.

>
> >     ;;; -*- lexical-binding: t -*-   | 1.174s, 1.144s, 1.177ą;s, 1.125s, 1.177s
> >     ;;; -*- lexical-binding: nil -*- | 1.037s, 1.061s, 1.037s, 1.037s, 0.991s
>
> Hmmm... ~13% slowdown isn't terribly bad, but it's indeed
> rather disappointing.  It'd be nice to try and compare the profiles
> in those two cases to try and see if the slowdown is just evenly spread
> (which would suck) or is neatly limited to a specific spot (which would
> be great).
>
> FWIW, the usual case where lexical-binding leads to bigger&slower code
> is when you pass a lambda-expression to another function and that
> lambda-expression has free variables (in the lexical-binding case, it
> requires building a closure which is rather costly currently, whereas in
> the dynamic-binding case it relies on the dynamic-scoping instead).
>
> So, for example, calls to `mapcar` where you pass a (lambda ...) that
> refers to variables from the context tend to be a bit more expensive
> (tho each iteration may be very slightly faster because the references
> to those variables can be slightly cheaper, so the overall impact may
> depend on the number of elements in the list).

FTR: I re-benchmarked as `(benchmark-run-compiled 10
(c-font-lock-fontify-region 0 (point-max)))`, and also with `emacs -Q`
to make sure that none of addons can interfere (I'm using
color-identifiers which may add an overhead to fontification). For the
same reason I disabled GC (locally I too have it only enabled to run
at "idle time").

It's interesting that the difference almost disappeared:

    nil: (7.463936164 0 0.0) (7.520960622 0 0.0) (7.526411695999999 0
0.0) (7.537842362999999 0 0.0)
    t:   (7.617106151000001 0 0.0) (7.635044875 0 0.0)
(7.6383228789999995 0 0.0) (7.598431915 0 0.0)

"nil" still seems to be faster, but it may as well be a statistical variation.

----

Either way, I'm happy, as you suggested, to look at per-function
overhead to see if there's any difference. Do you think it's still
worth it? And how do I do it though? Shall I do (profiler-start), and
then evaluate the benchmark?

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> Ah, thanks! Previously I only found `benchmark` function, but running
>
>      (benchmark 10 (c-font-lock-fontify-region 0 (point-max)))
>
> results in error "void-function jit-lock-bounds".

With `benchmark` you need to quote the argument (it's a function, not
a macro).

[ BTW, (point-min) is more better than 0.  ]

> FTR: I re-benchmarked as `(benchmark-run-compiled 10
> (c-font-lock-fontify-region 0 (point-max)))`, and also with `emacs -Q`
> to make sure that none of addons can interfere (I'm using
> color-identifiers which may add an overhead to fontification).  For the
> same reason I disabled GC (locally I too have it only enabled to run
> at "idle time").

Good (except for disabling GC, which makes the measurement different
from "real life").

> It's interesting that the difference almost disappeared:
>
>     nil: (7.463936164 0 0.0) (7.520960622 0 0.0) (7.526411695999999 0
> 0.0) (7.537842362999999 0 0.0)
>     t:   (7.617106151000001 0 0.0) (7.635044875 0 0.0)
> (7.6383228789999995 0 0.0) (7.598431915 0 0.0)
>
> "nil" still seems to be faster, but it may as well be a statistical variation.

It looks minor but it seems stable enough to be more than just
statistical variation.

> Either way, I'm happy, as you suggested, to look at per-function
> overhead to see if there's any difference. Do you think it's still
> worth it?

Hard to tell.

> And how do I do it though?  Shall I do (profiler-start), and
> then evaluate the benchmark?

Do (profiler-start 'cpu) before running the benchmark and
(profiler-report) afterwards. And then C-u RET on the top line to expand
it (recursively).


        Stefan

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

On Sat, 9 Mar 2019 at 18:06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > FTR: I re-benchmarked as `(benchmark-run-compiled 10
> > (c-font-lock-fontify-region 0 (point-max)))`, and also with `emacs -Q`
> > to make sure that none of addons can interfere (I'm using
> > color-identifiers which may add an overhead to fontification).  For the
> > same reason I disabled GC (locally I too have it only enabled to run
> > at "idle time").
>
> Good (except for disabling GC, which makes the measurement different
> from "real life").
>
> > It's interesting that the difference almost disappeared:
> >
> >     nil: (7.463936164 0 0.0) (7.520960622 0 0.0) (7.526411695999999 0
> > 0.0) (7.537842362999999 0 0.0)
> >     t:   (7.617106151000001 0 0.0) (7.635044875 0 0.0)
> > (7.6383228789999995 0 0.0) (7.598431915 0 0.0)
> >
> > "nil" still seems to be faster, but it may as well be a statistical variation.
>
> It looks minor but it seems stable enough to be more than just
> statistical variation.
>
> > Either way, I'm happy, as you suggested, to look at per-function
> > overhead to see if there's any difference. Do you think it's still
> > worth it?
>
> Hard to tell.
>
> > And how do I do it though?  Shall I do (profiler-start), and
> > then evaluate the benchmark?
>
> Do (profiler-start 'cpu) before running the benchmark and
> (profiler-report) afterwards. And then C-u RET on the top line to expand
> it (recursively).

Thanks, so, I did that, and without disabling GC results look quite odd.

For "lexical-binding: t" it is:

    - ...                                        2 100%
     - list                                      1  50%
      - -                                        1  50%
       - let                                     1  50%
        - let                                    1  50%
         - while                                 1  50%
          - funcall                              1  50%
           - #<compiled 0x15833de3b0c9>          1  50%
            - c-font-lock-fontify-region         1  50%
             - c-before-context-fl-expand-region 1  50%
              - mapc                             1  50%
               - #<compiled 0x15833db80701>      1  50%
                - c-context-expand-fl-region     1  50%
                 - c-fl-decl-end                 1  50%
                  - c-literal-start              1  50%
                   - c-state-semi-pp-to-literal  1  50%
                      c-parse-ps-state-below     1  50%
       Automatic GC                              1  50%

Whereas for "lexical-binding: nil" it's just:

    - ...           1 100%
       Automatic GC 1 100%
-----

With GC disabled it looks more interesting:

"lexical-binding: t":

    - ...                                        1 100%
     - list                                      1 100%
      - -                                        1 100%
       - let                                     1 100%
        - let                                    1 100%
         - while                                 1 100%
          - funcall                              1 100%
           - #<compiled 0x1585968733a1>          1 100%
            - c-font-lock-fontify-region         1 100%
             - c-before-context-fl-expand-region 1 100%
              - mapc                             1 100%
               - #<compiled 0x158596934cbd>      1 100%
                - c-context-expand-fl-region     1 100%
                 - c-fl-decl-end                 1 100%
                  - c-literal-start              1 100%
                   - c-state-semi-pp-to-literal  1 100%
                      c-parse-ps-state-below     1 100%
       Automatic GC                              0   0%

"lexical-binding: nil":
    - ...                                                1 100%
     - funcall-interactively                             1 100%
      - eval-expression                                  1 100%
       - eval                                            1 100%
        - mytest                                         1 100%
         - let                                           1 100%
          - list                                         1 100%
           - -                                           1 100%
            - let                                        1 100%
             - let                                       1 100%
              - while                                    1 100%
               - funcall                                 1 100%
                - #<compiled 0x1580e938690d>             1 100%
                 - c-font-lock-fontify-region            1 100%
                  - font-lock-default-fontify-region     1 100%
                   - font-lock-unfontify-region          1 100%
                      font-lock-default-unfontify-region 1 100%
       Automatic GC                                      0   0%

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

On Sun, 10 Mar 2019 at 18:16, Hi-Angel <hiangel999@gmail.com> wrote:
>
> On Sat, 9 Mar 2019 at 18:06, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > > FTR: I re-benchmarked as `(benchmark-run-compiled 10
> > > (c-font-lock-fontify-region 0 (point-max)))`, and also with `emacs -Q`
> > > to make sure that none of addons can interfere (I'm using
> > > color-identifiers which may add an overhead to fontification).  For the
> > > same reason I disabled GC (locally I too have it only enabled to run
> > > at "idle time").
> >
> > Good (except for disabling GC, which makes the measurement different
> > from "real life").
> >
> > > It's interesting that the difference almost disappeared:
> > >
> > >     nil: (7.463936164 0 0.0) (7.520960622 0 0.0) (7.526411695999999 0
> > > 0.0) (7.537842362999999 0 0.0)
> > >     t:   (7.617106151000001 0 0.0) (7.635044875 0 0.0)
> > > (7.6383228789999995 0 0.0) (7.598431915 0 0.0)
> > >
> > > "nil" still seems to be faster, but it may as well be a statistical variation.
> >
> > It looks minor but it seems stable enough to be more than just
> > statistical variation.
> >
> > > Either way, I'm happy, as you suggested, to look at per-function
> > > overhead to see if there's any difference. Do you think it's still
> > > worth it?
> >
> > Hard to tell.
> >
> > > And how do I do it though?  Shall I do (profiler-start), and
> > > then evaluate the benchmark?
> >
> > Do (profiler-start 'cpu) before running the benchmark and
> > (profiler-report) afterwards. And then C-u RET on the top line to expand
> > it (recursively).
>
> Thanks, so, I did that, and without disabling GC results look quite odd.
>
> For "lexical-binding: t" it is:
>
>     - ...                                        2 100%
>      - list                                      1  50%
>       - -                                        1  50%
>        - let                                     1  50%
>         - let                                    1  50%
>          - while                                 1  50%
>           - funcall                              1  50%
>            - #<compiled 0x15833de3b0c9>          1  50%
>             - c-font-lock-fontify-region         1  50%
>              - c-before-context-fl-expand-region 1  50%
>               - mapc                             1  50%
>                - #<compiled 0x15833db80701>      1  50%
>                 - c-context-expand-fl-region     1  50%
>                  - c-fl-decl-end                 1  50%
>                   - c-literal-start              1  50%
>                    - c-state-semi-pp-to-literal  1  50%
>                       c-parse-ps-state-below     1  50%
>        Automatic GC                              1  50%
>
> Whereas for "lexical-binding: nil" it's just:
>
>     - ...           1 100%
>        Automatic GC 1 100%
> -----
>
> With GC disabled it looks more interesting:
>
> "lexical-binding: t":
>
>     - ...                                        1 100%
>      - list                                      1 100%
>       - -                                        1 100%
>        - let                                     1 100%
>         - let                                    1 100%
>          - while                                 1 100%
>           - funcall                              1 100%
>            - #<compiled 0x1585968733a1>          1 100%
>             - c-font-lock-fontify-region         1 100%
>              - c-before-context-fl-expand-region 1 100%
>               - mapc                             1 100%
>                - #<compiled 0x158596934cbd>      1 100%
>                 - c-context-expand-fl-region     1 100%
>                  - c-fl-decl-end                 1 100%
>                   - c-literal-start              1 100%
>                    - c-state-semi-pp-to-literal  1 100%
>                       c-parse-ps-state-below     1 100%
>        Automatic GC                              0   0%
>
> "lexical-binding: nil":
>     - ...                                                1 100%
>      - funcall-interactively                             1 100%
>       - eval-expression                                  1 100%
>        - eval                                            1 100%
>         - mytest                                         1 100%
>          - let                                           1 100%
>           - list                                         1 100%
>            - -                                           1 100%
>             - let                                        1 100%
>              - let                                       1 100%
>               - while                                    1 100%
>                - funcall                                 1 100%
>                 - #<compiled 0x1580e938690d>             1 100%
>                  - c-font-lock-fontify-region            1 100%
>                   - font-lock-default-fontify-region     1 100%
>                    - font-lock-unfontify-region          1 100%
>                       font-lock-default-unfontify-region 1 100%
>        Automatic GC                                      0   0%

I found that a lot of code resides in `cc-engine.el`, so here's some
more statistics, tested with emacs -Q, and "nil" or "t" accord to
lexical-binding disabled/enabled in both cc-mode and cc-engine
simultaneously.

With GC:
    nil: (9.099007151 87 1.6701303489999995) (9.057014855 85 1.6550994100000007)
    t:   (9.17824316 95 1.792632212) (9.139527843 94 1.8019125349999991)

No GC:

    nil: (7.519520527999999 0 0.0) (7.576073185 0 0.0)
    t:   (7.522527273 0 0.0) (7.514555517 0 0.0

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> Thanks, so, I did that, and without disabling GC results look quite odd.
>
> For "lexical-binding: t" it is:
>
>     - ...                                        2 100%
>      - list                                      1  50%
>       - -                                        1  50%
>        - let                                     1  50%
>         - let                                    1  50%
>          - while                                 1  50%
>           - funcall                              1  50%
>            - #<compiled 0x15833de3b0c9>          1  50%
>             - c-font-lock-fontify-region         1  50%
>              - c-before-context-fl-expand-region 1  50%
>               - mapc                             1  50%
>                - #<compiled 0x15833db80701>      1  50%
>                 - c-context-expand-fl-region     1  50%
>                  - c-fl-decl-end                 1  50%
>                   - c-literal-start              1  50%
>                    - c-state-semi-pp-to-literal  1  50%
>                       c-parse-ps-state-below     1  50%
>        Automatic GC                              1  50%

The code didn't run long enough to get any meaningful profile:
profiler-start is sampling-based (and apparently in the above we only
got 2 samples) so make sure the code runs for several seconds (the more
the merrier).


        Stefan

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

On Sun, 10 Mar 2019 at 21:08, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > Thanks, so, I did that, and without disabling GC results look quite odd.
> >
> > For "lexical-binding: t" it is:
> >
> >     - ...                                        2 100%
> >      - list                                      1  50%
> >       - -                                        1  50%
> >        - let                                     1  50%
> >         - let                                    1  50%
> >          - while                                 1  50%
> >           - funcall                              1  50%
> >            - #<compiled 0x15833de3b0c9>          1  50%
> >             - c-font-lock-fontify-region         1  50%
> >              - c-before-context-fl-expand-region 1  50%
> >               - mapc                             1  50%
> >                - #<compiled 0x15833db80701>      1  50%
> >                 - c-context-expand-fl-region     1  50%
> >                  - c-fl-decl-end                 1  50%
> >                   - c-literal-start              1  50%
> >                    - c-state-semi-pp-to-literal  1  50%
> >                       c-parse-ps-state-below     1  50%
> >        Automatic GC                              1  50%
>
> The code didn't run long enough to get any meaningful profile:
> profiler-start is sampling-based (and apparently in the above we only
> got 2 samples) so make sure the code runs for several seconds (the more
> the merrier).

If you say "several seconds enough", then it's already there, the
benchmark with GC enabled takes 9 seconds.

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> If you say "several seconds enough", then it's already there, the
> benchmark with GC enabled takes 9 seconds.

Hmm... then there's a problem with the sampling.  Is this in a batch
Emacs session?  What OS?  I think this might deserve its own bug report.


        Stefan

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

On Sun, 10 Mar 2019 at 21:59, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > If you say "several seconds enough", then it's already there, the
> > benchmark with GC enabled takes 9 seconds.
>
> Hmm... then there's a problem with the sampling.  Is this in a batch
> Emacs session?  What OS?  I think this might deserve its own bug report.

It's `emacs -Q` ran one time. Archlinux here, but Emacs is not from a
repository, but a ≈month old version from git.

Something similar can be reproduced in 2 steps:

    1. wget https://gitlab.freedesktop.org/libinput/libinput/raw/9a2d6f55b1276da11dd9b2c4c8e22a405576dfea/src/libinput.h
    2. emacs -Q --eval "(progn (find-file \"./libinput.h\")
(profiler-start 'cpu) (benchmark-run-compiled 10
(c-font-lock-fontify-region 0 (point-max))) (profiler-report))"

Since by default cc-mode.el doesn't use lexical-binding, you should
get this result:
    - ...           1 100%
       Automatic GC 1 100%

Do you want me to open a report for above?

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> Do you want me to open a report for above?

Yes, please: this is definitely not right.


        Stefan

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

done https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34809

On Sun, 10 Mar 2019 at 23:48, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>
> > Do you want me to open a report for above?
>
> Yes, please: this is definitely not right.
>
>
>         Stefan

Re: Lexical binding doesn't seem to be faster?

[Hi-Angel]

So, it turned out to be a bug in linux 4.20.5 and earlier, so I
updated to 5.0, and everything is fine now.

I re-run tests, benchmark results now are tiny bit better for lexical
scope when GC is on, and are the same when GC is off:

With GC:
    t:   (8.451301031000002 87 1.5439787770000004), (8.492936438 85 1.527223621)
    nil: (8.654444538 87 1.5810563760000003), (8.549887916 85
1.5612690489999996)

No GC:
    t:   (7.548030708 0 0.0) (7.591869120999999 0 0.0)
    nil: (7.519884027 0 0.0) (7.577961806 0 0.0)

On Mon, 11 Mar 2019 at 00:22, Hi-Angel <hiangel999@gmail.com> wrote:
>
> done https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34809
>
> On Sun, 10 Mar 2019 at 23:48, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> >
> > > Do you want me to open a report for above?
> >
> > Yes, please: this is definitely not right.
> >
> >
> >         Stefan

Re: Lexical binding doesn't seem to be faster?

[Stefan Monnier]

> So, it turned out to be a bug in linux 4.20.5 and earlier, so I
> updated to 5.0, and everything is fine now.

Hmm... still at 4.18 here, so no wonder I wasn't seeing this problem.

> I re-run tests, benchmark results now are tiny bit better for lexical
> scope when GC is on, and are the same when GC is off:
>
> With GC:
>     t:   (8.451301031000002 87 1.5439787770000004), (8.492936438 85 1.527223621)
>     nil: (8.654444538 87 1.5810563760000003), (8.549887916 85
> 1.5612690489999996)
>
> No GC:
>     t:   (7.548030708 0 0.0) (7.591869120999999 0 0.0)
>     nil: (7.519884027 0 0.0) (7.577961806 0 0.0)

Hmm... looks like the difference might be statistically insignificant
after all (or maybe lexical-binding makes cc-engine.el marginally
faster, so it hides the slight slowdown in cc-mode.el).  I think you
might decide to drop this investigation, at least until you find more
clear evidence of a slowdown.


        Stefan


> On Mon, 11 Mar 2019 at 00:22, Hi-Angel <hiangel999@gmail.com> wrote:
>>
>> done https://debbugs.gnu.org/cgi/bugreport.cgi?bug=34809
>>
>> On Sun, 10 Mar 2019 at 23:48, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>> >
>> > > Do you want me to open a report for above?
>> >
>> > Yes, please: this is definitely not right.
>> >
>> >
>> >         Stefan