Re: Performance of lexical closures in interpreted code? (was Re: Execution speed)

[Stefan Monnier <monnier@iro.umontreal.ca>]

> Tangentially, that performance hit on the interpreted recursive
> code with lexical-binding enabled is pretty severe!  From well under
> 2 seconds (with dynamic binding) to well over 4 seconds (with lexical
> binding) in all my results.
> Is this a known/accepted trade-off for the better byte-compiled
> performance?

Yes and no: I consider the performance of interpreted code to be
basically irrelevant, so I haven't paid any attention to it when
introducing the lexical-binding functionality: when performance matters,
the code should be compiled.

> I'm assuming it's to do with overheads of lexical closures.

As you can guess from the above, I don't really know.  But I'd be
surprised if it's the case.  A more likely reason is the handling of the
lexical environment (e.g. all the let-rebinding of the dynamically-scoped
var which holds the lexical environment).

> The output npostavs generates indicates that it's largely (if
> not solely) on account of a vast quantity of garbage collection.
> Perhaps that is all the lexical environments being cleaned up?

Quite likely, yes.

> (Although I would have thought that was a requirement for the
> byte-compiled code as well, so I'm very much guessing.)

The lexical environment is mostly "compiled away".  In the bytecode, new
lexical vars are just pushed on the stack, whereas in interpreted code
a new lexical var means adding a new element to an alist (i.e. 2 `cons`).

> Can that be improved, or is this most likely to remain this way?

Of course it can be improved.  It might take a fair bit of effort, tho
(e.g. I think to be more efficient, you'll likely want to pass the
lexical-env as an additional argument rather than by storing it in
a global (dynamically-scoped) var, but this will require changes to all
the special forms (grep for UNEVALLED)).


        Stefan