"Bringing GNU Emacs to Native Code" at the European Lisp Symposium

"Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Kangas]

For some reason, this has not been announced here:

There was an interesting a talk today by Andrea Corallo on "Bringing
GNU Emacs to Native Code" at the European Lisp Symposium.[1]

Here is a link to Andrea's paper (page 68):
https://www.european-lisp-symposium.org/static/proceedings/2020.pdf

Here is a link to the talk (starts around 02:44:30):
https://www.twitch.tv/videos/604934409

(I'm assuming that twitch.tv is ridden with proprietary code, but I
couldn't find a better link.)

Best regards,
Stefan Kangas

1. https://www.european-lisp-symposium.org/2020/index.html

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Hi Stefan,

Stefan Kangas <stefan@marxist.se> writes:

> For some reason, this has not been announced here:

well I thought I generated enough traffic with that already :)

> There was an interesting a talk today by Andrea Corallo on "Bringing
> GNU Emacs to Native Code" at the European Lisp Symposium.[1]
>
> Here is a link to Andrea's paper (page 68):
> https://www.european-lisp-symposium.org/static/proceedings/2020.pdf
>
> Here is a link to the talk (starts around 02:44:30):
> https://www.twitch.tv/videos/604934409
>
> (I'm assuming that twitch.tv is ridden with proprietary code, but I
> couldn't find a better link.)

I'd like to upload the recording on some free software based streaming
platform but I'm totally lost on the subject.  Any suggestion?

> Best regards,
> Stefan Kangas
>
> 1. https://www.european-lisp-symposium.org/2020/index.html

Regards

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

Andrea Corallo <akrl@sdf.org> writes:

[...]
>
> I'd like to upload the recording on some free software based streaming
> platform but I'm totally lost on the subject.  Any suggestion?
>
[...]

Since you're already an SDF member, you can email their membership@ and
ask them to create an account for you on https://toobnix.org.  That's
where I've mirrored the videos from EmacsConf 2019.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[tomas]

[-- Attachment #1: Type: text/plain, Size: 192 bytes --]

On Tue, Apr 28, 2020 at 07:11:18PM +0200, Stefan Kangas wrote:

[...]

> https://www.european-lisp-symposium.org/static/proceedings/2020.pdf

Great. You ruined my weekend :-/

Thanks :)

-- t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Amin Bandali <bandali@gnu.org> writes:

> Since you're already an SDF member, you can email their membership@ and
> ask them to create an account for you on https://toobnix.org.  That's
> where I've mirrored the videos from EmacsConf 2019.

Hi Amin,

I tried to open it in the past with GNU LibreJS but failed to display
the page, but now checking better https://joinpeertube.org/ is claimed
to be Free Software.

I'll drop a mail then thanks for suggesting.

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Amin Bandali]

[-- Attachment #1: Type: text/plain, Size: 827 bytes --]

Hi Andrea,

Andrea Corallo <akrl@sdf.org> writes:

> Amin Bandali <bandali@gnu.org> writes:
>
>> Since you're already an SDF member, you can email their membership@ and
>> ask them to create an account for you on https://toobnix.org.  That's
>> where I've mirrored the videos from EmacsConf 2019.
>
> Hi Amin,
>
> I tried to open it in the past with GNU LibreJS but failed to display
> the page, but now checking better https://joinpeertube.org/ is claimed
> to be Free Software.
>

Indeed, PeerTube is free software (under AGPLv3+) but they sadly do not
currently mark the JavaScript frontend as such in a way recognized by
GNU LibreJS.  There is an open issue about it on their tracker [0].

[0]: https://github.com/Chocobozzz/PeerTube/issues/1254

>
> I'll drop a mail then thanks for suggesting.
>
>   Andrea

Cheers,
amin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 857 bytes --]

RE: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Drew Adams]

1. It's a good paper, and the work sounds great.

2. FWIW, I don't agree with this prognostication
or point of view, from the paper, starting after
"since":

 "The proposed compiler focuses on generating
  code for the new lexically scoped dialect only,
  since the dynamic one is considered obsolete
  and close to deprecation."

Dunno who, besides perhaps Stefan, considers
dynamic binding in Elisp to be "obsolete and
close to deprecation".  That would be a mistake.

IMO, Emacs Lisp should, like Common Lisp and for
even stronger reasons, continue to make use of
both dynamic and lexical binding.  Each has its
uses in Elisp.

The "even stronger" comes from the particular
use, for users in particular, described by RMS
in his 1981 defense of it:

https://www.gnu.org/software/emacs/emacs-paper.html#SEC17

https://www.gnu.org/software/emacs/emacs-paper.html#SEC18

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> Dunno who, besides perhaps Stefan, considers
> dynamic binding in Elisp to be "obsolete and
> close to deprecation".  That would be a mistake.

You're confusing "dynamic binding" with "the dynamically scoped dialect
of Elisp".  Dynamic binding is very much alive in "the lexically scoped
dialect of Elisp" with no plan to deprecate it.


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Dmitry Gutov]

On 29.04.2020 01:00, Drew Adams wrote:
> Dunno who, besides perhaps Stefan, considers
> dynamic binding in Elisp to be "obsolete and
> close to deprecation".  That would be a mistake.

Not dynamic binding, but the "dynamic scoped dialect".

> IMO, Emacs Lisp should, like Common Lisp and for
> even stronger reasons, continue to make use of
> both dynamic and lexical binding.  Each has its
> uses in Elisp.

Yes. And that's the "lexically scoped dialect".

RE: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Drew Adams]

> > Dunno who, besides perhaps Stefan, considers
> > dynamic binding in Elisp to be "obsolete and
> > close to deprecation".  That would be a mistake.
> 
> You're confusing "dynamic binding" with "the
> dynamically scoped dialect of Elisp".  Dynamic
> binding is very much alive in "the lexically scoped
> dialect of Elisp" with no plan to deprecate it.

You can say I'm confused, and you throw out
undefined terms like "the ___ scoped dialect of
Elisp".  (Undefined in both your mail and the
paper.  As one of the two main reviewers of the
paper, perhaps that's your undefined term?)

I'm talking about what Common Lisp calls
"indefinite scope and dynamic extent", i.e.,
what Common Lisp special variables have.

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html

What does your "dynamically scoped dialect of
Elisp" entail, that's not included in the use
of special variables (infinite scope and dynamic
extent)?  What's excluded in the limitation to
"the new lexically scoped dialect only"?

If all that you really mean is the behavior when
variable `lexical-binding' is nil, then that
really should have been made clear.  Emacs and
Elisp (e.g. the doc) make no mention of that
variable defining two different Lisp dialects.

If that's all that's meant, it's OK by me.  But
such terminology is misleading (IMO), without
the explanation.

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'd like to upload the recording on some free software based streaming
  > platform but I'm totally lost on the subject.  Any suggestion?

1. What software the _platform_ is based on is not crucial,
because that has no effect on the people who use that platform.

2. The crucial question for the users' freedom is whether the site
requires them. or leads them, to run any nonfree software,
including nonfree Javascript code.

3. Any web site can host a file of video in such a way that any browser
can view it without ANY Javascript.  Just put the file of video onto the site,
and tell people its URL.  Any modern graphical browser, when it encounters
a file of video, will stream it.

4. The only special thing about "video platform" sites is that they
have other auxiliary facilities, for things such as uploading
videos. tracking users who watch them, and restricting those users.
You don't need those things.

5. You do need a site that you can post the file on.
Someone here surely has a site and can post this file on it.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Tue, 28 Apr 2020 18:18:23 -0400
> Cc: Andrea Corallo <akrl@sdf.org>, Stefan Kangas <stefan@marxist.se>,
>  Emacs developers <emacs-devel@gnu.org>
> 
> > Dunno who, besides perhaps Stefan, considers
> > dynamic binding in Elisp to be "obsolete and
> > close to deprecation".  That would be a mistake.
> 
> You're confusing "dynamic binding" with "the dynamically scoped dialect
> of Elisp".

Perhaps you should define what "the dynamically scoped dialect of
Elisp" entails, then.  Because AFAIU the ELisp manual says that
dynamic bindings have dynamic scope.  It's easy to interpret that to
mean that the above two terms have the same practical meaning in
Emacs.

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Hi Drew,

Drew Adams <drew.adams@oracle.com> writes:

> 1. It's a good paper, and the work sounds great.

Thanks that's appreciated

> 2. FWIW, I don't agree with this prognostication
> or point of view, from the paper, starting after
> "since":
>
>  "The proposed compiler focuses on generating
>   code for the new lexically scoped dialect only,
>   since the dynamic one is considered obsolete
>   and close to deprecation."
>
> Dunno who, besides perhaps Stefan, considers
> dynamic binding in Elisp to be "obsolete and
> close to deprecation".  That would be a mistake.
>
> IMO, Emacs Lisp should, like Common Lisp and for
> even stronger reasons, continue to make use of
> both dynamic and lexical binding.  Each has its
> uses in Elisp.
>

As the reference in the previous phrase explains this is just about what
we control in Emacs with the `lexical-binding' variable.

Apologies if you think this could have been phrased better, I hope the
misunderstanding is clarified.

Regards

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> Perhaps you should define what "the dynamically scoped dialect of
> Elisp" entails, then.  Because AFAIU the ELisp manual says that
> dynamic bindings have dynamic scope.  It's easy to interpret that to
> mean that the above two terms have the same practical meaning in
> Emacs.

They do have the same meaning.  The difference is in what you apply it
to: a specific binding, or a language dialect.

Maybe we should stop talking about our language dialects being either
"dynamically scoped" or "lexically scoped" because it's reductive and
just talk about Elisp/d (the dialect where all bindings are dynamically
scoped by default, except those using `lexical-let`) and Elisp/l (the
dialect where all bindings are lexically scoped by default except those
applied to vars that have been declared as dynamically scoped)?


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: drew.adams@oracle.com,  akrl@sdf.org,  stefan@marxist.se,
>   emacs-devel@gnu.org
> Date: Wed, 29 Apr 2020 08:38:25 -0400
> 
> Maybe we should stop talking about our language dialects being either
> "dynamically scoped" or "lexically scoped" because it's reductive and
> just talk about Elisp/d (the dialect where all bindings are dynamically
> scoped by default, except those using `lexical-let`) and Elisp/l (the
> dialect where all bindings are lexically scoped by default except those
> applied to vars that have been declared as dynamically scoped)?

If that's what you meant, then the intent is clear.

RE: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Drew Adams]

> > FWIW, I don't agree with this prognostication
> > or point of view, from the paper, starting
> > after "since":
> >
> >  "The proposed compiler focuses on generating
> >   code for the new lexically scoped dialect only,
> >   since the dynamic one is considered obsolete
> >   and close to deprecation."
> >
> > Dunno who, besides perhaps Stefan, considers
> > dynamic binding in Elisp to be "obsolete and
> > close to deprecation".  That would be a mistake.
> >
> > IMO, Emacs Lisp should, like Common Lisp and for
> > even stronger reasons, continue to make use of
> > both dynamic and lexical binding.  Each has its
> > uses in Elisp.
> 
> As the reference in the previous phrase explains
> this is just about what we control in Emacs with
> the `lexical-binding' variable.

Dunno what previous phrase you refer to.  That
variable isn't mentioned in the paper (other
than appearing in a code example).

A suggestion would be to be explicit about this
in the future - or else explain the phrase.

I personally think the phrase used is confusing,
and perhaps misleading.  Yes, one could argue
that variable `lexical-binding' kind of splits
Elisp currently into two languages.  But that's
not a usual way of looking at it, and it's not
the way that Emacs talks about itself.

At some point the default value of that variable
may be (will likely be) `t' - a good thing, IMO.
At that point, there'll be no possibility to
speak of such "dialects".  My opinion would be
to also avoid speaking of them now.

Count me as one who was misled/confused by that
particular phrase, and doesn't think it's
helpful without some explanation.

> Apologies if you think this could have been
> phrased better, I hope the misunderstanding
> is clarified.

No need to apologize, at all.  It's clear to me
now; thank you for clarifying.

And thanks for the great work (!) and clear
paper about it.

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Hi all,

I've updated http://akrl.sdf.org/gccemacs.html with a download link in
OGV video format and alternatively a streaming option on the suggested
PeerTube instance.

Thanks

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Drew Adams <drew.adams@oracle.com> writes:

>> As the reference in the previous phrase explains
>> this is just about what we control in Emacs with
>> the `lexical-binding' variable.
>
> Dunno what previous phrase you refer to.  That
> variable isn't mentioned in the paper (other
> than appearing in a code example).
>
> A suggestion would be to be explicit about this
> in the future - or else explain the phrase.

I was referring to the phrase just before the one we are discussing:

"We point out that, since Emacs Lisp received in 2012 lexical scope
support, two different sub-languages are currently coexisting [15,
Sec. 8.1]."

> I personally think the phrase used is confusing,
> and perhaps misleading.  Yes, one could argue
> that variable `lexical-binding' kind of splits
> Elisp currently into two languages.  But that's
> not a usual way of looking at it, and it's not
> the way that Emacs talks about itself.

I agree with you that could have been stated more clearly without
assuming the user had visited the reference (this is not a correct
assumption).

>
>> Apologies if you think this could have been
>> phrased better, I hope the misunderstanding
>> is clarified.
>
> No need to apologize, at all.  It's clear to me
> now; thank you for clarifying.
>
> And thanks for the great work (!) and clear
> paper about it.

Thanks again.

  Andrea

-- 
akrl@sdf.org

RE: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Drew Adams]

>>> As the reference in the previous phrase explains
>>> this is just about what we control in Emacs with
>>> the `lexical-binding' variable.

>> A suggestion would be to be explicit about this
>> in the future - or else explain the phrase.
>
> I was referring to the phrase just before the
> one we are discussing:
> 
> "We point out that, since Emacs Lisp received in
> 2012 lexical scope support, two different
> sub-languages are currently coexisting [15, Sec. 8.1]."

I see, thanks.  So a reader who consults Sect 8.1
of that paper by Stefan will learn that he uses
the values of variable `lexical-binding' to define
two Elisp sub-languages.

Yes, that's certainly a complete reference.  But
that's not what I meant by suggesting to be
explicit about what you mean by the lexical Elisp
dialect, i.e., that you are, in effect, using
variable `lexical-binding' to define two dialects.

It's a lot clearer to just say that your work
assumes a non-nil value of that variable (and
there's nothing wrong with that), than it is to
either (1) hope that someone guesses what you
mean by the lexical Elisp dialect or (2) expect
that a reader will consult Sect 8.1 of Stefan's
paper and figure out what you mean from that.

> > I personally think the phrase used is confusing,
> > and perhaps misleading.  Yes, one could argue
> > that variable `lexical-binding' kind of splits
> > Elisp currently into two languages.  But that's
> > not a usual way of looking at it, and it's not
> > the way that Emacs talks about itself.
> 
> I agree with you that could have been stated more
> clearly without assuming the user had visited the
> reference (this is not a correct assumption).

No problem.  Hindsight's 20-20.  And perhaps
you thought that it's common to speak of such
Elisp dialects?  That might be a reasonable
assumption from reading Stefan's paper.  Now
you know that there's at least one Elisp user
who misunderstood.

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> It's a lot clearer to just say that your work
> assumes a non-nil value of that variable (and
> there's nothing wrong with that), than it is to
> either (1) hope that someone guesses what you
> mean by the lexical Elisp dialect or (2) expect
> that a reader will consult Sect 8.1 of Stefan's
> paper and figure out what you mean from that.

Or rather that this detail is mostly irrelevant in the context of his
paper.  For the same reason his paper doesn't explain how buffer-local
variables interact with dynamic scoping or a myriad of other details of
Elisp's semantics.


        Stefan


PS: Since his compiler takes the bytecode as input, it's largely
agnostic to the source's scoping rules.  I don't even know what makes
his code currently only work for Elisp/l and not Elisp/d.  It's most
likely a minor technical detail.

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

   > "The proposed compiler focuses on generating
   >  code for the new lexically scoped dialect only,
   >  since the dynamic one is considered obsolete
   >  and close to deprecation."

We might someday decide to deprecate dynamic binding mode, but that
would be years away.  Unless/until we do so, new features in Emacs
should support both.

It won't be difficult to support both.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

>    > "The proposed compiler focuses on generating
>    >  code for the new lexically scoped dialect only,
>    >  since the dynamic one is considered obsolete
>    >  and close to deprecation."
> We might someday decide to deprecate dynamic binding mode, but that
> would be years away.  Unless/until we do so, new features in Emacs
> should support both.

I don't see such a need, no.  Old code should work as well as before,
but I think it's perfectly OK to require using Elisp/l in order to
make use of some new features.  We already do that in various cases
(e.g. `gv-ref`, `thunk.el`, `generator.el` and I'm sure there are
others).

I might argue that old code doesn't need to be compiled to native code
since it ran fast enough with the byte-code.

That doesn't mean we should discourage making the native code compiler
work with old code, but just that it's secondary.


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Richard Stallman <rms@gnu.org> writes:

>    > "The proposed compiler focuses on generating
>    >  code for the new lexically scoped dialect only,
>    >  since the dynamic one is considered obsolete
>    >  and close to deprecation."
>
> We might someday decide to deprecate dynamic binding mode, but that
> would be years away.  Unless/until we do so, new features in Emacs
> should support both.
>
> It won't be difficult to support both.

Hi Richard,

I have not taken any design decision that excludes supporting dynamic
binding mode for the future.  I've been working on tasks that I consider
higher priority as completing the integration with Emacs and I'm
starting now with anonymous lambdas compilation.

We can always discuss the dynamic binding support but has also to be
considered that the performance uplift is expected to be smaller for
this.

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > but I think it's perfectly OK to require using Elisp/l in order to
  > make use of some new features.  We already do that in various cases
  > (e.g. `gv-ref`, `thunk.el`, `generator.el` and I'm sure there are
  > others).

Those issues are not comparable to this issue.  They are facilities
for use in code, and they have trouble with dynamic binding for
inherent reasons.  Even in lexical binding mode, they may fail to work
as expected with code that binds dynamic variables.

None of that applies to a compiler.  Any compiler that transforms Emacs Lisp
into something else must be able to deal with dynamic binding.  So there is
no reason it should have trouble with dynamic binding mode, that only means
it is not finished.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > We can always discuss the dynamic binding support but has also to be
  > considered that the performance uplift is expected to be smaller for
  > this.

Such is life.  But even if dynamic-default binding can't get the same
amount of speedup, it shoulod work correctly.

You might find a way to recognize that some bindings can't be seen by
any other functions, and treat them as lexical on the grounds that
doing so would not alter the result.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > We can always discuss the dynamic binding support but has also to be
>   > considered that the performance uplift is expected to be smaller for
>   > this.
>
> Such is life.  But even if dynamic-default binding can't get the same
> amount of speedup, it shoulod work correctly.

No doubt.  I realize I haven't explained my self very well:

I'm sure it would work, what I'm not sure of is if is a good technical
solution.

> You might find a way to recognize that some bindings can't be seen by
> any other functions, and treat them as lexical on the grounds that
> doing so would not alter the result.

This the point.  I think we could at best make this assumption only on
calls to C primitives (that we assume cannot be redefined) that are
known to be side effect free and not to use global variable bindings in
general.  Any other call must have all variables spilled first.

Given the frequency of calls and that almost all of these cannot be
proved to target one function of this very specific class, I think one
would end-up spilling almost always all variables anyway.

I always thought this is one of the main reasons why lexical scope was
introduced and I think this optimization is not worth being implemented.

I believe that if there will be a speed-up compiling dynamic code this
will come just from non having to decode byte-op-codes and manipulating
the execution stack.

In case the speed-up would come-up to be minimal imposing long
compilation times for a compiler that is unable of optimizing much would
be sad.

OTOH I think no one as data on this speed-up.  Given the discussed
suspect of poor performance I intentionally left this task aside to work
on more urgent tasks.  Nothing prevents to have a try on this later if
we think is really important.

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> None of that applies to a compiler.  Any compiler that transforms Emacs Lisp
> into something else must be able to deal with dynamic binding.

AFAIK it does *handle* such code (i.e. such code will run correctly), it
just doesn't optimize it (in this particular case it keeps it in `.elc` form).

> So there is no reason it should have trouble with dynamic binding
> mode, that only means it is not finished.

That's partly right.  But then again, there's always something else that
can be optimized.  You have to prioritize.  I think it's perfectly OK to
say that if you want higher performance you should start by adapting
your Elisp/d code to Elisp/l, especially since the compiler will be able
to do a much better job with Elisp/l than Elisp/d since when compiled
via libggcjit, a lexical variable will turn into (more or less) a plain
normal C variable whereas a dynamic variable will only be accessed via
the standard `symbol-value`, `set`, ... accessors.  For this reason, on
Elisp/d the native compiler won't be able to get much more speed up than
that resulting from removing the while+switch interpretive overhead,
whereas on Elisp/l it can go significantly further by allocating
variables to registers, removing type tests, boxing/unboxing pairs, ...


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> Such is life.  But even if dynamic-default binding can't get the same
> amount of speedup, it shoulod work correctly.

It does work correctly already.

> You might find a way to recognize that some bindings can't be seen by
> any other functions, and treat them as lexical on the grounds that
> doing so would not alter the result.

To get a good result out of this you need a non-trivial analysis
(there's actually a paper by a student of Mike Sperber that does exactly
that, BTW) and you still won't preserve the expected behavior when
advising or redefining functions.

It's much easier to adjust the source file to use Elisp/l instead of
Elisp/d and will give better results (and better compilation warnings,
etc...).


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > This the point.  I think we could at best make this assumption only on
  > calls to C primitives (that we assume cannot be redefined) that are
  > known to be side effect free and not to use global variable bindings in
  > general.  Any other call must have all variables spilled first.

You're concerned about the possibility of redefining Lisp functions
defined in other files, and that the new function definitions might
touch variables bound in your file.

I think it is legitimate to compile assuming they don't do that.  How
about seeing where you can get, that way?

  >   Given the discussed
  > suspect of poor performance I intentionally left this task aside to work
  > on more urgent tasks.  Nothing prevents to have a try on this later if
  > we think is really important.

That seems right to me.  I am not saying this particular issue is urgent.
Only that it should get done by and by.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> That seems right to me.  I am not saying this particular issue is urgent.
> Only that it should get done by and by.

I go a little step further: while it's good if someone does it (and
I think that it will likely come almost for free at some point),
I don't think it's necessary to do it eventually.


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > That seems right to me.  I am not saying this particular issue is urgent.
  > > Only that it should get done by and by.

  > I go a little step further: while it's good if someone does it (and
  > I think that it will likely come almost for free at some point),
  > I don't think it's necessary to do it eventually.

It is sloppy to leave part of the job unfinished.
We should finish the job.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

>   > I go a little step further: while it's good if someone does it (and
>   > I think that it will likely come almost for free at some point),
>   > I don't think it's necessary to do it eventually.
> It is sloppy to leave part of the job unfinished.
> We should finish the job.

We're talking about optimizing code: i.e., a job that can never be finished.


        Stefan

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Andrea Corallo]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > This the point.  I think we could at best make this assumption only on
>   > calls to C primitives (that we assume cannot be redefined) that are
>   > known to be side effect free and not to use global variable bindings in
>   > general.  Any other call must have all variables spilled first.
>
> You're concerned about the possibility of redefining Lisp functions
> defined in other files, and that the new function definitions might
> touch variables bound in your file.

Also in the same file.  The native compiler as the byte-compiler can
compile files as a whole or single functions too.  If one of these gets
redefined the assumptions used to compile all the others could become
wrong.

I suspect the assumption in discussion would be too weak and likely to
cause bugs that are very difficult to track down.

> I think it is legitimate to compile assuming they don't do that.  How
> about seeing where you can get, that way?
>
>   >   Given the discussed
>   > suspect of poor performance I intentionally left this task aside to work
>   > on more urgent tasks.  Nothing prevents to have a try on this later if
>   > we think is really important.
>
> That seems right to me.  I am not saying this particular issue is urgent.
> Only that it should get done by and by.

_I'm in favor_ of giving dynamic scope compilation a shot on day and see
how it runs (with the right prioritization).  But I'm not in favor of
designing and implementing complex passes that are dynamic scoped code
specific, especially if based on dangerous assumptions.

Is my believe that there are other tasks inside the compiler but also
outside of it in Emacs that deserves sooner man power I'd like to help
with.  This is just my optinion obviously.

Regards

  Andrea

-- 
akrl@sdf.org

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >   > I go a little step further: while it's good if someone does it (and
  > >   > I think that it will likely come almost for free at some point),
  > >   > I don't think it's necessary to do it eventually.
  > > It is sloppy to leave part of the job unfinished.
  > > We should finish the job.

  > We're talking about optimizing code: i.e., a job that can never be finished.

We're talking about implementing support, in the compilation package,
for the mode of operation which most Emacs code uses.

You may wish to deprecate that mode, but unless we do that soon,
it is no reason to leave that mode unsupported.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: "Bringing GNU Emacs to Native Code" at the European Lisp Symposium

[Stefan Monnier]

> You may wish to deprecate that mode, but unless we do that soon,
> it is no reason to leave that mode unsupported.

It is supported.  It is not optimized as effectively.
It's a question of optimization, not support.


        Stefan