Conditional binding and testing of `lexical-binding'

Conditional binding and testing of `lexical-binding'

[Drew Adams]

What's the best way, or a reasonable way, of
setting `lexical-binding' conditionally, so a
library can be used with both Emacs versions
that support lexical binding and versions
that don't support it?
___

As a simple example of testing, consider
something like this, in some context that
defines variable `x'.  (`x' could be a
function arg, a `let' var, or a `setq' var;
its binding could be lexical or dynamic.)

 (if (and (boundp 'lexical-binding)
          lexical-binding)
     (lambda (y) (something x y))
   `(lambda (y) (something ',x y)))

(Assume `x' doesn't occur in `y', etc.  By
hypothesis, occurrences of `x' can just be
_replaced_ by its value.  And the function
body could be any code, not necessarily a
single `something' function call, etc.)

That works.  (Of course, it has the drawback
that without lexical binding the value's not
a function; it's a list with car `lambda'.
So, not known to the byte-compiler to be a
function, not optimal, not elegant, not too
clean, etc.)

There are no doubt other ways to do something
similar.  (Feel free to suggest alternatives.)
___

But my question is really about conditionally
_setting_ `lexical-binding', so it can be tested.

Putting it in `Local Variables' at the end of
a file, and using `eval' to set its value (e.g.
conditionally, depending on Emacs version or
`boundp' or whatever), has no effect.  As the
doc says, we must instead set it in the first
file line.

And trying to set it conditionally this way
in a file apparently has no effect either:

 (ignore-errors (eval '(setq lexical-binding t)
                      t))

(That uses the 2-arg version of `eval', so it
raises an error in older Emacs versions; hence
the `ignore-errors'.)
___

It works to just put it in the first file line:

 ;;; ....... -*- lexical-binding:t -*-

In Emacs versions where that variable doesn't
exist, this apparently has no effect - the var
continues not to exist after the file's loaded.

[I don't see that the doc says that the var is
set only if it already exists (or whatever the
actual criterion is).  I was expecting that
that declaration would set `local-variables' to
`t' in older Emacs versions also, so I didn't
try it till after (unsuccessfully) trying other 
conditional approaches.  Shouldn't something be
said about this in the doc?]

Anyway, that first-line non-nil declaration
seems to work OK, e.g. with a test such as this:

 (and (boundp 'lexical-binding) lexical-binding)
___

Is this the thing to do?  If not, what advice
do you have for adapting a library to use
lexical binding when available (Emacs 24+) but
to also work when it's unavailable (Emacs < 24)?

[The doc just tells you how to convert code to
use lexical binding.  I see nothing about how
to code compatibly for old and new Emacs.]

Re: Conditional binding and testing of `lexical-binding'

[LdBeth]

>>>>> In <SJ0PR10MB5488D4372A6BCE46CF82D0E1F3479@SJ0PR10MB5488.namprd10.prod.outlook.com> 
>>>>>	Drew Adams <drew.adams@oracle.com> wrote:


>  (if (and (boundp 'lexical-binding)
>           lexical-binding)
>      (lambda (y) (something x y))
>    `(lambda (y) (something ',x y)))

You may use `static-if' to benifit from
byte-compiling.

> But my question is really about conditionally
> _setting_ `lexical-binding', so it can be tested.

I think the "lispy" way is to use:

(provide 'lexical-binding)

and use `featurep' to test it.

> Putting it in `Local Variables' at the end of
> a file, and using `eval' to set its value (e.g.
> conditionally, depending on Emacs version or
> `boundp' or whatever), has no effect.  As the
> doc says, we must instead set it in the first
> file line.

> And trying to set it conditionally this way
> in a file apparently has no effect either:

>  (ignore-errors (eval '(setq lexical-binding t)
>                       t))

> (That uses the 2-arg version of `eval', so it
> raises an error in older Emacs versions; hence
> the `ignore-errors'.)
> ___

> It works to just put it in the first file line:

>  ;;; ....... -*- lexical-binding:t -*-

> In Emacs versions where that variable doesn't
> exist, this apparently has no effect - the var
> continues not to exist after the file's loaded.

> [I don't see that the doc says that the var is
> set only if it already exists (or whatever the
> actual criterion is).  I was expecting that
> that declaration would set `local-variables' to
> `t' in older Emacs versions also, so I didn't
> try it till after (unsuccessfully) trying other 
> conditional approaches.  Shouldn't something be
> said about this in the doc?]

A more reliable way is to test

(static-if (assoc 'lexical-binding (buffer-local-variables))
   (provide 'lexical-binding))

instead of doing `boundp' test.

> Is this the thing to do?  If not, what advice
> do you have for adapting a library to use
> lexical binding when available (Emacs 24+) but
> to also work when it's unavailable (Emacs < 24)?

> [The doc just tells you how to convert code to
> use lexical binding.  I see nothing about how
> to code compatibly for old and new Emacs.]

Well, I think it's fine to just keep that library as-is.
Even in the current lastest release there are still many
builtin packages are not converted to using lexical binding
at all. Just name a few I know, supercite, enriched-mode 

Best,
LDB

Re: Conditional binding and testing of `lexical-binding'

[Po Lu]

LdBeth <andpuke@foxmail.com> writes:

> (static-if (assoc 'lexical-binding (buffer-local-variables))
>    (provide 'lexical-binding))
>
> instead of doing `boundp' test.

I think it should be runtime, because transferring compiled code between
Emacs 23 and more recent versions is relatively common IME.

>> Is this the thing to do?  If not, what advice
>> do you have for adapting a library to use
>> lexical binding when available (Emacs 24+) but
>> to also work when it's unavailable (Emacs < 24)?

>> [The doc just tells you how to convert code to
>> use lexical binding.  I see nothing about how
>> to code compatibly for old and new Emacs.]

+1, there should at least be a paragraph or two in the Lisp reference
manual about that.

OTOH, I don't think there's much to be gained by using lexical binding
at all in code that's supposed to be compatible with versions of Emacs
without lexical binding.

It's not that important.

Re: Conditional binding and testing of `lexical-binding'

[Stefan Monnier]

> OTOH, I don't think there's much to be gained by using lexical binding
> at all in code that's supposed to be compatible with versions of Emacs
> without lexical binding.

You get better byte-compiler warnings, and the native code generated
from it is of higher quality, so it's likely to be marginally faster in
such a build.

Nothing ground breaking, admittedly.


        Stefan

RE: [External] : Re: Conditional binding and testing of `lexical-binding'

[Drew Adams]

> > OTOH, I don't think there's much to be gained by using lexical binding
> > at all in code that's supposed to be compatible with versions of Emacs
> > without lexical binding.
> 
> You get better byte-compiler warnings, and the native code generated
> from it is of higher quality, so it's likely to be marginally faster in
> such a build.

Yes, I suggested that (but I said byte-compile,
not native-compile).  But of course that's only
when you compile with Emacs 24+.

It's the difference between Emacs knowing that
the value is a function versus it only being
able to know for sure that it's a list with a
`lambda' car.  When a compiler knows that it's
a function it can sometimes take advantage of
that - beyond just providing better warnings.

> Nothing ground breaking, admittedly.

Yes, I said that too.  Besides immediate minor
advantages, it presents the advantage of being
the code to use if/when the library later drops
compatibility with Emacs 23 and prior - that's
the part to keep, then.

RE: [External] : Re: Conditional binding and testing of `lexical-binding'

[Drew Adams]

> > (static-if (assoc 'lexical-binding (buffer-local-variables))
> >    (provide 'lexical-binding))
> > instead of doing `boundp' test.
> 
> I think it should be runtime, because transferring
> compiled code between Emacs 23 and more recent
> versions is relatively common IME.

My question is about taking advantage of Emacs
support for variable `lexical-binding' when
it's available (which means for Emacs 24+),
while still keeping compatibility for older
Emacs versions.

> >> [The doc just tells you how to convert code to
> >> use lexical binding.  I see nothing about how
> >> to code compatibly for old and new Emacs.]
> 
> +1, there should at least be a paragraph or two
> in the Lisp reference manual about that.

I suspect that this was left out or neglected
(un oublie) due to not thinking of developers
of 3rd-party code that intends to work with
multiple Emacs versions, including some with
and some without `lexical-binding' support.

I can only guess that such a use case just
hasn't occurred to whoever coded variable
`lexical-binding' in Emacs.

> OTOH, I don't think there's much to be gained
> by using lexical binding at all in code that's
> supposed to be compatible with versions of Emacs
> without lexical binding.
> 
> It's not that important.

I fear you're maybe not getting it.

Not all of the code in a given library needs
to be usable in each Emacs version that the
library is compatible with.

And sure, even for that code that is to be
usable both with and without lexical binding,
there's generally no _need_ to use it.

On the other hand, there's generally some
advantage to using it.  I mentioned as much
in the simple example I gave - the drawback
of returning a list with `lambda' car,
instead of returning an actual anonymous
function, is minor.

I didn't ask whether I should try to have
such compatibility in a library.  I asked
what the best way to do so is.  I mentioned
how I intend to do it, and asked for advice
about that (alternative ways, a better way).

Re: [External] : Re: Conditional binding and testing of `lexical-binding'

[Po Lu]

Drew Adams <drew.adams@oracle.com> writes:

> I fear you're maybe not getting it.

I admit I didn't look at your original post, so I probably missed some
context.

I'm speaking from my own experience of writing software that had to work
with both old and new versions of Emacs.  There, lexical binding did not
pose much of an advantage.

Thanks.

RE: [External] : Re: Conditional binding and testing of `lexical-binding'

[Drew Adams]

> >  (if (and (boundp 'lexical-binding)
> >           lexical-binding)
> >      (lambda (y) (something x y))
> >    `(lambda (y) (something ',x y)))
> 
> You may use `static-if' to benifit from
> byte-compiling.

I see no `static-if' in any Emacs release, from
emacs -Q.  (The most recent Emacs release is 27.2.)

Certainly it's not present in Emacs 23 or earlier,
which is where there's no variable `lexical-let'.
So clearly it can't be used in Emacs 23 to test
whether lexical binding is supported.

I wonder whether you might have misread my post.
This is about a library that intends to be usable
with Emacs releases prior to Emacs 24 (where that
var is introduced), as well as with later Emacs
releases.

This is not about _converting_ a library to always
use `lexical-binding', which would only be useful
for Emacs 24 and later.

It's about being able to take advantage of
`lexical-binding' for Emacs 24+, while still being
compatible with versions earlier than 24.

> > But my question is really about conditionally
> > _setting_ `lexical-binding', so it can be tested.
> 
> I think the "lispy" way is to use: 
> (provide 'lexical-binding)> and use `featurep' to test it.

See above, or reread my first post.  (And none of
my code is providing feature `lexical-binding'.
Just setting that var to t does not "provide" it
as a supported feature.  It's either supported by
a given version of Emacs or it's not.)

> A more reliable way

To do what?  This apparently has nothing to do
with the quoted text it followed (?).

> is to test
> (static-if (assoc 'lexical-binding (buffer-local-variables))
>    (provide 'lexical-binding))
> instead of doing `boundp' test.

Sorry, but that makes no sense to me.  Emacs
support for `lexical-binding' is true (for
Emacs 24+) regardless of whether that var is
buffer-local variable (and with any value).

> > Is this the thing to do?  If not, what advice
> > do you have for adapting a library to use
> > lexical binding when available (Emacs 24+) but
> > to also work when it's unavailable (Emacs < 24)?
>
> > [The doc just tells you how to convert code to
> > use lexical binding.  I see nothing about how
> > to code compatibly for old and new Emacs.]
> 
> Well, I think it's fine to just keep that library as-is.
> Even in the current lastest release there are still many
> builtin packages are not converted to using lexical binding
> at all. Just name a few I know, supercite, enriched-mode

Certainly it's OK to keep such a library as is.
I've been doing that for years.

But I'm asking how to let it take advantage of
lexical binding when that's available.  That's
the question I posed.  I said how I intend to
do that, but I asked if there's a better way.

Re: [External] : Re: Conditional binding and testing of `lexical-binding'

[LdBeth]

>>>>> In 
 <SJ0PR10MB548898B6BA9D69C29FABB6A9F3489@SJ0PR10MB5488.namprd10.prod.outlook.com> 
>>>>>	Drew Adams <drew.adams@oracle.com> wrote:
>> >  (if (and (boundp 'lexical-binding)
>> >           lexical-binding)
>> >      (lambda (y) (something x y))
>> >    `(lambda (y) (something ',x y)))
>> 
>> You may use `static-if' to benifit from
>> byte-compiling.

> I see no `static-if' in any Emacs release, from
> emacs -Q.  (The most recent Emacs release is 27.2.)

> Certainly it's not present in Emacs 23 or earlier,
> which is where there's no variable `lexical-let'.
> So clearly it can't be used in Emacs 23 to test
> whether lexical binding is supported.

It's a macro from APEL, so sorry for not making it clear
that it is not part of Emacs.
But I thought it would be helpful because it seems
you want to take the advantages of speed gain via byte
compiling.

APEL is a library that supports Emacs versions as
old as 18 or XEmacs.

https://directory.fsf.org/wiki/Apel

This is the definition of `static-if' macro.

```
(put 'static-if 'lisp-indent-function 2)
(defmacro static-if (cond then &rest else)
  "Like `if', but evaluate COND at compile time."
  (if (eval cond)
      then
    `(progn ,@ else)))
```

> I wonder whether you might have misread my post.
> This is about a library that intends to be usable
> with Emacs releases prior to Emacs 24 (where that
> var is introduced), as well as with later Emacs
> releases.

> This is not about _converting_ a library to always
> use `lexical-binding', which would only be useful
> for Emacs 24 and later.

> It's about being able to take advantage of
> `lexical-binding' for Emacs 24+, while still being
> compatible with versions earlier than 24.


>> > But my question is really about conditionally
>> > _setting_ `lexical-binding', so it can be tested.
>> 
>> I think the "lispy" way is to use: 
>> (provide 'lexical-binding)> and use `featurep' to test it.

> See above, or reread my first post.  (And none of
> my code is providing feature `lexical-binding'.
> Just setting that var to t does not "provide" it
> as a supported feature.  It's either supported by
> a given version of Emacs or it's not.)

>> A more reliable way

> To do what?  This apparently has nothing to do
> with the quoted text it followed (?).

>> is to test
>> (static-if (assoc 'lexical-binding (buffer-local-variables))
>>    (provide 'lexical-binding))
>> instead of doing `boundp' test.

> Sorry, but that makes no sense to me.  Emacs
> support for `lexical-binding' is true (for
> Emacs 24+) regardless of whether that var is
> buffer-local variable (and with any value).

Sorry for I was making that conclusion based on
the wrong assumption about how buffer local
variables works.

But I think instead of testing whether `lexical-binding'
has been bound,

```
(if (>= emacs-major-version 24)
    (provide 'lexical-binding))
```

testing major version number seems to be easier since
it is unlikely someone would take an old version of
Emacs and patch for lexical binding only, or use a
version above 24 but taking out the lexical binding
support.

And instead of doing the lexical binding support test
in every file, having the `provide' statement in one
file loaded before any other files and use `featurep'
as a universal test of that feature seems to be more
appropriate.

>> > Is this the thing to do?  If not, what advice
>> > do you have for adapting a library to use
>> > lexical binding when available (Emacs 24+) but
>> > to also work when it's unavailable (Emacs < 24)?
>>
>> > [The doc just tells you how to convert code to
>> > use lexical binding.  I see nothing about how
>> > to code compatibly for old and new Emacs.]
>> 
>> Well, I think it's fine to just keep that library as-is.
>> Even in the current lastest release there are still many
>> builtin packages are not converted to using lexical binding
>> at all. Just name a few I know, supercite, enriched-mode

> Certainly it's OK to keep such a library as is.
> I've been doing that for years.

> But I'm asking how to let it take advantage of
> lexical binding when that's available.  That's
> the question I posed.  I said how I intend to
> do that, but I asked if there's a better way.

I apologize again for not reading your post carefully.

However, as Stefan says, what lexical-binding provides
is "nothing ground breaking", even you've already
hit a critical performance bottle neck, converting
to lexical binding would help little.

In my opinion, adopting lexical-binding is mainly
for making the code cleaner. Having both lexical
binding and dynamic binding in one library, seems
to be contrary to that goal.

-- 
LDB

RE: [External] : Re: Conditional binding and testing of `lexical-binding'

[Drew Adams]

> Sorry for I was making that conclusion based on
> the wrong assumption about how buffer local
> variables works.
> 
> But I think instead of testing whether
> `lexical-binding' has been bound,
> (if (>= emacs-major-version 24)
>     (provide 'lexical-binding))

My question has nothing to do with
`provide'ing a `lexical-binding' feature.

Adding such a feature is neither needed
nor helpful.  (And with that name it
might even mislead.)

> testing major version number seems to be easier since
> it is unlikely someone would take an old version of
> Emacs and patch for lexical binding only, or use a
> version above 24 but taking out the lexical binding
> support.

Testing the Emacs version # is OK also,
of course.  But that coding difference
too is irrelevant for my question/post.
It's not about that, at all.

And in general it's a bit better to test
for a variable or function definition
(`(f)boundp') than it is to test for an
Emacs version.  For one thing, it lets
a reader of the code know specifically
what's needed from that version.

> And instead of doing the lexical binding support test
> in every file, having the `provide' statement in one
> file loaded before any other files and use `featurep'
> as a universal test of that feature seems to be more
> appropriate.

My question is specific to a single file.
It's even specific to particular parts of
a file.

> I apologize again for not reading your post carefully.

No problem.  At least you tried to help.
Thx.

> However, as Stefan says, what lexical-binding provides
> is "nothing ground breaking", even you've already
> hit a critical performance bottle neck, converting
> to lexical binding would help little.

I'm well aware of what it has to offer.  But thx.

> In my opinion, adopting lexical-binding is mainly
> for making the code cleaner. Having both lexical
> binding and dynamic binding in one library, seems
> to be contrary to that goal.

What I asked about and mentioned is not about
having lexical and dynamic binding in the same
library - that is, for the same Emacs version.

Variable `lexical-binding' is itself not about
having only lexical binding.  It's simply about
Emacs support for lexical binding _in addition_
to dynamic binding (what Common Lisp has offered
since its outset).

Re: Conditional binding and testing of `lexical-binding'

[Stefan Monnier]

> What's the best way, or a reasonable way, of
> setting `lexical-binding' conditionally,

I don't know what you mean by that, nor why you think it's necessary
for:

> so a library can be used with both Emacs versions that support lexical
> binding and versions that don't support it?

Adding the -*- lexical-binding:t -*- at the beginning of the file
does that (this is used by CC-mode, for example).


        Stefan

RE: [External] : Re: Conditional binding and testing of `lexical-binding'

[Drew Adams]

> > What's the best way, or a reasonable way, 
> > of setting `lexical-binding' conditionally,
> 
> I don't know what you mean by that, 

And yet I gave details to make it clear, as
well as examples.  Which part(s) didn't you
understand?

> nor why you think it's necessary for:

If you didn't understand what "it" is, then
how is it that you think it's not necessary
for what follows?

> > so a library can be used with both Emacs
> > versions that support lexical binding
> > and versions that don't support it?

There you have it: setting it conditionally
so it can be tested, to check whether the
Emacs version being used supports lexical
binding.

> Adding the -*- lexical-binding:t -*- at the
> beginning of the file does that

Thanks for confirming that this is the best
(the only?) way to do it.

If you read my mail you'll see that I figured
out, by experimenting, that it's _sufficient_
to do that, to be able to test even with old
Emacs versions (which don't have variable
`lexical-binding').

This isn't obvious (and it's undocumented,
AFAICT).

It wasn't clear, before experimenting, what
that declaration would mean for Emacs 23 or
earlier - which behavior:

1. It implicitly creates a defvar and sets the
   value to `t'.

2. It's ignored, so the variable is undefined
   (unbound).

3. Something else.

Before experimenting, I was guessing #1.  But
it seems to be #2.

Knowing this is important, for testing the 
variable.  As I said, the test I apparently
need is this:

 (and (boundp 'lexical-binding)
      lexical-binding)

[Emacs 22 has `bound-and-true-p', but older
versions don't have it.]

If it were #1 then this wouldn't help.  The
var would be `t' for all Emacs versions, even
those for which that variable has no effect.

Please consider documenting this in the manual.

It _looks_ very much like the declaration sets
var `lexical-binding' to t, regardless of the
Emacs version.  But it seems it does not.

> (this is used by CC-mode, for example).

I don't see that in cc-mode.el for the latest
Emacs release (which is 27.2).

Even if it'll be present for Emacs 28 or later
(when released), that's no proof or even a
hint that the given library is compatible with
Emacs 23, 22,...