bug#74145: 31.0.50; Default lexical-binding to t

[Stefan Monnier via "Bug reports for GNU Emacs, the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

> Great, thanks.

Updated patch below.

> Meanwhile, here's two more things I thought of:
> - I think there's some text in doc/lispref/tips.texi that will need
>   updating too.

I didn't think it necessary, but after taking another look at it,
I tweaked a few more things, indeed.

> - Should we provide any advice somewhere for Emacs Lisp authors that
>   want to support Emacs 30 or older?  For example, they will want to
>   keep the lexical-binding:t cookie.

Currently, the doc recommends to add the cookie "everywhere", and my
patch doesn't really change that, so I think we're OK in this respect
(the new change in `tips.texi` does mention that the cookie is needed
only for compatibility with Emacs<31).


        Stefan



[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Change-the-default-value-of-lexical-binding.patch --]
[-- Type: text/x-diff, Size: 15417 bytes --]

From e69585bfbca950bce14fb51db385a72f6ecb553d Mon Sep 17 00:00:00 2001
From: Stefan Monnier <monnier@iro.umontreal.ca>
Date: Thu, 31 Oct 2024 16:00:10 -0400
Subject: [PATCH] Change the default value of `lexical-binding`

* src/lread.c (syms_of_lread): Set `Vlexical_binding` to Qt.
(Fload, Feval_buffer): Don't assume the default value of
`lexical-binding` is nil.
* lisp/loadup.el: Don't assume the default value of `lexical-binding`
is nil.

* doc/misc/cl.texi (Porting Common Lisp):
* doc/lispref/tips.texi (Coding Conventions, Library Headers):
* doc/lispref/variables.texi (Variable Scoping, Selecting Lisp Dialect)
(Void Variables): Adjust to new default of `lexical-binding`.

* doc/lispref/edebug.texi (Edebug Eval): Remove out of date paragraph.

* doc/lispintro/emacs-lisp-intro.texi (How let Binds Variables)
(Lexical vs. Dynamic Binding Example, defvar): Only consider the
lexical dialect of ELisp.
---
 doc/lispintro/emacs-lisp-intro.texi | 26 +++++++----------------
 doc/lispref/edebug.texi             |  7 ------
 doc/lispref/tips.texi               |  6 ++++--
 doc/lispref/variables.texi          | 33 +++++++++++++++--------------
 doc/misc/cl.texi                    |  9 ++++----
 etc/NEWS                            |  6 ++++++
 lisp/loadup.el                      |  4 ++--
 src/lread.c                         | 14 ++++++++----
 8 files changed, 52 insertions(+), 53 deletions(-)

diff --git a/doc/lispintro/emacs-lisp-intro.texi b/doc/lispintro/emacs-lisp-intro.texi
index 49916235fbf..6cc7a5b3cf2 100644
--- a/doc/lispintro/emacs-lisp-intro.texi
+++ b/doc/lispintro/emacs-lisp-intro.texi
@@ -3754,21 +3754,12 @@ How let Binds Variables
 
 Emacs Lisp supports two different ways of binding variable names to
 their values.  These ways affect the parts of your program where a
-particular binding is valid.  For historical reasons, Emacs Lisp uses
-a form of variable binding called @dfn{dynamic binding} by default.
-However, in this manual we discuss the preferred form of binding,
-called @dfn{lexical binding}, unless otherwise noted (in the future,
-the Emacs maintainers plan to change the default to lexical binding).
+particular binding is valid.
+In this manual we discuss the preferred form of binding,
+called @dfn{lexical binding}, unless otherwise noted.
 If you have programmed in other languages before, you're likely
 already familiar with how lexical binding behaves.
 
-In order to use lexical binding in a program, you should add this to
-the first line of your Emacs Lisp file:
-
-@example
-;;; -*- lexical-binding: t -*-
-@end example
-
 For more information about this, @pxref{Variable Scoping, , ,
 elisp, The Emacs Lisp Reference Manual}.
 
@@ -3818,8 +3809,6 @@ Lexical vs. Dynamic Binding Example
 For example, see what happens in this code under lexical binding:
 
 @example
-;;; -*- lexical-binding: t -*-
-
 (setq x 0)
 
 (defun getx ()
@@ -3845,7 +3834,7 @@ Lexical vs. Dynamic Binding Example
 If we use dynamic binding instead, the behavior is different:
 
 @example
-;;; -*- lexical-binding: nil -*-
+(defvar x) ;; Use dynamic binding for 'x'.
 
 (setq x 0)
 
@@ -3867,8 +3856,9 @@ Lexical vs. Dynamic Binding Example
 @code{x}, since its binding is below the one from our @code{let}
 expression in the stack of bindings.
 
-(Some variables are also ``special'', and they are always dynamically
-bound even when @code{lexical-binding} is @code{t}.  @xref{defvar, ,
+(The @code{defvar} declaration above is said to make the variable
+``special'', which causes it to obey the dynamic binding rules instead of
+the default lexical binding rules.  @xref{defvar, ,
 Initializing a Variable with @code{defvar}}.)
 
 @node if
@@ -9229,7 +9219,7 @@ defvar
 The @code{defvar} special form is similar to @code{setq} in that it
 sets the value of a variable.  It is unlike @code{setq} in three ways:
 first, it marks the variable as ``special'' so that it is always
-dynamically bound, even when @code{lexical-binding} is @code{t}
+dynamically bound
 (@pxref{How let Binds Variables}).  Second, it only sets the value of
 the variable if the variable does not already have a value.  If the
 variable already has a value, @code{defvar} does not override the
diff --git a/doc/lispref/edebug.texi b/doc/lispref/edebug.texi
index 03fae67e528..fe617550b09 100644
--- a/doc/lispref/edebug.texi
+++ b/doc/lispref/edebug.texi
@@ -722,13 +722,6 @@ Edebug Eval
 pretty-printed in a separate buffer.
 @end table
 
-@cindex lexical binding (Edebug)
-  Edebug supports evaluation of expressions containing references to
-lexically bound symbols created by the following constructs in
-@file{cl.el}: @code{lexical-let}, @code{macrolet}, and
-@code{symbol-macrolet}.
-@c FIXME?  What about lexical-binding = t?
-
 @node Eval List
 @subsection Evaluation List Buffer
 
diff --git a/doc/lispref/tips.texi b/doc/lispref/tips.texi
index 14eabb3558f..2b1b79a7f4f 100644
--- a/doc/lispref/tips.texi
+++ b/doc/lispref/tips.texi
@@ -96,7 +96,7 @@ Coding Conventions
 alternative common prefixes, so long as they make sense.
 
 @item
-We recommend enabling @code{lexical-binding} in new code, and
+We recommend not disabling @code{lexical-binding} in new code, and
 converting existing Emacs Lisp code to enable @code{lexical-binding}
 if it doesn't already.  @xref{Selecting Lisp Dialect}.
 
@@ -1082,7 +1082,9 @@ Library Headers
 @end example
 
 @noindent
-The description should be contained in one line.  If the file needs to
+The description should be contained in one line.
+The @code{lexical-binding} is not necessary if you do not care about
+compatibility with Emacs<31.  If the file needs to
 set more variables in the @samp{-*-} specification, add it after
 @code{lexical-binding}.  If this would make the first line too long, use
 a Local Variables section at the end of the file.
diff --git a/doc/lispref/variables.texi b/doc/lispref/variables.texi
index af11e498d86..d4b179003e3 100644
--- a/doc/lispref/variables.texi
+++ b/doc/lispref/variables.texi
@@ -285,7 +285,7 @@ Local Variables
 @defspec letrec (bindings@dots{}) forms@dots{}
 This special form is like @code{let*}, but all the variables are bound
 before any of the local values are computed.  The values are then
-assigned to the locally bound variables.  This is only useful when
+assigned to the locally bound variables.  This is useful only when
 lexical binding is in effect, and you want to create closures that
 refer to bindings that would otherwise not yet be in effect when using
 @code{let*}.
@@ -351,7 +351,7 @@ Local Variables
 done so that the value returned by the call is the value of @var{body}
 itself, as is the case in the recursive call to @code{sum} above.
 
-@code{named-let} can only be used when lexical-binding is enabled.
+@code{named-let} can be used only when lexical-binding is enabled.
 @xref{Lexical Binding}.
 @end defspec
 
@@ -383,7 +383,7 @@ Void Variables
   We say that a variable is void if its symbol has an unassigned value
 cell (@pxref{Symbol Components}).
 
-  Under Emacs Lisp's default dynamic scoping rule (@pxref{Variable
+  Under Emacs Lisp's dynamic scoping rule (@pxref{Variable
 Scoping}), the value cell stores the variable's current (local or
 global) value.  Note that an unassigned value cell is @emph{not} the
 same as having @code{nil} in the value cell.  The symbol @code{nil} is
@@ -392,7 +392,7 @@ Void Variables
 to evaluate the variable signals a @code{void-variable} error, instead
 of returning a value.
 
-  Under the optional lexical scoping rule, the value cell only holds
+  Under the lexical scoping rule, the value cell holds only
 the variable's global value---the value outside of any lexical binding
 construct.  When a variable is lexically bound, the local value is
 determined by the lexical environment; hence, variables can have local
@@ -514,7 +514,7 @@ Defining Variables
 
 Note that specifying a value, even @code{nil}, marks the variable as
 special permanently.  Whereas if @var{value} is omitted then the
-variable is only marked special locally (i.e.@: within the current
+variable is marked special only locally (i.e.@: within the current
 lexical scope, or file if at the top-level).  This can be useful for
 suppressing byte compilation warnings, see @ref{Compiler Errors}.
 
@@ -706,8 +706,8 @@ Accessing Variables
   The usual way to reference a variable is to write the symbol which
 names it.  @xref{Symbol Forms}.
 
-  Occasionally, you may want to reference a variable which is only
-determined at run time.  In that case, you cannot specify the variable
+  Occasionally, you may want to reference a variable which is determined
+only at run time.  In that case, you cannot specify the variable
 name in the text of the program.  You can use the @code{symbol-value}
 function to extract the value.
 
@@ -1003,14 +1003,15 @@ Variable Scoping
 while the binding construct (such as the body of a @code{let} form) is
 being executed.
 
-  The old dynamic-only Emacs Lisp dialect is still the default in code
+  The new lexically scoped Emacs Lisp dialect is now the default in code
 loaded or evaluated from Lisp files that lack a dialect declaration.
-Eventually the modern dialect will be made the default.
-All Lisp files should declare the dialect used to ensure that they
-keep working correctly in the future.
+Lisp files which still need the old dynamically scoped dialect need to declare
+it explicitly to ensure that they keep working correctly, and should
+eventually be converted to use the new dialect to
+ensure that they keep working correctly in the future.
 
   The following subsections describe lexical binding and dynamic
-binding in greater detail, and how to enable lexical binding in Emacs
+binding in greater detail, and how to disable lexical binding in Emacs
 Lisp programs.
 
 @menu
@@ -1319,8 +1320,8 @@ Selecting Lisp Dialect
 
 @noindent
 for the old dynamic-only dialect.  When no declaration is present the
-old dialect is used, but this may change in a future release.
-The compiler will warn if no declaration is present.
+new dialect is used.  Support for the old dialect may be removed in
+a future release.
 
 When evaluating Emacs Lisp code directly using an @code{eval} call,
 lexical binding is enabled if the @var{lexical} argument to
@@ -1980,8 +1981,8 @@ File Local Variables
 
 @defvar permanently-enabled-local-variables
 Some local variable settings will, by default, be heeded even if
-@code{enable-local-variables} is @code{nil}.  By default, this is only
-the case for the @code{lexical-binding} local variable setting, but
+@code{enable-local-variables} is @code{nil}.  By default, this is the
+case only for the @code{lexical-binding} local variable setting, but
 this can be controlled by using this variable, which is a list of
 symbols.
 @end defvar
diff --git a/doc/misc/cl.texi b/doc/misc/cl.texi
index 113029700ec..395725944f0 100644
--- a/doc/misc/cl.texi
+++ b/doc/misc/cl.texi
@@ -4758,12 +4758,13 @@ Porting Common Lisp
 @item
 Lexical scoping.  In Common Lisp, function arguments and @code{let}
 bindings apply only to references physically within their bodies (or
-within macro expansions in their bodies).  Traditionally, Emacs Lisp
-uses @dfn{dynamic scoping} wherein a binding to a variable is visible
+within macro expansions in their bodies).  The is also the case in
+modern Emacs Lisp, but historically, Emacs Lisp
+used @dfn{dynamic scoping} wherein a binding to a variable is visible
 even inside functions called from the body.
 @xref{Dynamic Binding,,,elisp,GNU Emacs Lisp Reference Manual}.
-Lexical binding is available since Emacs 24.1, so be sure to set
-@code{lexical-binding} to @code{t} if you need to emulate this aspect
+Lexical binding is available since Emacs 24.1, but be sure not to set
+@code{lexical-binding} to @code{nil} if you need to emulate this aspect
 of Common Lisp.  @xref{Lexical Binding,,,elisp,GNU Emacs Lisp Reference Manual}.
 
 Here is an example of a Common Lisp code fragment that would fail in
diff --git a/etc/NEWS b/etc/NEWS
index d1c7303f976..5a480c5f68a 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -636,6 +636,12 @@ the directory into which the repository was cloned.
 \f
 * Incompatible Lisp Changes in Emacs 31.1
 
++++
+** 'lexical-binding' now defaults to t.
+The default ELisp dialect is now the lexically scoped dialect.
+Any code which has not yet been adapted should either be
+adapted or set 'lexical-binding' to nil explicitly.
+
 ** Nested backquotes are not supported any more in Pcase patterns.
 
 ** The 'rx' category name 'chinese-two-byte' must now be spelled correctly.
diff --git a/lisp/loadup.el b/lisp/loadup.el
index bd74a9d6aff..e676c338c14 100644
--- a/lisp/loadup.el
+++ b/lisp/loadup.el
@@ -654,9 +654,9 @@ comp-subr-arities-h
           (unwind-protect
               (let ((tmp-dump-mode dump-mode)
                     (dump-mode nil)
-                    ;; Set `lexical-binding' to nil by default
+                    ;; Set `lexical-binding' to its default value
                     ;; in the dumped Emacs.
-                    (lexical-binding nil))
+                    (lexical-binding (default-toplevel-value 'lexical-binding)))
                 (if (member tmp-dump-mode '("pdump" "pbootstrap"))
                     (dump-emacs-portable (expand-file-name output invocation-directory))
                   (dump-emacs output (if (eq system-type 'ms-dos)
diff --git a/src/lread.c b/src/lread.c
index ea0398196e3..9f56880e551 100644
--- a/src/lread.c
+++ b/src/lread.c
@@ -1522,7 +1522,7 @@ DEFUN ("load", Fload, Sload, 1, 5, 0,
      otherwise using a file-variable in the first line.  This is bound here
      so that it takes effect whether or not we use
      Vload_source_file_function.  */
-  specbind (Qlexical_binding, Qnil);
+  specbind (Qlexical_binding, Fdefault_toplevel_value (Qlexical_binding));
 
   Lisp_Object found_eff =
     is_native_elisp
@@ -1720,8 +1720,11 @@ DEFUN ("load", Fload, Sload, 1, 5, 0,
     }
   else
     {
-      if (lisp_file_lexical_cookie (Qget_file_char) == Cookie_Lex)
+      lexical_cookie_t lexc = lisp_file_lexical_cookie (Qget_file_char);
+      if (lexc == Cookie_Lex)
         Fset (Qlexical_binding, Qt);
+      else if (lexc == Cookie_Dyn)
+        Fset (Qlexical_binding, Qnil);
 
       if (! version || version >= 22)
         readevalloop (Qget_file_char, &input, hist_file_name,
@@ -2606,8 +2609,11 @@ DEFUN ("eval-buffer", Feval_buffer, Seval_buffer, 0, 5, "",
   specbind (Qstandard_output, tem);
   record_unwind_protect_excursion ();
   BUF_TEMP_SET_PT (XBUFFER (buf), BUF_BEGV (XBUFFER (buf)));
+  lexical_cookie_t lexc = lisp_file_lexical_cookie (buf);
   specbind (Qlexical_binding,
-	    lisp_file_lexical_cookie (buf) == Cookie_Lex ? Qt : Qnil);
+	    lexc == Cookie_Lex ? Qt
+	    : lexc == Cookie_Dyn ? Qnil
+	    : Fdefault_toplevel_value (Qlexical_binding));
   BUF_TEMP_SET_PT (XBUFFER (buf), BUF_BEGV (XBUFFER (buf)));
   readevalloop (buf, 0, filename,
 		!NILP (printflag), unibyte, Qnil, Qnil, Qnil);
@@ -6056,7 +6062,7 @@ syms_of_lread (void)
 This variable is automatically set from the file variables of an
 interpreted Lisp file read using `load'.  Unlike other file local
 variables, this must be set in the first line of a file.  */);
-  Vlexical_binding = Qnil;
+  Vlexical_binding = Qt;
   Fmake_variable_buffer_local (Qlexical_binding);
 
   DEFVAR_LISP ("eval-buffer-list", Veval_buffer_list,
-- 
2.39.5