Default lexical-binding to t

Default lexical-binding to t

[Stefan Monnier]

In bug#74145 I suggest that it is time to change the default of
`lexical-binding` to t.  Clearly this is a breaking change, but only for
those files which:

- Don't have a `lexical-binding` cookie.
- Have code which happens to behave differently under the new dialect
  (such code is not rare, but a lot of code works identically in the
  two dialects).

I believe by the time Emacs-31 will be released, such files will be
uncommon, and it is easy to fix them (either by adjusting he code, or
by slapping a `lexical-binding` cookie).


        Stefan

Re: Default lexical-binding to t

[Gerd Möllmann]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> In bug#74145 I suggest that it is time to change the default of
> `lexical-binding` to t.  Clearly this is a breaking change, but only for
> those files which:
>
> - Don't have a `lexical-binding` cookie.
> - Have code which happens to behave differently under the new dialect
>   (such code is not rare, but a lot of code works identically in the
>   two dialects).
>
> I believe by the time Emacs-31 will be released, such files will be
> uncommon, and it is easy to fix them (either by adjusting he code, or
> by slapping a `lexical-binding` cookie).

+1

Re: Default lexical-binding to t

[Karl Fogel]

On 01 Nov 2024, Stefan Monnier wrote:
>In bug#74145 I suggest that it is time to change the default of
>`lexical-binding` to t.  Clearly this is a breaking change, but 
>only for
>those files which:
>
>- Don't have a `lexical-binding` cookie.
>- Have code which happens to behave differently under the new 
>dialect
>  (such code is not rare, but a lot of code works identically in 
>  the
>  two dialects).
>
>I believe by the time Emacs-31 will be released, such files will 
>be
>uncommon, and it is easy to fix them (either by adjusting he 
>code, or
>by slapping a `lexical-binding` cookie).

+1  It's time.

Anecdata, but: a lot of the random Emacs packages I encounter in 
the wild lately have `lexical-binding` cookies already (and 
usually to t anyway).  Most recently 
https://github.com/kmonad/kbd-mode/blob/master/kbd-mode.el, for 
example.  No doubt there are also many that don't, but I think 
knowledge about the new dialect is diffused widely enough now that 
when Emacs' default switches over, some user of a given package 
will quickly recognize what's going on and submit the appropriate 
change to that package; in a few cases I have done so myself.

Best regards,
-Karl

Re: Default lexical-binding to t

[Visuwesh]

[வெள்ளி நவம்பர் 01, 2024] Stefan Monnier wrote:

> In bug#74145 I suggest that it is time to change the default of
> `lexical-binding` to t.  Clearly this is a breaking change, but only for
> those files which:
>
> - Don't have a `lexical-binding` cookie.

Would it make sense to alert/warn the user when loading such a file?  It
would cause less confusion to the user when the code is suddenly not
functioning just right.  If there was a warning, she could add the
lexical-binding cookie and call it a day (or nag the package maintainer
about it).

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Visuwesh <visuweshm@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Sat, 02 Nov 2024 16:18:56 +0530
> 
> [வெள்ளி நவம்பர் 01, 2024] Stefan Monnier wrote:
> 
> > In bug#74145 I suggest that it is time to change the default of
> > `lexical-binding` to t.  Clearly this is a breaking change, but only for
> > those files which:
> >
> > - Don't have a `lexical-binding` cookie.
> 
> Would it make sense to alert/warn the user when loading such a file?

Loading or visiting?  For the latter, we already have a prominent
indication on the mode line.

Re: Default lexical-binding to t

[Visuwesh]

On 2 November 2024 17:38:18 GMT+05:30, Eli Zaretskii <eliz@gnu.org> wrote:
>> From: Visuwesh <visuweshm@gmail.com>
>> Cc: emacs-devel@gnu.org
>> Date: Sat, 02 Nov 2024 16:18:56 +0530
>> 
>> [வெள்ளி நவம்பர் 01, 2024] Stefan Monnier wrote:
>> 
>> > In bug#74145 I suggest that it is time to change the default of
>> > `lexical-binding` to t.  Clearly this is a breaking change, but only for
>> > those files which:
>> >
>> > - Don't have a `lexical-binding` cookie.
>> 
>> Would it make sense to alert/warn the user when loading such a file?
>
>Loading or visiting?  For the latter, we already have a prominent
>indication on the mode line.

Loading.

Re: Default lexical-binding to t

[Stefan Monnier]

> Would it make sense to alert/warn the user when loading such a file?
> It would cause less confusion to the user when the code is suddenly
> not functioning just right.  If there was a warning, she could add the
> lexical-binding cookie and call it a day (or nag the package
> maintainer about it).

We could easily make the code of `load` emit a message when loading
a source file without a `lexical-binding` cookie, yes.
I did not include such a message in my patch because last time someone
suggested such a thing (Mattias, IIRC) it didn't seem very popular.
But that's easy to add, and I'd be happy to add it,


        Stefan

Re: Default lexical-binding to t

[Jim Porter]

On 11/2/2024 9:24 AM, Stefan Monnier wrote:
>> Would it make sense to alert/warn the user when loading such a file?
>> It would cause less confusion to the user when the code is suddenly
>> not functioning just right.  If there was a warning, she could add the
>> lexical-binding cookie and call it a day (or nag the package
>> maintainer about it).
> 
> We could easily make the code of `load` emit a message when loading
> a source file without a `lexical-binding` cookie, yes.
> I did not include such a message in my patch because last time someone
> suggested such a thing (Mattias, IIRC) it didn't seem very popular.
> But that's easy to add, and I'd be happy to add it,

+1 to the main topic (defaulting 'lexical-binding' to t); from the last 
time we discussed this, I think we've eliminated all the blockers to 
doing this now. I also think lexical-binding is the de facto standard at 
this point; I'm not sure I've seen any dynamic-binding Elisp in a while. 
(I know it's still out there but I've only seen it in very old packages.)

As for warning about a missing 'lexical-binding' cookie, maybe we could 
check the "Package-Requires" metadata and warn if the "emacs" 
requirement is less than 31, but there's no 'lexical-binding' cookie? If 
you write a package that supports older Emacsen, then you still need the 
cookie. For packages with a minimum Emacs of 31, no-cookie is fine.

(For files that don't report the minimum Emacs, maybe we can make a 
guess that the minimum is "current-emacs - 2", or some other number? I'm 
not sure about the specifics here, but this would give us a warning that 
eventually goes away after a couple of Emacs versions.)

Re: Default lexical-binding to t

[Stefan Kangas]

Jim Porter <jporterbugs@gmail.com> writes:

> As for warning about a missing 'lexical-binding' cookie, maybe we could
> check the "Package-Requires" metadata and warn if the "emacs"
> requirement is less than 31, but there's no 'lexical-binding' cookie? If
> you write a package that supports older Emacsen, then you still need the
> cookie. For packages with a minimum Emacs of 31, no-cookie is fine.
>
> (For files that don't report the minimum Emacs, maybe we can make a
> guess that the minimum is "current-emacs - 2", or some other number? I'm
> not sure about the specifics here, but this would give us a warning that
> eventually goes away after a couple of Emacs versions.)

I think disabling the byte compiler and mode line warnings for packages
that explicitly declare support only for Emacs 31 or later makes sense.

While we're at it, we could also disable the warning for built-in and
non-:core libraries.  That would let us remove the cookie from most
files in emacs.git, if we wanted to.

Re: Default lexical-binding to t

[Stefan Monnier]

> As for warning about a missing 'lexical-binding' cookie, maybe we could
> check the "Package-Requires" metadata and warn if the "emacs" requirement
> is less than 31, but there's no 'lexical-binding' cookie?

AFAIK we're talking about warning when *loading* `.el` files.
This should almost never happen for files installed via `package.el`
(which always compiles the files).

Also, I'm not fond of such "fanciness".  I think it would be complicated
to implement and brittle.

> (For files that don't report the minimum Emacs, maybe we can make a guess
> that the minimum is "current-emacs - 2", or some other number? I'm not sure
> about the specifics here, but this would give us a warning that eventually
> goes away after a couple of Emacs versions.)

IOW, you're suggesting that we get rid of the warning in Emacs-33?
I'm not opposed, but I'd rather leave that discussion to when we get to
Emacs-33.  🙂


        Stefan

Re: Default lexical-binding to t

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As for warning about a missing 'lexical-binding' cookie, maybe we could
>> check the "Package-Requires" metadata and warn if the "emacs" requirement
>> is less than 31, but there's no 'lexical-binding' cookie?
>
> AFAIK we're talking about warning when *loading* `.el` files.
> This should almost never happen for files installed via `package.el`
> (which always compiles the files).
>
> Also, I'm not fond of such "fanciness".  I think it would be complicated
> to implement and brittle.

I tend to agree, FWIW.

Re: Default lexical-binding to t

[Stefan Kangas]

Stefan Kangas <stefankangas@gmail.com> writes:

> I think disabling the byte compiler and mode line warnings for packages
> that explicitly declare support only for Emacs 31 or later makes sense.
>
> While we're at it, we could also disable the warning for built-in and
> non-:core libraries.  That would let us remove the cookie from most
> files in emacs.git, if we wanted to.

To be clear, here I'm talking only about disabling the _already
existing_ warnings that we have when byte-compiling, and in the mode
line.

Re: Default lexical-binding to t

[Sean Whitton]

Hello,

On Sat 02 Nov 2024 at 01:42pm -07, Jim Porter wrote:

> On 11/2/2024 9:24 AM, Stefan Monnier wrote:
>>> Would it make sense to alert/warn the user when loading such a file?
>>> It would cause less confusion to the user when the code is suddenly
>>> not functioning just right.  If there was a warning, she could add the
>>> lexical-binding cookie and call it a day (or nag the package
>>> maintainer about it).
>> We could easily make the code of `load` emit a message when loading
>> a source file without a `lexical-binding` cookie, yes.
>> I did not include such a message in my patch because last time someone
>> suggested such a thing (Mattias, IIRC) it didn't seem very popular.
>> But that's easy to add, and I'd be happy to add it,
>
> I also think lexical-binding is the de facto standard at this point; I'm
> not sure I've seen any dynamic-binding Elisp in a while. (I know it's still
> out there but I've only seen it in very old packages.)

Paredit is a notable example which doesn't have the cookie.

I don't know whether it actually uses any dynamic bindings but I think
there's some chance it does, based on the style of some parts I've read.

-- 
Sean Whitton

Re: Default lexical-binding to t

[Jim Porter]

On 11/2/2024 6:46 PM, Stefan Monnier wrote:
>> As for warning about a missing 'lexical-binding' cookie, maybe we could
>> check the "Package-Requires" metadata and warn if the "emacs" requirement
>> is less than 31, but there's no 'lexical-binding' cookie?
> 
> AFAIK we're talking about warning when *loading* `.el` files.
> This should almost never happen for files installed via `package.el`
> (which always compiles the files).

Hmm, maybe (possibly in addition to other warnings), we could add a 
Flymake check to ensure that 'lexical-binding' is present when the 
"Package-Requires" metadata says the package supports Emacs < 31? That 
way, package authors would hopefully see a warning when editing their 
package.

Re: Default lexical-binding to t

[Stefan Monnier]

> Hmm, maybe (possibly in addition to other warnings), we could add a Flymake
> check to ensure that 'lexical-binding' is present when the
> "Package-Requires" metadata says the package supports Emacs < 31? That way,
> package authors would hopefully see a warning when editing their package.

Emacs-30 already warns during compilation (and hence in flymake) when
the cookie is missing.


        Stefan

Re: Default lexical-binding to t

[Stefan Monnier]

> To be clear, here I'm talking only about disabling the _already
> existing_ warnings that we have when byte-compiling, and in the
> mode line.

I'd first like us to get experience with "lexical-binding defaults to t"
before considering removing the cookie for files.
I would hate having to re-add the cookie if we find out that there are
many more problems than expected and we revert my patch.


        Stefan

Re: Default lexical-binding to t

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> To be clear, here I'm talking only about disabling the _already
>> existing_ warnings that we have when byte-compiling, and in the
>> mode line.
>
> I'd first like us to get experience with "lexical-binding defaults to t"
> before considering removing the cookie for files.
> I would hate having to re-add the cookie if we find out that there are
> many more problems than expected and we revert my patch.

Fully agreed.

lexical-binding in paredit

[Stefan Monnier]

[-- Attachment #1: Type: text/plain, Size: 691 bytes --]

>> I also think lexical-binding is the de facto standard at this point; I'm
>> not sure I've seen any dynamic-binding Elisp in a while. (I know it's still
>> out there but I've only seen it in very old packages.)
> Paredit is a notable example which doesn't have the cookie.
> I don't know whether it actually uses any dynamic bindings but I think
> there's some chance it does, based on the style of some parts I've read.

After playing the janitor for a while I got the patch below and while
the code does have some of the typical traits of "old ELisp" code, it
doesn't seem to rely on dynamic scoping anywhere (tho I haven't actually
tested it, so this is no guarantee).


        Stefan

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: paredit.patch --]
[-- Type: text/x-diff, Size: 40410 bytes --]

2024-11-03  Stefan Monnier  <monnier@iro.umontreal.ca>

    * paredit.el: Use `lexical-binding`.  Prefer #' to quote function names.
    Move `defmacro`s outside of `eval-and-compile`.
    Add `declare`s to `defmacro`s.  Avoid obsolete `point-at-eol/bol`.
    Drop support for Emacs<22.
    (paredit-xemacs-p, paredit-gnu-emacs-p): Inline all calls (so the
    byte-compiler can throw away the irrelevant code) and then delete.
    (xcond): Delete macro, since after expanding `paredit-*emacs-p` it's
    obvious that all uses are already exhaustive.
    (paredit-sexp-error-type): Add FIXME.
    (paredit-initialize-comment-dwim): Delete function needed only for Emacs-21.
    (paredit-comment-dwim): Call `comment-normalize-vars` instead.
    (paredit-wrap-sexp): Silence warning about ((lambda (..) ..) ..).
    (paredit-region-active-p): Use `region-active-p` also in Emacs.


diff --git a/paredit.el b/paredit.el
index 87ed313716..bfd877e899 100644
--- a/paredit.el
+++ b/paredit.el
@@ -1,4 +1,4 @@
-;;; paredit.el --- minor mode for editing parentheses  -*- Mode: Emacs-Lisp -*-
+;;; paredit.el --- minor mode for editing parentheses  -*- Mode: Emacs-Lisp; lexical-binding: t; -*-
 
 ;; Copyright (C) 2005--2023 Taylor R. Campbell
 
@@ -60,7 +60,7 @@
 ;;; Send questions, bug reports, comments, feature suggestions, &c.,
 ;;; via email to the author's surname at paredit.org.
 ;;;
-;;; Paredit should run in GNU Emacs 21 or later and XEmacs 21.5.28 or
+;;; Paredit should run in GNU Emacs 22 or later and XEmacs 21.5.28 or
 ;;; later.
 \f
 ;;; The paredit minor mode, Paredit Mode, binds common character keys,
@@ -138,26 +138,10 @@
 \f
 (eval-and-compile
 
-  (defun paredit-xemacs-p ()
-    ;; No idea where I got this definition from.  Edward O'Connor
-    ;; (hober in #emacs) suggested the current definition.
-    ;;   (and (boundp 'running-xemacs)
-    ;;        running-xemacs)
-    (featurep 'xemacs))
-
-  (defun paredit-gnu-emacs-p ()
-    ;++ This could probably be improved.
-    (not (paredit-xemacs-p)))
-
-  (defmacro xcond (&rest clauses)
-    "Exhaustive COND.
-Signal an error if no clause matches."
-    `(cond ,@clauses
-           (t (error "XCOND lost."))))
-
-  (defalias 'paredit-warn (if (fboundp 'warn) 'warn 'message))
+  (defalias 'paredit-warn (if (fboundp 'warn) #'warn #'message))
 
   (defvar paredit-sexp-error-type
+    ;; FIXME: This has been `scan-error' in Emacs since 1997.
     (with-temp-buffer
       (insert "(")
       (condition-case condition
@@ -169,37 +153,32 @@ Signal an error if no clause matches."
                                  " other errors. "
                                  " This may cause obscure problems. "
                                  " Please upgrade Emacs."))
-               (car condition)))))
+               (car condition))))))
 
-  (defmacro paredit-handle-sexp-errors (body &rest handler)
-    `(condition-case ()
-         ,body
-       (,paredit-sexp-error-type ,@handler)))
+(defmacro paredit-handle-sexp-errors (body &rest handler)
+  (declare (indent 1) (debug t))
+  `(condition-case ()
+       ,body
+     (,paredit-sexp-error-type ,@handler)))
 
-  (put 'paredit-handle-sexp-errors 'lisp-indent-function 1)
+(defmacro paredit-ignore-sexp-errors (&rest body)
+  (declare (indent 0) (debug t))
+  `(paredit-handle-sexp-errors (progn ,@body)
+                               nil))
 
-  (defmacro paredit-ignore-sexp-errors (&rest body)
-    `(paredit-handle-sexp-errors (progn ,@body)
-       nil))
-
-  (put 'paredit-ignore-sexp-errors 'lisp-indent-function 0)
-
-  (defmacro paredit-preserving-column (&rest body)
-    "Evaluate BODY and restore point to former column, relative to code.
+(defmacro paredit-preserving-column (&rest body)
+  "Evaluate BODY and restore point to former column, relative to code.
 Assumes BODY will change only indentation.
 If point was on code, it moves with the code.
 If point was on indentation, it stays in indentation."
-    (let ((column (make-symbol "column"))
-          (indentation (make-symbol "indentation")))
-      `(let ((,column (paredit-current-column))
-             (,indentation (paredit-current-indentation)))
-         (let ((value (progn ,@body)))
-           (paredit-restore-column ,column ,indentation)
-           value))))
-
-  (put 'paredit-preserving-column 'lisp-indent-function 0)
-
-  nil)
+  (declare (indent 0) (debug t))
+  (let ((column (make-symbol "column"))
+        (indentation (make-symbol "indentation")))
+    `(let ((,column (paredit-current-column))
+           (,indentation (paredit-current-indentation)))
+       (let ((value (progn ,@body)))
+         (paredit-restore-column ,column ,indentation)
+         value))))
 \f
 ;;;; Minor Mode Definition
 
@@ -210,7 +189,7 @@ If point was on indentation, it stays in indentation."
   "Keymap for the paredit minor mode.")
 
 (defvar paredit-override-check-parens-function
-  (lambda (condition) (declare ignore condition) nil)
+  (lambda (_condition) nil) ;;A.k.a `#'ignore'
   "Function to tell whether unbalanced text should inhibit Paredit Mode.")
 
 ;;;###autoload
@@ -248,12 +227,12 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
   (paredit-mode -1))
 
 (defvar paredit-backward-delete-key
-  (xcond ((paredit-xemacs-p)    "BS")
-         ((paredit-gnu-emacs-p) "DEL")))
+  (cond ((featurep 'xemacs)    "BS")
+        (t                     "DEL")))
 
 (defvar paredit-forward-delete-keys
-  (xcond ((paredit-xemacs-p)    '("DEL"))
-         ((paredit-gnu-emacs-p) '("<delete>" "<deletechar>"))))
+  (cond ((featurep 'xemacs)    '("DEL"))
+        (t                     '("<delete>" "<deletechar>"))))
 \f
 ;;;; Paredit Keys
 
@@ -477,28 +456,26 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
 \f
 ;;;;; Command Examples
 
-(eval-and-compile
-  (defmacro paredit-do-commands (vars string-case &rest body)
-    (let ((spec     (nth 0 vars))
-          (keys     (nth 1 vars))
-          (fn       (nth 2 vars))
-          (examples (nth 3 vars)))
-      `(dolist (,spec paredit-commands)
-         (if (stringp ,spec)
-             ,string-case
-           (let ((,keys (let ((k (car ,spec)))
-                          (cond ((stringp k) (list k))
-                                ((listp k) k)
-                                (t (error "Invalid paredit command %s."
-                                          ,spec)))))
-                 (,fn (cadr ,spec))
-                 (,examples (cddr ,spec)))
-             ,@body)))))
-
-  (put 'paredit-do-commands 'lisp-indent-function 2))
+(defmacro paredit-do-commands (vars string-case &rest body)
+  (declare (indent 2) (debug (sexp form body)))
+  (let ((spec     (nth 0 vars))
+        (keys     (nth 1 vars))
+        (fn       (nth 2 vars))
+        (examples (nth 3 vars)))
+    `(dolist (,spec paredit-commands)
+       (if (stringp ,spec)
+           ,string-case
+         (let ((,keys (let ((k (car ,spec)))
+                        (cond ((stringp k) (list k))
+                              ((listp k) k)
+                              (t (error "Invalid paredit command %s."
+                                        ,spec)))))
+               (,fn (cadr ,spec))
+               (,examples (cddr ,spec)))
+           ,@body)))))
 
 (defun paredit-define-keys ()
-  (paredit-do-commands (spec keys fn examples)
+  (paredit-do-commands (spec keys fn _examples)
       nil       ; string case
     (dolist (key keys)
       (define-key paredit-mode-map (read-kbd-macro key) fn))))
@@ -513,7 +490,7 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
 (defun paredit-annotate-mode-with-examples ()
   (let ((contents
          (list (paredit-function-documentation 'paredit-mode))))
-    (paredit-do-commands (spec keys fn examples)
+    (paredit-do-commands (spec _keys fn examples)
         (push (concat "\n\f\n" spec "\n")
               contents)
       (let ((name (symbol-name fn)))
@@ -523,7 +500,7 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
                               (mapconcat (lambda (example)
                                            (concat
                                             "\n"
-                                            (mapconcat 'identity
+                                            (mapconcat #'identity
                                                        example
                                                        "\n  --->\n")
                                             "\n"))
@@ -532,20 +509,20 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
                               "\n  (no examples)\n"))
                   contents))))
     (put 'paredit-mode 'function-documentation
-         (apply 'concat (reverse contents))))
+         (apply #'concat (reverse contents))))
   ;; PUT returns the huge string we just constructed, which we don't
   ;; want it to return.
   nil)
 
 (defun paredit-annotate-functions-with-examples ()
-  (paredit-do-commands (spec keys fn examples)
+  (paredit-do-commands (spec _keys fn examples)
       nil       ; string case
     (put fn 'function-documentation
          (concat (paredit-function-documentation fn)
                  "\n\n\\<paredit-mode-map>\\[" (symbol-name fn) "]\n"
                  (mapconcat (lambda (example)
                               (concat "\n"
-                                      (mapconcat 'identity
+                                      (mapconcat #'identity
                                                  example
                                                  "\n  ->\n")
                                       "\n"))
@@ -581,7 +558,7 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
         (dolist (example examples)
           (let ((prefix "<td><table border=\"1\"><tr><td><table><tr><td><pre>")
                 (examples
-                 (mapconcat 'paredit-html-quote
+                 (mapconcat #'paredit-html-quote
                             example
                             (concat "</pre></td></tr>"
                                     "<tr><th>&darr;</th></tr>"
@@ -607,12 +584,12 @@ Paredit behaves badly if parentheses are unbalanced, so exercise
 
 (eval-and-compile
   (defun paredit-conc-name (&rest strings)
-    (intern (apply 'concat strings)))
+    (intern (apply #'concat strings))))
 
-  (defmacro define-paredit-pair (open close name)
-    `(progn
-       (defun ,(paredit-conc-name "paredit-open-" name) (&optional n)
-         ,(concat "Insert a balanced " name " pair.
+(defmacro define-paredit-pair (open close name)
+  `(progn
+     (defun ,(paredit-conc-name "paredit-open-" name) (&optional n)
+       ,(concat "Insert a balanced " name " pair.
 With a prefix argument N, put the closing " name " after N
   S-expressions forward.
 If the region is active, `transient-mark-mode' is enabled, and the
@@ -621,36 +598,36 @@ If the region is active, `transient-mark-mode' is enabled, and the
 If in a string or a comment, insert a single " name ".
 If in a character literal, do nothing.  This prevents changing what was
   in the character literal to a meaningful delimiter unintentionally.")
-         (interactive "P")
-         (cond ((or (paredit-in-string-p)
-                    (paredit-in-comment-p))
-                (insert ,open))
-               ((not (paredit-in-char-p))
-                (paredit-insert-pair n ,open ,close 'goto-char)
-                (save-excursion (backward-up-list) (indent-sexp)))))
-       (defun ,(paredit-conc-name "paredit-close-" name) ()
-         ,(concat "Move past one closing delimiter and reindent.
+       (interactive "P")
+       (cond ((or (paredit-in-string-p)
+                  (paredit-in-comment-p))
+              (insert ,open))
+             ((not (paredit-in-char-p))
+              (paredit-insert-pair n ,open ,close 'goto-char)
+              (save-excursion (backward-up-list) (indent-sexp)))))
+     (defun ,(paredit-conc-name "paredit-close-" name) ()
+       ,(concat "Move past one closing delimiter and reindent.
 \(Agnostic to the specific closing delimiter.)
 If in a string or comment, insert a single closing " name ".
 If in a character literal, do nothing.  This prevents changing what was
   in the character literal to a meaningful delimiter unintentionally.")
-         (interactive)
-         (paredit-move-past-close ,close))
-       (defun ,(paredit-conc-name "paredit-close-" name "-and-newline") ()
-         ,(concat "Move past one closing delimiter, add a newline,"
-                  " and reindent.
+       (interactive)
+       (paredit-move-past-close ,close))
+     (defun ,(paredit-conc-name "paredit-close-" name "-and-newline") ()
+       ,(concat "Move past one closing delimiter, add a newline,"
+                " and reindent.
 If there was a margin comment after the closing delimiter, preserve it
   on the same line.")
-         (interactive)
-         (paredit-move-past-close-and-newline ,close))
-       (defun ,(paredit-conc-name "paredit-wrap-" name)
-           (&optional argument)
-         ,(concat "Wrap the following S-expression.
+       (interactive)
+       (paredit-move-past-close-and-newline ,close))
+     (defun ,(paredit-conc-name "paredit-wrap-" name)
+         (&optional argument)
+       ,(concat "Wrap the following S-expression.
 See `paredit-wrap-sexp' for more details.")
-         (interactive "P")
-         (paredit-wrap-sexp argument ,open ,close))
-       (add-to-list 'paredit-wrap-commands
-                    ',(paredit-conc-name "paredit-wrap-" name)))))
+       (interactive "P")
+       (paredit-wrap-sexp argument ,open ,close))
+     (add-to-list 'paredit-wrap-commands
+                  ',(paredit-conc-name "paredit-wrap-" name))))
 
 (defvar paredit-wrap-commands '(paredit-wrap-sexp)
   "List of paredit commands that wrap S-expressions.
@@ -663,15 +640,15 @@ Used by `paredit-yank-pop'; for internal paredit use only.")
 
 ;;; Aliases for the old names.
 
-(defalias 'paredit-open-parenthesis 'paredit-open-round)
-(defalias 'paredit-close-parenthesis 'paredit-close-round)
+(defalias 'paredit-open-parenthesis #'paredit-open-round)
+(defalias 'paredit-close-parenthesis #'paredit-close-round)
 (defalias 'paredit-close-parenthesis-and-newline
-  'paredit-close-round-and-newline)
+  #'paredit-close-round-and-newline)
 
-(defalias 'paredit-open-bracket 'paredit-open-square)
-(defalias 'paredit-close-bracket 'paredit-close-square)
+(defalias 'paredit-open-bracket #'paredit-open-square)
+(defalias 'paredit-close-bracket #'paredit-close-square)
 (defalias 'paredit-close-bracket-and-newline
-  'paredit-close-square-and-newline)
+  #'paredit-close-square-and-newline)
 \f
 (defun paredit-move-past-close (close)
   (paredit-move-past-close-and close
@@ -710,7 +687,7 @@ If such a comment exists, delete the comment (including all leading
   and whose cdr is the point of the comment's initial semicolon,
   relative to the start of the line."
   (save-excursion
-    (paredit-skip-whitespace t (point-at-eol))
+    (paredit-skip-whitespace t (line-end-position))
     (and (eq ?\; (char-after))
          (not (eq ?\; (char-after (1+ (point)))))
          (not (or (paredit-in-string-p)
@@ -718,10 +695,10 @@ If such a comment exists, delete the comment (including all leading
          (let* ((start                  ;Move to before the semicolon.
                  (progn (backward-char) (point)))
                 (comment
-                 (buffer-substring start (point-at-eol))))
-           (paredit-skip-whitespace nil (point-at-bol))
-           (delete-region (point) (point-at-eol))
-           (cons comment (- start (point-at-bol)))))))
+                 (buffer-substring start (line-end-position))))
+           (paredit-skip-whitespace nil (line-beginning-position))
+           (delete-region (point) (line-end-position))
+           (cons comment (- start (line-beginning-position)))))))
 \f
 (defun paredit-insert-pair (n open close forward)
   (let* ((regionp
@@ -1006,7 +983,7 @@ If in a comment and if followed by invalid structure, call
   (cond ((paredit-in-string-p)
          (newline))
         ((paredit-in-comment-p)
-         (if (paredit-region-ok-p (point) (point-at-eol))
+         (if (paredit-region-ok-p (point) (line-end-position))
              (progn (newline-and-indent)
                     (paredit-ignore-sexp-errors (indent-sexp)))
              (indent-new-comment-line)))
@@ -1096,7 +1073,7 @@ If a list begins on the line after the point but ends on a different
 
 (defun paredit-semicolon-find-line-break-point ()
   (and (not (eolp))                   ;Implies (not (eobp)).
-       (let ((eol (point-at-eol)))
+       (let ((eol (line-end-position)))
          (save-excursion
            (catch 'exit
              (while t
@@ -1106,16 +1083,16 @@ If a list begins on the line after the point but ends on a different
                         ;; Successfully advanced by an S-expression.
                         ;; If that S-expression started on this line
                         ;; and ended on another one, break here.
-                        (cond ((not (eq eol (point-at-eol)))
+                        (cond ((not (eq eol (line-end-position)))
                                (throw 'exit
                                       (and (save-excursion
                                              (backward-sexp)
-                                             (eq eol (point-at-eol)))
+                                             (eq eol (line-end-position)))
                                            line-break-point)))
                               ((eobp)
                                (throw 'exit nil))))
                        ((save-excursion
-                          (paredit-skip-whitespace t (point-at-eol))
+                          (paredit-skip-whitespace t (line-end-position))
                           (or (eolp) (eobp) (eq (char-after) ?\;)))
                         ;; Can't move further, but there's no closing
                         ;; delimiter we're about to clobber -- either
@@ -1143,29 +1120,6 @@ If a list begins on the line after the point but ends on a different
       (paredit-ignore-sexp-errors (indent-sexp))
       (paredit-indent-sexps))))
 \f
-;;; This is all a horrible, horrible hack, primarily for GNU Emacs 21,
-;;; in which there is no `comment-or-uncomment-region'.
-
-(autoload 'comment-forward "newcomment")
-(autoload 'comment-normalize-vars "newcomment")
-(autoload 'comment-region "newcomment")
-(autoload 'comment-search-forward "newcomment")
-(autoload 'uncomment-region "newcomment")
-
-(defun paredit-initialize-comment-dwim ()
-  (require 'newcomment)
-  (if (not (fboundp 'comment-or-uncomment-region))
-      (defalias 'comment-or-uncomment-region
-        (lambda (beginning end &optional argument)
-          (interactive "*r\nP")
-          (if (save-excursion (goto-char beginning)
-                              (comment-forward (point-max))
-                              (<= end (point)))
-              (uncomment-region beginning end argument)
-              (comment-region beginning end argument)))))
-  (defalias 'paredit-initialize-comment-dwim 'comment-normalize-vars)
-  (comment-normalize-vars))
-
 (defvar paredit-comment-prefix-toplevel ";;; "
   "String of prefix for top-level comments aligned at the left margin.")
 
@@ -1193,7 +1147,7 @@ At the top level, where indentation is calculated to be at column 0,
   and if the point is after all code on the line, insert a single-
   semicolon margin comment at `comment-column'."
   (interactive "*P")
-  (paredit-initialize-comment-dwim)
+  (comment-normalize-vars)
   (cond ((paredit-region-active-p)
          (comment-or-uncomment-region (region-beginning)
                                       (region-end)
@@ -1215,7 +1169,7 @@ This is expected to be called only in `paredit-comment-dwim'; do not
       ;; COMMENT-P to true; if not, it will be nil.
       (while (progn
                (setq comment-p          ;t -> no error
-                     (comment-search-forward (point-at-eol) t))
+                     (comment-search-forward (line-end-position) t))
                (and comment-p
                     (or (paredit-in-string-p)
                         (paredit-in-char-p (1- (point))))))
@@ -1224,10 +1178,10 @@ This is expected to be called only in `paredit-comment-dwim'; do not
 
 (defun paredit-insert-comment ()
   (let ((code-after-p
-         (save-excursion (paredit-skip-whitespace t (point-at-eol))
+         (save-excursion (paredit-skip-whitespace t (line-end-position))
                          (not (eolp))))
         (code-before-p
-         (save-excursion (paredit-skip-whitespace nil (point-at-bol))
+         (save-excursion (paredit-skip-whitespace nil (line-beginning-position))
                          (not (bolp)))))
     (cond ((and (bolp)
                 (let ((indent
@@ -1378,8 +1332,8 @@ If `delete-active-region' is enabled and the mark is active and
   ;; the line break.  Refuse to do so if if moving the next line into
   ;; the comment would break structure.
   (if (eolp)
-      (let ((next-line-start (point-at-bol 2))
-            (next-line-end (point-at-eol 2)))
+      (let ((next-line-start (line-beginning-position 2))
+            (next-line-end (line-end-position 2)))
         (paredit-check-region next-line-start next-line-end))))
 
 (defun paredit-forward-delete-in-comment ()
@@ -1389,7 +1343,7 @@ If `delete-active-region' is enabled and the mark is active and
 (defun paredit-forward-delete-comment-start ()
   ;; Point precedes a comment start (not at eol).  Refuse to delete a
   ;; comment start if the comment contains unbalanced junk.
-  (paredit-check-region (+ (point) 1) (point-at-eol))
+  (paredit-check-region (+ (point) 1) (line-end-position))
   (delete-char +1))
 \f
 (defun paredit-backward-delete (&optional argument)
@@ -1496,7 +1450,7 @@ If `delete-active-region' is enabled and the mark is active and
         ;; Must call `paredit-in-string-p' before
         ;; `paredit-in-comment-p'.
         (not (or (paredit-in-string-p) (paredit-in-comment-p))))
-      (paredit-check-region (point) (point-at-eol)))
+      (paredit-check-region (point) (line-end-position)))
   (backward-delete-char-untabify +1))
 
 (defun paredit-backward-delete-maybe-comment-end ()
@@ -1507,7 +1461,7 @@ If `delete-active-region' is enabled and the mark is active and
   (if (save-excursion
         (backward-char)
         (and (not (paredit-in-string-p)) (paredit-in-comment-p)))
-      (paredit-check-region (point-at-eol) (point-at-bol)))
+      (paredit-check-region (line-end-position) (line-beginning-position)))
   (delete-char -1))
 \f
 ;;;; Killing
@@ -1533,7 +1487,7 @@ In that case, ensure there is at least one space between the
          (paredit-kill-line-in-string))
         ((paredit-in-comment-p)
          (paredit-kill-line-in-comment))
-        ((save-excursion (paredit-skip-whitespace t (point-at-eol))
+        ((save-excursion (paredit-skip-whitespace t (line-end-position))
                          (or (eolp) (eq (char-after) ?\; )))
          ;** Be careful about trailing backslashes.
          (if (paredit-in-char-p)
@@ -1542,7 +1496,7 @@ In that case, ensure there is at least one space between the
         (t (paredit-kill-sexps-on-line))))
 
 (defun paredit-kill-line-in-string ()
-  (if (save-excursion (paredit-skip-whitespace t (point-at-eol))
+  (if (save-excursion (paredit-skip-whitespace t (line-end-position))
                       (eolp))
       (kill-line)
     (save-excursion
@@ -1550,7 +1504,7 @@ In that case, ensure there is at least one space between the
       (if (paredit-in-string-escape-p)
           (backward-char))
       (kill-region (point)
-                   (min (point-at-eol)
+                   (min (line-end-position)
                         (cdr (paredit-string-start+end-points)))))))
 
 (defun paredit-kill-line-in-comment ()
@@ -1563,7 +1517,7 @@ In that case, ensure there is at least one space between the
   (if (paredit-in-char-p)               ; Move past the \ and prefix.
       (backward-char 2))                ; (# in Scheme/CL, ? in elisp)
   (let ((beginning (point))
-        (eol (point-at-eol)))
+        (eol (line-end-position)))
     (let ((end-of-list-p (paredit-forward-sexps-to-kill beginning eol)))
       ;; If we got to the end of the list and it's on the same line,
       ;; move backward past the closing delimiter before killing.  (This
@@ -1579,7 +1533,7 @@ In that case, ensure there is at least one space between the
                      ;; the last S-expression we found.  Be sure,
                      ;; though, not to kill any closing parentheses.
                      (if (and (not end-of-list-p)
-                              (eq (point-at-eol) eol))
+                              (eq (line-end-position) eol))
                          eol
                          (point)))))))
 \f
@@ -1588,10 +1542,10 @@ In that case, ensure there is at least one space between the
 ;;; and the closing delimiter both lie on this line.  Return true if
 ;;; the closing delimiter of this list is on this line, false if not.
 ;;;
-;;; beginning is (point), and eol is (point-at-eol).  Handling of
+;;; beginning is (point), and eol is (line-end-position).  Handling of
 ;;; `kill-whole-line' is trick, and probably kind of broken.
 
-(defun paredit-forward-sexps-to-kill (beginning eol)
+(defun paredit-forward-sexps-to-kill (_beginning eol)
   (let ((end-of-list-p nil) ;Have we hit a closing delimiter on this line?
         (firstp t))         ;Is this still the first line?
     (catch 'return
@@ -1610,7 +1564,7 @@ In that case, ensure there is at least one space between the
             ;; list.  Stop here, but record whether the closing
             ;; delimiter occurred on the starting line.
             (up-list)
-            (setq end-of-list-p (eq (point-at-eol) eol))
+            (setq end-of-list-p (eq (line-end-position) eol))
             (throw 'return nil))
           ;; We can move forward.  Where did we move to?  Stop if:
           ;;
@@ -1629,7 +1583,7 @@ In that case, ensure there is at least one space between the
                   (paredit-handle-sexp-errors   ;(b)
                       (progn (backward-sexp) nil)
                     t)
-                  (not (eq (point-at-eol) eol)))
+                  (not (eq (line-end-position) eol)))
               (throw 'return nil)))
         ;; Determined we can and should move forward.  Do so.
         (forward-sexp)
@@ -1662,8 +1616,8 @@ In that case, ensure there is at least one space between the
                           (point)))
                    ;; ...or just use the point past the newline, if
                    ;; we encounter a comment.
-                   (point-at-eol)))
-  (cond ((save-excursion (paredit-skip-whitespace nil (point-at-bol))
+                   (line-end-position)))
+  (cond ((save-excursion (paredit-skip-whitespace nil (line-beginning-position))
                          (bolp))
          ;; Nothing but indentation before the point, so indent it.
          (lisp-indent-line))
@@ -1687,7 +1641,7 @@ In that case, ensure there is at least one space between the
   (let ((argument (or argument 1)))
     (if (< argument 0)
         (paredit-backward-kill-word (- argument))
-      (dotimes (i argument)
+      (dotimes (_ argument)
         (let ((beginning (point)))
           (skip-syntax-forward " -")
           (let* ((parse-state (paredit-current-parse-state))
@@ -1721,7 +1675,7 @@ In that case, ensure there is at least one space between the
   (let ((argument (or argument 1)))
     (if (< argument 0)
         (paredit-forward-kill-word (- argument))
-      (dotimes (i argument)
+      (dotimes (_ argument)
         (if (not (or (bobp)
                      (eq (char-syntax (char-before)) ?w)))
             (let ((end (point)))
@@ -1789,14 +1743,14 @@ In that case, ensure there is at least one space between the
   (cond ((paredit-in-string-p)
          (paredit-copy-as-kill-in-string))
         ((paredit-in-comment-p)
-         (copy-region-as-kill (point) (point-at-eol)))
-        ((save-excursion (paredit-skip-whitespace t (point-at-eol))
+         (copy-region-as-kill (point) (line-end-position)))
+        ((save-excursion (paredit-skip-whitespace t (line-end-position))
                          (or (eolp) (eq (char-after) ?\; )))
          ;** Be careful about trailing backslashes.
          (save-excursion
            (if (paredit-in-char-p)
                (backward-char))
-           (copy-region-as-kill (point) (point-at-eol))))
+           (copy-region-as-kill (point) (line-end-position))))
         (t (paredit-copy-sexps-as-kill))))
 
 (defun paredit-copy-as-kill-in-string ()
@@ -1804,7 +1758,7 @@ In that case, ensure there is at least one space between the
     (if (paredit-in-string-escape-p)
         (backward-char))
     (copy-region-as-kill (point)
-                         (min (point-at-eol)
+                         (min (line-end-position)
                               (cdr (paredit-string-start+end-points))))))
 
 (defun paredit-copy-sexps-as-kill ()
@@ -1812,7 +1766,7 @@ In that case, ensure there is at least one space between the
     (if (paredit-in-char-p)
         (backward-char 2))
     (let ((beginning (point))
-          (eol (point-at-eol)))
+          (eol (line-end-position)))
       (let ((end-of-list-p (paredit-forward-sexps-to-kill beginning eol)))
         (if end-of-list-p (progn (up-list) (backward-char)))
         (copy-region-as-kill beginning
@@ -1821,9 +1775,9 @@ In that case, ensure there is at least one space between the
                                           (paredit-skip-whitespace t)
                                           (and (not (eq (char-after) ?\; ))
                                                (point)))
-                                        (point-at-eol)))
+                                        (line-end-position)))
                                    ((and (not end-of-list-p)
-                                         (eq (point-at-eol) eol))
+                                         (eq (line-end-position) eol))
                                     eol)
                                    (t
                                     (point))))))))
@@ -1862,7 +1816,7 @@ With a prefix argument, skip the balance check."
       (paredit-check-region-for-delete:char-quote start start-state
                                                   end end-state))))
 
-(defun paredit-check-region-for-delete:depth (start start-state end end-state)
+(defun paredit-check-region-for-delete:depth (_start start-state _end end-state)
   (let ((start-depth (nth 0 start-state))
         (end-depth (nth 0 end-state)))
     (if (not (= start-depth end-depth))
@@ -1870,7 +1824,7 @@ With a prefix argument, skip the balance check."
                start-depth
                end-depth))))
 
-(defun paredit-check-region-for-delete:string (start start-state end end-state)
+(defun paredit-check-region-for-delete:string (_start start-state _end end-state)
   (let ((start-string-p (nth 3 start-state))
         (end-string-p (nth 3 end-state)))
     (if (not (eq start-string-p end-string-p))
@@ -1879,7 +1833,7 @@ With a prefix argument, skip the balance check."
                (if end-string-p "" "not ")))))
 \f
 (defun paredit-check-region-for-delete:comment
-    (start start-state end end-state)
+    (_start start-state end end-state)
   (let ((start-comment-state (nth 4 start-state))
         (end-comment-state (nth 4 end-state)))
     (if (not (or (eq start-comment-state end-comment-state)
@@ -1894,7 +1848,7 @@ With a prefix argument, skip the balance check."
                                (eq end-comment-state nil)))
                       (save-excursion
                         (goto-char end)
-                        (paredit-region-ok-p (point) (point-at-eol))))))
+                        (paredit-region-ok-p (point) (line-end-position))))))
         (error "Mismatched comment state: %s"
                (cond ((and (integerp start-comment-state)
                            (integerp end-comment-state))
@@ -1915,7 +1869,7 @@ With a prefix argument, skip the balance check."
                               end-comment-state)))))))
 
 (defun paredit-check-region-for-delete:char-quote
-    (start start-state end end-state)
+    (_start start-state _end end-state)
   (let ((start-char-quote (nth 5 start-state))
         (end-char-quote (nth 5 end-state)))
     (if (not (eq start-char-quote end-char-quote))
@@ -1929,18 +1883,18 @@ With a prefix argument, skip the balance check."
 \f
 ;;;; Point Motion
 
-(eval-and-compile
-  (defmacro defun-motion (name bvl doc &rest body)
-    `(defun ,name ,bvl
-       ,doc
-       ,(xcond ((paredit-xemacs-p)
-                '(interactive "_"))
-               ((paredit-gnu-emacs-p)
-                ;++ Not sure this is sufficient for the `^'.
-                (if (fboundp 'handle-shift-selection)
-                    '(interactive "^p")
-                    '(interactive "p"))))
-       ,@body)))
+(defmacro defun-motion (name bvl doc &rest body)
+  (declare (indent 2) (debug defun))
+  `(defun ,name ,bvl
+     ,doc
+     ,(cond ((featurep 'xemacs)
+             '(interactive "_"))
+            (t
+                                   ;++ Not sure this is sufficient for the `^'.
+             (if (fboundp 'handle-shift-selection)
+                 '(interactive "^p")
+               '(interactive "p"))))
+     ,@body))
 
 (defun-motion paredit-forward (&optional arg)
   "Move forward an S-expression, or up an S-expression forward.
@@ -1948,8 +1902,8 @@ If there are no more S-expressions in this one before the closing
   delimiter, move past that closing delimiter; otherwise, move forward
   past the S-expression following the point."
   (let ((n (or arg 1)))
-    (cond ((< 0 n) (dotimes (i n)       (paredit-move-forward)))
-          ((< n 0) (dotimes (i (- n))   (paredit-move-backward))))))
+    (cond ((< 0 n) (dotimes (_ n)       (paredit-move-forward)))
+          ((< n 0) (dotimes (_ (- n))   (paredit-move-backward))))))
 
 (defun-motion paredit-backward (&optional arg)
   "Move backward an S-expression, or up an S-expression backward.
@@ -1957,8 +1911,8 @@ If there are no more S-expressions in this one before the opening
   delimiter, move past that opening delimiter backward; otherwise,
   move backward past the S-expression preceding the point."
   (let ((n (or arg 1)))
-    (cond ((< 0 n) (dotimes (i n)       (paredit-move-backward)))
-          ((< n 0) (dotimes (i (- n))   (paredit-move-forward))))))
+    (cond ((< 0 n) (dotimes (_ n)       (paredit-move-backward)))
+          ((< n 0) (dotimes (_ (- n))   (paredit-move-forward))))))
 \f
 (defun paredit-move-forward ()
   (cond ((paredit-in-string-p)
@@ -1995,7 +1949,7 @@ If there are no more S-expressions in this one before the opening
 \f
 ;;;; Window Positioning
 
-(defalias 'paredit-recentre-on-sexp 'paredit-recenter-on-sexp)
+(defalias 'paredit-recentre-on-sexp #'paredit-recenter-on-sexp)
 
 (defun paredit-recenter-on-sexp (&optional n)
   "Recenter the screen on the S-expression following the point.
@@ -2145,11 +2099,11 @@ By default OPEN and CLOSE are round delimiters."
   (let ((open (or open ?\( ))
         (close (or close ?\) )))
     (paredit-handle-sexp-errors
-        ((lambda (n) (paredit-insert-pair n open close 'goto-char))
-         (cond ((integerp argument) argument)
-               ((consp argument) (paredit-count-sexps-forward))
-               ((paredit-region-active-p) nil)
-               (t 1)))
+        (let ((n (cond ((integerp argument) argument)
+                       ((consp argument) (paredit-count-sexps-forward))
+                       ((paredit-region-active-p) nil)
+                       (t 1))))
+          (paredit-insert-pair n open close 'goto-char))
       (insert close)
       (backward-char)))
   (save-excursion (backward-up-list) (indent-sexp)))
@@ -2326,8 +2280,10 @@ If the point is on an S-expression, such as a string or a symbol, not
         ;;
         ;; XXX What if the *column* of the starting point was preserved
         ;; too?  Should we avoid reindenting in that case?
-        (if (not (eq (save-excursion (goto-char indent-start) (point-at-eol))
-                     (save-excursion (goto-char indent-end) (point-at-eol))))
+        (if (not (eq (save-excursion (goto-char indent-start)
+                                     (line-end-position))
+                     (save-excursion (goto-char indent-end)
+                                     (line-end-position))))
             (indent-region indent-start indent-end nil))))))
 \f
 ;;; The effects of convolution on the surrounding whitespace are pretty
@@ -2741,7 +2697,7 @@ Both must be lists, strings, or atoms; error if there is a mismatch."
          ;; (foo)| ;x\n(bar) => (foo | ;x\nbar), not (foo|  ;x\nbar).
          (and (not (eolp))
               (save-excursion
-                (paredit-skip-whitespace t (point-at-eol))
+                (paredit-skip-whitespace t (line-end-position))
                 (eq (char-after) ?\;)))
          ;; (foo)|(bar) => (foo| bar), not (foo|bar).
          (and (= left-point right-point)
@@ -2842,10 +2798,8 @@ If TRAILING-P is nil, skip leading whitespace; otherwise, skip trailing
            limit))    ; is a comment end, not newline, in Lisp mode.
 
 (defalias 'paredit-region-active-p
-  (xcond ((paredit-xemacs-p) 'region-active-p)
-         ((paredit-gnu-emacs-p)
-          (lambda ()
-            (and mark-active transient-mark-mode)))))
+  (cond ((fboundp 'region-active-p) #'region-active-p)
+        (t (lambda () (and mark-active transient-mark-mode)))))
 
 (defun paredit-hack-kill-region (start end)
   "Kill the region between START and END.
@@ -2886,16 +2840,17 @@ Don't reindent the line starting at START, however."
       (error "Incorrectly related points: %S, %S" start end))
   (save-excursion
     (goto-char start)
-    (let ((bol (point-at-bol)))
+    (let ((bol (line-beginning-position)))
       ;; Skip all S-expressions that end on the starting line, but
       ;; don't go past `end'.
-      (if (and (save-excursion (goto-char end) (not (eq bol (point-at-bol))))
+      (if (and (save-excursion (goto-char end)
+                               (not (eq bol (line-beginning-position))))
                (paredit-handle-sexp-errors
                    (catch 'exit
                      (while t
                        (save-excursion
                          (forward-sexp)
-                         (if (not (eq bol (point-at-bol)))
+                         (if (not (eq bol (line-beginning-position)))
                              (throw 'exit t))
                          (if (not (< (point) end))
                              (throw 'exit nil)))
@@ -2904,9 +2859,9 @@ Don't reindent the line starting at START, however."
           (progn
             ;; Point is still on the same line, but precedes an
             ;; S-expression that ends on a different line.
-            (if (not (eq bol (point-at-bol)))
+            (if (not (eq bol (line-beginning-position)))
                 (error "Internal error -- we moved forward a line!"))
-            (goto-char (+ 1 (point-at-eol)))
+            (goto-char (+ 1 (line-end-position)))
             (if (not (<= (point) end))
                 (error "Internal error -- we frobnitzed the garfnut!"))
             (indent-region (point) end nil))))))
@@ -3062,7 +3017,7 @@ This is independent of context -- it doesn't check what state the
   ;; Like current-column, but respects field boundaries in interactive
   ;; modes like ielm.  For use only with paredit-restore-column, which
   ;; works relative to point-at-bol.
-  (- (point) (point-at-bol)))
+  (- (point) (line-beginning-position)))
 
 (defun paredit-current-indentation ()
   (save-excursion
@@ -3077,7 +3032,7 @@ This is independent of context -- it doesn't check what state the
   ;; left).
   (let ((indentation* (paredit-current-indentation)))
     (goto-char
-     (+ (point-at-bol)
+     (+ (line-beginning-position)
         (cond ((not (< column indentation))
                (+ column (- indentation* indentation)))
               ((<= indentation* column) indentation*)

Re: Default lexical-binding to t

[Andrea Corallo]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> In bug#74145 I suggest that it is time to change the default of
> `lexical-binding` to t.  Clearly this is a breaking change, but only for
> those files which:
>
> - Don't have a `lexical-binding` cookie.
> - Have code which happens to behave differently under the new dialect
>   (such code is not rare, but a lot of code works identically in the
>   two dialects).
>
> I believe by the time Emacs-31 will be released, such files will be
> uncommon, and it is easy to fix them (either by adjusting he code, or
> by slapping a `lexical-binding` cookie).

I support this, together with the idea of having a warning when loading
file without the cookie.  We can always remove the warning (and the
cookies) in the future when/if we feel it's time.

  Andrea

Re: lexical-binding in paredit

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>> I also think lexical-binding is the de facto standard at this point; I'm
>>> not sure I've seen any dynamic-binding Elisp in a while. (I know it's still
>>> out there but I've only seen it in very old packages.)
>> Paredit is a notable example which doesn't have the cookie.
>> I don't know whether it actually uses any dynamic bindings but I think
>> there's some chance it does, based on the style of some parts I've read.
>
> After playing the janitor for a while I got the patch below and while
> the code does have some of the typical traits of "old ELisp" code, it
> doesn't seem to rely on dynamic scoping anywhere (tho I haven't actually
> tested it, so this is no guarantee).

It seems to work fine in some light testing, FWIW.

Re: lexical-binding in paredit

[Taylor R Campbell]

I have pushed some changes to https://paredit.org/paredit.git master.

I enabled `lexical-binding: t' (it would be astonishing to me -- and
an embarrassing mistake -- if anything in paredit.el relied on dynamic
binding other than for special variables like `show-paren-mode' or
`delete-active-region') and I pacified all the warnings that came up
while running the tests.

Were there any other important changes in the patch that was attached?
Since paredit.el still works fine in GNU Emacs 21, I don't see the
need for the large amount of churn to drop support for it, but maybe I
missed some other important fix in the churn.

FYI, there is an automatic test suite in the repository if you want to
test changes -- just run ./check.sh.

Re: lexical-binding in paredit

[Stefan Monnier]

> Since paredit.el still works fine in GNU Emacs 21, I don't see the
> need for the large amount of churn to drop support for it, but maybe I

The only part of the patch related to support for older Emacsen was the
part described as:

    (paredit-initialize-comment-dwim): Delete function needed only for Emacs-21.
    (paredit-comment-dwim): Call `comment-normalize-vars` instead.


> missed some other important fix in the churn.

Nothing important, no:

    Add `declare`s to `defmacro`s.

would help Edebugging, but I guess that hasn't been a problem so far.

    Avoid obsolete `point-at-eol/bol`.

silences compiler warnings but apparently not in the version of Emacs
you're using.

    (paredit-xemacs-p, paredit-gnu-emacs-p): Inline all calls (so the
    byte-compiler can throw away the irrelevant code) and then delete.
    (xcond): Delete macro, since after expanding `paredit-*emacs-p` it's

makes the source code simpler and the .elc code shorter.


        Stefan

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > - Don't have a `lexical-binding` cookie.
  > - Have code which happens to behave differently under the new dialect
  >   (such code is not rare, but a lot of code works identically in the
  >   two dialects).

  > I believe by the time Emacs-31 will be released, such files will be
  > uncommon, and it is easy to fix them (either by adjusting he code, or
  > by slapping a `lexical-binding` cookie).

It seems that people are forming ideas of how frequent dynamic binding
files are by looking at code in active development.  Of course it is
rare there.  But user have a lot of old Lisp files that they haven't
used for a while.  Some of them, someone will try to use someday.

So we should never expect the user commnity to quickly eliminate
constructs that we plan to break.  We should never rush to break it.

Many users upgrade Emacs only occasionally, and often skip one or more
versions.  And people have Lisp code they have not run in some years
-- but that doesn't mean they never will.  Those disused files may be
more numerous than the files people regularly use.

I think we should prepare for the change in Emacs 31, and go the rest
of the way in Emacs 33 or 34.  Emacs 31 should go all out to urge
users to fix their Lisp files to specify lexical-binding.

For now, we should improve the warning given for a file that fails to
specify lexical-binding.  Let's make it explicitly say what we want
users to do.  "To avoid confusion later, add lexical-binding to the
file's local variables now."

For warning when visiting a file, instead of a cryptic "/d", it whould
show clear instructions.

Can we add that trivial change to 31.2?

The more cases in which we notify users, and the more explicitly we do
so, the sooner users will follow that advice.  With this
encouragement, I think we can expect that few programs that don't
specify lexical-binding will remain after a few more years.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Jean Louis]

* Stefan Monnier <monnier@iro.umontreal.ca> [2024-11-03 16:59]:
> > To be clear, here I'm talking only about disabling the _already
> > existing_ warnings that we have when byte-compiling, and in the
> > mode line.
> 
> I'd first like us to get experience with "lexical-binding defaults to t"
> before considering removing the cookie for files.
> I would hate having to re-add the cookie if we find out that there are
> many more problems than expected and we revert my patch.

How can I remain with the file that likes to have global binding?

Can I use the negative cookie in that case?

-- 
Jean Louis

Re: Default lexical-binding to t

[tomas]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

On Mon, Nov 04, 2024 at 11:35:53AM +0300, Jean Louis wrote:

[...]

> How can I remain with the file that likes to have global binding?

You mean "dynamic binding", I guess.

In that case: just set lexical binding to nil at the top of the
file:

;;; ...  -*- lexical-binding: nil -*-

(It's in the manual, "12.10.4 Selecting Lisp Dialect". The present
discussion is just about changing the /default/ (i.e. no explicit
cookie) from dynamic to lexical.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Default lexical-binding to t

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> It seems that people are forming ideas of how frequent dynamic binding
> files are by looking at code in active development.  Of course it is
> rare there.  But user have a lot of old Lisp files that they haven't
> used for a while.  Some of them, someone will try to use someday.

It is far too early even to arrive at this conclusion.  I have said it
before and I'll say it again.  Even actively developed packages on the
EmacsWiki are still to be amended with lexical binding directives.

  https://www.emacswiki.org/emacs/info%2b.el
  (Last updated Mon Jan 29 15:53:30 2024 (-0800))

For instance.

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 04 Nov 2024 00:34:25 -0500
> 
> For now, we should improve the warning given for a file that fails to
> specify lexical-binding.  Let's make it explicitly say what we want
> users to do.  "To avoid confusion later, add lexical-binding to the
> file's local variables now."

If we urge people to specify lexical-binding in their files, then what
is the point of making lexical-binding the default?

> For warning when visiting a file, instead of a cryptic "/d", it whould
> show clear instructions.
> 
> Can we add that trivial change to 31.2?

You mean, 31.1, I guess?

> The more cases in which we notify users, and the more explicitly we do
> so, the sooner users will follow that advice.  With this
> encouragement, I think we can expect that few programs that don't
> specify lexical-binding will remain after a few more years.

What bothers me is how do I know whether a given Lisp file of mine
will have problems under lexical-binding.  Do we have any tools which
will help me determine that?

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,  emacs-devel@gnu.org
> Date: Mon, 04 Nov 2024 17:39:44 +0800
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> > It seems that people are forming ideas of how frequent dynamic binding
> > files are by looking at code in active development.  Of course it is
> > rare there.  But user have a lot of old Lisp files that they haven't
> > used for a while.  Some of them, someone will try to use someday.
> 
> It is far too early even to arrive at this conclusion.  I have said it
> before and I'll say it again.  Even actively developed packages on the
> EmacsWiki are still to be amended with lexical binding directives.
> 
>   https://www.emacswiki.org/emacs/info%2b.el
>   (Last updated Mon Jan 29 15:53:30 2024 (-0800))

How do you propose to encourage people to convert sooner rather than
later?  If we just wait for that to happen spontaneously, we will wait
forever.

Re: Default lexical-binding to t

[Alfred M. Szmidt]

   Richard Stallman <rms@gnu.org> writes:

   > It seems that people are forming ideas of how frequent dynamic binding
   > files are by looking at code in active development.  Of course it is
   > rare there.  But user have a lot of old Lisp files that they haven't
   > used for a while.  Some of them, someone will try to use someday.

   It is far too early even to arrive at this conclusion.  I have said it
   before and I'll say it again.  Even actively developed packages on the
   EmacsWiki are still to be amended with lexical binding directives.

     https://www.emacswiki.org/emacs/info%2b.el
     (Last updated Mon Jan 29 15:53:30 2024 (-0800))

   For instance.

I am often of that old annoying guard that wants to keep things as it,
so take it with the ptich of salt it deserves.

But I think that holding back on lexical-binding is worse than pushing
it forward.  The quicker it happens, the better.  Even if it breaks
existing code that is being updated, the net benefit worth it.

It is a big change, and to mentionin it in big letters in NEWS should
be enough at this point.

RE: [External] : Re: Default lexical-binding to t

[Drew Adams]

>    Even actively developed packages on the EmacsWiki are
>    still to be amended with lexical binding directives.
> 
>    https://www.emacswiki.org/emacs/info*2b.el
>
>    For instance.
> 
> I am often of that old annoying guard that wants to keep things as it,
> so take it with the ptich of salt it deserves.
> 
> But I think that holding back on lexical-binding is worse than pushing
> it forward.  The quicker it happens, the better.  Even if it breaks
> existing code that is being updated, the net benefit worth it.
> 
> It is a big change, and to mentionin it in big letters in NEWS should
> be enough at this point.

+1.

FWIW, I have no problem adding a `lexical-binding: nil'
cookie to my libraries that haven't yet been converted
to use `lexical-binding: t'.

(Ultimately I hope to convert all to `lexical-binding:
t', but in some cases that could take a while.)

I think the message to users should be, _in priority_:

 0. Make a backup of your files, for safe keeping.

 1. Add a `lexical-binding: nil' cookie to each file.

 2. Convert code in a file to lexical binding, then
    change the file's cookie to `lexical-binding: t'.

    OR, if you don't care about compatibility with
    release  NN.N or older, just remove the cookie
    altogether (`lexical-binding: t' is the default).

#1 is always easier than #2.  It should be the first
priority - just fix things so they work as before.
#1 requires no thinking.

#2 is desirable, but can require some care/thinking.

The message should make clear that if you care about
backward compatibility in any way then it's a good
idea to have an explicit `lexical-binding' cookie,
whether nil or t.

(It's also for that reason that I wonder whether the
code provided by GNU Emacs shouldn't continue,
"forever", to explicitly use `lexical-binding: t'.
Wouldn't hurt.)

_______

That said, there are no doubt lots of files out there
that are not "actively maintained", but that "just
work".  And many of them are likely useful and used,
even by users who upgrade to new Emacs releases.

Those libraries are more problematic.  It would be
good for GNU Emacs to provide a document describing
symptoms that indicate that a file might need a
`lexical-binding: nil' cookie.  The symptoms should
be described clearly to "end" users who just use
libraries and aren't necessarily Elisp-oriented.
Emacs has loads of such users, I expect (and hope).

The message should assure users that adding
`lexical-binding: nil' likely won't hurt anything,
and it's likely needed, given the symptoms described.
IOW: _try_ adding that cookie.  If it doesn't help,
then...

Re: Default lexical-binding to t

[Stefan Monnier]

> What bothers me is how do I know whether a given Lisp file of mine
> will have problems under lexical-binding.  Do we have any tools which
> will help me determine that?

It's generally impossible to do that reliably (it can be reduced to
a variant of the halting problem), but the problem is exactly the same
as that of converting from dynbind to lexbind (which we do describe in
the manual).

I.e. the tool we offer is to look at the output of the byte-compiler
when compiling the file with lexical binding enabled (either by adding
the cookie manually or by compiling with an Emacs which defaults to it):
all the "unused var" and "reference to free var" warnings are hints at
places where the switch from dynbind to lexbind *might* change
the semantics.

Other places worth looking at are those that call `eval`, `symbol-value`, or
`intern`, or things like backquoted lambdas used at runtime (as opposed to
inside a macro), or `lambda`s hidden inside quoted constants
(e.g. a big alist stored in a `defconst` where some of the entries
contain `lambda`s).

But in all those cases, you need to look at the code and try to
understand enough of what it does (as well as enough of how dynbind and
lexbind differ) to figure out whether dynbind-vs-lexbind will make
a difference.

IOW, nothing beats actually testing the code, IME.  In 99% of the cases,
changes introduced by the switch will manifest as "void variable" errors.


        Stefan

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But I think that holding back on lexical-binding is worse than pushing
  > it forward.  The quicker it happens, the better.

What precisely does "it" refer to in that sentence?
What does it include, and what does it exclude?

The words assume that "it" will be done completely, sooner or later,
and the only question is how fast.  I think that implies that "it"
doss not include updating all of peoples old-but-working Lisp files so
that they would run in new Emacs versions.

If we were talking only about live code, programs that people
maintain, maybe your point would be valid.  By assumption, those
programs will be updated before too long,

But the old-but-working programs may not be touched again until years
from now.  When that happens, they simply won't work, and the people
who might want them won't know what the problem is.

We need to think of those programs _too_.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  emacs-devel@gnu.org
> Date: Tue, 05 Nov 2024 15:24:07 -0500
> 
> > What bothers me is how do I know whether a given Lisp file of mine
> > will have problems under lexical-binding.  Do we have any tools which
> > will help me determine that?
> 
> It's generally impossible to do that reliably (it can be reduced to
> a variant of the halting problem), but the problem is exactly the same
> as that of converting from dynbind to lexbind (which we do describe in
> the manual).

Too bad.

> IOW, nothing beats actually testing the code, IME.  In 99% of the cases,
> changes introduced by the switch will manifest as "void variable" errors.

Consider the plight of someone who over the years have accrued
hundreds of lines of Lisp in their init and customization files, and
now needs somehow to walk over all of that and see if anything needs
changing, or risk breakage in something that has been working for
eons.  It would be nice if we could help those poor souls, and not
just hang them to dry.

Re: Default lexical-binding to t

[Stefan Monnier]

> Consider the plight of someone who over the years have accrued
> hundreds of lines of Lisp in their init and customization files, and
> now needs somehow to walk over all of that and see if anything needs
> changing, or risk breakage in something that has been working for
> eons.  It would be nice if we could help those poor souls, and not
> just hang them to dry.

They should likely add a `lexical-binding:nil` cookie and move on.

Of course, they can also change this cookie to `t` temporarily every
once in a while to see if it uncovers a problem or not, so the
conversion can be done bit by bit.


        Stefan

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 06 Nov 2024 07:50:43 -0500
> 
> > Consider the plight of someone who over the years have accrued
> > hundreds of lines of Lisp in their init and customization files, and
> > now needs somehow to walk over all of that and see if anything needs
> > changing, or risk breakage in something that has been working for
> > eons.  It would be nice if we could help those poor souls, and not
> > just hang them to dry.
> 
> They should likely add a `lexical-binding:nil` cookie and move on.

In each and every file?  Not a simple thing to do.

> Of course, they can also change this cookie to `t` temporarily every
> once in a while to see if it uncovers a problem or not, so the
> conversion can be done bit by bit.

For some code that gets executed only rarely, this is not practical.

Can you tell what are the benefits of turning on lexical-binding by
default, relative to what we have now?  Which files/uses are affected,
and how?  Maybe if we see all the potential effects, we could find a
nicer way forward.

Re: Default lexical-binding to t

[Stefan Monnier]

>> They should likely add a `lexical-binding:nil` cookie and move on.
> In each and every file?

Yup.

> Not a simple thing to do.

It doesn't require any thought, so I consider it simple.
Also, there shouldn't be so terribly many such files remaining nowadays.

But if experience shows it to be a problem, I guess we could provide
a command that automates it.

>> Of course, they can also change this cookie to `t` temporarily every
>> once in a while to see if it uncovers a problem or not, so the
>> conversion can be done bit by bit.
> For some code that gets executed only rarely, this is not practical.

I don't see why.

> Can you tell what are the benefits of turning on lexical-binding by
> default, relative to what we have now?

IME, nowadays lexbind is the "de facto" default used for all
new ELisp code, including code snippets posted around the web.
So using dynbind when there's no cookie is often a source of errors
and confusion.


        Stefan

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 06 Nov 2024 11:32:10 -0500
> 
> > Can you tell what are the benefits of turning on lexical-binding by
> > default, relative to what we have now?
> 
> IME, nowadays lexbind is the "de facto" default used for all
> new ELisp code, including code snippets posted around the web.
> So using dynbind when there's no cookie is often a source of errors
> and confusion.

Thanks, but I asked about specifics, not in general.  And this was the
less important part of my question; you elided the more important
parts.

I'm trying to find a way of moving forward with this while minimizing
objections and complaints, so please help me finding the auxiliary
measures which will make this possible and hopefully less painful.

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 06, 2024 at 11:32:10 -0500, Stefan Monnier wrote:
> >> They should likely add a `lexical-binding:nil` cookie and move on.
> > In each and every file?

> Yup.

> > Not a simple thing to do.

> It doesn't require any thought, so I consider it simple.
> Also, there shouldn't be so terribly many such files remaining nowadays.

> But if experience shows it to be a problem, I guess we could provide
> a command that automates it.

> >> Of course, they can also change this cookie to `t` temporarily every
> >> once in a while to see if it uncovers a problem or not, so the
> >> conversion can be done bit by bit.
> > For some code that gets executed only rarely, this is not practical.

> I don't see why.

> > Can you tell what are the benefits of turning on lexical-binding by
> > default, relative to what we have now?

> IME, nowadays lexbind is the "de facto" default used for all
> new ELisp code, including code snippets posted around the web.
> So using dynbind when there's no cookie is often a source of errors
> and confusion.

Maybe briefly explaining the philosophy of lexical binding somewhere
(maybe NEWS) would help.

I still don't understand it.  I go along with lexical binding because
quite a lot of people cleverer than me have said it is good.  But why?

It doesn't seem to make code run faster (I think CC Mode became
marginally slower after the conversion to lexical binding).  OK, we can
create closures, but so what?  What user features in Emacs have become
possible, or practical, because of lexical binding?

I'm not trying to be argumentative here.  I'm just puzzled about it, as
I have been for quite a number of years.

>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Jim Porter]

On 11/5/2024 12:24 PM, Stefan Monnier wrote:
>> What bothers me is how do I know whether a given Lisp file of mine
>> will have problems under lexical-binding.  Do we have any tools which
>> will help me determine that?
> 
> It's generally impossible to do that reliably (it can be reduced to
> a variant of the halting problem), but the problem is exactly the same
> as that of converting from dynbind to lexbind (which we do describe in
> the manual).

How about a less-reliable tool that catches some easy-to-detect 
problems? For example, look in every defun and see if the macro-expanded 
form of the function body refers to a variable that is neither 1) 
declared with 'defvar' or 2) let-bound in that function. This logic 
might not be exactly what we want since I haven't thought very long 
about it, but something along these lines could hopefully catch many errors.

Even if we can't catch 100% of issues, catching 80% of them would make 
users' lives easier.

Re: Default lexical-binding to t

[Stefan Monnier]

>>> What bothers me is how do I know whether a given Lisp file of mine
>>> will have problems under lexical-binding.  Do we have any tools which
>>> will help me determine that?
>> It's generally impossible to do that reliably (it can be reduced to
>> a variant of the halting problem), but the problem is exactly the same
>> as that of converting from dynbind to lexbind (which we do describe in
>> the manual).
> How about a less-reliable tool that catches some easy-to-detect problems?
> For example, look in every defun and see if the macro-expanded form of the
> function body refers to a variable that is neither 1) declared with 'defvar'
> or 2) let-bound in that function. This logic might not be exactly what we
> want since I haven't thought very long about it, but something along these
> lines could hopefully catch many errors.

That's what the compiler's warnings give you (plus a few related cases
such as the vars that are let-bound but not used within that let's
lexical scope).


        Stefan

Re: Default lexical-binding to t

[Jose A. Ortega Ruiz]

On Wed, Nov 06 2024, Jim Porter wrote:

> On 11/5/2024 12:24 PM, Stefan Monnier wrote:
>>> What bothers me is how do I know whether a given Lisp file of mine
>>> will have problems under lexical-binding.  Do we have any tools which
>>> will help me determine that?
>> It's generally impossible to do that reliably (it can be reduced to
>> a variant of the halting problem), but the problem is exactly the same
>> as that of converting from dynbind to lexbind (which we do describe in
>> the manual).
>
> How about a less-reliable tool that catches some easy-to-detect problems? For
> example, look in every defun and see if the macro-expanded form of the
> function body refers to a variable that is neither 1) declared with 'defvar'
> or 2) let-bound in that function. This logic might not be exactly what we want
> since I haven't thought very long about it, but something along these lines
> could hopefully catch many errors.
>
> Even if we can't catch 100% of issues, catching 80% of them would make users'
> lives easier.

Apologies in advance if this has been suggested and discarded before, or
if I am missing something obvious, but wouldn't it be possible to have
an additional level of indirection with a new variable,
default-lexical-binding, settable in, say, early-init.el? It's not a
real solution, but at least people with lots of trouble with the new
default could just set it off in one place.

Just my 2c,
jao
-- 
A student came to the master and asked, for the master was one of them
who knew such things: "Does Emacs have the Buddha nature?" The master
contemplated this for some time, and answered: "I don't see why not,
it has about everything else."

Re: Default lexical-binding to t

[John Yates]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

Being fundamentally a programmer in languages that support only lexical
binding, I have little sense of the mechanics underlying dynamic binding.
That said, I wonder if something along these line could help a bit:

If a "void variable" error is about to be thrown and the source defaulted
> to dynamic binding then
>
>    - Perform a dynamic lookup of the offending variable
>
>
>    - If the lookup is unsuccessful then issue the same error as always
>
>
>    - If the lookup is successful, then modify the diagnostic to mention
>    the possibility of a dynamic / lexical binding clash
>
> /john

On Wed, Nov 6, 2024 at 2:27 PM Stefan Monnier <monnier@iro.umontreal.ca>
wrote:

> >>> What bothers me is how do I know whether a given Lisp file of mine
> >>> will have problems under lexical-binding.  Do we have any tools which
> >>> will help me determine that?
> >> It's generally impossible to do that reliably (it can be reduced to
> >> a variant of the halting problem), but the problem is exactly the same
> >> as that of converting from dynbind to lexbind (which we do describe in
> >> the manual).
> > How about a less-reliable tool that catches some easy-to-detect problems?
> > For example, look in every defun and see if the macro-expanded form of
> the
> > function body refers to a variable that is neither 1) declared with
> 'defvar'
> > or 2) let-bound in that function. This logic might not be exactly what we
> > want since I haven't thought very long about it, but something along
> these
> > lines could hopefully catch many errors.
>
> That's what the compiler's warnings give you (plus a few related cases
> such as the vars that are let-bound but not used within that let's
> lexical scope).
>
>
>         Stefan
>
>
>

-- 
John Yates
505 Tremont St, #803
Boston, MA 02116

[-- Attachment #2: Type: text/html, Size: 3373 bytes --]

Re: Default lexical-binding to t

[Joost Kremers]

On Wed, Nov 06 2024, Alan Mackenzie wrote:
> Maybe briefly explaining the philosophy of lexical binding somewhere
> (maybe NEWS) would help.
>
> I still don't understand it.  I go along with lexical binding because
> quite a lot of people cleverer than me have said it is good.  But why?

The Elisp manual has a few short remarks on it:

(info "(elisp) Lexical Binding")

,----
| Lexical binding was introduced to Emacs, as an optional feature, in
| version 24.1.  We expect its importance to increase with time.  Lexical
| binding opens up many more opportunities for optimization, so programs
| using it are likely to run faster in future Emacs versions.  Lexical
| binding is also more compatible with concurrency, which was added to
| Emacs in version 26.1.
`----

and:

(info "(elisp) Dynamic Binding")

,----
|    Note that when code using Dynamic Binding is native compiled the
| native compiler will not perform any Lisp specific optimization.
`----

AFAIU, dynamic binding is especially dangerous if you're in the habit of
setting variables without let-binding them. A coding style that was perhaps
more common in the past than it is now.

-- 
Joost Kremers
Life has its moments

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Joost.

On Wed, Nov 06, 2024 at 21:48:05 +0100, Joost Kremers wrote:
> On Wed, Nov 06 2024, Alan Mackenzie wrote:
> > Maybe briefly explaining the philosophy of lexical binding somewhere
> > (maybe NEWS) would help.

> > I still don't understand it.  I go along with lexical binding because
> > quite a lot of people cleverer than me have said it is good.  But why?

> The Elisp manual has a few short remarks on it:

> (info "(elisp) Lexical Binding")

> ,----
> | Lexical binding was introduced to Emacs, as an optional feature, in
> | version 24.1.  We expect its importance to increase with time.  Lexical
> | binding opens up many more opportunities for optimization, so programs
> | using it are likely to run faster in future Emacs versions.

This was several major Emacs versions ago.  Has anybody actually done
any measurements to demonstrate how effective this optimisation has
been?  When I tried comparing dynamic vs. lexical .elc versions of CC
Mode scrolling through xdisp.c, dynamic took 8.3247s, lexical took
8.3285s.  That's the same, within measurement accuracy.

> | Lexical
> | binding is also more compatible with concurrency, which was added to
> | Emacs in version 26.1.
> `----

I wonder what "more" compatible means.  Concurrency doesn't appear to be
a major feature in Emacs, as far as I can tell.

> and:

> (info "(elisp) Dynamic Binding")

> ,----
> |    Note that when code using Dynamic Binding is native compiled the
> | native compiler will not perform any Lisp specific optimization.
> `----

This looks like a choice not to support dynamic binding as well as
lexical binding.  In my CC Mode benchmark (see above) with native
compiled code, the lexical binding version was somewhat faster than the
dynamic binding one, around 10%.  But the native compiled code was
scarcely faster than the byte compiled code.

> AFAIU, dynamic binding is especially dangerous if you're in the habit of
> setting variables without let-binding them. A coding style that was perhaps
> more common in the past than it is now.

I think you get a warning from the byte compiler for this, and have done
for many Emacs versions.

With lexical binding, I get the impression of an unstoppable juggernaut,
something with momentum of its own, careering ahead without anybody
being able to stop it or even question it.  I doubt it will do Emacs any
harm, once all the old .el files have been equipped with a -*- lexical
binding: nil -*- comment.  But I do wonder whether the massive time and
effort its development and proliferation have taken up were worth it.

> -- 
> Joost Kremers
> Life has its moments

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

> On Wed, Nov 06, 2024 at 21:48:05 +0100, Joost Kremers wrote:
>> (info "(elisp) Lexical Binding")
>
>> ,----
>> | Lexical binding was introduced to Emacs, as an optional feature, in
>> | version 24.1.  We expect its importance to increase with time.  Lexical
>> | binding opens up many more opportunities for optimization, so programs
>> | using it are likely to run faster in future Emacs versions.
>
> This was several major Emacs versions ago.  Has anybody actually done
> any measurements to demonstrate how effective this optimisation has
> been?  When I tried comparing dynamic vs. lexical .elc versions of CC
> Mode scrolling through xdisp.c, dynamic took 8.3247s, lexical took
> 8.3285s.  That's the same, within measurement accuracy.

Let me first point out that, at least to my mind, the most important
benefit of lexbind is that it's easier to use and reason about: the
byte-compiler can catch more mistakes, and it eliminates an entire class
of bugs (with dynbind, even presumed "local" variables can be
manipulated by any function you call).

Consider a simple example like this:

    (defun another-function ()
      (setq x 3))

    (defun foo ()
      (let ((x 1) (y 2))
        (another-function)
        (+ x y)))

This code is clearly bad, but regardless of that we get with dynbind,

    (foo) => 5

whereas with lexbind,

    (foo) => 3

Now, it's also true that theory tells us that lexbind should indeed
often lead to better performance, since the compiler can do
optimizations on such code that are hard or impractical with dynbind.

For example, if you compile and then disassemble the above function
`foo` using something like

    (byte-compile #'foo)
    (disassemble #'foo (current-buffer))

you will see that, with dynbind, this comes out to 11 byte-code
instructions, many of which are about checking for new values of `x` or
`y`, basically in case they have gotten new values after calling
`another-function`.  With lexbind, the byte-compiler can constant fold
`x` and `y`, and it comes out to 4 instructions instead.

Such optimizations will of course not matter for all Emacs Lisp code.
I'm also not sure that we have exhausted all optimization opportunities
that lexbind opens up in the current byte-compiler, but Mattias
Engdegård would know more about this.  (BTW, if that's true, it should
perhaps not come as a surprise, given that it started out as a compiler
for dynbind.)

The optimizations will also not improve performance in every single use
case.  The scrolling benchmark you did with CC Mode might be an example
of this.  I would guess that things like displaying lots of images,
which mostly takes place in C, are also not much faster.

Re: Default lexical-binding to t

[Sean Whitton]

Hello,

On Wed 06 Nov 2024 at 05:42pm GMT, Alan Mackenzie wrote:

> OK, we can create closures, but so what?

Sometimes being able to use closures enables you to experiment with code
in certain ways that lead you to come up with solutions to problems you
didn't think you could solve.

Without closures, sure, the solution is still expressible, but you might
not have ever come up with it.

It's the thing about good notation making the mathematical discoveries
even possible, even though the mathematical truths are not dependent on
the notation we adopt.

-- 
Sean Whitton

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I believe by the time Emacs-31 will be released, such files will be
  > > uncommon, and it is easy to fix them (either by adjusting he code, or
  > > by slapping a `lexical-binding` cookie).

  > I support this, together with the idea of having a warning when loading
  > file without the cookie.  We can always remove the warning (and the
  > cookies) in the future when/if we feel it's time.

If the goal is to lead ysers to indicate this explicitly in each file,
it is enough to complain about files that lack a file local variable
specification for this variable.  It could do this whenever it visits
a file in Emacs Lisp mode.

If we want to increase the pressure to label old files explicitly,
we could go beyond a warning.  For instance, we could ask the user
whether. "Is this an old file thjt is supposed to use dynamic binding?"

If the user answers yes to that, Emacs could offer to modify the file,
adding a tag.

By contrast, to change the interpretation of files that don't specify
the variable is ineffective for the purpuse.  But it will waste a lot
of long-time users' time in figuring out what went wrong.

Eli wrote:

  > If we urge people to specify lexical-binding in their files, then what
  > is the point of making lexical-binding the default?

Perhaps there is no reason to change the default itself ;-).  Perhaps
what we _should_ want is to put an end, among somewhat-maintained
programs, to dependence on the old defaull.

The methods described above will do the job better and faster than
changing the default, and much more gently too.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > For now, we should improve the warning given for a file that fails to
  > > specify lexical-binding.  Let's make it explicitly say what we want
  > > users to do.  "To avoid confusion later, add lexical-binding to the
  > > file's local variables now."

  > > For warning when visiting a file, instead of a cryptic "/d", it whould
  > > show clear instructions.

  > > Can we add that trivial change to 31.2?

  > You mean, 31.1, I guess?

Sorry, I meant 30.2.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What bothers me is how do I know whether a given Lisp file of mine
  > will have problems under lexical-binding.  Do we have any tools which
  > will help me determine that?

I don't know of any easy way to do that.  It requires software to
analyze the code and deduce how it will function.

However, to determine automatically _some_ value of `lexical-binding'
that will fowk in that file is easy: use nil.

In many cases, t would work too, but determining whether it would
calls for some intelligence.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Eli Zaretskii]

> Date: Wed, 6 Nov 2024 22:50:57 +0000
> Cc: emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> But I do wonder whether the massive time and effort its development
> and proliferation have taken up were worth it.

This could be said about almost every major feature in Emacs that
fundamentally changes the internals.  Are the time, energy, and
massive efforts I invested (and am still investing) in bidirectional
editing support worth it?  I don't know; there are days when I think
it was all a terrible waste.  What about all the multiple changes in
the internals that bring us 3.7% speedup in some benchmark?  What
about support for Lisp threads?  What about the addition of positions
to symbols? in how many error-message situations is this really
important?  And XInput2, and touch devices, etc. etc.

Emacs moves forward because someone whom we trust to be an expert in
some area or to understand well enough what our users might want has
an itch to scratch, not because we have some magic future insight.
Hindsight is always 20-20, but it is also unfair, given our
development model and the pull of core developers and expertise that
we can command.

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: monnier@iro.umontreal.ca, emacs-devel@gnu.org
> Date: Wed, 06 Nov 2024 22:46:43 -0500
> 
>   > > For now, we should improve the warning given for a file that fails to
>   > > specify lexical-binding.  Let's make it explicitly say what we want
>   > > users to do.  "To avoid confusion later, add lexical-binding to the
>   > > file's local variables now."
> 
>   > > For warning when visiting a file, instead of a cryptic "/d", it whould
>   > > show clear instructions.
> 
>   > > Can we add that trivial change to 31.2?
> 
>   > You mean, 31.1, I guess?
> 
> Sorry, I meant 30.2.

It depends on the warning, how frequently it will be shown, and how
annoying could it be as result.  Also how complex and safe is the code
to implement that.

In general, users don't expect significant functionality changes in
bugfix releases, only in XX.1 major releases.

Re: Default lexical-binding to t

[Joost Kremers]

Hi Eli,
> Are the time, energy, and
> massive efforts I invested (and am still investing) in bidirectional
> editing support worth it?  I don't know; there are days when I think
> it was all a terrible waste.

FWIW, I was extremely grateful for that work. At the time bidi support
became available in Emacs, I needed to work with Arabic texts quite a bit,
and it was great that I could do that from the comfort of my favourite
editor.

-- 
Joost Kremers
Life has its moments

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Joost Kremers <joostkremers@fastmail.fm>
> Cc: Alan Mackenzie <acm@muc.de>,  emacs-devel@gnu.org
> Date: Thu, 07 Nov 2024 09:07:28 +0100
> 
> Hi Eli,
> > Are the time, energy, and
> > massive efforts I invested (and am still investing) in bidirectional
> > editing support worth it?  I don't know; there are days when I think
> > it was all a terrible waste.
> 
> FWIW, I was extremely grateful for that work. At the time bidi support
> became available in Emacs, I needed to work with Arabic texts quite a bit,
> and it was great that I could do that from the comfort of my favourite
> editor.

Thanks.  But there are other voices in this regard.

Re: Default lexical-binding to t

[tomas]

[-- Attachment #1: Type: text/plain, Size: 829 bytes --]

On Thu, Nov 07, 2024 at 10:45:21AM +0200, Eli Zaretskii wrote:
> > From: Joost Kremers <joostkremers@fastmail.fm>
> > Cc: Alan Mackenzie <acm@muc.de>,  emacs-devel@gnu.org
> > Date: Thu, 07 Nov 2024 09:07:28 +0100
> > 
> > Hi Eli,
> > > Are the time, energy, and
> > > massive efforts I invested (and am still investing) in bidirectional
> > > editing support worth it?  I don't know; there are days when I think
> > > it was all a terrible waste.
> > 
> > FWIW, I was extremely grateful for that work. At the time bidi support
> > became available in Emacs, I needed to work with Arabic texts quite a bit,
> > and it was great that I could do that from the comfort of my favourite
> > editor.
> 
> Thanks.  But there are other voices in this regard.

Count me also on the "thank you!" side.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Stefan.

On Wed, Nov 06, 2024 at 16:46:52 -0800, Stefan Kangas wrote:
> Alan Mackenzie <acm@muc.de> writes:

> > On Wed, Nov 06, 2024 at 21:48:05 +0100, Joost Kremers wrote:
> >> (info "(elisp) Lexical Binding")

> >> ,----
> >> | Lexical binding was introduced to Emacs, as an optional feature, in
> >> | version 24.1.  We expect its importance to increase with time.  Lexical
> >> | binding opens up many more opportunities for optimization, so programs
> >> | using it are likely to run faster in future Emacs versions.

> > This was several major Emacs versions ago.  Has anybody actually done
> > any measurements to demonstrate how effective this optimisation has
> > been?  When I tried comparing dynamic vs. lexical .elc versions of CC
> > Mode scrolling through xdisp.c, dynamic took 8.3247s, lexical took
> > 8.3285s.  That's the same, within measurement accuracy.

> Let me first point out that, at least to my mind, the most important
> benefit of lexbind is that it's easier to use and reason about: the
> byte-compiler can catch more mistakes, and it eliminates an entire class
> of bugs (with dynbind, even presumed "local" variables can be
> manipulated by any function you call).

Lexical binding makes debugging more difficult.  For example, the
compilation process discards the names of parameters to functions.

> Consider a simple example like this:

>     (defun another-function ()
>       (setq x 3))

>     (defun foo ()
>       (let ((x 1) (y 2))
>         (another-function)
>         (+ x y)))

> This code is clearly bad, but regardless of that we get with dynbind,

>     (foo) => 5

> whereas with lexbind,

>     (foo) => 3

Yes.  Both results are clearly correct.

> Now, it's also true that theory tells us that lexbind should indeed
> often lead to better performance, since the compiler can do
> optimizations on such code that are hard or impractical with dynbind.

But the overhead of having to construct closures for internal functions
tends to counter other optimisations.

> For example, if you compile and then disassemble the above function
> `foo` using something like

>     (byte-compile #'foo)
>     (disassemble #'foo (current-buffer))

> you will see that, with dynbind, this comes out to 11 byte-code
> instructions, many of which are about checking for new values of `x` or
> `y`, basically in case they have gotten new values after calling
> `another-function`.  With lexbind, the byte-compiler can constant fold
> `x` and `y`, and it comes out to 4 instructions instead.

As a counter, try compiling the following under both styles of bindings
and printing out the disassemblies (M-: (hack-local-variables) is your
friend when changing the lexical-binding setting):

;; -*- lexical-binding: nil -*-
(defvar external-function nil)

(defun foo (a)
  (funcall external-function
	   (lambda (b)
	     (+ a b))))

Under dynamic binding this is 8 byte-code instructions.  With lexical
binding, it's 11, including closure creation instructions.  Should foo
get called in a tight loop, these instructions will be slow.  There are
no such problems with the dynamic version.

> Such optimizations will of course not matter for all Emacs Lisp code.
> I'm also not sure that we have exhausted all optimization opportunities
> that lexbind opens up in the current byte-compiler, but Mattias
> Engdegård would know more about this.  (BTW, if that's true, it should
> perhaps not come as a surprise, given that it started out as a compiler
> for dynbind.)

> The optimizations will also not improve performance in every single use
> case.  The scrolling benchmark you did with CC Mode might be an example
> of this.  I would guess that things like displaying lots of images,
> which mostly takes place in C, are also not much faster.

Most of the time in that scrolling is taken by font locking, which is
primarily compiled Lisp.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Eli.

On Thu, Nov 07, 2024 at 08:14:43 +0200, Eli Zaretskii wrote:
> > Date: Wed, 6 Nov 2024 22:50:57 +0000
> > Cc: emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > But I do wonder whether the massive time and effort its development
> > and proliferation have taken up were worth it.

> This could be said about almost every major feature in Emacs that
> fundamentally changes the internals.  Are the time, energy, and
> massive efforts I invested (and am still investing) in bidirectional
> editing support worth it?  I don't know; there are days when I think
> it was all a terrible waste.  What about all the multiple changes in
> the internals that bring us 3.7% speedup in some benchmark?  What
> about support for Lisp threads?  What about the addition of positions
> to symbols? in how many error-message situations is this really
> important?  And XInput2, and touch devices, etc. etc.

No, the difference is that most major features are implemented by one
person (or a small team) and are complete in themselves.  Lexical
binding involved significant effort from lots and lots of contributors,
each having to modify "his own" files as lexical binding steadily
morphed from being optional to being compulsory.

I still wonder who made the decision to convert the entire code base,
and when.  Lexical binding, I believe, was first implemented by Miles
Bader, 20 or 25 years ago.  Maybe Gerd, maybe Richard made the decision.
Or maybe nobody made the decision, and the conversion became autonomous,
moving ahead with its own momentum, with nobody really in charge.

> Emacs moves forward because someone whom we trust to be an expert in
> some area or to understand well enough what our users might want has
> an itch to scratch, not because we have some magic future insight.
> Hindsight is always 20-20, but it is also unfair, given our
> development model and the pull of core developers and expertise that
> we can command.

I can't remember any discussion of the technical merits and demerits of
lexical binding taking place on this list.  I've been subscribed for
over 20 years.  Maybe I missed it.  It seems strange, that's all.

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Andrea Corallo]

Alan Mackenzie <acm@muc.de> writes:

> Hello, Joost.
>
> On Wed, Nov 06, 2024 at 21:48:05 +0100, Joost Kremers wrote:
>> On Wed, Nov 06 2024, Alan Mackenzie wrote:
>> > Maybe briefly explaining the philosophy of lexical binding somewhere
>> > (maybe NEWS) would help.
>
>> > I still don't understand it.  I go along with lexical binding because
>> > quite a lot of people cleverer than me have said it is good.  But why?
>
>> The Elisp manual has a few short remarks on it:
>
>> (info "(elisp) Lexical Binding")
>
>> ,----
>> | Lexical binding was introduced to Emacs, as an optional feature, in
>> | version 24.1.  We expect its importance to increase with time.  Lexical
>> | binding opens up many more opportunities for optimization, so programs
>> | using it are likely to run faster in future Emacs versions.
>
> This was several major Emacs versions ago.  Has anybody actually done
> any measurements to demonstrate how effective this optimisation has
> been?  When I tried comparing dynamic vs. lexical .elc versions of CC
> Mode scrolling through xdisp.c, dynamic took 8.3247s, lexical took
> 8.3285s.  That's the same, within measurement accuracy.
>
>> | Lexical
>> | binding is also more compatible with concurrency, which was added to
>> | Emacs in version 26.1.
>> `----
>
> I wonder what "more" compatible means.  Concurrency doesn't appear to be
> a major feature in Emacs, as far as I can tell.
>
>> and:
>
>> (info "(elisp) Dynamic Binding")
>
>> ,----
>> |    Note that when code using Dynamic Binding is native compiled the
>> | native compiler will not perform any Lisp specific optimization.
>> `----
>
> This looks like a choice not to support dynamic binding as well as
> lexical binding.  In my CC Mode benchmark (see above) with native
> compiled code, the lexical binding version was somewhat faster than the
> dynamic binding one, around 10%.  But the native compiled code was
> scarcely faster than the byte compiled code.

I'm pretty sure the native compiler could be improved on dynamic binding
support, OTOH dynamic binding code is way less optimizable as many of
the technics the native compiler uses in order to layout efficient code
would not be applicable.

  Andrea

Re: Default lexical-binding to t

[Christopher Howard]

Hi, I just wanted to mention one little thing (feature request?): I noticed that, if I include this

```
;;; ...  -*- lexical-binding: nil -*-
```

in order to purposefully set the lexical-binding to dynamic — I'm a rebel, I guess — then the mode-line still gives the "/d" a warning face. In my theme, that shows up as bright red. But if I set it to "t" instead, then I get a "/l" without a warning color. I don't see why, if I purposely chose dynamic binding, that Emacs should treat it as a bad thing which I need to be warned about.

-- 
Christopher Howard

Re: Default lexical-binding to t

[Dmitry Gutov]

On 07/11/2024 23:23, Alan Mackenzie wrote:
> I can't remember any discussion of the technical merits and demerits of
> lexical binding taking place on this list.  I've been subscribed for
> over 20 years.  Maybe I missed it.  It seems strange, that's all.

I can suggest two points:

- Lexical binding is closer to the how bindings work in contemporary 
languages, or ones that are most popular these days anyway. Which makes 
transition easier, and so helps with writing and  debugging new Elisp.

- The "lexical binding" dialect of Elisp is easier to analyze for the 
byte-compiler. Not just in terms of performance, but to offer warnings 
about potential typos. Such detection won't work with "dynamic binding" 
Elisp: any variable with a weird name can be intended to be bound in the 
caller.

Re: Default lexical-binding to t

[Andrea Corallo]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > I believe by the time Emacs-31 will be released, such files will be
>   > > uncommon, and it is easy to fix them (either by adjusting he code, or
>   > > by slapping a `lexical-binding` cookie).
>
>   > I support this, together with the idea of having a warning when loading
>   > file without the cookie.  We can always remove the warning (and the
>   > cookies) in the future when/if we feel it's time.
>
> If the goal is to lead ysers to indicate this explicitly in each file,
> it is enough to complain about files that lack a file local variable
> specification for this variable.  It could do this whenever it visits
> a file in Emacs Lisp mode.

But a user could one day upgrade Emacs to a new emacs (with
lexical-binding t) and, without any feedback from Emacs, load an old file
with no cookie that was designed for dynamic bindings now interpreted as
lexical.  Most people load files without first fisiting them, so I still
think the load time warning idea has some advantage.

> If we want to increase the pressure to label old files explicitly,
> we could go beyond a warning.  For instance, we could ask the user
> whether. "Is this an old file thjt is supposed to use dynamic binding?"
>
> If the user answers yes to that, Emacs could offer to modify the file,
> adding a tag.

Yep that would be doable too.  I think increasing the pressure to label
old files explicitly is the goal here.

> By contrast, to change the interpretation of files that don't specify
> the variable is ineffective for the purpuse.  But it will waste a lot
> of long-time users' time in figuring out what went wrong.
>
> Eli wrote:
>
>   > If we urge people to specify lexical-binding in their files, then what
>   > is the point of making lexical-binding the default?
>
> Perhaps there is no reason to change the default itself ;-).  Perhaps
> what we _should_ want is to put an end, among somewhat-maintained
> programs, to dependence on the old defaull.
>
> The methods described above will do the job better and faster than
> changing the default, and much more gently too.

I believe having code with no cookie interpreted as lexical is anyway
the (final) goal on this subject.  Given all new code is lexically
scoped, one day would be nice not to have to keep on writing the cookie.
The cookie AFAIU was a workaraund put in place to have the coexistence
of the two dialects possible and allow for the transition, it is/was a
useful tool but would be nice in the future not to be forced to use it
everywhere.

Regards

  Andrea

Re: Default lexical-binding to t

[Andrea Corallo]

Po Lu <luangruo@yahoo.com> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> It seems that people are forming ideas of how frequent dynamic binding
>> files are by looking at code in active development.  Of course it is
>> rare there.  But user have a lot of old Lisp files that they haven't
>> used for a while.  Some of them, someone will try to use someday.
>
> It is far too early even to arrive at this conclusion.  I have said it
> before and I'll say it again.  Even actively developed packages on the
> EmacsWiki are still to be amended with lexical binding directives.
>
>   https://www.emacswiki.org/emacs/info%2b.el
>   (Last updated Mon Jan 29 15:53:30 2024 (-0800))

Drew replied in this thread the following:

> +1.
> 
> FWIW, I have no problem adding a `lexical-binding: nil'
> cookie to my libraries that haven't yet been converted
> to use `lexical-binding: t'.
> 
> (Ultimately I hope to convert all to `lexical-binding:
> t', but in some cases that could take a while.)

Po Lu I'd be curious to know if you have other examples.

Regards

  Andrea

Re: Default lexical-binding to t

[Stefan Kangas]

Christopher Howard <christopher@librehacker.com> writes:

> Hi, I just wanted to mention one little thing (feature request?): I
> noticed that, if I include this
>
> ``` ;;; ...  -*- lexical-binding: nil -*- ```
>
> in order to purposefully set the lexical-binding to dynamic — I'm a
> rebel, I guess — then the mode-line still gives the "/d" a warning
> face. In my theme, that shows up as bright red. But if I set it to "t"
> instead, then I get a "/l" without a warning color. I don't see why,
> if I purposely chose dynamic binding, that Emacs should treat it as a
> bad thing which I need to be warned about.

Yes, that is on purpose.  See (info "(elisp) Coding Conventions"):

   • We recommend enabling ‘lexical-binding’ in new code, and converting
     existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
     already.  *Note Selecting Lisp Dialect::.

Re: Default lexical-binding to t

[Stefan Kangas]

Alan Mackenzie <acm@muc.de> writes:

>> Consider a simple example like this:
>
>>     (defun another-function ()
>>       (setq x 3))
>
>>     (defun foo ()
>>       (let ((x 1) (y 2))
>>         (another-function)
>>         (+ x y)))
>
>> This code is clearly bad, but regardless of that we get with dynbind,
>
>>     (foo) => 5
>
>> whereas with lexbind,
>
>>     (foo) => 3
>
> Yes.  Both results are clearly correct.

The question is not if Emacs correctly implements the dynbind and
lexbind semantics, but which of the two is more desirable.

In my view, lexbind provides the superior semantics for let-binding
non-special variables, assuming that we care about program correctness.
It completely eliminates an entire class of bugs.

You could perhaps argue that it's just up to the programmer to check
which variables are used in every function they call (and the functions
they call in turn, etc.), but that quickly gets impractical.  You could
also argue that rarely does this become an issue in practice, and that
might be true even.  It's also rare that people get killed in car
accidents, yet we still try to make roads and cars as safe as possible.

Consider also which semantics are better if we were to add concurrency
into the mix.  Having two concurrent processes competing about the same
`x` variable is not my idea of a good time.

>> Now, it's also true that theory tells us that lexbind should indeed
>> often lead to better performance, since the compiler can do
>> optimizations on such code that are hard or impractical with dynbind.
>
> But the overhead of having to construct closures for internal functions
> tends to counter other optimisations.

That might be true, but luckily we don't need to create closures for
internal functions everywhere.

BTW, if you see problems with lexbind in debugging, then I recommend
opening bug reports so we can discuss how to improve things.

Re: Default lexical-binding to t

[tomas]

[-- Attachment #1: Type: text/plain, Size: 924 bytes --]

On Fri, Nov 08, 2024 at 12:38:08AM +0000, Stefan Kangas wrote:
> Christopher Howard <christopher@librehacker.com> writes:
> 
> > Hi, I just wanted to mention one little thing (feature request?): I
> > noticed that, if I include this
> >
> > ``` ;;; ...  -*- lexical-binding: nil -*- ```
> >
> > in order to purposefully set the lexical-binding to dynamic — I'm a
> > rebel, I guess — then the mode-line still gives the "/d" a warning
> > face. In my theme, that shows up as bright red. But if I set it to "t"
> > instead, then I get a "/l" without a warning color. I don't see why,
> > if I purposely chose dynamic binding, that Emacs should treat it as a
> > bad thing which I need to be warned about.
> 
> Yes, that is on purpose.  See (info "(elisp) Coding Conventions"):

That sounds counterproductive, IMO. If the user intentionally choses dynamic
binding, I'd rather not warn.

Cheers
-- 
t

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: Default lexical-binding to t

[Eli Zaretskii]

> Date: Thu, 7 Nov 2024 21:23:38 +0000
> Cc: joostkremers@fastmail.fm, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > > But I do wonder whether the massive time and effort its development
> > > and proliferation have taken up were worth it.
> 
> > This could be said about almost every major feature in Emacs that
> > fundamentally changes the internals.  Are the time, energy, and
> > massive efforts I invested (and am still investing) in bidirectional
> > editing support worth it?  I don't know; there are days when I think
> > it was all a terrible waste.  What about all the multiple changes in
> > the internals that bring us 3.7% speedup in some benchmark?  What
> > about support for Lisp threads?  What about the addition of positions
> > to symbols? in how many error-message situations is this really
> > important?  And XInput2, and touch devices, etc. etc.
> 
> No, the difference is that most major features are implemented by one
> person (or a small team) and are complete in themselves.  Lexical
> binding involved significant effort from lots and lots of contributors,
> each having to modify "his own" files as lexical binding steadily
> morphed from being optional to being compulsory.

That is factually not true.  You seem to compare the contributions to
lexical binding and to the other features with different scales, or
maybe you simply misremember.  Lexical binding is mainly the work of
Stefan, exactly like bidi support is mainly mine.  But in the same
way, many others contributed in both.

As for the need to adapt to changes: it is also common.  My init files
are quite stable, and still I have in them parts that depend on Emacs
versions, and each new major release ends up with additions and
modifications there.  There's nothing surprising here: new releases
bring new variables and features, and OTOH change some defaults which
I don't like, so I need to adapt.  And so do others.

> I still wonder who made the decision to convert the entire code base,
> and when.

We all did.  If you are against this, you are in a tiny minority, and
should probably stop fighting this war, because it was lost long ago.
Trying to argue now against lexical binding is basically a waste of
everyone's time, yours included.  So please let's not go there,
because this issue is not on the table, not for a very long time.

> > Emacs moves forward because someone whom we trust to be an expert in
> > some area or to understand well enough what our users might want has
> > an itch to scratch, not because we have some magic future insight.
> > Hindsight is always 20-20, but it is also unfair, given our
> > development model and the pull of core developers and expertise that
> > we can command.
> 
> I can't remember any discussion of the technical merits and demerits of
> lexical binding taking place on this list.  I've been subscribed for
> over 20 years.  Maybe I missed it.  It seems strange, that's all.

If you don't remember such discussions, then you missed them, yes.  We
had quite a lot of them over the years.

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Christopher Howard <christopher@librehacker.com>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  Joost Kremers
>  <joostkremers@fastmail.fm>,  emacs-devel@gnu.org
> Date: Thu, 07 Nov 2024 13:28:12 -0900
> 
> Hi, I just wanted to mention one little thing (feature request?): I noticed that, if I include this
> 
> ```
> ;;; ...  -*- lexical-binding: nil -*-
> ```
> 
> in order to purposefully set the lexical-binding to dynamic — I'm a rebel, I guess — then the mode-line still gives the "/d" a warning face. In my theme, that shows up as bright red. But if I set it to "t" instead, then I get a "/l" without a warning color. I don't see why, if I purposely chose dynamic binding, that Emacs should treat it as a bad thing which I need to be warned about.

Thanks, please submit a bug report about this, so that we won't forget
it.

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 8 Nov 2024 00:38:08 +0000
> Cc: Joost Kremers <joostkremers@fastmail.fm>, emacs-devel@gnu.org
> 
> Christopher Howard <christopher@librehacker.com> writes:
> 
> > Hi, I just wanted to mention one little thing (feature request?): I
> > noticed that, if I include this
> >
> > ``` ;;; ...  -*- lexical-binding: nil -*- ```
> >
> > in order to purposefully set the lexical-binding to dynamic — I'm a
> > rebel, I guess — then the mode-line still gives the "/d" a warning
> > face. In my theme, that shows up as bright red. But if I set it to "t"
> > instead, then I get a "/l" without a warning color. I don't see why,
> > if I purposely chose dynamic binding, that Emacs should treat it as a
> > bad thing which I need to be warned about.
> 
> Yes, that is on purpose.  See (info "(elisp) Coding Conventions"):
> 
>    • We recommend enabling ‘lexical-binding’ in new code, and converting
>      existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
>      already.  *Note Selecting Lisp Dialect::.

I don't think this justifies nagging a user who explicitly wants some
file(s) to be treated as using dynamic binding.  The above is just a
recommendation, not a hard requirement.  We are not going to remove
dynamic binding from Emacs, are we?  Then why annoy people who want to
use it, for presumably good reasons?

Re: Default lexical-binding to t

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Kangas <stefankangas@gmail.com>
>> Date: Fri, 8 Nov 2024 00:38:08 +0000
>> Cc: Joost Kremers <joostkremers@fastmail.fm>, emacs-devel@gnu.org
>>
>> Christopher Howard <christopher@librehacker.com> writes:
>>
>> > Hi, I just wanted to mention one little thing (feature request?): I
>> > noticed that, if I include this
>> >
>> > ``` ;;; ...  -*- lexical-binding: nil -*- ```
>> >
>> > in order to purposefully set the lexical-binding to dynamic — I'm a
>> > rebel, I guess — then the mode-line still gives the "/d" a warning
>> > face. In my theme, that shows up as bright red. But if I set it to "t"
>> > instead, then I get a "/l" without a warning color. I don't see why,
>> > if I purposely chose dynamic binding, that Emacs should treat it as a
>> > bad thing which I need to be warned about.
>>
>> Yes, that is on purpose.  See (info "(elisp) Coding Conventions"):
>>
>>    • We recommend enabling ‘lexical-binding’ in new code, and converting
>>      existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
>>      already.  *Note Selecting Lisp Dialect::.
>
> I don't think this justifies nagging a user who explicitly wants some
> file(s) to be treated as using dynamic binding.  The above is just a
> recommendation, not a hard requirement.

I don't consider this as much more than a gentle nudge and reminder.
The mode line indicator has been with us for a few releases already, and
I personally don't see compelling reasons to change that decision now.

Re: Default lexical-binding to t

[Suhail Singh]

Stefan Kangas <stefankangas@gmail.com> writes:

>>>    • We recommend enabling ‘lexical-binding’ in new code, and converting
>>>      existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
>>>      already.  *Note Selecting Lisp Dialect::.
>>
>> I don't think this justifies nagging a user who explicitly wants some
>> file(s) to be treated as using dynamic binding.  The above is just a
>> recommendation, not a hard requirement.
>
> I don't consider this as much more than a gentle nudge and reminder.
> The mode line indicator has been with us for a few releases already, and
> I personally don't see compelling reasons to change that decision now.

If the goal, eventually, is to have lexical binding as the default, then
it would help to have all code that uses dynamic binding to (eventually)
use an explicitly set the lexical-binding buffer-local variable.  To
that end, while it makes sense to issue a warning for implicit uses of
dynamic binding currently, it is unclear why explicit uses should also
warrant a warning.

I think it would help to distinguish (in the mode line) whether the
dynamic binding is being explicitly requested (which will continue to be
supported in the future) vs when the dynamic binding is the result of a
current default (one that will eventually be changing).  Having a
warning face apply _only_ in the latter situation would be a reasonable
approach.

-- 
Suhail

Re: Default lexical-binding to t

[Stefan Kangas]

Suhail Singh <suhailsingh247@gmail.com> writes:

> Stefan Kangas <stefankangas@gmail.com> writes:
>
>>>>    • We recommend enabling ‘lexical-binding’ in new code, and converting
>>>>      existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
>>>>      already.  *Note Selecting Lisp Dialect::.
>>>
>>> I don't think this justifies nagging a user who explicitly wants some
>>> file(s) to be treated as using dynamic binding.  The above is just a
>>> recommendation, not a hard requirement.
>>
>> I don't consider this as much more than a gentle nudge and reminder.
>> The mode line indicator has been with us for a few releases already, and
>> I personally don't see compelling reasons to change that decision now.
>
> If the goal, eventually, is to have lexical binding as the default, then
> it would help to have all code that uses dynamic binding to (eventually)
> use an explicitly set the lexical-binding buffer-local variable.  To
> that end, while it makes sense to issue a warning for implicit uses of
> dynamic binding currently, it is unclear why explicit uses should also
> warrant a warning.

Sorry, I misunderstood.

I'm fine with removing the warning for _explicit_ uses of dynbind.

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Fri, 8 Nov 2024 12:11:34 +0000
> Cc: christopher@librehacker.com, acm@muc.de, joostkremers@fastmail.fm, 
> 	emacs-devel@gnu.org
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> From: Stefan Kangas <stefankangas@gmail.com>
> >> Date: Fri, 8 Nov 2024 00:38:08 +0000
> >> Cc: Joost Kremers <joostkremers@fastmail.fm>, emacs-devel@gnu.org
> >>
> >> Christopher Howard <christopher@librehacker.com> writes:
> >>
> >> > Hi, I just wanted to mention one little thing (feature request?): I
> >> > noticed that, if I include this
> >> >
> >> > ``` ;;; ...  -*- lexical-binding: nil -*- ```
> >> >
> >> > in order to purposefully set the lexical-binding to dynamic — I'm a
> >> > rebel, I guess — then the mode-line still gives the "/d" a warning
> >> > face. In my theme, that shows up as bright red. But if I set it to "t"
> >> > instead, then I get a "/l" without a warning color. I don't see why,
> >> > if I purposely chose dynamic binding, that Emacs should treat it as a
> >> > bad thing which I need to be warned about.
> >>
> >> Yes, that is on purpose.  See (info "(elisp) Coding Conventions"):
> >>
> >>    • We recommend enabling ‘lexical-binding’ in new code, and converting
> >>      existing Emacs Lisp code to enable ‘lexical-binding’ if it doesn't
> >>      already.  *Note Selecting Lisp Dialect::.
> >
> > I don't think this justifies nagging a user who explicitly wants some
> > file(s) to be treated as using dynamic binding.  The above is just a
> > recommendation, not a hard requirement.
> 
> I don't consider this as much more than a gentle nudge and reminder.

Why is there a need to nudge someone who already took the necessary
action?

> The mode line indicator has been with us for a few releases already, and
> I personally don't see compelling reasons to change that decision now.

We could remove the warning face from the indication if the setting is
explicit.  Would that be a problem?

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Eli.

On Fri, Nov 08, 2024 at 08:58:40 +0200, Eli Zaretskii wrote:
> > Date: Thu, 7 Nov 2024 21:23:38 +0000
> > Cc: joostkremers@fastmail.fm, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > > But I do wonder whether the massive time and effort its development
> > > > and proliferation have taken up were worth it.

> > > This could be said about almost every major feature in Emacs that
> > > fundamentally changes the internals.  Are the time, energy, and
> > > massive efforts I invested (and am still investing) in bidirectional
> > > editing support worth it?  I don't know; there are days when I think
> > > it was all a terrible waste.  What about all the multiple changes in
> > > the internals that bring us 3.7% speedup in some benchmark?  What
> > > about support for Lisp threads?  What about the addition of positions
> > > to symbols? in how many error-message situations is this really
> > > important?  And XInput2, and touch devices, etc. etc.

> > No, the difference is that most major features are implemented by one
> > person (or a small team) and are complete in themselves.  Lexical
> > binding involved significant effort from lots and lots of contributors,
> > each having to modify "his own" files as lexical binding steadily
> > morphed from being optional to being compulsory.

> That is factually not true.  You seem to compare the contributions to
> lexical binding and to the other features with different scales, or
> maybe you simply misremember.

They are different.  Most features, such as bidi, are self-contained -
they are very useful for some, or many, users, but they don't upset the
fundaments of Emacs.  Lexical binding was different.  Essentially every
..el file in Emacs had to be modified because of it.

[ .... ]

> > I still wonder who made the decision to convert the entire code base
> > [to lexical binding], and when.

> We all did.

When?  And how?

> If you are against this, you are in a tiny minority, and should
> probably stop fighting this war, because it was lost long ago.

I'm not.  I'm against the process, or lack of process, by which it was
introduced.

I don't think there ever was an explicit decision taken - a thread on
emacs-devel, which ended up with something such as:

    "I think we're all agreed that the advantages of lexical binding
    outweigh the disadvantages, and that from now on we'll work towards
    converting the entire Emacs code base."

..  If there had been such an explicit decision process, we might have
ended up with a better result.  I'm thinking here of the discarding of
functions' parameter lists in .elc files, and their replacement by
numbers coding up some of that information.  For example 514 meaning
exactly two parameters.

[ .... ]

> > I can't remember any discussion of the technical merits and demerits of
> > lexical binding taking place on this list.  I've been subscribed for
> > over 20 years.  Maybe I missed it.  It seems strange, that's all.

> If you don't remember such discussions, then you missed them, yes.  We
> had quite a lot of them over the years.

Yes, there have been some, even some initiated by me, over the past few
years.  But none in the context of deciding whether or not to introduce
lexical binding.  [This is what I meant to say in my last paragraph, but
failed.]

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Eli Zaretskii]

> Date: Fri, 8 Nov 2024 14:01:30 +0000
> Cc: joostkremers@fastmail.fm, emacs-devel@gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> > If you are against this, you are in a tiny minority, and should
> > probably stop fighting this war, because it was lost long ago.
> 
> I'm not.  I'm against the process, or lack of process, by which it was
> introduced.

This "lack of process" is our process.

> I don't think there ever was an explicit decision taken - a thread on
> emacs-devel, which ended up with something such as:
> 
>     "I think we're all agreed that the advantages of lexical binding
>     outweigh the disadvantages, and that from now on we'll work towards
>     converting the entire Emacs code base."

This isn't a parliament.  We discuss stuff, and then whoever is
motivated to do the job does it as well as he/she can, taking the
discussions into consideration.

Take this last discussion about reported nested loads as part of an
error message, and consider your own behavior there.  This is how
everything works here, always had, like it or not.

> Yes, there have been some, even some initiated by me, over the past few
> years.  But none in the context of deciding whether or not to introduce
> lexical binding.

That's because no one was really against it, believe it or not.

Re: Default lexical-binding to t

[Sebastián Monía]

Andrea Corallo <acorallo@gnu.org> writes:
>
> Most people load files without first fisiting them, so I still
> think the load time warning idea has some advantage.

They _really_ shouldn't do that, though... :)

-- 
Sebastián Monía
https://site.sebasmonia.com/

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Sebastián Monía <sebastian@sebasmonia.com>
> Cc: Richard Stallman <rms@gnu.org>,  emacs-devel@gnu.org
> Date: Fri, 08 Nov 2024 10:53:23 -0500
> 
> Andrea Corallo <acorallo@gnu.org> writes:
> >
> > Most people load files without first fisiting them, so I still
> > think the load time warning idea has some advantage.
> 
> They _really_ shouldn't do that, though... :)

Why not?

Re: Default lexical-binding to t

[Sebastián Monía]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Sebastián Monía <sebastian@sebasmonia.com>
>> Cc: Richard Stallman <rms@gnu.org>,  emacs-devel@gnu.org
>> Date: Fri, 08 Nov 2024 10:53:23 -0500
>> 
>> Andrea Corallo <acorallo@gnu.org> writes:
>> >
>> > Most people load files without first fisiting them, so I still
>> > think the load time warning idea has some advantage.
>> 
>> They _really_ shouldn't do that, though... :)
>
> Why not?

Isn't it general considered a bad idea to run arbitrary code without
taking a look first?


-- 
Sebastián Monía
https://site.sebasmonia.com/

Re: Default lexical-binding to t

[Joost Kremers]

On Fri, Nov 08 2024, Sebastián Monía wrote:
> Isn't it general considered a bad idea to run arbitrary code without
> taking a look first?

I currently have almost 140 Emacs packages installed. Heck, my Linux
install consists of around 1650 packages. Oh, and of course the Python
packages in every virtual environment I create... I'm not going to check
all their sources first before installing them. 😉

-- 
Joost Kremers
Life has its moments

Re: Default lexical-binding to t

[Stefan Monnier]

>> |    Note that when code using Dynamic Binding is native compiled the
>> | native compiler will not perform any Lisp specific optimization.
> This looks like a choice not to support dynamic binding as well as
> lexical binding.

As Andrea mentions, it's not quite so simple: by its very nature,
dynamic scoping makes static understanding of the code's behavior
is harder.  To take an example from, my PL course: 🙂

    (let ((x 3)) (+ (f y) x))

with static scoping, the compiler can trivially rewrite this:

    (let ((x 3)) (+ (f y) x))

=> (constant propagation)

    (let ((x 3)) (+ (f y) 3))

=> (dead variable elimination)

    (+ (f y) 3)

but with dynamic scoping, both steps require the compiler to convince
itself that no code reachable from `f` can refer to or modify `x`.
Our compiler is very far from performing enough code analysis to know
anything about `f` when we compile the above code, so it gives up on the
optimization right away.

Of course, this doesn't affect only compilers but also humans.

It's part of the reason why static scoping is used by virtually all
programming languages, whereas dynamic scoping is supported only by
a select few programming languages and is virtually never the default.


        Stefan

Re: Default lexical-binding to t

[Stefan Monnier]

> Apologies in advance if this has been suggested and discarded before, or
> if I am missing something obvious, but wouldn't it be possible to have
> an additional level of indirection with a new variable,
> default-lexical-binding, settable in, say, early-init.el?  It's not a
> real solution, but at least people with lots of trouble with the new
> default could just set it off in one place.

The patch I submitted in the relevant bugreport allows that.
I'm not sure how much we want to advertise it, tho.  I was thinking of
keeping it only as a kind of testing/debugging tool, a bit like
`bidi-display-reordering`.


        Stefan

Re: Default lexical-binding to t

[Stefan Monnier]

> Being fundamentally a programmer in languages that support only lexical
> binding, I have little sense of the mechanics underlying dynamic binding.
> That said, I wonder if something along these line could help a bit:
>
> If a "void variable" error is about to be thrown and the source defaulted
> to dynamic binding then
>
>    - Perform a dynamic lookup of the offending variable
>
>
>    - If the lookup is unsuccessful then issue the same error as always
>
>
>    - If the lookup is successful, then modify the diagnostic to mention
>    the possibility of a dynamic / lexical binding clash

That's already what happens.


        Stefan

Re: Default lexical-binding to t

[Eli Zaretskii]

> From: Sebastián Monía <sebastian@sebasmonia.com>
> Cc: acorallo@gnu.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Fri, 08 Nov 2024 12:28:55 -0500
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> >> From: Sebastián Monía <sebastian@sebasmonia.com>
> >> Cc: Richard Stallman <rms@gnu.org>,  emacs-devel@gnu.org
> >> Date: Fri, 08 Nov 2024 10:53:23 -0500
> >> 
> >> Andrea Corallo <acorallo@gnu.org> writes:
> >> >
> >> > Most people load files without first fisiting them, so I still
> >> > think the load time warning idea has some advantage.
> >> 
> >> They _really_ shouldn't do that, though... :)
> >
> > Why not?
> 
> Isn't it general considered a bad idea to run arbitrary code without
> taking a look first?

Don't you do that with all the Lisp files that come with Emacs?

Re: Default lexical-binding to t

[Alan Mackenzie]

Hello, Eli.

On Fri, Nov 08, 2024 at 17:19:57 +0200, Eli Zaretskii wrote:
> > Date: Fri, 8 Nov 2024 14:01:30 +0000
> > Cc: joostkremers@fastmail.fm, emacs-devel@gnu.org
> > From: Alan Mackenzie <acm@muc.de>

> > > If you are against this, you are in a tiny minority, and should
> > > probably stop fighting this war, because it was lost long ago.

> > I'm not.  I'm against the process, or lack of process, by which it was
> > introduced.

> This "lack of process" is our process.

It's normal, even here, when introducing possibly breaking changes to
announce them and deal with objections/bugs/incompatibilities.  For
example, you did this on introducing line numbers in the display engine.
This worked very well, in that several bugs got fixed before landing it
on master.

Maybe this did happen with lexical binding.  If it did, I can't find it.

> > I don't think there ever was an explicit decision taken - a thread on
> > emacs-devel, which ended up with something such as:

> >     "I think we're all agreed that the advantages of lexical binding
> >     outweigh the disadvantages, and that from now on we'll work towards
> >     converting the entire Emacs code base."

> This isn't a parliament.  We discuss stuff, and then whoever is
> motivated to do the job does it as well as he/she can, taking the
> discussions into consideration.

> Take this last discussion about reported nested loads as part of an
> error message, and consider your own behavior there.  This is how
> everything works here, always had, like it or not.

> > Yes, there have been some, even some initiated by me, over the past few
> > years.  But none in the context of deciding whether or not to introduce
> > lexical binding.

> That's because no one was really against it, believe it or not.

OK.  But surely somebody, somewhere, had some reservations?

-- 
Alan Mackenzie (Nuremberg, Germany).

Re: Default lexical-binding to t

[Stefan Monnier]

> It's normal, even here, when introducing possibly breaking changes to
> announce them and deal with objections/bugs/incompatibilities.  For
> example, you did this on introducing line numbers in the display engine.
> This worked very well, in that several bugs got fixed before landing it
> on master.
>
> Maybe this did happen with lexical binding.  If it did, I can't find it.

In our "Evolution of Emacs Lisp" paper we do mention some of the steps
along the way.  Lexical binding in Emacs Lisp dates back to the
"rewrite" of CL back in Emacs-18 which added `lexical-let` and various
other options were discussed, attempted, etc...

Not sure how that's the relevant to the current discussion:

> OK.  But surely somebody, somewhere, had some reservations?

No doubt.  But the other choices just sucked even more.


        Stefan

Re: Default lexical-binding to t

[Sebastián Monía]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Sebastián Monía <sebastian@sebasmonia.com>
>> Eli Zaretskii <eliz@gnu.org> writes:
>> >> From: Sebastián Monía <sebastian@sebasmonia.com>
>> >> Andrea Corallo <acorallo@gnu.org> writes:
>> >> >
>> >> > Most people load files without first fisiting them, so I still
>> >> > think the load time warning idea has some advantage.
>> >> 
>> >> They _really_ shouldn't do that, though... :)
>> >
>> > Why not?
>> 
>> Isn't it general considered a bad idea to run arbitrary code without
>> taking a look first?
>
> Don't you do that with all the Lisp files that come with Emacs?

Yes! but in this context I understood Andrea meant "loading some random
lone file", which is different from a vetted (or at least, well known)
package, and very different from code included with Emacs.

If some file has no explicit binding cookie, it is relativaly old, maybe
even unmaintaned. No? Since using explicit lexical binding has been the
recommendation for a while now.


-- 
Sebastián Monía
https://site.sebasmonia.com/

RE: [External] : Re: Default lexical-binding to t

[Drew Adams]

> If some file has no explicit binding cookie, it
> is relativaly old, maybe even unmaintaned. No? 

No.

And even a new/maintained file might reasonably
have no such cookie - especially one with `nil'.

> Since using explicit lexical binding has been
> the recommendation for a while now.

The current state is still, and has long been,
that no cookie means dynamic binding by default.
Currently there's little reason for a file that
wants dynamic binding to add "lexical-binding:nil".

It doesn't hurt for such a file to add that now,
but there's no great reason to do so yet.

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Lexical binding makes debugging more difficult.  For example, the
  > compilation process discards the names of parameters to functions.

C uses lexical binding, and yet GCC manages to say where to find the
function's local variables.  Even in an optimized compilation,
it does that pretty well.

If the byte compiler doesn't do such a good job, how about fixing that?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > IMO. If the user intentionally choses dynamic
  > binding, I'd rather not warn.

If dynamic binding mode someday becomes so unusual that users get
confused when they encounter it, or may not know about it at all,
it would be useful to warn the user in a special way
when the file specifies dynamic binding.

In tis case, Emacs should not nag the user by offering to change the
file.  What is useful is making the fact of dynamic binding in that
file more visible to the user.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Default lexical-binding to t

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >> > Most people load files without first fisiting them, so I still
  > >> > think the load time warning idea has some advantage.
  > >> 
  > >> They _really_ shouldn't do that, though... :)
  > >
  > > Why not?

  > Isn't it general considered a bad idea to run arbitrary code without
  > taking a look first?

If it is on your laptop, you could be quite sure no one else
has changed it since you last worked on it, 10 years ago.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)