bug#66706: [PATCH] Automatic elisp dialect insertion

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 257 bytes --]

This patch inserts the lexical cookie in new Elisp files automatically.
It helps users by making it less likely that they forget to add it, and eliminates some drudgery.
Their code will be more future-safe, and more robust and performant here and now.


[-- Attachment #2: 0001-Automatic-Elisp-dialect-declaration-insertion.patch --]
[-- Type: application/octet-stream, Size: 3948 bytes --]

From 1f046584a58da54b32b6be8c17d00c990453a177 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Sun, 22 Oct 2023 16:25:28 +0200
Subject: [PATCH] Automatic Elisp dialect declaration insertion

Insert the `;;; -*- lexical-binding:t -*-` cookie and set
`lexical-binding` to `t` when the user visits a new Elisp file, unless
this feature is disabled.  `auto-insert-mode` takes precedence.

* etc/NEWS: Announce.
* lisp/progmodes/elisp-mode.el (emacs-lisp-mode): Add hook.
(elisp-auto-dialect-declaration): New defcustom.
(elisp--insert-auto-dialect-declaration): New function.
---
 etc/NEWS                     | 11 +++++++++++
 lisp/progmodes/elisp-mode.el | 35 ++++++++++++++++++++++++++++++++++-
 2 files changed, 45 insertions(+), 1 deletion(-)

diff --git a/etc/NEWS b/etc/NEWS
index d0880669752..b87f51a134c 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -797,6 +797,17 @@ This argument specifies the prettifying algorithm to use.
 
 ** Emacs Lisp mode
 
+*** Automatic Elisp dialect declaration insertion.
+When visiting a new Elisp file, Emacs will now automatically insert a
+';;; -*- lexical-binding: t -*-' line to declare the modern Elisp
+lexical-binding dialect, and set the 'lexical-binding' variable
+in the buffer to 't'.
+
+This mechanism is controlled by the new 'elisp-auto-dialect-declaration'
+user option.  It will only insert a declaration into an empty buffer:
+if the buffer already had text added by means of 'auto-insert-mode'
+then it will not do anything.
+
 ---
 *** ',@' now has 'prefix' syntax.
 Previously, the '@' character, which normally has 'symbol' syntax,
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ff90a744ea3..8ac32ecd077 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -363,7 +363,40 @@ emacs-lisp-mode
   (add-hook 'flymake-diagnostic-functions #'elisp-flymake-checkdoc nil t)
   (add-hook 'flymake-diagnostic-functions
               #'elisp-flymake-byte-compile nil t)
-  (add-hook 'context-menu-functions #'elisp-context-menu 10 t))
+  (add-hook 'context-menu-functions #'elisp-context-menu 10 t)
+  ;; Add this hook sufficient late to give other hooks (like `auto-insert')
+  ;; the opportunity to insert something with higher priority.
+  (add-hook 'find-file-hook #'elisp--insert-auto-dialect-declaration 50 t))
+
+(defcustom elisp-auto-dialect-declaration 'lexical
+  "Dialect declaration automatically inserted in new Elisp buffers.
+The declaration is \";;; -*- lexical-binding: t -*-\".
+It is only inserted when an empty non-existing file is visited.
+Possible values are:
+  `lexical'  declare use of the modern lexical binding dialect.
+  `nil'      do not automatically insert any declaration.
+
+If `auto-insert-mode' is used to put something in the buffer instead,
+then no declaration is inserted."
+  :type '(choice (const :tag "Lexical binding (modern)" lexical)
+                 (const :tag "No automatic declaration" nil))
+  :group 'lisp
+  :version "30.1")
+
+(defun elisp--insert-auto-dialect-declaration ()
+  "Insert the `elisp-auto-dialect-declaration' selection in a new empty buffer.
+Otherwise, do nothing."
+  (when (and (eq elisp-auto-dialect-declaration 'lexical)
+             (not buffer-read-only)
+             (zerop (buffer-size))
+             ;; Don't modify a buffer corresponding to an existing empty file.
+             (not (and buffer-file-name (file-exists-p buffer-file-name))))
+    (let ((was-modified (buffer-modified-p)))
+      (insert ";;; -*- lexical-binding: t -*-\n")
+      (setq-local lexical-binding t)
+      ;; Mark the buffer unmodified (unless it was modified before)
+      ;; so that the user isn't bothered when killing it or quitting Emacs.
+      (set-buffer-modified-p was-modified))))
 
 ;; Font-locking support.
 
-- 
2.32.0 (Apple Git-132)

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> This patch inserts the lexical cookie in new Elisp files
> automatically.  It helps users by making it less likely that they
> forget to add it, and eliminates some drudgery.

Yes, please,

> Their code will be more future-safe, and more robust and performant here and now.

I wouldn't make any claim about "performant": there are important cases
where we execute lexically scoped code more efficiently, but there are
also cases (hopefully less important) where the reverse is true.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Mon, 23 Oct 2023 19:46:14 +0200
> 
> This patch inserts the lexical cookie in new Elisp files automatically.
> It helps users by making it less likely that they forget to add it, and eliminates some drudgery.
> Their code will be more future-safe, and more robust and performant here and now.

This could be an optional feature, but not the default.  It is
un-Emacs'y to insert stuff into user files or buffers without an
explicit consent.  We have several auto-insert features in Emacs, and
they are all optional, so this one must be optional as well.

Thanks.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

>> This patch inserts the lexical cookie in new Elisp files automatically.
>> It helps users by making it less likely that they forget to add it, and eliminates some drudgery.
>> Their code will be more future-safe, and more robust and performant here and now.

Sounds good to me.

> This could be an optional feature, but not the default.  It is
> un-Emacs'y to insert stuff into user files or buffers without an
> explicit consent.  We have several auto-insert features in Emacs, and
> they are all optional, so this one must be optional as well.

Yeah, it's not something we usually do.  But we also don't change Emacs
Lisp in such a fundamental way very often (once every 40 years, give or
take?).  So I think veering from what we usually do is justified.

I also note that this can only help and will never hurt users.

If anyone happens to hate this, they can easily disable it.  Whereas, on
the other hand, making the feature optional makes it much less useful.

So my vote would be in favor of this change.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

23 okt. 2023 kl. 21.21 skrev Stefan Kangas <stefankangas@gmail.com>:

> Yeah, it's not something we usually do.  But we also don't change Emacs
> Lisp in such a fundamental way very often (once every 40 years, give or
> take?).  So I think veering from what we usually do is justified.

It's actually a rather timid feature as language modes go. They are supposed to act in their users' best interests, after all.

> I also note that this can only help and will never hurt users.

Right. This always informs our designs.

The feature is carefully designed not to get in the way. It is not opinionated: it does not add any other header, footer, copyright text, preamble or anything else.

Not including the cookie at all is the odd choice, and it is not unreasonable having to type C-p C-k in that rare case.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

[-- Attachment #1: Type: text/plain, Size: 168 bytes --]

Here is an updated patch, now with a test.

Not sure if this belongs in the manual, but if so it's probably in the Elisp manual ('Selecting Lisp Dialect'), right?


[-- Attachment #2: 0001-Automatic-Elisp-dialect-declaration-insertion.patch --]
[-- Type: application/octet-stream, Size: 5661 bytes --]

From 66270adb8cd546f4839fa6a476312063e01b7fa0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Mattias=20Engdeg=C3=A5rd?= <mattiase@acm.org>
Date: Sun, 22 Oct 2023 16:25:28 +0200
Subject: [PATCH] Automatic Elisp dialect declaration insertion

Insert the `;;; -*- lexical-binding:t -*-` cookie and set
`lexical-binding` to `t` when the user visits a new Elisp file, unless
this feature is disabled.  `auto-insert-mode` takes precedence.

* etc/NEWS: Announce.
* lisp/progmodes/elisp-mode.el (emacs-lisp-mode): Add hook.
(elisp-auto-dialect-declaration): New defcustom.
(elisp--insert-auto-dialect-declaration): New function.
* test/lisp/progmodes/elisp-mode-tests.el
(elisp-auto-dialect-declaration): New test.
---
 etc/NEWS                                | 11 ++++++++
 lisp/progmodes/elisp-mode.el            | 35 ++++++++++++++++++++++++-
 test/lisp/progmodes/elisp-mode-tests.el | 25 ++++++++++++++++++
 3 files changed, 70 insertions(+), 1 deletion(-)

diff --git a/etc/NEWS b/etc/NEWS
index 8becfae7bb9..d09abd42313 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -804,6 +804,17 @@ This argument specifies the prettifying algorithm to use.
 
 ** Emacs Lisp mode
 
+*** Automatic Elisp dialect declaration insertion.
+When visiting a new Elisp file, Emacs will now automatically insert a
+';;; -*- lexical-binding: t -*-' line to declare the modern Elisp
+lexical-binding dialect, and set the 'lexical-binding' variable
+in the buffer to 't'.
+
+This mechanism is controlled by the new 'elisp-auto-dialect-declaration'
+user option.  It will only insert a declaration into an empty buffer:
+if the buffer already had text added by means of 'auto-insert-mode'
+then it will not do anything.
+
 ---
 *** ',@' now has 'prefix' syntax.
 Previously, the '@' character, which normally has 'symbol' syntax,
diff --git a/lisp/progmodes/elisp-mode.el b/lisp/progmodes/elisp-mode.el
index ff90a744ea3..8ac32ecd077 100644
--- a/lisp/progmodes/elisp-mode.el
+++ b/lisp/progmodes/elisp-mode.el
@@ -363,7 +363,40 @@ emacs-lisp-mode
   (add-hook 'flymake-diagnostic-functions #'elisp-flymake-checkdoc nil t)
   (add-hook 'flymake-diagnostic-functions
               #'elisp-flymake-byte-compile nil t)
-  (add-hook 'context-menu-functions #'elisp-context-menu 10 t))
+  (add-hook 'context-menu-functions #'elisp-context-menu 10 t)
+  ;; Add this hook sufficient late to give other hooks (like `auto-insert')
+  ;; the opportunity to insert something with higher priority.
+  (add-hook 'find-file-hook #'elisp--insert-auto-dialect-declaration 50 t))
+
+(defcustom elisp-auto-dialect-declaration 'lexical
+  "Dialect declaration automatically inserted in new Elisp buffers.
+The declaration is \";;; -*- lexical-binding: t -*-\".
+It is only inserted when an empty non-existing file is visited.
+Possible values are:
+  `lexical'  declare use of the modern lexical binding dialect.
+  `nil'      do not automatically insert any declaration.
+
+If `auto-insert-mode' is used to put something in the buffer instead,
+then no declaration is inserted."
+  :type '(choice (const :tag "Lexical binding (modern)" lexical)
+                 (const :tag "No automatic declaration" nil))
+  :group 'lisp
+  :version "30.1")
+
+(defun elisp--insert-auto-dialect-declaration ()
+  "Insert the `elisp-auto-dialect-declaration' selection in a new empty buffer.
+Otherwise, do nothing."
+  (when (and (eq elisp-auto-dialect-declaration 'lexical)
+             (not buffer-read-only)
+             (zerop (buffer-size))
+             ;; Don't modify a buffer corresponding to an existing empty file.
+             (not (and buffer-file-name (file-exists-p buffer-file-name))))
+    (let ((was-modified (buffer-modified-p)))
+      (insert ";;; -*- lexical-binding: t -*-\n")
+      (setq-local lexical-binding t)
+      ;; Mark the buffer unmodified (unless it was modified before)
+      ;; so that the user isn't bothered when killing it or quitting Emacs.
+      (set-buffer-modified-p was-modified))))
 
 ;; Font-locking support.
 
diff --git a/test/lisp/progmodes/elisp-mode-tests.el b/test/lisp/progmodes/elisp-mode-tests.el
index 4fa869c773f..2a18b204ef1 100644
--- a/test/lisp/progmodes/elisp-mode-tests.el
+++ b/test/lisp/progmodes/elisp-mode-tests.el
@@ -1131,5 +1131,30 @@ test-indentation
                         (emacs-lisp-mode)
                         (indent-region (point-min) (point-max)))))
 
+(defun elisp-mode-tests--insert-gibberish ()
+  (insert "gibberish\n")
+  (set-buffer-modified-p nil))
+
+(ert-deftest elisp-auto-dialect-declaration ()
+  (let ((file "does-not-exist.el"))
+    (should-not (file-exists-p file))
+    ;; Try with `elisp-auto-dialect-declaration' off and on.
+    (dolist (enabled '(nil lexical))
+      (let ((elisp-auto-dialect-declaration enabled))
+        ;; Try with a competing insertion that uses the `find-file-hook'.
+        (dolist (auto-ins '(nil t))
+          (let ((find-file-hook
+                 (if auto-ins
+                     (cons 'elisp-mode-tests--insert-gibberish find-file-hook)
+                   find-file-hook)))
+            (save-current-buffer
+              (find-file file)
+              (should (equal (buffer-string)
+                             (cond (auto-ins "gibberish\n")
+                                   (enabled ";;; -*- lexical-binding: t -*-\n")
+                                   (t ""))))
+              (should (equal (buffer-modified-p) nil))
+              (kill-buffer))))))))
+
 (provide 'elisp-mode-tests)
 ;;; elisp-mode-tests.el ends here
-- 
2.32.0 (Apple Git-132)

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Tue, 24 Oct 2023 19:31:33 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>,
>  66706@debbugs.gnu.org,
>  Stefan Monnier <monnier@iro.umontreal.ca>
> 
> Here is an updated patch, now with a test.

As mentioned earlier, please make the automatic insertion be off by
default, and please reflect that in the NEWS entry.

Thanks.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> As mentioned earlier, please make the automatic insertion be off by
> default, and please reflect that in the NEWS entry.

That makes the patch completely useless.  There's already an opt-in
solution (via `auto-insert`), so I'd oppose this patch if it's OFF by
default since it's just added complexity.

The whole point of this patch is to help those users who don't
know better.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Kangas]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That makes the patch completely useless.  There's already an opt-in
> solution (via `auto-insert`), so I'd oppose this patch if it's OFF by
> default since it's just added complexity.

Indeed.  I also can't see any downsides to making it non-optional.

I think we should mainly ask ourselves if this feature will help users.
In my opinion, the answer is clearly yes, in almost all cases.  For the
rest, a very small minority indeed, the patch provides a user option.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> As mentioned earlier, please make the automatic insertion be off by
>> default, and please reflect that in the NEWS entry.
>
> That makes the patch completely useless.  There's already an opt-in
> solution (via `auto-insert`), so I'd oppose this patch if it's OFF by
> default since it's just added complexity.
>
> The whole point of this patch is to help those users who don't
> know better.
>
>
>         Stefan

I want this off by default.  We teach users to enable lexical binding in
the Lisp introduction and a myriad of other places; inserting lexical
binding cookies is not "the icing on the cake", but a gesture that
implies we should override the judgement of our users, and are willing
to browbeat them into abiding by such notions of ours.

Similar measures taken by other software bespeaks the existence of a
sense of superiority in their developers; doubtless, they think it
beyond their users to read several pages of documentation pertaining to
lexical binding, and to make from that an informed decision.

Users who know what lexical-binding controls will enable it without any
wheedling, should they so desire.  Users who don't or won't will delete
the cookie, then repine about its presence; quite justly at that, for
software which holds its users in such low regard is nothing except
impertinent.

We never insert text to enforce our conceptions of good practice in any
major mode.  Such is the first step down a very precipitous slope, which
culminates in the adoption of the haughty attitude that other software
has seized upon with particular zeal.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Po Lu <luangruo@yahoo.com> writes:

> I want this off by default.  We teach users to enable lexical binding in
> the Lisp introduction and a myriad of other places; inserting lexical

I was mistaken here.  Lexical binding is briefly mentioned as
non-default behavior.

So why are we attempting to remedy this pitfall of the novice in Emacs
Lisp mode by inserting a feature hitherto _undocumented_ in our manual
for the uninitiated?  Everything suggest to me that documenting lexical
binding in the manual is the proper course of action.

> binding cookies is not "the icing on the cake", but a gesture that
> implies we should override the judgement of our users, and are willing
> to browbeat them into abiding by such notions of ours.
>
> Similar measures taken by other software bespeaks the existence of a
> sense of superiority in their developers; doubtless, they think it
> beyond their users to read several pages of documentation pertaining to
> lexical binding, and to make from that an informed decision.
>
> Users who know what lexical-binding controls will enable it without any
> wheedling, should they so desire.  Users who don't or won't will delete
> the cookie, then repine about its presence; quite justly at that, for
> software which holds its users in such low regard is nothing except
> impertinent.
>
> We never insert text to enforce our conceptions of good practice in any
> major mode.  Such is the first step down a very precipitous slope, which
> culminates in the adoption of the haughty attitude that other software
> has seized upon with particular zeal.

I recognize this argument is grounded wholly on principle, but principle
itself is as important a consideration as any other.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I want this off by default.  We teach users to enable lexical binding in
> the Lisp introduction and a myriad of other places; inserting lexical
> binding cookies is not "the icing on the cake", but a gesture that
> implies we should override the judgement of our users, and are willing
> to browbeat them into abiding by such notions of ours.

The lexbind dialect is the dialect we recommend:

    Lately Emacs is moving towards using lexical binding in more and
    more places, with the goal of eventually making lexical binding the
    default.  In particular, all Emacs Lisp source files and the
    @file{*scratch*} buffer use lexical scoping.

So why should a new file default to using the dynbind dialect instead?

> Similar measures taken by other software bespeaks the existence of a
> sense of superiority in their developers; doubtless,

We simply can't satisfy everyone: either we impose the dynbind
dialect by default (the current situation) or we impose the lexbind
dialect by default.  In both cases you can look at it as something
imposed out of a sense of superiority, but I don't think that is
a useful way to look at it.

> they think it beyond their users to read several pages of
> documentation pertaining to lexical binding, and to make from that an
> informed decision.

Those users who read all the docs can easily turn that option off if
they so wish.  We shouldn't decide on what's a good defaults based
on them.

> We never insert text to enforce our conceptions of good practice in any
> major mode.  Such is the first step down a very precipitous slope, which
> culminates in the adoption of the haughty attitude that other software
> has seized upon with particular zeal.

Just to be clear: the long term goal is to eliminate the dynbind
dialect, or at least make it be an opt-in that requires taking extra
steps, such as adding a cookie to the file, whereas the lexbind dialect
should be the default.  Not because dynbind is evil.  Not because it
imposes an undue burden on the implementation.
But because having two dialects imposes an undue burden on our users.
Those who read all the docs may not care, but all the others are
vulnerable to the usual "I copied the code from that webpage and it
doesn't work" because the code on that webpage used the other dialect.

So the question is how to move from here to there in the least
painful way and I believe that inserting this cookie by default is a step
that will make this transition less painful.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Mattias Engdegård <mattias.engdegard@gmail.com>,
>   stefankangas@gmail.com,  66706@debbugs.gnu.org
> Date: Tue, 24 Oct 2023 15:19:46 -0400
> 
> > As mentioned earlier, please make the automatic insertion be off by
> > default, and please reflect that in the NEWS entry.
> 
> That makes the patch completely useless.  There's already an opt-in
> solution (via `auto-insert`), so I'd oppose this patch if it's OFF by
> default since it's just added complexity.
> 
> The whole point of this patch is to help those users who don't
> know better.

But we already have ample features that help those "who don't know
better".  We turned on lexical-binding by default in *scratch* and
other situations, and we have the prominent indication in the mode
line when dynamic binding is in effect.  How much more is needed, and
why?

This looks like an overkill to me.  Almost like tyranny.  And
inserting stuff into a buffer without the user's say-so is completely
against the Emacs spirit.

So I'm very much against this.  I can live with it if it's off by
default, but if you and others cannot live with such a compromise,
then neither will I.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Kangas <stefankangas@gmail.com>
> Date: Tue, 24 Oct 2023 13:22:03 -0700
> Cc: Mattias Engdegård <mattias.engdegard@gmail.com>, 
> 	66706@debbugs.gnu.org
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > That makes the patch completely useless.  There's already an opt-in
> > solution (via `auto-insert`), so I'd oppose this patch if it's OFF by
> > default since it's just added complexity.
> 
> Indeed.  I also can't see any downsides to making it non-optional.
> 
> I think we should mainly ask ourselves if this feature will help users.
> In my opinion, the answer is clearly yes, in almost all cases.  For the
> rest, a very small minority indeed, the patch provides a user option.

We should also ask ourselves if this is not "too much".  I think it
is, since a clear indication of dynamic binding, with a face that
makes it stand out, already exists on the mode line.  I think this is
enough.  It is not Emacs's business to force users into something they
don't want to do.  Emacs never did that, and should not do that in the
future.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> The lexbind dialect is the dialect we recommend:
>
>     Lately Emacs is moving towards using lexical binding in more and
>     more places, with the goal of eventually making lexical binding the
>     default.  In particular, all Emacs Lisp source files and the
>     @file{*scratch*} buffer use lexical scoping.
>
> So why should a new file default to using the dynbind dialect instead?

Because that is the default, rather than a mere recommendation.  A
suggestion is an idea its recipients are meant to judge for themselves,
on the grounds of its merits and deficiencies.

> We simply can't satisfy everyone: either we impose the dynbind
> dialect by default (the current situation) or we impose the lexbind
> dialect by default.  In both cases you can look at it as something
> imposed out of a sense of superiority, but I don't think that is
> a useful way to look at it.

There is no quandary between default values here.  This change is borne
out of both a desire to _retain the status quo_, and a belief that our
users are of such low intellectual caliber as to require instruction for
even the most elementary of tasks, so that the one pious way of writing
Emacs Lisp code is inculcated into them from the outset.

> Those users who read all the docs can easily turn that option off if
> they so wish.  We shouldn't decide on what's a good defaults based
> on them.

As mentioned above, we are not adjuding which form of variable binding
is to be enabled by default.

> Just to be clear: the long term goal is to eliminate the dynbind
> dialect, or at least make it be an opt-in that requires taking extra
> steps, such as adding a cookie to the file, whereas the lexbind dialect
> should be the default.  Not because dynbind is evil.  Not because it
> imposes an undue burden on the implementation.
> But because having two dialects imposes an undue burden on our users.
> Those who read all the docs may not care, but all the others are
> vulnerable to the usual "I copied the code from that webpage and it
> doesn't work" because the code on that webpage used the other dialect.

I find this categorical dismissal of the intellectual or problem-solving
capacity, and perhaps fortitude if you will, of Emacs users very
patronizing, needless to say with my Emacs user hat on.  Furthermore, it
is quite dubitable that minor customizations of the sort found on most
web pages are suceptible to influence by the type of variable binding
used.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> But because having two dialects imposes an undue burden on our users.
>> Those who read all the docs may not care, but all the others are
>> vulnerable to the usual "I copied the code from that webpage and it
>> doesn't work" because the code on that webpage used the other dialect.
> I find this categorical dismissal of the intellectual or problem-solving
> capacity, and perhaps fortitude if you will, of Emacs users very
> patronizing, needless to say with my Emacs user hat on.

There's no patronizing, here.  Just experience.  It *is* a source of
confusion.  I've seen enough such examples during my time in SX where
people give a valid answer, followed by comments like:

   "doesn't work here" --Mr.Foo
   "I assumed you have lexical-binding enabled, try enabling it and see
    if it works" --Mr.Bar

Sometimes the original poster tried to avoid the problem by including
the `-*- lexical-binding:t -*-` cookie in his snippet, but the other
user just copied that snippet to his existing file so the cookie didn't
end up on the first line.

I don't doubt that our users have the capacity to solve those problems.
It's just that my sadism is already satisfied by all the other problems
we foist on them.

I myself get bitten by the different dialects every once in a while,
sometimes even copying code within Emacs itself (typically taking
a chunk of code from a buffer and running it inside `M-:` or `M-x
ielm`).

> Furthermore, it is quite dubitable that minor customizations of the
> sort found on most web pages are suceptible to influence by the type
> of variable binding used.

And yet.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> enough.  It is not Emacs's business to force users into something they
> don't want to do.

Not inserting a cookie (i.e. what we do now) forces the use of the
dynbind dialect.  In my experience, most users either don't know which
dialect they want or want the lexbind dialect.

This is reflected in the fact that almost all the code I find out there,
is either using lexbind or is old: the only code still using dynbind is
the code that started its life before lexbind existed, pretty much.
And this is to be expected: lexical scoping is the standard in virtually
all programming languages (e.g. Python, JavaScript, C, C++, Java, C#,
Rust, ...)  so programmers are much more at ease with that choice.

So we currently do "force users into something they don't want to do"
and the patch will simply let us do that less often.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Mattias Engdegård
>  <mattias.engdegard@gmail.com>,  66706@debbugs.gnu.org,
>   stefankangas@gmail.com
> Date: Tue, 24 Oct 2023 22:01:25 -0400
> 
> Just to be clear: the long term goal is to eliminate the dynbind
> dialect, or at least make it be an opt-in that requires taking extra
> steps, such as adding a cookie to the file, whereas the lexbind dialect
> should be the default.  Not because dynbind is evil.  Not because it
> imposes an undue burden on the implementation.
> But because having two dialects imposes an undue burden on our users.
> Those who read all the docs may not care, but all the others are
> vulnerable to the usual "I copied the code from that webpage and it
> doesn't work" because the code on that webpage used the other dialect.
> 
> So the question is how to move from here to there in the least
> painful way and I believe that inserting this cookie by default is a step
> that will make this transition less painful.

I thought we already decided on that: we should make lexical-binding
the default at some point.  Until then, I see no reason why the
prominent indication on the mode line should not be enough.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Kangas]

Eli Zaretskii <eliz@gnu.org> writes:

> We should also ask ourselves if this is not "too much".  I think it
> is, since a clear indication of dynamic binding, with a face that
> makes it stand out, already exists on the mode line.  I think this is
> enough.

I expect the users most helped by this to not even register that warning
on the mode line.  It's quite inconspicuous.  I never see it, myself.

We have had the mode-line warning for years, yet most user
configurations still implicitly use "lexical-binding:nil".  Just have a
look at the configurations posted on GitHub, Reddit and elsewhere.

This will mean that a lot of users will have a Bad Time (TM) when we
eventually change the default to lexical-binding:t.  We can help
mitigate that pain, or we can choose not to.

The controversial choice here is to see this situation coming yet refuse
to take reasonable steps to help.  That would be unfortunate, and a
disservice to our users.

> It is not Emacs's business to force users into something they don't
> want to do.

No one disagrees with that, AFAICT.  The proposed patch has a user
option to disable the new behavior.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Nikolay Kudryavtsev]

I think a better perspective to frame this discussion is this:

Lets say you go to a restaurant and you order a duck. The waiter asks 
whether you want a white duck or a yellow duck. And then you're stuck 
thinking about the differences in gastronomic quantities of both.

Same thing here. Assuming the user is reasonable, but not knowledgeable 
yet, there's a risk that forcefully adding that cookie would derail such 
a user solving some trivial problem into learning the differences 
between dynamic and lexical scoping way before he really has to actually 
care about them.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 14:48, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> I myself get bitten by the different dialects every once in a while,
> sometimes even copying code within Emacs itself (typically taking
> a chunk of code from a buffer and running it inside `M-:` or `M-x
> ielm`).

+1

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 15:36, Nikolay Kudryavtsev wrote:
> Same thing here. Assuming the user is reasonable, but not knowledgeable 
> yet, there's a risk that forcefully adding that cookie would derail such 
> a user solving some trivial problem into learning the differences 
> between dynamic and lexical scoping way before he really has to actually 
> care about them.

Knowing the difference can be helpful, though.

E.g. flymake in emacs-lisp-mode isn't very useful without lexbind.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> There's no patronizing, here.  Just experience.  It *is* a source of
> confusion.  I've seen enough such examples during my time in SX where
> people give a valid answer, followed by comments like:
>
>    "doesn't work here" --Mr.Foo
>    "I assumed you have lexical-binding enabled, try enabling it and see
>     if it works" --Mr.Bar

OK, and why is it such a reply does not suffice?  And why would any
measure short of initializing lexical-binding to t eliminate these
people, when so many have failed in the past?

> I don't doubt that our users have the capacity to solve those problems.
> It's just that my sadism is already satisfied by all the other problems
> we foist on them.

Then please inure yourself to such trifling matters of theirs.  Those
who are ignorant of lexical binding will run afoul of it once, learn
their lesson, and from that time forward routinely check whether it is
enabled.

You are not obliged to guarantee that no person stumbles across this
pitfall, and we are long past the juncture after which the law of
diminishing returns begins to assert itself in full force.  Such
"safeguards" as those proposed here will irk the informed and the
ignorant alike, while the latter, which is to say those who elect not to
read the documentation, take notice of the prominently decorated mode
line lighter, or heed advice given to them on forums such as
StackExchange, will continue not to enable lexical binding where it is
due.  As they have always done.

I am not the only person empty of the élan for lexical binding that
serves as the impetus for these increasingly heavy-handed measures,
incidentally.  Eli seems to agree with me.

> I myself get bitten by the different dialects every once in a while,
> sometimes even copying code within Emacs itself (typically taking
> a chunk of code from a buffer and running it inside `M-:` or `M-x
> ielm`).

How will the automatic insertion of lexical binding cookies in new files
affect M-: or IELM, both of which already execute lexically bound code?

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 15:17, Stefan Kangas wrote:
> This will mean that a lot of users will have a Bad Time (TM) when we
> eventually change the default to lexical-binding:t.  We can help
> mitigate that pain, or we can choose not to.

There will be such users, but their number will likely be lower than 
some might expect, simply because the lexical dialect very much 
resembles how programs are written in other languages these days.

To use a feature specifically available only in dynbind (e.g. access a 
local variable from a caller function), one would really have to know 
about it. But of course there can be typos, wrong refactorings, etc, 
which could contribute.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 15:03, Eli Zaretskii wrote:
>> From: Stefan Monnier<monnier@iro.umontreal.ca>
>> Cc: Eli Zaretskii<eliz@gnu.org>,  Mattias Engdegård
>>   <mattias.engdegard@gmail.com>,66706@debbugs.gnu.org,
>>    stefankangas@gmail.com
>> Date: Tue, 24 Oct 2023 22:01:25 -0400
>>
>> Just to be clear: the long term goal is to eliminate the dynbind
>> dialect, or at least make it be an opt-in that requires taking extra
>> steps, such as adding a cookie to the file, whereas the lexbind dialect
>> should be the default.  Not because dynbind is evil.  Not because it
>> imposes an undue burden on the implementation.
>> But because having two dialects imposes an undue burden on our users.
>> Those who read all the docs may not care, but all the others are
>> vulnerable to the usual "I copied the code from that webpage and it
>> doesn't work" because the code on that webpage used the other dialect.
>>
>> So the question is how to move from here to there in the least
>> painful way and I believe that inserting this cookie by default is a step
>> that will make this transition less painful.
> I thought we already decided on that: we should make lexical-binding
> the default at some point.  Until then, I see no reason why the
> prominent indication on the mode line should not be enough.

I wasn't aware of that indication myself, FWIW. Or forgot.

Regarding intermediate steps toward making 'lexical-binding: t' the 
default, are you of the opinion that we've made enough of them now? And 
will just be able to flip the default in some future release?

All in all, I'd say the byte-compilation warning "file has no 
‘lexical-binding’ directive" which we already show might suffice. But it 
only helps after the user learns how to byte-compile files and/or enable 
flymake-mode.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Dmitry Gutov <dmitry@gutov.dev> writes:

> I wasn't aware of that indication myself, FWIW. Or forgot.
>
> Regarding intermediate steps toward making 'lexical-binding: t' the
> default, are you of the opinion that we've made enough of them now?
> And will just be able to flip the default in some future release?
>
> All in all, I'd say the byte-compilation warning "file has no
> ‘lexical-binding’ directive" which we already show might suffice. But
> it only helps after the user learns how to byte-compile files and/or
> enable flymake-mode.

We cannot make lexical binding the default until an adequately
proficient writer is found to explain it within the Emacs Lisp
introduction.  Any volunteers?  For doing so would certainly be more
conducive towards that goal.

But today's software types are generally more interested in the much
simpler (if doubtfully efficacious) expedient that is coercion.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 16:20, Po Lu wrote:
> We cannot make lexical binding the default until an adequately
> proficient writer is found to explain it within the Emacs Lisp
> introduction.

Do we have an explanation there of "dynamic binding"? Or mentions of any 
features that wouldn't work in the context of lexical binding (e.g. 
referencing a local variable of a caller function)?

If so, those would be the places to start with.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> Date: Wed, 25 Oct 2023 16:06:43 +0300
> Cc: luangruo@yahoo.com, mattias.engdegard@gmail.com, 66706@debbugs.gnu.org,
>  stefankangas@gmail.com
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 25/10/2023 15:03, Eli Zaretskii wrote:
> >> From: Stefan Monnier<monnier@iro.umontreal.ca>
> >> Cc: Eli Zaretskii<eliz@gnu.org>,  Mattias Engdegård
> >>   <mattias.engdegard@gmail.com>,66706@debbugs.gnu.org,
> >>    stefankangas@gmail.com
> >> Date: Tue, 24 Oct 2023 22:01:25 -0400
> >>
> Regarding intermediate steps toward making 'lexical-binding: t' the 
> default, are you of the opinion that we've made enough of them now? And 
> will just be able to flip the default in some future release?

I thought we already answered this question by YES.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> OK, and why is it such a reply does not suffice?  And why would any
> measure short of initializing lexical-binding to t eliminate these
> people, when so many have failed in the past?

Initializing lexical-binding to t by default is currently not an option,
because it would introduce too much breakage.  For this reason, we need
to take smaller steps that will later make this change possible.

> How will the automatic insertion of lexical binding cookies in new files
> affect M-: or IELM, both of which already execute lexically bound code?

Because it will reduce the occurrence of the case where I copy a chunk
of code which relies on dynbind and then incorrectly execute it in
lexbind mode in `M-:` or ielm.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> I thought we already decided on that: we should make lexical-binding
> the default at some point.

The question is how to get to that point.

My understanding is that we're waiting for enough files/packages to use
lexbind so that the change will not introduce too much breakage.

> Until then, I see no reason why the prominent indication on the mode
> line should not be enough.

It's much easier to write code directly in lexbind dialect than to go
back and change existing code to adapt to the lexbind dialect.

Since "we should make lexical-binding the default at some point",
all files which come without `lexical-binding:t` cookie will need to go
through the harder path, compared to those who started their life
with that cookie.
So inserting that cookie in new files would help our users take the
easier road.

For that same reason, it will cause less breakage when the default is
changed, and thus will let us make this change sooner rather than later.

Of course, maybe I'm not being bold enough and we shouldn't wait: let's
change the default `lexical-binding` to t right away and just tell
people to add a `lexical-binding:nil` cookie if they don't know how (or
don't want to) adjust their code to the new dialect.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Eli Zaretskii <eliz@gnu.org>,  Mattias Engdegård
>  <mattias.engdegard@gmail.com>,  66706@debbugs.gnu.org,
>   stefankangas@gmail.com
> Date: Wed, 25 Oct 2023 10:56:50 -0400
> 
> > OK, and why is it such a reply does not suffice?  And why would any
> > measure short of initializing lexical-binding to t eliminate these
> > people, when so many have failed in the past?
> 
> Initializing lexical-binding to t by default is currently not an option,
> because it would introduce too much breakage.  For this reason, we need
> to take smaller steps that will later make this change possible.

I think if turning lexical-binding on by default is not yet possible,
we should wait until it's possible.  I see no need for any
intermediate steps, certainly not for a step such as the one suggested
here.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: luangruo@yahoo.com,  mattias.engdegard@gmail.com,
>   66706@debbugs.gnu.org,  stefankangas@gmail.com
> Date: Wed, 25 Oct 2023 11:11:52 -0400
> 
> > Until then, I see no reason why the prominent indication on the mode
> > line should not be enough.
> 
> It's much easier to write code directly in lexbind dialect than to go
> back and change existing code to adapt to the lexbind dialect.

If you think that inserting the cookie is okay, then it should also be
okay to turn lexical-binding on by default.  The effect is the same.

> Of course, maybe I'm not being bold enough and we shouldn't wait: let's
> change the default `lexical-binding` to t right away and just tell
> people to add a `lexical-binding:nil` cookie if they don't know how (or
> don't want to) adjust their code to the new dialect.

If this is what you were arguing for, I could at least understand it.
The measure proposed here I don't even understand.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Dmitry Gutov]

On 25/10/2023 19:08, Eli Zaretskii wrote:
>> Of course, maybe I'm not being bold enough and we shouldn't wait: let's
>> change the default `lexical-binding` to t right away and just tell
>> people to add a `lexical-binding:nil` cookie if they don't know how (or
>> don't want to) adjust their code to the new dialect.
> If this is what you were arguing for, I could at least understand it.
> The measure proposed here I don't even understand.

FWIW, if we flipped lexical-binding to t in the next release, that 
should also solve all user scenarios that were presented in this discussion.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> FWIW, if we flipped lexical-binding to t in the next release, that should
> also solve all user scenarios that were presented in this discussion.

Sounds good to me.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

25 okt. 2023 kl. 18.08 skrev Eli Zaretskii <eliz@gnu.org>:

> If you think that inserting the cookie is okay, then it should also be
> okay to turn lexical-binding on by default.  The effect is the same.

No, the automatic cookie insertion doesn't affect existing code.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Mattias Engdegård <mattias.engdegard@gmail.com>
> Date: Wed, 25 Oct 2023 20:19:40 +0200
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>,
>  luangruo@yahoo.com,
>  66706@debbugs.gnu.org,
>  stefankangas@gmail.com
> 
> 25 okt. 2023 kl. 18.08 skrev Eli Zaretskii <eliz@gnu.org>:
> 
> > If you think that inserting the cookie is okay, then it should also be
> > okay to turn lexical-binding on by default.  The effect is the same.
> 
> No, the automatic cookie insertion doesn't affect existing code.

Which is even worse, because some files will use lexical-binding, and
others will use dynamic binding.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

25 okt. 2023 kl. 20.40 skrev Eli Zaretskii <eliz@gnu.org>:

>> No, the automatic cookie insertion doesn't affect existing code.
> 
> Which is even worse, because some files will use lexical-binding, and
> others will use dynamic binding.

In other words just like now (but better, since hardly any new code will use dynamic binding).

It depends on how much sympathy you have with people having existing code written in the old dialect. It could be argued that they should have converted their code long ago, but that assumes that they know that they actually have that problem.

Only now do we have a compiler warning telling them, which is only effective if (a) people recompile their code when they upgrade Emacs (which they don't need to do because we are almost too good at .elc backward compatibility, and some run their code interpreted all the time) and (b) they look at at the warning and decide to take action.

We can and probably should try to devise other methods to catch those falling through the net, but the assumption was also that we should give them some time between being aware of the problem and being forced to do something. Of course, this can be discussed.

Indeed, ripping off the bandage quickly can also sometimes be the best solution.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Mattias Engdegård <mattias.engdegard@gmail.com> writes:

> In other words just like now (but better, since hardly any new code
> will use dynamic binding).

That's a theory, one inconsistent with the results of all other measures
to promote lexical binding that we have taken so far.

> It depends on how much sympathy you have with people having existing
> code written in the old dialect. It could be argued that they should
> have converted their code long ago, but that assumes that they know
> that they actually have that problem.

The mere existence of dynamically bound code is not a problem, and
nothing enjoins users to convert their code.  This is the sort of
fallacious thinking that leads people to propose draconian measures such
as the subject of this bug report.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> OK, and why is it such a reply does not suffice?  And why would any
>> measure short of initializing lexical-binding to t eliminate these
>> people, when so many have failed in the past?
>
> Initializing lexical-binding to t by default is currently not an
> option, because it would introduce too much breakage.  For this
> reason, we need to take smaller steps that will later make this change
> possible.

As previously mentioned, the law of diminishing returns now applies in
full force.  The intensity of our measures has grown substantially from
merely introducing lexical binding, to converting all our code to use
it, displaying a prominent indicator in the mode line, and ultimately
generating warnings when dynamically bound code is compiled.  The
proposed measure ups the ante from wheedling to browbeating, with
scarcely any evidence to substantiate the assertion that it will reduce
the (already negligible) prevalence of dynamic binding.  What's next,
duress?  Perhaps that will first take the form of a prohibition on
dynamically bound packages in ELPA, and gradually toughen thereafter?

One genuine step towards enabling lexical binding by default is the
revision of the Lisp introduction to describe it.  Of course time and
again, people here have demonstrated themselves so much more eager for
imperiousness towards users than the comparatively bland task of
writing.

Or perhaps such users of Emacs as those qualified to undertake this
writing don't share your fervor for lexical binding.  Which can only be
a good thing, as it were.

> Because it will reduce the occurrence of the case where I copy a chunk
> of code which relies on dynbind and then incorrectly execute it in
> lexbind mode in `M-:` or ielm.

The incidence of such code is low as it stands.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> FWIW, if we flipped lexical-binding to t in the next release, that
>> should
>> also solve all user scenarios that were presented in this
>> discussion.
>
> Sounds good to me.
>
>
>         Stefan

Who will update the Emacs Lisp introduction?

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

On 10/25/2023 6:20 AM, Po Lu via Bug reports for GNU Emacs, the Swiss 
army knife of text editors wrote:
> We cannot make lexical binding the default until an adequately
> proficient writer is found to explain it within the Emacs Lisp
> introduction.  Any volunteers?  For doing so would certainly be more
> conducive towards that goal.

If we can get some high-level agreement on things that this explanation 
should include (I'm not sure how much detail we want to go into in the 
ELisp Intro), I could take a stab at it.

In particular, what's missing from the following paragraph[1]?

> Another way to think about let is that it is like a setq that is temporary and local. The values set by let are automatically undone when the let is finished. The setting only affects expressions that are inside the bounds of the let expression. In computer science jargon, we would say the binding of a symbol is visible only in functions called in the let form; in Emacs Lisp, the default scoping is dynamic, not lexical. (The non-default lexical binding is not discussed in this manual.) 
[1] 
https://www.gnu.org/software/emacs/manual/html_node/eintr/Prevent-confusion.html

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

On 10/25/2023 12:09 PM, Mattias Engdegård wrote:
> Only now do we have a compiler warning telling them, which is only effective if (a) people recompile their code when they upgrade Emacs (which they don't need to do because we are almost too good at .elc backward compatibility, and some run their code interpreted all the time) and (b) they look at at the warning and decide to take action.

Assuming we don't want to flip the default for 'lexical-binding' just 
yet, would it be possible to warn the user any time they load 
interpreted code that uses dynamic binding?

We could potentially do the same for .elc files that were implicitly 
compiled with dynamic binding (i.e. without a "lexical-binding: nil" 
cookie), but in my opinion it would suffice to warn users when they 
recompile that file. There's nothing I'm aware of that makes the 
dynamically-bound .elc *wrong*; it's just not preferred, and one day, 
you might get a lexically-bound .elc instead.

Or we could just flip the default now.

- Jim

bug#66706: [PATCH] Automatic elisp dialect insertion

[Michael Heerdegen]

Dmitry Gutov <dmitry@gutov.dev> writes:

> There will be such users, but their number will likely be lower than
> some might expect, simply because the lexical dialect very much
> resembles how programs are written in other languages these days.

Short version of my reply: Emacs users are a different group than Elisp
package developers.  Let's help them by forcing a lexical binding cookie
in the config file instead of simply making their Emacs potentially not
starting up.

Long version: I guess most new users have not read much of the Emacs
manual before trying Emacs and starting a user config file.  Most of
them will not have started their config file using a lexical-binding:t
cookie.  That means: Most new users started their learning experience of
Elisp using the dynamical dialect.  Since Lisp is not that widespread,
not all Emacs users are programmers, and not all programmers take the
time to care about such a detail as long as the tool works, I would
think that the numbers of users which will see a broken config would be
higher than you expect.  And a lot of them will not know immediately how
to fix it, because this might make not so trivial changes necessary.
Without having their tool at hand.  Not cool.

This would be an unfriendly and annoying thing we would do to some
users.  Especially when Emacs is still perfectly able to interpret the
file.  I really hope we find a different way.

We could temporarily make a cookie mandatory for user config files, and
make Emacs barf at startup if a cookie missing (or ask about what to do:
insert the cookie for you, ask which dialect to use for this time), but
still use dynamical binding at least for a few releases so that Emacs
will still start up, instead of forcing lexical-binding interpretation
on a file that has only ever been interpreted using dynamical binding in
the past.


Michael.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jim Porter <jporterbugs@gmail.com> writes:

> If we can get some high-level agreement on things that this
> explanation should include (I'm not sure how much detail we want to go
> into in the ELisp Intro), I could take a stab at it.
>
> In particular, what's missing from the following paragraph[1]?

I'm not certain.  My recollection of the previous discussion is that the
entire manual must be revised to relate lexical binding to the complete
novice, without sacrificing the clarity or simplicity of the text in the
process.  The latter is a real pickle, for its original author has since
left us, and no other technical writers of his stature remain at our
disposal.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> In particular, what's missing from the following paragraph[1]?
>
>> Another way to think about let is that it is like a setq that is temporary
>> and local. The values set by let are automatically undone when the let is
>> finished. The setting only affects expressions that are inside the bounds
>> of the let expression. In computer science jargon, we would say the
>> binding of a symbol is visible only in functions called in the let form;
>> in Emacs Lisp, the default scoping is dynamic, not lexical. (The
>> non-default lexical binding is not discussed in this manual.) 

Nothing is "missing" but the above describes dynamic binding, static
binding has a different behavior which is not like "a setq at the
beginning which is undone at the end".  So this paragraph needs to be
fully rewritten.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

On 10/25/2023 5:51 PM, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
>> In particular, what's missing from the following paragraph[1]?
>>
>>> Another way to think about let is that it is like a setq that is temporary
>>> and local. The values set by let are automatically undone when the let is
>>> finished. The setting only affects expressions that are inside the bounds
>>> of the let expression. In computer science jargon, we would say the
>>> binding of a symbol is visible only in functions called in the let form;
>>> in Emacs Lisp, the default scoping is dynamic, not lexical. (The
>>> non-default lexical binding is not discussed in this manual.)
> 
> Nothing is "missing" but the above describes dynamic binding, static
> binding has a different behavior which is not like "a setq at the
> beginning which is undone at the end".  So this paragraph needs to be
> fully rewritten.

This is helpful. The bit about 'setq' seemed a bit roundabout (to say 
the least), and while I can see how it might help explain 'let' to a 
novice, it's probably more likely to mislead them instead.

I'll start with a patch here then. I think this is also a prime spot to 
add an example or two that would actually show lexical binding in action 
(i.e. a sample where the code would do something different under dynamic 
binding).

- Jim

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Jim Porter <jporterbugs@gmail.com> writes:

> This is helpful. The bit about 'setq' seemed a bit roundabout (to say
> the least), and while I can see how it might help explain 'let' to a
> novice, it's probably more likely to mislead them instead.
>
> I'll start with a patch here then. I think this is also a prime spot
> to add an example or two that would actually show lexical binding in
> action (i.e. a sample where the code would do something different
> under dynamic binding).
>
> - Jim

Does anyone recall the # of the previous bug report this subject was
last explored in detail in?  Where IIRC this boiled down to the absence
of someone like Bob Chassell?

bug#66706: [PATCH] Automatic elisp dialect insertion

[Drew Adams]

> In particular, what's missing from the following paragraph[1]?
> 
> > Another way to think about let is that it is like a setq that is
> temporary and local. The values set by let are automatically undone when
> the let is finished. The setting only affects expressions that are inside
> the bounds of the let expression. In computer science jargon, we would say
> the binding of a symbol is visible only in functions called in the let
> form; in Emacs Lisp, the default scoping is dynamic, not lexical. (The
> non-default lexical binding is not discussed in this manual.)

This bit would be far off-base for a description
of lexical let-binding:

  when the let is finished
  ^^^^            ^^^^^^^^

That's excusable for the Intro Lisp manual back
when Elisp had only dynamic binding.  It fits only
dynamic, not lexical, let-binding.

A description of a lexical let binding should talk
about lexical scope - a limit you can see "on the
page" (it's lexical) - a limit on "where" in code,
not "when" in code evaluation.

"Finished" is nearly as bad (for lexical binding),
as it has a (strong) connotation of process/time.
For lexical binding it's about where the let sexp
ends, not when let processing finishes.

OTOH, "local" _is_ pertinent for lexical, but it's
about being local _lexically_: local means inside
the let sexp.  A lexical binding doesn't exist
outside that scope.

And this bit is not true for dynamic binding:

  The setting only affects expressions that are
  inside the bounds of the let expression.

It affects expressions that are evaluated within
the extent (time) of the dynamic let binding.
There's _no_ limit on the scope, dynamically -
it's indefinite: anything, anywhere, can refer to
a thing that's dynamically bound.

If looking for a clear and palatable description
then let me suggest starting with the way lexical
and dynamic binding are presented in CLTL2:

https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node43.html#SECTION00700000000000000000

And from the moment the language has both lexical
and dynamic binding (as it does now), a description
of let needs to also get into the fact that it can
bind dynamic (special) variables also - the overall
behavior is more complicated to describe because of
that.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Drew Adams]

> Assuming we don't want to flip the default for 'lexical-binding' just
> yet, would it be possible to warn the user any time they load
> interpreted code that uses dynamic binding?

I hope you don't really mean code that
uses dynamic binding.

I hope you just mean code defined with
`lexical-binding'=nil.  Such code can
perfectly well use dynamic binding.
And it will.  Always.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Drew Adams]

> Nothing is "missing" but the above describes dynamic binding, static
> binding has a different behavior which is not like "a setq at the
> beginning which is undone at the end".  So this paragraph needs to be
> fully rewritten.

Exactly.  You put it simpler than I did. ;-)

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

On 10/25/2023 6:19 PM, Jim Porter wrote:
> I'll start with a patch here then. I think this is also a prime spot to 
> add an example or two that would actually show lexical binding in action 
> (i.e. a sample where the code would do something different under dynamic 
> binding).

Here's a first attempt. I'm not sure I'm entirely happy with it (the 
digression into setting 'lexical-binding' to 't' is a bit disruptive), 
but hopefully it's an improvement. Of course, we can keep adjusting this 
further as needed.

[-- Attachment #2: 0001-Introduce-let-using-lexical-binding-in-the-Lisp-Intr.patch --]
[-- Type: text/plain, Size: 4916 bytes --]

From 6bc9bbbb98105f700bb8d5b04e8de5e261efa777 Mon Sep 17 00:00:00 2001
From: Jim Porter <jporterbugs@gmail.com>
Date: Wed, 25 Oct 2023 20:43:57 -0700
Subject: [PATCH] Introduce 'let' using lexical binding in the Lisp
 Introduction

* doc/lispintro/emacs-lisp-intro.texi (Prevent confusion): Rename to...
(Why Use let?): ... this, and rework the explanation to discuss
lexical binding (including how to enable it).
---
 doc/lispintro/emacs-lisp-intro.texi | 81 +++++++++++++++++++++--------
 1 file changed, 58 insertions(+), 23 deletions(-)

diff --git a/doc/lispintro/emacs-lisp-intro.texi b/doc/lispintro/emacs-lisp-intro.texi
index fce7583fe91..ebbcc08b9ff 100644
--- a/doc/lispintro/emacs-lisp-intro.texi
+++ b/doc/lispintro/emacs-lisp-intro.texi
@@ -3587,39 +3587,74 @@ let
 @code{let} special form prevents this kind of confusion.
 
 @menu
-* Prevent confusion::
+* Why Use let?::
 * Parts of let Expression::
 * Sample let Expression::
 * Uninitialized let Variables::
 @end menu
 
 @ifnottex
-@node Prevent confusion
-@unnumberedsubsec @code{let} Prevents Confusion
+@node Why Use let?
+@unnumberedsubsec Why Use @code{let}?
 @end ifnottex
 
 @cindex @samp{local variable} defined
 @cindex @samp{variable, local}, defined
-The @code{let} special form prevents confusion.  @code{let} creates a
-name for a @dfn{local variable} that overshadows any use of the same
-name outside the @code{let} expression.  This is like understanding
-that whenever your host refers to ``the house'', he means his house, not
-yours.  (Symbols used in argument lists work the same way.
-@xref{defun, , The @code{defun} Macro}.)
-
-Local variables created by a @code{let} expression retain their value
-@emph{only} within the @code{let} expression itself (and within
-expressions called within the @code{let} expression); the local
-variables have no effect outside the @code{let} expression.
-
-Another way to think about @code{let} is that it is like a @code{setq}
-that is temporary and local.  The values set by @code{let} are
-automatically undone when the @code{let} is finished.  The setting
-only affects expressions that are inside the bounds of the @code{let}
-expression.  In computer science jargon, we would say the binding of
-a symbol is visible only in functions called in the @code{let} form;
-in Emacs Lisp, the default scoping is dynamic, not lexical.  (The
-non-default lexical binding is not discussed in this manual.)
+The @code{let} special form provides a way to confine your variables
+to a particular section of your code (in computer science jargon, a
+``scope'').  @code{let} creates a name for a @dfn{local variable} that
+overshadows any use of the same name outside the @code{let} expression
+(we call this ``binding'' the variable).  This prevents any accidental
+usage of these variables outside of the @code{let} expression.  This
+is like understanding that whenever your host refers to ``the house'',
+he means his house, not yours.  (Symbols used in argument lists work
+the same way.  @xref{defun, , The @code{defun} Macro}.)
+
+@cindex lexical binding
+@cindex binding, lexical
+@cindex dynamic binding
+@cindex binding, dynamic
+Before we begin discussing @code{let} in detail, we must first mention
+an important note.  For historical reasons, Emacs Lisp uses a form of
+variable binding called ``dynamic binding''.  However, this manual
+will discuss the preferred form of binding, called ``lexical binding''
+(if you have programmed in other languages before, you're likely
+already familiar with how lexical binding behaves).  In order to use
+lexical binding, you should add something like this to the first line
+of your Emacs Lisp file:
+
+@example
+;;; -*- lexical-binding: t -*-
+@end example
+
+For more information about this, @pxref{Selecting Lisp Dialect, , ,
+elisp, The Emacs Lisp Reference Manual}.
+
+With that out of the way, we can return to discussing @code{let}.
+Local variables created by a @code{let} expression hold their value
+@emph{only} within the body of the @code{let} expression itself; the
+local variables have no effect outside of the @code{let} expression.
+This means that inside the @code{let} body, calling @code{setq}
+for a variable named by the @code{let} expression will set the value
+of the @emph{local} variable of that name.  This also means that
+outside of the @code{let} body, calling @code{setq} for a variable
+named by the @code{let} expression will @emph{not} affect that local
+variable.
+
+For example, if you call a function inside of a @code{let}
+body, that function's body would be unable to ``see'' (or modify) the
+value of a local variable from the @code{let} expression:
+
+@example
+(setq x 1)
+
+(defun getx ()
+  x)
+
+(let ((x 2))
+  (get-x))
+     @result{} 1
+@end example
 
 @code{let} can create more than one variable at once.  Also,
 @code{let} gives each variable it creates an initial value, either a
-- 
2.25.1

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

On 10/25/2023 7:34 PM, Drew Adams wrote:
>> Assuming we don't want to flip the default for 'lexical-binding' just
>> yet, would it be possible to warn the user any time they load
>> interpreted code that uses dynamic binding?
> 
> I hope you don't really mean code that
> uses dynamic binding.

Yeah, that was poorly worded. I should have said "warn the user any time 
they load interpreted code without a 'lexical-binding' cookie".

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> Date: Wed, 25 Oct 2023 17:07:00 -0700
> Cc: Eli Zaretskii <eliz@gnu.org>, 66706@debbugs.gnu.org,
>  mattias.engdegard@gmail.com, Stefan Monnier <monnier@iro.umontreal.ca>,
>  stefankangas@gmail.com
> From: Jim Porter <jporterbugs@gmail.com>
> 
> In particular, what's missing from the following paragraph[1]?
> 
> > Another way to think about let is that it is like a setq that is temporary and local. The values set by let are automatically undone when the let is finished. The setting only affects expressions that are inside the bounds of the let expression. In computer science jargon, we would say the binding of a symbol is visible only in functions called in the let form; in Emacs Lisp, the default scoping is dynamic, not lexical. (The non-default lexical binding is not discussed in this manual.) 
> [1] 
> https://www.gnu.org/software/emacs/manual/html_node/eintr/Prevent-confusion.html

IMNSHO, it is woefully incomplete.  Much more needs to be said.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> Date: Wed, 25 Oct 2023 17:07:55 -0700
> Cc: luangruo@yahoo.com, 66706@debbugs.gnu.org, monnier@iro.umontreal.ca,
>  stefankangas@gmail.com
> From: Jim Porter <jporterbugs@gmail.com>
> 
> On 10/25/2023 12:09 PM, Mattias Engdegård wrote:
> > Only now do we have a compiler warning telling them, which is only effective if (a) people recompile their code when they upgrade Emacs (which they don't need to do because we are almost too good at .elc backward compatibility, and some run their code interpreted all the time) and (b) they look at at the warning and decide to take action.
> 
> Assuming we don't want to flip the default for 'lexical-binding' just 
> yet, would it be possible to warn the user any time they load 
> interpreted code that uses dynamic binding?

We already do: on the mode line.  Doing more would be an annoyance.
E.g., many init files use dynamic binding.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Jim Porter]

On 10/25/2023 8:48 PM, Jim Porter wrote:
> Here's a first attempt. I'm not sure I'm entirely happy with it (the 
> digression into setting 'lexical-binding' to 't' is a bit disruptive), 
> but hopefully it's an improvement. Of course, we can keep adjusting this 
> further as needed.

On second thought, let's discuss the documentation work in a separate 
bug so as not to hijack this one: bug#66756.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Eli Zaretskii <eliz@gnu.org> writes:

> We already do: on the mode line.  Doing more would be an annoyance.

What's more, it would be disrespect.  And I think that goes far beyond
the pale of behavior acceptable from software.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Stefan Kangas <stefankangas@gmail.com>,  Eli Zaretskii <eliz@gnu.org>,
>   mattias.engdegard@gmail.com,  66706@debbugs.gnu.org,
>   monnier@iro.umontreal.ca
> Date: Thu, 26 Oct 2023 02:31:57 +0200
> 
> Dmitry Gutov <dmitry@gutov.dev> writes:
> 
> > There will be such users, but their number will likely be lower than
> > some might expect, simply because the lexical dialect very much
> > resembles how programs are written in other languages these days.
> 
> Short version of my reply: Emacs users are a different group than Elisp
> package developers.  Let's help them by forcing a lexical binding cookie
> in the config file instead of simply making their Emacs potentially not
> starting up.

If this is about init files, we could instead make processing init
files assume lexical-binding by default.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> Date: Wed, 25 Oct 2023 20:48:04 -0700
> From: Jim Porter <jporterbugs@gmail.com>
> Cc: 66706@debbugs.gnu.org, mattias.engdegard@gmail.com,
>  Po Lu <luangruo@yahoo.com>, Dmitry Gutov <dmitry@gutov.dev>,
>  stefankangas@gmail.com, Eli Zaretskii <eliz@gnu.org>
> 
> On 10/25/2023 6:19 PM, Jim Porter wrote:
> > I'll start with a patch here then. I think this is also a prime spot to 
> > add an example or two that would actually show lexical binding in action 
> > (i.e. a sample where the code would do something different under dynamic 
> > binding).
> 
> Here's a first attempt. I'm not sure I'm entirely happy with it (the 
> digression into setting 'lexical-binding' to 't' is a bit disruptive), 
> but hopefully it's an improvement. Of course, we can keep adjusting this 
> further as needed.

Thanks.

The challenge in updating the Lisp Introduction manual is to try to
keep its informal and reader-friendly style as much as possible.  It
is not just another ELisp Reference manual!  So please try to keep
that in mind when you write the text, and in particular try not to
modify the existing text that is still accurate -- it was written by a
master, and each word there counts, even if it looks at first sight as
not important.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Mattias Engdegård]

26 okt. 2023 kl. 02.07 skrev Jim Porter <jporterbugs@gmail.com>:

> Assuming we don't want to flip the default for 'lexical-binding' just yet, would it be possible to warn the user any time they load interpreted code that uses dynamic binding?

If you mean .el files without a cookie, then definitely yes.

> We could potentially do the same for .elc files that were implicitly compiled with dynamic binding (i.e. without a "lexical-binding: nil" cookie), but in my opinion it would suffice to warn users when they recompile that file.

Not sure how you would detect that.
Ideally we should try to find a way to recompile code (and emit warnings) without inconveniencing the user too much.

> Or we could just flip the default now.

If we had a modest pot of money, we could sponsor young and coming (or old and going) programmers for converting code using dynamic binding that is still in regular use.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Nikolay Kudryavtsev]

Sure, knowing the difference is extremely helpful. But still, I think 
it's one of the more intermediate topics, which should be hidden from 
users up until they absolutely have to know about it.

The ideal of Emacs has always been about bringing the extensibility even 
to the non-technical people. Where someone could learn Elisp and use it 
even without any other technical background.

For someone like this, there's already a steep learning curve and a 
conscious effort must be still be expanded to keep that curve in check, 
and maybe even smoothen it where possible.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Nikolay Kudryavtsev]

26.10.2023 11:32, Mattias Engdegård writes:
> If we had a modest pot of money, we could sponsor young and coming (or 
> old and going) programmers for converting code using dynamic binding 
> that is still in regular use.

Dynamic binding allows you to tweak some behavior when you have to 
conform to an existing API, even when it has not been properly designed 
to to support your use case. We just don't know how much 3rd party code 
uses it in this perfectly valid way and refactoring those mature API to 
support even some of the use cases like that may not be realistically 
doable or worth it in the first place.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> What's more, it would be disrespect.

These kinds of arguments make me realize that we're deep in
bikeshed territory.


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Po Lu via Bug reports for GNU Emacs, the Swiss army knife of text editors]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> These kinds of arguments make me realize that we're deep in
> bikeshed territory.

The argument stands wherever we may be, so I'm not willing to compromise
in this department.  Emacs is not a truncheon its users are to be
bludgeoned with.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> These kinds of arguments make me realize that we're deep in
>> bikeshed territory.
>
> The argument stands wherever we may be, so I'm not willing to
> compromise in this department.  Emacs is not a truncheon its users are
> to be bludgeoned with.

Luckily, "le ridicule ne tue pas"  :-)


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Drew Adams]

> Dynamic binding allows you to tweak some behavior when you have to
> conform to an existing API, even when it has not been properly designed
> to to support your use case. We just don't know how much 3rd party code
> uses it in this perfectly valid way and refactoring those mature API to
> support even some of the use cases like that may not be realistically
> doable or worth it in the first place.

I believe this discussion is not about dynamic
binding as such; it's about having the default
be lexical binding.

That is, it's about having, by default, the
behavior you get with lexical-binding=nil.

That doesn't prevent any use of dynamic binding.
What it does is make Elisp behave like Common
Lisp wrt lexical/dynamic: It makes lexical
binding the _default_; nothing more.  To get
dynamic binding for something you need to tell
Lisp that it's "special".

And yes, everything you said about dynamic
binding is correct: it is extremely useful, in
particular for an application such as Emacs.

But wrt that there should be no special need
to refactor anything.  The problem (need for
rewriting/refactoring) is code for code that
expects or depends on dynamic binding by
default.

Long ago RMS summarized the special advantages
dynamic binding offers for an app such as Emacs:

https://www.gnu.org/software/emacs/emacs-paper.html#SEC17

https://www.gnu.org/software/emacs/emacs-paper.html#SEC18

bug#66706: [PATCH] Automatic elisp dialect insertion

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > Short version of my reply: Emacs users are a different group than Elisp
> > package developers.  Let's help them by forcing a lexical binding cookie
> > in the config file instead of simply making their Emacs potentially not
> > starting up.
>
> If this is about init files, we could instead make processing init
> files assume lexical-binding by default.

That's exactly what we should avoid because it can make Emacs not start
up for init files that require the current default of dynamical binding.

We also should not assume dynamical binding: lexical binding is the
default.  Emacs should not guess: it should ask the user and then add a
cookie.  That's my point.

Michael.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: dmitry@gutov.dev,  mattias.engdegard@gmail.com,  66706@debbugs.gnu.org,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Fri, 27 Oct 2023 05:14:11 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > Short version of my reply: Emacs users are a different group than Elisp
> > > package developers.  Let's help them by forcing a lexical binding cookie
> > > in the config file instead of simply making their Emacs potentially not
> > > starting up.
> >
> > If this is about init files, we could instead make processing init
> > files assume lexical-binding by default.
> 
> That's exactly what we should avoid because it can make Emacs not start
> up for init files that require the current default of dynamical binding.

How many of init files do indeed require dynamic binding?  And why?

If a new Emacs version refuses to start because something in the init
files goes against some new Emacs feature, we have --debug-init and
other facilities to debug and fix those.  Why should lexical-binding
be considered different from any other backward-incompatible change
that we sometimes do?

In any case, how will injecting the cookie help those users, exactly?
If people who write init files don't understand the implications of
lexical-binding, their Emacs will fail to start, something that you
think is a catastrophe.  And if they do understand it, their init
files are already compatible with lexical-binding.

> We also should not assume dynamical binding: lexical binding is the
> default.  Emacs should not guess: it should ask the user and then add a
> cookie.  That's my point.

Your point seems to be incompatible with our plan, which is to turn on
lexical-binding by default at some point.  We don't want to have a
schism of two separate flavors of Lisp, we want only one.  The current
situation is a transitional period, not the ideal.  You seem to be
suggesting that we should keep this situation forever, and that is not
what we decided, AFAIU.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> How many of init files do indeed require dynamic binding?  And why?

It was the default, and when I read what others mention ("look at the
code is posted here and there"), I guess there are a lot.  And exactly
those users will have problems with the upgrade.

If you look at the code that is posted in emacs-help, you see that some
people use a coding style that heavily depends on dynamical binding.

>
> If a new Emacs version refuses to start because something in the init
> files goes against some new Emacs feature, we have --debug-init and
> other facilities to debug and fix those.

I don't think that a debugger is that helpful for rewriting that stuff.

> Why should lexical-binding be considered different from any other
> backward-incompatible change that we sometimes do?

As I said: it's unnecessary to break Emacs startup when we can just be
nice and use the other dialect.  We can still force users to rewrite
their init file in nicer ways.

> In any case, how will injecting the cookie help those users, exactly?
> If people who write init files don't understand the implications of
> lexical-binding, their Emacs will fail to start, something that you
> think is a catastrophe.  And if they do understand it, their init
> files are already compatible with lexical-binding.

If Emacs starts and it doesn't find a lexical-binding cookie in the init
file, it might be written in the dynamically binding dialect.  I suggest
that Emacs then prompts the user: hey, your init file lacks that cookie,
if you upgraded your Emacs recently, it is likely that I should
interpret it as dynamically binding, although lexical-binding is the
new default.  If you just installed Emacs, you should use "lexical-binding".

Use lexical binding for now (y-or-n)?

And then you would be instructed to add a cookie for the init file.
With a hint where to look up the relevant information about the two
dialects.

> Your point seems to be incompatible with our plan, which is to turn on
> lexical-binding by default at some point.  We don't want to have a
> schism of two separate flavors of Lisp, we want only one.  The current
> situation is a transitional period, not the ideal.  You seem to be
> suggesting that we should keep this situation forever, and that is not
> what we decided, AFAIU.

No, I don't think you fully understand.  I only want to help the users
with the transition.  Nice programs do that, they help you a bit with
necessary changes to your config after major changes.

I suggest this treatment only for the init file.  And also only for a
transitional period (I already said that though...).

Michael.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: dmitry@gutov.dev,  mattias.engdegard@gmail.com,  66706@debbugs.gnu.org,
>   stefankangas@gmail.com,  monnier@iro.umontreal.ca
> Date: Fri, 27 Oct 2023 09:24:19 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Your point seems to be incompatible with our plan, which is to turn on
> > lexical-binding by default at some point.  We don't want to have a
> > schism of two separate flavors of Lisp, we want only one.  The current
> > situation is a transitional period, not the ideal.  You seem to be
> > suggesting that we should keep this situation forever, and that is not
> > what we decided, AFAIU.
> 
> No, I don't think you fully understand.  I only want to help the users
> with the transition.

I don't think it will help.  I guess we disagree about that.

bug#66706: [PATCH] Automatic elisp dialect insertion

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Your point seems to be incompatible with our plan, which is to turn on
> lexical-binding by default at some point.

What's missing before we can do that, in your opinion?


        Stefan

bug#66706: [PATCH] Automatic elisp dialect insertion

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Michael Heerdegen <michael_heerdegen@web.de>,  dmitry@gutov.dev,
>   mattias.engdegard@gmail.com,  66706@debbugs.gnu.org,
>   stefankangas@gmail.com
> Date: Fri, 27 Oct 2023 10:41:42 -0400
> 
> > Your point seems to be incompatible with our plan, which is to turn on
> > lexical-binding by default at some point.
> 
> What's missing before we can do that, in your opinion?

Our decision?