bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

This would be sometimes useful, and more consistent with other modes like
dired and diff.  Is there any reason to not do it?

Ergus: I've cc'ed you because this is potentially related to your recent
discussion in emacs-devel about "out of sources compilation"
(compilation buffers use comint-mode).

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> Cc: Ergus <spacibba@aol.com>
> From: Augusto Stoffel <arstoffel@gmail.com>
> Date: Tue, 02 Apr 2024 07:54:46 +0200
> 
> This would be sometimes useful, and more consistent with other modes like
> dired and diff.  Is there any reason to not do it?

It doesn't sound right to me to do that by default, since comint is
used for shell-like interpreters, and those tend to change directories
at will.  Which means that dir-locals for some random directory
doesn't necessarily take such modes into consideration.

If you need that for some particular use case, can't you call it from
comint-mode-hook or something?

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

On Tue,  2 Apr 2024 at 14:58, Eli Zaretskii wrote:

> It doesn't sound right to me to do that by default, since comint is
> used for shell-like interpreters, and those tend to change directories
> at will.  Which means that dir-locals for some random directory
> doesn't necessarily take such modes into consideration.

This observation makes sense, but it mostly applies to the good old
'M-x shell', not to 'M-x project-shell', other language interpreters, or
to compilation buffers.

By the way, I now realize that 'M-x compile' doesn't use comint-mode
by default.  Which raises the same question: should compilation-mode
call hack-dir-local-variables-non-file-buffer?

> If you need that for some particular use case, can't you call it from
> comint-mode-hook or something?

Sure, it's an easy customization, but the question is whether it's the
expected default behavior. :-)

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 70136@debbugs.gnu.org,  spacibba@aol.com
> Date: Tue, 02 Apr 2024 16:03:23 +0200
> 
> By the way, I now realize that 'M-x compile' doesn't use comint-mode
> by default.  Which raises the same question: should compilation-mode
> call hack-dir-local-variables-non-file-buffer?

Maybe.  What kind of directory-specific variables relevant to
compilation-mode would make sense?

> > If you need that for some particular use case, can't you call it from
> > comint-mode-hook or something?
> 
> Sure, it's an easy customization, but the question is whether it's the
> expected default behavior. :-)

The only way to answer that is if we see a flood of requests to have
that by default.  Without that, local customizations are perfectly
adequate.

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

On Tue,  2 Apr 2024 at 18:11, Eli Zaretskii wrote:

> Maybe.  What kind of directory-specific variables relevant to
> compilation-mode would make sense?

There are some many.  To say just the first that comes to mind:
compilation-error-regexp-alist.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

[-- Attachment #1: Type: text/plain, Size: 444 bytes --]

Since compilation buffers go as far as to print the directory they're
running on at the top of the buffer, I think it's pretty clear they
should receive dir-local variables.

So I'd suggest the attached patch, which does that and also removes a
more limited mechanism I added some time ago to allow compilation with
project-specific settings.  I've CC'ed Stefan since at the time he kind
of supported the changes I'm now suggesting to remove.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Add-dir-local-variables-to-compilation-buffers.patch --]
[-- Type: text/x-patch, Size: 2296 bytes --]

From e29f1849c278ebf55aa67470b5f35263ecf989f3 Mon Sep 17 00:00:00 2001
From: Augusto Stoffel <arstoffel@gmail.com>
Date: Sun, 14 Apr 2024 11:07:02 +0200
Subject: [PATCH] Add dir-local variables to compilation buffers

* lisp/progmodes/compile.el (compilation-mode): Arrange for dir-local
variables to be hacked into to non-file compilation buffers.
(compilation-start): Remove a less flexible solution to set up the
compilation environment (cf. discussion in bug#50607).
---
 lisp/progmodes/compile.el | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/lisp/progmodes/compile.el b/lisp/progmodes/compile.el
index b18eb81fee1..f81c1edb8c3 100644
--- a/lisp/progmodes/compile.el
+++ b/lisp/progmodes/compile.el
@@ -1929,9 +1929,6 @@ compilation-start
 	    (replace-regexp-in-string "-mode\\'" "" (symbol-name mode))))
 	 (thisdir default-directory)
 	 (thisenv compilation-environment)
-         (buffer-path (and (local-variable-p 'exec-path) exec-path))
-         (buffer-env (and (local-variable-p 'process-environment)
-                          process-environment))
 	 outwin outbuf)
     (with-current-buffer
 	(setq outbuf
@@ -2004,12 +2001,6 @@ compilation-start
         ;; NB: must be done after (funcall mode) as that resets local variables
         (setq-local compilation-directory thisdir)
         (setq-local compilation-environment thisenv)
-        (if buffer-path
-            (setq-local exec-path buffer-path)
-          (kill-local-variable 'exec-path))
-        (if buffer-env
-            (setq-local process-environment buffer-env)
-          (kill-local-variable 'process-environment))
 	(if highlight-regexp
             (setq-local compilation-highlight-regexp highlight-regexp))
         (if (or compilation-auto-jump-to-first-error
@@ -2372,6 +2363,11 @@ compilation-mode
   ;; some other input event happens.
   (setq-local jit-lock-defer-time nil)
   (setq buffer-read-only t)
+  (unless (buffer-file-name)
+    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
+      (set sym #'hack-dir-local-variables-non-file-buffer)
+      ;; Ensure hack-dir-locals is called only after a derived mode is set.
+      (push sym delayed-mode-hooks)))
   (run-mode-hooks 'compilation-mode-hook))
 
 ;;;###autoload
-- 
2.44.0

bug#70136: 30.0.50; comint-mode doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 70136@debbugs.gnu.org,  spacibba@aol.com
> Date: Sun, 14 Apr 2024 11:16:48 +0200
> 
> On Tue,  2 Apr 2024 at 18:11, Eli Zaretskii wrote:
> 
> > Maybe.  What kind of directory-specific variables relevant to
> > compilation-mode would make sense?
> 
> There are some many.  To say just the first that comes to mind:
> compilation-error-regexp-alist.

I must say this is not very convincing.

What bothers me is that .dir-locals.el in a directory is not meant for
compilation-mode and similar modes, so settings there that are not
specific to modes could get in the way in *compilation* and "grep*
buffers.  But if this is optional behavior, by default off, I don't
think I'd mind.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 70136@debbugs.gnu.org,  Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Sun, 14 Apr 2024 11:27:28 +0200
> 
> Since compilation buffers go as far as to print the directory they're
> running on at the top of the buffer, I think it's pretty clear they
> should receive dir-local variables.
> 
> So I'd suggest the attached patch, which does that and also removes a
> more limited mechanism I added some time ago to allow compilation with
> project-specific settings.  I've CC'ed Stefan since at the time he kind
> of supported the changes I'm now suggesting to remove.

Thanks, but I think this should be optional behavior, by default off,
because it could cause trouble in directory trees which already have
.dir-locals.el that were not intended to affect compilation-mode (and
its descendants, like Grep).

Also, this needs a NEWS entry, I think.

> +  (unless (buffer-file-name)
> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
> +      (set sym #'hack-dir-local-variables-non-file-buffer)
> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
> +      (push sym delayed-mode-hooks)))

Why such a complicated way of using the symbol of a function that's
defined in a preloaded Lisp file?  Am I missing some subtlety here?

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

On Sun, 14 Apr 2024 at 13:21, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Cc: 70136@debbugs.gnu.org,  Stefan Monnier <monnier@iro.umontreal.ca>
>> Date: Sun, 14 Apr 2024 11:27:28 +0200
>> 
>> Since compilation buffers go as far as to print the directory they're
>> running on at the top of the buffer, I think it's pretty clear they
>> should receive dir-local variables.
>> 
>> So I'd suggest the attached patch, which does that and also removes a
>> more limited mechanism I added some time ago to allow compilation with
>> project-specific settings.  I've CC'ed Stefan since at the time he kind
>> of supported the changes I'm now suggesting to remove.
>
> Thanks, but I think this should be optional behavior, by default off,
> because it could cause trouble in directory trees which already have
> .dir-locals.el that were not intended to affect compilation-mode (and
> its descendants, like Grep).

This seems rather hypothetical to me.  Do you have a concrete example?
The dir locals mechanism is very precise and easy to use.  Anything not
intended to affect compilation buffers should be put under prog-mode (or
a descendant), text-mode, or whatever else it's actually intended for.

On the other hand, imagine this situation: you're working on a project
with very long lines in some files, so you want your grep buffers to
look more compact.  Then you type

  M-x add-dir-local-variable RET grep-mode RET truncate-lines RET t RET

Wouldn't you be very confused that this doesn't work?

> Also, this needs a NEWS entry, I think.
>
>> +  (unless (buffer-file-name)
>> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
>> +      (set sym #'hack-dir-local-variables-non-file-buffer)
>> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
>> +      (push sym delayed-mode-hooks)))
>
> Why such a complicated way of using the symbol of a function that's
> defined in a preloaded Lisp file?  Am I missing some subtlety here?

This is because delayed-mode-hooks is a _list of hooks_, not a hook.
Adding (the name of) a function to it doesn't work.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: 70136@debbugs.gnu.org,  monnier@iro.umontreal.ca
> Date: Mon, 15 Apr 2024 19:10:05 +0200
> 
> On Sun, 14 Apr 2024 at 13:21, Eli Zaretskii wrote:
> 
> >> From: Augusto Stoffel <arstoffel@gmail.com>
> >> Cc: 70136@debbugs.gnu.org,  Stefan Monnier <monnier@iro.umontreal.ca>
> >> Date: Sun, 14 Apr 2024 11:27:28 +0200
> >> 
> >> Since compilation buffers go as far as to print the directory they're
> >> running on at the top of the buffer, I think it's pretty clear they
> >> should receive dir-local variables.
> >> 
> >> So I'd suggest the attached patch, which does that and also removes a
> >> more limited mechanism I added some time ago to allow compilation with
> >> project-specific settings.  I've CC'ed Stefan since at the time he kind
> >> of supported the changes I'm now suggesting to remove.
> >
> > Thanks, but I think this should be optional behavior, by default off,
> > because it could cause trouble in directory trees which already have
> > .dir-locals.el that were not intended to affect compilation-mode (and
> > its descendants, like Grep).
> 
> This seems rather hypothetical to me.  Do you have a concrete example?

You are entitled to your opinions, but this is clearly a change in
behavior that will affect a lot of users (since compilation-mode and
its descendants are very popular and widely used).  Therefore, I don't
understand why you need concrete examples: the issue is crystal clear
just by thinking about it.

> The dir locals mechanism is very precise and easy to use.  Anything not
> intended to affect compilation buffers should be put under prog-mode (or
> a descendant), text-mode, or whatever else it's actually intended for.

On the contrary, .dir-locals.el affects every file in a directory, so
it is quite a blunt weapon.

> On the other hand, imagine this situation: you're working on a project
> with very long lines in some files, so you want your grep buffers to
> look more compact.  Then you type
> 
>   M-x add-dir-local-variable RET grep-mode RET truncate-lines RET t RET
> 
> Wouldn't you be very confused that this doesn't work?

No, I would not.

> 
> > Also, this needs a NEWS entry, I think.
> >
> >> +  (unless (buffer-file-name)
> >> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
> >> +      (set sym #'hack-dir-local-variables-non-file-buffer)
> >> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
> >> +      (push sym delayed-mode-hooks)))
> >
> > Why such a complicated way of using the symbol of a function that's
> > defined in a preloaded Lisp file?  Am I missing some subtlety here?
> 
> This is because delayed-mode-hooks is a _list of hooks_, not a hook.
> Adding (the name of) a function to it doesn't work.

I'm asking mainly about the need to use make-symbol.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Juri Linkov]

> On the other hand, imagine this situation: you're working on a project
> with very long lines in some files, so you want your grep buffers to
> look more compact.  Then you type
>
>   M-x add-dir-local-variable RET grep-mode RET truncate-lines RET t RET
>
> Wouldn't you be very confused that this doesn't work?

I agree this would be confusing.

IMHO, the problem is more wide and it's that dir-local variables
are not supported in non-file buffers by default.  Maybe a new
option could enable them not only in compilation buffers,
but in all non-file buffers?  Or such a new option could be
a list of non-file modes that should support dir-local variables.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Juri Linkov <juri@linkov.net>
> Cc: Eli Zaretskii <eliz@gnu.org>,  70136@debbugs.gnu.org,
>   monnier@iro.umontreal.ca
> Date: Tue, 16 Apr 2024 09:33:51 +0300
> 
> > On the other hand, imagine this situation: you're working on a project
> > with very long lines in some files, so you want your grep buffers to
> > look more compact.  Then you type
> >
> >   M-x add-dir-local-variable RET grep-mode RET truncate-lines RET t RET
> >
> > Wouldn't you be very confused that this doesn't work?
> 
> I agree this would be confusing.
> 
> IMHO, the problem is more wide and it's that dir-local variables
> are not supported in non-file buffers by default.  Maybe a new
> option could enable them not only in compilation buffers,
> but in all non-file buffers?  Or such a new option could be
> a list of non-file modes that should support dir-local variables.

I'm okay with such opt-in behavior.  We can later make it the default
if enough people like it and not many complain about it.

Thanks.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> You are entitled to your opinions, but this is clearly a change in
> behavior that will affect a lot of users (since compilation-mode and
> its descendants are very popular and widely used).  Therefore, I don't
> understand why you need concrete examples: the issue is crystal clear
> just by thinking about it.

FWIW, back in 2010 (commit 8117868f0ce6) when we added support for
dir-locals to non-file buffers, we did it without even a config var to
turn it off.

AFAICT the `dir-locals.el` format should already be sufficiently
flexible to make it easy for users annoyed by the new behavior to
recover the old behavior (without affecting older Emacsen).

I think we should make an effort to enable dir-locals in as many buffers
as makes sense (but that can't be all buffers, because many buffers
aren't really related to any particular place in the file system, in
which case using the dir-locals setting of the directory that happens to
be current when the buffer was created is too arbitrary).


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Augusto Stoffel <arstoffel@gmail.com>,  70136@debbugs.gnu.org
> Date: Tue, 16 Apr 2024 17:49:08 -0400
> 
> > You are entitled to your opinions, but this is clearly a change in
> > behavior that will affect a lot of users (since compilation-mode and
> > its descendants are very popular and widely used).  Therefore, I don't
> > understand why you need concrete examples: the issue is crystal clear
> > just by thinking about it.
> 
> FWIW, back in 2010 (commit 8117868f0ce6) when we added support for
> dir-locals to non-file buffers, we did it without even a config var to
> turn it off.

That's not the same.  Also, we did quite a few things wrong regarding
backward compatibility over the years, and I don't want us to repeat
past mistakes.

> AFAICT the `dir-locals.el` format should already be sufficiently
> flexible to make it easy for users annoyed by the new behavior to
> recover the old behavior (without affecting older Emacsen).
> 
> I think we should make an effort to enable dir-locals in as many buffers
> as makes sense (but that can't be all buffers, because many buffers
> aren't really related to any particular place in the file system, in
> which case using the dir-locals setting of the directory that happens to
> be current when the buffer was created is too arbitrary).

Like I said: I'm okay with this change provided that it is opt-in.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> FWIW, back in 2010 (commit 8117868f0ce6) when we added support for
>> dir-locals to non-file buffers, we did it without even a config var to
>> turn it off.
> That's not the same.  Also, we did quite a few things wrong regarding
> backward compatibility over the years, and I don't want us to repeat
> past mistakes.

I can relate to that, but I can't remember bug reports (nor questions
from confused users in other channels) when we made that change, so
I don't see why we should consider that specific past choice to be
a "past mistake".
Also, I'm not seeing why "That's not the same".

> Like I said: I'm okay with this change provided that it is opt-in.

The problem with that is discovery.  Should we add a message like
"ignoring dir-locals.  See obey-dir-local-variables-in-all-non-file-buffers"?

And of course a related question is what kind of granularity to use for
the "opt-in"?  Will we add a new var every time we notice another (set
of) buffers for which we should apply dir-local vars, or would it be OK
to have a single variable?

If it's OK to have a single var: why should this var not
apply to diff-mode, log-edit-mode, ...?
Or should this var contain a list of modes which allow it, so we can
make it default to (diff-mode log-edit-mode ...)?  And maybe also allow
a t value?

And since this var is needed only to avoid breaking backward
compatibility, it would be desirable to have a plan to get rid of it in
the longer term.


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

On Tue, 16 Apr 2024 at 09:33, Juri Linkov wrote:

> IMHO, the problem is more wide and it's that dir-local variables
> are not supported in non-file buffers by default.  Maybe a new
> option could enable them not only in compilation buffers,
> but in all non-file buffers?

One would have to exclude temporary and internal work buffers, for
efficiency reasons.

And then there's also the case of buffers that don't really "belong" to
a directory, such as *Help*.  Do you think there's a reliable way to
distinguish those from the rest?

(By the way, *Help* and friends get stuck forever in whatever
default-directory they were first created; so for instance if you do
C-x C-f in a *Help* buffer, you will see a random directory as
starting point...)

>  Or such a new option could be a list of non-file modes that should
> support dir-local variables.

I'm not a big fan of this idea.  The .dir-locals.el file is already a
list of modes, you can decide there which modes you want to be affected.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Augusto Stoffel]

On Tue, 16 Apr 2024 at 22:58, Stefan Monnier wrote:

> And since this var is needed only to avoid breaking backward
> compatibility, it would be desirable to have a plan to get rid of it in
> the longer term.

I definitely agree with this.  Eli's worry is about retaining
compatibility with buggy code (in this case, a hypothetical
.dir-locals.el that doesn't properly map variables to the desired
modes), so the compatibility workarounds, if any at all, should be
transitional.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: arstoffel@gmail.com,  70136@debbugs.gnu.org
> Date: Tue, 16 Apr 2024 22:58:10 -0400
> 
> >> FWIW, back in 2010 (commit 8117868f0ce6) when we added support for
> >> dir-locals to non-file buffers, we did it without even a config var to
> >> turn it off.
> > That's not the same.  Also, we did quite a few things wrong regarding
> > backward compatibility over the years, and I don't want us to repeat
> > past mistakes.
> 
> I can relate to that, but I can't remember bug reports (nor questions
> from confused users in other channels) when we made that change, so
> I don't see why we should consider that specific past choice to be
> a "past mistake".

I didn't mean to say that introduction of dir-locals specifically was
a mistake, I meant that in general, to make the point that not
everything we did before can be taken as a good recipe for imitation.

> Also, I'm not seeing why "That's not the same".

Because introducing a new feature is qualitatively different: it can
have no backward-compatibility problems, since no one can possibly
have existing customizations for it.

> > Like I said: I'm okay with this change provided that it is opt-in.
> 
> The problem with that is discovery.

It always is, with opt-in features.  But that doesn't mean we should
turn each new feature on, just to make it more discoverable.  There
are other considerations, and some are more important than
discoverability.

> Should we add a message like
> "ignoring dir-locals.  See obey-dir-local-variables-in-all-non-file-buffers"?

The time for April 1 jokes has come and passed this year, no? ;-)

> And of course a related question is what kind of granularity to use for
> the "opt-in"?  Will we add a new var every time we notice another (set
> of) buffers for which we should apply dir-local vars, or would it be OK
> to have a single variable?

There's no such dilemma in this case, because this feature was
proposed to be controlled by users to begin with.  So the variable was
already proposed, the discussion is just about its default value.

> And since this var is needed only to avoid breaking backward
> compatibility, it would be desirable to have a plan to get rid of it in
> the longer term.

I believe I suggested such a plan in my previous messages.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Also, I'm not seeing why "That's not the same".
>
> Because introducing a new feature is qualitatively different: it can
> have no backward-compatibility problems, since no one can possibly
> have existing customizations for it.

That commit I referred to had AFAICT the same effect as the one
discussed here: it made some modes (diff-mode, log-edit-mode, and a few
more) obey dir-locals whereas they didn't before.
And dir-locals existed since several years before that.

Why would it be more likely for them to have .dir-locals which
accidentally affect grep-mode than diff-mode/log-edit-mode/...?

AFAICT it risked the exact same backward compatibility problems.

>> Should we add a message like
>> "ignoring dir-locals.  See obey-dir-local-variables-in-all-non-file-buffers"?
> The time for April 1 jokes has come and passed this year, no? ;-)

I'm quite serious.  From where I stand, I think most users would
want to enable this feature in they have a situation where it affects
the behavior of Emacs.


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  70136@debbugs.gnu.org,
>   monnier@iro.umontreal.ca
> Date: Wed, 17 Apr 2024 10:16:03 +0200
> 
> On Tue, 16 Apr 2024 at 09:33, Juri Linkov wrote:
> 
> >  Or such a new option could be a list of non-file modes that should
> > support dir-local variables.
> 
> I'm not a big fan of this idea.  The .dir-locals.el file is already a
> list of modes, you can decide there which modes you want to be affected.

The difference is that if we say this should be in .dir-locals.el, we
place the burden on the users, whereas if we do what Juri suggested,
we solve this ourselves at least for modes that come with Emacs.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  70136@debbugs.gnu.org
> Date: Wed, 17 Apr 2024 10:31:42 +0200
> 
> On Tue, 16 Apr 2024 at 22:58, Stefan Monnier wrote:
> 
> > And since this var is needed only to avoid breaking backward
> > compatibility, it would be desirable to have a plan to get rid of it in
> > the longer term.
> 
> I definitely agree with this.  Eli's worry is about retaining
> compatibility with buggy code (in this case, a hypothetical
> .dir-locals.el that doesn't properly map variables to the desired
> modes), so the compatibility workarounds, if any at all, should be
> transitional.

This is not so much about buggy .dir-locals.el, this is about
non-buggy .dir-locals.el that have customizations for nil mode --
those customizations might not expect non-file modes to be affected.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: arstoffel@gmail.com,  70136@debbugs.gnu.org
> Date: Wed, 17 Apr 2024 08:59:50 -0400
> 
> >> Also, I'm not seeing why "That's not the same".
> >
> > Because introducing a new feature is qualitatively different: it can
> > have no backward-compatibility problems, since no one can possibly
> > have existing customizations for it.
> 
> That commit I referred to had AFAICT the same effect as the one
> discussed here: it made some modes (diff-mode, log-edit-mode, and a few
> more) obey dir-locals whereas they didn't before.
> And dir-locals existed since several years before that.
> 
> Why would it be more likely for them to have .dir-locals which
> accidentally affect grep-mode than diff-mode/log-edit-mode/...?
> 
> AFAICT it risked the exact same backward compatibility problems.

So we should risk it again?

> >> Should we add a message like
> >> "ignoring dir-locals.  See obey-dir-local-variables-in-all-non-file-buffers"?
> > The time for April 1 jokes has come and passed this year, no? ;-)
> 
> I'm quite serious.  From where I stand, I think most users would
> want to enable this feature in they have a situation where it affects
> the behavior of Emacs.

If you are right, we will be able to make this on by default in Emacs
31 (assuming we introduce the opt-in feature in Emacs 30, that is: no
one has yet shown the code, so we are discussing a highly theoretical
feature).

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Bruno Barbier]

Hi,

When setting debugger-print-function to prin1, Emacs 30.0.50 may fail
to enter the debugger.

Emacs displays something like:

    Entering debugger...
    make-text-button: Args out of range: 67, 3000


The correct behavior is to enter the debugger, with something like this:

    Debugger entered--Lisp error: (wrong-type-argument stringp nil)
      #f(compiled-function (&rest args) "Start a program in a subprocess.  ...
      ...
      make-process(:name "mandatory" :command "ls" :stderr err-buf)

The problem doesn't occur with Emacs 29.3.

To manually reproduce, execute the following code block in a new Emacs
(started with '-Q'):

  #+begin_src elisp
    (progn 
      (setq debugger-print-function 'prin1)
      
      (defun my-useless-advice (fun &rest args)
         (apply fun args))

      (advice-add 'make-process :around #'my-useless-advice)

      (make-process :name "mandatory"
                    :command "ls"
                    :stderr 'err-buf))
  #+end_src
  
It seems that backtrace.el prints using prin1, but infers where the
button should be using cl-prin1.

My personal workaround is to stop setting debugger-print-function to
prin1 (I customized it a few years ago because I found that cl-prin1 was
really slow at the time, I guess it shouldn't be the case anymore).

Best,

Bruno


In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.18.0) of 2024-04-17 built on keynux
Windowing system distributor 'The X.Org Foundation', version 11.0.12101011
System Description: Gentoo Linux

Configured using:
 'configure --prefix=/home/bruno/local --with-x-toolkit=lucid
 --without-toolkit-scroll-bars --without-tree-sitter
 --without-native-compilation --without-modules --without-xwidgets
 --without-threads --without-pop --without-mailutils
 --without-compress-install --without-hesiod --without-gameuser
 --without-lcms2 --without-wide-int --without-kerberos
 --without-kerberos5 --with-sound=no --without-ns --without-gsettings
 --without-gconf --without-libotf --without-m17n-flt
 --with-gif=ifavailable --with-harfbuzz'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM HARFBUZZ JPEG LIBXML2
NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SQLITE3 TIFF WEBP X11 XDBE XIM
XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LC_CTYPE: en_US.UTF-8
  value of $LANG: C.UTF8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  minibuffer-regexp-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
sendmail rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
warnings icons cl-loaddefs cl-lib rmc iso-transl tooltip cconv eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/x-win x-win term/common-win x-dnd touch-screen tool-bar dnd
fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode lisp-mode prog-mode register page tab-bar menu-bar rfn-eshadow
isearch easymenu timer select scroll-bar mouse jit-lock font-lock syntax
font-core term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote dbusbind inotify dynamic-setting
font-render-setting cairo x-toolkit xinput2 x multi-tty move-toolbar
make-network-process emacs)

Memory information:
((conses 16 40047 17385) (symbols 48 5270 0) (strings 32 13441 1036)
 (string-bytes 1 323186) (vectors 16 9362)
 (vector-slots 8 111771 9938) (floats 8 23 28) (intervals 56 305 0)
 (buffers 984 11))

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> That commit I referred to had AFAICT the same effect as the one
>> discussed here: it made some modes (diff-mode, log-edit-mode, and a few
>> more) obey dir-locals whereas they didn't before.
>> And dir-locals existed since several years before that.
>> 
>> Why would it be more likely for them to have .dir-locals which
>> accidentally affect grep-mode than diff-mode/log-edit-mode/...?
>> 
>> AFAICT it risked the exact same backward compatibility problems.
>
> So we should risk it again?

Well, that risk paid off because by all accounts it seems that noone
suffered (and some people benefited), so that argues in favor of taking
that risk again.


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: arstoffel@gmail.com,  70136@debbugs.gnu.org
> Date: Wed, 17 Apr 2024 13:55:55 -0400
> 
> >> That commit I referred to had AFAICT the same effect as the one
> >> discussed here: it made some modes (diff-mode, log-edit-mode, and a few
> >> more) obey dir-locals whereas they didn't before.
> >> And dir-locals existed since several years before that.
> >> 
> >> Why would it be more likely for them to have .dir-locals which
> >> accidentally affect grep-mode than diff-mode/log-edit-mode/...?
> >> 
> >> AFAICT it risked the exact same backward compatibility problems.
> >
> > So we should risk it again?
> 
> Well, that risk paid off because by all accounts it seems that noone
> suffered (and some people benefited), so that argues in favor of taking
> that risk again.

That's not the kind of risk management and mitigation that I'm used
to.  It's like saying "crossing that junction on red light went well
and paid off, so let's risk that again on this next junction".

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> That's not the kind of risk management and mitigation that I'm used
> to.  It's like saying "crossing that junction on red light went well
> and paid off, so let's risk that again on this next junction".

🙂
Except that I'd seriously doubt anyone would get seriously injured if
a dir-locals setting misfires.


        Stefan


PS: admittedly, I do cross on red lights on a regular basis (not when
    driving, tho).

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Eli Zaretskii]

> From: Bruno Barbier <brubar.cs@gmail.com>
> Date: Wed, 17 Apr 2024 15:29:14 +0200
> 
> When setting debugger-print-function to prin1, Emacs 30.0.50 may fail
> to enter the debugger.
> 
> Emacs displays something like:
> 
>     Entering debugger...
>     make-text-button: Args out of range: 67, 3000
> 
> 
> The correct behavior is to enter the debugger, with something like this:
> 
>     Debugger entered--Lisp error: (wrong-type-argument stringp nil)
>       #f(compiled-function (&rest args) "Start a program in a subprocess.  ...
>       ...
>       make-process(:name "mandatory" :command "ls" :stderr err-buf)
> 
> The problem doesn't occur with Emacs 29.3.
> 
> To manually reproduce, execute the following code block in a new Emacs
> (started with '-Q'):
> 
>   #+begin_src elisp
>     (progn 
>       (setq debugger-print-function 'prin1)
>       
>       (defun my-useless-advice (fun &rest args)
>          (apply fun args))
> 
>       (advice-add 'make-process :around #'my-useless-advice)
> 
>       (make-process :name "mandatory"
>                     :command "ls"
>                     :stderr 'err-buf))
>   #+end_src
>   
> It seems that backtrace.el prints using prin1, but infers where the
> button should be using cl-prin1.
> 
> My personal workaround is to stop setting debugger-print-function to
> prin1 (I customized it a few years ago because I found that cl-prin1 was
> really slow at the time, I guess it shouldn't be the case anymore).

Stefan, is this supposed to be supported?

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Eli Zaretskii]

> Cc: 70436@debbugs.gnu.org
> Date: Sat, 20 Apr 2024 11:24:11 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > From: Bruno Barbier <brubar.cs@gmail.com>
> > Date: Wed, 17 Apr 2024 15:29:14 +0200
> > 
> > When setting debugger-print-function to prin1, Emacs 30.0.50 may fail
> > to enter the debugger.
> > 
> > Emacs displays something like:
> > 
> >     Entering debugger...
> >     make-text-button: Args out of range: 67, 3000
> > 
> > 
> > The correct behavior is to enter the debugger, with something like this:
> > 
> >     Debugger entered--Lisp error: (wrong-type-argument stringp nil)
> >       #f(compiled-function (&rest args) "Start a program in a subprocess.  ...
> >       ...
> >       make-process(:name "mandatory" :command "ls" :stderr err-buf)
> > 
> > The problem doesn't occur with Emacs 29.3.
> > 
> > To manually reproduce, execute the following code block in a new Emacs
> > (started with '-Q'):
> > 
> >   #+begin_src elisp
> >     (progn 
> >       (setq debugger-print-function 'prin1)
> >       
> >       (defun my-useless-advice (fun &rest args)
> >          (apply fun args))
> > 
> >       (advice-add 'make-process :around #'my-useless-advice)
> > 
> >       (make-process :name "mandatory"
> >                     :command "ls"
> >                     :stderr 'err-buf))
> >   #+end_src
> >   
> > It seems that backtrace.el prints using prin1, but infers where the
> > button should be using cl-prin1.
> > 
> > My personal workaround is to stop setting debugger-print-function to
> > prin1 (I customized it a few years ago because I found that cl-prin1 was
> > really slow at the time, I guess it shouldn't be the case anymore).
> 
> Stefan, is this supposed to be supported?

Btw, if I also set backtrace-print-function to prin1, the above recipe
works as expected.  So maybe we should document that changing
debugger-print-function should also change backtrace-print-function?
Or even use the value of debugger-print-function as the default value
of backtrace-print-function?

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Stefan, is this supposed to be supported?

Yes.


        Stefan

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

I pushed a quick fix to `master`.


        Stefan

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: brubar.cs@gmail.com,  70436@debbugs.gnu.org
> Date: Sat, 20 Apr 2024 11:24:56 -0400
> 
> I pushed a quick fix to `master`.

Thanks, I guess we should now close this bug?

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Bruno Barbier]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Stefan Monnier <monnier@iro.umontreal.ca>
>> Cc: brubar.cs@gmail.com,  70436@debbugs.gnu.org
>> Date: Sat, 20 Apr 2024 11:24:56 -0400
>> 
>> I pushed a quick fix to `master`.
>
> Thanks, I guess we should now close this bug?

After installing Stefan fix, the bug is gone.  Thanks.

I've personally switched back using the default value cl-prin1, and
didn't notice any issue since.


Thanks Eli, Stefan.

Bruno

bug#70436: 30.0.50; Fail to enter the debugger when using prin1 (instead of cl-prin1)

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> Thanks, I guess we should now close this bug?
> After installing Stefan fix, the bug is gone.  Thanks.
> I've personally switched back using the default value cl-prin1, and
> didn't notice any issue since.

Thanks Bruno, I'm happy both work well for you now.
Closing,


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Juri Linkov]

> @@ -2372,6 +2363,11 @@ compilation-mode
>    ;; some other input event happens.
>    (setq-local jit-lock-defer-time nil)
>    (setq buffer-read-only t)
> +  (unless (buffer-file-name)
> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
> +      (set sym #'hack-dir-local-variables-non-file-buffer)
> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
> +      (push sym delayed-mode-hooks)))
>    (run-mode-hooks 'compilation-mode-hook))

Thanks for the patch.  I confirm it completely fixes the bug
reported at the end of bug#68570.

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Dmitry Gutov]

On 02/05/2024 09:17, Juri Linkov wrote:
>> @@ -2372,6 +2363,11 @@ compilation-mode
>>     ;; some other input event happens.
>>     (setq-local jit-lock-defer-time nil)
>>     (setq buffer-read-only t)
>> +  (unless (buffer-file-name)
>> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
>> +      (set sym #'hack-dir-local-variables-non-file-buffer)
>> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
>> +      (push sym delayed-mode-hooks)))
>>     (run-mode-hooks 'compilation-mode-hook))
> Thanks for the patch.  I confirm it completely fixes the bug
> reported at the end of bug#68570.

I'd like to express my support for the change as well (got here from 
bug#74631).

When we added it to some modes in Emacs 24, the announcement had what 
seems like advice to call this functions in more others (see the 
corresponding NEWS).

The current opt-out options seem sufficient IMHO (the Compilation mode, 
they would also include setting enable-dir-local-variables to nil 
locally in the mode hook).

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Eli Zaretskii]

> Date: Fri, 20 Dec 2024 05:09:24 +0200
> Cc: Eli Zaretskii <eliz@gnu.org>, 70136@debbugs.gnu.org,
>  Stefan Monnier <monnier@iro.umontreal.ca>
> From: Dmitry Gutov <dmitry@gutov.dev>
> 
> On 02/05/2024 09:17, Juri Linkov wrote:
> >> @@ -2372,6 +2363,11 @@ compilation-mode
> >>     ;; some other input event happens.
> >>     (setq-local jit-lock-defer-time nil)
> >>     (setq buffer-read-only t)
> >> +  (unless (buffer-file-name)
> >> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
> >> +      (set sym #'hack-dir-local-variables-non-file-buffer)
> >> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
> >> +      (push sym delayed-mode-hooks)))
> >>     (run-mode-hooks 'compilation-mode-hook))
> > Thanks for the patch.  I confirm it completely fixes the bug
> > reported at the end of bug#68570.
> 
> I'd like to express my support for the change as well (got here from 
> bug#74631).

I agreed with the change, provided it's opt-in.  That was 7 months
ago.

> When we added it to some modes in Emacs 24, the announcement had what 
> seems like advice to call this functions in more others (see the 
> corresponding NEWS).
> 
> The current opt-out options seem sufficient IMHO (the Compilation mode, 
> they would also include setting enable-dir-local-variables to nil 
> locally in the mode hook).

Sorry, I don't understand what you are saying here.  ("Emacs 24"?  I
thought we were discussing this for Emacs 30, for which it is now too
late, and Emacs 31?)

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Stefan Monnier via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>> @@ -2372,6 +2363,11 @@ compilation-mode
>>    ;; some other input event happens.
>>    (setq-local jit-lock-defer-time nil)
>>    (setq buffer-read-only t)
>> +  (unless (buffer-file-name)
>> +    (let ((sym (make-symbol "hack-dir-local-variables-non-file-buffer")))
>> +      (set sym #'hack-dir-local-variables-non-file-buffer)
>> +      ;; Ensure hack-dir-locals is called only after a derived mode is set.
>> +      (push sym delayed-mode-hooks)))
>>    (run-mode-hooks 'compilation-mode-hook))
>
> Thanks for the patch.  I confirm it completely fixes the bug
> reported at the end of bug#68570.

Clearly, I'm in favor, tho I find the specific code above rather ugly.
Could we improve it?
Maybe setting a `enable-dir-local-variables-non-file-buffer` variable,
which is then obeyed in the hack-local-* functions?


        Stefan

bug#70136: 30.0.50; compilation-mode [was: comint-mode] doesn't call hack-dir-local-variables-non-file-buffer

[Dmitry Gutov]

On 20/12/2024 09:29, Eli Zaretskii wrote:
>> When we added it to some modes in Emacs 24, the announcement had what
>> seems like advice to call this functions in more others (see the
>> corresponding NEWS).
>>
>> The current opt-out options seem sufficient IMHO (the Compilation mode,
>> they would also include setting enable-dir-local-variables to nil
>> locally in the mode hook).
> Sorry, I don't understand what you are saying here.  ("Emacs 24"?  I
> thought we were discussing this for Emacs 30, for which it is now too
> late, and Emacs 31?)

I mean this in NEWS.24 (the last sentence in particular):

   *** Directory local variables can apply to some file-less buffers.
   Affected modes include dired, vc-dir, and log-edit.  For example,
   adding "(diff-mode . ((mode . whitespace)))" to .dir-locals.el will
   turn on 'whitespace-mode' for *vc-diff* buffers.  Modes should call
   'hack-dir-local-variables-non-file-buffer' to support this.