bug#53507: 27.1; syntax highlight in the eval-expression prompt

bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

There is some progress thanks to Stefan who gave a hint on the emacs help list.

There is a check in font-core.el which disables font lock for buffers starting
with a space and the minibuffer is like that:

;; Don't turn on Font Lock mode if we don't have a display (we're running a
;; batch job) or if the buffer is invisible (the name starts with a space).

(when (or noninteractive (eq (aref (buffer-name) 0) ?\s))
 (setq font-lock-mode nil))

https://github.com/emacs-mirror/emacs/blob/master/lisp/font-core.el#L133

If one comments these lines, reevalutes the function and then goes to 
eval-expression, pastes this code into it:

  (progn (setq font-lock-mode t) (emacs-lisp-mode))

and evals this code right there in context (C-x C-e) then the lisp 
code in the minibuffer gets font locking.

This is good, the only problem left to solve is that snippet affects
quitting from the minibuffer for some reason.

bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Lars Ingebrigtsen]

ndame <laszlomail@protonmail.com> writes:

> ;; Don't turn on Font Lock mode if we don't have a display (we're running a
> ;; batch job) or if the buffer is invisible (the name starts with a space).
>
> (when (or noninteractive (eq (aref (buffer-name) 0) ?\s))
>  (setq font-lock-mode nil))
>
> https://github.com/emacs-mirror/emacs/blob/master/lisp/font-core.el#L133
>
> If one comments these lines, reevalutes the function and then goes to 
> eval-expression, pastes this code into it:
>
>   (progn (setq font-lock-mode t) (emacs-lisp-mode))
>
> and evals this code right there in context (C-x C-e) then the lisp 
> code in the minibuffer gets font locking.

Yes -- but I don't think this is something we can change at this point.
It's likely to break various things.

So if you want to do font locking things in the minibuffer, you have to
use minibuffer-setup-hook (or one of the other hooks) and set things
explicitly.

So I don't think there's anything to be done on the Emacs side here, but
perhaps I'm misunderstanding what this report is about?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>
> Yes -- but I don't think this is something we can change at this point.
> It's likely to break various things.

Naturally, I did not post the final solution. If you check the bug
history they you can see that up to know it was not clear why font
lock does not work in the eval-expression prompt:

https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46092

Stefan gave a hint that it is disabled in font lock mode and
this is the case. That's why I posted that proof of concept
code which shows that font locking does work there if we
remove the disabling code.

It should be implemented properly, hopefully in emacs 29, because
syntax highlight is useful to get elisp code easier to read, and
that's also true for code written into the eval-expression prompt.

>
> So I don't think there's anything to be done on the Emacs side here, but
> perhaps I'm misunderstanding what this report is about?

This report is a wish so that it's implemented for core emacs, so
font lock works by default in the eval expression prompt too, since
font lock is enabled by default in emacs for emacs lisp, so the
user rightly expects that it also works for code typed into
eval-expression.

The problem is as I wrote at the end of my previous mail, if you
follow my steps and enable font lock there then you see that font
locking elisp code does work, but then you can't quit the prompt
with C-g for some reason.

This is still a roadblock which should be investigated why this is.

In order to highlight elisp in eval-expression I changed the major
mode of the minibuffer to emacs-lisp-mode. Is that the reason?
If not so then what is? If so then why is changing the mode stops
C-g from working there?

So there are still roadblocks which have to be overcome to get
this feature work in emacs.

bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Lars Ingebrigtsen]

ndame <laszlomail@protonmail.com> writes:

> Naturally, I did not post the final solution. If you check the bug
> history they you can see that up to know it was not clear why font
> lock does not work in the eval-expression prompt:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=46092

You opened a new bug (bug#53507), so the context wasn't clear.  I'll
merge them.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>
> You opened a new bug (bug#53507), so the context wasn't clear. I'll
> merge them.

Thanks. I sent the reply to 46092@debbugs.gnu.org, but I also
CC'd emacs-bugs which apparently I shouldn't have done.

Sorry about the confusion.

bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Michael Heerdegen]

ndame via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs@gnu.org> writes:

> In order to highlight elisp in eval-expression I changed the major
> mode of the minibuffer to emacs-lisp-mode. Is that the reason?
> If not so then what is? If so then why is changing the mode stops
> C-g from working there?

C-g is part of the mode that is normally active in the minibuffer,
`minibuffer-mode'.

Michael.

bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

On Wednesday, January 26th, 2022 at 1:13 AM, Michael Heerdegen <michael_heerdegen@web.de> wrote:
> C-g is part of the mode that is normally active in the minibuffer,
>

Thanks, makes sense. I thought C-g was handled by some deeper layer.

So then I guess it's only a matter of merging the minibuffer keymap
with emacs lisp mode map in the eval-expression prompt.

bug#46092: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Drew Adams]

> So then I guess it's only a matter of merging the minibuffer
> keymap with emacs lisp mode map in the eval-expression prompt.

FWIW -

I know that this enhancement request is about having 
Lisp font-locking while typing a sexp to evaluate.

My remarks are only partly related, I guess, and are
offered as food for thought.

1. Font-locking a sexp you're typing isn't very
   important.

   Not saying it can't or shouldn't happen.  But
   compared to font-locking the _resulting value_,
   it's really not so important, IMHO.

2. Turning on `emacs-lisp-mode' for reading the sexp
   is overkill.  Unless you prevent it, it entails
   running the mode hook, which can do all kinds of
   things and might take a while.  It's likely to be
   problematic.

3. Although in principle it's only orthogonal, what
   I'd propose is to concentrate on the _result_
   sexp, instead of the input sexp to evaluate.

___

This is what I do, for #3.  I think it, or similar,
makes sense also for vanilla Emacs:

1. Bind `M-:' to `pp-eval-expression', not to
   `eval-expression'.  This means you can get the
   output in the echo area or a separate buffer,
   and that buffer can be better Lisp-enabled:
   font-lock, undo, `emacs-lisp-mode'.

2. Optionally show the result in a tooltip.
   Provide a user option, but commands can flip
   the behavior on the fly (with a prefix arg).

3. `pp-display-expression':

   * When result is shown in a buffer: Put
     buffer in `emacs-lisp-mode', with undo
     and font-locked. 
   * But use NO `emacs-lisp-mode-hook' or
     `change-major-mode-hook'.

   `pp-eval-expression':

   * Read the input sexp with completion.
     Completion uses a PP+ keymap, which is like
     `read-expression-map' but with some Elisp
     key bindings.
   * By default, use `pp-display-expression'.
     But with non-zero prefix arg insert result
     into current buffer, with or without
     double-quotes if result is a string (prefix
     arg < 0 means without).
   * Respect pp versions of print options:
     `pp-eval-expression-print-length',
     `pp-eval-expression-print-level', and
     `pp-max-tooltip-size'.  (No reason to
     force the same behavior for pretty-printing
     as for nonpretty-printing.)
   * Respect `eval-expression-debug-on-error'.
   * Return result of evaluation (as well as
     having it as car variable `values').

The code for pp+.el is here:

https://www.emacswiki.org/emacs/download/pp%2b.el

bug#46092: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Michael Heerdegen]

Drew Adams <drew.adams@oracle.com> writes:

> 1. Font-locking a sexp you're typing isn't very
>    important.

Agreed.  Doesn't harm OTOH.

> 2. Turning on `emacs-lisp-mode' for reading the sexp
>    is overkill.  Unless you prevent it, it entails
>    running the mode hook, which can do all kinds of
>    things and might take a while.  It's likely to be
>    problematic.

The advantage would be that the user gets all his related settings
gratis.  But I guess making that the default is not acceptable anyway.

> 3. Although in principle it's only orthogonal, what
>    I'd propose is to concentrate on the _result_
>    sexp, instead of the input sexp to evaluate.

The minibuffer part is also important.  We do have completion and eldoc
when reading input, which is already very good.  Font-Lock is a minor
issue but is still helpful.  A part that currently is not that well
covered in the minibuffer, in my opinion, is multi-line expression
editing.

Michael.

bug#53507: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Drew Adams]

> A part that currently is not that well covered in the
> minibuffer, in my opinion, is multi-line expression editing.

I suppose just multiline editing is the main thing
that needs love - i.e., independent of what the
input might be.

As for multiline Lisp expression editing (as opposed
to better support for multiline input in general), I
guess what you mean is support for indentation and
operations on sexps, e.g. motion.  And you mentioned
font-locking.  Is there something more than that?
___

But as for `eval-expression' enhancements, I repeat
that IMO the main need for improvement is wrt the
output.  I think it's far more common to deal with a
complex output value than it is to input a complex
sexp for evaluation.

You'll no doubt point out that if we had better
input support users would use `M-:' to evaluate
more complex sexps. ;-)

Still, I think it'll remain the case that complex
sexps are eval'd more often in other ways (e.g.
`C-x C-e'), and that complex sexps will continue to
be more common on output than on input.
___

Yes, my comments here are no doubt a bit of a
distraction from the request - sorry.  But we could
have better treatment of the output immediately.
And we could bind `M-:' to `pp-eval-expression' by
default, now.

bug#46092: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

>  Font-locking a sexp you're typing isn't very
>   important.

It may not be important for you, because you type
only simple expressions into eval-expression , but
others can use it differently.

I often paste multiline elisp snippets (e.g. 8-10
lines) into eval-expression which operate on the
current buffer (transforming content, etc). I'm
most of the time interestested in buffer changes,
not the result of the expression.

Such a snippet often needs small tewakings depending
on the current buffer content. In this case I
retrieve the snippet from the history of the eval
expression prompt and edit it right the in the
minibuffer.

When editing multiline snippets there, font locking
is important, because it makes it easier to read the
code (strings, etc.), just like in case of code  in
regular elisp buffers.

bug#53507: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Drew Adams]

> >  Font-locking a sexp you're typing isn't very
> >  important.
> 
> It may not be important for you, because you type
> only simple expressions into eval-expression , but
> others can use it differently.

I meant that, relative to the need for
reasonable support of the _output_
(examination, editing, undo), it's less
important, IMO.

But I wandered from the topic by bringing
up the output.  Sorry for that.

> I often paste multiline elisp snippets (e.g. 8-10
> lines) into eval-expression which operate on the
> current buffer (transforming content, etc). I'm
> most of the time interestested in buffer changes,
> not the result of the expression.

Yes, that's a reasonable use case as well.

Personally, even when I use `M-:' mainly for
a side effect, I'm often interested in the
resulting value as well.

The `M-:' dialog could be improved in both
ways.  And a start for output enhancement
is to bind it to `pp-eval-expression' by
default, instead of `eval-expression'.

> Such a snippet often needs small tewakings depending
> on the current buffer content. In this case I
> retrieve the snippet from the history of the eval
> expression prompt and edit it right the in the
> minibuffer.

I understand.

> When editing multiline snippets there, font locking
> is important, because it makes it easier to read the
> code (strings, etc.), just like in case of code  in
> regular elisp buffers.

Yes, understood.

bug#53507: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[ndame via Bug reports for GNU Emacs, the Swiss army knife of text editors]

> Personally, even when I use `M-:' mainly for a side effect, I'm often  > interested in the resulting value as well. The` M-:' dialog could be
> improved in both ways.

I agree that when eval-expression outputs a non-trivial expression
then the output should be formatted properly and maybe it should
be popped to a  buffer, because the echo area is not suitable
for examining a complex expression when it disappears with the
next command.

Why don't you open a separate bug about this?

bug#46092: [External] : bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Drew Adams]

> Why don't you open a separate bug about this?

Took me a while, but I've done that (#53592).

(Probably just another waste of time, though.
And it's possible that I already went through
all of this previously.  Dunno.)

bug#53507: bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Michael Heerdegen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So I don't think there's anything to be done on the Emacs side here, but
> perhaps I'm misunderstanding what this report is about?

Do we want to support syntax highlighting in eval-expression by default?
I would vote for it.

Michael.

bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Do we want to support syntax highlighting in eval-expression by default?
> I would vote for it.

I think that might be nice, but I'd have to see how it works in practice
first.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#53507: bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Michael Heerdegen]

[-- Attachment #1: Type: text/plain, Size: 291 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> > Do we want to support syntax highlighting in eval-expression by default?
> > I would vote for it.
>
> I think that might be nice, but I'd have to see how it works in practice
> first.

Privately I am using a hack like this for quite a while:


[-- Attachment #2: read-expr.el --]
[-- Type: application/emacs-lisp, Size: 1297 bytes --]

[-- Attachment #3: Type: text/plain, Size: 41 bytes --]


and I am quite happy with it.

Michael.

bug#53507: bug#46092: bug#53507: 27.1; syntax highlight in the eval-expression prompt

[Lars Ingebrigtsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Privately I am using a hack like this for quite a while:

After playing with it for a few minutes, it seems very nice.  Perhaps
push it to master (with a user option to switch it on/off)?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no