bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Klaus-Dieter Bauer]

[-- Attachment #1.1: Type: text/plain, Size: 3601 bytes --]

Hello!

In `emacs-lisp-mode', the counting of the nesting level of forms seems
to be broken in some subtle way. Consider e.g.

    (form
     (
      ) WEIRD HIGHLIGHT
        x) ;; Unexepected Indentation

It looks like "WEIRD HIGHLIGHT" is wrongly highlighted as junk after a
surplus closing parenthesis, and the subsequent form is also weirdly
indented.

So this particular example is artificial, it occurred for me when I was
writing testing-code with in some personal emacs-lisp project:

    ;; Example from actual code (abbreviated)
    (ert-deftest expr--parser-rule-def-firsts ()
      (myert-test-results #'expr--parser-rule-def-firsts
        `("The `firsts' function doesn't yet enforce, that every rule
          must have a resolvable first."
          (((&def RuleA RuleB)
            (&def RuleB RuleA))) => WEIRD-HIGHLIGHT-HERE
            :WEIRD-INDENTATION-HERE)))

I confirmed the issue in Emacs 25.1.1 64bit (from Chocolatey), running
that same Emacs with "runemacs -q", and with Emacs 26.0.50.1
(self-compiled).

For reference I have attached an elisp file with examples and a
screenshot of how it looks on my system.

regards, Klaus

--------------------------------------------------

In GNU Emacs 25.1.1 (x86_64-w64-mingw32)
 of 2016-09-17 built on KAEL
Windowing system distributor 'Microsoft Corp.', version 10.0.14393
Configured using:
 'configure --prefix=/tmp/emacs --without-imagemagick 'CFLAGS=-O2
 -fomit-frame-pointer -g0''

Configured features:
XPM JPEG TIFF GIF PNG RSVG SOUND DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB
TOOLKIT_SCROLL_BARS

Important settings:
  value of $LANG: DEA
  locale-coding-system: cp1252

Major mode: Emacs-Lisp

Minor modes in effect:
  diff-auto-refine-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message dired format-spec rfc822 mml
mml-sec password-cache epg epg-config gnus-util mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail
rfc2047 rfc2045 ietf-drums mm-util help-fns help-mode cl-loaddefs pcase
cl-lib mail-prsvr mail-utils vc-git diff-mode easymenu easy-mmode
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel dos-w32 ls-lisp disp-table w32-win w32-vars
term/common-win tool-bar dnd fontset image regexp-opt fringe
tabulated-list newcomment elisp-mode lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese charscript case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer cl-preloaded nadvice loaddefs button faces
cus-face macroexp files text-properties overlay sha1 md5 base64 format
env code-pages mule custom widget hashtable-print-readable backquote
w32notify dbusbind w32 multi-tty make-network-process emacs)

Memory information:
((conses 16 94172 5940)
 (symbols 56 20219 0)
 (miscs 48 89 106)
 (strings 32 17676 3993)
 (string-bytes 1 489683)
 (vectors 16 12293)
 (vector-slots 8 433157 4526)
 (floats 8 164 100)
 (intervals 56 581 40)
 (buffers 976 20))

--=-=-=--

[-- Attachment #1.2: Type: text/html, Size: 9421 bytes --]

[-- Attachment #2: emacs-indent-bug.el --]
[-- Type: application/octet-stream, Size: 817 bytes --]

;;; -*- mode: emacs-lisp; coding: utf-8-unix -*-

() => x ;; No weird coloring
(
 ) this is colored unexpectedly
;; indent is fine here

`(form
  ((() 
    ())) => 
    x);; Unexepected indentation, by two spaces too much.

(form
 (
  ) WEIRD HIGHLIGHT
    x) ;; Unexepected indentation in a different way.

`(form 
  (
   x) =>
   x) ;; Unexpected indentation in a different way


    ;; Example from actual code (abbreviated)
    (ert-deftest expr--parser-rule-def-firsts ()
      (myert-test-results #'expr--parser-rule-def-firsts
        `("The `firsts' function doesn't yet enforce, that every rule
          must have a resolvable first."
          (((&def RuleA RuleB) 
            (&def RuleB RuleA))) => WEIRD-HIGHLIGHT-HERE
            :WEIRD-INDENTATION-HERE)))


) Here the coloring is probably intended.

[-- Attachment #3: emacs-indent-bug.png --]
[-- Type: image/png, Size: 44175 bytes --]

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Philipp Stephani]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

Klaus-Dieter Bauer <bauer.klaus.dieter@gmail.com> schrieb am Sa., 26. Nov.
2016 um 00:13 Uhr:

> Hello!
>
> In `emacs-lisp-mode', the counting of the nesting level of forms seems
> to be broken in some subtle way. Consider e.g.
>
>     (form
>      (
>       ) WEIRD HIGHLIGHT
>         x) ;; Unexepected Indentation
>
> It looks like "WEIRD HIGHLIGHT" is wrongly highlighted as junk after a
> surplus closing parenthesis, and the subsequent form is also weirdly
> indented.
>
>
This is working as intended (i.e. not a bug). Lisp-mode explicitly tests
for this. When you hover over the highlighted part, you get a tooltip
"Hidden behind deeper element; move to another line?"
While not a syntax error, there's such a strong convention to avoid such
formatting that the Lisp modes warn unconditionally about it.

[-- Attachment #2: Type: text/html, Size: 2463 bytes --]

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Noam Postavsky]

tags 25030 + wontfix
quit

Philipp Stephani <p.stephani2@gmail.com> writes:

> This is working as intended (i.e. not a bug). Lisp-mode explicitly
> tests for this. When you hover over the highlighted part, you get a
> tooltip "Hidden behind deeper element; move to another line?" 
> While not a syntax error, there's such a strong convention to avoid
> such formatting that the Lisp modes warn unconditionally about it.

Since there's been no further movement on this, I'm closing as wontfix.

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Andy Moreton]

On Sun 27 Nov 2016, Philipp Stephani wrote:

> Klaus-Dieter Bauer <bauer.klaus.dieter@gmail.com> schrieb am Sa., 26. Nov.
> 2016 um 00:13 Uhr:
>
>> Hello!
>>
>> In `emacs-lisp-mode', the counting of the nesting level of forms seems
>> to be broken in some subtle way. Consider e.g.
>>
>>     (form
>>      (
>>       ) WEIRD HIGHLIGHT
>>         x) ;; Unexepected Indentation
>>
>> It looks like "WEIRD HIGHLIGHT" is wrongly highlighted as junk after a
>> surplus closing parenthesis, and the subsequent form is also weirdly
>> indented.
>>
>>
> This is working as intended (i.e. not a bug). Lisp-mode explicitly tests
> for this. When you hover over the highlighted part, you get a tooltip
> "Hidden behind deeper element; move to another line?"
> While not a syntax error, there's such a strong convention to avoid such
> formatting that the Lisp modes warn unconditionally about it.

This may not be a bug, but it is certainly a mis-feature.

Warning should be reserved for syntax which may have unintended or
surprising semantics. Indentation that does not follow a convention is
not wrong either systacically or semantically.

Please remove this broken mis-feature.

    AndyM

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Drew Adams]

> This may not be a bug, but it is certainly a mis-feature.
> 
> Warning should be reserved for syntax which may have unintended or
> surprising semantics. Indentation that does not follow a convention is
> not wrong either systacically or semantically.
> 
> Please remove this broken mis-feature.

I agree that warnings are not for such things.  Emacs too
often uses "warnings" for things that are not warnings.

On the other hand, I do appreciate this highlighting, though
at first I didn't think I would.  I think a different face
should be used for this - this is *warning* about anything.
That would let users control whether it is actually highlighted
(e.g., by resetting the face attributes to nil).

This is a duplicate of bug #18163, BTW.  And the thread
repeats that one...

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Noam Postavsky]

unarchive 18163
merge 25030 18163
quit

Andy Moreton <andrewjmoreton@gmail.com> writes:

> This may not be a bug, but it is certainly a mis-feature.
>
> Warning should be reserved for syntax which may have unintended or
> surprising semantics. Indentation that does not follow a convention is
> not wrong either systacically or semantically.

I'm not convinced by this.  Code with unconventional indendation has
surprising syntax to a human reader (or from another perspective, when
I'm writing code which indents strangely, that clues me in that I've
written some unintended syntax), therefore, it seems a warning is
exactly appropriate.

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Andy Moreton]

On Mon 19 Mar 2018, Noam Postavsky wrote:

> unarchive 18163
> merge 25030 18163
> quit
>
> Andy Moreton <andrewjmoreton@gmail.com> writes:
>
>> This may not be a bug, but it is certainly a mis-feature.
>>
>> Warning should be reserved for syntax which may have unintended or
>> surprising semantics. Indentation that does not follow a convention is
>> not wrong either systacically or semantically.
>
> I'm not convinced by this.  Code with unconventional indendation has
> surprising syntax to a human reader (or from another perspective, when
> I'm writing code which indents strangely, that clues me in that I've
> written some unintended syntax), therefore, it seems a warning is
> exactly appropriate.

I disagree. The interpreter and byte compiler do not care about the
indentation style that you choose for your code: the syntax and
semantics are unaffected.

Style choices should not produce warnings. An indication that code layout is
following an unusual style may be useful, but it should be optional, and
it should not use the warning face (it should have a separate face that can be
customised independently of the warning face).

    AndyM

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Drew Adams]

> >> This may not be a bug, but it is certainly a mis-feature.
> >>
> >> Warning should be reserved for syntax which may have unintended or
> >> surprising semantics. Indentation that does not follow a convention is
> >> not wrong either systacically or semantically.
> >
> > I'm not convinced by this.  Code with unconventional indendation has
> > surprising syntax to a human reader (or from another perspective, when
> > I'm writing code which indents strangely, that clues me in that I've
> > written some unintended syntax), therefore, it seems a warning is
> > exactly appropriate.
> 
> I disagree. The interpreter and byte compiler do not care about the
> indentation style that you choose for your code: the syntax and
> semantics are unaffected.
> 
> Style choices should not produce warnings. An indication that code layout
> is following an unusual style may be useful, but it should be optional, and
> it should not use the warning face (it should have a separate face that can
> be customised independently of the warning face).

What Andy said.  This has nothing to do with byte-compiling
(or interpreting, for that matter).

There is nothing wrong with having optional (especially opt-in)
indications of flouting conventional style.  And even then we
should not use, or inherit from, the warning face.

It should be easy for users to give the face(s) used for
stylistic highlighting different appearance(s) from standard
Emacs faces that have other meanings.

Error and warning faces are to be avoided for anything that is
not an error or warning.  And any faces used by the byte-compiler
should be about something relevant to byte-compiling.

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Noam Postavsky]

Drew Adams <drew.adams@oracle.com> writes:

>> I disagree. The interpreter and byte compiler do not care about the
>> indentation style that you choose for your code: the syntax and
>> semantics are unaffected.
>> 
>> Style choices should not produce warnings. An indication that code layout
>> is following an unusual style may be useful, but it should be optional, and
>> it should not use the warning face (it should have a separate face that can
>> be customised independently of the warning face).
>
> What Andy said.  This has nothing to do with byte-compiling
> (or interpreting, for that matter).

I don't understand why you guys are all of a sudden talking about
byte-compiling.

> It should be easy for users to give the face(s) used for
> stylistic highlighting different appearance(s) from standard
> Emacs faces that have other meanings.

Anyway, if this just about reorganizing the faces used, I have no
objections.

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Andy Moreton]

On Mon 19 Mar 2018, Noam Postavsky wrote:

> Drew Adams <drew.adams@oracle.com> writes:
>
>>> I disagree. The interpreter and byte compiler do not care about the
>>> indentation style that you choose for your code: the syntax and
>>> semantics are unaffected.
>>> 
>>> Style choices should not produce warnings. An indication that code layout
>>> is following an unusual style may be useful, but it should be optional, and
>>> it should not use the warning face (it should have a separate face that can
>>> be customised independently of the warning face).
>>
>> What Andy said.  This has nothing to do with byte-compiling
>> (or interpreting, for that matter).
>
> I don't understand why you guys are all of a sudden talking about
> byte-compiling.

The point is that the unusual code layout does not cause any problems for
elisp code, and it does not matter if the code is interpreted or
byte-compiled: the result is the same, namely that indentation style has
no effect on the meaning of the code.

    AndyM

bug#25030: 25.1; Unexpected indentation and syntax-highlighting in `emacs-lisp-mode'

[Noam Postavsky]

retitle 25030 elisp: highlighting of unexpected indentation should use separate face from highlight of error functions
severity 25030 wishlist
tags 25030 =
reopen 25030
quit

Andy Moreton <andrewjmoreton@gmail.com> writes:

> The point is that the unusual code layout does not cause any problems for
> elisp code, and it does not matter if the code is interpreted or
> byte-compiled: the result is the same, namely that indentation style has
> no effect on the meaning of the code.

This seems like an argument against the byte compiler emitting style
warnings, but since this bug is not about byte compiler warnings, I
don't understand why you are bringing it up.

bug#25030: elisp: highlighting of unexpected indentation should use separate face from highlight of error functions

[Lars Ingebrigtsen]

Philipp Stephani <p.stephani2@gmail.com> writes:

> This is working as intended (i.e. not a bug). Lisp-mode explicitly tests for this. When
> you hover over the highlighted part, you get a tooltip "Hidden behind deeper
> element; move to another line?" 

So I think this is working as designed, and I'm therefore closing this
bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no