bug#36358: Indentation of not matched braces in latex-mode

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

If I use 'fill-paragraph' (M-q) on paragraph, inside things like
\footnote{} or \emph{}, if they're long, text is indented with 2
spaces - look below.  I was able to overcome this by setting
'tex-indent-basic' from 2 to 0, but I think it will affect other
indentations I may want.  Perhaps adding another variable to control
this kind of indentation would be good idea?  AUCTEX has:

      User Option: TeX-brace-indent-level
      - Number of spaces to add to the indentation for each '{' not
        matched by a '}'.

So maybe we could "borrow" it from them?

Lacus tincidunt ultrices.  Lorem ipsum dolor sit amet, consectetuer
adipiscing elit.\footnote{Quisque ullamcorper placerat ipsum.  Cras
   nibh.  Morbi vel justo vitae lacus tincidunt ultrices.  Lorem ipsum
   dolor sit amet, consectetuer adipiscing elit.}  In hac habitasse
platea dictumst.  Integer tempus convallis augue.  Etiam facilisis.
Nunc elementum fermentum wisi.  Integer tempus convallis augue.


S. U.


In GNU Emacs 26.2 (build 1, i686-w64-mingw32)
  of 2019-04-13 built on CIRROCUMULUS
Repository revision: fd1b34bfba8f3f6298df47c8e10b61530426f749
Windowing system distributor 'Microsoft Corp.', version 6.1.7601

bug#36358: Indentation of not matched braces in latex-mode

[Stefan Monnier]

> If I use 'fill-paragraph' (M-q) on paragraph, inside things like
> \footnote{} or \emph{}, if they're long, text is indented with 2
> spaces - look below.

I just pushed the patch below which lets you specify commands whose args
shouldn't be indented this way.

Note that the patch is for Emacs's bundled tex-mode, whereas you seem to
be using AUCTeX, which uses its own indentation code.


        Stefan


diff --git a/etc/NEWS b/etc/NEWS
index 74a8bbe8fa..0f764cd998 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -475,6 +475,9 @@ current and the previous or the next line, as before.
 \f
 * Changes in Specialized Modes and Packages in Emacs 27.1
 
+** tex-mode
+*** 'latex-noindent-commands' stops indenting arguments of \emph and friends
+
 ** byte compiler
 *** byte-compile-dynamic is now obsolete
 This is because on the one hand it suffers from various misbehaviors in corner
diff --git a/lisp/textmodes/tex-mode.el b/lisp/textmodes/tex-mode.el
index 9c91d27b94..4511354a3d 100644
--- a/lisp/textmodes/tex-mode.el
+++ b/lisp/textmodes/tex-mode.el
@@ -2803,9 +2803,19 @@ tex-indent-arg
 (defvar tex-indent-basic 2)
 (defvar tex-indent-item tex-indent-basic)
 (defvar tex-indent-item-re "\\\\\\(bib\\)?item\\>")
-(defvar latex-noindent-environments '("document"))
-(put 'latex-noindent-environments 'safe-local-variable
-     (lambda (x) (null (delq t (mapcar #'stringp x)))))
+(defcustom latex-noindent-environments '("document")
+  "Environments whose content is not indented by `tex-indent-basic'."
+  :type '(repeat string)
+  :safe (lambda (x) (lambda (x) (memq nil (mapcar #'stringp x))))
+  :group 'tex-file
+  :version "27.1")
+
+(defcustom latex-noindent-commands '("emph" "footnote")
+  "Commands for which `tex-indent-basic' should not be used."
+  :type '(repeat string)
+  :safe (lambda (x) (memq nil (mapcar #'stringp x)))
+  :group 'tex-file
+  :version "27.1")
 
 (defvar tex-latex-indent-syntax-table
   (let ((st (make-syntax-table tex-mode-syntax-table)))
@@ -2912,9 +2922,17 @@ latex-find-indent
 	       (current-column)
 	     ;; We're the first element after a hanging brace.
 	     (goto-char up-list-pos)
-	     (+ (if (and (looking-at "\\\\begin *{\\([^\n}]+\\)")
+	     (+ (if (if (eq (char-after) ?\{)
+                        (save-excursion
+                          (skip-chars-backward " \t")
+                          (let ((end (point)))
+                            (skip-chars-backward "a-zA-Z")
+                            (and (eq (char-before) ?\\)
+                                 (member (buffer-substring (point) end)
+                                         latex-noindent-commands))))
+                      (and (looking-at "\\\\begin *{\\([^\n}]+\\)")
 			 (member (match-string 1)
-				 latex-noindent-environments))
+				 latex-noindent-environments)))
 		    0 tex-indent-basic)
 		indent (latex-find-indent 'virtual))))
 	  ;; We're now at the "beginning" of a line.
diff --git a/test/manual/indent/latex-mode.tex b/test/manual/indent/latex-mode.tex
index 55c8e7033b..d314b98b48 100644
--- a/test/manual/indent/latex-mode.tex
+++ b/test/manual/indent/latex-mode.tex
@@ -8,4 +8,8 @@
 \url{/lib/modules/$(uname -r)},  %bug#11953.
 and install the appropriate \url{gspca-modules} package.
 
+Footnotes and emphasis shouldn't be indented \footnote{as can be seen here,
+for example}, \emph{or there
+as well}.
+
 \end{document}

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

> I just pushed the patch below which lets you specify commands whose
> args shouldn't be indented this way.

Hmmm... or I don't understand how "specify commands whose args
shouldn't be indented" will work in practice, or this may be wrong
path.  It's not about \emph{} or \footnote{}, it's about every command
which use '{}' (braces), so in latex... most of them(?).  Therefore if
you let people add them to the list it'll grow really big - just look
at font styles: \textrm{}, \textit{}, \textmd{}, \textbf{}, \textup{},
\textsl{}, \textsf{}, \textsc{}, \texttt{}, \textnormal{} - we could
probably use "\text*", but there are other commands, for example the
ones we define as new in preamble.

Unless this patch is temporary lifebuoy for those who don't want to
set 'tex-indent-basic' to 0, then well... carry on. :)

> Note that the patch is for Emacs's bundled tex-mode, whereas you
> seem to be using AUCTeX, which uses its own indentation code.

No, I'm _not_ using AUCTeX, I'm using latex-mode.  I only pointed to
AUCTeX, because when I was looking for answer to this problem, I found
solution (helpful internet) in AUCTeX - 'TeX-brace-indent-level'
variable - and wanted something similar in latex-mode.

Also I found code in theirs (AUCTeX) TEX.EL, which may help a bit.
It may, but since I basically know no Elisp, I may be wrong.

(defcustom TeX-brace-indent-level 2
   "*The level of indentation produced by an open brace."
   :group 'TeX-indentation
   :type 'integer)

(defun TeX-brace-count-line ()
   "Count number of open/closed braces."
   (save-excursion
     (let ((count 0) (limit (line-end-position)) char)
       (while (progn
	       (skip-chars-forward "^{}\\\\" limit)
	       (when (and (< (point) limit) (not (TeX-in-comment)))
		 (setq char (char-after))
		 (forward-char)
		 (cond ((eq char ?\{)
			(setq count (+ count TeX-brace-indent-level)))
		       ((eq char ?\})
			(setq count (- count TeX-brace-indent-level)))
		       ((eq char ?\\)
			(when (< (point) limit)
			  (forward-char)
			  t))))))
       count)))

This variable is also used in theirs LATEX.EL - L3366, L3503.  Both
functions too big to paste, I think.

bug#36358: Indentation of not matched braces in latex-mode

[Stefan Monnier]

> Hmmm... or I don't understand how "specify commands whose args
> shouldn't be indented" will work in practice, or this may be wrong
> path.  It's not about \emph{} or \footnote{}, it's about every command
> which use '{}' (braces), so in latex... most of them(?).  Therefore if
> you let people add them to the list it'll grow really big - just look
> at font styles: \textrm{}, \textit{}, \textmd{}, \textbf{}, \textup{},
> \textsl{}, \textsf{}, \textsc{}, \texttt{}, \textnormal{} - we could
> probably use "\text*", but there are other commands, for example the
> ones we define as new in preamble.

Most of the ones I define are commands for which I do want the args to
be indented.  I typically use them like this:

    prefix: \TAlign{
      foo \\
      bar
    }

where \TAlign expands to a kind of `tabular` environment.

Similarly, I want

    \section{foo bar baz
      toto titi tata}

rather than

    \section{foo bar baz
    toto titi tata}

> Unless this patch is temporary lifebuoy for those who don't want to
> set 'tex-indent-basic' to 0, then well... carry on. :)

It's not meant as a temporary solution.  More a tentative solution.

Thinking more about it, the main distinction is whether the command is
used "within a paragraph" or not.  But I don't really know how to
precisely characterize this notion of "within a paragraph".

>> Note that the patch is for Emacs's bundled tex-mode, whereas you
>> seem to be using AUCTeX, which uses its own indentation code.
> No, I'm _not_ using AUCTeX, I'm using latex-mode.  I only pointed to
> AUCTeX, because when I was looking for answer to this problem, I found
> solution (helpful internet) in AUCTeX - 'TeX-brace-indent-level'
> variable - and wanted something similar in latex-mode.

Good.

> Also I found code in theirs (AUCTeX) TEX.EL, which may help a bit.
> It may, but since I basically know no Elisp, I may be wrong.
> (defcustom TeX-brace-indent-level 2
[...]
> (defun TeX-brace-count-line ()

It seems to indent by 2 regardless of the command, just like what you
don't want.


        Stefan

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

I really don't like the solution, because it'll force people to add
environments/commands they don't want to be indented and it may be
long list.

I may be wrong but do we need to include environments?  Shouldn't they
be indented by default?  I guess it depends on preferences and because
it costs nothing why not have it as well...  but then again as you
wrote (see quote below) "(...) the main distinction is whether the
command is used "within a paragraph" or not (...)", so we don't really
need environments.

> Thinking more about it, the main distinction is whether the command
> is used "within a paragraph" or not.  But I don't really know how to
> precisely characterize this notion of "within a paragraph".

This is the main problem of this bug, i.e. how to make text "flow"
better in paragraph (environments can be indented).  For now we can
populate variable '*-commands' with commands commonly used inside
paragraph, like the one I wrote in previous message.

Idea: Maybe if its at the beginning of the line and there is no text
after '}' - indent? And if it has text after OR before OR is
surrounded by it - noindet.  This would fit for things like 'section',
'emph' would be indented as well but it would not be inside paragraph
but as a separate being, so it can be indented.  I didn't check other
commands/env so there could be exceptions...

> It seems to indent by 2 regardless of the command, just like what
> you don't want.

Because this is the default value, and with setq we can change it to 0.

Also when I byte compiled tex-mode.el with your changes, there was a
warning about unused 'x' or something like that - could it be one of
'lambda (x)' in 'defcustom latex-noindent-environments'?

Also I think declarations doesn't work(1) or are mixed(2):
{\small Quisque ullamcorper placerat ipsum.  Cras nibh.  Morbi
   vel justo vitae lacus tincidunt ultrices.  Lorem ipsum dolor sit
   amet, consectetuer adipiscing elit.}

{\small\emph{Quisque ullamcorper placerat ipsum.  Cras nibh.  Morbi
vel justo vitae lacus tincidunt ultrices.}  Lorem ipsum dolor sit
   amet, consectetuer adipiscing elit.}

bug#36358: Indentation of not matched braces in latex-mode

[Stefan Monnier]

> I may be wrong but do we need to include environments?

I don't understand the question.  AFAIK this discussion and the patch
I installed is independent from what happens to environments.

> Idea: Maybe if its at the beginning of the line and there is no text
> after '}' - indent? And if it has text after OR before OR is
> surrounded by it - noindet.  This would fit for things like 'section',
> 'emph' would be indented as well but it would not be inside paragraph
> but as a separate being, so it can be indented.  I didn't check other
> commands/env so there could be exceptions...

Hmm... I can try something like that.  Making indentation depend on text
after the } is a bit unorthodox (we generally refrain from making
indentation depend on text that comes later), but it might be
worthwhile here.

>> It seems to indent by 2 regardless of the command, just like what
>> you don't want.
> Because this is the default value, and with setq we can change it to 0.

Yes, of course, but I meant that it also does it for all commands.

> Also when I byte compiled tex-mode.el with your changes, there was a
> warning about unused 'x' or something like that - could it be one of
> 'lambda (x)' in 'defcustom latex-noindent-environments'?

I believe this was fixed a few hours later by Basil.
Do you still see it with the current code?

> Also I think declarations doesn't work(1) or are mixed(2):
> {\small Quisque ullamcorper placerat ipsum.  Cras nibh.  Morbi
>   vel justo vitae lacus tincidunt ultrices.  Lorem ipsum dolor sit
>   amet, consectetuer adipiscing elit.}
>
> {\small\emph{Quisque ullamcorper placerat ipsum.  Cras nibh.  Morbi
> vel justo vitae lacus tincidunt ultrices.}  Lorem ipsum dolor sit
>   amet, consectetuer adipiscing elit.}

Hmm... that looks wrong indeed: will have to investigate, thanks.


        Stefan

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

> I don't understand the question.  AFAIK this discussion and the
> patch I installed is independent from what happens to environments.

Sorry, for some reason I thought that `latex-noindent-environments'
was also added in this patch.

>> Idea: Maybe if its at the beginning of the line and there is no text
>> after `}' - indent?
>
> Hmm... I can try something like that.

Perhaps to solve the problem, you only need to write code for `indent'
condition and put something like - otherwise => noindent.  In this
case there would be no need to define `noindent' as text before/after
or surrounded by text - one of this would indicate that brackets are
inside paragraph.  This way we would made two cases: if separated =>
indent, if inside paragraph => noindent.  Or do the opposite define
`noindent' and otherwise - indent.

If this would work, we wouldn't need `latex-noindent-commands'.  But
there are probably quite a few ``styles'' of how people write latex, so
tests will be needed.

> Yes, of course, but I meant that it also does it for all commands.

So... this sounds like `tex-indent-basic'... maybe they limit the
range somehow later in the code.

> I believe this was fixed a few hours later by Basil.
> Do you still see it with the current code?

It's OK after the update.

>> ... about declarations...
>
> Hmm... that looks wrong indeed: will have to investigate, thanks.

The problem is that we have two constructs `\something{...}' and
`{\something...}' and if I understand correctly your patch after
reaching `{' moves back and compare word between `\' and `{' with the
list of `latex-noindent-commands' - if match then don't indent.  In
case of declaration, command is after `{' not before.

So, perhaps, how far it goes backward can be (already is?) limited to
SPC (& perhaps beginning of the line)?  Then if it won't find
backward, it should look forward again to SPC (& perhaps to EOL) and
compare word between `\' and SPC.

Another problem may arise from nested declarations.  As for them,
maybe add _second_ `\' as limit?  But the rest of declarations will be
ignored - or simply mention this nuisance in doc string.  Maybe look
for second `\' compare word, then to the next one `\' and compare,
until it'll find SPC instead of `\'.  Currently they indent by the
value of `tex-indent-basic' no mater how many of them is there.

Funny thing - currently (e26.2), each nested command adds value of
`tex-indent-basic' to the end value of indent, for example
(setq t-i-b 2):

\textbf{\textit{\textsc{hypertext links, including those to hypertext
       links, including those to external documents and URLs
       external}}} documents and URLs.

However, your patch works and adding any of them to commands list will
decrease indent by 2 (in this example), i.e. patch works for nested
commands.

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

In the meantime I think I found better solution to setting
'tex-indent-basic' to 0.  While leaving it at default value '2', I put
into my init.el this:

(add-hook 'tex-mode-hook
	  (lambda ()
	    (setq fill-indent-according-to-mode nil)))

aaand...  everything works like I wanted.  So maybe this is solution
to this bug?

bug#36358: Indentation of not matched braces in latex-mode

[Stefan Monnier]

> In the meantime I think I found better solution to setting
> 'tex-indent-basic' to 0.  While leaving it at default value '2', I put
> into my init.el this:
>
> (add-hook 'tex-mode-hook
> 	  (lambda ()
> 	    (setq fill-indent-according-to-mode nil)))
>
> aaand...  everything works like I wanted.  So maybe this is solution
> to this bug?

I think it's a workaround more than a solution.  If you ever hit TAB
within a paragraph, you'll see the undesired indentation again.


        Stefan

bug#36358: Indentation of not matched braces in latex-mode

[Basil L. Contovounesios]

Sebastian Urban <mrsebastianurban@gmail.com> writes:

> In the meantime I think I found better solution to setting
> 'tex-indent-basic' to 0.  While leaving it at default value '2', I put
> into my init.el this:
>
> (add-hook 'tex-mode-hook
> 	  (lambda ()
> 	    (setq fill-indent-according-to-mode nil)))
>
> aaand...  everything works like I wanted.  So maybe this is solution
> to this bug?

Note that fill-indent-according-to-mode is not automatically
buffer-local, so using setq instead of setq-local changes its value
globally.

-- 
Basil

bug#36358: Indentation of not matched braces in latex-mode

[Sebastian Urban]

So, I tried to write something that could work better:
1st method: noindent, unless indent; basically current approach
             reversed: indent, unless noindent;
2nd method: no text before AND after - indent, otherwise don't.

Although, I'm not sure, what to do with things like:
    Some paragraph...

    {\small Quisque ullamcorper placerat ipsum.  Cras nibh.  Morbi
      vel justo vitae lacus tincidunt ultrices.  Lorem ipsum dolor sit
      amet, consectetuer adipiscing elit.}

    Other paragraph...
if it's not inside of a paragraph, should it be indented?

BTW I did not test them thoroughly...

=======================================
1. Check if command should be indented.
=======================================

By default it won't indent commands, unless they are in
latex-indent-commands.  Commands like section or TAlign, will have to
be added to that var.  This could be done for the most common
commands, like mentioned "section", by default.  This, basically,
reverses current approach, and I think that it'll be easier (less
commands typed into var) to tell what should be indented, than what
shouldn't.  But I didn't use LaTeX extensively, so I may be wrong.

--8<---------------cut here---------------start------------->8---
--- old/tex-mode.el	2020-06-26 18:34:05.000000000 +0200
+++ new/tex-mode.el	2020-07-09 11:22:15.246268900 +0200
@@ -2802,8 +2802,8 @@
    :group 'tex-file
    :version "27.1")

-(defcustom latex-noindent-commands '("emph" "footnote")
-  "Commands for which `tex-indent-basic' should not be used."
+(defcustom latex-indent-commands '("section")
+  "Commands for which `tex-indent-basic' should be used."
    :type '(repeat string)
    :safe (lambda (x) (not (memq nil (mapcar #'stringp x))))
    :group 'tex-file
@@ -2915,17 +2915,20 @@
  	     ;; We're the first element after a hanging brace.
  	     (goto-char up-list-pos)
  	     (+ (if (if (eq (char-after) ?\{)
-                        (save-excursion
-                          (skip-chars-backward " \t")
-                          (let ((end (point)))
-                            (skip-chars-backward "a-zA-Z")
-                            (and (eq (char-before) ?\\)
-                                 (member (buffer-substring (point) end)
-                                         latex-noindent-commands))))
+                        (unless (save-excursion
+				  (skip-chars-backward " \t")
+				  (if (eq (char-before) ?\])
+				      (backward-list))
+				  (let ((end (point)))
+                                    (skip-chars-backward "a-zA-Z")
+                                    (and (eq (char-before) ?\\)
+					 (member (buffer-substring (point) end)
+						 latex-indent-commands))))
+                          t)
                        (and (looking-at "\\\\begin *{\\([^\n}]+\\)")
-			 (member (match-string 1)
-				 latex-noindent-environments)))
-		    0 tex-indent-basic)
+                           (member (match-string 1)
+                                   latex-noindent-environments)))
+                    0 tex-indent-basic)
  		indent (latex-find-indent 'virtual))))
  	  ;; We're now at the "beginning" of a line.
  	  ((not (and (not virtual) (eq (char-after) ?\\)))
--8<---------------cut here---------------end--------------->8---


=======================================================
2. Check if command/declaration(?) is within paragraph.
=======================================================

This checks if there is nothing but "beginning of the line" (ignoring
whitespaces) before command or "{" AND nothing but "end of the line"
(ignoring whitespaces) after "}" - if "yes" then indent, otherwise
don't.

The good thing about this, is that we don't need
latex-noindent-commands.  The bad thing is, that constructs like:
    prefix: \TAlign{
      foo\\
      bar
    }
won't work, because of the "prefix:" at BOL - it'll think it's
a paragraph.

If, "not-in-paragraph" declarations, like:
    {\delaration Integer tempus convallis augue.  Etiam facilisis.
    Nunc elementum fermentum wisi.  Integer tempus convallis augue.}
should NOT be indented (like in example above), then:
- uncomment: (skip-chars-backward " \t");
- delete: (eq (char-after) ?\{) AND "or" from first condition of "and".

--8<---------------cut here---------------start------------->8---
--- old/tex-mode.el	2020-06-26 18:34:05.000000000 +0200
+++ new/tex-mode.el	2020-07-09 12:42:14.067428300 +0200
@@ -2802,13 +2802,6 @@
    :group 'tex-file
    :version "27.1")

-(defcustom latex-noindent-commands '("emph" "footnote")
-  "Commands for which `tex-indent-basic' should not be used."
-  :type '(repeat string)
-  :safe (lambda (x) (not (memq nil (mapcar #'stringp x))))
-  :group 'tex-file
-  :version "27.1")
-
  (defvar tex-latex-indent-syntax-table
    (let ((st (make-syntax-table tex-mode-syntax-table)))
      (modify-syntax-entry ?$ "." st)
@@ -2915,17 +2908,22 @@
  	     ;; We're the first element after a hanging brace.
  	     (goto-char up-list-pos)
  	     (+ (if (if (eq (char-after) ?\{)
-                        (save-excursion
-                          (skip-chars-backward " \t")
-                          (let ((end (point)))
-                            (skip-chars-backward "a-zA-Z")
-                            (and (eq (char-before) ?\\)
-                                 (member (buffer-substring (point) end)
-                                         latex-noindent-commands))))
+			(not (and (save-excursion
+				    ;; (skip-chars-backward " \t")
+				    (cond ((eq (char-before) ?\]) (backward-list)))
+				    (skip-chars-backward "a-zA-Z")
+				    (cond ((or (eq (char-before) ?\\)
+					       (eq (char-after) ?\{))
+					   (skip-chars-backward " \t\\\\")
+					   (bolp))))
+				  (save-excursion
+				    (forward-list)
+				    (skip-chars-forward " \t")
+				    (eolp))))
                        (and (looking-at "\\\\begin *{\\([^\n}]+\\)")
-			 (member (match-string 1)
-				 latex-noindent-environments)))
-		    0 tex-indent-basic)
+                           (member (match-string 1)
+                                   latex-noindent-environments)))
+                    0 tex-indent-basic)
  		indent (latex-find-indent 'virtual))))
  	  ;; We're now at the "beginning" of a line.
  	  ((not (and (not virtual) (eq (char-after) ?\\)))
--8<---------------cut here---------------end--------------->8---


=============================
? Indentation in itemize env.
=============================

I'm not sure how it suppose to look, but when in example below
    \begin{itemize}#
      \item foo*
      \item bar
    \end{itemize}
the cursor is where "#" is, and I press <RET> - cursor moves to the
next line and is auto-indented by the value of tex-indent-basic.  But
when the cursor is at the position of "*", and I press <TAB>, the
indentation is gone (zero).  The question is, should \item(s) be
indented or not?  If they should, maybe code below will help.

--8<---------------cut here---------------start------------->8---
--- old/tex-mode.el	2020-06-26 18:34:05.000000000 +0200
+++ new/tex-mode.el	2020-07-09 11:04:11.583564000 +0200
@@ -2880,7 +2880,13 @@
         ;; Default (maybe an argument)
         (let ((pos (point))
  	     ;; Outdent \item if necessary.
-	     (indent (if (looking-at tex-indent-item-re) (- tex-indent-item) 0))
+	     (indent (if (looking-at tex-indent-item-re)
+			 (if (save-excursion
+			       (forward-line -1)
+			       (beginning-of-line)
+			       (looking-at "\\\\begin *{\\([^\n}]+\\)"))
+			     0 (- tex-indent-item))
+		       0))
  	     up-list-pos)
  	 ;; Find the previous point which determines our current indentation.
  	 (condition-case err
--8<---------------cut here---------------end--------------->8---


Anyway, I hope it'll help... somehow.

S. U.