My humble additions to AUCTeX

My humble additions to AUCTeX

[Marcin Borkowski]

Hi all,

I've recently started putting together some functions to enhance
AUCTeX.  Currently, a very limited set is ready (getting info about
the "token" (in the TeX sense) at point and movement by tokens), and
this is not really meant for an average B. L. User yet (there are no
keybindings). Still, I thought I might share it:
https://github.com/mbork/tex-plus.el .  As mentioned in the readme
file, I plan to add a few things.  Currently I'm thinking about
replacing M-f, M-d etc. with commands acting on tokens if at a control
word and on words otherwise (so that e.g. an unescaped backslash at the
beginning is treated as the part of "word").  Later, I'd like to
implement some functions to work with \left ... \right, \bigl ... \bigr
and similar delimiters.

I'd be very thankful for any opinions/thoughts, bug reports or feature
requests. I have very limited spare time now, but I'll have more of it
in a few weeks, so I'll be able to devote more time to work on this.
Also, I still consider myself an Elisp newbie, so it is well possible
that I did violate some conventions ar good style...

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: My humble additions to AUCTeX

[Nicolas Richard]

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:

> Hi all,
>
> I've recently started putting together some functions to enhance
> AUCTeX.  Currently, a very limited set is ready (getting info about
> the "token" (in the TeX sense) at point and movement by tokens)

Could you state some of the problems you are solving ? ISTR
token-by-token motion was a problem for me at some point in the past,
but I think these problems disappeared. I use C-M-f and C-M-b.

-- 
Nicolas.

Re: My humble additions to AUCTeX

[Marcin Borkowski]

Dnia 2014-01-13, o godz. 17:49:55
Nicolas Richard <theonewiththeevillook@yahoo.fr> napisał(a):

> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
> 
> > Hi all,
> >
> > I've recently started putting together some functions to enhance
> > AUCTeX.  Currently, a very limited set is ready (getting info about
> > the "token" (in the TeX sense) at point and movement by tokens)
> 
> Could you state some of the problems you are solving ? ISTR
> token-by-token motion was a problem for me at some point in the past,
> but I think these problems disappeared. I use C-M-f and C-M-b.

Do I guess correctly that you did some (or more) LaTeX typesetting, but
not too much low-level TeX programming, dear Watson? ;)

Imagine this: <!>\these\are\four\tokens, where <!> denotes the point.
Then, C-M-f moves too far; I want to move by one token (i.e. to get
\these<!>\are\four\tokens), not by a whole string of them at the same
level of curly braces.

Also, I want to level up the word commands to token-aware commands.
Imagine this: \these\are\four\tokens<!>.  Now, if you press M-DEL, the
backslash does not get deleted; if I made its syntax "word", pressing
M-DEL would kill *everything* shown here. And I want to be able to kill
just the string "\tokens" with one keystroke.

Notice that, as I hinted at the beginning, strings of consecutive TeX
commands are rather rare in LaTeX documents; but when doing low-level
TeX programming, such situations are *extremely* common, and I'd find
more fine-grained control over movement very useful then.

Last but not least, I aim at commands to work on matching pairs of
\left...\right, \bigl...\bigr etc. delimiters.  Therefore, I want to be
able to walk through the text token by token.  (I did look into
smartparen, and it doesn't seem to be sophisticated enough to do what I
need.)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: My humble additions to AUCTeX

[Stefan Monnier]

> Also, I want to level up the word commands to token-aware commands.

That'd be a mistake.  In Emacs, a "word" is not a "token".

For those people who prefer symbol-navigation over word-navigation,
Emacs-24.4 offers superword-mode.  And indeed, it would be nice to make
superword-mode understand TeX's notion of "symbol".


        Stefan

Re: My humble additions to AUCTeX

[Marcin Borkowski]

Dnia 2014-01-13, o godz. 15:47:46
Stefan Monnier <monnier@iro.umontreal.ca> napisał(a):

> > Also, I want to level up the word commands to token-aware commands.
> 
> That'd be a mistake.  In Emacs, a "word" is not a "token".

Could you elaborate on this?  IMHO, both "word" and "\token" are
somehow word-ish in a LaTeX file.  And to clarify: I want my word
commands to act normally *unless* the point is on a token starting with
a backslash, and only then treat it as a word.  Does that sound better?

> For those people who prefer symbol-navigation over word-navigation,
> Emacs-24.4 offers superword-mode.  And indeed, it would be nice to
> make superword-mode understand TeX's notion of "symbol".

Thanks, I'll definitely look into it, though I'm using Emacs 24.3 now.

>         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: My humble additions to AUCTeX

[Marcin Borkowski]

Dnia 2014-01-14, o godz. 10:57:20
Marcin Borkowski <mbork@wmi.amu.edu.pl> napisał(a):

> Could you elaborate on this?
> [...]
> Thanks, I'll definitely look into it, though I'm using Emacs 24.3 now.

OK, so I did look at your discussion with Ted about superword-mode, and
I saw that you already *did* elaborate on this in that thread.

What is interesting in the (La)TeX case, is that you have human
language interspersed with a programming language.  In case of
documents, the human part is much larger; in case of packages/classes,
the programming part is much larger.  Maybe it's wrong that AUCTeX uses
the same mode for both.  Anyway, my plan now is to postpone work on
"tokens-as-words" functions (well, just for my personal setup, I'll
redefine M-f & friends;)), and wait for some more free time to study
Emacs 24.4's superword mode.

Thanks again for your suggestion!

> >         Stefan

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: My humble additions to AUCTeX

[Stefan Monnier]

>> > Also, I want to level up the word commands to token-aware commands.
>> That'd be a mistake.  In Emacs, a "word" is not a "token".
> Could you elaborate on this?

Emacs's "word" is "a sequence of alphanumeric characters".  Always.

Its behavior is meant to reflect the "human text" notion of a "word"
rather than the related notion of a "token" from the current
programming language.  So the behavior of word-commands should not be
affected by the major mode.


        Stefan

Re: My humble additions to AUCTeX

[Marcin Borkowski]

On 2014-01-13, at 21:47, Stefan Monnier <monnier@iro.umontreal.ca> wrote:

>> Also, I want to level up the word commands to token-aware commands.
>
> That'd be a mistake.  In Emacs, a "word" is not a "token".
>
> For those people who prefer symbol-navigation over word-navigation,
> Emacs-24.4 offers superword-mode.  And indeed, it would be nice to make
> superword-mode understand TeX's notion of "symbol".

OK, so let me revive this (over a year!) old thread.  (Last time I wrote
in it, I said that I'd look into new Emacs' superword-mode.  Well,
I just did.)

Here's the catch (and the summary of what I want to accomplish).  I'd
like M-f and M-b treat \these \things \as \words in AUCTeX mode.  The
rationale is that they are really the basic components of a (La)TeX
code.

Now the advice Stefan gave me was to utilize superword-mode.  I can see
a few problems with this approach.  First is subjective: I looked into
the code, and have no idea how find-word-boundary-function-table works.
From what I /did/ understand is that in superword-mode, M-f (somehow?)
calls (forward-symbol 1) and M-b calls (forward-symbol -1).

Now, I could somehow substitute my function for forward-symbol (assuming
that I will understand how that table works).

However, as I alluded in one of my previous posts, in LaTeX files the
code and (human-language) text are heavily interspersed.  Take this
line, for instance:

I \emph{really} like \LaTeX.

Assume that the point is at the beginning of line (I will denote it by
the vertical line, |.)  After a few subsequent presses of M-f, this is
what I would like to achieve:

I| \emph{really} like \LaTeX.
I \emph|{really} like \LaTeX.
I \emph{really|} like \LaTeX.
I \emph{really} like| \LaTeX.
I \emph{really} like \LaTeX|.

This is in fact how Emacs operates by default.  However, imagine that
I start with the point at the end of line, and press M-b repeatedly.
Here's what I'd like to see:

I \emph{really} like \LaTeX.
I \emph{really} like |\LaTeX.
I \emph{really} |like \LaTeX.
I \emph{|really} like \LaTeX.
I |\emph{really} like \LaTeX.

As you can see, it's different from the normal Emacs operation.

OTOH, this@is@treated@as@one@symbol in AUCTeX; but email@address
shouldn't.  So when the point is after `foo@bar', and the user presses
M-b, we should land at the `b'; but when the point if after `\foo@bar',
and the user presses M-b, we should land at the backslash.  (In fact,
it's even more complicated than that, with the \makeatletter and stuff,
but let's forget about it now.)

Yet another example: I'd like M-b to move to the backslash when the
point is after the comma in `foo\,'.  Normally, both (backward-word) and
(forward-symbol -1) travel all the way to the `f'.

In fact, I would also like a (single) M-f or M-b to skip the /entire/
math formula ($...$ or \(...\)) when I'm on its delimiter (this is
probably even more unorthodox, but would be very handy: M-f and M-b
would allow me to move between parts of the formula when I'm inside, but
would treat it as a single entity when I'm outside!)

Now I could more or less easily code these functions and e.g. define
a suitable minor mode.  My question is not how to do this, but /where/
to put my functions, so to speak.  IOW, what is the best/suggested
practice here?  Should I just turn it on in AUCTeX mode?  (Probably not
the best idea ever.)  Should I utilize superword mode?  (Might not be
a good idea either, since it might not work the way I want it to
/outside/ TeX commands.)  Should I define my own minor mode where
I redefine M-f and M-b?  (And M-d etc.?  This seems the best solution to
me.)

>         Stefan

Best,

-- 
Marcin Borkowski               This email was proudly sent
http://mbork.pl                from my Emacs.

Re: My humble additions to AUCTeX

[Artur Malabarba]

>Yet another example: I'd like M-b to move to the backslash when the point
is after the comma in `foo\,'.  Normally, both (backward-word) and
(forward-symbol -1) travel all the way to the `f'.

This is the problem I had as well, when trying to add superword mode to
AucTex. This is ultimately a limitation of emacs syntax table. AFAIK,
there's no way to define \foo as a symbol while keeping \foo\foo as two
separate symbols. That is, the backslash is a symbol boundary, but not a
symbol constituent.

It's perfectly possible to manually write a mode that fixes this behaviour
for the purpose of M-f and M-b, of course. And I think that would be a
worthy addition to AucTex.

Re: My humble additions to AUCTeX

[Marcin Borkowski]

On 2015-01-20, at 17:32, Artur Malabarba <bruce.connor.am@gmail.com> wrote:

>>Yet another example: I'd like M-b to move to the backslash when the point
> is after the comma in `foo\,'.  Normally, both (backward-word) and
> (forward-symbol -1) travel all the way to the `f'.
>
> This is the problem I had as well, when trying to add superword mode to
> AucTex. This is ultimately a limitation of emacs syntax table. AFAIK,
> there's no way to define \foo as a symbol while keeping \foo\foo as two
> separate symbols. That is, the backslash is a symbol boundary, but not a
> symbol constituent.

Exactly.

> It's perfectly possible to manually write a mode that fixes this behaviour
> for the purpose of M-f and M-b, of course. And I think that would be a
> worthy addition to AucTex.

That's what I thought.  (Especially for people who do some LaTeX
package/class writing.)  I even started hacking at it about a year ago,
but Stefan discouraged me a bit (in the sense: directed me to
superword-mode, and I didn't look at it until today).

Best,

-- 
Marcin Borkowski               This email was proudly sent
http://mbork.pl                from my Emacs.

Re: My humble additions to AUCTeX

[Stefan Monnier]

> AucTex. This is ultimately a limitation of emacs syntax table. AFAIK,
> there's no way to define \foo as a symbol while keeping \foo\foo as two
> separate symbols.

FWIW, I'd really like to see syntax-tables extended to finite state
machines so they can match any token described by a regular expression.


        Stefan

Re: My humble additions to AUCTeX

[Rusi]

On Wednesday, January 21, 2015 at 12:13:00 AM UTC+5:30, Stefan Monnier wrote:
> > AucTex. This is ultimately a limitation of emacs syntax table. AFAIK,
> > there's no way to define \foo as a symbol while keeping \foo\foo as two
> > separate symbols.
> 
> FWIW, I'd really like to see syntax-tables extended to finite state
> machines so they can match any token described by a regular expression.

[Somewhat OT]
It may be a good idea to build ragel (or some such)
into the guts of emacs
http://www.colm.net/open-source/ragel/

Re: My humble additions to AUCTeX

[Nicolas Richard]

Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
> Do I guess correctly that you did some (or more) LaTeX typesetting, but
> not too much low-level TeX programming, dear Watson? ;)

Indeed : I did some lower level (albeit trivial) TeX programming at some
point, but not recently -- that might explain why I thought the problems
had disappeared. Another reason is that I had changed the syntax class
of backslash character in my .emacs (I made it ".", i.e. punctuation),
which explains why I use C-M-f (but that doesn't address most of the
points you raised).

> Imagine this: <!>\these\are\four\tokens, where <!> denotes the point.
> Then, C-M-f moves too far; I want to move by one token (i.e. to get
> \these<!>\are\four\tokens), not by a whole string of them at the same
> level of curly braces.

In order to move by token, I made one further change this morning in my
local setup:
(modify-syntax-entry ?\\ ". p" LaTeX-mode-syntax-table)
which lets C-M-b skip the backslash ("p" here means "prefix").

> Also, I want to level up the word commands to token-aware commands.
> Imagine this: \these\are\four\tokens<!>.  Now, if you press M-DEL, the
> backslash does not get deleted; if I made its syntax "word", pressing
> M-DEL would kill *everything* shown here. And I want to be able to kill
> just the string "\tokens" with one keystroke.

C-M-b C-M-k would work with the above modification. Not exactly "one
keystroke", indeed, but close enough for me.

> Last but not least, I aim at commands to work on matching pairs of
> \left...\right, \bigl...\bigr etc. delimiters.  Therefore, I want to be
> able to walk through the text token by token.  (I did look into
> smartparen, and it doesn't seem to be sophisticated enough to do what I
> need.)

To work around that problem, I write this:
\paren{foobar} or \paren*{foobar}
which gives (foobar) (Starred version says to adapt height to the content)
It's as simple as loading mathtools and having
\DeclarePairedDelimiter{\paren}{(}{)}
in the preamble.

-- 
Nico.

Re: My humble additions to AUCTeX

[Marcin Borkowski]

Dnia 2014-01-14, o godz. 11:29:09
Nicolas Richard <theonewiththeevillook@yahoo.fr> napisał(a):

> Marcin Borkowski <mbork@wmi.amu.edu.pl> writes:
> > Do I guess correctly that you did some (or more) LaTeX typesetting,
> > but not too much low-level TeX programming, dear Watson? ;)
> 
> Indeed : I did some lower level (albeit trivial) TeX programming at
> some point, but not recently -- that might explain why I thought the
> problems had disappeared. Another reason is that I had changed the
> syntax class of backslash character in my .emacs (I made it ".", i.e.
> punctuation), which explains why I use C-M-f (but that doesn't
> address most of the points you raised).

Exactly.  So I guess that now you can see the problem I'm solving;).
(Though I didn't really know about C-M-f, so thanks for that tip, too.
I'm quite used to edit human-language documents in Emacs, but not as
good as editing programs, it seems.  I'll have to reread the relevant
portions of the manual.)

> > Imagine this: <!>\these\are\four\tokens, where <!> denotes the
> > point. Then, C-M-f moves too far; I want to move by one token (i.e.
> > to get \these<!>\are\four\tokens), not by a whole string of them at
> > the same level of curly braces.
> 
> In order to move by token, I made one further change this morning in
> my local setup:
> (modify-syntax-entry ?\\ ". p" LaTeX-mode-syntax-table)
> which lets C-M-b skip the backslash ("p" here means "prefix").

Interesting.  It solves *some* problems, but not all.  For instance,
it does not (obviously) see that the second backslash in "\\whatever"
is escaped and should not be treated as the first character of a
token.

> > Also, I want to level up the word commands to token-aware commands.
> > Imagine this: \these\are\four\tokens<!>.  Now, if you press M-DEL,
> > the backslash does not get deleted; if I made its syntax "word",
> > pressing M-DEL would kill *everything* shown here. And I want to be
> > able to kill just the string "\tokens" with one keystroke.
> 
> C-M-b C-M-k would work with the above modification. Not exactly "one
> keystroke", indeed, but close enough for me.

Right, though definitely not close enough for me.  Especially that
e.g. M-b doesn't do *anything* reasonable when used on a token
beginning with a backslash, and (at least for me) it begs for a
suitable modification.  As Stefan pointed out, this is a bad idea
outside my personal setup, so I'm going to examine superword-mode to
incorporate TeX's meaning of "symbol" into its "superword" notion.

> > Last but not least, I aim at commands to work on matching pairs of
> > \left...\right, \bigl...\bigr etc. delimiters.  Therefore, I want
> > to be able to walk through the text token by token.  (I did look
> > into smartparen, and it doesn't seem to be sophisticated enough to
> > do what I need.)
> 
> To work around that problem, I write this:
> \paren{foobar} or \paren*{foobar}
> which gives (foobar) (Starred version says to adapt height to the
> content) It's as simple as loading mathtools and having
> \DeclarePairedDelimiter{\paren}{(}{)}
> in the preamble.

Interesting, I completely forgot about this feature of mathtools.
Thanks!  This, however, still does not solve the problem.  It has an
advantage of replacing the (very plain-TeX-like) \left..\right,
\bigl..\bigr etc. syntax with something much more LaTeX-y, but is not
widespread enough.  Though I might change my TeX habits, people who
send their papers to the journal I'm working for probably won't.  And
making authors stick to *any* LaTeX conventions is, well, somewhere
between impossible and inconceivable;).  (I still quite often get
papers written using LaTeX 2.09 commands.  They should have been dead
for some fifteen years now...  I also get papers with hand-crafted
section titles, complete with all caps, \vskips and of course
hardcoded section numbers, footnotes numbered by hand and inserted
using the $^1$, $^2$ syntax and similar things.)

(Also, \DeclarePairedDelimiter does not help when the delimiters *do
not pair up*, like in the left-open interval $(0,1]$, though probably
defining macros like \lointerval for that purpose would be the right
way to deal with that kind of situation, especially that you could
easily change the appearance of the parens in the intervals globally.)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University

Re: My humble additions to AUCTeX

[jack-mac]

Le lundi 13 janvier 2014 00:28:18 UTC+1, Marcin Borkowski a écrit :
> Also, I still consider myself an Elisp newbie, so it is well possible
> that I did violate some conventions ar good style...
> 
> http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski

In this file, you wrote:
(defun TeX+-letter (&optional at-is-letter)
  "Returns a character class matching letters (including \"@\" if
  AT-IS-LETTER is true)."
  (concat "a-zA-Z"
          (if at-is-letter "@")))

(defun TeX+-looking-at-letter (&optional at-is-letter)
  "Returns t if the point is at a letter (including \"@\" if
AT-IS-LETTER; default is not)."
  (looking-at (concat "[" (TeX+-letter at-is-letter) "]")))


These function create a new string each time they are called (even if at-is-letter is nil).

For efficiency purpose, I would suggest to "compile" them by hand into something like:

(defconst TeX+-letter-ccml        "a-zA-Z"
  "A character class matching letters")

(defconst TeX+-letter-and-at-ccml (concat TeX+-letter-ccml "@")
  "A character class matching letters including \"@\"")

(defconst TeX+-letter-re          (concat "[" TeX+-letter-ccml        "]")
  "A regexp for character class matching letters")

(defconst TeX+-letter-and-at-re   (concat "[" TeX+-letter-and-at-ccml "]")
  "A regexp for character class matching letters including \"@\"")

(defun TeX+-letter (&optional at-is-letter)
  "Returns a character class matching letters (including \"@\" if
  AT-IS-LETTER is true)."
  (if at-is-letter TeX+-letter-and-at-ccml TeX+-letter-ccml))


(defun TeX+-looking-at-letter (&optional at-is-letter)
  "Returns t if the point is at a letter (including \"@\" if
AT-IS-LETTER; default is not)."
  (looking-at (if at-is-letter TeX+-letter-and-at-re TeX+-letter-re)))

HTH

)jack(

Re: My humble additions to AUCTeX

[Marcin Borkowski]

Dnia 2014-01-13, o godz. 08:35:32
jack-mac <duthen.mac@gmail.com> napisał(a):

> 
> Le lundi 13 janvier 2014 00:28:18 UTC+1, Marcin Borkowski a écrit :
> > Also, I still consider myself an Elisp newbie, so it is well
> > possible that I did violate some conventions ar good style...
> > 
> > http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
> 
> In this file, you wrote:
> (defun TeX+-letter (&optional at-is-letter)
>   "Returns a character class matching letters (including \"@\" if
>   AT-IS-LETTER is true)."
>   (concat "a-zA-Z"
>           (if at-is-letter "@")))
> 
> (defun TeX+-looking-at-letter (&optional at-is-letter)
>   "Returns t if the point is at a letter (including \"@\" if
> AT-IS-LETTER; default is not)."
>   (looking-at (concat "[" (TeX+-letter at-is-letter) "]")))
> 
> 
> These function create a new string each time they are called (even if
> at-is-letter is nil).
> 
> For efficiency purpose, I would suggest to "compile" them by hand
> into something like:
> 
> (defconst TeX+-letter-ccml        "a-zA-Z"
>   "A character class matching letters")
> 
> (defconst TeX+-letter-and-at-ccml (concat TeX+-letter-ccml "@")
>   "A character class matching letters including \"@\"")
> 
> (defconst TeX+-letter-re          (concat "[" TeX+-letter-ccml
> "]") "A regexp for character class matching letters")
> 
> (defconst TeX+-letter-and-at-re   (concat "[" TeX+-letter-and-at-ccml
> "]") "A regexp for character class matching letters including \"@\"")
> 
> (defun TeX+-letter (&optional at-is-letter)
>   "Returns a character class matching letters (including \"@\" if
>   AT-IS-LETTER is true)."
>   (if at-is-letter TeX+-letter-and-at-ccml TeX+-letter-ccml))
> 
> 
> (defun TeX+-looking-at-letter (&optional at-is-letter)
>   "Returns t if the point is at a letter (including \"@\" if
> AT-IS-LETTER; default is not)."
>   (looking-at (if at-is-letter TeX+-letter-and-at-re TeX+-letter-re)))
> 
> HTH
> 
> )jack(

Thanks a lot!  In a case of a trade-of between efficiency and code
legibility, I'm in favor of code legibility (unless actual tests prove
that optimization is really needed), but here I can see no problems
like that.  (Well, apart from the a bit more complex situation of
LaTeX3, where there are more characters with the "letter" catcode in
TeXspeak, but still there are at most 3 cases of what "letter" means.)

Best,

-- 
Marcin Borkowski
http://octd.wmi.amu.edu.pl/en/Marcin_Borkowski
Adam Mickiewicz University