non word abbrevs

non word abbrevs

[Jean-Christophe Helary]

I'm trying to create abbrevs that do this

-> →
<- ← 

etc.

but Emacs won't allow that because the abbrevs are not "word constituents".

-- 
Jean-Christophe Helary @brandelune
https://mac4translators.blogspot.com
https://sr.ht/~brandelune/omegat-as-a-book/

Re: non word abbrevs

[Stefan Monnier via Users list for the GNU Emacs text editor]

Jean-Christophe Helary [2021-10-31 14:45:04] wrote:
> I'm trying to create abbrevs that do this
>
> -> →
> <- ← 
>
> etc.
>
> but Emacs won't allow that because the abbrevs are not "word constituents".

You need to change the `:regexp` on your abbrev table.
See `define-abbrev-table`.


        Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

> On Nov 1, 2021, at 4:04, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:
> 
> Jean-Christophe Helary [2021-10-31 14:45:04] wrote:
>> I'm trying to create abbrevs that do this
>> 
>> -> →
>> <- ← 
>> 
>> etc.
>> 
>> but Emacs won't allow that because the abbrevs are not "word constituents".
> 
> You need to change the `:regexp` on your abbrev table.
> See `define-abbrev-table`.

(define-abbrev-table 'arrows-abbrev-table
  '(("->" "→")
    ("<-"  "←")
    ("<->" "⇆")
    ("==>" "⇒")
    ("<==" "⇐")
    ("<=>" "⇔")
    )
    "Arrows as defined on my macOS side."
    :regexp "\(<?[-=]>?\)")

I must be missing something. I evaluate the expression but the abbrevs do not work.

Jean-Christophe 

Re: non word abbrevs

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean-Christophe Helary wrote:

> (define-abbrev-table 'arrows-abbrev-table
>   '(("->" "→")
>     ("<-"  "←")
>     ("<->" "⇆")
>     ("==>" "⇒")
>     ("<==" "⇐")
>     ("<=>" "⇔")
>     )
>     "Arrows as defined on my macOS side."
>     :regexp "\(<?[-=]>?\)")
>
> I must be missing something. I evaluate the expression but
> the abbrevs do not work.

Try with just one arrow - the simplest - and see if you can
find the regexp that works for that ...

-- 
underground experts united
https://dataswamp.org/~incal

Re: non word abbrevs

[Stefan Monnier]

>     :regexp "\(<?[-=]>?\)")
> I must be missing something.

Hmmm... backslashes?  ;-)
The above string is the same as "(<?[-=]>?)" (in recentish Emacsen the
above backslashes should presumably be highlighted in
a font-lock-warning color for that reason).

Beware also that this regexp is matched backwards and stops as soon as
it finds a match, so it finds the *shortest* match rather than the
longest match.  IOW the <? part will always match the empty string.


        Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

> On Nov 1, 2021, at 21:03, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>>    :regexp "\(<?[-=]>?\)")
>> I must be missing something.
> 
> Hmmm... backslashes?  ;-)
> The above string is the same as "(<?[-=]>?)" (in recentish Emacsen the
> above backslashes should presumably be highlighted in
> a font-lock-warning color for that reason).

I'm trying in *scratch* with lisp-mode on a recent "master" and I don't get that...

> Beware also that this regexp is matched backwards and stops as soon as
> it finds a match, so it finds the *shortest* match rather than the
> longest match.  IOW the <? part will always match the empty string.

I've changed it to:

:regexp "\\([<>=-]+\\)"

this time using: 

(setq local-abbrev-table arrows-abbrev-table)

And I get a nice message that reads like this:

[## 0 0 0 0 0 0 0 0 0 0 <== <=> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ==> 0 0 0 0 0 0 0 0 0 <- 0 0 0 0 0 0 0 0 <-> 0 0 0 -> 0 0 0]

And I guess that's a list that includes possible matches but I'm not seeing => / <= so I wonder.

Then, I enable abbrev-mode but nothing gets transformed...

:-( Jean-Christophe 

Re: non word abbrevs

[Stefan Monnier via Users list for the GNU Emacs text editor]

>>>    :regexp "\(<?[-=]>?\)")
>>> I must be missing something.
>> Hmmm... backslashes?  ;-)
>> The above string is the same as "(<?[-=]>?)" (in recentish Emacsen the
>> above backslashes should presumably be highlighted in
>> a font-lock-warning color for that reason).
>
> I'm trying in *scratch* with lisp-mode on a recent "master" and I don't get that...

I just tried it on my end with both `master` and Emacs-27.1

    emacs -Q
    :regexp "\(<?[-=]>?\)")

and the two backslashes got the `font-lock-warning-face` (the strings
are colored a kind of brick red and the warning is in red so it doesn't
stand out as much as I'd like but it's visible).

I wonder why you don't see that on your end.

> :regexp "\\([<>=-]+\\)"

And since Emacs will only use the shortest match, it will only use
one-char-long matches :-(

Try something like: "[^<>=-]\\([<>=-]+\\)"
[ Yes, this shortest match business is a PITA.  ]

> (setq local-abbrev-table arrows-abbrev-table)
>
> And I get a nice message that reads like this:
>
> [## 0 0 0 0 0 0 0 0 0 0 <== <=> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ==>
> 0 0 0 0 0 0 0 0 0 <- 0 0 0 0 0 0 0 0 <-> 0 0 0 -> 0 0 0]

That's just the printed version of the vector that's used as an
"obarray".  It's mostly meaningless.

> And I guess that's a list that includes possible matches but I'm not seeing => / <= so I wonder.

As I said it's mostly meaningless: it includes *some* of the symbols but
not all.  I had a patch which introduced a real obarray type (instead
of abusing vectors), which could then be printed "properly", a bit like
hash tables, but it had its share of problems.

> Then, I enable abbrev-mode but nothing gets transformed...

That's because it only single-char matches occurred and you had no
single-char abbrev defined?


        Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

> On Nov 1, 2021, at 21:43, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:
> 
>>>>   :regexp "\(<?[-=]>?\)")
>>>> I must be missing something.
>>> Hmmm... backslashes?  ;-)
>>> The above string is the same as "(<?[-=]>?)" (in recentish Emacsen the
>>> above backslashes should presumably be highlighted in
>>> a font-lock-warning color for that reason).
>> 
>> I'm trying in *scratch* with lisp-mode on a recent "master" and I don't get that...
> 
> I just tried it on my end with both `master` and Emacs-27.1
> 
>    emacs -Q
>    :regexp "\(<?[-=]>?\)")
> 
> and the two backslashes got the `font-lock-warning-face` (the strings
> are colored a kind of brick red and the warning is in red so it doesn't
> stand out as much as I'd like but it's visible).

Ok, I just rebuilt the latest master and emacs -Q shows them... And I am *not* going to dive into my init.el at this time of the day...

>> :regexp "\\([<>=-]+\\)"
> 
> And since Emacs will only use the shortest match, it will only use
> one-char-long matches :-(
> 
> Try something like: "[^<>=-]\\([<>=-]+\\)"
> [ Yes, this shortest match business is a PITA.  ]

My abbrevs are arrows composed of < or > at the tips and - or = with a minimum of 2 characters and a max of 3:

→ ⇒
← ⇐
↔ ⇔

In a different world, I'd try [<>=-]{2,3}

Is that what you mean ?

JC

Re: non word abbrevs

[Leo Butler]

Jean-Christophe Helary <lists@traduction-libre.org> writes:

>> On Nov 1, 2021, at 21:43, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:
>> 
>>>>>   :regexp "\(<?[-=]>?\)")
>>>>> I must be missing something.
>>>> Hmmm... backslashes?  ;-)
>>>> The above string is the same as "(<?[-=]>?)" (in recentish Emacsen the
>>>> above backslashes should presumably be highlighted in
>>>> a font-lock-warning color for that reason).
>>> 
>>> I'm trying in *scratch* with lisp-mode on a recent "master" and I don't get that...
>> 
>> I just tried it on my end with both `master` and Emacs-27.1
>> 
>>    emacs -Q
>>    :regexp "\(<?[-=]>?\)")
>> 
>> and the two backslashes got the `font-lock-warning-face` (the strings
>> are colored a kind of brick red and the warning is in red so it doesn't
>> stand out as much as I'd like but it's visible).
>
> Ok, I just rebuilt the latest master and emacs -Q shows them... And I am *not* going to dive into my init.el at this time of the day...
>
>>> :regexp "\\([<>=-]+\\)"
>> 
>> And since Emacs will only use the shortest match, it will only use
>> one-char-long matches :-(
>> 
>> Try something like: "[^<>=-]\\([<>=-]+\\)"
>> [ Yes, this shortest match business is a PITA.  ]
>
> My abbrevs are arrows composed of < or > at the tips and - or = with a minimum of 2 characters and a max of 3:
>
> → ⇒
> ← ⇐
> ↔ ⇔
>
> In a different world, I'd try [<>=-]{2,3}
>
> Is that what you mean ?

Try M-x regexp-builder RET

This is a fast way to check your regexps are matching the way you
intend.

Btw, it might be easier to do something like

(regexp-opt '("=>" "==>") "[^>=<]\\(?:")
===> "[^>=<]\\(?:=\\(?:=?>\\)\\)"

Btw, I just checked that regexp with regexp-builder and it matched only
2 strings in this buffer.

FWIW,
Leo

Re: non word abbrevs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

On Mon, Nov 01, 2021 at 10:17:28PM +0900, Jean-Christophe Helary wrote:
> 
> 
> > On Nov 1, 2021, at 21:43, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

[...]

> >> :regexp "\\([<>=-]+\\)"

Oh, yes: always double your backslashes (well /nearly/ [1] always).
The first ones are for Lisp's string syntax.

[...]

> In a different world, I'd try [<>=-]{2,3}

Emacs is your world, then. Well, nearly :) It groks {} repetitions
(cf. the docs). Note that they are spelt with a backslash (don't
forget to double it in string context). This would give:

      "...[<>=-]\\{2,3\\}..."

in your case (the dots are ellipses, for the rest of your regexp).

Cheers

[1] When you are writing the regexp in a Lisp string. Not when
   you are entering it interactively, e.g. in the minibuffer.

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: non word abbrevs

[Stefan Monnier]

>> Try something like: "[^<>=-]\\([<>=-]+\\)"
>> [ Yes, this shortest match business is a PITA.  ]
>
> My abbrevs are arrows composed of < or > at the tips and - or = with
> a minimum of 2 characters and a max of 3:
>
> → ⇒
> ← ⇐
> ↔ ⇔
>
> In a different world, I'd try [<>=-]{2,3}
>
> Is that what you mean ?

No, the [^<>=-] is the important part: it tells Emacs to keep looking
(going backward) for a longer match.

If you use "[<>=-]\\{2,3\\}" (which does exist in our beloved ELisp
world), then Emacs's regexp matcher will go as follows:

- Let's say we're in a line with "<=> hello", with point just after the ">".
- The regexp search will first try to match (backward) ">" against the
  regexp, which will fail.
- It will then try to match "=>" against the regexp, which will succeed.
- At this point it stops and fails to notice that the regexp would also
  have matched "<=>", so you won't get the abbrev expansion you wanted.


-- Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

Ok, so this time I try your regexp (thank you by the way):

> On Nov 1, 2021, at 21:13, Jean-Christophe Helary <lists@traduction-libre.org> wrote:


(define-abbrev-table 'arrows-abbrev-table
 '(("->" "→")
   ("<-"  "←")
   ("<->" "⇆")
   ("==>" "⇒")
   ("<==" "⇐")
   ("<=>" "⇔")
   )
   "Arrows as defined on my macOS side."
   :regexp "[^<>=-]\\([<>=-]+\\)")

(setq local-abbrev-table arrows-abbrev-table)

and I have the abbrev-mode enabled in the *scratch* buffer where I run that, and nothing happens when I type -> for ex... 😢

-- 
Jean-Christophe Helary @brandelune
https://mac4translators.blogspot.com
https://sr.ht/~brandelune/omegat-as-a-book/

Re: non word abbrevs

[Stefan Monnier]

Jean-Christophe Helary [2021-11-06 17:20:58] wrote:
> Ok, so this time I try your regexp (thank you by the way):
>> On Nov 1, 2021, at 21:13, Jean-Christophe Helary <lists@traduction-libre.org> wrote:
> (define-abbrev-table 'arrows-abbrev-table
>  '(("->" "→")
>    ("<-"  "←")
>    ("<->" "⇆")
>    ("==>" "⇒")
>    ("<==" "⇐")
>    ("<=>" "⇔")
>    )
>    "Arrows as defined on my macOS side."
>    :regexp "[^<>=-]\\([<>=-]+\\)")
>
> (setq local-abbrev-table arrows-abbrev-table)
>
> and I have the abbrev-mode enabled in the *scratch* buffer where I run
>  that, and nothing happens when I type -> for ex... 😢

I tried the above and then `M-x expand-abbrev` and it worked for my test
case, so I guess the problem is in the auto-triggering of abbrevs down
in `self-insert-command`.

And indeed I see:

    if (!NILP (BVAR (current_buffer, abbrev_mode))
        && synt != Sword
        && NILP (BVAR (current_buffer, read_only))
        && PT > BEGV
        && (SYNTAX (!NILP (BVAR (current_buffer, enable_multibyte_characters))
                    ? XFIXNAT (Fprevious_char ())
                    : UNIBYTE_TO_CHAR (XFIXNAT (Fprevious_char ())))
            == Sword))

so it looks like the "auto expand abbrev" feature only works right after
a word char, so basically only works for abbrevs whose last char has
word syntax.

You might want to `M-x report-emacs-bug` and request that this
constraint be made more flexible.


        Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

Thank you Stephan for checking all that.

> On Nov 7, 2021, at 7:32, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> I tried the above and then `M-x expand-abbrev` and it worked for my test
> case, so I guess the problem is in the auto-triggering of abbrevs down
> in `self-insert-command`.
> 
> And indeed I see:
> 
>    if (!NILP (BVAR (current_buffer, abbrev_mode))
>        && synt != Sword
>        && NILP (BVAR (current_buffer, read_only))
>        && PT > BEGV
>        && (SYNTAX (!NILP (BVAR (current_buffer, enable_multibyte_characters))
>                    ? XFIXNAT (Fprevious_char ())
>                    : UNIBYTE_TO_CHAR (XFIXNAT (Fprevious_char ())))
>            == Sword))
> 
> so it looks like the "auto expand abbrev" feature only works right after
> a word char, so basically only works for abbrevs whose last char has
> word syntax.

Which kind of defeats the idea of being able to define a regex in the first place...

> You might want to `M-x report-emacs-bug` and request that this
> constraint be made more flexible.

I will. Thank you again.

-- 
Jean-Christophe Helary @brandelune
https://mac4translators.blogspot.com
https://sr.ht/~brandelune/omegat-as-a-book/

Re: non word abbrevs

[Stefan Monnier via Users list for the GNU Emacs text editor]

>> so it looks like the "auto expand abbrev" feature only works right after
>> a word char, so basically only works for abbrevs whose last char has
>> word syntax.
> Which kind of defeats the idea of being able to define a regex in the first place...

Not completely: the original motivation for adding that `:regexp`
keyword was to handle abbrevs of the form "`foo" as well as others of
the form "foo/bar" ;-)

>> You might want to `M-x report-emacs-bug` and request that this
>> constraint be made more flexible.
> I will. Thank you again.

In the mean time, you might be able to implement
a `post-self-insert-hook` which runs expands your abbrevs.
You could start with something like:

    (add-hook 'post-self-insert-hook #'my-expand-arrow-abbrevs)
    (defun my-expand-arrow-abbrevs ()
      (if (memq (char-before) '(?= ?< ?-))
          (expand-abbrev)))


-- Stefan

Re: non word abbrevs

[Jean-Christophe Helary]

> On Nov 7, 2021, at 12:15, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:

> Not completely: the original motivation for adding that `:regexp`
> keyword was to handle abbrevs of the form "`foo" as well as others of
> the form "foo/bar" ;-)

And the overall feature seems severely under-documented... Honestly, I barely understood your explanations...

Maybe expecting to have abbrevs work way was a mistake, but it does not seem possible to have (easy...) arbitrary string expansion in out-of-the-box-emacs without abbrevs.

Would there be another more idiomatic way ?
I know about assigning key shortcuts to arbitrary string insertion, but that needlessly fills the keymaps with shortcuts that could be put to better use *and* the combination would be longer to enter than a normal abbrev...

> 
>>> You might want to `M-x report-emacs-bug` and request that this
>>> constraint be made more flexible.
>> I will. Thank you again.
> 
> In the mean time, you might be able to implement
> a `post-self-insert-hook` which runs expands your abbrevs.

Christmas in November ? :-)

> You could start with something like:
> 
>    (add-hook 'post-self-insert-hook #'my-expand-arrow-abbrevs)
>    (defun my-expand-arrow-abbrevs ()
>      (if (memq (char-before) '(?= ?< ?-))
>          (expand-abbrev)))

I'll try that and will report. Thank you !


-- 
Jean-Christophe Helary @brandelune
https://mac4translators.blogspot.com
https://sr.ht/~brandelune/omegat-as-a-book/

Re: non word abbrevs

[Emanuel Berg via Users list for the GNU Emacs text editor]

Stefan Monnier via Users list for the GNU Emacs text editor wrote:

> In the mean time, you might be able to implement
> a `post-self-insert-hook` which runs expands your abbrevs.
> You could start with something like:
>
>     (add-hook 'post-self-insert-hook #'my-expand-arrow-abbrevs)
>     (defun my-expand-arrow-abbrevs ()
>       (if (memq (char-before) '(?= ?< ?-))
>           (expand-abbrev)))

Isn't it faster to type something, e.g. "leftarr" or
something, anyway? I mean, compared with typing "<-" ...

But I get it one would like it configurable anyway, and not
a bug.

-- 
underground experts united
https://dataswamp.org/~incal

Re: non word abbrevs

[Emanuel Berg via Users list for the GNU Emacs text editor]

Jean-Christophe Helary wrote:

>> On Nov 7, 2021, at 12:15, Stefan Monnier via Users list for
>> the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:
>
>> Not completely: the original motivation for adding that `:regexp`
>> keyword was to handle abbrevs of the form "`foo" as well as others of
>> the form "foo/bar" ;-)
>
> And the overall feature seems severely under-documented...
> Honestly, I barely understood your explanations...

It doesn't execute if the last char is for example a dash ...

Try an abbrev that is "a-b" (not "a to b", the chars a, dash,
and b), the "a-b" abbrev will work as b is the last char, not
the dash this time.

(dash is "HYPHEN-MINUS" officially :))

> Maybe expecting to have abbrevs work way was a mistake, but
> it does not seem possible to have (easy...) arbitrary string
> expansion in out-of-the-box-emacs without abbrevs.

It is easy! Try setup an abbrev that is "aaaz" to "bbbz", with
the punctuation chars it doesn't work since it doesn't
trigger, that's a bug, good work YOU for finding it! Your code
works fine with "aaaz" to "bbbz", well, details aside since
you tried to do something else.

> Would there be another more idiomatic way ?

No, you did it the right way, Emacs did it the wrong way, but
as long as we have our captain going at - I don't know how
many percent? - but surely plenty enough to fix these kind of
bugs :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: non word abbrevs

[Kevin Vigouroux via Users list for the GNU Emacs text editor]

Jean-Christophe Helary <lists@traduction-libre.org> writes:

>> On Nov 7, 2021, at 12:15, Stefan Monnier via Users list for the GNU Emacs text editor <help-gnu-emacs@gnu.org> wrote:
>
>> Not completely: the original motivation for adding that `:regexp`
>> keyword was to handle abbrevs of the form "`foo" as well as others of
>> the form "foo/bar" ;-)
>
> And the overall feature seems severely under-documented... Honestly, I barely understood your explanations...
>
> Maybe expecting to have abbrevs work way was a mistake, but it does not seem possible to have (easy...) arbitrary string expansion in out-of-the-box-emacs without abbrevs.
>
> Would there be another more idiomatic way ?
> I know about assigning key shortcuts to arbitrary string insertion,
> but that needlessly fills the keymaps with shortcuts that could be put
> to better use *and* the combination would be longer to enter than a
> normal abbrev...
>

You can insert a Unicode character with `insert-char'. This does not
shorten the typing but at least you can insert any character easily.

In word processors, you can insert these characters if they are
available in the dialog box called “Special Characters”.

>> 
>>>> You might want to `M-x report-emacs-bug` and request that this
>>>> constraint be made more flexible.
>>> I will. Thank you again.
>> 
>> In the mean time, you might be able to implement
>> a `post-self-insert-hook` which runs expands your abbrevs.
>
> Christmas in November ? :-)
>
>> You could start with something like:
>> 
>>    (add-hook 'post-self-insert-hook #'my-expand-arrow-abbrevs)
>>    (defun my-expand-arrow-abbrevs ()
>>      (if (memq (char-before) '(?= ?< ?-))
>>          (expand-abbrev)))
>
> I'll try that and will report. Thank you !

-- 
Best regards,
Kevin Vigouroux

Re: non word abbrevs

[Jean-Christophe Helary]

>> Maybe expecting to have abbrevs work way was a mistake, but it does not seem possible to have (easy...) arbitrary string expansion in out-of-the-box-emacs without abbrevs.
>> 
>> Would there be another more idiomatic way ?
>> I know about assigning key shortcuts to arbitrary string insertion,
>> but that needlessly fills the keymaps with shortcuts that could be put
>> to better use *and* the combination would be longer to enter than a
>> normal abbrev...
>> 
> 
> You can insert a Unicode character with `insert-char'. This does not
> shorten the typing but at least you can insert any character easily.

Thank you. I am really just interested in having "proper" text expansions like what I have on the OS side. I fail to see the logic of limiting abbrevs to "word characters".


-- 
Jean-Christophe Helary @brandelune
https://mac4translators.blogspot.com
https://sr.ht/~brandelune/omegat-as-a-book/

Re: non word abbrevs

[Stefan Monnier via Users list for the GNU Emacs text editor]

> Thank you. I am really just interested in having "proper" text expansions
> like what I have on the OS side. I fail to see the logic of limiting abbrevs
> to "word characters".

Maybe part of the motivation was the performance cost of checking
abbrevs after every key stroke, but I suspect the main issue is avoiding
"false positives":

Say you hit in sequence the three keys `< = >` and you have abbrev rules
for `<=` and for `<=>`, you'd probably not want the first abbrev rule to
apply after you hit `< =` because you'd prefer for Emacs to wait a bit
more and apply the second abbrev rule.

For words, this notion of boundary is fairly standard.  But for other
chars, it's not so clear.
So to lift this restriction we need something more.


        Stefan