jinx

jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

I saw the announcement about the jinx spelling package.  It sounds
very powerful.  Could it replace ispell.el?  If so, do people think
that would be a good idea?  If not, why not?

Same questions for flyspell.el.

What languages is jinx written in?  What does the libenchant library
do, and how does it combine or work with Emacs?  How does it combine
or work with aspell?

The description talks about "native modules" but doesn't say what they
are or what they do.  Can anyone tell us?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> I saw the announcement about the jinx spelling package.  It sounds
> very powerful.  Could it replace ispell.el?  If so, do people think
> that would be a good idea?  If not, why not?

I think it would be an idea worth considering, but that would mean that
Emacs would have another compile-time dependency.

> Same questions for flyspell.el.
>
> What languages is jinx written in?  

https://git.savannah.gnu.org/cgit/emacs/elpa.git/tree/?h=externals/jinx

jinx is written as a Emacs module, so one part is Emacs Lisp and the
other one is C.

>                                     What does the libenchant library
> do, and how does it combine or work with Emacs?  How does it combine
> or work with aspell?

https://abiword.github.io/enchant/

"Enchant is a library (and command-line program) that wraps a number of
different spelling libraries and programs with a consistent
interface. By using Enchant, you can use a wide range of spelling
libraries, including some specialised for particular languages, without
needing to program to each library's interface."

ispell already supports enchant, but it communicates via IPC instead of
the library interface.

> The description talks about "native modules" but doesn't say what they
> are or what they do.  Can anyone tell us?

I am guessing that is just a typo, and Daniel means (elisp) Writing
Dynamic Modules.

-- 
Philip Kaludercic

Re: jinx

[Michael Eliachevitch]

[-- Attachment #1: Type: text/plain, Size: 3548 bytes --]


The technical questions where answered by Philip. I tried the package today and can comment from a user perspective:

> I saw the announcement about the jinx spelling package.  It sounds
> very powerful.  Could it replace ispell.el?  If so, do people think
> that would be a good idea?  If not, why not?
>
> Same questions for flyspell.el.

These packages are used in different ways and may complement each other.

jinx spell-checks the visible text and underlines errors. It's very fast at finding errors due to focusing on the visible text and using the libenchant API instead of subprocess calls. But it currently lacks the ability to check the entire buffer and e.g. jump to the next error or list all errors.

In its README, the jinx author writes that flyspell only checks the work under point. However, it doesn't mention that one can run flyspell-buffer (or flycheck-region) to check the entire buffer for errors, so that's not a huge issue. But jinx just feels more snappy in that regard, incorrect are just highlighted immediately without having to do anything except having the mode enabled.

But as far as I know it doesn't have yet the ability to check and list the spelling mistakes in the entire buffer. Therefore, I will probably still be using ispell-buffer to iterate through all spelling errors.

What I personally enjoy most about jinx are the ergonomics of being able to exclude certain syntactic categories from spell-checking. One convenient way are  `jinx-include-faces' and `jinx-exclude-faces', which are alists of major modes to faces to include/ignore. Also there's `jinx-exclude-regexps' which is an alist from major modes to regexes. For example made it easy for me to get rid of false-positives on macros in LaTeX [fn 1]. Some spell-checking programs (aspell, hunspell) have LaTeX-detection built-in, but it doesn't always work reliably and is not as easily configurable. And that functionality doesn't extend to other markup languages. This is also useful for e.g. org-mode or checking comments and docstrings in programming languages.


Ergonomic-wise I also enjoyed that `jinx-languages', which configures the language, can be a list of language (locale) strings (or just a single string).. I'll use that for my private notes, which tend to be a mix of German and English. `ispell-dictionary' in comparison can only be a single string. (Though through online search I found that when used with hunspell one can provide a comma-seperated list of locales to ispell-dictionary and hunspell will recognise that).

I feel like from a user-perspective, jinx, possibly extended, could replace most use-cases of flyspell. Alternatively, one could also try to add a feature to make flyspell always check the visible text in the buffer, which would need to happen asynchronously I don't know whether technically that would be possible  One could also consider trying to port some of its other ideas back to the existing spell-checking packages in emacs, like using mode-alists for the faces-configuration.

[fn 1]: While writing the mail I looked into the flyspell documentation and that is has similar features via `flyspell-prog-text-faces' and `flyspell-tex-command-regexp', but those are less flexible. `flyspell-prog-text-faces' doesn't work for modes that do not inherit from prog-mode and I can't configure it on a mode-specific granularity, except by using major-mode-hooks and dir- or file-local variables.

--
Michael Eliachevitch
Public PGP Key: https://keyoxide.org/hkp/546908c782383ad0e7d894ec1b8f95c8125dce31

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: jinx

[João Pedro]

Hey Michael,

On Wed, March 29 2023 22:46, Michael Eliachevitch <m.eliachevitch@posteo.de> wrote:

> But it currently lacks the ability to check the entire buffer

Little correction here. It does have a "check the entire buffer"
functionality, just call jinx-correct with a universal-argument (C-u).

Best regards,

-- 
João Pedro de A. Paula

Re: jinx

[Eli Zaretskii]

> From: Michael Eliachevitch <m.eliachevitch@posteo.de>
> Cc: emacs-devel@gnu.org
> Date: Wed, 29 Mar 2023 22:46:01 +0000
> 
> [fn 1]: While writing the mail I looked into the flyspell documentation and that is has similar features via `flyspell-prog-text-faces' and `flyspell-tex-command-regexp', but those are less flexible. `flyspell-prog-text-faces' doesn't work for modes that do not inherit from prog-mode and I can't configure it on a mode-specific granularity, except by using major-mode-hooks and dir- or file-local variables.

It should be easy to extend these flyspell features to be more
flexible.  Patches welcome.

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > ispell already supports enchant, but it communicates via IPC instead of
  > the library interface.

Would it be possible for jinx to do this?
Would it be better overall for jinx to do this?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But as far as I know it doesn't have yet the ability to check and
  > list the spelling mistakes in the entire buffer. Therefore, I will
  > probably still be using ispell-buffer to iterate through all
  > spelling errors.

It was pointed out that there is a way to do this.  Of course, if we
install it in Emacs we could adjust the calling conventions however we
think users will like.  We could make `ispell-buffer' call jinx
if that's what we want.

Given that, do you still think you would have a need for the existing
`ispell-buffer' that uses ispell.el, if jinx were always available?


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 31 Mar 2023 00:29:23 -0400
> 
>   > But as far as I know it doesn't have yet the ability to check and
>   > list the spelling mistakes in the entire buffer. Therefore, I will
>   > probably still be using ispell-buffer to iterate through all
>   > spelling errors.
> 
> It was pointed out that there is a way to do this.  Of course, if we
> install it in Emacs we could adjust the calling conventions however we
> think users will like.  We could make `ispell-buffer' call jinx
> if that's what we want.
> 
> Given that, do you still think you would have a need for the existing
> `ispell-buffer' that uses ispell.el, if jinx were always available?

We cannot assume that "jinx is always available", since it requires
Emacs to be linked against the enchant library.

Anyway, after looking at jinx, I must say that I don't see why we
should replace ispell and flyspell with jinx.  From my POV, ispell and
flyspell provide all the required spell-checking functionalities:

  . spell-check a word
  . spell-check a region
  . spell-check the entire buffer
  . spell-check comments and strings in a prog-mode buffer
  . spell-check as you type

jinx spell-checks the visible portion of the window.  I don't see why
we would need that.  Suppose I scroll the window by 2 lines: why would
I want the new visible portion to be spell-checked again, when the
overlap with the last check is so large?  In any case, if someone
wants a handy command to spell-check only the part that is currently
visible in the window, we can add such a command to ispell.el.  (Btw,
jinx relies on window-end for this, and window-end is a problematic
function which cannot be trusted.)

Another feature of jinx is the ability to skip some buffer text, e.g.,
relying on font-lock faces or user-defined regexps to identify the
parts that should be skipped.  We have similar features:
spell-checking of comments and strings, but if that is not enough, we
could extend these capabilities in ispell and flyspell by simple
enough additions.

All those are minor extensions of the existing spell-checking
capabilities that require no deep changes in the infrastructure and
documentation, which have been stable and useful for many years.

Having jinx as an ELPA package is fine, of course.  Likewise, it's
fine if someone wants to work on redesigning and reimplementing the
low level of ispell.el so that it could optionally use libenchant if
that is available.  But I see no reason to throw out the window the
whole spelling infrastructure we have and replacing it with what jinx
provides.  That would require not just coding and testing, but also
rewriting the manual and will require people to re-learn how they use
spell-checking features in Emacs.  I see no justification for that,
especially since jinx doesn't bring any new spell-checking technology
to Emacs -- libenchant uses those same spell-checking programs, aspell
and hunspell, that we have been using for ages.

As an aside, a technology I would like to see in Emacs is grammar
checking.  Last I looked, there are no free grammar-checking libraries
or services out there, even just for the English language.  But if
that's no longer true, then adding such capabilities to Emacs will be
very welcome, I think, since word-based spell-checking is clearly not
enough these days.

Re: jinx

[Gregory Heytings]

>
> As an aside, a technology I would like to see in Emacs is grammar 
> checking.  Last I looked, there are no free grammar-checking libraries 
> or services out there, even just for the English language.  But if 
> that's no longer true, then adding such capabilities to Emacs will be 
> very welcome, I think, since word-based spell-checking is clearly not 
> enough these days.
>

I think that's no longer true, LanguageTool is available (and has been for 
~20 years): https://github.com/languagetool-org/languagetool .  But it's 
written in Java...

Re: jinx

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > ispell already supports enchant, but it communicates via IPC instead of
>   > the library interface.
>
> Would it be possible for jinx to do this?
> Would it be better overall for jinx to do this?

Do what?  Jinx /does/ use the enchant library, ispell /does/ use IPC.

-- 
Philip Kaludercic

Re: jinx

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Another feature of jinx is the ability to skip some buffer text, e.g.,
> relying on font-lock faces or user-defined regexps to identify the
> parts that should be skipped.  We have similar features:
> spell-checking of comments and strings, but if that is not enough, we
> could extend these capabilities in ispell and flyspell by simple
> enough additions.

Somewhat off-topic, but it would be great if ispell and flyspell could
use the same logic for skipping portions of buffer text.  Take for
example (La)TeX mode: ispell uses `ispell-tex-skip-alists' and flyspell
uses `flyspell-tex-command-regexp' (which is not very powerful) and then
a function added to `flyspell-generic-check-word-predicate' (AFAIU).  I
once wrote an addition for AUCTeX (tex-ispell.el) which extends
`ispell-tex-skip-alists' for many LaTeX packages but had hard times
making it work with flyspell as well, skipping multi-line text portions
was something I couldn't solve for flyspell, e.g.:

\begin{verbatim}
Please skip the entire text in this environment.
\end{verbatim}

Best, Arash

Re: jinx

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> Date: Fri, 31 Mar 2023 20:33:02 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Another feature of jinx is the ability to skip some buffer text, e.g.,
> > relying on font-lock faces or user-defined regexps to identify the
> > parts that should be skipped.  We have similar features:
> > spell-checking of comments and strings, but if that is not enough, we
> > could extend these capabilities in ispell and flyspell by simple
> > enough additions.
> 
> Somewhat off-topic, but it would be great if ispell and flyspell could
> use the same logic for skipping portions of buffer text.

Please submit patches to do that, it shouldn't be hard.

(I wonder why the built-in TeX support in the spell-checker doesn't do
this job, it's supposed to be adequate, and all Emacs needs to do is
to tell the speller it is working on TeX text.  So I don't even
understand why we need to skip something on our own in these cases.)

Re: jinx

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> Please submit patches to do that, it shouldn't be hard.

I tried it as I wrote tex-ispell.el but I didn't understand flyspell.el
enough back then.  Maybe I give it a new try, but I wouldn't hold my
breath for that.

> (I wonder why the built-in TeX support in the spell-checker doesn't do
> this job, it's supposed to be adequate, and all Emacs needs to do is
> to tell the speller it is working on TeX text.  So I don't even
> understand why we need to skip something on our own in these cases.)

I can only tell for hunspell: It has built-in support for skipping of
many macro arguments and environments[1], but:

  a) It is not complete (and probably will never be)
  b) It cannot work for user defined macros/environments, and this is
  where AUCTeX/ispell shine: AUCTeX can parse user defined
  macros/environments and add them automatically to
  `ispell-tex-skip-alists'.
  c) Another advantage of `ispell-tex-skip-alists' is that one can write
  a function and put that in `ispell-tex-skip-alists' for certain
  macros/environments: That gives us a lot of flexibility to precisely
  skip or check.

Best, Arash

Footnotes:
[1]  https://github.com/hunspell/hunspell/blob/e7ccb97fde9d7e9a81c920a3e255ad98426692f8/src/parsers/latexparser.cxx#L53

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > jinx spell-checks the visible portion of the window.  I don't see why
  > we would need that.  Suppose I scroll the window by 2 lines: why would
  > I want the new visible portion to be spell-checked again, when the
  > overlap with the last check is so large?

You mean, it doesn't remember which part has already been checked?
I took for granted it would optimize by not rechecking what was
already on the window.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > >   > ispell already supports enchant, but it communicates via IPC instead of
  > >   > the library interface.
  > >
  > > Would it be possible for jinx to do this?
  > > Would it be better overall for jinx to do this?

  > Do what?

Communicate with enchant via IPC.

I understood the other person to be saying that communicating that way
is an advantage for ispell.  If it is an advantage, could jinx be made
to do it that way?

Maybe perse did not mean that this was an advantage.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: m.eliachevitch@posteo.de, emacs-devel@gnu.org
> Date: Fri, 31 Mar 2023 23:11:55 -0400
> 
>   > jinx spell-checks the visible portion of the window.  I don't see why
>   > we would need that.  Suppose I scroll the window by 2 lines: why would
>   > I want the new visible portion to be spell-checked again, when the
>   > overlap with the last check is so large?
> 
> You mean, it doesn't remember which part has already been checked?

It seems to rely on JIT font-lock mechanism to keep track of what was
already spell-checked.

But my point was not about efficiency, it was about the user
experience: why would I want to have the visible portion marked with
mis-spelling indications at all times?  This is not a natural way of
dealing with spelling mistakes.

And my main point was that if someone does want to have a mode or a
command to mark spelling mistakes in a visible portion of the buffer,
we can easily add that to the existing spell-checking commands and
modes.  No need to switch to a completely different implementation to
have that feature.

Re: jinx

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 31 Mar 2023 23:11:57 -0400
> 
>   > >   > ispell already supports enchant, but it communicates via IPC instead of
>   > >   > the library interface.
>   > >
>   > > Would it be possible for jinx to do this?
>   > > Would it be better overall for jinx to do this?
> 
>   > Do what?
> 
> Communicate with enchant via IPC.
> 
> I understood the other person to be saying that communicating that way
> is an advantage for ispell.  If it is an advantage, could jinx be made
> to do it that way?
> 
> Maybe perse did not mean that this was an advantage.

AFAIU, the IPC interface that is bypassed is the one between Emacs and
Enchant, implemented in ispell.el.  The IPC that is NOT bypassed is
the one between Enchant and the speller used as its back-end: Aspell
or Hunspell.

Personally, I don't think this is a significant matter: the IPC
interface used by ispell.el is not slow, because we keep the
spell-checking process running at all times.  What might be slow is
the Flyspell's reliance on post-command-hook to spell-check as you
type: this slows down Emacs in some important scenarios.  But again:
if we want to improve that aspect, we don't need jinx, we should
redesign and reimplement how we pass words to the speller using more
efficient mechanisms, such as JIT font-lock hooks.  I believe there's
already an ELPA package which attempts to do something like that.

Re: jinx

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> Date: Fri, 31 Mar 2023 21:35:25 +0200
> 
> > (I wonder why the built-in TeX support in the spell-checker doesn't do
> > this job, it's supposed to be adequate, and all Emacs needs to do is
> > to tell the speller it is working on TeX text.  So I don't even
> > understand why we need to skip something on our own in these cases.)
> 
> I can only tell for hunspell: It has built-in support for skipping of
> many macro arguments and environments[1], but:
> 
>   a) It is not complete (and probably will never be)
>   b) It cannot work for user defined macros/environments, and this is
>   where AUCTeX/ispell shine: AUCTeX can parse user defined
>   macros/environments and add them automatically to
>   `ispell-tex-skip-alists'.
>   c) Another advantage of `ispell-tex-skip-alists' is that one can write
>   a function and put that in `ispell-tex-skip-alists' for certain
>   macros/environments: That gives us a lot of flexibility to precisely
>   skip or check.

These issues are not TeX-specific, so they should be handled in a more
general way, not bound to TeX/LaTeX.  In particular, any "tex-skip"
features should not supplant the TeX support built into the speller,
they should only _add_ skip regexps for stuff that the speller cannot
support, like user-defined macros.

I'm not sure I understand item c), though: when and why would one want
to spell-check macros and environments?

Re: jinx

[Arash Esbati]

Eli Zaretskii <eliz@gnu.org> writes:

> These issues are not TeX-specific, so they should be handled in a more
> general way, not bound to TeX/LaTeX.  In particular, any "tex-skip"
> features should not supplant the TeX support built into the speller,
> they should only _add_ skip regexps for stuff that the speller cannot
> support, like user-defined macros.

Agreed, in general.  The only thing is that we are dealing with
different spellers (ispell, aspell, hunspell, nuspell?) and I assume
they have different level of built-in support for "tex-skips" (I haven't
checked though).  My example with verbatim environment was related to
"multi-line skip" in flyspell; it can be replaced with any other
environment which does a similar thing.

> I'm not sure I understand item c), though: when and why would one want
> to spell-check macros and environments?

Sorry, I meant to say arguments/bodies of macros/environments, not the
name of macros and environments.

Best, Arash

Re: jinx

[Eli Zaretskii]

> From: Arash Esbati <arash@gnu.org>
> Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> Date: Sat, 01 Apr 2023 09:42:40 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I'm not sure I understand item c), though: when and why would one want
> > to spell-check macros and environments?
> 
> Sorry, I meant to say arguments/bodies of macros/environments, not the
> name of macros and environments.

This is again not TeX-specific.  Similar issues exist in Texinfo and
in other markup languages.  The annoyance is usually minor: it's
enough to tell the speller to "accept" a word just once, since the
number of markup codes is usually very small.  But it would be nicer,
of course, if Emacs could automatically skip markup in each major
mode.

We should keep in mind that spell-checking technical text will
inevitably produce quite a few false positives, due to the jargon,
acronyms, file names, and other similar stuff.  I don't see how this
could be solved in principle without risking false negatives, which is
worse.

Re: jinx

[Emanuel Berg]

Eli Zaretskii wrote:

> We cannot assume that "jinx is always available", since it
> requires Emacs to be linked against the enchant library.
>
> Anyway, after looking at jinx, I must say that I don't see
> why we should replace ispell and flyspell with jinx. From my
> POV, ispell and flyspell provide all the required
> spell-checking functionalities:
>
>   . spell-check a word
>   . spell-check a region
>   . spell-check the entire buffer
>   . spell-check comments and strings in a prog-mode buffer
>   . spell-check as you type

Had to do some work on some of that back when I set it up,
don't know if that has since then been moved into vanilla
Emacs - here is the file anyway

;;; -*- lexical-binding: t -*-
;;
;; this file:
;;   https://dataswamp.org/~incal/emacs-init/spell.el
;;
;; Debian bug:
;;   ispell-init-process: Illegal format hash table
;;   /usr/lib/ispell/svenska.hash - expected magic2 0x9602, got 0x414c
;;   $ sudo buildhash /usr/share/dict/svenska /usr/lib/ispell/svenska.aff /usr/lib/ispell/svenska.hash

(require 'ispell)

(when (eq system-type 'gnu/linux)
  (load-file "/usr/share/dictionaries-common/site-elisp/debian-ispell.el")
  (load-file "/var/cache/dictionaries-common/emacsen-ispell-dicts.el") )

(setq ispell-program-name   "ispell")
(setq ispell-silently-savep t)

;; single word

(defun spell-word (word)
  (with-temp-buffer
    (save-excursion
      (insert word) )
    (condition-case nil
        (not (ispell-word))
      (error nil) )))
;; (spell-word "length") ; t
;; (spell-word "lenght") ; nil

(defun spell-this-word (dict)
  "`ispell-word' the preceding word with DICT."
  (ispell-change-dictionary dict)
  (ispell-word) )

(let ((eng-dict "american-insane"))
  (defun spell-english ()
    (interactive)
    (spell eng-dict) )

  (defun word ()
    (interactive)
    (spell-this-word eng-dict) ))
;; test: hello / hxllo

(let ((swe-dict "svenska"))
  (defun spell-swedish ()
    (interactive)
    (spell swe-dict) )

  (defun ord ()
    (interactive)
    (spell-this-word swe-dict) ))
;; test: hej / hxj

;; buffer / whole / dwim

(defun is-code ()
  (member major-mode '(c++-mode
                       c-mode
                       emacs-lisp-mode
                       prolog-mode
                       python-mode
                       sh-mode
                       Shell-script-mode) )) ; add more ...

(defun is-message ()
  (eq major-mode 'message-mode) )

(defun spell (dict)
  "Spell with DICT.
If region is active, use `ispell-region';
if editing code, `ispell-comments-and-strings';
if writing a message, `ispell-message';
else, `ispell-buffer'"
  (ispell-change-dictionary dict)
  (save-window-excursion
    (save-excursion
      (cond
       ((use-region-p) (ispell-region (region-beginning) (region-end)))
       ((is-message)   (ispell-message))
       ((is-code)      (ispell-comments-and-strings))
       (t              (ispell-buffer)) ))))

-- 
underground experts united
https://dataswamp.org/~incal

Re: jinx

[Augusto Stoffel]

On Fri, 31 Mar 2023 at 20:33, Arash Esbati wrote:

> Somewhat off-topic, but it would be great if ispell and flyspell could
> use the same logic for skipping portions of buffer text.  Take for
> example (La)TeX mode: ispell uses `ispell-tex-skip-alists' and flyspell
> uses `flyspell-tex-command-regexp' (which is not very powerful) and then
> a function added to `flyspell-generic-check-word-predicate' (AFAIU).  I
> once wrote an addition for AUCTeX (tex-ispell.el) which extends
> `ispell-tex-skip-alists' for many LaTeX packages but had hard times
> making it work with flyspell as well, skipping multi-line text portions
> was something I couldn't solve for flyspell, e.g.:
>
> \begin{verbatim}
> Please skip the entire text in this environment.
> \end{verbatim}
>
> Best, Arash

The logic used by Flyspell could easily be used by ispell.el, but not
vice-versa.  However, the flyspell-generic-check-word-predicate API also
has some issues (e.g., a minor mode can't plug into it easily).  Also,
it's quite messy in that some modes are supported in flyspell.el, while
all other are expected to implement their own predicate (of course, the
latter is a better approach).

So I'd suggest a generic API on these lines:

#+begin_src emacs-lisp
(defvar spelling-ignore-functions nil
  "Hook used to determine if a word should be excluded from spell checking.
These functions are called with two arguments, the start and end
positions of a word, until a non-nil value is returned.  Any
result other than nil or `never' indicates that the word should
be excluded from spell checking.

Modes derived from `prog-mode' should leave it up to the spell
checker to decide whether or not to ignore comments, strings or
code regions.  They may provide more specific rules, if desired,
but this typically not needed.

MAYBE ALSO: These functions should make no assumptions about the
point position, and are free to move it.")

(defun spelling-ignored-p (start end)
  "Return non-nil if the word between START and END should not be spell checked.
See `spelling-ignore-functions' for information on how this is
determined."
  (save-excursion
    (memq (run-hook-with-args-until-success spelling-ignore-functions start end)
          '(nil never))))
#+end_src

The above would go into a “neutral” place like simple.el.  Each mode
would be responsible to define its own hooks and each spell-checking
package could use it as it sees fit.

Re: jinx

[Augusto Stoffel]

On Fri, 31 Mar 2023 at 21:35, Arash Esbati wrote:

>> (I wonder why the built-in TeX support in the spell-checker doesn't do
>> this job, it's supposed to be adequate, and all Emacs needs to do is
>> to tell the speller it is working on TeX text.  So I don't even
>> understand why we need to skip something on our own in these cases.)
>
> I can only tell for hunspell: It has built-in support for skipping of
> many macro arguments and environments[1], but:
>
>   a) It is not complete (and probably will never be)
>   b) It cannot work for user defined macros/environments, and this is
>   where AUCTeX/ispell shine: AUCTeX can parse user defined
>   macros/environments and add them automatically to
>   `ispell-tex-skip-alists'.
>   c) Another advantage of `ispell-tex-skip-alists' is that one can write
>   a function and put that in `ispell-tex-skip-alists' for certain
>   macros/environments: That gives us a lot of flexibility to precisely
>   skip or check.

A more basic reason why this couldn't work is, the parser is stateful
and you need to look at an arbitrary amount of preceding text to figure
out the context at point.  So you can't rely on the spellchecker TeX
parser if you feed single words or lines into it, as Flyspell et al do.

Re: jinx

[Augusto Stoffel]

On Sat,  1 Apr 2023 at 08:56, Eli Zaretskii wrote:

>> From: Richard Stallman <rms@gnu.org>
>> Cc: m.eliachevitch@posteo.de, emacs-devel@gnu.org
>> Date: Fri, 31 Mar 2023 23:11:55 -0400
>> 
>>   > jinx spell-checks the visible portion of the window.  I don't see why
>>   > we would need that.  Suppose I scroll the window by 2 lines: why would
>>   > I want the new visible portion to be spell-checked again, when the
>>   > overlap with the last check is so large?
>> 
>> You mean, it doesn't remember which part has already been checked?
>
> It seems to rely on JIT font-lock mechanism to keep track of what was
> already spell-checked.
>
> But my point was not about efficiency, it was about the user
> experience: why would I want to have the visible portion marked with
> mis-spelling indications at all times?  This is not a natural way of
> dealing with spelling mistakes.

FWIW, this is what any word processor out there does.  Personally I find
it much more natural.

Re: jinx

[Augusto Stoffel]

On Fri, 31 Mar 2023 at 23:11, Richard Stallman wrote:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > >   > ispell already supports enchant, but it communicates via IPC instead of
>   > >   > the library interface.
>   > >
>   > > Would it be possible for jinx to do this?
>   > > Would it be better overall for jinx to do this?
>
>   > Do what?
>
> Communicate with enchant via IPC.
>
> I understood the other person to be saying that communicating that way
> is an advantage for ispell.  If it is an advantage, could jinx be made
> to do it that way?
>
> Maybe perse did not mean that this was an advantage.

It is totally feasible to write a decent spell-checking package relying
just on IPC, but it would be very helpful (for this an many other
purposes) to optimize Emacs's subprocess communication routines, which
are clearly pretty slow.

In fact, jinx is a fork of my (also recent) jit-spell package, which
communicates with the spellchecker through pipes using the `ispell -a'
interface.  Jinx was created because its author experienced some latency
issues while using jit-spell.  I was unable to reproduce his
observations, although I do observe relatively high CPU usage while
typing with jit-spell mode on.

Re: jinx

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  m.eliachevitch@posteo.de,
>   emacs-devel@gnu.org
> Date: Sat, 01 Apr 2023 10:29:13 +0200
> 
> So I'd suggest a generic API on these lines:
> 
> #+begin_src emacs-lisp
> (defvar spelling-ignore-functions nil
>   "Hook used to determine if a word should be excluded from spell checking.
> These functions are called with two arguments, the start and end
> positions of a word, until a non-nil value is returned.  Any
> result other than nil or `never' indicates that the word should
> be excluded from spell checking.

If we pass START and END, why does the region have to be a single
"word"? why not let the caller specify a larger region to be skipped?

> Modes derived from `prog-mode' should leave it up to the spell
> checker to decide whether or not to ignore comments, strings or
> code regions.  They may provide more specific rules, if desired,
> but this typically not needed.

Why?  I thought the purpose of this is to override what the
spell-checker knows, and that includes comments and strings, if
needed.  So I see no reason for such a recommendation.

> (defun spelling-ignored-p (start end)
>   "Return non-nil if the word between START and END should not be spell checked.
> See `spelling-ignore-functions' for information on how this is
> determined."
>   (save-excursion

Why do we need save-excursion?

> The above would go into a “neutral” place like simple.el.

Why not ispell.el?  All the spelling functions use ispell.el anyway.
simple.el is preloaded, so having this there bloats every Emacs
session for no good reason, IMO.

Re: jinx

[Augusto Stoffel]

On Sat,  1 Apr 2023 at 14:21, Eli Zaretskii wrote:

>> From: Augusto Stoffel <arstoffel@gmail.com>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  rms@gnu.org,  m.eliachevitch@posteo.de,
>>   emacs-devel@gnu.org
>> Date: Sat, 01 Apr 2023 10:29:13 +0200
>> 
>> So I'd suggest a generic API on these lines:
>> 
>> #+begin_src emacs-lisp
>> (defvar spelling-ignore-functions nil
>>   "Hook used to determine if a word should be excluded from spell checking.
>> These functions are called with two arguments, the start and end
>> positions of a word, until a non-nil value is returned.  Any
>> result other than nil or `never' indicates that the word should
>> be excluded from spell checking.
>
> If we pass START and END, why does the region have to be a single
> "word"?

This is a tricky point.  The notion of "word", for spell checking
purposes, is outside of the major mode's control.  For instance, some
languages include the apostrophe or hyphen as a word char, some don't.

Passing START and END means that the major mode can use the information
of how the spell-checker did the tokenization.  Typically, though, I
expect most function will look only at START or END.

>  why not let the caller specify a larger region to be skipped?

Then the return value can't be a boolean; it has to be a list of
regions, which is too complicated.

>> Modes derived from `prog-mode' should leave it up to the spell
>> checker to decide whether or not to ignore comments, strings or
>> code regions.  They may provide more specific rules, if desired,
>> but this typically not needed.
>
> Why?  I thought the purpose of this is to override what the
> spell-checker knows, and that includes comments and strings, if
> needed.  So I see no reason for such a recommendation.

Maybe.  But the above is exactly how Flyspell's
flyspell-generic-check-word-predicate works, and it makes sense IMO.

>> (defun spelling-ignored-p (start end)
>>   "Return non-nil if the word between START and END should not be spell checked.
>> See `spelling-ignore-functions' for information on how this is
>> determined."
>>   (save-excursion
>
> Why do we need save-excursion?

If we remove it here, then most spelling-ignore-functions will need to
call save excursion on their own, which would probably be fine as well.

>> The above would go into a “neutral” place like simple.el.
>
> Why not ispell.el?  All the spelling functions use ispell.el anyway.
> simple.el is preloaded, so having this there bloats every Emacs
> session for no good reason, IMO.

jit-spell only uses ispell.el to start a process and jinx doesn't use it
at all.  To me it makes sense to untangle the spellchecking support API
for major modes from any particular implementation.  But I guess this is
not a major issue.

Re: jinx

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: arash@gnu.org,  rms@gnu.org,  m.eliachevitch@posteo.de,
>   emacs-devel@gnu.org
> Date: Sat, 01 Apr 2023 13:39:40 +0200
> 
> On Sat,  1 Apr 2023 at 14:21, Eli Zaretskii wrote:
> 
> > If we pass START and END, why does the region have to be a single
> > "word"?
> 
> This is a tricky point.  The notion of "word", for spell checking
> purposes, is outside of the major mode's control.  For instance, some
> languages include the apostrophe or hyphen as a word char, some don't.
> 
> Passing START and END means that the major mode can use the information
> of how the spell-checker did the tokenization.  Typically, though, I
> expect most function will look only at START or END.
> 
> >  why not let the caller specify a larger region to be skipped?
> 
> Then the return value can't be a boolean; it has to be a list of
> regions, which is too complicated.

I'm afraid I don't quite follow.  I actually don't understand why we
need END here.  Why not call the function with some buffer position,
and let it return nil (meaning don't skip) or a buffer position, which
means skip until that position?

IOW, skipping text in at least some situation needs to skip multiple
words, perhaps even multiple lines, and the skip function should be
allowed to specify that in one go.  Right?

> >> (defun spelling-ignored-p (start end)
> >>   "Return non-nil if the word between START and END should not be spell checked.
> >> See `spelling-ignore-functions' for information on how this is
> >> determined."
> >>   (save-excursion
> >
> > Why do we need save-excursion?
> 
> If we remove it here, then most spelling-ignore-functions will need to
> call save excursion on their own, which would probably be fine as well.

Yes, I don't think we should second-guess what those functions will
do.

> >> The above would go into a “neutral” place like simple.el.
> >
> > Why not ispell.el?  All the spelling functions use ispell.el anyway.
> > simple.el is preloaded, so having this there bloats every Emacs
> > session for no good reason, IMO.
> 
> jit-spell only uses ispell.el to start a process and jinx doesn't use it
> at all.

jinx is not in Emacs, so we don't have to solve its problems.  And
jit-spell uses ispell.el, so it will be able to use any function
there.

> To me it makes sense to untangle the spellchecking support API
> for major modes from any particular implementation.  But I guess this is
> not a major issue.

Not a major issue indeed, at least not yet.  If and when needed, we
could have a general spell.el file that would hold all the
spell-checking stuff not related to particular implementations.
Although in my mind ispell.el is such a "base-line" file already, as
every spelling feature basically uses some of its infrastructure.

Re: jinx

[Augusto Stoffel]

On Sat,  1 Apr 2023 at 14:54, Eli Zaretskii wrote:

> I'm afraid I don't quite follow.  I actually don't understand why we
> need END here.  Why not call the function with some buffer position,
> and let it return nil (meaning don't skip) or a buffer position, which
> means skip until that position?
>
> IOW, skipping text in at least some situation needs to skip multiple
> words, perhaps even multiple lines, and the skip function should be
> allowed to specify that in one go.  Right?

You are making too many assumptions about how the spell checker logic
works.  What you propose would be really handy for ispell-region and the
like but is not usable by Flyspell -- for the same reason Flyspell
doesn't use ispell-skip-region-alist.

We should come up with an API that any spell-checking package, present
of future, could use.

>> > Why not ispell.el?  All the spelling functions use ispell.el anyway.
>> > simple.el is preloaded, so having this there bloats every Emacs
>> > session for no good reason, IMO.
>> 
>> jit-spell only uses ispell.el to start a process and jinx doesn't use it
>> at all.
>
> jinx is not in Emacs, so we don't have to solve its problems.  And
> jit-spell uses ispell.el, so it will be able to use any function
> there.

We don't need to solve jinx's problems, but the API should be convenient
for any third-party package to integrate with, and that's why I brought
up the example.

Re: jinx

[Peter Oliver]

On Sat, 1 Apr 2023, Eli Zaretskii wrote:

> AFAIU, the IPC interface that is bypassed is the one between Emacs and
> Enchant, implemented in ispell.el.  The IPC that is NOT bypassed is
> the one between Enchant and the speller used as its back-end: Aspell
> or Hunspell.

With libenchant (and hence Jinx), both IPCs are avoided for most spellchecking providers (including Aspell, Hunspell, Nuspell).

https://github.com/AbiWord/enchant/tree/master/providers

-- 
Peter Oliver

Re: jinx

[Eli Zaretskii]

> From: Augusto Stoffel <arstoffel@gmail.com>
> Cc: arash@gnu.org,  rms@gnu.org,  m.eliachevitch@posteo.de,
>   emacs-devel@gnu.org
> Date: Sat, 01 Apr 2023 14:32:33 +0200
> 
> On Sat,  1 Apr 2023 at 14:54, Eli Zaretskii wrote:
> 
> > I'm afraid I don't quite follow.  I actually don't understand why we
> > need END here.  Why not call the function with some buffer position,
> > and let it return nil (meaning don't skip) or a buffer position, which
> > means skip until that position?
> >
> > IOW, skipping text in at least some situation needs to skip multiple
> > words, perhaps even multiple lines, and the skip function should be
> > allowed to specify that in one go.  Right?
> 
> You are making too many assumptions about how the spell checker logic
> works.  What you propose would be really handy for ispell-region and the
> like but is not usable by Flyspell -- for the same reason Flyspell
> doesn't use ispell-skip-region-alist.

This depends on where we decide to plug this feature into Flyspell, I
think.  If we plug this into it before it gets to process the result
of flyspell-get-word, then all we need to do is make sure
flyspell-get-word returns nil for stretches of text that need to be
skipped.

> We should come up with an API that any spell-checking package, present
> of future, could use.

I think what I proposed fits the bill.  But it's just an idea.

> >> jit-spell only uses ispell.el to start a process and jinx doesn't use it
> >> at all.
> >
> > jinx is not in Emacs, so we don't have to solve its problems.  And
> > jit-spell uses ispell.el, so it will be able to use any function
> > there.
> 
> We don't need to solve jinx's problems, but the API should be convenient
> for any third-party package to integrate with, and that's why I brought
> up the example.

I don't see how loading a single Lisp file could be of any
inconvenience.

Adding functions to preloaded packages just because we think they will
make it convenient to some third-party package is not TRT.

Re: jinx

[Eli Zaretskii]

> Date: Sat, 1 Apr 2023 13:43:05 +0100 (BST)
> From: Peter Oliver <p.d.oliver@mavit.org.uk>
> cc: emacs-devel@gnu.org
> 
> On Sat, 1 Apr 2023, Eli Zaretskii wrote:
> 
> > AFAIU, the IPC interface that is bypassed is the one between Emacs and
> > Enchant, implemented in ispell.el.  The IPC that is NOT bypassed is
> > the one between Enchant and the speller used as its back-end: Aspell
> > or Hunspell.
> 
> With libenchant (and hence Jinx), both IPCs are avoided for most spellchecking providers (including Aspell, Hunspell, Nuspell).
> 
> https://github.com/AbiWord/enchant/tree/master/providers

That's a two-edge sword: it means an Emacs built with that will be
wedded to a particular version of the speller.  I'm not sure this is a
good idea (but won't object to supporting such a build, if someone
wants that).

Spell-checking is an interactive process, which proceeds word by word,
and most of the time is taken by the user deciding which alternative
to choose.  So the speed of communicating with the spell-checker
should not by itself be an issue.

Re: jinx

[Lynn Winebarger]

On Sat, Apr 1, 2023 at 3:20 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Arash Esbati <arash@gnu.org>
> > Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> > Date: Fri, 31 Mar 2023 21:35:25 +0200
> >
> > > (I wonder why the built-in TeX support in the spell-checker doesn't do
> > > this job, it's supposed to be adequate, and all Emacs needs to do is
> > > to tell the speller it is working on TeX text.  So I don't even
> > > understand why we need to skip something on our own in these cases.)
> >
> > I can only tell for hunspell: It has built-in support for skipping of
> > many macro arguments and environments[1], but:
> >
> >   a) It is not complete (and probably will never be)
> >   b) It cannot work for user defined macros/environments, and this is
> >   where AUCTeX/ispell shine: AUCTeX can parse user defined
> >   macros/environments and add them automatically to
> >   `ispell-tex-skip-alists'.
> >   c) Another advantage of `ispell-tex-skip-alists' is that one can write
> >   a function and put that in `ispell-tex-skip-alists' for certain
> >   macros/environments: That gives us a lot of flexibility to precisely
> >   skip or check.
>
> These issues are not TeX-specific, so they should be handled in a more
> general way, not bound to TeX/LaTeX.  In particular, any "tex-skip"
> features should not supplant the TeX support built into the speller,
> they should only _add_ skip regexps for stuff that the speller cannot
> support, like user-defined macros.
>
> I'm not sure I understand item c), though: when and why would one want
> to spell-check macros and environments?

If's completely possible to have TeX files with typos in the macros or
environments.  I wouldn't mind having some mechanism for detecting
such typos and suggesting correct possibilities.  Whether that should
be done through a spell-checking interface or not, I don't know.
Obviously I'd want the dictionary for spell-checking those lexemes to
be distinct from the ones used for checking ordinary text.

Please note I am not advocating for such a feature, I am only
responding to your question.

Lynn

Re: jinx

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 785 bytes --]

On Sat, 1 Apr 2023, Eli Zaretskii wrote:

>> With libenchant (and hence Jinx), both IPCs are avoided for most spellchecking providers (including Aspell, Hunspell, Nuspell).
>>
>> https://github.com/AbiWord/enchant/tree/master/providers
>
> That's a two-edge sword: it means an Emacs built with that will be
> wedded to a particular version of the speller.  I'm not sure this is a
> good idea (but won't object to supporting such a build, if someone
> wants that).

I’d be fine with that, since I get all of this pre-built from my distribution.

But anyway, I’m not sure that that’s true.  If you want to take a major upgrade to your spellchecker, I would expect it to be sufficient to update its dynamically loaded Enchant plugin, not everything that uses libenchant.

-- 
Peter Oliver

Re: jinx

[Arash Esbati]

[-- Attachment #1: Type: text/plain, Size: 1257 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

> This is again not TeX-specific.  Similar issues exist in Texinfo and
> in other markup languages.  The annoyance is usually minor: it's
> enough to tell the speller to "accept" a word just once, since the
> number of markup codes is usually very small.

I see your point, but I think Emacs could do better.

> But it would be nicer, of course, if Emacs could automatically skip
> markup in each major mode.

I second this: Emacs should skip markup in each major mode, and it is a
pity that the support for things like texinfo and/or org-mode is
somewhat poor.  This was my motivation to add a library to AUCTeX for
ispell and adjust most of it style files to use it.

> We should keep in mind that spell-checking technical text will
> inevitably produce quite a few false positives, due to the jargon,
> acronyms, file names, and other similar stuff.  I don't see how this
> could be solved in principle without risking false negatives, which is
> worse.

True.  OTOH, Emacs could skip the markup where we know it can be
ignored, take for example @file{filename}.  Currently, both ispell and
flyspell try to correct the filename argument.  Here a nonsensical
example how flyspell looks in texinfo-mode with Emacs -Q:

[-- Attachment #2: flyspell-texinfo.png --]
[-- Type: image/png, Size: 2525 bytes --]

[-- Attachment #3: Type: text/plain, Size: 298 bytes --]


So if I could make a wish, I'd like to see a general infrastructure
where one can do things like in `ispell-skip-region-alist', looking for
font-lock and all the other techniques lying around and ispell, flyspell
and others would use this infrastructure and not bake their own stuff.

Best, Arash

Re: jinx

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> > From: Arash Esbati <arash@gnu.org>
> > Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> > Date: Sat, 01 Apr 2023 09:42:40 +0200
> >
> > Eli Zaretskii <eliz@gnu.org> writes:
> >
> > > I'm not sure I understand item c), though: when and why would one want
> > > to spell-check macros and environments?
> >
> > Sorry, I meant to say arguments/bodies of macros/environments, not the
> > name of macros and environments.
>
> This is again not TeX-specific.  Similar issues exist in Texinfo and
> in other markup languages.  The annoyance is usually minor: it's
> enough to tell the speller to "accept" a word just once, since the
> number of markup codes is usually very small.  But it would be nicer,
> of course, if Emacs could automatically skip markup in each major
> mode.

Wasn't that the purpose of `flyspell-prog-mode'?

Michael.

Re: jinx

[Michael Eliachevitch]

[-- Attachment #1: Type: text/plain, Size: 2670 bytes --]


On 2023-04-03 at 14:32 +02, Michael Heerdegen <michael_heerdegen@web.de> wrote:

>> This is again not TeX-specific.  Similar issues exist in Texinfo and
>> in other markup languages.  The annoyance is usually minor: it's
>> enough to tell the speller to "accept" a word just once, since the
>> number of markup codes is usually very small.  But it would be nicer,
>> of course, if Emacs could automatically skip markup in each major
>> mode.
>
> Wasn't that the purpose of `flyspell-prog-mode'?

Yes, `flyspell-prog-mode' and `flyspell-prog-text-faces' together offer similar functionality to what jinx does by default, only that it allows configuring the faces on per-major-mode basis and has include- and exclude-faces. As I understood the discussion, ideally we could add this to flyspell and ispell in a consistent way.

I'm also not sure if the distinction between `flyspell-mode' and `flyspell-prog-mode' is still needed, you can just customize the text-mode to not exclude any faces for instance. It doesn't hurt, we can just enable `flyspell-prog-mode' everywhere. But ispell doesn't have a prog-mode. If we want to have common include/exclude faces for ispell and flyspell and be consistent with those packages, one could think about adding an ispell-prog-mode for consistency, though that seems weird to me.

Of course `flyspell-prog-mode' one can achieve very similar functionality to jinx by using per-major-mode hooks and dir-local variables to customize `flyspell-prog-text-faces'  for each mode, but it would be nice for that to be simpler. And naming wise, initially I was confused by `flyspell-prog-mode' because I thought this is programming-mode-specific and might need for the mode to inherit from `prog-mode', but we have plenty of modes with font-lock that inherit from text-mode or conf-mode.

An example: I noticed that both jinx and flyspell underline "pgpmime" string in the MML signature snippet in my email buffer, which has the major mode `mu4e-compose-mode' inheriting from `message-mode' and in turn from text-mode. In jinx, I can fix the error message by a simple

    (cl-pushnew 'message-mml (alist-get 'message-mode jinx-exclude-faces))

The cited text is still checked, because for non-prog-modes I didn't specify any jinx-include-faces and thus it checks all non-exclude faces. In flyspell, I could add `flyspell-prog-mode' to `message-mode-hook' and also hook a function that locally sets the `flyspell-prog-text-faces' to the default text-face and the four message-cited-faces, hoping I didn't forget any other faces that text might have in the email compose buffer. So it feels a bit more annoying to configure.

Cheers, Michael

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 519 bytes --]

Re: jinx

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Cc: Arash Esbati <arash@gnu.org>,  rms@gnu.org,  m.eliachevitch@posteo.de,
>   emacs-devel@gnu.org
> Date: Mon, 03 Apr 2023 14:32:36 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > > From: Arash Esbati <arash@gnu.org>
> > > Cc: rms@gnu.org,  m.eliachevitch@posteo.de,  emacs-devel@gnu.org
> > > Date: Sat, 01 Apr 2023 09:42:40 +0200
> > >
> > > Eli Zaretskii <eliz@gnu.org> writes:
> > >
> > > > I'm not sure I understand item c), though: when and why would one want
> > > > to spell-check macros and environments?
> > >
> > > Sorry, I meant to say arguments/bodies of macros/environments, not the
> > > name of macros and environments.
> >
> > This is again not TeX-specific.  Similar issues exist in Texinfo and
> > in other markup languages.  The annoyance is usually minor: it's
> > enough to tell the speller to "accept" a word just once, since the
> > number of markup codes is usually very small.  But it would be nicer,
> > of course, if Emacs could automatically skip markup in each major
> > mode.
> 
> Wasn't that the purpose of `flyspell-prog-mode'?

Markup languages are usually not descendants of prog-modes.

Or maybe I misunderstand what you are saying.

Re: jinx

[Michael Eliachevitch]

On 2023-04-03 at 17:26 +03, Eli Zaretskii <eliz@gnu.org> wrote:

>> From: Michael Heerdegen <michael_heerdegen@web.de>
>> Wasn't that the purpose of `flyspell-prog-mode'?
>
> Markup languages are usually not descendants of prog-modes.
>
> Or maybe I misunderstand what you are saying.

`flyspell-prog-mode' doesn't have any relation to prog-mode. It is a function (not really a mode) that enables flyspell-mode and sets the buffer-local variable `flyspell-generic-check-word-predicate'.

--8<---------------cut here---------------start------------->8---
  (defun flyspell-prog-mode ()
    "Turn on `flyspell-mode' for comments and strings."
    (interactive)
    (setq flyspell-generic-check-word-predicate
          #'flyspell-generic-progmode-verify)
    (flyspell-mode 1)
    (run-hooks 'flyspell-prog-mode-hook))
--8<---------------cut here---------------end--------------->8---

The function `flyspell-generic-progmode-verify' checks whether the face at point is a member of `flyspell-prog-text-faces'. By default it only checks comment and string faces, which is useful in programming modes. But flyspell-prog-mode can be used in any other mode as well `flyspell-prog-text-faces', and one can set `flyspell-prog-text-faces' buffer-locally via setq-local to e.g. only include non-markup. Though the flyspell documentation talks about it being for programming modes, so that seems to be a hack.

Until I read the code above today, I wasn't aware of `flyspell-generic-check-word-predicate', as it's not a defcustom-variable. It might serve as a customization entrypoint, i.e. one could bind it to a function that e.g. checks for include- and exclude-lists like jinx.

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > `flyspell-prog-mode' doesn't have any relation to prog-mode. It is
  > a function (not really a mode) that enables flyspell-mode and sets
  > the buffer-local variable `flyspell-generic-check-word-predicate'.

It seems to act somewhat like a major mode but not entirely.
Also, its name seems to suggest something different.  That can
cause confusion.

Is it possible to rename this to a clearer name?  Perhaps
program-text-flyspell-mode, or something else?  And then make it a
proper major mode?, following all the conventions for major modes?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Michael Heerdegen]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > `flyspell-prog-mode' doesn't have any relation to prog-mode. It is
>   > a function (not really a mode) that enables flyspell-mode and sets
>   > the buffer-local variable `flyspell-generic-check-word-predicate'.
>
> It seems to act somewhat like a major mode but not entirely.
> Also, its name seems to suggest something different.  That can
> cause confusion.
>
> Is it possible to rename this to a clearer name?  Perhaps
> program-text-flyspell-mode, or something else?  And then make it a
> proper major mode?, following all the conventions for major modes?

A major mode?? I think you misunderstood what it does.  It's simply a
slightly modified flyspell-mode, conceptually clearly a minor-mode, see
the doc string ("Turn on `flyspell-mode' for comments and strings").

What the "don't spell-check language keywords" functionality skips
depends on the major-mode.  That's all.

Michael.

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Is it possible to rename this to a clearer name?  Perhaps
  > program-text-flyspell-mode, or something else?  And then make it a
  > proper major mode?, following all the conventions for major modes?

Another idea is, text-within-code-flyspell-mode.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Is it possible to rename this to a clearer name?  Perhaps
  > > program-text-flyspell-mode, or something else?  And then make it a
  > > proper major mode?, following all the conventions for major modes?

  > A major mode?? I think you misunderstood what it does.  It's simply a
  > slightly modified flyspell-mode, conceptually clearly a minor-mode, 

I stand corrected.

But the name is still unhelpful.  It seems to enable Flyspell mode for
strings and comments only.  But you'd never guess that from its name.
The name suggests it is a mode function, but I doesn't follow the
conventions for those.

Can those things be corrected?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: jinx

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

> Another idea is, text-within-code-flyspell-mode.

How about `syntactical-flyspell-mode'?

Re: jinx

[Arash Esbati]

Richard Stallman <rms@gnu.org> writes:

> Another idea is, text-within-code-flyspell-mode.

ispell.el has `ispell-comments-and-strings', so I'd suggest
`flyspell-comments-and-strings-mode'.

Best, Arash

Re: jinx

[Emanuel Berg]

Arash Esbati wrote:

>> Another idea is, text-within-code-flyspell-mode.
>
> ispell.el has `ispell-comments-and-strings', so I'd suggest
> `flyspell-comments-and-strings-mode'.

Can any of them spell a single word? If not, here is code
for that.

(defun spell-word (word)
  (with-temp-buffer
    (save-excursion
      (insert word) )
    (condition-case nil
        (not (ispell-word))
      (error nil) )))
;; (spell-word "length") ; t
;; (spell-word "lenght") ; nil

(defun spell-this-word (dict)
  "`ispell-word' the word with DICT."
  (ispell-change-dictionary dict)
  (ispell-word) )

(let ((eng-dict "american-insane"))
  (defun spell-english ()
    (interactive)
    (spell eng-dict) )

  (defun word ()
    (interactive)
    (spell-this-word eng-dict) ))
;; test: hello / hxlle

(let ((swe-dict "svenska"))
  (defun spell-swedish ()
    (interactive)
    (spell swe-dict) )

  (defun ord ()
    (interactive)
    (spell-this-word swe-dict) ))
;; test: hej / hxj

Full source:
  https://dataswamp.org/~incal/emacs-init/spell.el

-- 
underground experts united
https://dataswamp.org/~incal

Re: jinx

[Philip Kaludercic]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > > Is it possible to rename this to a clearer name?  Perhaps
>   > > program-text-flyspell-mode, or something else?  And then make it a
>   > > proper major mode?, following all the conventions for major modes?
>
>   > A major mode?? I think you misunderstood what it does.  It's simply a
>   > slightly modified flyspell-mode, conceptually clearly a minor-mode, 
>
> I stand corrected.
>
> But the name is still unhelpful.  It seems to enable Flyspell mode for
> strings and comments only.  But you'd never guess that from its name.
> The name suggests it is a mode function, but I doesn't follow the
> conventions for those.
>
> Can those things be corrected?

It should be possible to add a defalias with a different name, but
personally I don't think the name is that confusing.  The docstring is
pretty clear as well.  Then again, I know about prog-mode which is
something that newcomers might not?

Re: jinx

[Michael Heerdegen]

Philip Kaludercic <philipk@posteo.net> writes:

> > But the name is still unhelpful.  It seems to enable Flyspell mode for
> > strings and comments only.  But you'd never guess that from its name.
> > The name suggests it is a mode function, but I doesn't follow the
> > conventions for those.
> >
> > Can those things be corrected?
>
> It should be possible to add a defalias with a different name, but
> personally I don't think the name is that confusing.  The docstring is
> pretty clear as well.  Then again, I know about prog-mode which is
> something that newcomers might not?

Could be the intention was to abbreviate something like "flyspell mode
for programming" and that the name doesn't refer to `prog-mode' at all.

If that is true (I don't know) - it would be a very bad name indeed.

Michael.

Re: jinx

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Could be the intention was to abbreviate something like "flyspell mode
> for programming" and that the name doesn't refer to `prog-mode' at all.

Seems `flyspell-prog-mode' is even older than `prog-mode'.

Michael.

Re: jinx

[Eli Zaretskii]

> From: Michael Heerdegen <michael_heerdegen@web.de>
> Date: Wed, 05 Apr 2023 13:25:54 +0200
> 
> Michael Heerdegen <michael_heerdegen@web.de> writes:
> 
> > Could be the intention was to abbreviate something like "flyspell mode
> > for programming" and that the name doesn't refer to `prog-mode' at all.
> 
> Seems `flyspell-prog-mode' is even older than `prog-mode'.

Since flyspell-prog-mode just sets up a mechanism for skipping
user-defined words, I actually see no reason to have "prog" in that
name, none at all.  E.g., as already mentioned, text markup modes may
wish using that, and they have nothing to do with programming
languages.  Likewise, spell-checking some other formatted text, like
email messages.

It's a bad name, that's all.  The question is, can we come up with a
significantly better one, given the generality of the feature?

Re: jinx

[Rudolf Adamkovič]

Gregory Heytings <gregory@heytings.org> writes:

> I think that's no longer true, LanguageTool is available (and has been for 
> ~20 years): https://github.com/languagetool-org/languagetool .  But it's 
> written in Java...

There is also Vale [1], a program written in Go and released under the
MIT license.  It now even supports the Org mode format (along with
Markdown, AsciiDoc, reStructuredText, HTML, and XML).

[1] https://github.com/errata-ai/vale
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and
if it were so, it would be; but as it isn't, it ain't.  That's logic.'"
-- Lewis Carroll, Through the Looking Glass, 1871/1872

Rudolf Adamkovič <salutis@me.com> [he/him]
Studenohorská 25
84103 Bratislava
Slovakia

Re: jinx

[Michael Heerdegen]

Eli Zaretskii <eliz@gnu.org> writes:

> It's a bad name, that's all.  The question is, can we come up with a
> significantly better one, given the generality of the feature?

I have created bug#62677: "Need to find a better name  for
flyspell-prog-mode" for this issue.

Yes, we only need a better name.  "flyspell-restricted-mode"?

But maybe let's better continue in that bug report.

Thx,

Michael.

Re: jinx

[Philip Kaludercic]

Rudolf Adamkovič <salutis@me.com> writes:

> Gregory Heytings <gregory@heytings.org> writes:
>
>> I think that's no longer true, LanguageTool is available (and has been for 
>> ~20 years): https://github.com/languagetool-org/languagetool .  But it's 
>> written in Java...
>
> There is also Vale [1], a program written in Go and released under the
> MIT license.  It now even supports the Org mode format (along with
> Markdown, AsciiDoc, reStructuredText, HTML, and XML).
>
> [1] https://github.com/errata-ai/vale

This seems to be on a different level than what LanguageTool does.  Vale
and other alternatives are style-checkers that catch mistakes matching
some fixed rules, without any "understanding" (linguistic model) of the
underlying language.  IIUC it could be compared to font-locking using
regular expressions or a library like Tree Sitter.

Re: jinx

[Gustavo Barros]

Hi Arash,
Hi All,

Arash Esbati <arash@gnu.org> writes:

> Somewhat off-topic, but it would be great if ispell and flyspell could
> use the same logic for skipping portions of buffer text.  Take for
> example (La)TeX mode: ispell uses `ispell-tex-skip-alists' and flyspell
> uses `flyspell-tex-command-regexp' (which is not very powerful) and then
> a function added to `flyspell-generic-check-word-predicate' (AFAIU).  I
> once wrote an addition for AUCTeX (tex-ispell.el) which extends
> `ispell-tex-skip-alists' for many LaTeX packages but had hard times
> making it work with flyspell as well, skipping multi-line text portions
> was something I couldn't solve for flyspell, e.g.:

I've been trying pretty hard to make `jinx` work with AUCTeX this
week, and perhaps I can chime in. And I also come from a background of
heavily customizing `flyspell` for LaTeX.

Jinx relies heavily on font-lock to be able to ignore certain parts of
the buffer according to their faces. This is a powerful and fast
approach, but comes with some challenges.

I tried to summarize the main ones I found at an issue there
(https://github.com/minad/jinx/issues/25#issuecomment-1512876646, see
also https://github.com/minad/jinx/issues/40), and I shared there the
following summary:

  ;; The way font-lock works in AUCTeX, as done by 'font-latex.el', poses some
  ;; challenges to the face based predicate for ignoring (or not) certain
  ;; parts of the buffer used by Jinx.
  ;;
  ;; 1. All optional arguments are fontified with a single face,
  ;; 'font-lock-variable-name-face', for all classes and all commands, and
  ;; that's hard-coded inside 'font-latex-match-command-with-arguments' with
  ;; no viable way to selectively control it.  Even if arguably most optional
  ;; arguments are meant to be ignored, not all of them are, and there is no
  ;; way to distinguish the cases based on faces alone.
  ;;
  ;; 2. For each keyword class, and hence for every command in the class, a
  ;; single face is applied to all mandatory arguments.  Therefore, if a macro
  ;; has one argument which should be spell checked and another that should
  ;; not, there is no way to distinguish them based on faces alone.
  ;;
  ;; 3. Since the keyword classes are populated with criteria other than spell
  ;; checking in mind it is not always easy to decide if a class of commands
  ;; should have its arguments ignored or not.  Take the "reference" class,
  ;; for example, which includes most cross-referencing commands, whose
  ;; arguments are labels, but also "\footnote", "\footnotetext" and
  ;; "\marginpar".  Besides, some keyword classes share the same face.
  ;;
  ;; 4. Given that (currently) the face predicate is called first by Jinx, at
  ;; the time the regexp predicate is called, point is already inside the
  ;; argument, meaning a regexp for the whole macro won't match, and which
  ;; macro that argument comes from can only be found out by expensive
  ;; backwards searching.
  ;;
  ;; To some extend, number 3 can be dealt with by tweaking font-lock with
  ;; regular tools ('font-latex-add-keywords' etc.), possibly requiring
  ;; personal style files in many cases.  How much this is a meaningful way to
  ;; deal with the problem, I'm not sure.
  ;;
  ;; A reordering of the ignore predicates may help with 4, but it doesn't
  ;; fully solve the problem either.  Because it is easy to use a regexp to
  ;; ignore a command which the faces predicate would not, but not the other
  ;; way around.  So, if you have a command using a face which is part of
  ;; 'jinx-exclude-faces' but should be spell checked, the only way to deal
  ;; with it would be to remove the face altogether from 'jinx-exclude-faces',
  ;; and then handling all the other commands which use that face be ignored
  ;; by regexps.  Not an appealing route.
  ;;
  ;; But numbers 1 and 2 remain impossible to tackle with current machinery
  ;; since there's no way to distinguish the parts of a regexp what should be
  ;; spell checked from those that shouldn't.

It is quite possible though to complement font-lock to cover these
gaps though. My attempt at it ran in the following lines:
https://github.com/minad/jinx/issues/25#issuecomment-1511954696

By now I've generalized this a little, and can reach outstanding
results solely based on font-locking, faces, plus some extra added
command/argument specific properties. Better than what I had on a
heavily customized `flyspell` for the same purpose. I don't add a
single TeX specific regexp to my config.

It must be said though that `flyspell` already supports custom
predicates through `flyspell-generic-check-word-predicate`. So using
the same approach with `flyspell` is a matter of "plugging it in"
there. With faces (and some little extra properties to guide spell
checking) in hand, the predicate itself is trivial.

Best,
Gustavo.