ispell.el and pipes

ispell.el and pipes

[André A. Gomes]

Hi Emacs,

I'm a layman, so please instruct me.

In the beginning was Ispell (the spelling program) and ispell.el (the
Emacs interface).  The communication is via pipes or ptys.

But today there's Aspell (among others), which provides an external C
API.

Could Emacs leverage it?  What are the pros and cons of pipes/ptys and
the C API for Emacs' sake?

Thanks.


--
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Date: Tue, 16 Nov 2021 17:42:53 +0300
> 
> In the beginning was Ispell (the spelling program) and ispell.el (the
> Emacs interface).  The communication is via pipes or ptys.
> 
> But today there's Aspell (among others), which provides an external C
> API.
> 
> Could Emacs leverage it?  What are the pros and cons of pipes/ptys and
> the C API for Emacs' sake?

The main con is that Someone™ will have to write the code to use the C
API and integrate that into ispell.el.  There are also other cons:

  . the C API provided by Aspell isn't shared by other spell-checkers
    we support (AFAIK)
  . using the C API will need Emacs to be linked against the Aspell
    library, which makes the build requirements and procedures more
    complex
  . different versions of Aspell may introduce binary
    incompatibilities, whose accommodation in Emacs will complicate
    our own code

Re: ispell.el and pipes

[André A. Gomes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: André A. Gomes <andremegafone@gmail.com>
>> Date: Tue, 16 Nov 2021 17:42:53 +0300
>> 
>> In the beginning was Ispell (the spelling program) and ispell.el (the
>> Emacs interface).  The communication is via pipes or ptys.
>> 
>> But today there's Aspell (among others), which provides an external C
>> API.
>> 
>> Could Emacs leverage it?  What are the pros and cons of pipes/ptys and
>> the C API for Emacs' sake?
>
> The main con is that Someone™ will have to write the code to use the C
> API and integrate that into ispell.el.  There are also other cons:
>
>   . the C API provided by Aspell isn't shared by other spell-checkers
>     we support (AFAIK)
>   . using the C API will need Emacs to be linked against the Aspell
>     library, which makes the build requirements and procedures more
>     complex
>   . different versions of Aspell may introduce binary
>     incompatibilities, whose accommodation in Emacs will complicate
>     our own code

Makes sense.

Ispell.el is quite creepy.  I'm working on some fixes.


-- 
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[André A. Gomes]

André A. Gomes <andremegafone@gmail.com> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>>> From: André A. Gomes <andremegafone@gmail.com>
>>> Date: Tue, 16 Nov 2021 17:42:53 +0300
>>>
>>> In the beginning was Ispell (the spelling program) and ispell.el (the
>>> Emacs interface).  The communication is via pipes or ptys.
>>>
>>> But today there's Aspell (among others), which provides an external C
>>> API.
>>>
>>> Could Emacs leverage it?  What are the pros and cons of pipes/ptys and
>>> the C API for Emacs' sake?
>>
>> The main con is that Someone™ will have to write the code to use the C
>> API and integrate that into ispell.el.  There are also other cons:
>>
>>   . the C API provided by Aspell isn't shared by other spell-checkers
>>     we support (AFAIK)
>>   . using the C API will need Emacs to be linked against the Aspell
>>     library, which makes the build requirements and procedures more
>>     complex
>>   . different versions of Aspell may introduce binary
>>     incompatibilities, whose accommodation in Emacs will complicate
>>     our own code

After some thought and research, the only sane way would be to use the
enchant C API.  It claims to be a wrapper around any spell backend
(ispell, aspell, hunspell, etc).  I can't help with this.  But maybe it
will motivate a soul.

> Ispell.el is quite creepy.  I'm working on some fixes.

Here's the creepiest thing so far, and I must share it so that it will
be properly discussed before I can send patches.

Ispell.el uses ascii characters for the user to choose the right
spelling suggestions.  This was probably OK in the pre-historic era of
ispell (the program), since it only suggests words with
Damerau–Levenshtein distance of 1.  Most of the time, there would be 10
or less suggestions, and that justifies why the documentation
erroneously states that the user must press a digit to choose the right
one.

In reality, the valid ascii characters are those starting at 0 (?0,
i.e. 48) up to ?~ (i.e. 126), minus the characters that are bound to
certain commands obviously.  It is like that since, at least, 1994.  I'm
surprised, to say the least.  Is it even possible to select, i.e. type,
a characters that is above ?~???

A few thoughts below.

Why is there a *Choices* buffer in the first place?  IMO, the
*Completions* buffer should be used.

If the old farts (no offence) insist on using *Choices*, then using
ascii characters is silly.  There should be a variable that gives an
upper-bound for spelling suggestions perhaps.  If that bound would be 10
by default, then just use the digits.  Otherwise, the characters used
should be latin lower-case english letters.  (Typing digits is hard).
Either one or two letters, depending on the number of suggestions.  That
gives as much as (expt 26 2) choices, which is more than enough.

Thoughts?


--
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 22:40:19 +0300
> 
> If the old farts (no offence) insist on using *Choices*, then using
> ascii characters is silly.  There should be a variable that gives an
> upper-bound for spelling suggestions perhaps.  If that bound would be 10
> by default, then just use the digits.  Otherwise, the characters used
> should be latin lower-case english letters.  (Typing digits is hard).
> Either one or two letters, depending on the number of suggestions.  That
> gives as much as (expt 26 2) choices, which is more than enough.

A tempest in a teapot, if you ask me.  I'm a heavy user of
spell-checking, always have been.  IME, the correct choice is almost
always '0', rarely '1', i.e. the very first candidates.  If you see
something else, switch to a different speller.  Or use the drop-down
menu of Flyspell ("C-c $").

So I wouldn't burn cycles on trying to improve the UI here.  But
that's me.

(The historical reason why Ispell uses digits is that this is how the
UI of the original Ispell worked; Hunspell still behaves the same even
today.)

Re: ispell.el and pipes

[André A. Gomes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: André A. Gomes <andremegafone@gmail.com>
>> Cc: emacs-devel@gnu.org
>> Date: Tue, 16 Nov 2021 22:40:19 +0300
>>
>> If the old farts (no offence) insist on using *Choices*, then using
>> ascii characters is silly.  There should be a variable that gives an
>> upper-bound for spelling suggestions perhaps.  If that bound would be 10
>> by default, then just use the digits.  Otherwise, the characters used
>> should be latin lower-case english letters.  (Typing digits is hard).
>> Either one or two letters, depending on the number of suggestions.  That
>> gives as much as (expt 26 2) choices, which is more than enough.
>
> A tempest in a teapot, if you ask me.  I'm a heavy user of
> spell-checking, always have been.  IME, the correct choice is almost
> always '0', rarely '1', i.e. the very first candidates.  If you see
> something else, switch to a different speller.  Or use the drop-down
> menu of Flyspell ("C-c $").

What speller do you use, if I may ask?

Well, it's a tempest in a teapot, since you ignored my starting point.
There's a bug, afaik, and I've described it.  Let me remind you below.

> Is it even possible to select, i.e. type, a characters that is above
> ?~???

> (The historical reason why Ispell uses digits is that this is how the
> UI of the original Ispell worked; Hunspell still behaves the same even
> today.)

Same for aspell.  I'm well aware of these historical details.

On another subject, I wonder what would be worth to burn cycles on.  And
how to deal with the idiosyncrasies of the gatekeepers of this
community.


--
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[Robert Pluim]

>>>>> On Tue, 16 Nov 2021 23:40:45 +0300, André A. Gomes <andremegafone@gmail.com> said:
    André> On another subject, I wonder what would be worth to burn cycles on.  And
    André> how to deal with the idiosyncrasies of the gatekeepers of this
    André> community.

etc/TODO has some suggestions. Perhaps we should add "work on
improving your communication skills to avoid ad hominems" to it. I
should perhaps work on that too :-)

Robert
-- 

Re: ispell.el and pipes

[tomas]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

On Wed, Nov 17, 2021 at 10:33:54AM +0100, Robert Pluim wrote:
> >>>>> On Tue, 16 Nov 2021 23:40:45 +0300, André A. Gomes <andremegafone@gmail.com> said:
>     André> On another subject, I wonder what would be worth to burn cycles on.  And
>     André> how to deal with the idiosyncrasies of the gatekeepers of this
>     André> community.
> 
> etc/TODO has some suggestions. Perhaps we should add "work on
> improving your communication skills to avoid ad hominems" to it. I
> should perhaps work on that too :-)

Everyone should, from time to time :)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: ispell.el and pipes

[André A. Gomes]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Tue, 16 Nov 2021 23:40:45 +0300, André A. Gomes <andremegafone@gmail.com> said:
>     André> On another subject, I wonder what would be worth to burn cycles on.  And
>     André> how to deal with the idiosyncrasies of the gatekeepers of this
>     André> community.
>
> etc/TODO has some suggestions. Perhaps we should add "work on
> improving your communication skills to avoid ad hominems" to it. I
> should perhaps work on that too :-)

I never observed/felt ad hominems.

The issue is different.  Time is spent writing and explaining your point
of view.  Sadly, little effort is actually put on understanding.  It's
easy to evade a conversation, but disrespectful.  At the end of the day,
the sacred gatekeepers have the final word.

What am I doing here?  This is a waste of time.


--
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[Robert Pluim]

>>>>> On Wed, 17 Nov 2021 14:01:48 +0300, André A. Gomes <andremegafone@gmail.com> said:

    André> Robert Pluim <rpluim@gmail.com> writes:
    >>>>>>> On Tue, 16 Nov 2021 23:40:45 +0300, André A. Gomes <andremegafone@gmail.com> said:
    André> On another subject, I wonder what would be worth to burn cycles on.  And
    André> how to deal with the idiosyncrasies of the gatekeepers of this
    André> community.
    >> 
    >> etc/TODO has some suggestions. Perhaps we should add "work on
    >> improving your communication skills to avoid ad hominems" to it. I
    >> should perhaps work on that too :-)

    André> I never observed/felt ad hominems.

Saying 'how to deal with the idiosyncrasies of the gatekeepers of
community' when engaging in a disagreement with Eli comes across as ad
hominem to me. If Iʼm mistaken I apologise.

    André> The issue is different.  Time is spent writing and explaining your point
    André> of view.  Sadly, little effort is actually put on understanding.  It's
    André> easy to evade a conversation, but disrespectful.  At the end of the day,
    André> the sacred gatekeepers have the final word.

And again: 'sacred gatekeepers'. Nobody here is sacred, and theyʼre
not 'gatekeepers'. That word has taken on a very negative connotation
in recent years.

Robert
-- 

Re: ispell.el and pipes

[André A. Gomes]

Robert Pluim <rpluim@gmail.com> writes:

>>>>>> On Wed, 17 Nov 2021 14:01:48 +0300, André A. Gomes <andremegafone@gmail.com> said:
>
>     André> Robert Pluim <rpluim@gmail.com> writes:
>     >>>>>>> On Tue, 16 Nov 2021 23:40:45 +0300, André A. Gomes <andremegafone@gmail.com> said:
>     André> On another subject, I wonder what would be worth to burn cycles on.  And
>     André> how to deal with the idiosyncrasies of the gatekeepers of this
>     André> community.
>     >> 
>     >> etc/TODO has some suggestions. Perhaps we should add "work on
>     >> improving your communication skills to avoid ad hominems" to it. I
>     >> should perhaps work on that too :-)
>
>     André> I never observed/felt ad hominems.
>
> Saying 'how to deal with the idiosyncrasies of the gatekeepers of
> community' when engaging in a disagreement with Eli comes across as ad
> hominem to me. If Iʼm mistaken I apologise.

Interesting.  I won't even try to explain my point of view anymore.

Apologies to Eli.


-- 
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[tomas]

[-- Attachment #1: Type: text/plain, Size: 1147 bytes --]

On Wed, Nov 17, 2021 at 12:08:10PM +0100, Robert Pluim wrote:

[...]

> Saying 'how to deal with the idiosyncrasies of the gatekeepers of
> community' when engaging in a disagreement with Eli comes across as ad
> hominem to me. If Iʼm mistaken I apologise.

It is, at least unnecessarily loaded.

>     André> The issue is different.  Time is spent writing and explaining your point
>     André> of view.  Sadly, little effort is actually put on understanding.  It's
>     André> easy to evade a conversation, but disrespectful.  At the end of the day,
>     André> the sacred gatekeepers have the final word.
> 
> And again: 'sacred gatekeepers'. Nobody here is sacred, and theyʼre
> not 'gatekeepers'. That word has taken on a very negative connotation
> in recent years.

I think this negative connotation is intended.


And "...little effort is actually...". That's, I think, provably false.

Perhaps André is frustrated that the process takes more time and
involvement he hoped for, and this is understandable, but trying to
put all the blame at someone else's doorstep isn't credible to me.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: ispell.el and pipes

[dick]

It is a common mistake to dismiss *ad hominem*.  In all the top-10 lists for
logical fallacies, *ad hominem* often appears first when it shouldn't be on
the list at all.

A Bayesian argument can easily be made to decrease the posterior likelihood of
a proposal's correctness by pointing out the foibles and motives of the person
making it.  People have accused me of bloviative grandstanding, and I must
concede my narcissism does cast doubt on my code proposals.

While it's not nice, it's not exactly wrong.

Re: ispell.el and pipes

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Tue, 16 Nov 2021 23:40:45 +0300
> 
> What speller do you use, if I may ask?

I'm a long-time and happy user of Hunspell.

> > Is it even possible to select, i.e. type, a characters that is above
> > ?~???

It _is_ possible to select it, but maybe inconvenient, and showing
such "characters" could maybe confuse (if we ever get to showing so
many candidates: e.g., Hunspell limits the number of suggestions to
just 15, AFAICT).  So if you or someone else wants to submit patches
to skip all the codepoints between 128 and 160, I'm okay with that.

> On another subject, I wonder what would be worth to burn cycles on.

We record that in etc/TODO.

> And how to deal with the idiosyncrasies of the gatekeepers of this
> community.

Either live with them or replace them?

Re: ispell.el and pipes

[tomas]

[-- Attachment #1: Type: text/plain, Size: 196 bytes --]

On Wed, Nov 17, 2021 at 07:08:25AM -0500, dick wrote:

[...]

> While it's not nice, it's not exactly wrong.

I think the point was being rather made about the
first, not the second.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: ispell.el and pipes

[André A. Gomes]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Is it even possible to select, i.e. type, a characters that is above
>> > ?~???
>
> It _is_ possible to select it, but maybe inconvenient, and showing
> such "characters" could maybe confuse (if we ever get to showing so
> many candidates: e.g., Hunspell limits the number of suggestions to
> just 15, AFAICT).  So if you or someone else wants to submit patches
> to skip all the codepoints between 128 and 160, I'm okay with that.

Could you please explain me how can I select those?  For instance,
"\200".

I've always used aspell.  For example, triggering M-$ on "twr" yields
100 suggestions.

What happens between codepoints 128 and 160?  Thank you.


--
André A. Gomes
"Free Thought, Free World"

Re: ispell.el and pipes

[Eli Zaretskii]

> From: André A. Gomes <andremegafone@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Wed, 17 Nov 2021 16:23:37 +0300
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > Is it even possible to select, i.e. type, a characters that is above
> >> > ?~???
> >
> > It _is_ possible to select it, but maybe inconvenient, and showing
> > such "characters" could maybe confuse (if we ever get to showing so
> > many candidates: e.g., Hunspell limits the number of suggestions to
> > just 15, AFAICT).  So if you or someone else wants to submit patches
> > to skip all the codepoints between 128 and 160, I'm okay with that.
> 
> Could you please explain me how can I select those?  For instance,
> "\200".

It's just a codepoint, so you need a keyboard that can send it.  Not
many will do that without tinkering, admittedly.

> I've always used aspell.  For example, triggering M-$ on "twr" yields
> 100 suggestions.

Like I said: switch to a better speller.  Hunspell suggests just 7.

Maybe we should limit the list to some reasonably small length.

> What happens between codepoints 128 and 160?

They are non-printable characters, so not many keyboards are able to
produce them.  After that, you have Latin-1, so much easier.

Re: ispell.el and pipes

[André A. Gomes]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: André A. Gomes <andremegafone@gmail.com>
>> Cc: emacs-devel@gnu.org
>> Date: Wed, 17 Nov 2021 16:23:37 +0300
>>
>> Eli Zaretskii <eliz@gnu.org> writes:
>>
>> >> > Is it even possible to select, i.e. type, a characters that is above
>> >> > ?~???
>> >
>> > It _is_ possible to select it, but maybe inconvenient, and showing
>> > such "characters" could maybe confuse (if we ever get to showing so
>> > many candidates: e.g., Hunspell limits the number of suggestions to
>> > just 15, AFAICT).  So if you or someone else wants to submit patches
>> > to skip all the codepoints between 128 and 160, I'm okay with that.
>>
>> Could you please explain me how can I select those?  For instance,
>> "\200".
>
> It's just a codepoint, so you need a keyboard that can send it.  Not
> many will do that without tinkering, admittedly.

I use the US altgr-intl layout so I have access to some of those
codepoints.  For instance, ?á.  However, when I try to select that
option from the *Choices* buffer, it doesn't work.  I'm not sure what's
going on, or how to debug it.

>> I've always used aspell.  For example, triggering M-$ on "twr" yields
>> 100 suggestions.
>
> Like I said: switch to a better speller.  Hunspell suggests just 7.

I find your (valuable) suggestion orthogonal to the topic.  Ispell.el is
supposed to be agnostic, and aspell is the "official" GNU spell
program.

> Maybe we should limit the list to some reasonably small length.

I find this suggestion reasonable.  I'll play with that along with the
suggestion to skip between 128 and 160.


--
André A. Gomes
"Free Thought, Free World"