Grammar checking in Emacs

Grammar checking in Emacs

[Daniel Martín]

Emacs has good support for spell checking text written in multiple
languages.  However, there is currently no support for grammar checking,
and I think that feature may be useful for people writing long documents
and books with Emacs.

I've found a few discussions in emacs-devel about grammar checking, and
the general feeling in the past has been that grammar checking is not
useful enough, because the tools only provide some text linting
functionality that just offers some basic style suggestions.

The question is if those feelings about grammar checking still hold
true.  I think one of the most popular free programs to do grammar
checking is LanguageTool (https://languagetool.org).  I've quickly
tested it and it detected some non-trivial grammar mistakes, in English
and Spanish at least.  I didn't see false positives, which is a good
thing in this kind of tools.  You can try the tool from its website
without installing it on your computer.

Is there interest in having this feature in Emacs?  Perhaps as an ELPA
package, if the technology is not mature enough to have it in Core?

Re: Grammar checking in Emacs

[Stefan Monnier]

> The question is if those feelings about grammar checking still hold
> true.  I think one of the most popular free programs to do grammar
> checking is LanguageTool (https://languagetool.org).  I've quickly
> tested it and it detected some non-trivial grammar mistakes, in English
> and Spanish at least.  I didn't see false positives, which is a good
> thing in this kind of tools.  You can try the tool from its website
> without installing it on your computer.

IIUC (after looking at https://github.com/languagetool-org/languagetool)
this is structured as a grammar-checking server (written in Java) that
can run on a remote host and then client applications connect to it and
presumably send their text to it a bit like they'd do with an LSP
server, except the clients aren't expected to launch the server.

Sadly it's not packaged for Debian (see
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
the details).  I can't see any info about using it on Fedora either.
AFAICT the most popular way to install it is as a "snap" package, which
I consider to be a euphemism for a blob :-(

> Is there interest in having this feature in Emacs?  Perhaps as an ELPA
> package, if the technology is not mature enough to have it in Core?

I think it would be nice to provide a package for such grammar checking,
yes.  We should be careful not to encourage users to just connect to
someone else's LaguageTool server, since that would be SaaSS, but other
than that I see no reason not to have such a package in GNU ELPA.
I don't expect many users to have such a server, so it's probably not
widespread enough to justify having it in core, currently.


        Stefan

Re: Grammar checking in Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

On Sun, Nov 14, 2021 at 09:46:19PM -0500, Stefan Monnier wrote:
> > The question is if those feelings about grammar checking still hold
> > true.  I think one of the most popular free programs to do grammar
> > checking is LanguageTool (https://languagetool.org) [...]

Gah: https://www.googletagmanager.com/gtm.js?id='+i+dl;var n=d.querySelector('[nonce]');

Now on my blacklist.

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Grammar checking in Emacs

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 1129 bytes --]

On Sun, 14 Nov 2021, Stefan Monnier wrote:

> IIUC (after looking at https://github.com/languagetool-org/languagetool)
> this is structured as a grammar-checking server (written in Java) that
> can run on a remote host and then client applications connect to it and
> presumably send their text to it a bit like they'd do with an LSP
> server, except the clients aren't expected to launch the server.

I recently enhanced flycheck-languagetool so that it’s able to start its own server if so configured, so I’m sure something similar could be done with the various other LanguageTool modes.

> Sadly it's not packaged for Debian (see
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
> the details).  I can't see any info about using it on Fedora either.

Fedora have pretty much given up on packaging Java applications now, I think.

> AFAICT the most popular way to install it is as a "snap" package, which
> I consider to be a euphemism for a blob :-(

You can unzip prebuilt JARs from https://languagetool.org/download/LanguageTool-stable.zip, which seems simpler but similarly blobby.

-- 
Peter Oliver

Re: Grammar checking in Emacs

[Stefan Monnier]

> I recently enhanced flycheck-languagetool so that it’s able to start its own
> server if so configured, so I’m sure something similar could be done with
> the various other LanguageTool modes.

Any chance you could port it to `flymake` and contribute it to GNU ELPA?


        Stefan

Re: Grammar checking in Emacs

[Philip Kaludercic]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I recently enhanced flycheck-languagetool so that it’s able to start its own
>> server if so configured, so I’m sure something similar could be done with
>> the various other LanguageTool modes.
>
> Any chance you could port it to `flymake` and contribute it to GNU ELPA?

Should this use flymake or should ispell be generalized to handle
grammar checking as well?  Usually flymake doesn't offer you direct
suggestions on how to fix code (or at least I haven't seen it yet),
while ispell gives me a list of possible fixes.

-- 
	Philip Kaludercic

Re: Grammar checking in Emacs

[Eli Zaretskii]

> From: Philip Kaludercic <philipk@posteo.net>
> Date: Tue, 16 Nov 2021 18:01:48 +0000
> Cc: emacs-devel@gnu.org, Daniel Martín <mardani29@yahoo.es>
> 
> Stefan Monnier <monnier@iro.umontreal.ca> writes:
> 
> > Any chance you could port it to `flymake` and contribute it to GNU ELPA?
> 
> Should this use flymake or should ispell be generalized to handle
> grammar checking as well?  Usually flymake doesn't offer you direct
> suggestions on how to fix code (or at least I haven't seen it yet),
> while ispell gives me a list of possible fixes.

Ispell's interface for the list of fixes is not really appropriate for
grammar checking, since in the latter case one frequently needs to
replace entire phrases, and also because in many cases the checker
needs to display an elaborate explanation of the mistake.  So I think
IU-wise neither Ispell nor Flymake's default presentation are good
candidates, perhaps something like flymake-show-buffer-diagnostics.

Caveat: my experience with using grammar checkers is very limited.

Re: Grammar checking in Emacs

[Stefan Monnier]

> Should this use flymake or should ispell be generalized to handle
> grammar checking as well?

I don't think ispell/flyspell are particularly well suited for
today's expectations.  I'd welcome a new interface to ispell based on
something like jit-lock or flymake.

> Usually flymake doesn't offer you direct suggestions on how to fix
> code (or at least I haven't seen it yet), while ispell gives me a list
> of possible fixes.

I think flymake would benefit from improvements to make this possible,
and several flymake backends could take advantage (e.g. checkdoc).

For me the question is rather whether flymake performs well enough when
opening a 10MB file.


        Stefan

Re: Grammar checking in Emacs

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 531 bytes --]

On Mon, 15 Nov 2021, Stefan Monnier wrote:

>> I recently enhanced flycheck-languagetool so that it’s able to start its own
>> server if so configured, so I’m sure something similar could be done with
>> the various other LanguageTool modes.
>
> Any chance you could port it to `flymake` and contribute it to GNU ELPA?

I’m not a `flymake` user myself, so I don’t think I’d be the best person for the job, but I’ve suggested it at <https://github.com/emacs-languagetool/flymake-languagetool/issues/3>.

-- 
Peter Oliver

Re: Grammar checking in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > AFAICT the most popular way to install it is as a "snap" package, which
  > > I consider to be a euphemism for a blob :-(

  > You can unzip prebuilt JARs from https://languagetool.org/download/LanguageTool-stable.zip, which seems simpler but similarly blobby.

In principle, those JARs could make it easier to see the full list of
contents and determine what nonfree files there are, if any.  But it
looks like you don't think so.  Would you please expand on why you
don't think so?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Grammar checking in Emacs

[Peter Oliver]

[-- Attachment #1: Type: text/plain, Size: 1177 bytes --]

On Tue, 16 Nov 2021, Richard Stallman wrote:

>  > > AFAICT the most popular way to install it is as a "snap" package, which
>  > > I consider to be a euphemism for a blob :-(
>
>  > You can unzip prebuilt JARs from https://languagetool.org/download/LanguageTool-stable.zip, which seems simpler but similarly blobby.
>
> In principle, those JARs could make it easier to see the full list of
> contents and determine what nonfree files there are, if any.  But it
> looks like you don't think so.  Would you please expand on why you
> don't think so?

I was thinking only about our ability to modify the contents.

If a reputable GNU/Linux distribution includes a software package, then we can be sure that it’s known how to reproducibly build it from source.  If it becomes too hard for distributions to build from source, and people come to rely on developer-supplied pre-built artefacts instead (be they binaries, bytecode, container images, or whatever the next thing is), then I would be less confident.

My main point was to draw attention to the availability of the JARs.  The mention of blobbyness was only intended as a throwaway aside, to be honest.

-- 
Peter Oliver

Re: Grammar checking in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Sadly it's not packaged for Debian (see
  > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
  > the details).  I can't see any info about using it on Fedora either.
  > AFAICT the most popular way to install it is as a "snap" package, which
  > I consider to be a euphemism for a blob :-(

I think the same.

I am starting to think that the GNU Project needs to adopt standard
not to steer people towards programs which are released primarily via
containers, on the grounds that it is too hard to tell whether they
are actually free.

This would also cover the programs that are built using package library
managers such as pip and cargo, which don't provide a way to reject
the nonfree packages they contain.  Basically, the difference between
these and container managers are not significant for this issue.

Some of the packages are free, and some of them may work with
only free packages, but unless we can say how to install them
in a way that you can verify is free, we cannot that that as known.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Grammar checking in Emacs

[Ihor Radchenko]

Richard Stallman <rms@gnu.org> writes:

>   > Sadly it's not packaged for Debian (see
>   > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
>   > the details).  I can't see any info about using it on Fedora either.
>   > AFAICT the most popular way to install it is as a "snap" package, which
>   > I consider to be a euphemism for a blob :-(
>
> I think the same.
>
> I am starting to think that the GNU Project needs to adopt standard
> not to steer people towards programs which are released primarily via
> containers, on the grounds that it is too hard to tell whether they
> are actually free.

Not opposing Richard's statements in general, I would like to provide
several clarifications about LanguageTool:
1. It does not _require_ a server but can be used as a command line tool
2. It's source code is available via
   git clone https://github.com/languagetool-org/languagetool
   It is licensed under LGPL.

Best,
Ihor

Re: Grammar checking in Emacs

[Stefan Monnier]

> Not opposing Richard's statements in general, I would like to provide
> several clarifications about LanguageTool:
> 1. It does not _require_ a server but can be used as a command line tool
> 2. It's source code is available via
>    git clone https://github.com/languagetool-org/languagetool
>    It is licensed under LGPL.

Indeed.  And IIUC the main reason why it's not in Debian (other than
lack of time/dedication to package it up) seems to be that its language
models have a murky copyright because they've been trained on text files
whose copyright isn't itself clean.

In my view, this is not a problem of software freedom, so I do consider
LanguageTool to be philosophically perfectly acceptable.
It's just technically rather annoying to install :-(
[ Installing a JAR file is no solution, since you then have to worry
  about keeping it up-to-date.  ]


        Stefan

Re: Grammar checking in Emacs

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> In my view, this is not a problem of software freedom, so I do consider
> LanguageTool to be philosophically perfectly acceptable.
> It's just technically rather annoying to install :-(
> [ Installing a JAR file is no solution, since you then have to worry
>   about keeping it up-to-date.  ]
>

Just interested in why you find installing a jar file so problematic.
Given it is essentially just a zip archive, how is installing a jar file
and keeping it up-to-date any different from installing any binary
package? Even when you clone a repository and build from sources, you
still have to pull updates and rebuild. Is their something fundamentally
different with jar or is it simply a different build system and the
discomfort is due to it being less familiar?

Re: Grammar checking in Emacs

[Stefan Monnier]

> Just interested in why you find installing a jar file so problematic.
> Given it is essentially just a zip archive, how is installing a jar file
> and keeping it up-to-date any different from installing any binary
> package?

Not sure what you mean by "installing any binary package", but the way
I normally install a package is "apt install ....", and the "keeping it
up-to-date" is done mostly automatically by updating the whole system on
a regular basis (including via the `unattended-upgrade` package).

> Even when you clone a repository and build from sources, you
> still have to pull updates and rebuild.

Indeed, it sucks just about as much.

> Is their something fundamentally different with jar or is it simply
> a different build system and the discomfort is due to it being
> less familiar?

No, I'm comparing to something that's directly supported by
my distribution (Debian).


        Stefan

Re: Grammar checking in Emacs

[Alexandre Garreau]

Le lundi 15 novembre 2021, 03:46:19 CET Stefan Monnier a écrit :
> > The question is if those feelings about grammar checking still hold
> > true.  I think one of the most popular free programs to do grammar
> > checking is LanguageTool (https://languagetool.org).  I've quickly
> > tested it and it detected some non-trivial grammar mistakes, in
> > English
> > and Spanish at least.  I didn't see false positives, which is a good
> > thing in this kind of tools.  You can try the tool from its website
> > without installing it on your computer.
> 
> IIUC (after looking at https://github.com/languagetool-org/languagetool)
> this is structured as a grammar-checking server (written in Java) that
> can run on a remote host and then client applications connect to it and
> presumably send their text to it a bit like they'd do with an LSP
> server, except the clients aren't expected to launch the server.
> 
> Sadly it's not packaged for Debian (see
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
> the details).  I can't see any info about using it on Fedora either.
> AFAICT the most popular way to install it is as a "snap" package, which
> I consider to be a euphemism for a blob :-(

a very heavy and hype blob

> > Is there interest in having this feature in Emacs?  Perhaps as an ELPA
> > package, if the technology is not mature enough to have it in Core?
> 
> I think it would be nice to provide a package for such grammar checking,
> yes.  We should be careful not to encourage users to just connect to
> someone else's LaguageTool server, since that would be SaaSS, but other
> than that I see no reason not to have such a package in GNU ELPA. I
> don't expect many users to have such a server, so it's probably not
> widespread enough to justify having it in core, currently.

I wonder if the increasing tendency to write software extensions as 
servers instead of libraries is calculated not only to promote SaaSS and 
dependency, but even to promote a way of interfacing software that allows 
circumventing copyleft…

Re: Grammar checking in Emacs

[Stefan Monnier]

> I wonder if the increasing tendency to write software extensions as 
> servers instead of libraries is calculated not only to promote SaaSS and 
> dependency, but even to promote a way of interfacing software that allows 
> circumventing copyleft…

I'm pretty sure that's of no concern.
"Don't attribute to malice that which ...."


        Stefan

Re: Grammar checking in Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

On Tue, Nov 16, 2021 at 09:30:07AM -0500, Stefan Monnier wrote:
> > I wonder if the increasing tendency to write software extensions as 
> > servers instead of libraries is calculated not only to promote SaaSS and 
> > dependency, but even to promote a way of interfacing software that allows 
> > circumventing copyleft…
> 
> I'm pretty sure that's of no concern.
> "Don't attribute to malice that which ...."

OTOH, there goes my favourite, let's call it "Bernie's Bastard" [1], [2].

  "Any sufficiently advanced malice is indistinguishable from stupidity"

It's dangerous, because it tends to spawn conspiracy theories,

Cheers
[1] https://en.wikipedia.org/wiki/Clarke%27s_Laws
[2] https://en.wikipedia.org/wiki/Hanlon's_Razor

 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Grammar checking in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > I wonder if the increasing tendency to write software extensions as 
  > > servers instead of libraries is calculated not only to promote SaaSS and 
  > > dependency, but even to promote a way of interfacing software that allows 
  > > circumventing copyleft…

  > I'm pretty sure that's of no concern.
  > "Don't attribute to malice that which ...."

I too think it unlikely that this reflects a plan intended to harm the
free software movement.  I don't think they care that much about us.
But then, their motives are a side issue; what matters is the effect.
Whether it is a plan to harm us, a plan to increase their profits, a
plan to make users increasingly captive, or no plan at all, we need to
address the effect on our goals.

Whether intentionally or not, designing computational features to be
used over the network tends to deny users the possibility of control
over their computing. and to weaken our defenses.  It also tends to
create more opportunities for general surveillance.

What should we do about it?  We should push it away from us.  We
should, by policy, _not_ add features to Emacs that naturally work
with across-the-network computational features, and instead add new
Emacs features whose natural mode of operation works with code running
on the same computer as Emacs.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: Grammar checking in Emacs

[tomas]

[-- Attachment #1: Type: text/plain, Size: 454 bytes --]

On Sun, Nov 14, 2021 at 09:46:19PM -0500, Stefan Monnier wrote:

[...]

> Sadly it's not packaged for Debian (see
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403619 suggests for
> the details).  I can't see any info about using it on Fedora either.
> AFAICT the most popular way to install it is as a "snap" package, which
> I consider to be a euphemism for a blob :-(

May I use /that/ quote? With proper attribution, of course :-)

Cheers
 - t

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Grammar checking in Emacs

[Rudolf Adamkovič]

Daniel Martín <mardani29@yahoo.es> writes:

> Is there interest in having this feature in Emacs?  Perhaps as an ELPA
> package, if the technology is not mature enough to have it in Core?

I would *love* to see grammar checking support in core Emacs, but I would not want to see Emacs tied to LanguageTool. I would welcome a set of hooks (in the more general sense), perhaps similar to Flymake but easier to set up. I currently use Vale and Flymake to do both grammar and spell checking, and I find the experience quite painful. I disabled Emacs internal spell checking, for I do not want to maintain two sets of regular expressions, one for grammar checking and one for spell checking, that define what parts of text to ignore.

Rudy
-- 
"'Contrariwise,' continued Tweedledee, 'if it was so, it might be; and if it were so, it would be; but as it isn't, it ain't. That's logic.'" -- Lewis Carroll, Through the Looking Glass

Rudolf Adamkovič <salutis@me.com>
Studenohorská 25
84103 Bratislava
Slovakia

[he/him]

Re: Grammar checking in Emacs

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Grammar checking is a very desirable feature to provide in the GNU
system.  To provide this feature in the GNU system means that the
system includes free programs that do the job.

These programs don't have to be part of Emacs.  They don't have to be
GNU packages -- it's fine if they are developed and released by
others.  The requisites are: they have to be free/libre, and they have
to be easy install and run on your own computer so that they work with
Emacs without requiring any network connection.

They should not push people into dependence on servers run by others
for a task which doesn't inherently involve someone else.  That
conflicts with the deepest form of our goal: for users to have control
of their computing.

If some existing free software comes close to doing this, that's
great.  If it doesn't quite get all the way there, would people like
to work on bringing it the rest of the way?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)