`ispell-message' don't spell the subject line if it starts with a square bracket

`ispell-message' don't spell the subject line if it starts with a square bracket

[Emanuel Berg]

Start with -Q, then do `compose-mail', write something which
is incorrectly spelled in the Subject header line, then do
`ispell-message'.

It works but if you start the subject line with a left square
bracket, [, spelling the subject line doesn't happen. If you
remove it, the subject line gets spell-checked as well as
the body.

You cannot prevent the body from being spelled with
this method.

Probably a bug, right?

So to exemplify how it looks to me

To: 
Subject: [this will prevent spelling] spel1 th1s
From: Emanuel Berg <incal@ebc.mail-host-address-is-not-set>
--text follows this line--
spel1 th1s

--
hello

-- 
underground experts united
https://dataswamp.org/~incal

Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Eli Zaretskii]

> From: Emanuel Berg <incal@dataswamp.org>
> Date: Thu, 09 May 2024 18:00:57 +0200
> 
> Start with -Q, then do `compose-mail', write something which
> is incorrectly spelled in the Subject header line, then do
> `ispell-message'.
> 
> It works but if you start the subject line with a left square
> bracket, [, spelling the subject line doesn't happen. If you
> remove it, the subject line gets spell-checked as well as
> the body.
> 
> You cannot prevent the body from being spelled with
> this method.
> 
> Probably a bug, right?

No, it's a feature.  ispell-message spell-checks only the text written
by the user who composed the message.  So it doesn't spell-check parts
that are quotations from someone else message (the one to which this
one replies), and it doesn't spell-check Subject if it comes from
another message: if it starts with "Re:" (which indicates the Subject
comes from the message to which you reply), or if it starts with "[",
which indicates a forwarded message (so the Subject comes from the
message you are forwarding).

[SOLV3D] Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Emanuel Berg]

Eli Zaretskii wrote:

> No, it's a feature [...] it doesn't spell-check Subject if
> it comes from another message: if it starts with "Re:"
> (which indicates the Subject comes from the message to which
> you reply), or if it starts with "[", which indicates
> a forwarded message

I think "Fwd:" is the convention for forwarded messages, but
you are right, people add such little notes sometimes, like
[SOLVED], [OFF LIST] and so on and then most often the subject
line shouldn't be spell-checked except for within the square
brackets maybe :)

-- 
underground experts united
https://dataswamp.org/~incal

Re: [SOLV3D] Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Karl Fogel]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

On 09 May 2024, Emanuel Berg wrote:
>Eli Zaretskii wrote:
>
>> No, it's a feature [...] it doesn't spell-check Subject if
>> it comes from another message: if it starts with "Re:"
>> (which indicates the Subject comes from the message to which
>> you reply), or if it starts with "[", which indicates
>> a forwarded message
>
>I think "Fwd:" is the convention for forwarded messages, but
>you are right, people add such little notes sometimes, like
>[SOLVED], [OFF LIST] and so on and then most often the subject
>line shouldn't be spell-checked except for within the square
>brackets maybe :)

Yes: I've seen "Fwd:" used, and also "[FWD]".

But there are other little things that people put in square 
brackets that probably *shouldn't* block spell-check.  For 
example, "[PATCH]" and "[PROPOSAL]".

I haven't tested the attached patch, but if there is consensus on 
this general direction of fix, I would be happy to finalize it 
(updating it based on suggestions here) and test it.

Best regards,
-Karl


[-- Attachment #2: ispell-subject-check-narrow-exclusion.diff --]
[-- Type: text/plain, Size: 557 bytes --]

diff --git lisp/textmodes/ispell.el lisp/textmodes/ispell.el
index 17af1f1d926..bd4c8d3de7c 100644
--- lisp/textmodes/ispell.el
+++ lisp/textmodes/ispell.el
@@ -4060,7 +4060,10 @@ ispell-message
 		(progn
 		  (goto-char (match-end 0))
 		  (if (and (not (looking-at ".*\\<Re\\>"))
-			   (not (looking-at "\\[")))
+			   (not (looking-at "Fwd:"))
+			   (not (looking-at "\\[FWD"))
+			   (not (looking-at "\\[SOLVED"))
+			   (not (looking-at "\\[OFF[ -]LIST")))
 		      (progn
 			(setq case-fold-search old-case-fold-search)
 			(ispell-region (point)

Re: [SOLV3D] Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Emanuel Berg]

Karl Fogel wrote:

>> I think "Fwd:" is the convention for forwarded messages,
>> but you are right, people add such little notes sometimes,
>> like [SOLVED], [OFF LIST] and so on and then most often the
>> subject line shouldn't be spell-checked except for within
>> the square brackets maybe :)
>
> Yes: I've seen "Fwd:" used, and also "[FWD]".
>
> But there are other little things that people put in square
> brackets that probably *shouldn't* block spell-check.
> For example, "[PATCH]" and "[PROPOSAL]".

With [SOLVED] and [OFF LIST] there should already be a "Re:"
and with [FWD] there is a "Fwd:" only in a different form.
There is also a "was:".

I think it would be enough too look for those three including
variations and after any one occur, don't spell.

-- 
underground experts united
https://dataswamp.org/~incal

Re: [SOLV3D] Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Emanuel Berg]

Karl Fogel wrote:

>>> No, it's a feature [...] it doesn't spell-check Subject if
>>> it comes from another message: if it starts with "Re:"
>>> (which indicates the Subject comes from the message to
>>> which you reply), or if it starts with "[", which
>>> indicates a forwarded message
>>
>> I think "Fwd:" is the convention for forwarded messages,
>> but you are right, people add such little notes sometimes,
>> like [SOLVED], [OFF LIST] and so on and then most often the
>> subject line shouldn't be spell-checked except for within
>> the square brackets maybe :)
>
> Yes: I've seen "Fwd:" used, and also "[FWD]".
>
> But there are other little things that people put in square
> brackets that probably *shouldn't* block spell-check.
> For example, "[PATCH]" and "[PROPOSAL]".

Another way to approach the thing is to determine in some
other way if the message is a first-in-a-thread on, the
initial mail or post, and if it is, spell the subject, if
not, don't.

But one would still look for headers and header data, maybe
cleaner and more robust than parsing the subject line.

(I would not recommend adding individual hard-coded [CASES] to
the source except for the very common "Re:" etc, they are
already in the Gnus source BTW as regexps and can be reused
for this, likely.)
 
-- 
underground experts united
https://dataswamp.org/~incal

Re: [SOLV3D] Re: `ispell-message' don't spell the subject line if it starts with a square bracket

[Eli Zaretskii]

> From: Karl Fogel <kfogel@red-bean.com>
> Date: Thu, 09 May 2024 19:23:12 -0500
> 
> On 09 May 2024, Emanuel Berg wrote:
> >Eli Zaretskii wrote:
> >
> >> No, it's a feature [...] it doesn't spell-check Subject if
> >> it comes from another message: if it starts with "Re:"
> >> (which indicates the Subject comes from the message to which
> >> you reply), or if it starts with "[", which indicates
> >> a forwarded message
> >
> >I think "Fwd:" is the convention for forwarded messages, but
> >you are right, people add such little notes sometimes, like
> >[SOLVED], [OFF LIST] and so on and then most often the subject
> >line shouldn't be spell-checked except for within the square
> >brackets maybe :)
> 
> Yes: I've seen "Fwd:" used, and also "[FWD]".
> 
> But there are other little things that people put in square 
> brackets that probably *shouldn't* block spell-check.  For 
> example, "[PATCH]" and "[PROPOSAL]".
> 
> I haven't tested the attached patch, but if there is consensus on 
> this general direction of fix, I would be happy to finalize it 
> (updating it based on suggestions here) and test it.

I agree to the general direction, but the actual implementation you
propose is IMO wrong.  Try forwarding a message from Rmail or Gnus
with the default settings, and you will see that "[" is not followed
by any telltale string; the very fact that the subject is in "[...]"
_is_ the only indication of the fact that the message is being
forwarded.  So removing the "\\[" match is not TRT here.

AFAICT, MH-E uses yet another format for forwarded messages.  And I
don't know what other Emacs-based MUAs do with subject of forwarded
and replied messages.

OTOH, just having the "Fwd: " prefix doesn't avoid spell-checking the
Subject, which is a bug of an opposite kind (false negative).

So I think we should leave the "\\[" pattern alone, and instead
override the no-spell-checking decision if only part of the subject is
in brackets.

If we also want not to spell-check the part inside the brackets, we
could instead skip the bracketed part.  (If the entire Subject is
bracketed, as in the default-formatted forwarded messages, that would
avoid spell-checking all of it.)  Not sure how important that is,
since the text inside the brackets is usually auto-generated and thus
correct (see the examples you posted).

There's a more general issue here: ispell-message currently doesn't
know about all the localized variants of "Re:" and "Fwd:", although
the MUAs generally do: see, for example, rmail-re-abbrevs.  So bonus
points for detecting replied and forwarded Subject lines using these
patterns.

P.S. It's too bad this discussion is here and not on the bug tracker.
Bugs, even potential bugs, should _always_ be reported using "M-x
report-emacs-bug".