bug#49926: [PATCH] Fix NNIMAP search command in the gnus

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Jan Stranik via Bug reports for GNU Emacs, the Swiss army knife of text editors]

In the version 27.2 of emacs, the nnimap search commands stopped
working in gnus.

Good example is a command AT to find a referred
thread in the summary buffer. The observed behaviour was that the
search command returned no error.

The problem turned out to be in the way how imap query is sent to the
server. The function nnimap-make-thread-query used a format function
with foramt specifier %S. For string values with formatting the string
returned is in the format #("string" ....). The result was tha the
query sent to the server looked like:
  23:52:00 [stranik.org] 1980 UID SEARCH (OR HEADER REFERENCES #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)) HEADER Message-Id #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)))

which is an invalid query.

The change formats the string with %s specifier which discards text
properties.
---
 lisp/gnus/nnimap.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 3cf65453f3..33bd6ee5a8 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -2295,11 +2295,11 @@ Return the server's response to the SELECT or EXAMINE command."
 		    "")))
 	 (value
 	  (format
-	   "(OR HEADER REFERENCES %S HEADER Message-Id %S)"
+	   "(OR HEADER REFERENCES %s HEADER Message-Id %s)"
 	   id id)))
     (dolist (refid refs value)
       (setq value (format
-		   "(OR (OR HEADER Message-Id %S HEADER REFERENCES %S) %s)"
+		   "(OR (OR HEADER Message-Id %s HEADER REFERENCES %s) %s)"
 		   refid refid value)))))
 
 
-- 
2.31.1

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Eric Abrahamsen]

Jan Stranik via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs@gnu.org> writes:

> In the version 27.2 of emacs, the nnimap search commands stopped
> working in gnus.
>
> Good example is a command AT to find a referred
> thread in the summary buffer. The observed behaviour was that the
> search command returned no error.
>
> The problem turned out to be in the way how imap query is sent to the
> server. The function nnimap-make-thread-query used a format function
> with foramt specifier %S. For string values with formatting the string
> returned is in the format #("string" ....). The result was tha the
> query sent to the server looked like:
>   23:52:00 [stranik.org] 1980 UID SEARCH (OR HEADER REFERENCES #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)) HEADER Message-Id #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)))
>
> which is an invalid query.
>
> The change formats the string with %s specifier which discards text
> properties.

But that also removes the quoting around the message ids -- are we sure
that's still valid?

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Eric Abrahamsen]

On 08/08/21 14:29 PM, Jan Stranik wrote:
> On 08/07/21 16:06 PM, Eric Abrahamsen wrote:
>> Jan Stranik via "Bug reports for GNU Emacs, the Swiss army knife of text
>> editors" <bug-gnu-emacs@gnu.org> writes:
>>
>>> In the version 27.2 of emacs, the nnimap search commands stopped
>>> working in gnus.
>>>
>>> Good example is a command AT to find a referred
>>> thread in the summary buffer. The observed behaviour was that the
>>> search command returned no error.
>>>
>>> The problem turned out to be in the way how imap query is sent to the
>>> server. The function nnimap-make-thread-query used a format function
>>> with foramt specifier %S. For string values with formatting the string
>>> returned is in the format #("string" ....). The result was tha the
>>> query sent to the server looked like:
>>>   23:52:00 [stranik.org] 1980 UID SEARCH (OR HEADER REFERENCES
>>> #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)) HEADER
>>> Message-Id #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg
>>> chg)))
>>>
>>> which is an invalid query.
>>>
>>> The change formats the string with %s specifier which discards text
>>> properties.
>>
>> But that also removes the quoting around the message ids -- are we sure
>> that's still valid?

(Putting debbugs back in the cc)

> You're right. Per the specification, the strings are expected to be
> either in literal or quotes syntax
> (https://datatracker.ietf.org/doc/html/rfc3501#section-4.3).
>
> Adding quotes back to make the strings to be in quoted syntax.
>
> Interestingly, dovecot seems to accept the strings without quotes as
> well.

I've found dovecot to be generally pretty permissive, though obviously
we want to stick closely to the RFCs here.

> New patch attached.

This seems fine, but just thinking out loud: is there anything else the
%S could potentially get us, here? Extra quoting of special characters?

Our other option would be to explicitly remove properties from the
strings beforehand; I'm just wondering if one approach is preferable
over another.

Eric

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> (Putting debbugs back in the cc)

[...]

>> New patch attached.

The new version of the patch didn't make it to the bug tracker.  Could
you re-send it, Jan?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 359 bytes --]


On 11/06/21 03:39 AM, Lars Ingebrigtsen wrote:
> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> (Putting debbugs back in the cc)
>
> [...]
>
>>> New patch attached.
>
> The new version of the patch didn't make it to the bug tracker.  Could
> you re-send it, Jan?

He sent that directly to me, attached here. I guess I don't see any
problem with it.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-Fix-NNIMAP-search-command-in-the-gnus.patch --]
[-- Type: text/x-patch, Size: 1774 bytes --]

From 7f877856766b3e6b668ce4dc52130a3367f5dc6f Mon Sep 17 00:00:00 2001
From: Jan Stranik <jan@stranik.org>
Date: Sat, 7 Aug 2021 00:17:03 -0400
Subject: [PATCH] Fix NNIMAP search command in the gnus

In the version 27.2 of emacs, the nnimap search commands stopped
working in gnus.

Good example is a command AT to find a referred
thread in the summary buffer. The observed behaviour was that the
search command returned no error.

The problem turned out to be in the way how imap query is sent to the
server. The function nnimap-make-thread-query used a format function
with foramt specifier %S. For string values with formatting the string
returned is in the format #("string" ....). The result was tha the
query sent to the server looked like:
  23:52:00 [stranik.org] 1980 UID SEARCH (OR HEADER REFERENCES #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)) HEADER Message-Id #("<87pmurac3u.fsf@stranik.org>" 0 28 (ws-butler-chg chg)))

which is an invalid query.

The change formats the string with %s specifier which discards text
properties.
---
 lisp/gnus/nnimap.el | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index 3cf65453f3..6e3c411bf2 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -2295,11 +2295,11 @@ Return the server's response to the SELECT or EXAMINE command."
 		    "")))
 	 (value
 	  (format
-	   "(OR HEADER REFERENCES %S HEADER Message-Id %S)"
+	   "(OR HEADER REFERENCES \"%s\" HEADER Message-Id \"%s\")"
 	   id id)))
     (dolist (refid refs value)
       (setq value (format
-		   "(OR (OR HEADER Message-Id %S HEADER REFERENCES %S) %s)"
+		   "(OR (OR HEADER Message-Id \"%s\" HEADER REFERENCES \"%s\") %s)"
 		   refid refid value)))))
 
 
-- 
2.31.1

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> -	   "(OR HEADER REFERENCES %S HEADER Message-Id %S)"
> +	   "(OR HEADER REFERENCES \"%s\" HEADER Message-Id \"%s\")"
>  	   id id)))

This is not correct -- this would yield an incorrect syntax for IDs that
contain a " character.  I've now fixed this in emacs-28 by just removing
the text properties from `id' instead.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#49926: [PATCH] Fix NNIMAP search command in the gnus

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> -	   "(OR HEADER REFERENCES %S HEADER Message-Id %S)"
>> +	   "(OR HEADER REFERENCES \"%s\" HEADER Message-Id \"%s\")"
>>  	   id id)))
>
> This is not correct -- this would yield an incorrect syntax for IDs that
> contain a " character.  I've now fixed this in emacs-28 by just removing
> the text properties from `id' instead.

Sounds good.