bug#42131: Cannot split to IMAP folders

bug#42131: Cannot split to IMAP folders

[David A. Greene]

I am running a local dovecot instance to serve IMAP to gnus.  I have set
up splitting rules which `B q` shows work correctly:

  This message would go to home/school

However, `B r` results in:

  nnimap-request-accept-article: Writing to process: Bad file descriptor, *nnimap*

Dovecot shows this in the log:

  Jun 29 15:27:02 imap(dag)<14994><2s+d4T6pHsN/AAAB>: Debug: Mailbox work/INBOX: Mailbox opened because: SELECT
  Jun 29 15:27:02 imap(dag)<14994><2s+d4T6pHsN/AAAB>: Debug: Mailbox work/INBOX: Mailbox opened because: SELECT
  Jun 29 15:27:02 imap(dag)<14994><2s+d4T6pHsN/AAAB>: Debug: Mailbox work/INBOX: UID 2429: Opened mail because: full mail
  Jun 29 15:27:02 imap(dag)<14994><2s+d4T6pHsN/AAAB>: Debug: Mailbox mail.misc: Mailbox opened because: APPEND
  Jun 29 15:27:02 imap(dag)<14994><2s+d4T6pHsN/AAAB>: Info: Too many invalid IMAP commands. in=1444 out=626842 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=622647

I'm not sure where "mail.misc" is coming from.

I am successfully able to move messages (`B m`) to other folders.

  Moving to home/school: (1933)...

Any help would be greatly appreciated!

Gnus v5.13
GNU Emacs 26.2 (build 1, x86_64-apple-darwin18.2.0, NS appkit-1671.20 Version 10.14.3 (Build 18D109))
 of 2019-04-12

bug#42131: Cannot split to IMAP folders

[Greene, David]

[-- Attachment #1: Type: text/plain, Size: 167 bytes --]

Any ideas of what to try or additional information I can provide to aid debugging?

I’d really like to use gnus but this is preventing that.

David







[-- Attachment #2: Type: text/html, Size: 3103 bytes --]

bug#42131: Cannot split to IMAP folders

[Eric Abrahamsen]

"Greene, David" <david.greene@hpe.com> writes:

> Any ideas of what to try or additional information I can provide to aid debugging?
>
> I’d really like to use gnus but this is preventing that.

Maybe set `nnimap-log-command' to t, and also `nnmail-split-tracing' --
once you try splitting that should give you two buffers, "*imap-log" and
"*Split-Trace*", that might contain some more useful hints.

"mail.misc" is the default group to put split mail into, when none of
the user's split definitions match.

Eric

bug#42131: Cannot split to IMAP folders

[Stefan Kangas]

tags 42131 + moreinfo
thanks

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> "Greene, David" <david.greene@hpe.com> writes:
>
>> Any ideas of what to try or additional information I can provide to aid debugging?
>>
>> I’d really like to use gnus but this is preventing that.
>
> Maybe set `nnimap-log-command' to t, and also `nnmail-split-tracing' --
> once you try splitting that should give you two buffers, "*imap-log" and
> "*Split-Trace*", that might contain some more useful hints.
>
> "mail.misc" is the default group to put split mail into, when none of
> the user's split definitions match.

That was 6 weeks ago.

Did you find out anything more?

Best regards,
Stefan Kangas

bug#42131: Cannot split to IMAP folders

[Greene, David]

I did this:

  ;; Turn on debugging
  (setq imap-log t)
  (setq nnimap-log-command t)
  (setq nnmail-split-tracing t)

But after a 'B r' I see no *imap-log* or *Split-Trace* buffers.

David

On 8/19/20, 7:23 PM, "Stefan Kangas" <stefan@marxist.se> wrote:

    tags 42131 + moreinfo
    thanks
    
    Eric Abrahamsen <eric@ericabrahamsen.net> writes:
    
    > "Greene, David" <david.greene@hpe.com> writes:
    >
    >> Any ideas of what to try or additional information I can provide to aid debugging?
    >>
    >> I’d really like to use gnus but this is preventing that.
    >
    > Maybe set `nnimap-log-command' to t, and also `nnmail-split-tracing' --
    > once you try splitting that should give you two buffers, "*imap-log" and
    > "*Split-Trace*", that might contain some more useful hints.
    >
    > "mail.misc" is the default group to put split mail into, when none of
    > the user's split definitions match.
    
    That was 6 weeks ago.
    
    Did you find out anything more?
    
    Best regards,
    Stefan Kangas
    

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

"Greene, David" <dag@hpe.com> writes:

> I did this:
>
>   ;; Turn on debugging
>   (setq imap-log t)
>   (setq nnimap-log-command t)
>   (setq nnmail-split-tracing t)

That should be

(setq nnimap-record-commands t)
(setq nnmail-debug-splitting t)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42131: Cannot split to IMAP folders

[Eric Abrahamsen]

On 08/20/20 19:57 PM, Lars Ingebrigtsen wrote:
> "Greene, David" <dag@hpe.com> writes:
>
>> I did this:
>>
>>   ;; Turn on debugging
>>   (setq imap-log t)
>>   (setq nnimap-log-command t)
>>   (setq nnmail-split-tracing t)
>
> That should be
>
> (setq nnimap-record-commands t)
> (setq nnmail-debug-splitting t)

Oops, sorry!

bug#42131: Cannot split to IMAP folders

[David Greene]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> "Greene, David" <dag@hpe.com> writes:
>
>> I did this:
>>
>>   ;; Turn on debugging
>>   (setq imap-log t)
>>   (setq nnimap-log-command t)
>>   (setq nnmail-split-tracing t)
>
> That should be
>
> (setq nnimap-record-commands t)
> (setq nnmail-debug-splitting t)

Thank you!  With this *imapo-log* shows:

10:01:57 [localhost] 529 CAPABILITY
10:01:57 [localhost] 530 ENABLE QRESYNC
10:01:57 [localhost] 531 SELECT "hpe/INBOX"
10:01:58 [localhost] 532 SELECT "hpe/INBOX"
10:01:58 [localhost] 533 UID FETCH 1427,2429,3664,3858,3954,10448,10545,10560,10972:10973,11224,11291,11731,11952,12758,13114,13405,13529,13863,13876,14954,15096,15202,15307,15312,15664,15791,15890,15913,16062,16161,16235,16778,16988,17395,18205,20358,20458,20461:20462,20465,20609,21073,21124:21125,21128,21186,21196,21362,21799,21982,22013,22120,22208,23347,24041,24113:24115,24281,24335,24420,24424:24425,24878,25123,25200,27074,27473:27474,27579,27597,27683,27694,27697,27776,27784,27787:27794 (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups Cc)])
10:02:04 [localhost] 534 SELECT "hpe/INBOX"
10:02:04 [localhost] 535 UID STORE 2429 -FLAGS.SILENT (\Flagged \Answered gnus-expire gnus-dormant gnus-score gnus-save gnus-download gnus-forward)
10:02:04 [localhost] 536 SELECT "hpe/INBOX"
10:02:04 [localhost] 537 UID FETCH 2429 BODY.PEEK[]
10:02:04 [localhost] 538 APPEND "mail.misc" {622647}

mail.misc is suspicious because B q says the message should go to a
different folder, on that exists.  There is no mail.misc IMAP folder so
maybe that's why I'm having trouble.

I still don't see a *Split-Trace* buffer.

David

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

David Greene <dag@hpe.com> writes:

> Thank you!  With this *imapo-log* shows:

[...]

> 10:02:04 [localhost] 536 SELECT "hpe/INBOX"
> 10:02:04 [localhost] 537 UID FETCH 2429 BODY.PEEK[]
> 10:02:04 [localhost] 538 APPEND "mail.misc" {622647}
>
> mail.misc is suspicious because B q says the message should go to a
> different folder, on that exists.  There is no mail.misc IMAP folder so
> maybe that's why I'm having trouble.

Hm.  Odd.

> I still don't see a *Split-Trace* buffer.

Your IMAP log didn't really look much like a typical splitting session
at all -- was this after tapping `g' in the group buffer?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42131: Cannot split to IMAP folders

[David Greene]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> I still don't see a *Split-Trace* buffer.
>
> Your IMAP log didn't really look much like a typical splitting session
> at all -- was this after tapping `g' in the group buffer?

I typed B r with the point on the article in the Summary buffer and
selected nnimap as the split method.

Is there some other way I should test it?

David

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

David Greene <dag@hpe.com> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>>> I still don't see a *Split-Trace* buffer.
>>
>> Your IMAP log didn't really look much like a typical splitting session
>> at all -- was this after tapping `g' in the group buffer?
>
> I typed B r with the point on the article in the Summary buffer and
> selected nnimap as the split method.

Sorry, I forgot that we were talking about respooling, not splitting --
there's no split-trace buffer when doing respooling.

Anyway, this was the bug:

> 10:02:04 [localhost] 538 APPEND "mail.misc" {622647}
>
> mail.misc is suspicious because B q says the message should go to a
> different folder, on that exists.  There is no mail.misc IMAP folder so
> maybe that's why I'm having trouble.

Do you have any idea where mail.misc may be coming from?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42131: Cannot split to IMAP folders

[David Greene]

Lars Ingebrigtsen <larsi@gnus.org> writes:
>> I typed B r with the point on the article in the Summary buffer and
>> selected nnimap as the split method.
>
> Sorry, I forgot that we were talking about respooling, not splitting --
> there's no split-trace buffer when doing respooling.

I didn't know there was a distinction.  Is there a way to manually
"split" a message like with respooling?

> Anyway, this was the bug:
>
>> 10:02:04 [localhost] 538 APPEND "mail.misc" {622647}
>>
>> mail.misc is suspicious because B q says the message should go to a
>> different folder, on that exists.  There is no mail.misc IMAP folder so
>> maybe that's why I'm having trouble.
>
> Do you have any idea where mail.misc may be coming from?

I don't.  The only instance of that string that I found in my config is
in a commented line.

               -David

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

David Greene <dag@hpe.com> writes:

>> Do you have any idea where mail.misc may be coming from?
>
> I don't.  The only instance of that string that I found in my config is
> in a commented line.

I just noticed:

(defvar gnus-group-split-default-catch-all-group "mail.misc"
  "Group name (or arbitrary fancy split) with default splitting rules.
Used by `gnus-group-split' and `gnus-group-split-update' as a fallback
split, in case none of the group-based splits matches.")

Hm, no, it's not used by these commands, 

Then there's:

(defcustom nnmail-split-fancy "mail.misc"
  "Incoming mail can be split according to this fancy variable.

But you had a different value here?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42131: Cannot split to IMAP folders

[Andreas Schwab]

On Sep 08 2020, Lars Ingebrigtsen wrote:

> Then there's:
>
> (defcustom nnmail-split-fancy "mail.misc"
>   "Incoming mail can be split according to this fancy variable.

That is only used by fancy splitting, ie if nnmail-split-methods is set
to 'nnmail-split-fancy.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

bug#42131: Cannot split to IMAP folders

[David Greene]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>>> Do you have any idea where mail.misc may be coming from?
>>
>> I don't.  The only instance of that string that I found in my config is
>> in a commented line.
>
> I just noticed:
>
> (defvar gnus-group-split-default-catch-all-group "mail.misc"
>   "Group name (or arbitrary fancy split) with default splitting rules.
> Used by `gnus-group-split' and `gnus-group-split-update' as a fallback
> split, in case none of the group-based splits matches.")
>
> Hm, no, it's not used by these commands, 

Which commands?

> Then there's:
>
> (defcustom nnmail-split-fancy "mail.misc"
>   "Incoming mail can be split according to this fancy variable.
>
> But you had a different value here?

No, I didn't set nnmail-split-fancy, but I did set nnimap-split-fancy
and also nnimap-split-methods = nnimap-split-fancy.

I set them via gnu-select-method:

  (setq gnus-select-method '(nnimap "Local"
                                    (nnimap-address "localhost")
                                    (nnimap-authenticator login)
                                    (nnimap-stream network)
                                    (nnimap-inbox '("work/INBOX" "home/INBOX" "gmail/INBOX"))
                                    (nnimap-split-fancy
                                     (|
                                      ("Subject:" "Nightly.*test results" "work/testing")
                                      (from "myschool.org" "home/school")
                                      ;; Really leave it where it is?
                                      (nil)
                                      ))
                                    (nnimap-split-methods nnimap-split-fancy)

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

David Greene <dag@hpe.com> writes:

>> Hm, no, it's not used by these commands, 
>
> Which commands?

We were discussing `B q' and `B r', I think?

> No, I didn't set nnmail-split-fancy, but I did set nnimap-split-fancy
> and also nnimap-split-methods = nnimap-split-fancy.
>
> I set them via gnu-select-method:
>
>   (setq gnus-select-method '(nnimap "Local"
>                                     (nnimap-address "localhost")
>                                     (nnimap-authenticator login)
>                                     (nnimap-stream network)
>                                     (nnimap-inbox '("work/INBOX" "home/INBOX" "gmail/INBOX"))
>                                     (nnimap-split-fancy
>                                      (|
>                                       ("Subject:" "Nightly.*test results" "work/testing")
>                                       (from "myschool.org" "home/school")
>                                       ;; Really leave it where it is?
>                                       (nil)
>                                       ))
>                                     (nnimap-split-methods nnimap-split-fancy)

Ah, that's where it's coming from.  `B q' is this:

(defun gnus-summary-respool-query (&optional silent trace)

[...]

		 ;; nnimap has its own splitting variables.
		 (let ((nnmail-split-methods
			(cond
			 ((eq nnimap-split-methods 'default)
			  nnmail-split-methods)
			 (nnimap-split-methods
			  nnimap-split-methods)
			 (nnimap-split-fancy
			  'nnmail-split-fancy)))
		       (nnmail-split-fancy (or nnimap-split-fancy
					       nnmail-split-fancy)))


So I'm guessing your fallback nil means that the machinery is falling
back on the fallback group "mail.misc" for fancy mail splitting...
Hm...  but that should be shadowed here.  But I think we're on the trail
of the bug now.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#42131: Cannot split to IMAP folders

[David Greene]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Ah, that's where it's coming from.  `B q' is this:
>
> (defun gnus-summary-respool-query (&optional silent trace)
>
> [...]
>
> 		 ;; nnimap has its own splitting variables.
> 		 (let ((nnmail-split-methods
> 			(cond
> 			 ((eq nnimap-split-methods 'default)
> 			  nnmail-split-methods)
> 			 (nnimap-split-methods
> 			  nnimap-split-methods)
> 			 (nnimap-split-fancy
> 			  'nnmail-split-fancy)))
> 		       (nnmail-split-fancy (or nnimap-split-fancy
> 					       nnmail-split-fancy)))
>
>
> So I'm guessing your fallback nil means that the machinery is falling
> back on the fallback group "mail.misc" for fancy mail splitting...
> Hm...  but that should be shadowed here.  But I think we're on the trail
> of the bug now.

That still doesn't explain why "B q" reports the correct group but "B r"
goes to mail.misc.  Do they not use the same selection logic?

Also, since I am apparently not "splitting" using those commands, how
would I actually test splitting?

                   -David

bug#42131: Cannot split to IMAP folders

[Lars Ingebrigtsen]

David Greene <dag@hpe.com> writes:

> That still doesn't explain why "B q" reports the correct group but "B r"
> goes to mail.misc.  Do they not use the same selection logic?

They should, so that sounds like the bug.

> Also, since I am apparently not "splitting" using those commands, how
> would I actually test splitting?

Hitting `g' in the group buffer should split the email (unless you've
disabled INBOX splitting).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no