Re: Gnus slow with IMAP when changing ISP's

[Stefan Monnier <monnier@iro.umontreal.ca>]

> I use gnus from work and from home. I have a different ISP in each
> place. When I start gnus from one of those places (say home) I can
> access my IMAP account almost right away, however, if I go to the
> other place (my work) checking for new mail takes forever.

By "forever" do you mean that it is slow, or that it really literally
never terminates (i.e. it doesn't work at all)?

I see a similar problem, but in my case it's not just slow, Gnus just
sits there and never terminates.  It looks like the problem has to with
NAT and/or with going from one private subnet to another and/or changing
IP address.  In any case what I see happen is that the underlying
connection is stuck and doesn't timeout and Gnus's imap backend doesn't
have its own timeout to make up for it.

For what it's worth, I've been using the patch below, so that I just hit
C-g when the connection is stuck, and then retry the command which causes
the connection to be re-created from scratch and everything works
fine again.


        Stefan


--- imap.el.~1.21.~	2009-09-28 09:00:51.000000000 -0400
+++ imap.el	2009-11-19 11:28:18.000000000 -0500
@@ -1313,7 +1313,8 @@
     imap-namespace))
 
 (defun imap-send-command-wait (command &optional buffer)
-  (imap-wait-for-tag (imap-send-command command buffer) buffer))
+  (with-local-quit            ; See backtrace in imap-accept-process-output.
+    (imap-wait-for-tag (imap-send-command command buffer) buffer)))
 
 (defun imap-logout (&optional buffer)
   (or buffer (setq buffer (current-buffer)))
@@ -2029,6 +2030,61 @@
 	  (imap-send-command-1 cmdstr))
       tag)))
 
+(defun imap-accept-process-output (proc &rest args)
+  ;; FIXME: This is sometimes called with quit inhibited:
+  ;; Tried to fix it with a `with-local-quit' in imap-send-command-wait.
+  ;; Debugger entered--Lisp error: (error "Blocking call to accept-process-output with quit inhibited!!")
+  ;;   accept-process-output(#<process imap> 0 100 nil)
+  ;;   apply(accept-process-output #<process imap> (0 100))
+  ;;   byte-code("\302\303\b	#\207" [proc args apply accept-process-output] 4)
+  ;;   imap-accept-process-output(#<process imap> 0 100)
+  ;;   imap-wait-for-tag(1937 nil)
+  ;;   imap-send-command-wait("SELECT \"inbox\"")
+  ;;   imap-mailbox-select-1("inbox" nil)
+  ;;   imap-mailbox-select("inbox")
+  ;;   nnimap-possibly-change-group("inbox" "diro")
+  ;;   nnimap-request-article-part(36260 "BODY.PEEK[]" BODYDETAIL "inbox" "diro" nil detail)
+  ;;   nnimap-request-article(36260 "inbox" "diro" nil)
+  ;;   gnus-request-article(36260 "nnimap+diro:inbox")
+  ;;   gnus-async-prefetch-article("nnimap+diro:inbox" t #<buffer *Summary nnimap+diro:inbox*> t)
+  ;;   gnus-async-article-callback(t "nnimap+diro:inbox" 36261 #<marker at 12447484 in  *Async Prefetch Article*> #<buffer *Summary nnimap+diro:inbox*> t)
+  ;;   (lambda (arg) (gnus-async-article-callback arg "nnimap+diro:inbox" 36261 #<marker at 12447484 in  *Async Prefetch Article*> #<buffer *Summary nnimap+diro:inbox*> t))(t)
+  ;;   nnimap-callback(36261 (lambda (arg) (gnus-async-article-callback arg "nnimap+diro:inbox" 36261 #<marker at 12447484 in  *Async Prefetch Article*> #<buffer *Summary nnimap+diro:inbox*> t)) #<buffer  *Async Prefetch Article*>)
+  ;;   (lambda nil (nnimap-callback 36261 (lambda ... ...) #<buffer  *Async Prefetch Article*>))()
+  ;;   run-hooks(imap-fetch-data-hook)
+  ;;   imap-parse-fetch(6731)
+  ;;   imap-parse-response()
+  ;;   imap-arrival-filter(#<process imap> "evaluated the resulting lisp. The lisp looks like this:\r\n\r\n(fset 'findwordfile\r\n   (lambda (&optional arg) \"Keyboard macro.\" (interactive \"p\")\r\n(kmacro-exec-ring-item (quote (\"=13file\" 0 \"%d\")) arg)))\r\n\r\nWhen trying to use the macro, I get this in my minibuffer:\r\n\r\nfindwordfile: Symbol's function definition is void: kmacro-exec-ring-item\r\n\r\nIf I put (require 'kmacro) at the beginning of my file containing\r\nmacros it works.\r\n\r\nIs there a missing autoload before this line?\r\n(defun kmacro-exec-ring-item (item arg)\r\n\r\nThanks for your work.\r\n\r\n/Mats\r\n\r\nIf Emacs crashed, and you have the Emacs process in the gdb debugger,\r\nplease include the output from the following gdb commands:\r\n    `bt full' and `xbacktrace'.\r\nIf you would like to further debug the crash, please read the file\r\nc:/ProgramFiles/emacs-22.2/etc/DEBUG for instructions.\r\n\r\n\r\nIn GNU Emacs 22.2.1 (i386-mingw-nt5.0.2195)\r\n of 2008-03-26 on RELEASE\r\nWindowing system distributor `Microsoft Corp.', version 5.0.2195\r\nconfigured using `configure --with-gcc (3.4)'\r\n\r\nImportant settings:\r\n  value of $LC_ALL: nil\r\n  value of $LC_COLLATE: nil\r\n  value of $LC_CTYPE: nil\r\n  value of $LC_MESSAGES: nil\r\n  value of $LC_MONETARY: nil\r\n  value of $LC_NUMERIC: nil\r\n  value of $LC_TIME: nil\r\n  value of $LANG: SVE\r\n  locale-coding-system: cp1252\r\n  default-enable-multibyte-characters: t\r\n\r\nMajor mode: Emacs-Lisp\r\n\r\nMinor modes in effect:\r\n  encoded-kbd-mode: t\r\n  tooltip-mode: t\r\n  tool-bar-mode: t\r\n  mouse-wheel-mode: t\r\n  menu-bar-mode: t\r\n  file-name-shadow-mode: t\r\n  global-font-lock-mode: t\r\n  font-lock-mode: t\r\n  blink-cursor-mode: t\r\n  unify-8859-on-encoding-mode: t\r\n  utf-translate-cjk-mode: t\r\n  auto-compression-mode: t\r\n  line-number-mode: t\r\n\r\nRecent input:\r\nC-x C-f C-a C-k c : / t e m p / f w f . e l <return>\r\nM-> C-x C-e M-< M-x f i n d w <tab> <return> M-x e m\r\n<backspace> <backspace> r e p o r t SPC e m a <tab>\r\n<return>\r\n\r\nRecent messages:\r\nLoading encoded-kb...done\r\nFor information about GNU Emacs and the GNU system, type C-h C-a.\r\nfwf.el has auto save data; consider M-x recover-this-file\r\nMark set\r\n(lambda (&optional arg) \"Keyboard macro.\" (interactive \"p\")\r\n(kmacro-exec-ring-item (quote (\"=13file\" 0 \"%d\")) arg))\r\nMark set\r\nfindwordfile: Symbol's function definition is void: kmacro-exec-ring-item\r\nLoading emacsbug...\r\nLoading regexp-opt...done\r\nLoading emacsbug...done\r\n\r\n\r\n\r\n\r\n------------=_1217977809-519-0\r\nContent-Type: message/rfc822\r\nContent-Disposition: inline\r\nContent-Transfer-Encoding: 7bit\r\n\r\nX-Spam-Checker-Version: SpamAssassin 3.2.3-bugs.debian.org_2005_01_02\r\n	(2007-08-08) on rzlab.ucr.edu\r\nX-Spam-Level: \r\nX-Spam-Status: No, score=-3.2 required=4.0 tests=AWL,BAYES_00 autolearn=ham\r\n	version=3.2.3-bugs.debian.org_2005_01_02\r\nReceived: (at 331-done) by emacsbugs.donarmstrong.com; 5 Aug 2008 23:00:06 +0000\r\nReceived: from cyd (CYD.MIT.EDU [18.115.2.24])\r\n	by rzlab.ucr.edu (8.13.8/8.13.8/Debian-3) with ESMTP id m75N02Ac029108\r\n	for <331-done@emacsbugs.donarmstrong.com>; Tue, 5 Aug 2008 16:00:03 -0700\r\nReceived: by cyd (Postfix, from userid 1000)\r\n	id 027C957E338; Tue,  5 Aug 2008 19:00:30 -0400 (EDT)\r\nFrom: Chong Yidong <cyd@stupidchicken.com>\r\nTo: \"Matzi Kratzi\" <matzikratzi@gmail.com>\r\nCc: 331-done@emacsbugs.donarmstrong.com\r\nSubject: Re: Bug in kmacro.el?\r\nDate: Tue, 05 Aug 2008 19:00:29 -0400\r\nMessage-ID: <87hc9zxe2a.fsf@stupidchicken.com>\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=us-ascii\r\n\r\nThanks.  I've checked in a fix (autoloading kmacro-exec-ring-item, as\r\nyou suggested).\r\n\r\n\r\n------------=_1217977809-519-0--\r\n\r\n)\r\n1935 OK UID FETCH completed\r\n")
+  ;;   accept-process-output(#<process imap> 0 100 nil)
+  ;;   apply(accept-process-output #<process imap> (0 100))
+  ;;   byte-code("\302\303\b	#\207" [proc args apply accept-process-output] 4)
+  ;;   imap-accept-process-output(#<process imap> 0 100)
+  ;;   imap-wait-for-tag(1936 nil)
+  ;;   imap-send-command-wait("EXAMINE \"diro\"")
+  ;;   imap-mailbox-select-1("diro" examine)
+  ;;   imap-mailbox-examine-1("diro")
+  ;;   imap-message-copyuid-1("diro")
+  ;;   imap-message-copy("36250" "diro" dontcreate nil " *nnimap* diro")
+  ;;   nnimap-request-accept-article("diro" "diro" t)
+  ;;   gnus-request-accept-article("nnimap+diro:diro" nil t t)
+  ;;   eval((gnus-request-accept-article "nnimap+diro:diro" (quote nil) t t))
+  ;;   nnimap-request-move-article(36250 "inbox" "diro" (gnus-request-accept-article "nnimap+diro:diro" (quote nil) t t) t t)
+  ;;   gnus-request-move-article(36250 "nnimap+diro:inbox" "diro" (gnus-request-accept-article "nnimap+diro:diro" (quote nil) t t) t t)
+  ;;   gnus-summary-move-article(nil)
+  ;;   call-interactively(gnus-summary-move-article nil nil)
+  (condition-case nil
+      (apply 'accept-process-output proc args)
+    (quit
+     ;; The IMAP connection can get stuck (not sure why: it may have
+     ;; to do with NAT routers, changing private network, or changing
+     ;; your IP address).
+     ;; Something somewhere should catch this and timeout
+     ;; after a little while, but it seems like currently it doesn't
+     ;; happen, so instead let the user hit C-g to cause the
+     ;; connection to be dropped, so it can be re-opened from scratch.
+     (delete-process proc)
+     (signal 'quit nil))))
+
 (defun imap-wait-for-tag (tag &optional buffer)
   (with-current-buffer (or buffer (current-buffer))
     (let (imap-have-messaged)
@@ -2040,7 +2096,8 @@
 	  (unless (< len 10)
 	    (setq imap-have-messaged t)
 	    (message "imap read: %dk" len))
-	  (accept-process-output imap-process
+	  (imap-accept-process-output
+           imap-process
 				 (truncate imap-read-timeout)
 				 (truncate (* (- imap-read-timeout
 						 (truncate imap-read-timeout))