bug#9831: 24.0.90; o and c-o in RMAIL change buffer

bug#9831: 24.0.90; o and c-o in RMAIL change buffer

[john ffitch]

When reading mail o writes the message to another file, or buffer if
it is loaded.  It also changes to that buffer and this seriously
interferes with work flow, as it is inconsistent with when the file is
not in a buffer. This seems fairly recent behaviour and is causing
significant problems.  Could it stay in the originating buffer, or
have an option so to do?



In GNU Emacs 24.0.90.7 (x86_64-suse-linux-gnu, GTK+ Version 2.22.1)
 of 2011-10-20 on harvey
Windowing system distributor `The X.Org Foundation', version 11.0.10903000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=local
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Emacs-Lisp

Minor modes in effect:
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  eldoc-mode: t
  auto-image-file-mode: t
  show-paren-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<return> / j u <backspace> o i n SPC c s o u n d <return> 
<help-echo> <switch-frame> <help-echo> C-x C-f <backspace> 
<return> <next> <next> <next> <next> <next> <next> 
<down> <down> d <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> C-x C-f G 
N U _ 2 1 / e m <tab> t e <backspace> <backspace> / 
t <tab> <return> <switch-frame> <switch-frame> C-h 
k o C-x b C-g C-x C-f ~ / R M A I L <return> y <help-echo> 
<help-echo> <help-echo> <down-mouse-1> <mouse-movement> 
<mouse-1> q C-h l o C-g C-f C-g C-g C-g C-h k C-o <help-echo> 
C-h k o C-h k C-o <help-echo> <help-echo> <help-echo> 
<down-mouse-1> <mouse-2> <help-echo> <down-mouse-1> 
<mouse-2> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> M-x e m a c s - b <tab> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> r e p o <tab> r <tab> <ret
urn>

Recent messages:
Invalid face reference: my-trailing-space-face
Invalid face reference: my-tab-face [7 times]
Invalid face reference: my-trailing-space-face
Invalid face reference: my-tab-face [7 times]
Invalid face reference: my-trailing-space-face
Invalid face reference: my-tab-face [7 times]
Invalid face reference: my-trailing-space-face
Invalid face reference: my-tab-face [7 times]
Invalid face reference: my-trailing-space-face
Invalid face reference: my-tab-face [7 times]

Load-path shadows:
None found.

Features:
(shadow mailalias emacsbug vc-bzr find-func rmailout sendmail
mime-compose mail-alias-menu mailcrypt mail-extr rmailkwd rmailmm
message rfc822 mml mml-sec mm-decode mm-bodies mm-encode mailabbrev
gmm-utils mailheader mail-parse rfc2231 rmail rfc2047 rfc2045
ietf-drums mail-utils network-stream auth-source eieio byte-opt
bytecomp byte-compile cconv macroexp assoc gnus-util mm-util
mail-prsvr password-cache starttls tls erc-menu erc-join erc-ring
erc-networks erc-pcomplete pcomplete comint ring erc-track erc-match
erc-button wid-edit erc-fill erc-stamp erc-netsplit erc-goodies erc
erc-backend erc-compat format-spec thingatpt pp dired help-mode eldoc
cal-julian image-file crypt crypt++ crypt+pgp-pub paren uniquify
advice help-fns advice-preload view cal-china cal-bahai cal-islam
cal-hebrew lunar solar cal-dst appt diary-lib diary-loaddefs holidays
hol-loaddefs regexp-opt cal-menu easymenu calendar cal-loaddefs time
time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win
x-dnd tool-bar dnd fontset image fringe lisp-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
loaddefs button faces cus-face files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process
dynamic-setting system-font-setting font-render-setting move-toolbar
gtk x-toolkit x multi-tty emacs)

==John ffitch

bug#9831: narrowing the bug down

[Mark Lillibridge]

I have also run into this bug on Emacs 23.3.  It is quite annoying.

So far I have determined that the bug is caused by the following lines:

rmailout.el:386:
    ;; FIXME should re-use existing windows.
    (if (rmail-summary-exists)
	(rmail-select-summary (rmail-update-summary)))

Commenting out those two lines makes the bug go away.

- Mark

bug#9831: narrowing the bug down

[Mark Lillibridge]

Some further tracking reveals:

    The problem is in rmail-update-summary; rmail-update-summary calls
(indirectly) rmail-summary in the normal unfiltered summary case:

(defun rmail-update-summary (&rest ignore)
  (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))



(defun rmail-summary ()
  "Display a summary of all messages, one line per message."
  (interactive)
  (rmail-new-summary "All" '(rmail-summary) nil)
  (unless (get-buffer-window rmail-buffer)
    (rmail-summary-beginning-of-message)))


Commenting out the last two lines of the above makes the bug go away (at
least for unfiltered summaries):

(defun rmail-summary ()
  "Display a summary of all messages, one line per message."
  (interactive)
  (rmail-new-summary "All" '(rmail-summary) nil)
)
;  (unless (get-buffer-window rmail-buffer)
;    (rmail-summary-beginning-of-message)))


A bit further tracking shows that the last call is the culprit:

(defun rmail-summary-beginning-of-message ()
  "Show current message from the beginning."
  (interactive)
  (rmail-summary-show-message 'BEG))

(defun rmail-summary-show-message (where)
  "Show current mail message.
Position it according to WHERE which can be BEG or END"
  (if (and (one-window-p) (not pop-up-frames))
      ;; If there is just one window, put the summary on the top.
      (let ((buffer rmail-buffer))
	(split-window (selected-window) rmail-summary-window-size)
	(select-window (frame-first-window))
	(rmail-pop-to-buffer rmail-buffer)
	;; If pop-to-buffer did not use that window, delete that
	;; window.  (This can happen if it uses another frame.)
	(or (eq buffer (window-buffer (next-window (frame-first-window))))
	    (delete-other-windows)))
    (rmail-pop-to-buffer rmail-buffer))
  (cond
   ((eq where 'BEG)
	(goto-char (point-min))
	(search-forward "\n\n"))
   ((eq where 'END)
	(goto-char (point-max))
	(recenter (1- (window-height))))
   )
  (rmail-pop-to-buffer rmail-summary-buffer))


    I'm not sure how to go about fixing this; extra dynamically-bound
variable that stops rmail-update-summary called functions from
displaying the summary in a window?

- Mark

bug#9831: cause of bug found! [PATCH]

[Mark Lillibridge]

    Because this bug doesn't occur in Emacs 22, I compared to that code's
version of rmail-summary:


[Rmail 22]
(defun rmail-summary ()
  "Display a summary of all messages, one line per message."
  (interactive)
  (rmail-new-summary "All" '(rmail-summary) nil))

[Rmail 23.3]
(defun rmail-summary ()
  "Display a summary of all messages, one line per message."
  (interactive)
  (rmail-new-summary "All" '(rmail-summary) nil)
  (unless (get-buffer-window rmail-buffer)
    (rmail-summary-beginning-of-message)))


    As you can see, some well-meaning person added the functionality of
move-to-start-of-message to the display summary command ('h') and broke
rmail-output and associated functions.  I checked and none of the other
summary generating functions (e.g., rmail-summary-by-labels) have this
functionality (added).


    On the assumption that somebody wanted this functionality, here is
one way to patch things:

ts-rhel5 [159]% ( setenv LC_ALL C ; setenv TZ UTC0 ; diff -Naur original-rmailsum.el rmailsum.el ) 
--- original-rmailsum.el        2011-10-22 21:03:14.834090000 +0000
+++ rmailsum.el 2011-10-22 21:22:00.443672000 +0000
@@ -71,16 +71,22 @@
 
 ;; Regenerate the contents of the summary
 ;; using the same selection criterion as last time.
+;; If current buffer is the summary buffer (rather than the Rmail
+;; buffer), then does not make summary displayed if not already
+;; displayed.
 ;; M-x revert-buffer in a summary buffer calls this function.
 (defun rmail-update-summary (&rest ignore)
   (apply (car rmail-summary-redo) (cdr rmail-summary-redo)))
 
 ;;;###autoload
-(defun rmail-summary ()
-  "Display a summary of all messages, one line per message."
+(defun rmail-summary (&rest no-display)
+  "Display a summary of all messages, one line per message.
+
+If no-display is set, updates/creates the summary buffer, but does not
+necessarily display it."
   (interactive)
-  (rmail-new-summary "All" '(rmail-summary) nil)
-  (unless (get-buffer-window rmail-buffer)
+  (rmail-new-summary "All" '(rmail-summary t) nil)
+  (unless (or no-display (get-buffer-window rmail-buffer))
     (rmail-summary-beginning-of-message)))
 
 ;;;###autoload


I believe it is now safe to remove the FIXME from rmailout.el:

    ;; FIXME should re-use existing windows.
    (if (rmail-summary-exists)
	(rmail-select-summary (rmail-update-summary)))

- Mark

bug#9831: cause of bug found! [PATCH]

[martin rudalics]

 >     Because this bug doesn't occur in Emacs 22, I compared to that code's
 > version of rmail-summary:

I'm not convinced that the issue you see is related to that reported by
the OP.  But since I'm not familiar with rmail could you please explain
to me what happens and what should happen below.

 > [Rmail 22]
 > (defun rmail-summary ()
 >   "Display a summary of all messages, one line per message."
 >   (interactive)
 >   (rmail-new-summary "All" '(rmail-summary) nil))
 >
 > [Rmail 23.3]
 > (defun rmail-summary ()
 >   "Display a summary of all messages, one line per message."
 >   (interactive)
 >   (rmail-new-summary "All" '(rmail-summary) nil)
 >   (unless (get-buffer-window rmail-buffer)
 >     (rmail-summary-beginning-of-message)))
 >
 >
 >     As you can see, some well-meaning person added the functionality of
 > move-to-start-of-message to the display summary command ('h') and broke
 > rmail-output and associated functions.  I checked and none of the other
 > summary generating functions (e.g., rmail-summary-by-labels) have this
 > functionality (added).

I seem to understand that you show in some window a buffer called
rmail-buffer, presumably containing some messages you read.  Now you
want to produce a summary in a buffer called rmail-summary-buffer and do
this by invoking the command `rmail-summary'.  That command winds up by
calling `rmail-summary-show-message' which does `rmail-pop-to-buffer' on
rmail-buffer (I don't understand why it does do that).  Anyway, since
that buffer already appears on some window, `rmail-summary-show-message'
should in principle reuse that window and IIUC not change that window's
point, i.e., not change what you see in that window.

But if my summary is correct then

   (rmail-new-summary "All" '(rmail-summary) nil)

should always make sure that rmail-buffer appears in some window and the
test coming next

   (unless (get-buffer-window rmail-buffer)
     (rmail-summary-beginning-of-message)))

should always fail (unless rmail-buffer is shown on another frame) so no
such deliberate movement should occur.

However, my summary apparently fails to tell what you see, so could you
please tell me what happens instead and why?

And, as mentioned above, I don't understand how what you describe here
corresponds to the bug reported by John: His seems a problem with the
command invoked by typing `o' yours when typing `h'.

martin

bug#9831: cause of bug found! [PATCH]

[Mark Lillibridge]

>   >     Because this bug doesn't occur in Emacs 22, I compared to that code's
>   > version of rmail-summary:
>  
>  I'm not convinced that the issue you see is related to that reported by
>  the OP.  But since I'm not familiar with rmail could you please explain
>  to me what happens and what should happen below.

    Sorry, more background.  The bug OP and I am reporting is as
follows: we have two Rmail buffers, say A and B, each with summary
buffers.  However, only A and it's summary are displayed in windows.  We
then output the current message from A to B via 'o'.  The bug is that at
this point the summary for B becomes displayed when it should not.

    Why?  The filing code updates the summary for the buffer the
messages being filed to (e.g., B) so that it shows the message just
added to that buffer if appropriate.  This should not cause that summary
to be displayed but it does due to the bug.

    Why?  The summary is updated via (rmail-update-summary).
Historically, this does not cause the updated buffer to be displayed,
but because of the bug if this summary was produced by rmail-summary, it
will be displayed.

    Why? rmail-update-summary makes a saved function call (depending on
the filtering requested, a different call is necessary to rebuild the
summary) to update the summary.  If the summary was originally created via
rmail-summary, then the saved call is (rmail-summary), which because of
the bug displays the summary.

    Why?  Because someone incorrectly added code to display the summary
buffer on summary update to rmail-summary.

    I changed the code so that rmail-summary when called by the user
(e.g., via 'h') does always display the summary but does not do so when
called via rmail-update-summary.

    Is this more clear?  I think the part you were unclear about is that
there are two Rmail buffers involved, each with their own summary.

- Mark

bug#9831: cause of bug found! [PATCH]

[martin rudalics]

 >     Sorry, more background.  The bug OP and I am reporting is as
 > follows: we have two Rmail buffers, say A and B, each with summary
 > buffers.  However, only A and it's summary are displayed in windows.  We
 > then output the current message from A to B via 'o'.  The bug is that at
 > this point the summary for B becomes displayed when it should not.

I'm probably too silly to understand.  John was talking about "o" not
doing the right thing, but IIUC "o" calls `rmail-output' and not
`rmail-summary-output' in his case.  At least that's what I deduct from
his "When reading mail o writes the message to another file, or buffer
if it is loaded" and the doc-string of `rmail-output' saying "Append
this message to mail file FILE-NAME".  Then John says that "It also
changes to that buffer and this seriously interferes with work flow, as
it is inconsistent with when the file is not in a buffer" but
unfortunately I don't understand what "changes to that buffer" means in
this context.

Moreover, John was saying that "This seems fairly recent behaviour and
is causing significant problems" but I don't find any recent reference
to a change of `rmail-summary' in the Logs.  Finally, John nowhere
talked about point moving to some inconvenient position.  John could you
please clarify these issues?

 >     Why?  The filing code updates the summary for the buffer the
 > messages being filed to (e.g., B) so that it shows the message just
 > added to that buffer if appropriate.  This should not cause that summary
 > to be displayed but it does due to the bug.
 >
 >     Why?  The summary is updated via (rmail-update-summary).
 > Historically, this does not cause the updated buffer to be displayed,

Can you tell me when and where this was changed?

 > but because of the bug if this summary was produced by rmail-summary, it
 > will be displayed.
 >
 >     Why? rmail-update-summary makes a saved function call (depending on
 > the filtering requested, a different call is necessary to rebuild the
 > summary) to update the summary.  If the summary was originally created via
 > rmail-summary, then the saved call is (rmail-summary), which because of
 > the bug displays the summary.
 >
 >     Why?  Because someone incorrectly added code to display the summary
 > buffer on summary update to rmail-summary.

According to our Logs `rmail-update-summary' hasn't been changed for
many years.

 >     I changed the code so that rmail-summary when called by the user
 > (e.g., via 'h') does always display the summary but does not do so when
 > called via rmail-update-summary.
 >
 >     Is this more clear?  I think the part you were unclear about is that
 > there are two Rmail buffers involved, each with their own summary.

I still suppose your's is a different bug.  But I suspect that any of
these bugs may have its cause in a recent change of the buffer display
routines.  Unfortunately, I'm not of much help here since I don't use
rmail.

martin

bug#9831: cause of bug found! [PATCH]

[Mark Lillibridge]

>   >     Sorry, more background.  The bug OP and I am reporting is as
>   > follows: we have two Rmail buffers, say A and B, each with summary
>   > buffers.  However, only A and it's summary are displayed in windows.  We
>   > then output the current message from A to B via 'o'.  The bug is that at
>   > this point the summary for B becomes displayed when it should not.
>  
>  I'm probably too silly to understand.  John was talking about "o" not
>  doing the right thing, but IIUC "o" calls `rmail-output' and not
>  `rmail-summary-output' in his case.  At least that's what I deduct from
>  his "When reading mail o writes the message to another file, or buffer
>  if it is loaded" and the doc-string of `rmail-output' saying "Append
>  this message to mail file FILE-NAME".  Then John says that "It also
>  changes to that buffer and this seriously interferes with work flow, as
>  it is inconsistent with when the file is not in a buffer" but
>  unfortunately I don't understand what "changes to that buffer" means in
>  this context.

    Yes, 'o' calls rmail-output from an Rmail buffer and
rmail-summary-output from the associated summary buffer.  Both suffer
from the bug we are talking about.

    What John means by "changes to that buffer" is that his window
showing rmail-buffer A changes to a *different* rmail-buffer, namely the
one he was outputting the message to.  Note that this buffer change does
not occur when the targeted rmail file is not held in a buffer, hence
John's comments about inconsistency.



>   > but because of the bug if this summary was produced by rmail-summary, it
>   > will be displayed.
>   >
>   >     Why? rmail-update-summary makes a saved function call (depending on
>   > the filtering requested, a different call is necessary to rebuild the
>   > summary) to update the summary.  If the summary was originally created via
>   > rmail-summary, then the saved call is (rmail-summary), which because of
>   > the bug displays the summary.
>   >
>   >     Why?  Because someone incorrectly added code to display the summary
>   > buffer on summary update to rmail-summary.
>  
>  According to our Logs `rmail-update-summary' hasn't been changed for
>  many years.

    I never said that function got changed; remember that it is an
indirection function.  One of the functions it can call, namely
rmail-summary, has been changed since Rmail 22.  I don't have convenient
access to the source control system so I can't tell you when that change
was made.


>  I still suppose your's is a different bug.  But I suspect that any of
>  these bugs may have its cause in a recent change of the buffer display
>  routines.  Unfortunately, I'm not of much help here since I don't use
>  rmail.

    Let's ask John if my patch makes his bug go away.  It certainly
makes mine go way.

- Mark

bug#9831: Your bug report re: o and c-o in RMAIL change buffer

[Mark Lillibridge]

Hi John.

You reported an Rmail bug October 22:

        When reading mail o writes the message to another file, or
    buffer if it is loaded.  It also changes to that buffer and this
    seriously interferes with work flow, as it is inconsistent with when
    the file is not in a buffer. This seems fairly recent behaviour and
    is causing significant problems.  Could it stay in the originating
    buffer, or have an option so to do?

We think we have a patch for your bug.  Could you please try the
following:

  * first, figure out again how to reproduce the problem you are seeing


  * evaluate the following elisp code:

(defun rmail-summary (&rest no-display)
  "Display a summary of all messages, one line per message.

If no-display is set, updates/creates the summary buffer, but does not
necessarily display it."
  (interactive)
  (rmail-new-summary "All" '(rmail-summary t) nil)
  (unless (or no-display (get-buffer-window rmail-buffer))
    (rmail-summary-beginning-of-message)))

  [just put mark and point around the above text then do M-x eval-region]


  * is the problem gone now?

    If that didn't work, can you please provide a more detailed
description of the problem you are seeing, including how to reproduce
it?

- Thanks,
  Mark

bug#9831: cause of bug found! [PATCH]

[martin rudalics]

>     Yes, 'o' calls rmail-output from an Rmail buffer and
> rmail-summary-output from the associated summary buffer.  Both suffer
> from the bug we are talking about.

Aha.

>     What John means by "changes to that buffer" is that his window
> showing rmail-buffer A changes to a *different* rmail-buffer, namely the
> one he was outputting the message to.  Note that this buffer change does
> not occur when the targeted rmail file is not held in a buffer, hence
> John's comments about inconsistency.

OK.  I believe you now.

>     Let's ask John if my patch makes his bug go away.  It certainly
> makes mine go way.

Good idea.

Thanks, martin

bug#9831: o and c-o in RMAIL change buffer

[Glenn Morris]

close 9831 24.0.92
stop

The OP: a) hasn't replied; b) did not mention summaries at all; and c)
said this was "fairly recent" behaviour; whereas the issue identified
occurs since 23.1, but only when summaries are present.

Nevertheless, I think you probably identified the problem, and I
committed a fix.

Summary:

emacs -Q
C-u M-x rmail RET ~/xmail RET   ; non-empty folder
h
C-x 1 
C-u M-x rmail RET ~/RMAIL RET  ; non-empty folder
o RET    ; output message from RMAIL to xmail

At this point, xmail buffers appear.