bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

If I have two invocations of emacs and in one I have modified but not
saved my RMAIL file, and then I try to read my mail in the other,
rmail-insert-inbox-text gives the error "RMAIL file ... is locked".  I
think it should call ask-user-about-lock instead, so I have the
opportunity to steal the lock and so forth.

This happens to me because I run emacs on the same server machine from
home and work, so if I go home without saving my mail file, I will get
this problem when I tried to read my mail from home.  But it seems to me
that it has only started happening recently, even though the code in
rmail-insert-inbox-text has been there a long time.  Previously I always
got the usual query: "... locked by kdo... (pid ....): (s, q, p, ?)?".
Perhaps it was formerly trying to modify file earlier in the process,
before the explicit check at the start of rmail-insert-inbox-text, and
so generating the usual query.

                                        Ken

In GNU Emacs 24.4.50.2 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
 of 2014-05-31 on cosmos
Repository revision: 115761 kdo@cosmos.phy.tufts.edu-20140521165819-mg0m9yc98nkdigwg
Windowing system distributor `The Cygwin/X Project', version 11.0.10903000
System Description:	Ubuntu 12.04.4 LTS

Configured features:
XPM JPEG TIFF GIF PNG SOUND NOTIFY LIBXML2 FREETYPE XFT ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
M-x r e p o r t - b u g <return>

Recent messages:
Loading /home/kdo/emacs-init.el (source)...done
Loading /home/kdo/emacs-init-mail.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug sendmail rmailmm message dired
format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode mailabbrev
gmm-utils mailheader mail-parse rfc2231 rmail rfc2047 rfc2045 ietf-drums
mm-util help-fns mail-prsvr mail-utils shell pcomplete diff-mode
easy-mmode thingatpt inf-lisp easymenu comint ansi-color ring warnings
cl-macs cl gv cl-loaddefs cl-lib time-date tooltip electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
inotify dynamic-setting font-render-setting x-toolkit x multi-tty emacs)

Memory information:
((conses 16 91223 6890)
 (symbols 48 19727 0)
 (miscs 40 86 95)
 (strings 32 16889 5090)
 (string-bytes 1 467813)
 (vectors 16 11696)
 (vector-slots 8 396271 4956)
 (floats 8 76 53)
 (intervals 56 169 0)
 (buffers 960 11)
 (heap 1024 37890 602))

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Ken Olum <kdo@cosmos.phy.tufts.edu>
> Date: Thu, 05 Jun 2014 16:10:25 -0400
> 
> If I have two invocations of emacs and in one I have modified but not
> saved my RMAIL file, and then I try to read my mail in the other,
> rmail-insert-inbox-text gives the error "RMAIL file ... is locked".  I
> think it should call ask-user-about-lock instead, so I have the
> opportunity to steal the lock and so forth.
> 
> This happens to me because I run emacs on the same server machine from
> home and work, so if I go home without saving my mail file, I will get
> this problem when I tried to read my mail from home.  But it seems to me
> that it has only started happening recently, even though the code in
> rmail-insert-inbox-text has been there a long time.  Previously I always
> got the usual query: "... locked by kdo... (pid ....): (s, q, p, ?)?".
> Perhaps it was formerly trying to modify file earlier in the process,
> before the explicit check at the start of rmail-insert-inbox-text, and
> so generating the usual query.

Can you please give a complete recipe starting with "emacs -Q" in both
sessions?  I tried something I thought you were doing, but couldn't
reproduce it.  I got the "locked by NNNN (pid nnn): (s, q, p, ?)"
message, as you expected.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

To reproduce:

Start with some messages in your RMAIL file, but no new mail waiting to
be copied from your inbox.

emacs -Q
M-x rmail
e
Type space, backspace, so that buffer is modified.

Then, somewhere else:

emacs -Q
M-x rmail
Get error "RMAIL file RMAIL is locked"

                                        Ken

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Ken Olum <kdo@cosmos.phy.tufts.edu>
> Cc: 17706@debbugs.gnu.org
> Date: Fri, 06 Jun 2014 11:09:50 -0400
> 
> To reproduce:
> 
> Start with some messages in your RMAIL file, but no new mail waiting to
> be copied from your inbox.
> 
> emacs -Q
> M-x rmail
> e
> Type space, backspace, so that buffer is modified.
> 
> Then, somewhere else:
> 
> emacs -Q
> M-x rmail
> Get error "RMAIL file RMAIL is locked"

Thanks, but I still cannot reproduce.

Does "M-x rmail" on your system fetch mail from some inbox?  On my
system, it doesn't, so I replaced "M-x rmail" in your recipe with
"C-u M-x rmail SOME-MBOX-FILE RET" in both sessions, and left the rest
of the recipe unaltered.  With that, I don't see the message you cite.

I will try on another system, maybe this is something local.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> Date: Fri, 06 Jun 2014 18:39:22 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 17706@debbugs.gnu.org
> 
> Does "M-x rmail" on your system fetch mail from some inbox?

Answering myself: YES.

This message comes from rmail-insert-inbox-text.  But I don't think it
was changed recently, so I wonder in what Emacs version you don't see
it.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

   From: Eli Zaretskii <eliz@gnu.org>
   Date: Fri, 06 Jun 2014 18:50:23 +0300

   I wonder in what Emacs version you don't see it.

I don't see it in 23.3.  The reason is that this version has a bug in
which it deletes newlines from the end of the mail file before getting
new mail.  So it modifies the file before calling
rmail-insert-inbox-text and so gives the ask-user-about-lock query
before calling rmail-insert-inbox-text.

My recollection is that this didn't happen either in builds that I made
from trunk a few months ago, but I could easily be confused.  In
particular, I have a bunch of rmail hacks, and if one of these were
modifying the buffer before rmail-insert-inbox-text it would be the same
as above.  In 24.3 (using -Q to make sure nothing strange is going on),
I get the "RMAIL file RMAIL is locked".  So I think we should not worry
too much about why it seems only to be bothering me now.

                                        Ken

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Ken Olum <kdo@cosmos.phy.tufts.edu>
> Cc: 17706@debbugs.gnu.org
> Date: Fri, 06 Jun 2014 12:29:33 -0400
> 
>    From: Eli Zaretskii <eliz@gnu.org>
>    Date: Fri, 06 Jun 2014 18:50:23 +0300
> 
>    I wonder in what Emacs version you don't see it.
> 
> I don't see it in 23.3.  The reason is that this version has a bug in
> which it deletes newlines from the end of the mail file before getting
> new mail.  So it modifies the file before calling
> rmail-insert-inbox-text and so gives the ask-user-about-lock query
> before calling rmail-insert-inbox-text.
> 
> My recollection is that this didn't happen either in builds that I made
> from trunk a few months ago, but I could easily be confused.  In
> particular, I have a bunch of rmail hacks, and if one of these were
> modifying the buffer before rmail-insert-inbox-text it would be the same
> as above.  In 24.3 (using -Q to make sure nothing strange is going on),
> I get the "RMAIL file RMAIL is locked".  So I think we should not worry
> too much about why it seems only to be bothering me now.

Maybe we should improve what Emacs does in this case by calling
ask-user-about-lock instead of just erroring out?  The information
with which to call ask-user-about-lock is already available at that
point, since it is the value returned by file-locked-p.

Then the prompt will be what you expected it to be, and will also be
consistent with what Emacs does when 2 sessions try to modify the same
file.

Would you like to prepare a patch along these lines?

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

   From: Eli Zaretskii <eliz@gnu.org>
   Date: Sat, 07 Jun 2014 11:34:39 +0300

   Would you like to prepare a patch along these lines?

Will do.

                                 Ken

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

   From: Eli Zaretskii <eliz@gnu.org>
   Date: Sat, 07 Jun 2014 11:34:39 +0300

   Maybe we should improve what Emacs does in this case by calling
   ask-user-about-lock instead of just erroring out?

Thinking about this more, I'm not sure it is such a good idea.  For one
thing, there is the possibility of races, since we aren't actually
locking the file but just checking if it is locked.  Perhaps more
importantly, if the user says to steal the file, I don't know how to
actually do so.  This is normally done by lock_file after
ask-user-about-lock returns t.   And if the user says to ignore the
lock, I don't know how to prevent this query again when the buffer
actually is modified.

So maybe it would be better to actually lock the file.  Then we'd be
using file locks as intended.  The way I'd do it is to lock the file in
rmail-get-new-mail-1 before calling rmail-insert-inbox-text and then
unlock it after saving the file with the new messages or after
discovering that there are none.

Unfortunately, there doesn't seem to be any lisp interface to lock_file,
but I guess I could set buffer-modified-p and then call lock-buffer.

What do you think?

                                Ken

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Ken Olum <kdo@cosmos.phy.tufts.edu>
> Cc: 17706@debbugs.gnu.org
> Date: Sat, 14 Jun 2014 08:16:39 -0400
> 
> So maybe it would be better to actually lock the file.  Then we'd be
> using file locks as intended.  The way I'd do it is to lock the file in
> rmail-get-new-mail-1 before calling rmail-insert-inbox-text and then
> unlock it after saving the file with the new messages or after
> discovering that there are none.
> 
> Unfortunately, there doesn't seem to be any lisp interface to lock_file,
> but I guess I could set buffer-modified-p and then call lock-buffer.
> 
> What do you think?

Sounds OK to me.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Ken Olum]

[-- Attachment #1: Type: text/plain, Size: 281 bytes --]

Here is a patch to mark the buffer modified (which locks the file) in
rmail-get-new-mail-1 before calling rmail-insert-inbox-text, and mark it
unmodified if rmail-insert-inbox-text returns (or is aborted) without
modifying the buffer.

                                        Ken


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: rmail-lock.patch --]
[-- Type: text/x-diff, Size: 2182 bytes --]

=== modified file 'lisp/mail/rmail.el'
--- lisp/mail/rmail.el	2014-05-21 16:58:19 +0000
+++ lisp/mail/rmail.el	2014-06-26 19:18:00 +0000
@@ -1816,9 +1816,20 @@
 	;; Read in the contents of the inbox files, renaming them as
 	;; necessary, and adding to the list of files to delete
 	;; eventually.
-	(if file-name
-	    (rmail-insert-inbox-text files nil)
-	  (setq delete-files (rmail-insert-inbox-text files t)))
+	(unwind-protect
+	    (progn
+	      ;;Set modified now to lock the file, so that we don't encounter
+	      ;;locking problems later in the middle of reading the mail.
+	      (set-buffer-modified-p t)
+	      (if file-name
+		  (rmail-insert-inbox-text files nil)
+		(setq delete-files (rmail-insert-inbox-text files t))))
+	  ;;If there was no new mail, or we aborted before actually trying
+	  ;;to get any, mark buffer unmodified.  Otherwise the buffer is
+	  ;;correctly marked modified and the file locked until we save
+	  ;; out the new mail.
+	  (if (= (point-min) (point-max)))
+	      (set-buffer-modified-p nil))
 	;; Scan the new text and convert each message to
 	;; Rmail/mbox format.
 	(goto-char (point-min))
@@ -1968,11 +1979,6 @@
     size))
 
 (defun rmail-insert-inbox-text (files renamep)
-  ;; Detect a locked file now, so that we avoid moving mail
-  ;; out of the real inbox file.  (That could scare people.)
-  (or (memq (file-locked-p buffer-file-name) '(nil t))
-      (error "RMAIL file %s is locked"
-	     (file-name-nondirectory buffer-file-name)))
   (let (file tofile delete-files movemail popmail got-password password)
     (while files
       ;; Handle remote mailbox names specially; don't expand as filenames

=== modified file 'lisp/ChangeLog'
--- lisp/ChangeLog	2014-06-26 19:46:02 +0000
+++ lisp/ChangeLog	2014-06-26 19:46:30 +0000
@@ -1,3 +1,10 @@
+2014-06-26  Ken Olum  <kdo@cosmos.phy.tufts.edu>
+
+	* mail/rmail.el (rmail-get-new-mail-1): Lock file before
+	calling rmail-insert-inbox-text.  Unlock if
+	rmail-insert-inbox-text doesn't modify buffer.
+	(rmail-insert-inbox-text): Remove lock-checking code
+
 2014-06-26  Leo Liu  <sdl.web@gmail.com>
 
 	* skeleton.el (skeleton-end-hook): Default to nil and move the

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Ken Olum <kdo@cosmos.phy.tufts.edu>
> Cc: 17706@debbugs.gnu.org
> Date: Thu, 26 Jun 2014 15:52:22 -0400
> 
> Here is a patch to mark the buffer modified (which locks the file) in
> rmail-get-new-mail-1 before calling rmail-insert-inbox-text, and mark it
> unmodified if rmail-insert-inbox-text returns (or is aborted) without
> modifying the buffer.

Thanks.  This looks OK to me.  I will wait a few days for comments,
and commit then.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Glenn Morris]

Eli Zaretskii wrote:

> Thanks.  This looks OK to me.  I will wait a few days for comments,
> and commit then.

My only comment is that this looks like the last "tiny change" we can
accept without an assignment.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: Ken Olum <kdo@cosmos.phy.tufts.edu>,  17706@debbugs.gnu.org
> Date: Fri, 27 Jun 2014 11:54:14 -0400
> 
> Eli Zaretskii wrote:
> 
> > Thanks.  This looks OK to me.  I will wait a few days for comments,
> > and commit then.
> 
> My only comment is that this looks like the last "tiny change" we can
> accept without an assignment.

It changes exactly 5 lines of code, all the rest is either existing
code or comments.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Glenn Morris]

Eli Zaretskii wrote:

> It changes exactly 5 lines of code, all the rest is either existing
> code or comments.

And added to the previous changes from the same author?

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Glenn Morris <rgm@gnu.org>
> Cc: kdo@cosmos.phy.tufts.edu,  17706@debbugs.gnu.org
> Date: Fri, 27 Jun 2014 15:54:25 -0400
> 
> Eli Zaretskii wrote:
> 
> > It changes exactly 5 lines of code, all the rest is either existing
> > code or comments.
> 
> And added to the previous changes from the same author?

2 more lines of code.

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Mark Lillibridge]

    This bug is still present in Emacs 24.5, but a claimed patch has
been provided.  At a minimum, this should be moved to outstanding bugs
-- normal bug; patch available unless there's a problem with the patch.
Ideally, the patch should get applied... :-)

- Mark

bug#17706: 24.4.50; locked file error in rmail-insert-inbox-text

[Eli Zaretskii]

> From: Mark Lillibridge <mdl@alum.mit.edu>
> Date: Mon, 23 May 2016 13:23:12 -0700
> 
> 
>     This bug is still present in Emacs 24.5, but a claimed patch has
> been provided.  At a minimum, this should be moved to outstanding bugs
> -- normal bug; patch available unless there's a problem with the patch.
> Ideally, the patch should get applied... :-)

Sorry, I applied it now.  Unfortunately, it's too late for Emacs 25.1,
so the change is on the master branch, and will be in the next release
after that.