bug#20280: 25.0.50; Auto revert mode appears to be non-functional

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Christopher Carpenter]

M-x find-file /tmp/some-file
M-x auto-revert-mode

**** IN A SEPARATE PROCESS CAUSE THE FILE TO CHANGE ***

Expected: Buffer automatically reverts
Actual: Buffer shows a M in mode line signifying the file has changed,
but does not revert.

Additional details: This appears to affect auto-revert-mode,
auto-revert-tail-mode, and magit's auto revert on branch switching.



In GNU Emacs 25.0.50.1 (x86_64-apple-darwin14.1.0, NS appkit-1344.72 Version 10.10.2 (Build 14C1514))
 of 2015-04-08 on Christophers-MacBook-Pro.local
Repository revision: bf87b4d5a83cff934b469a95c14d782f68ea963a
Windowing system distributor `Apple', version 10.3.1344
Configured using:
 `configure --prefix=/usr/local/Cellar/emacs/HEAD
 --enable-locallisppath=/usr/local/share/emacs/site-lisp
 --infodir=/usr/local/Cellar/emacs/HEAD/share/info/emacs
 --with-file-notification=gfile --without-dbus --with-gnutls --with-rsvg
 --with-imagemagick --without-popmail --with-ns
 --disable-ns-self-contained'

Configured features:
RSVG IMAGEMAGICK NOTIFY ACL GNUTLS LIBXML2 ZLIB

Important settings:
  locale-coding-system: utf-8-unix

Major mode: Text

Minor modes in effect:
  magit-auto-revert-mode: t
  auto-revert-mode: t
  diff-auto-refine-mode: t
  global-company-mode: t
  company-mode: t
  show-smartparens-global-mode: t
  show-smartparens-mode: t
  smartparens-global-mode: t
  smartparens-mode: t
  erc-spelling-mode: t
  erc-log-mode: t
  erc-services-mode: t
  erc-list-mode: t
  erc-menu-mode: t
  erc-autojoin-mode: t
  erc-ring-mode: t
  erc-networks-mode: t
  erc-pcomplete-mode: t
  erc-track-mode: t
  erc-track-minor-mode: t
  erc-match-mode: t
  erc-button-mode: t
  erc-fill-mode: t
  erc-stamp-mode: t
  erc-netsplit-mode: t
  erc-irccontrols-mode: t
  erc-noncommands-mode: t
  erc-move-to-prompt-mode: t
  erc-readonly-mode: t
  global-rbenv-mode: t
  projectile-global-mode: t
  projectile-mode: t
  show-paren-mode: t
  display-time-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  helm-mode: t
  shell-dirtrack-mode: t
  helm-autoresize-mode: t
  recentf-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  size-indication-mode: t
  column-number-mode: t
  line-number-mode: t

Recent messages:
Checking 259 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/gnus...
Checking 57 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/eshell...
Checking 70 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/erc...
Checking 34 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/emulation...
Checking 159 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/emacs-lisp...
Checking 24 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/cedet...
Checking 57 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/calendar...
Checking 87 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/calc...
Checking 115 files in /usr/local/Cellar/emacs/HEAD/share/emacs/25.0.50/lisp/obsolete...
Checking for load-path shadows...done

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug sendmail magit-key-mode magit view
git-rebase-mode git-commit-mode log-edit pcvs-util add-log w3m-form
w3m-filter w3m-cookie w3m-bookmark w3m-tabmenu w3m-session w3m doc-view
jka-compr image-mode timezone w3m-hist w3m-fb bookmark-w3m w3m-ems
w3m-ccl ccl w3m-favicon w3m-image w3m-proc w3m-util ido helm-misc
sane-term autorevert filenotify helm-command helm-elisp helm-eval vc-git
diff-mode winner rainbow-delimiters rainbow-identifiers color
company-files company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-capf
company-cmake company-xcode company-clang company-semantic company-eclim
company-template company-css company-nxml company-bbdb company
solarized-light-theme solarized yaml-mode smartparens-config
smartparens-ruby smartparens-html smartparens shen-mode find-lisp geiser
jade-mode sws-mode web-mode whitespace epa-file epa epg ob-dot
org-install erc-spelling flyspell ispell erc-log erc-services erc-list
erc-menu erc-join erc-ring erc-networks erc-pcomplete erc-track
erc-match erc-button erc-fill erc-stamp erc-netsplit erc-goodies erc
erc-backend erc-compat scss-mode flymake derived haml-mode markdown-mode
css-mode rspec-mode ruby-block inf-ruby yari ruby-electric ruby-mode
smie rbenv smart-mode-line-respectful-theme smart-mode-line mule-util
rich-minority helm-projectile projectile ibuf-ext ibuffer dash json-mode
json-reformat json-snatcher js json imenu cc-mode cc-fonts cc-guess
cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
exec-path-from-shell multi-term term disp-table ehelp color-theme
saveplace paren time cus-start cus-load slime-fancy slime-trace-dialog
slime-fontifying-fu slime-package-fu slime-references
slime-compiler-notes-tree slime-scratch slime-presentations bridge
slime-fuzzy slime-fancy-trace slime-fancy-inspector slime-c-p-c
slime-editing-commands slime-autodoc slime-repl slime-parse slime etags
xref arc-mode archive-mode hyperspec yasnippet help-mode cl two-column
helm-mode helm-files rx image-dired tramp tramp-compat tramp-loaddefs
trampver shell dired-x dired-aux ffap thingatpt helm-buffers
helm-elscreen helm-tags helm-bookmark helm-adaptive helm-info bookmark
pp helm-locate helm-help helm-org org org-macro org-footnote
org-pcomplete pcomplete org-list org-faces org-entities noutline outline
org-version ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp
org-src ob-keys ob-comint ob-core ob-eval org-compat org-macs
org-loaddefs find-func cal-menu calendar cal-loaddefs helm-match-plugin
helm-grep helm-regexp helm-plugin grep helm-external helm-net browse-url
xml helm-utils helm-config async-bytecomp async helm-aliases helm advice
easy-mmode edmacro kmacro helm-source eieio-compat recentf tree-widget
wid-edit gnutls mm-archive message dired format-spec rfc822 mml mml-sec
mailabbrev gmm-utils mailheader mm-decode mm-bodies mm-encode mail-utils
network-stream nsm starttls url-http tls mail-parse rfc2231 rfc2047
rfc2045 ietf-drums url-gw url-cache url-auth url url-proxy url-privacy
url-expand url-methods url-history url-cookie url-domsuf url-util
mailcap url-handlers url-parse auth-source cl-macs eieio eieio-core
cl-generic byte-opt gv gnus-util mm-util help-fns mail-prsvr
password-cache url-vars finder-inf go-mode-autoloads info easymenu
slime-autoloads package epg-config compile comint ansi-color ring
bytecomp byte-compile cl-extra seq cl-loaddefs pcase cl-lib cconv server
time-date tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel ns-win tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment elisp-mode lisp-mode prog-mode register
page menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock
font-lock syntax facemenu font-core frame cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote make-network-process
gfilenotify cocoa ns multi-tty emacs)

Memory information:
((conses 16 684808 81787)
 (symbols 48 59590 8)
 (miscs 40 243 1037)
 (strings 32 175858 21705)
 (string-bytes 1 6527220)
 (vectors 16 73113)
 (vector-slots 8 1857583 53167)
 (floats 8 1142 847)
 (intervals 56 4783 3512)
 (buffers 976 36))

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Eli Zaretskii]

> From: Christopher Carpenter <mordocai@mordocai.net>
> Date: Wed, 08 Apr 2015 12:20:10 -0500
> 
> 
> M-x find-file /tmp/some-file
> M-x auto-revert-mode
> 
> **** IN A SEPARATE PROCESS CAUSE THE FILE TO CHANGE ***
> 
> Expected: Buffer automatically reverts
> Actual: Buffer shows a M in mode line signifying the file has changed,
> but does not revert.

I cannot reproduce this, with today's Git repository.

Does this happen in "emacs -Q"?
Does this happen with any file, or just with files in /tmp?

Could be Darwin-specific (I don't have access to such a system).

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Christopher Carpenter]

Alright, got this to reproduce with emacs -Q

M-x wasn't working for me in emacs -Q on Mac so I had to be a bit inventive.

I'm guessing it is a darwin specific issue like you said.

Exact steps for me:

Start with emacs -Q

Put: (global-set-key (kbd "<f2> a") 'auto-revert-mode) in *scratch*

Run above lisp command using C-x e

C-x f ~/test.txt

<f2> a

*** It says Auto-Revert mode enabled in current buffer. ARev shows in mode
line ***

In separate terminal, echo "blargh" > test.txt

*** Nothing happens in test.txt buffer. ***

>> From: Christopher Carpenter <mordocai@mordocai.net>
>> Date: Wed, 08 Apr 2015 12:20:10 -0500
>>
>>
>> M-x find-file /tmp/some-file
>> M-x auto-revert-mode
>>
>> **** IN A SEPARATE PROCESS CAUSE THE FILE TO CHANGE ***
>>
>> Expected: Buffer automatically reverts
>> Actual: Buffer shows a M in mode line signifying the file has changed,
>> but does not revert.
>
> I cannot reproduce this, with today's Git repository.
>
> Does this happen in "emacs -Q"?
> Does this happen with any file, or just with files in /tmp?
>
> Could be Darwin-specific (I don't have access to such a system).
>
>

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Michael Albinus]

"Christopher Carpenter" <mordocai@mordocai.net> writes:

> Alright, got this to reproduce with emacs -Q
>
> M-x wasn't working for me in emacs -Q on Mac so I had to be a bit inventive.
>
> I'm guessing it is a darwin specific issue like you said.
>
> Exact steps for me:
>
> Start with emacs -Q
>
> Put: (global-set-key (kbd "<f2> a") 'auto-revert-mode) in *scratch*
>
> Run above lisp command using C-x e
>
> C-x f ~/test.txt
>
> <f2> a
>
> *** It says Auto-Revert mode enabled in current buffer. ARev shows in mode
> line ***
>
> In separate terminal, echo "blargh" > test.txt
>
> *** Nothing happens in test.txt buffer. ***

You seem to use gfilenotify. What happens, if you set auto-revert-use-notify
to nil, and start to watch another file?

Best regards, Michael.

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Christopher Carpenter]

> "Christopher Carpenter" <mordocai@mordocai.net> writes:
>
>> Alright, got this to reproduce with emacs -Q
>>
>> M-x wasn't working for me in emacs -Q on Mac so I had to be a bit
>> inventive.
>>
>> I'm guessing it is a darwin specific issue like you said.
>>
>> Exact steps for me:
>>
>> Start with emacs -Q
>>
>> Put: (global-set-key (kbd "<f2> a") 'auto-revert-mode) in *scratch*
>>
>> Run above lisp command using C-x e
>>
>> C-x f ~/test.txt
>>
>> <f2> a
>>
>> *** It says Auto-Revert mode enabled in current buffer. ARev shows in
>> mode
>> line ***
>>
>> In separate terminal, echo "blargh" > test.txt
>>
>> *** Nothing happens in test.txt buffer. ***
>
> You seem to use gfilenotify. What happens, if you set
> auto-revert-use-notify
> to nil, and start to watch another file?
>
> Best regards, Michael.
>
>

Ah, excellent. That does indeed appear to work. I'm using brew and this is
the first time I've compiled from source instead of using a pre-built
binary. I assume that is what caused the difference, not code changes in
emacs HEAD.

Unless this is actually a gfilenotify <-> emacs problem(rather than a
gfilenotify on mac problem) I'd consider this solved/not a bug. Sorry for
wasting time!

Thanks,
Christopher Carpenter

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Michael Albinus]

"Christopher Carpenter" <mordocai@mordocai.net> writes:

Hi Christopher,

> Unless this is actually a gfilenotify <-> emacs problem(rather than a
> gfilenotify on mac problem) I'd consider this solved/not a bug. Sorry for
> wasting time!

Like Eli, I cannot test on a Mac. If you don't mind, I would like to
misuse you for further investigations.

Even if it is a glib-on-the-mac problem, I would prefer that file
notifications are not enabled then when building Emacs. This must be
determined robustly.

glib on OSX (BSD, in fact) uses either kqueue, or it polls for file
notifications. Maybe you could check, whether libkqueue is linked to Emacs.

Furthermore, it might be worth to set traces before running the
tests. Some thing like this:

(require 'filenotify)
(require 'trace)
(dolist (elt (all-completions "file-notify-" obarray 'functionp))
  (trace-function-background (intern elt)))

You might observe the buffer *trace-output*, whether file notifications
events arrive.

> Thanks,
> Christopher Carpenter

Best regards, Michael.

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Christopher Carpenter]

Hello again Michael,

It doesn't look like libkqueue is linked to my emacs executable to me.

➜  bin git:(master) otool -L emacs-25.0.50
emacs-25.0.50:
	/System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
(compatibility version 45.0.0, current version 1344.72.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
(compatibility version 1.0.0, current version 275.0.0)
	/usr/local/lib/librsvg-2.2.dylib (compatibility version 39.0.0, current
version 39.3.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version
1213.0.0)
	/usr/local/lib/libgio-2.0.0.dylib (compatibility version 4401.0.0,
current version 4401.0.0)
	/usr/local/lib/libgdk_pixbuf-2.0.0.dylib (compatibility version 3001.0.0,
current version 3001.8.0)
	/usr/local/lib/libgobject-2.0.0.dylib (compatibility version 4401.0.0,
current version 4401.0.0)
	/usr/local/lib/libglib-2.0.0.dylib (compatibility version 4401.0.0,
current version 4401.0.0)
	/usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 10.0.0,
current version 10.3.0)
	/usr/local/lib/libcairo.2.dylib (compatibility version 11403.0.0, current
version 11403.2.0)
	/usr/local/lib/libMagickWand-6.Q16.2.dylib (compatibility version 3.0.0,
current version 3.0.0)
	/usr/local/lib/libMagickCore-6.Q16.2.dylib (compatibility version 3.0.0,
current version 3.0.0)
	/usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version
10.9.0)
	/usr/lib/libncurses.5.4.dylib (compatibility version 5.4.0, current
version 5.4.0)
	/usr/local/lib/libgnutls.28.dylib (compatibility version 70.0.0, current
version 70.6.0)
	/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.5)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
(compatibility version 150.0.0, current version 1152.0.0)
	/System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
(compatibility version 64.0.0, current version 600.0.0)
	/System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
(compatibility version 1.0.0, current version 1.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
(compatibility version 300.0.0, current version 1152.14.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version
228.0.0)

I ran the code you gave there, and did multiple writes to the file. The
following is the only text that appeared in *trace-output* and I believe
it appeared when I activated auto-revert, not when I wrote to the file.

======================================================================
1 -> (file-notify-add-watch "/Users/ccarpenter/test.txt" (change
attribute-chan\
ge) auto-revert-notify-handler)
| 2 -> (file-notify--descriptor 1080241180 "test.txt")
| 2 <- file-notify--descriptor: 1080241180
1 <- file-notify-add-watch: 1080241180

From my amateur view it looks like we are not getting any notifications.

Does that help any?

Christopher Carpenter


> "Christopher Carpenter" <mordocai@mordocai.net> writes:
>
> Hi Christopher,
>
>> Unless this is actually a gfilenotify <-> emacs problem(rather than a
>> gfilenotify on mac problem) I'd consider this solved/not a bug. Sorry
>> for
>> wasting time!
>
> Like Eli, I cannot test on a Mac. If you don't mind, I would like to
> misuse you for further investigations.
>
> Even if it is a glib-on-the-mac problem, I would prefer that file
> notifications are not enabled then when building Emacs. This must be
> determined robustly.
>
> glib on OSX (BSD, in fact) uses either kqueue, or it polls for file
> notifications. Maybe you could check, whether libkqueue is linked to
> Emacs.
>
> Furthermore, it might be worth to set traces before running the
> tests. Some thing like this:
>
> (require 'filenotify)
> (require 'trace)
> (dolist (elt (all-completions "file-notify-" obarray 'functionp))
>   (trace-function-background (intern elt)))
>
> You might observe the buffer *trace-output*, whether file notifications
> events arrive.
>
>> Thanks,
>> Christopher Carpenter
>
> Best regards, Michael.
>
>

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Michael Albinus]

"Christopher Carpenter" <mordocai@mordocai.net> writes:

> Hello again Michael,

Hi Christopher,

> It doesn't look like libkqueue is linked to my emacs executable to me.

Yes. But glib is linked.

> I ran the code you gave there, and did multiple writes to the file. The
> following is the only text that appeared in *trace-output* and I believe
> it appeared when I activated auto-revert, not when I wrote to the file.
>
> ======================================================================
> 1 -> (file-notify-add-watch "/Users/ccarpenter/test.txt" (change
> attribute-chan\
> ge) auto-revert-notify-handler)
> | 2 -> (file-notify--descriptor 1080241180 "test.txt")
> | 2 <- file-notify--descriptor: 1080241180
> 1 <- file-notify-add-watch: 1080241180
>
> From my amateur view it looks like we are not getting any notifications.

Yes.

> Does that help any?

Yes. When glib is not linked with a native file notification library
like inotify or kqueue, it should poll in order to find file changes. It
doesn't seem to do that in your case.

I will dig deeper into this, trying to find out what makes glib to
refuse to poll. Maybe we have an error bringing glib into the main loop,
or there is something we miss to configure for glib, or it is simply a
glib error.

It might take some days, until I'll be back.

> Christopher Carpenter

Best regards, Michael.

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> I will dig deeper into this, trying to find out what makes glib to
> refuse to poll. Maybe we have an error bringing glib into the main loop,
> or there is something we miss to configure for glib, or it is simply a
> glib error.

glib's main loop integration in Emacs is implemented by xg_select, a
replacement for pselect in case glib is linked to Emacs. However, in
file process.c:4915, we have the code

--8<---------------cut here---------------start------------->8---
#if defined (HAVE_NS)
          nfds = ns_select
#elif defined (HAVE_GLIB)
	  nfds = xg_select
#else
	  nfds = pselect
#endif
            (max (max_process_desc, max_input_desc) + 1,
             &Available,
             (check_write ? &Writeok : 0),
             NULL, &timeout, NULL);
--8<---------------cut here---------------end--------------->8---

That means, that in your case on OS X ns_select is called, instead of
xg_select.

I have no idea how to solve this properly.

>> Christopher Carpenter

Best regards, Michael.

bug#20280: 25.0.50; Auto revert mode appears to be non-functional

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> That means, that in your case on OS X ns_select is called, instead of
> xg_select.
>
> I have no idea how to solve this properly.

Paul Eggert has committed patch 200c2b10faf298bf65e8b6dbd0cb9ef00b2f95d6,
which suppresses file notification if HAVE_NS. This is not a real
solution, but we have no better way to handle it. I'm closing the bug.

Best regards, Michael.