bug#21435: 25.0.50; file-notify has problems after renames

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

file-notify using the inotify backend reports wrong events after
directory renames.  For example, with

(let* ((flags '(change attribute-change))
       (handler (lambda (event)
		  (message "WATCH: %S" event)))
       (x "/tmp/x")
       (y "/tmp/x/y"))
  (make-directory x)
  (make-directory y)
  (file-notify-add-watch x flags handler)
  (file-notify-add-watch y flags handler))

the following events are reported after these shell operations:

  1. $ touch /tmp/x/y/test1
     WATCH: ((12) created "/tmp/x/y/test1")
     WATCH: ((12) attribute-changed "/tmp/x/y/test1")
  2. $ mv /tmp/x/y /tmp/x/z
     WATCH: ((11) renamed "/tmp/x/y" "/tmp/x/z") [2 times]
  3. $ touch /tmp/x/z/test2
     WATCH: ((12) created "/tmp/x/y/test2")
     WATCH: ((12) attribute-changed "/tmp/x/y/test2")

In 3, the file created is actually /tmp/x/z/test2, not /tmp/x/y/test2.

There is also a problem with single-file watches:

(let* ((flags '(change attribute-change))
       (handler (lambda (event)
		  (message "WATCH: %S" event)))
       (x "/tmp/x"))
  (with-temp-buffer
    (insert "TEST\n")
    (write-file x))
  (file-notify-add-watch x flags handler))

  1. $ mv /tmp/x /tmp/y
     WATCH: ((1 . "x") renamed "/tmp/x" "/tmp/y") [2 times]
  2. $ echo "more" >> /tmp/y
     ;; no notification
  3. $ touch /tmp/x
     WATCH: ((1 . "x") created "/tmp/x")
     WATCH: ((1 . "x") changed "/tmp/x")

So in the case of single-file watches, the watch seems to stick to a
filename rather than a file.  Well, I think that's ok but then I would
expect it to be similar with directories, e.g., in the first example
step 3 should not report a notification because I intended to watch
/tmp/x/y and not /tmp/x/z.

Another thing is that renames always seem to be reported twice (although
the example in the manual lists only one event but that example has
probably gathered with the gfilenotify backend).



In GNU Emacs 25.0.50.10 (x86_64-unknown-linux-gnu, GTK+ Version 3.16.6)
 of 2015-09-08
Repository revision: 6cd64aaf1317bdd96f29fa2a7c3d49172e79bc37
Windowing system distributor 'The X.Org Foundation', version 11.0.11702000
System Description:	Arch Linux

Configured using:
 'configure 'CFLAGS=-g -ggdb3 -O1''

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LC_MONETARY: de_DE.utf8
  value of $LC_NUMERIC: de_DE.utf8
  value of $LC_TIME: de_DE.utf8
  value of $LANG: en_US.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  diff-auto-refine-mode: t
  TeX-PDF-mode: t
  TeX-source-correlate-mode: t
  global-company-mode: t
  company-mode: t
  shell-dirtrack-mode: t
  paredit-mode: t
  global-aggressive-indent-mode: t
  aggressive-indent-mode: t
  highlight-symbol-mode: t
  outline-minor-mode: t
  pdf-occur-global-minor-mode: t
  recentf-mode: t
  highlight-parentheses-mode: t
  global-subword-mode: t
  subword-mode: t
  save-place-mode: t
  savehist-mode: t
  show-paren-mode: t
  ivy-mode: t
  minibuffer-depth-indicate-mode: t
  electric-pair-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
WATCH: (((10) attribute-changed "/home/horn/x/y/bar"))
Auto-saving...
WATCH: (((10) created "/home/horn/x/y/foo"))
WATCH: (((10) attribute-changed "/home/horn/x/y/foo"))
Mark set [2 times]
Undo! [10 times]
Indenting region...done
WATCH: (((10) deleted "/home/horn/x/y/bar"))
WATCH: (((10) deleted "/home/horn/x/y/foo"))
WATCH: (((9) deleted "/home/horn/x/z"))

Load-path shadows:
~/Repos/el/auctex/lpath hides ~/Repos/el/gnus/lisp/lpath
~/Repos/el/highlight-symbol.el/highlight-symbol hides /home/horn/.emacs.d/elpa/highlight-symbol-20150816.628/highlight-symbol
~/Repos/el/gnus/lisp/md4 hides /home/horn/Repos/el/emacs/lisp/md4
~/Repos/el/gnus/lisp/color hides /home/horn/Repos/el/emacs/lisp/color
~/Repos/el/gnus/lisp/format-spec hides /home/horn/Repos/el/emacs/lisp/format-spec
~/Repos/el/gnus/lisp/password-cache hides /home/horn/Repos/el/emacs/lisp/password-cache
~/Repos/el/gnus/lisp/hex-util hides /home/horn/Repos/el/emacs/lisp/hex-util
~/Repos/el/gnus/lisp/dns-mode hides /home/horn/Repos/el/emacs/lisp/textmodes/dns-mode
~/Repos/el/gnus/lisp/dig hides /home/horn/Repos/el/emacs/lisp/net/dig
~/Repos/el/gnus/lisp/hmac-md5 hides /home/horn/Repos/el/emacs/lisp/net/hmac-md5
~/Repos/el/gnus/lisp/ntlm hides /home/horn/Repos/el/emacs/lisp/net/ntlm
~/Repos/el/gnus/lisp/hmac-def hides /home/horn/Repos/el/emacs/lisp/net/hmac-def
~/Repos/el/gnus/lisp/rfc2104 hides /home/horn/Repos/el/emacs/lisp/net/rfc2104
~/Repos/el/gnus/lisp/sasl-ntlm hides /home/horn/Repos/el/emacs/lisp/net/sasl-ntlm
~/Repos/el/gnus/lisp/sasl-cram hides /home/horn/Repos/el/emacs/lisp/net/sasl-cram
~/Repos/el/gnus/lisp/dns hides /home/horn/Repos/el/emacs/lisp/net/dns
~/Repos/el/gnus/lisp/sasl hides /home/horn/Repos/el/emacs/lisp/net/sasl
~/Repos/el/gnus/lisp/tls hides /home/horn/Repos/el/emacs/lisp/net/tls
~/Repos/el/gnus/lisp/sasl-scram-rfc hides /home/horn/Repos/el/emacs/lisp/net/sasl-scram-rfc
~/Repos/el/gnus/lisp/netrc hides /home/horn/Repos/el/emacs/lisp/net/netrc
~/Repos/el/gnus/lisp/sasl-digest hides /home/horn/Repos/el/emacs/lisp/net/sasl-digest
~/Repos/el/gnus/lisp/uudecode hides /home/horn/Repos/el/emacs/lisp/mail/uudecode
~/Repos/el/gnus/lisp/binhex hides /home/horn/Repos/el/emacs/lisp/mail/binhex
~/Repos/el/gnus/lisp/hashcash hides /home/horn/Repos/el/emacs/lisp/mail/hashcash
~/Repos/el/gnus/lisp/canlock hides /home/horn/Repos/el/emacs/lisp/gnus/canlock
~/Repos/el/gnus/lisp/nneething hides /home/horn/Repos/el/emacs/lisp/gnus/nneething
~/Repos/el/gnus/lisp/mm-encode hides /home/horn/Repos/el/emacs/lisp/gnus/mm-encode
~/Repos/el/gnus/lisp/mm-util hides /home/horn/Repos/el/emacs/lisp/gnus/mm-util
~/Repos/el/gnus/lisp/rfc2047 hides /home/horn/Repos/el/emacs/lisp/gnus/rfc2047
~/Repos/el/gnus/lisp/nnml hides /home/horn/Repos/el/emacs/lisp/gnus/nnml
~/Repos/el/gnus/lisp/gnus-cus hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-cus
~/Repos/el/gnus/lisp/gnus-range hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-range
~/Repos/el/gnus/lisp/gnus-int hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-int
~/Repos/el/gnus/lisp/gnus-cloud hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-cloud
~/Repos/el/gnus/lisp/spam-stat hides /home/horn/Repos/el/emacs/lisp/gnus/spam-stat
~/Repos/el/gnus/lisp/nnmh hides /home/horn/Repos/el/emacs/lisp/gnus/nnmh
~/Repos/el/gnus/lisp/gnus-mlspl hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-mlspl
~/Repos/el/gnus/lisp/deuglify hides /home/horn/Repos/el/emacs/lisp/gnus/deuglify
~/Repos/el/gnus/lisp/gnus-gravatar hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-gravatar
~/Repos/el/gnus/lisp/nngateway hides /home/horn/Repos/el/emacs/lisp/gnus/nngateway
~/Repos/el/gnus/lisp/ietf-drums hides /home/horn/Repos/el/emacs/lisp/gnus/ietf-drums
~/Repos/el/gnus/lisp/mail-parse hides /home/horn/Repos/el/emacs/lisp/gnus/mail-parse
~/Repos/el/gnus/lisp/gnus-salt hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-salt
~/Repos/el/gnus/lisp/nnimap hides /home/horn/Repos/el/emacs/lisp/gnus/nnimap
~/Repos/el/gnus/lisp/gnus-draft hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-draft
~/Repos/el/gnus/lisp/mail-source hides /home/horn/Repos/el/emacs/lisp/gnus/mail-source
~/Repos/el/gnus/lisp/messcompat hides /home/horn/Repos/el/emacs/lisp/gnus/messcompat
~/Repos/el/gnus/lisp/pop3 hides /home/horn/Repos/el/emacs/lisp/gnus/pop3
~/Repos/el/gnus/lisp/nnmaildir hides /home/horn/Repos/el/emacs/lisp/gnus/nnmaildir
~/Repos/el/gnus/lisp/nnheader hides /home/horn/Repos/el/emacs/lisp/gnus/nnheader
~/Repos/el/gnus/lisp/gnus-cite hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-cite
~/Repos/el/gnus/lisp/nndiary hides /home/horn/Repos/el/emacs/lisp/gnus/nndiary
~/Repos/el/gnus/lisp/gnus-diary hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-diary
~/Repos/el/gnus/lisp/nnfolder hides /home/horn/Repos/el/emacs/lisp/gnus/nnfolder
~/Repos/el/gnus/lisp/gnus-art hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-art
~/Repos/el/gnus/lisp/gnus-demon hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-demon
~/Repos/el/gnus/lisp/mml-sec hides /home/horn/Repos/el/emacs/lisp/gnus/mml-sec
~/Repos/el/gnus/lisp/nnir hides /home/horn/Repos/el/emacs/lisp/gnus/nnir
~/Repos/el/gnus/lisp/mm-partial hides /home/horn/Repos/el/emacs/lisp/gnus/mm-partial
~/Repos/el/gnus/lisp/gnus-registry hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-registry
~/Repos/el/gnus/lisp/gnus-icalendar hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-icalendar
~/Repos/el/gnus/lisp/compface hides /home/horn/Repos/el/emacs/lisp/gnus/compface
~/Repos/el/gnus/lisp/gnus-fun hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-fun
~/Repos/el/gnus/lisp/gnus-start hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-start
~/Repos/el/gnus/lisp/smiley hides /home/horn/Repos/el/emacs/lisp/gnus/smiley
~/Repos/el/gnus/lisp/gnus-picon hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-picon
~/Repos/el/gnus/lisp/spam-report hides /home/horn/Repos/el/emacs/lisp/gnus/spam-report
~/Repos/el/gnus/lisp/nntp hides /home/horn/Repos/el/emacs/lisp/gnus/nntp
~/Repos/el/gnus/lisp/nnnil hides /home/horn/Repos/el/emacs/lisp/gnus/nnnil
~/Repos/el/gnus/lisp/nndir hides /home/horn/Repos/el/emacs/lisp/gnus/nndir
~/Repos/el/gnus/lisp/gnus-srvr hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-srvr
~/Repos/el/gnus/lisp/smime hides /home/horn/Repos/el/emacs/lisp/gnus/smime
~/Repos/el/gnus/lisp/nnvirtual hides /home/horn/Repos/el/emacs/lisp/gnus/nnvirtual
~/Repos/el/gnus/lisp/gnus-notifications hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-notifications
~/Repos/el/gnus/lisp/nnspool hides /home/horn/Repos/el/emacs/lisp/gnus/nnspool
~/Repos/el/gnus/lisp/gnus-group hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-group
~/Repos/el/gnus/lisp/gnus-bcklg hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-bcklg
~/Repos/el/gnus/lisp/gnus-util hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-util
~/Repos/el/gnus/lisp/gnus-sieve hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-sieve
~/Repos/el/gnus/lisp/nndraft hides /home/horn/Repos/el/emacs/lisp/gnus/nndraft
~/Repos/el/gnus/lisp/nnagent hides /home/horn/Repos/el/emacs/lisp/gnus/nnagent
~/Repos/el/gnus/lisp/gnus-spec hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-spec
~/Repos/el/gnus/lisp/gnus-bookmark hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-bookmark
~/Repos/el/gnus/lisp/mml1991 hides /home/horn/Repos/el/emacs/lisp/gnus/mml1991
~/Repos/el/gnus/lisp/rfc2231 hides /home/horn/Repos/el/emacs/lisp/gnus/rfc2231
~/Repos/el/gnus/lisp/yenc hides /home/horn/Repos/el/emacs/lisp/gnus/yenc
~/Repos/el/gnus/lisp/gnus-undo hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-undo
~/Repos/el/gnus/lisp/ecomplete hides /home/horn/Repos/el/emacs/lisp/gnus/ecomplete
~/Repos/el/gnus/lisp/legacy-gnus-agent hides /home/horn/Repos/el/emacs/lisp/gnus/legacy-gnus-agent
~/Repos/el/gnus/lisp/utf7 hides /home/horn/Repos/el/emacs/lisp/gnus/utf7
~/Repos/el/gnus/lisp/rtree hides /home/horn/Repos/el/emacs/lisp/gnus/rtree
~/Repos/el/gnus/lisp/gnus-uu hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-uu
~/Repos/el/gnus/lisp/gnus-ml hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-ml
~/Repos/el/gnus/lisp/sieve hides /home/horn/Repos/el/emacs/lisp/gnus/sieve
~/Repos/el/gnus/lisp/gnus hides /home/horn/Repos/el/emacs/lisp/gnus/gnus
~/Repos/el/gnus/lisp/mml hides /home/horn/Repos/el/emacs/lisp/gnus/mml
~/Repos/el/gnus/lisp/message hides /home/horn/Repos/el/emacs/lisp/gnus/message
~/Repos/el/gnus/lisp/mml-smime hides /home/horn/Repos/el/emacs/lisp/gnus/mml-smime
~/Repos/el/gnus/lisp/gnus-eform hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-eform
~/Repos/el/gnus/lisp/gnus-agent hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-agent
~/Repos/el/gnus/lisp/gnus-logic hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-logic
~/Repos/el/gnus/lisp/mm-extern hides /home/horn/Repos/el/emacs/lisp/gnus/mm-extern
~/Repos/el/gnus/lisp/nndoc hides /home/horn/Repos/el/emacs/lisp/gnus/nndoc
~/Repos/el/gnus/lisp/sieve-manage hides /home/horn/Repos/el/emacs/lisp/gnus/sieve-manage
~/Repos/el/gnus/lisp/mm-decode hides /home/horn/Repos/el/emacs/lisp/gnus/mm-decode
~/Repos/el/gnus/lisp/starttls hides /home/horn/Repos/el/emacs/lisp/gnus/starttls
~/Repos/el/gnus/lisp/gnus-dired hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-dired
~/Repos/el/gnus/lisp/nnbabyl hides /home/horn/Repos/el/emacs/lisp/gnus/nnbabyl
~/Repos/el/gnus/lisp/nnmbox hides /home/horn/Repos/el/emacs/lisp/gnus/nnmbox
~/Repos/el/gnus/lisp/gnus-win hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-win
~/Repos/el/gnus/lisp/gnus-async hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-async
~/Repos/el/gnus/lisp/mm-url hides /home/horn/Repos/el/emacs/lisp/gnus/mm-url
~/Repos/el/gnus/lisp/gnus-html hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-html
~/Repos/el/gnus/lisp/gssapi hides /home/horn/Repos/el/emacs/lisp/gnus/gssapi
~/Repos/el/gnus/lisp/mml2015 hides /home/horn/Repos/el/emacs/lisp/gnus/mml2015
~/Repos/el/gnus/lisp/nnrss hides /home/horn/Repos/el/emacs/lisp/gnus/nnrss
~/Repos/el/gnus/lisp/gnus-mh hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-mh
~/Repos/el/gnus/lisp/gnus-sum hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-sum
~/Repos/el/gnus/lisp/nnweb hides /home/horn/Repos/el/emacs/lisp/gnus/nnweb
~/Repos/el/gnus/lisp/mail-prsvr hides /home/horn/Repos/el/emacs/lisp/gnus/mail-prsvr
~/Repos/el/gnus/lisp/nnmairix hides /home/horn/Repos/el/emacs/lisp/gnus/nnmairix
~/Repos/el/gnus/lisp/plstore hides /home/horn/Repos/el/emacs/lisp/gnus/plstore
~/Repos/el/gnus/lisp/rfc2045 hides /home/horn/Repos/el/emacs/lisp/gnus/rfc2045
~/Repos/el/gnus/lisp/gnus-msg hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-msg
~/Repos/el/gnus/lisp/spam-wash hides /home/horn/Repos/el/emacs/lisp/gnus/spam-wash
~/Repos/el/gnus/lisp/gnus-score hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-score
~/Repos/el/gnus/lisp/mm-uu hides /home/horn/Repos/el/emacs/lisp/gnus/mm-uu
~/Repos/el/gnus/lisp/spam hides /home/horn/Repos/el/emacs/lisp/gnus/spam
~/Repos/el/gnus/lisp/mm-view hides /home/horn/Repos/el/emacs/lisp/gnus/mm-view
~/Repos/el/gnus/lisp/sieve-mode hides /home/horn/Repos/el/emacs/lisp/gnus/sieve-mode
~/Repos/el/gnus/lisp/html2text hides /home/horn/Repos/el/emacs/lisp/gnus/html2text
~/Repos/el/gnus/lisp/gnus-ems hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-ems
~/Repos/el/gnus/lisp/registry hides /home/horn/Repos/el/emacs/lisp/gnus/registry
~/Repos/el/gnus/lisp/auth-source hides /home/horn/Repos/el/emacs/lisp/gnus/auth-source
~/Repos/el/gnus/lisp/gravatar hides /home/horn/Repos/el/emacs/lisp/gnus/gravatar
~/Repos/el/gnus/lisp/flow-fill hides /home/horn/Repos/el/emacs/lisp/gnus/flow-fill
~/Repos/el/gnus/lisp/gmm-utils hides /home/horn/Repos/el/emacs/lisp/gnus/gmm-utils
~/Repos/el/gnus/lisp/mailcap hides /home/horn/Repos/el/emacs/lisp/gnus/mailcap
~/Repos/el/gnus/lisp/gnus-delay hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-delay
~/Repos/el/gnus/lisp/mm-bodies hides /home/horn/Repos/el/emacs/lisp/gnus/mm-bodies
~/Repos/el/gnus/lisp/mm-archive hides /home/horn/Repos/el/emacs/lisp/gnus/mm-archive
~/Repos/el/gnus/lisp/rfc1843 hides /home/horn/Repos/el/emacs/lisp/gnus/rfc1843
~/Repos/el/gnus/lisp/gnus-kill hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-kill
~/Repos/el/gnus/lisp/qp hides /home/horn/Repos/el/emacs/lisp/gnus/qp
~/Repos/el/gnus/lisp/score-mode hides /home/horn/Repos/el/emacs/lisp/gnus/score-mode
~/Repos/el/gnus/lisp/gnus-topic hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-topic
~/Repos/el/gnus/lisp/gnus-cache hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-cache
~/Repos/el/gnus/lisp/nnmail hides /home/horn/Repos/el/emacs/lisp/gnus/nnmail
~/Repos/el/gnus/lisp/gnus-vm hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-vm
~/Repos/el/gnus/lisp/gnus-sync hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-sync
~/Repos/el/gnus/lisp/nnoo hides /home/horn/Repos/el/emacs/lisp/gnus/nnoo
~/Repos/el/gnus/lisp/nnregistry hides /home/horn/Repos/el/emacs/lisp/gnus/nnregistry
~/Repos/el/gnus/lisp/gnus-dup hides /home/horn/Repos/el/emacs/lisp/gnus/gnus-dup
~/Repos/el/gnus/lisp/parse-time hides /home/horn/Repos/el/emacs/lisp/calendar/parse-time
~/Repos/el/gnus/lisp/time-date hides /home/horn/Repos/el/emacs/lisp/calendar/time-date

Features:
(tramp-cache flow-fill gnus-fun vc vc-dispatcher vc-git diff-mode
texmathp nndoc url-handlers preview prv-emacs auto-dictionary flyspell
ispell tex-buf reftex-dcr reftex-auc reftex reftex-vars font-latex latex
tex-style tex dbus tex-mode latexenc smiley gnus-bcklg qp gnus-async
gnus-ml mailalias smtpmail hippie-exp shadow sort gnus-cite emacsbug
sendmail mm-archive crm debbugs-gnu add-log debbugs soap-client url-http
url-auth url-gw warnings hl-line nndraft nnmh rot13 utf-7 gnutls
network-stream nsm starttls nnml nnnil gnus-agent gnus-srvr gnus-score
score-mode nnvirtual gnus-cache gnus-demon nntp spam spam-stat gnus-uu
yenc gnus-msg gnus-gravatar mail-extr gravatar gnus-topic nnir
gnus-registry registry eieio-compat eieio-base th-private smex ido seq
filenotify debug eieio-opt speedbar sb-image ezimage dframe colir color
company-files company-oddmuse company-keywords company-etags
company-gtags company-dabbrev-code company-dabbrev company-capf
company-cmake company-xcode company-clang company-semantic company-eclim
company-template company-css company-nxml company-bbdb company
stratego-mode greql-mode tg-mode generic preview-latex tex-site
auto-loads cider tramp-sh cider-debug cider-browse-ns cider-inspector
cider-mode cider-repl cider-eldoc cider-interaction spinner arc-mode
archive-mode cider-overlays cider-doc org-table org org-macro
org-footnote org-pcomplete org-list org-faces org-entities org-version
ob-emacs-lisp ob ob-tangle ob-ref ob-lob ob-table ob-exp org-src ob-keys
ob-comint ob-core ob-eval org-compat org-macs org-loaddefs cal-menu
calendar cal-loaddefs cider-test cider-stacktrace cider-client
cider-util nrepl-client tramp tramp-compat tramp-loaddefs trampver shell
pcomplete queue ewoc etags xref project dash clojure-mode paredit
aggressive-indent epa-file epa epg rdictcc google-contacts-message
google-contacts xml url-cache google-oauth google-contacts-gnus gnus-art
mm-uu mml2015 mm-view mml-smime smime dig gnus-sum gnus-group gnus-undo
gnus-start gnus-cloud nnimap nnmail mail-source tls utf7 netrc nnoo
parse-time gnus-spec gnus-int gnus-range gnus-win gnus gnus-ems
gnus-compat url url-proxy url-privacy url-expand url-methods url-history
url-cookie url-domsuf url-util url-parse auth-source password-cache
url-vars mailcap nnheader gnus-util dired-x em-term term ehelp esh-opt
esh-ext esh-util highlight-symbol thingatpt boxquote rect ecomplete
message rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 rfc2047 rfc2045 ietf-drums mm-util help-fns mail-prsvr
mailabbrev mail-utils gmm-utils mailheader server yasnippet disp-table
noutline outline pdf-occur ibuf-ext ibuffer tablist tablist-filter
semantic/wisent/comp semantic/wisent semantic/wisent/wisent
semantic/util-modes semantic/util semantic semantic/tag semantic/lex
semantic/fw eieio byte-opt bytecomp byte-compile cconv eieio-core
mode-local find-func cedet dired pdf-isearch let-alist pdf-misc imenu
pdf-tools compile comint ansi-color cus-edit cus-start cus-load pdf-view
bookmark pp jka-compr pdf-cache pdf-info tq pdf-util format-spec
image-mode browse-kill-ring derived recentf tree-widget wid-edit
highlight-parentheses cl iedit iedit-lib cl-extra help-mode hydra lv
counsel swiper cap-words superword subword saveplace savehist paren ivy
delsel icomplete mb-depth ace-window easy-mmode cl-macs gv avy ring
smart-mode-line-respectful-theme smart-mode-line-light-theme cl-seq
smart-mode-line rich-minority rx bs elec-pair edmacro kmacro cl-loaddefs
cl-lib gnus-load subr-x pcase tsdh-light-theme finder-inf
memory-usage-autoloads advice info package easymenu epg-config time-date
mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win term/common-win x-dnd tool-bar dnd fontset
image regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cl-generic cham
georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese charscript case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer cl-preloaded nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote dbusbind inotify dynamic-setting
system-font-setting font-render-setting move-toolbar gtk x-toolkit x
multi-tty make-network-process emacs)

Memory information:
((conses 16 966399 50632)
 (symbols 48 59145 97)
 (miscs 40 580 1231)
 (strings 32 188617 43232)
 (string-bytes 1 6426636)
 (vectors 16 62335)
 (vector-slots 8 1828918 146254)
 (floats 8 1054 1533)
 (intervals 56 7960 3005)
 (buffers 976 66)
 (heap 1024 100878 51951))

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Date: Tue, 08 Sep 2015 10:47:00 +0200
> 
> file-notify using the inotify backend reports wrong events after
> directory renames.  For example, with
> 
> (let* ((flags '(change attribute-change))
>        (handler (lambda (event)
> 		  (message "WATCH: %S" event)))
>        (x "/tmp/x")
>        (y "/tmp/x/y"))
>   (make-directory x)
>   (make-directory y)
>   (file-notify-add-watch x flags handler)
>   (file-notify-add-watch y flags handler))
> 
> the following events are reported after these shell operations:
> 
>   1. $ touch /tmp/x/y/test1
>      WATCH: ((12) created "/tmp/x/y/test1")
>      WATCH: ((12) attribute-changed "/tmp/x/y/test1")
>   2. $ mv /tmp/x/y /tmp/x/z
>      WATCH: ((11) renamed "/tmp/x/y" "/tmp/x/z") [2 times]
>   3. $ touch /tmp/x/z/test2
>      WATCH: ((12) created "/tmp/x/y/test2")
>      WATCH: ((12) attribute-changed "/tmp/x/y/test2")
> 
> In 3, the file created is actually /tmp/x/z/test2, not /tmp/x/y/test2.

With w32notify, I see something very similar:

  1. $ touch /tmp/x/y/test1
     WATCH: (133902432 created "d:/tmp/x/y/test1")
     WATCH: (133902432 changed "d:/tmp/x/y/test1")
     WATCH: (159542912 changed "d:/tmp/x/y")
  2. $ mv /tmp/x/y /tmp/x/z
     WATCH: (159542912 renamed "d:/tmp/x/y" "d:/tmp/x/z")

(Here the watching thread exist with abnormal status.)

  3. $ touch /tmp/x/z/test2
     WATCH: (133902432 created "d:/tmp/x/y/test2")
     WATCH: (133902432 changed "d:/tmp/x/y/test2")
     WATCH: (159542912 changed "d:/tmp/x/z")

> There is also a problem with single-file watches:
> 
> (let* ((flags '(change attribute-change))
>        (handler (lambda (event)
> 		  (message "WATCH: %S" event)))
>        (x "/tmp/x"))
>   (with-temp-buffer
>     (insert "TEST\n")
>     (write-file x))
>   (file-notify-add-watch x flags handler))
> 
>   1. $ mv /tmp/x /tmp/y
>      WATCH: ((1 . "x") renamed "/tmp/x" "/tmp/y") [2 times]
>   2. $ echo "more" >> /tmp/y
>      ;; no notification
>   3. $ touch /tmp/x
>      WATCH: ((1 . "x") created "/tmp/x")
>      WATCH: ((1 . "x") changed "/tmp/x")

On Windows:

  1. $ mv /tmp/x /tmp/y
     WATCH: (159487136 renamed "d:/tmp/x" "d:/tmp/x")
  2. $ echo "more" >> /tmp/y
     ;; no notification
  3. $ touch /tmp/x
     WATCH: (159487136 created "d:/tmp/x")
     WATCH: (159487136 changed "d:/tmp/x")

> So in the case of single-file watches, the watch seems to stick to a
> filename rather than a file.  Well, I think that's ok but then I would
> expect it to be similar with directories, e.g., in the first example
> step 3 should not report a notification because I intended to watch
> /tmp/x/y and not /tmp/x/z.

I don't see any problems here.  filenotify.el just reports to us what
the notification back-end receives.  It doesn't (and shouldn't, IMO)
include any non-trivial logic to "fix" the back-ends in these tricky
cases, because it has no idea what the application will want to do
with that.  Its role is to present a platform-independent API that
uses one of the available back-ends, that's all.

IOW, technically, the file/directory you watched disappeared from the
filesystem, so what is reported afterwards is IMO entirely resonable
at this low level.

One application we have on top of filenotify.el is auto-revert.el.
Now, if you see some user-level problems there when files are
renamed/deleted, please report them.

> Another thing is that renames always seem to be reported twice (although
> the example in the manual lists only one event but that example has
> probably gathered with the gfilenotify backend).

This doesn't happen with w32notify.  filenotify.el includes some code
to produce a single notification out of the 2 reported by back-ends in
this case; perhaps something doesn't work there with inotify.  Can you
step through the code and see why?

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> IOW, technically, the file/directory you watched disappeared from the
> filesystem, so what is reported afterwards is IMO entirely resonable
> at this low level.

Well, inotify sends then the low-level event `delete-self', which is
different from the usual `delete'. filenotify.el translates both to the
upper-level event `deleted'. In case of `delete-self', filenotify.el
could do something more, like removing the watch. Do we want this?

> One application we have on top of filenotify.el is auto-revert.el.
> Now, if you see some user-level problems there when files are
> renamed/deleted, please report them.
>
>> Another thing is that renames always seem to be reported twice (although
>> the example in the manual lists only one event but that example has
>> probably gathered with the gfilenotify backend).
>
> This doesn't happen with w32notify.  filenotify.el includes some code
> to produce a single notification out of the 2 reported by back-ends in
> this case; perhaps something doesn't work there with inotify.  Can you
> step through the code and see why?

Two days ago (commit dbdc459a48091f5953faf14bcaaa7e6d37fbf024), I've
changed filenotify.el to fire 2 events `renamed' in case the directories
of the source and target are different. This was triggered by a user
report, that he wants to have auto-revert-mode for two different
directories under dired control. So the event is sent for the two
different handlers activated by the respective *-add-watch calls.

I've tested for inotify, it works (confirmed by that user). The use case
of that user was, that a file has been moved outside Emacs from one
directory to the other. An he wanted to see it in both dired buffers,
immediately.

I couldn't test this for w32notify, but it should behave like this since
Sunday. I haven't changed it in gfilenotify yet, it's on my todo
list. And the manual must be adapted, as Tassilo has said.

Best regards, Michael.

PS: Tassilo, you write great test cases. Do you want to contribute to
test/automated/file-notify-tests.el?

PPS: Forgive me slow progress; I'm still suffering from health problems.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: Tassilo Horn <tsdh@gnu.org>,  21435@debbugs.gnu.org
> Date: Tue, 08 Sep 2015 20:04:20 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > IOW, technically, the file/directory you watched disappeared from the
> > filesystem, so what is reported afterwards is IMO entirely resonable
> > at this low level.
> 
> Well, inotify sends then the low-level event `delete-self', which is
> different from the usual `delete'. filenotify.el translates both to the
> upper-level event `deleted'. In case of `delete-self', filenotify.el
> could do something more, like removing the watch. Do we want this?

I don't think filenotify.el should remove the watch on its own, but it
could send an event that would allow the application do that if it
wants to.

> > One application we have on top of filenotify.el is auto-revert.el.
> > Now, if you see some user-level problems there when files are
> > renamed/deleted, please report them.
> >
> >> Another thing is that renames always seem to be reported twice (although
> >> the example in the manual lists only one event but that example has
> >> probably gathered with the gfilenotify backend).
> >
> > This doesn't happen with w32notify.  filenotify.el includes some code
> > to produce a single notification out of the 2 reported by back-ends in
> > this case; perhaps something doesn't work there with inotify.  Can you
> > step through the code and see why?
> 
> Two days ago (commit dbdc459a48091f5953faf14bcaaa7e6d37fbf024), I've
> changed filenotify.el to fire 2 events `renamed' in case the directories
> of the source and target are different. This was triggered by a user
> report, that he wants to have auto-revert-mode for two different
> directories under dired control. So the event is sent for the two
> different handlers activated by the respective *-add-watch calls.

But then Tassilo's "renames always seem to be reported twice" is
inaccurate: this should only happen when a file is moved to another
directory.

> I've tested for inotify, it works (confirmed by that user). The use case
> of that user was, that a file has been moved outside Emacs from one
> directory to the other. An he wanted to see it in both dired buffers,
> immediately.
> 
> I couldn't test this for w32notify, but it should behave like this since
> Sunday.

If you can show some simple test case, I can run it.

> PPS: Forgive me slow progress; I'm still suffering from health problems.

Wish you to recover quickly.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> Two days ago (commit dbdc459a48091f5953faf14bcaaa7e6d37fbf024), I've
>> changed filenotify.el to fire 2 events `renamed' in case the
>> directories of the source and target are different. This was
>> triggered by a user report, that he wants to have auto-revert-mode
>> for two different directories under dired control. So the event is
>> sent for the two different handlers activated by the respective
>> *-add-watch calls.
>
> But then Tassilo's "renames always seem to be reported twice" is
> inaccurate: this should only happen when a file is moved to another
> directory.

See my other mail: the test if the source/target are different were done
by testing if the descriptors are not `eq' but since inotify descriptors
are conses, that test returned nil even though they were structurally
equal.

>> PS: Tassilo, you write great test cases. Do you want to contribute to
>> test/automated/file-notify-tests.el?

Yes, but right now I can't because all these corner cases like the ones
from my last two reports are largely undocumented so I don't know what
to assert.  I myself don't have a strong opinion on the behavior of
file-notify as long as it's consistent across the different
`file-notify--library' possibilities (at least mostly, e.g., I can live
with created/deleted events in place of renamed in case the backend
doesn't support this) and the documentation is a bit more definite.

>> PPS: Forgive me slow progress; I'm still suffering from health
>> problems.
>
> Wish you to recover quickly.

Ditto!

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

>> But then Tassilo's "renames always seem to be reported twice" is
>> inaccurate: this should only happen when a file is moved to another
>> directory.
>
> See my other mail: the test if the source/target are different were done
> by testing if the descriptors are not `eq' but since inotify descriptors
> are conses, that test returned nil even though they were structurally
> equal.

Thanks for fixing this. I always had in mind that
`file-notify--descriptor' returns the same object, and maybe it did even
so in the past, but now it composes the cons itself. So we must test for
`equal', indeed.

>>> PS: Tassilo, you write great test cases. Do you want to contribute to
>>> test/automated/file-notify-tests.el?
>
> Yes, but right now I can't because all these corner cases like the ones
> from my last two reports are largely undocumented so I don't know what
> to assert.  I myself don't have a strong opinion on the behavior of
> file-notify as long as it's consistent across the different
> `file-notify--library' possibilities (at least mostly, e.g., I can live
> with created/deleted events in place of renamed in case the backend
> doesn't support this) and the documentation is a bit more definite.

Even the straigt forward cases are not covered well in
file-notify-tests.el. There is a test whether events arrive, but the
nature of the events is not asserted.

You could try with what you expect, at least for inotify, and Eli and I
could check then for the gfilenotify and w32notify cases. If we have the
same results, fine; if we have different results it would be a good
chance for us to harmonize.

And starting with inotify is also good because this is the library used
on hydra. We do not want to make Glenn sad because of failed tests.

>>> PPS: Forgive me slow progress; I'm still suffering from health
>>> problems.
>>
>> Wish you to recover quickly.
>
> Ditto!

Won't happen, unfortunately. It still takes months.

> Bye,
> Tassilo

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Michael Albinus <michael.albinus@gmx.de> writes:

>>>> PS: Tassilo, you write great test cases. Do you want to contribute
>>>> to test/automated/file-notify-tests.el?
>>
>> Yes, but right now I can't because all these corner cases like the ones
>> from my last two reports are largely undocumented so I don't know what
>> to assert.  I myself don't have a strong opinion on the behavior of
>> file-notify as long as it's consistent across the different
>> `file-notify--library' possibilities (at least mostly, e.g., I can live
>> with created/deleted events in place of renamed in case the backend
>> doesn't support this) and the documentation is a bit more definite.
>
> Even the straigt forward cases are not covered well in
> file-notify-tests.el.  There is a test whether events arrive, but the
> nature of the events is not asserted.
>
> You could try with what you expect, at least for inotify, and Eli and
> I could check then for the gfilenotify and w32notify cases. If we have
> the same results, fine; if we have different results it would be a
> good chance for us to harmonize.

That's right.  I'll try adding some tests whenever I find some time.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Tassilo Horn <tsdh@gnu.org> writes:

>> Even the straigt forward cases are not covered well in
>> file-notify-tests.el.  There is a test whether events arrive, but the
>> nature of the events is not asserted.
>>
>> You could try with what you expect, at least for inotify, and Eli and
>> I could check then for the gfilenotify and w32notify cases. If we
>> have the same results, fine; if we have different results it would be
>> a good chance for us to harmonize.
>
> That's right.  I'll try adding some tests whenever I find some time.

Ok, I gave it a whirl and now the `file-notify--test-event-handler' also
records all events in a new variable `file-notify--test-events' for
later analysis.  `file-notify-test02-events' now uses that feature to
check if the received events are the expected ones in the expected
order.

That already revealed two problems:

  1. Now `file-notify-test02-events-remote' fails because after every
     expected `changed' event an additional `attribute-changed' event is
     received.  This is wrong because when adding the watch, only
     '(change) is given as FLAGS argument, not '(change
     attribute-change).

  2. When I change the watch FLAGS to '(change attribute-change), there
     are still no attribute-changed events received in the local case.

And a question: Will the events read by `file-notify--wait-for-events'
still be processed by the handler function?

And what's the intention of (file-notify--wait-for-events 5
file-notify--test-results)?  The timeout of 5 is reasonable, but the
UNTIL argument here just defines that it waits until the very first of
possibly up to nine yet missing events is awaited here, or do I get
something wrong?

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

Hi Tassilo,

> Ok, I gave it a whirl and now the `file-notify--test-event-handler' also
> records all events in a new variable `file-notify--test-events' for
> later analysis.  `file-notify-test02-events' now uses that feature to
> check if the received events are the expected ones in the expected
> order.
>
> That already revealed two problems:
>
>   1. Now `file-notify-test02-events-remote' fails because after every
>      expected `changed' event an additional `attribute-changed' event is
>      received.  This is wrong because when adding the watch, only
>      '(change) is given as FLAGS argument, not '(change
>      attribute-change).

I'll contact the Tramp maintainer about :-)

>   2. When I change the watch FLAGS to '(change attribute-change), there
>      are still no attribute-changed events received in the local case.

attribute-change happens when you change file permissions, or
modification time w/o changing the file contents. Something like

--8<---------------cut here---------------start------------->8---
(progn
  (require 'filenotify)
  (defalias 'myhandler 'ignore)
  (file-notify-add-watch "/tmp" '(change attribute-change) 'myhandler)
  (trace-function 'file-notify-handle-event)
  (trace-function 'myhandler))

# echo xxx > /tmp/xxx

======================================================================
1 -> (file-notify-handle-event (file-notify (1 (create) "xxx" 0) file-notify-callback))
| 2 -> (myhandler ((1) created "/tmp/xxx"))
| 2 <- myhandler: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (1 (modify) "xxx" 0) file-notify-callback))
| 2 -> (myhandler ((1) changed "/tmp/xxx"))
| 2 <- myhandler: nil
1 <- file-notify-handle-event: nil

# chmod 777 /tmp/xxx

======================================================================
1 -> (file-notify-handle-event (file-notify (1 (attrib) "xxx" 0) file-notify-callback))
| 2 -> (myhandler ((1) attribute-changed "/tmp/xxx"))
| 2 <- myhandler: nil
1 <- file-notify-handle-event: nil

# touch /tmp/xxx

======================================================================
1 -> (file-notify-handle-event (file-notify (1 (attrib) "xxx" 0) file-notify-callback))
| 2 -> (myhandler ((1) attribute-changed "/tmp/xxx"))
| 2 <- myhandler: nil
1 <- file-notify-handle-event: nil
--8<---------------cut here---------------end--------------->8---

> And a question: Will the events read by `file-notify--wait-for-events'
> still be processed by the handler function?

Yes, they should.

> And what's the intention of (file-notify--wait-for-events 5
> file-notify--test-results)?  The timeout of 5 is reasonable, but the
> UNTIL argument here just defines that it waits until the very first of
> possibly up to nine yet missing events is awaited here, or do I get
> something wrong?

Well, this is an open point here. When waiting for events, you don't
know how many events will arrive for a given file operation. See the
first command above, "echo", it is good for 2 events.

So I've taken this heuristics, that after arrival of the first event the
other ones will arrive soon. I simply don't know better ...

gfilenotify could profit from the changes-done-hint event, which is
exactly this kind of information. But we don't have a similar mechanism
for inotify or w32notify, as far as I'm aware.

If you know of a better check for "alle events we wait for have arrived"
- go on.

> Bye,
> Tassilo

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Michael Albinus <michael.albinus@gmx.de> writes:

>> That already revealed two problems:
>>
>>   1. Now `file-notify-test02-events-remote' fails because after every
>>      expected `changed' event an additional `attribute-changed' event is
>>      received.  This is wrong because when adding the watch, only
>>      '(change) is given as FLAGS argument, not '(change
>>      attribute-change).
>
> I'll contact the Tramp maintainer about :-)

Deliver him my best wishes. :-)

>>   2. When I change the watch FLAGS to '(change attribute-change), there
>>      are still no attribute-changed events received in the local case.
>
> attribute-change happens when you change file permissions, or
> modification time w/o changing the file contents. Something like

Ah, ok, so when you write to a file you'll only get a `changed' event,
and not an additional `attribute-changed' event for the changed
modification time.  So basically, attribute changes are subsumed by
`changed' and `created' events.

By the way, I think it could be hard to test `attribute-changed' events
because those probably depend on the filesystem and mount options on the
machine where the tests are run, e.g., if access time recording is
enabled or not.

>> And a question: Will the events read by
>> `file-notify--wait-for-events' still be processed by the handler
>> function?
>
> Yes, they should.

Ok, good.

>> And what's the intention of (file-notify--wait-for-events 5
>> file-notify--test-results)?  The timeout of 5 is reasonable, but the
>> UNTIL argument here just defines that it waits until the very first
>> of possibly up to nine yet missing events is awaited here, or do I
>> get something wrong?
>
> Well, this is an open point here.  When waiting for events, you don't
> know how many events will arrive for a given file operation.  See the
> first command above, "echo", it is good for 2 events.

In the tests we know that our test files do not exist initially, so I
expect the first write to a file to result in a `created' followed by an
`changed' event.  And for other operations I have similar expectations,
so in the test02 I know that exactly nine events should be received
(unless my expectations are wrong).

> If you know of a better check for "alle events we wait for have
> arrived" - go on.

In the concrete case where I know it should be nine events, I could use
(= 9 (length file-notify--test-events)) as the UNTIL argument.

I've added a new macro to the tests now which lets you do things this
way:

--8<---------------cut here---------------start------------->8---
        ;; Check creation, change, and deletion.
        (file-notify--test-with-events
            3 3 (lambda (events)
                  (should (equal '(created changed deleted)
                                 (mapcar #'cadr events))))
          (write-region
           "any text" nil file-notify--test-tmpfile nil 'no-message)
          (delete-file file-notify--test-tmpfile))
--8<---------------cut here---------------end--------------->8---

This means we're waiting for 3 events for at most 3 seconds, and then
apply the lambda to the received events.  The rest is the code which
causes the events to be emitted.

Another thing: the remote tests, especially the test03-autorevert one,
take really, really long (maybe 30 seconds).  I saw that this uses some
mock TRAMP method which suggests it is a mockup connection which can
probably simulate a fast or a slow connection.  If so, I'd prefer to
have a reasonably fast one so that I don't try to avoid running all
tests frequently.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

>>>   1. Now `file-notify-test02-events-remote' fails because after every
>>>      expected `changed' event an additional `attribute-changed' event is
>>>      received.  This is wrong because when adding the watch, only
>>>      '(change) is given as FLAGS argument, not '(change
>>>      attribute-change).
>>
>> I'll contact the Tramp maintainer about :-)
>
> Deliver him my best wishes. :-)

Done. He's working on the problem.

> Ah, ok, so when you write to a file you'll only get a `changed' event,
> and not an additional `attribute-changed' event for the changed
> modification time.  So basically, attribute changes are subsumed by
> `changed' and `created' events.
>
> By the way, I think it could be hard to test `attribute-changed' events
> because those probably depend on the filesystem and mount options on the
> machine where the tests are run, e.g., if access time recording is
> enabled or not.

Yes. And I also expect here the most notable differences between the
native libraries. Not checked yet, 'tho.

> I've added a new macro to the tests now which lets you do things this
> way:
>
>         ;; Check creation, change, and deletion.
>         (file-notify--test-with-events
>             3 3 (lambda (events)
>                   (should (equal '(created changed deleted)
>                                  (mapcar #'cadr events))))
>           (write-region
>            "any text" nil file-notify--test-tmpfile nil 'no-message)
>           (delete-file file-notify--test-tmpfile))
>
> This means we're waiting for 3 events for at most 3 seconds, and then
> apply the lambda to the received events.  The rest is the code which
> causes the events to be emitted.

That's good, yes. Please add a docstring to that macro (boring, I know,
but it will help other people like Eli and me).

One point: you don't call any longer `read-event'. I believe it still
makes sense; IIRC I did it because sometimes you must trigger Emacs to
check all its file descriptors for new events.

> Another thing: the remote tests, especially the test03-autorevert one,
> take really, really long (maybe 30 seconds).  I saw that this uses some
> mock TRAMP method which suggests it is a mockup connection which can
> probably simulate a fast or a slow connection.  If so, I'd prefer to
> have a reasonably fast one so that I don't try to avoid running all
> tests frequently.

Well, the mockup method is a silly one and it even doesn't require a
remote connection, so it shall be faster than any other Tramp method.
And I do not experience serious delays when testing.

You could alway skip all the remote tests. Call

# env REMOTE_TEMPORARY_FILE_DIRECTORY=/dev/null make -C test/automated file-notify-tests

> Bye,
> Tassilo

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Michael Albinus <michael.albinus@gmx.de> writes:

>> I've added a new macro to the tests now which lets you do things this
>> way:
>>
>>         ;; Check creation, change, and deletion.
>>         (file-notify--test-with-events
>>             3 3 (lambda (events)
>>                   (should (equal '(created changed deleted)
>>                                  (mapcar #'cadr events))))
>>           (write-region
>>            "any text" nil file-notify--test-tmpfile nil 'no-message)
>>           (delete-file file-notify--test-tmpfile))
>>
>> This means we're waiting for 3 events for at most 3 seconds, and then
>> apply the lambda to the received events.  The rest is the code which
>> causes the events to be emitted.
>
> That's good, yes. Please add a docstring to that macro (boring, I
> know, but it will help other people like Eli and me).

Done.

> One point: you don't call any longer `read-event'.  I believe it still
> makes sense; IIRC I did it because sometimes you must trigger Emacs to
> check all its file descriptors for new events.

I still do.  That macro uses `file-notify--wait-for-events' to wait
until all N events have arrived before calling the ASSERT-FN.

>> Another thing: the remote tests, especially the test03-autorevert
>> one, take really, really long (maybe 30 seconds).  I saw that this
>> uses some mock TRAMP method which suggests it is a mockup connection
>> which can probably simulate a fast or a slow connection.  If so, I'd
>> prefer to have a reasonably fast one so that I don't try to avoid
>> running all tests frequently.
>
> Well, the mockup method is a silly one and it even doesn't require a
> remote connection, so it shall be faster than any other Tramp method.
> And I do not experience serious delays when testing.

Hm, then I wonder what causes these extreme delays here.  The remote
autorevert test really takes at least 20 seconds here.  But I have no
issues using TRAMP with its ssh method for accessing really remote
files...

> You could alway skip all the remote tests.

As your tester, I obviously want to run all tests. :-)

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

>>> I've added a new macro to the tests now which lets you do things this
>>> way:
>>>
>>>         ;; Check creation, change, and deletion.
>>>         (file-notify--test-with-events
>>>             3 3 (lambda (events)
>>>                   (should (equal '(created changed deleted)
>>>                                  (mapcar #'cadr events))))
>>>           (write-region
>>>            "any text" nil file-notify--test-tmpfile nil 'no-message)
>>>           (delete-file file-notify--test-tmpfile))
>>>
>>> This means we're waiting for 3 events for at most 3 seconds, and then
>>> apply the lambda to the received events.  The rest is the code which
>>> causes the events to be emitted.
>>
>> That's good, yes. Please add a docstring to that macro (boring, I
>> know, but it will help other people like Eli and me).
>
> Done.

Thanks. Another point is whether we always can expect that all native
libraries will report exactly the same number of events, and then
whether they report them exactly in the same order (you expect '(created
changed deleted) here). Let's see how the test cases evolve, maybe we
need more tolerant tests.

>> One point: you don't call any longer `read-event'.  I believe it still
>> makes sense; IIRC I did it because sometimes you must trigger Emacs to
>> check all its file descriptors for new events.
>
> I still do.  That macro uses `file-notify--wait-for-events' to wait
> until all N events have arrived before calling the ASSERT-FN.

Yep, I've overseen this. OK.

> Hm, then I wonder what causes these extreme delays here.  The remote
> autorevert test really takes at least 20 seconds here.  But I have no
> issues using TRAMP with its ssh method for accessing really remote
> files...

Now I see it also, but only when running in interactive mode. In batch
mode, there is no serious delay. Will contact the Tramp maintainer, again.

> Bye,
> Tassilo

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Michael Albinus <michael.albinus@gmx.de> writes:

> Thanks. Another point is whether we always can expect that all native
> libraries will report exactly the same number of events, and then
> whether they report them exactly in the same order (you expect
> '(created changed deleted) here). Let's see how the test cases evolve,
> maybe we need more tolerant tests.

Yes, I'm quite strict right now and IMHO in the current tests, that's
the only sensible order of events.  But of course if it turns out that
with some notification backend we can't achieve that guarantee, it'll be
no problem to relax the assertions a bit.

>> Hm, then I wonder what causes these extreme delays here.  The remote
>> autorevert test really takes at least 20 seconds here.  But I have no
>> issues using TRAMP with its ssh method for accessing really remote
>> files...
>
> Now I see it also, but only when running in interactive mode. In batch
> mode, there is no serious delay. Will contact the Tramp maintainer, again.

Yes, you are right.  I always executed the test interactively and now
tried running them from the command line.  Indeed, that's way faster.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  21435@debbugs.gnu.org
> Date: Fri, 11 Sep 2015 14:17:03 +0200
> 
> Michael Albinus <michael.albinus@gmx.de> writes:
> 
> > Thanks. Another point is whether we always can expect that all native
> > libraries will report exactly the same number of events, and then
> > whether they report them exactly in the same order (you expect
> > '(created changed deleted) here). Let's see how the test cases evolve,
> > maybe we need more tolerant tests.
> 
> Yes, I'm quite strict right now and IMHO in the current tests, that's
> the only sensible order of events.  But of course if it turns out that
> with some notification backend we can't achieve that guarantee, it'll be
> no problem to relax the assertions a bit.

I think if we ever find that the order is not guaranteed, we should
sort the events into the "right" order in filenotify.el.

Once again, the sole reason for having filenotify.el is to present a
unified API to applications, so that they don't need to know anything
about the back-ends and their idiosyncrasies.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

>>> Hm, then I wonder what causes these extreme delays here.  The remote
>>> autorevert test really takes at least 20 seconds here.  But I have no
>>> issues using TRAMP with its ssh method for accessing really remote
>>> files...
>>
>> Now I see it also, but only when running in interactive mode. In batch
>> mode, there is no serious delay. Will contact the Tramp maintainer, again.
>
> Yes, you are right.  I always executed the test interactively and now
> tried running them from the command line.  Indeed, that's way faster.

Should be fixed now. The problem was due to the curly quotes in the
*Messages* buffer; something I've never understood why it is needed.

> Bye,
> Tassilo

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

> Tassilo Horn <tsdh@gnu.org> writes:
>
>>>>   1. Now `file-notify-test02-events-remote' fails because after every
>>>>      expected `changed' event an additional `attribute-changed' event is
>>>>      received.  This is wrong because when adding the watch, only
>>>>      '(change) is given as FLAGS argument, not '(change
>>>>      attribute-change).
>>>
>>> I'll contact the Tramp maintainer about :-)
>>
>> Deliver him my best wishes. :-)
>
> Done. He's working on the problem.

Should be fixed now.

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Michael Albinus <michael.albinus@gmx.de> writes:

>>>>>   1. Now `file-notify-test02-events-remote' fails because after every
>>>>>      expected `changed' event an additional `attribute-changed' event is
>>>>>      received.  This is wrong because when adding the watch, only
>>>>>      '(change) is given as FLAGS argument, not '(change
>>>>>      attribute-change).
>>>>
>>>> I'll contact the Tramp maintainer about :-)
>>>
>>> Deliver him my best wishes. :-)
>>
>> Done. He's working on the problem.
>
> Should be fixed now.

Yes, it is.  Thanks!

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> Well, inotify sends then the low-level event `delete-self', which is
>> different from the usual `delete'. filenotify.el translates both to the
>> upper-level event `deleted'. In case of `delete-self', filenotify.el
>> could do something more, like removing the watch. Do we want this?
>
> I don't think filenotify.el should remove the watch on its own, but it
> could send an event that would allow the application do that if it
> wants to.

I will think about. And check also what's possible with gfilenotify.

>> I've tested for inotify, it works (confirmed by that user). The use case
>> of that user was, that a file has been moved outside Emacs from one
>> directory to the other. An he wanted to see it in both dired buffers,
>> immediately.
>> 
>> I couldn't test this for w32notify, but it should behave like this since
>> Sunday.
>
> If you can show some simple test case, I can run it.

Well, something like this:

--8<---------------cut here---------------start------------->8---
(progn
  (require 'filenotify)
  (defalias 'myhandler1 'ignore)
  (defalias 'myhandler2 'ignore)
  (file-notify-add-watch "/tmp" '(change) 'myhandler1)
  (file-notify-add-watch "~/tmp" '(change) 'myhandler2)
  (trace-function 'file-notify-handle-event)
  (trace-function 'myhandler1)
  (trace-function 'myhandler2))
--8<---------------cut here---------------end--------------->8---

Then you do outside Emacs (inotify case):

--8<---------------cut here---------------start------------->8---
# echo xxx >/tmp/xxx

======================================================================
1 -> (file-notify-handle-event (file-notify (1 (create) "xxx" 0) file-notify-callback))
| 2 -> (myhandler1 ((1) created "/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (1 (modify) "xxx" 0) file-notify-callback))
| 2 -> (myhandler1 ((1) changed "/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil

# mv /tmp/xxx ~/tmp/

======================================================================
1 -> (file-notify-handle-event (file-notify (1 (moved-from) "xxx" 49278) file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (2 (moved-to) "xxx" 49278) file-notify-callback))
| 2 -> (myhandler1 ((1) renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
| 2 <- myhandler1: nil
| 2 -> (myhandler2 ((2) renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
| 2 <- myhandler2: nil
1 <- file-notify-handle-event: nil
--8<---------------cut here---------------end--------------->8---

That looks good. The same scenario for gfilenotify needs some improvements:

--8<---------------cut here---------------start------------->8---
# echo xxx >/tmp/xxx

======================================================================
1 -> (file-notify-handle-event (file-notify (3924112 created "/tmp/xxx") file-notify-callback))
| 2 -> (myhandler1 (3924112 created "/tmp/xxx"))
| 2 <- myhandler1: nil
| 2 -> (myhandler1 (3924112 created "/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (3924112 changed "/tmp/xxx") file-notify-callback))
| 2 -> (myhandler1 (3924112 changed "/tmp/xxx"))
| 2 <- myhandler1: nil
| 2 -> (myhandler1 (3924112 changed "/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (3924112 changes-done-hint "/tmp/xxx") file-notify-callback))
1 <- file-notify-handle-event: nil

# mv /tmp/xxx ~/tmp/

======================================================================
1 -> (file-notify-handle-event (file-notify (3924112 moved "/tmp/xxx" "/home/albinus/tmp/xxx") file-notify-callback))
| 2 -> (myhandler1 (3924112 renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
| 2 <- myhandler1: nil
| 2 -> (myhandler1 (3924112 renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil
--8<---------------cut here---------------end--------------->8---

myhandler1 is always called twice (why?). And in the rename case, the
call of myhandler2 is missing.

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Wed, 09 Sep 2015 20:41:55 +0200
> 
> >> I couldn't test this for w32notify, but it should behave like this since
> >> Sunday.
> >
> > If you can show some simple test case, I can run it.
> 
> Well, something like this:
> 
> --8<---------------cut here---------------start------------->8---
> (progn
>   (require 'filenotify)
>   (defalias 'myhandler1 'ignore)
>   (defalias 'myhandler2 'ignore)
>   (file-notify-add-watch "/tmp" '(change) 'myhandler1)
>   (file-notify-add-watch "~/tmp" '(change) 'myhandler2)
>   (trace-function 'file-notify-handle-event)
>   (trace-function 'myhandler1)
>   (trace-function 'myhandler2))
> --8<---------------cut here---------------end--------------->8---
> 
> Then you do outside Emacs (inotify case):
> 
> --8<---------------cut here---------------start------------->8---
> # echo xxx >/tmp/xxx
> 
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (1 (create) "xxx" 0) file-notify-callback))
> | 2 -> (myhandler1 ((1) created "/tmp/xxx"))
> | 2 <- myhandler1: nil
> 1 <- file-notify-handle-event: nil
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (1 (modify) "xxx" 0) file-notify-callback))
> | 2 -> (myhandler1 ((1) changed "/tmp/xxx"))
> | 2 <- myhandler1: nil
> 1 <- file-notify-handle-event: nil

This behaves the same on w32:

======================================================================
1 -> (file-notify-handle-event (file-notify (100286560 added "xxx") file-notify-callback))
| 2 -> (myhandler1 (100286560 created "d:/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100286560 modified "xxx") file-notify-callback))
| 2 -> (myhandler1 (100286560 changed "d:/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil

> # mv /tmp/xxx ~/tmp/
> 
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (1 (moved-from) "xxx" 49278) file-notify-callback))
> 1 <- file-notify-handle-event: nil
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (2 (moved-to) "xxx" 49278) file-notify-callback))
> | 2 -> (myhandler1 ((1) renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
> | 2 <- myhandler1: nil
> | 2 -> (myhandler2 ((2) renamed "/tmp/xxx" "/home/albinus/tmp/xxx"))
> | 2 <- myhandler2: nil
> 1 <- file-notify-handle-event: nil
> --8<---------------cut here---------------end--------------->8---
> 
> That looks good.

This doesn't work at all on w32 (I used d:/usr/eli/data instead of
~/tmp), it reports removal and addition (and also a bogus 2nd
removal):

======================================================================
1 -> (file-notify-handle-event (file-notify (100286608 removed "xxx") file-notify-callback))
| 2 -> (myhandler2 (100286608 deleted "d:/usr/eli/data/xxx"))
| 2 <- myhandler2: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100286608 added "xxx") file-notify-callback))
| 2 -> (myhandler2 (100286608 created "d:/usr/eli/data/xxx"))
| 2 <- myhandler2: nil
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100286560 removed "xxx") file-notify-callback))
| 2 -> (myhandler1 (100286560 deleted "d:/tmp/xxx"))
| 2 <- myhandler1: nil
1 <- file-notify-handle-event: nil

Let me know what I should look into or which additional information I
can give you about this.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> # mv /tmp/xxx ~/tmp/
>
> This doesn't work at all on w32 (I used d:/usr/eli/data instead of
> ~/tmp), it reports removal and addition (and also a bogus 2nd
> removal):
>
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (100286608 removed "xxx")
> file-notify-callback))
> | 2 -> (myhandler2 (100286608 deleted "d:/usr/eli/data/xxx"))
> | 2 <- myhandler2: nil
> 1 <- file-notify-handle-event: nil
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (100286608 added "xxx")
> file-notify-callback))
> | 2 -> (myhandler2 (100286608 created "d:/usr/eli/data/xxx"))
> | 2 <- myhandler2: nil
> 1 <- file-notify-handle-event: nil
> ======================================================================
> 1 -> (file-notify-handle-event (file-notify (100286560 removed "xxx")
> file-notify-callback))
> | 2 -> (myhandler1 (100286560 deleted "d:/tmp/xxx"))
> | 2 <- myhandler1: nil
> 1 <- file-notify-handle-event: nil
>
> Let me know what I should look into or which additional information I
> can give you about this.

`file-notify-handle-event' is called directly by the low-level library,
w32notify here. It sends the events

(file-notify (100286608 removed "xxx"))
(file-notify (100286608 added "xxx"))
(file-notify (100286560 removed "xxx"))

The first two events are raised by the file name handler for d:/usr/eli/data.
Maybe "xxx" did exist already in that directory? In this case, it would be
OK to remove it first.

The last event comes from from file name handler for d:/tmp - this looks
OK. Well, the order of the events is not as expected (the third event
shall be the first one), but we never gave a promise for a canonical order.

I would say, that w32notify does not send the renamed-from and
renamed-to events, as expected. Maybe they are sent only in case of
renaming a file in the same directory? Say

# mv /tmp/xxx /tmp/yyy

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Thu, 10 Sep 2015 13:09:36 +0200
> 
> file-notify-handle-event' is called directly by the low-level library,
> w32notify here. It sends the events
> 
> (file-notify (100286608 removed "xxx"))
> (file-notify (100286608 added "xxx"))
> (file-notify (100286560 removed "xxx"))

Yes.

> The first two events are raised by the file name handler for d:/usr/eli/data.
> Maybe "xxx" did exist already in that directory?

No, it didn't.  I don't know why w32 sends this strange notification,
but the fact is it does.  (That doesn't happen when moving a
directory, btw, only when moving files.)

> The last event comes from from file name handler for d:/tmp - this looks
> OK. Well, the order of the events is not as expected (the third event
> shall be the first one), but we never gave a promise for a canonical order.
> 
> I would say, that w32notify does not send the renamed-from and
> renamed-to events, as expected. Maybe they are sent only in case of
> renaming a file in the same directory?

Yes, that's exactly what happens.  Which IMO is entirely reasonable,
since each watch watches only a single directory.  That inotify has
some kind of "global" perspective on such rename events is a bonus,
but we cannot expect that.

This, of course, breaks the basic assumption of the design intended to
provide this feature:

> Two days ago (commit dbdc459a48091f5953faf14bcaaa7e6d37fbf024), I've
> changed filenotify.el to fire 2 events `renamed' in case the directories
> of the source and target are different. This was triggered by a user
> report, that he wants to have auto-revert-mode for two different
> directories under dired control. So the event is sent for the two
> different handlers activated by the respective *-add-watch calls.

The design expects 2 'move'/'renamed' events, but that's not
guaranteed, and doesn't happen on w32.  If we want to conflate
'removed' followed by 'added' into a rename across directories, we
will need changes in filenotify.el, and will risk false positives,
because it could really be a deletion followed by a creation of a file
by the same name.

However, if all we want is to make sure the destination directory gets
a notification (so it could auto-revert), then this already happens on
MS-Windows (see the 'created' event above), and therefore nothing
should be done on Windows to support the user request above.

Therefore, I submit that a better solution would be to make inotify
emulate what w32notify does, i.e. produce a synthetic 'added' event in
the destination directory when we get a 'moved-to' event that
specifies a destination directory different from the source.

Finally, two more comments about this:

 . I wish such changes were discussed, and the various alternatives
   examined, before the code is changed

 . I'm not sure this kind of non-trivial logic is something that
   belongs to filenotify.el; it could well have a better place in
   auto-revert.el instead, as that is the level where the logic is
   needed and understood, or even in the Dired-specific function that
   auto-reverts a directory

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> (file-notify (100286608 removed "xxx"))
>> (file-notify (100286608 added "xxx"))
>> (file-notify (100286560 removed "xxx"))
>>
>> The first two events are raised by the file name handler for d:/usr/eli/data.
>> Maybe "xxx" did exist already in that directory?
>
> No, it didn't.  I don't know why w32 sends this strange notification,
> but the fact is it does.  (That doesn't happen when moving a
> directory, btw, only when moving files.)

Hmm, maybe a bug in the MS Windows library?.

However, likely we have no practical problem. The first `removed' event
reports the removal of a non-existing file. Nothing a handler shall be
concerned about (except the test case Tassilo writes for this ...)

>> I would say, that w32notify does not send the renamed-from and
>> renamed-to events, as expected. Maybe they are sent only in case of
>> renaming a file in the same directory?
>
> Yes, that's exactly what happens.  Which IMO is entirely reasonable,
> since each watch watches only a single directory.  That inotify has
> some kind of "global" perspective on such rename events is a bonus,
> but we cannot expect that.

Of course. inotify does this by adding an additional cookie to the
`moved-from' and `moved-to' events. A pair of such events with the same
cookie belongs together. See the example I have shown:

1 -> (file-notify-handle-event (file-notify (1 (moved-from) "xxx" 49278) file-notify-callback))
1 -> (file-notify-handle-event (file-notify (2 (moved-to) "xxx" 49278) file-notify-callback))

> This, of course, breaks the basic assumption of the design intended to
> provide this feature:
>
>> Two days ago (commit dbdc459a48091f5953faf14bcaaa7e6d37fbf024), I've
>> changed filenotify.el to fire 2 events `renamed' in case the directories
>> of the source and target are different. This was triggered by a user
>> report, that he wants to have auto-revert-mode for two different
>> directories under dired control. So the event is sent for the two
>> different handlers activated by the respective *-add-watch calls.
>
> The design expects 2 'move'/'renamed' events, but that's not
> guaranteed, and doesn't happen on w32.

There's no guarantee. We must say that there will be `renamed' events
only when possible. And if not, one must expect two events `deleted' and
`created', handled by the respective handler.

> If we want to conflate 'removed' followed by 'added' into a rename
> across directories, we will need changes in filenotify.el, and will
> risk false positives, because it could really be a deletion followed
> by a creation of a file by the same name.

We will conflate them only when possible. inotify helps us by setting a
proper cookie. If w32notify cannot provide this, then we will deliver
`deleted' and `created'.

> However, if all we want is to make sure the destination directory gets
> a notification (so it could auto-revert), then this already happens on
> MS-Windows (see the 'created' event above), and therefore nothing
> should be done on Windows to support the user request above.

It's not only the destination directory, it's also the source directory
which matters. Remember the initial use case, two dired buffers under
`auto-revert-mode' control. The moved file must appear in the
destination dired buffer, and it must disappear in the source dired buffer.

> Therefore, I submit that a better solution would be to make inotify
> emulate what w32notify does, i.e. produce a synthetic 'added' event in
> the destination directory when we get a 'moved-to' event that
> specifies a destination directory different from the source.

Not necessary I believe. Due to inotify cookie mechanism, it will work
robustly. And don't forget gfilenotify, which sends a `rename' event
already on low-level.

> Finally, two more comments about this:

>  . I wish such changes were discussed, and the various alternatives
>    examined, before the code is changed

You are right, as always. I didn't expect that there would be such
diasagreement.

OTOH, I have encouraged Tassilo to improve file-notify-tests.el just in
order to have a common base of understanding. The tested (and suceeded)
behaviour will be what low-level libraries and filenotify.el have agreed.

>  . I'm not sure this kind of non-trivial logic is something that
>    belongs to filenotify.el; it could well have a better place in
>    auto-revert.el instead, as that is the level where the logic is
>    needed and understood, or even in the Dired-specific function that
>    auto-reverts a directory

If we only deliver `removed' and `created' events, none of those
libraries would have a chance to pair them to a rename action. Essential
information, like inotify cookies, will be lost.

And yes, this information will be needed. Recently, I saw a discussion
on sx, whether Emacs' `auto.revert-mode' could also support file
renaming. That is, when a buffer is associated by a file, and that file
is renamed outside Emacs, Emacs shall rename `buffer-name' and
`buffer-file-name', and then revert. Nice idea ...

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Thu, 10 Sep 2015 19:37:26 +0200
> 
> > However, if all we want is to make sure the destination directory gets
> > a notification (so it could auto-revert), then this already happens on
> > MS-Windows (see the 'created' event above), and therefore nothing
> > should be done on Windows to support the user request above.
> 
> It's not only the destination directory, it's also the source directory
> which matters. Remember the initial use case, two dired buffers under
> `auto-revert-mode' control. The moved file must appear in the
> destination dired buffer, and it must disappear in the source dired buffer.

Yes, and both events happen in this scenario, on Windows as well as on
GNU/Linux.  So the fact that the move is not reported as a move will
not cause any problems in this use case.

> > Therefore, I submit that a better solution would be to make inotify
> > emulate what w32notify does, i.e. produce a synthetic 'added' event in
> > the destination directory when we get a 'moved-to' event that
> > specifies a destination directory different from the source.
> 
> Not necessary I believe. Due to inotify cookie mechanism, it will work
> robustly. And don't forget gfilenotify, which sends a `rename' event
> already on low-level.

I'm guessing that gfilenotify only does that when its back-end is
inotify.  E.g., I doubt that it could do this when it uses its
fallback polling method.

> >  . I'm not sure this kind of non-trivial logic is something that
> >    belongs to filenotify.el; it could well have a better place in
> >    auto-revert.el instead, as that is the level where the logic is
> >    needed and understood, or even in the Dired-specific function that
> >    auto-reverts a directory
> 
> If we only deliver `removed' and `created' events, none of those
> libraries would have a chance to pair them to a rename action.

They shouldn't rely on that in the first place, since this is
unreliable, as we just saw.

And in the case in point, it's unnecessary anyway, since all you need
is to have events in both source and destination.  These events do not
have to be 'rename' events.

> Essential information, like inotify cookies, will be lost.

On filenotify.el level, yes.  I thought filenotify.el exists to try to
present a more or less unified interface independent of the back-end.
If such differences in back-end behavior are seen by clients of
filenotify.el, then how is it different from invoking the back-end
directly?

> And yes, this information will be needed. Recently, I saw a discussion
> on sx, whether Emacs' `auto.revert-mode' could also support file
> renaming. That is, when a buffer is associated by a file, and that file
> is renamed outside Emacs, Emacs shall rename `buffer-name' and
> `buffer-file-name', and then revert. Nice idea ...

The problem we are discussing does not exist in this scenario, AFAIU.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> > However, if all we want is to make sure the destination directory gets
>> > a notification (so it could auto-revert), then this already happens on
>> > MS-Windows (see the 'created' event above), and therefore nothing
>> > should be done on Windows to support the user request above.
>> 
>> It's not only the destination directory, it's also the source directory
>> which matters. Remember the initial use case, two dired buffers under
>> `auto-revert-mode' control. The moved file must appear in the
>> destination dired buffer, and it must disappear in the source dired buffer.
>
> Yes, and both events happen in this scenario, on Windows as well as on
> GNU/Linux.  So the fact that the move is not reported as a move will
> not cause any problems in this use case.

For that dired case, yes. I was answering to your statement "if all we
want is to make sure the destination directory ..."

>> Not necessary I believe. Due to inotify cookie mechanism, it will work
>> robustly. And don't forget gfilenotify, which sends a `rename' event
>> already on low-level.
>
> I'm guessing that gfilenotify only does that when its back-end is
> inotify.  E.g., I doubt that it could do this when it uses its
> fallback polling method.

Maybe, I haven't checked. But the point is that we send a `renamed'
event only we can trust there is a file move. Otherwise, we send
`deleted' and `created'.

>> >  . I'm not sure this kind of non-trivial logic is something that
>> >    belongs to filenotify.el; it could well have a better place in
>> >    auto-revert.el instead, as that is the level where the logic is
>> >    needed and understood, or even in the Dired-specific function that
>> >    auto-reverts a directory
>> 
>> If we only deliver `removed' and `created' events, none of those
>> libraries would have a chance to pair them to a rename action.
>
> They shouldn't rely on that in the first place, since this is
> unreliable, as we just saw.

Nope. When filenotify.el sends a `renamed' event, it must be
reliable. Because we got it from gfilenotify, because we could pair the
events via inotify cookies, whatever. When it is not reliable, we send
`deleted' and `created', which has less semantics than `renamed'.

> And in the case in point, it's unnecessary anyway, since all you need
> is to have events in both source and destination.  These events do not
> have to be 'rename' events.

In the use cases we know today, you are right. But there might be other
use cases where it matters. And again, `renamed' events provide more
information than single `deleted' and `created' events.

>> Essential information, like inotify cookies, will be lost.
>
> On filenotify.el level, yes.  I thought filenotify.el exists to try to
> present a more or less unified interface independent of the back-end.
> If such differences in back-end behavior are seen by clients of
> filenotify.el, then how is it different from invoking the back-end
> directly?

There is already a difference: native gfilenotify gives us a `rename'
event. Shall we convert it to `deleted' and `created'? This would reduce
the information.

>> And yes, this information will be needed. Recently, I saw a discussion
>> on sx, whether Emacs' `auto.revert-mode' could also support file
>> renaming. That is, when a buffer is associated by a file, and that file
>> is renamed outside Emacs, Emacs shall rename `buffer-name' and
>> `buffer-file-name', and then revert. Nice idea ...
>
> The problem we are discussing does not exist in this scenario, AFAIU.

That scenario would work only if there is a `renamed' event. How else
autorevert could decide, that a file has been moved? What is its new name?

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Thu, 10 Sep 2015 20:20:54 +0200
> 
> >> >  . I'm not sure this kind of non-trivial logic is something that
> >> >    belongs to filenotify.el; it could well have a better place in
> >> >    auto-revert.el instead, as that is the level where the logic is
> >> >    needed and understood, or even in the Dired-specific function that
> >> >    auto-reverts a directory
> >> 
> >> If we only deliver `removed' and `created' events, none of those
> >> libraries would have a chance to pair them to a rename action.
> >
> > They shouldn't rely on that in the first place, since this is
> > unreliable, as we just saw.
> 
> Nope. When filenotify.el sends a `renamed' event, it must be
> reliable.

That's not what I meant.  What I meant was that applications cannot
rely on getting a 'renamed' event when files are moved between
directories, they should be prepared to deal with 'deleted' and
'created' anyway.

> > And in the case in point, it's unnecessary anyway, since all you need
> > is to have events in both source and destination.  These events do not
> > have to be 'rename' events.
> 
> In the use cases we know today, you are right. But there might be other
> use cases where it matters. And again, `renamed' events provide more
> information than single `deleted' and `created' events.

Those use cases will have to handle 'deleted' and 'created', if they
want to work on all supported platforms.  By sending sometimes
'renamed' and sometimes 'deleted' followed by 'created', we ask the
users to do more work, and gratuitously expose them to platform
differences that filenotify.el was supposed to conceal.

> >> Essential information, like inotify cookies, will be lost.
> >
> > On filenotify.el level, yes.  I thought filenotify.el exists to try to
> > present a more or less unified interface independent of the back-end.
> > If such differences in back-end behavior are seen by clients of
> > filenotify.el, then how is it different from invoking the back-end
> > directly?
> 
> There is already a difference: native gfilenotify gives us a `rename'
> event. Shall we convert it to `deleted' and `created'?

Yes.

> This would reduce the information.

Applications that don't want to lose that information can always call
back-ends directly.  By using filenotify.el, they agree to losing some
information, and in return gain uniformity and less coding.

> >> And yes, this information will be needed. Recently, I saw a discussion
> >> on sx, whether Emacs' `auto.revert-mode' could also support file
> >> renaming. That is, when a buffer is associated by a file, and that file
> >> is renamed outside Emacs, Emacs shall rename `buffer-name' and
> >> `buffer-file-name', and then revert. Nice idea ...
> >
> > The problem we are discussing does not exist in this scenario, AFAIU.
> 
> That scenario would work only if there is a `renamed' event. How else
> autorevert could decide, that a file has been moved? What is its new name?

In the same directory, there _is_ a 'renamed' event.  We are
discussing a situation when a file was moved to another directory,
which is not what this scenario is about.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> > They shouldn't rely on that in the first place, since this is
>> > unreliable, as we just saw.
>> 
>> Nope. When filenotify.el sends a `renamed' event, it must be
>> reliable.
>
> That's not what I meant.  What I meant was that applications cannot
> rely on getting a 'renamed' event when files are moved between
> directories, they should be prepared to deal with 'deleted' and
> 'created' anyway.

Ahh, I misunderstood you. And yes, I don't believe it is a burden for
applications to react on `renamed', `deleted' and `created'.

> Those use cases will have to handle 'deleted' and 'created', if they
> want to work on all supported platforms.  By sending sometimes
> 'renamed' and sometimes 'deleted' followed by 'created', we ask the
> users to do more work, and gratuitously expose them to platform
> differences that filenotify.el was supposed to conceal.

I believe we don't have platform differences. On all supported
platforms, we could compose a `renamed' event. gfilenotify sends
`rename', inotify sends the `moved-from' / `moved-to' pair, and
w32notify sends the `renamed-from' / `renamed-to' pair.

None of the platforms guarantees, that a move operation will result in
those events, remember the case of moving a file from a local disk to a
mounted disk (a share for MS-Windows). Then they send their `deleted' /
`created' equivalents.

What we do is harmonizing a little bit the different native
libraries. We don't do anything else.

>> There is already a difference: native gfilenotify gives us a `rename'
>> event. Shall we convert it to `deleted' and `created'?
>
> Yes.

Here I disagree.

>> This would reduce the information.
>
> Applications that don't want to lose that information can always call
> back-ends directly.  By using filenotify.el, they agree to losing some
> information, and in return gain uniformity and less coding.

See the example above. It would be a horror for them to handle all the
different low-level events. filenotify.el would simplify it. And again,
I don't believe it is too much work for an application, to handle a
`renamed' event, when they are already capable to handle `deleted' and
`created' events.

> In the same directory, there _is_ a 'renamed' event.  We are
> discussing a situation when a file was moved to another directory,
> which is not what this scenario is about.

A handler of an application always knows which directory it is
watching. Where's the problem?

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> Another thing is that renames always seem to be reported twice
>> (although the example in the manual lists only one event but that
>> example has probably gathered with the gfilenotify backend).
>
> This doesn't happen with w32notify.  filenotify.el includes some code
> to produce a single notification out of the 2 reported by back-ends in
> this case; perhaps something doesn't work there with inotify.  Can you
> step through the code and see why?

I stepped through inotify_callback in inotify.c which can read 64 byte
from inotifyfd where the size of one inotify event is 32 and that
produces 2 emacs events (although I cannot see which events are
created).  The Locals GUD frame always shows just nil for the `event'
local variable even after the assignments to it and I stepped into the
then-branch of an `if (!NILP(event.arg))' which makes it pretty clear
that it cannot be nil.  Why is that?

Then I turned to `file-notify-callback'.  That receives one `move-from'
event and one `move-to' event which I figured out by adding a message.
The problem is that as soon as I edebug the function, I can only see the
first call for the `mode-from' event.  After stepping through it, I
won't be put into the debugger for the second event as if it has been
discarded in the meantime.  My handle function is also not run when
edebugging.  Is that expected that when edebugging event handlers one
might miss events?

Anyway, eventually I found and fixed the culprit which simply was that
the watch descriptors of the pending and the current event were compared
with `eq' which is not valid because inotify descriptors are conses.
I've changed the comparison to `equal' which fixes the double-reporting
issue.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: 21435@debbugs.gnu.org
> Date: Tue, 08 Sep 2015 21:05:00 +0200
> 
> I stepped through inotify_callback in inotify.c which can read 64 byte
> from inotifyfd where the size of one inotify event is 32 and that
> produces 2 emacs events (although I cannot see which events are
> created).  The Locals GUD frame always shows just nil for the `event'
> local variable even after the assignments to it and I stepped into the
> then-branch of an `if (!NILP(event.arg))' which makes it pretty clear
> that it cannot be nil.  Why is that?

Could be some gud bug.  Do you see the same when you invoke GDB from
the shell?

> Then I turned to `file-notify-callback'.  That receives one `move-from'
> event and one `move-to' event which I figured out by adding a message.
> The problem is that as soon as I edebug the function, I can only see the
> first call for the `mode-from' event.  After stepping through it, I
> won't be put into the debugger for the second event as if it has been
> discarded in the meantime.  My handle function is also not run when
> edebugging.  Is that expected that when edebugging event handlers one
> might miss events?

I always use 'message' when debugging event-driven code.  I just don't
trust edebug enough in these cases.

> Anyway, eventually I found and fixed the culprit which simply was that
> the watch descriptors of the pending and the current event were compared
> with `eq' which is not valid because inotify descriptors are conses.
> I've changed the comparison to `equal' which fixes the double-reporting
> issue.

Thanks.  But meanwhile Michale explained that this behavior was on
purpose, due to a recent change.  Do you see 2 events only when a file
was moved to a different directory, or also when it is renamed in the
same directory?

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> I stepped through inotify_callback in inotify.c which can read 64
>> byte from inotifyfd where the size of one inotify event is 32 and
>> that produces 2 emacs events (although I cannot see which events are
>> created).  The Locals GUD frame always shows just nil for the `event'
>> local variable even after the assignments to it and I stepped into
>> the then-branch of an `if (!NILP(event.arg))' which makes it pretty
>> clear that it cannot be nil.  Why is that?
>
> Could be some gud bug.  Do you see the same when you invoke GDB from
> the shell?

Oh, my GDB foo on the command line ends with making backtraces. ;-)

But just typing in "info locals" in the *gud-bootstrap-emacs* buffer is
about the same, no?  And that shows

  event = {kind = FILE_NOTIFY_EVENT, part = scroll_bar_nowhere, code = 0, modifiers = 0, x = 0, y = 0, timestamp = 0, frame_or_window = 0, arg = 0}

whereas the relevant line in the locals GUD window is just

  struct input_event event        nil

So that looks like a bug.  I guess I should report this, or will you fix
this right now?

>> Is that expected that when edebugging event handlers one might miss
>> events?
>
> I always use 'message' when debugging event-driven code.  I just don't
> trust edebug enough in these cases.

Ok.

>> Anyway, eventually I found and fixed the culprit which simply was
>> that the watch descriptors of the pending and the current event were
>> compared with `eq' which is not valid because inotify descriptors are
>> conses.  I've changed the comparison to `equal' which fixes the
>> double-reporting issue.
>
> Thanks.  But meanwhile Michale explained that this behavior was on
> purpose, due to a recent change.  Do you see 2 events only when a file
> was moved to a different directory, or also when it is renamed in the
> same directory?

When renaming in the same directory.  I think Michaels case is where you
move a file from watched directory A to watched directory B, and then
you want the rename event reported to both handlers.  The case I fixed
is just that both source and target watch descriptor are actually the
same but not identical Lisp objects.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Tassilo Horn <tsdh@gnu.org>
> Cc: 21435@debbugs.gnu.org
> Date: Tue, 08 Sep 2015 21:47:50 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I stepped through inotify_callback in inotify.c which can read 64
> >> byte from inotifyfd where the size of one inotify event is 32 and
> >> that produces 2 emacs events (although I cannot see which events are
> >> created).  The Locals GUD frame always shows just nil for the `event'
> >> local variable even after the assignments to it and I stepped into
> >> the then-branch of an `if (!NILP(event.arg))' which makes it pretty
> >> clear that it cannot be nil.  Why is that?
> >
> > Could be some gud bug.  Do you see the same when you invoke GDB from
> > the shell?
> 
> Oh, my GDB foo on the command line ends with making backtraces. ;-)
> 
> But just typing in "info locals" in the *gud-bootstrap-emacs* buffer is
> about the same, no?  And that shows
> 
>   event = {kind = FILE_NOTIFY_EVENT, part = scroll_bar_nowhere, code = 0, modifiers = 0, x = 0, y = 0, timestamp = 0, frame_or_window = 0, arg = 0}
> 
> whereas the relevant line in the locals GUD window is just
> 
>   struct input_event event        nil
> 
> So that looks like a bug.  I guess I should report this, or will you fix
> this right now?

Please report it, with a test case.  A C struct can never be nil, so
this is a clear bug.

bug#21435: 25.0.50; file-notify has problems after renames

[Tassilo Horn]

Eli Zaretskii <eliz@gnu.org> writes:

>> But just typing in "info locals" in the *gud-bootstrap-emacs* buffer is
>> about the same, no?  And that shows
>> 
>>   event = {kind = FILE_NOTIFY_EVENT, part = scroll_bar_nowhere, code = 0,
>> modifiers = 0, x = 0, y = 0, timestamp = 0, frame_or_window = 0, arg = 0}
>> 
>> whereas the relevant line in the locals GUD window is just
>> 
>>   struct input_event event        nil
>> 
>> So that looks like a bug.  I guess I should report this, or will you fix
>> this right now?
>
> Please report it, with a test case.  A C struct can never be nil, so
> this is a clear bug.

Done: bug#21438.

But it actually seems to be intentional.  GUD uses the --simple-values
option when getting the values of the locals, and then all variables of
complex types have no value and nil is printed.  Well, in the report I
suggest to use something like <non-simple value> instead to make it
obvious that the value is not nil but simply not displayed.

Bye,
Tassilo

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Tassilo Horn <tsdh@gnu.org> writes:

> file-notify using the inotify backend reports wrong events after
> directory renames.

This is also fixed, the test case for inotify and Tramp pass. So the bug
report could be closed once there is a final confirmation for w32notify.

Changes for the gfilenotify case will follow.

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Date: Sun, 20 Sep 2015 19:26:21 +0200
> Cc: 21435@debbugs.gnu.org
> 
> Tassilo Horn <tsdh@gnu.org> writes:
> 
> > file-notify using the inotify backend reports wrong events after
> > directory renames.
> 
> This is also fixed, the test case for inotify and Tramp pass. So the bug
> report could be closed once there is a final confirmation for w32notify.

w32notify reports 2 changed events in a row:

  Test file-notify-test02-events condition:
      (ert-test-failed
       ((should
	 (equal '...
	  (mapcar ... events)))
	:form
	(equal
	 (created changed deleted)
	 (created changed changed deleted))
	:value nil :explanation
	(proper-lists-of-different-length 3 4
					  (created changed deleted)
					  (created changed changed deleted)
					  first-mismatch-at 2)))

This is the "copy" part of the test.  Interestingly, sometimes there
are 2 separate "changed" events and sometimes only 1.  Does
filenotify.el try to conflate several consecutive events of the same
kind into one?  In any case, I guess we will have to allow either one
or 2 "changed" events there.

The other parts also have differences.  Here are the diffs I need to
make all the tests pass:

diff --git a/test/automated/file-notify-tests.el b/test/automated/file-notify-tests.el
index 9d66f03..1aaf4b7 100644
--- a/test/automated/file-notify-tests.el
+++ b/test/automated/file-notify-tests.el
@@ -286,7 +286,7 @@ (ert-deftest file-notify-test02-events ()
         (file-notify--test-with-events
             3 (file-notify--test-timeout)
             (lambda (events)
-              (should (equal '(created changed deleted)
+              (should (equal '(created changed changed deleted)
                              (mapcar #'cadr events))))
           (write-region
            "any text" nil file-notify--test-tmpfile nil 'no-message)
@@ -318,7 +318,7 @@ (ert-deftest file-notify-test02-events ()
         (file-notify--test-with-events
             2 (file-notify--test-timeout)
             (lambda (events)
-              (should (equal '(attribute-changed attribute-changed)
+              (should (equal '(changed changed changed)
                              (mapcar #'cadr events))))
           (write-region
            "any text" nil file-notify--test-tmpfile nil 'no-message)
@@ -331,9 +331,9 @@ (ert-deftest file-notify-test02-events ()
         ;; `file-notify--test-events' has been set correctly.
         (should (equal (mapcar #'cadr file-notify--test-events)
                        '(created changed deleted
-                                 created changed deleted
+                                 created changed changed deleted
                                  created changed renamed
-                                 attribute-changed attribute-changed)))
+                                 changed changed changed)))
 
         (should file-notify--test-results)
         (dolist (result file-notify--test-results)

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

> w32notify reports 2 changed events in a row:
>
>   Test file-notify-test02-events condition:
>       (ert-test-failed
>        ((should
> 	 (equal '...
> 	  (mapcar ... events)))
> 	:form
> 	(equal
> 	 (created changed deleted)
> 	 (created changed changed deleted))
> 	:value nil :explanation
> 	(proper-lists-of-different-length 3 4
> 					  (created changed deleted)
> 					  (created changed changed deleted)
> 					  first-mismatch-at 2)))
>
> This is the "copy" part of the test.  Interestingly, sometimes there
> are 2 separate "changed" events and sometimes only 1.  Does
> filenotify.el try to conflate several consecutive events of the same
> kind into one?  In any case, I guess we will have to allow either one
> or 2 "changed" events there.

No, filenotify.el handles them as they are. The only change in the event
flow is, when two consecutive deleted+created events are combined to a
renamed event.

However, as said I have extended the test case a little bit. There are
now set-file-times and set-file-modes calls, which shall result in
attribute-changed events. And those events shall be suppressed, because
we start file notifications with (file-notify-add-watch ... '(change) ...)

How does w32notify report attribute changes? Could you, please, call
(trace-output 'file-notify-handle-event) and rerun file-notify-test02-events?

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Mon, 21 Sep 2015 08:25:58 +0200
> 
> However, as said I have extended the test case a little bit. There are
> now set-file-times and set-file-modes calls, which shall result in
> attribute-changed events. And those events shall be suppressed, because
> we start file notifications with (file-notify-add-watch ... '(change) ...)
> 
> How does w32notify report attribute changes?

As documented: it returns 'modified'.

> Could you, please, call (trace-output 'file-notify-handle-event) and
> rerun file-notify-test02-events?

The trace for one run is below:

======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-from ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-to ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-from ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-to ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-from ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-to ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 added "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-from "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 renamed-to "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 removed "file-notify-test35884xN") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#-emacsa05544") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified ".#file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil
======================================================================
1 -> (file-notify-handle-event (file-notify (100268128 modified "file-notify-test3588rnH") file-notify-callback))
1 <- file-notify-handle-event: nil

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

Hi Eli,

>> However, as said I have extended the test case a little bit. There are
>> now set-file-times and set-file-modes calls, which shall result in
>> attribute-changed events. And those events shall be suppressed, because
>> we start file notifications with (file-notify-add-watch ... '(change) ...)
>> 
>> How does w32notify report attribute changes?
>
> As documented: it returns 'modified'.

Hmm. So it does not distinguish between "contents changed" and
"attribute changed", as the other backends do.

What a pity! We must document it then.

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Mon, 21 Sep 2015 12:13:23 +0200
> 
> >> How does w32notify report attribute changes?
> >
> > As documented: it returns 'modified'.
> 
> Hmm. So it does not distinguish between "contents changed" and
> "attribute changed", as the other backends do.

Indeed, it doesn't.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> >> How does w32notify report attribute changes?
>> >
>> > As documented: it returns 'modified'.
>> 
>> Hmm. So it does not distinguish between "contents changed" and
>> "attribute changed", as the other backends do.
>
> Indeed, it doesn't.

I've committed a patch, which reflects this in file-notify-tests.el. And
I've added also a note about to the manual.

Could you, pls, check?

Best regards, Michael.

bug#21435: 25.0.50; file-notify has problems after renames

[Eli Zaretskii]

> From: Michael Albinus <michael.albinus@gmx.de>
> Cc: tsdh@gnu.org,  21435@debbugs.gnu.org
> Date: Mon, 21 Sep 2015 15:11:48 +0200
> 
> I've committed a patch, which reflects this in file-notify-tests.el. And
> I've added also a note about to the manual.
> 
> Could you, pls, check?

Checked, fixed, and pushed.

Thanks.

bug#21435: 25.0.50; file-notify has problems after renames

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

>> I've committed a patch, which reflects this in file-notify-tests.el. And
>> I've added also a note about to the manual.
>> 
>> Could you, pls, check?
>
> Checked, fixed, and pushed.

Thanks. I'm closing the bug.

> Thanks.

Best regards, Michael.