It seems that this not a bug, but a user experience issue. It is not convenient to tweak vim and
other apps which can modify file.

What about the following logic:
When the auto-revert module known the file is renamed or deleted, delete the watch
and start a one shot short timer(about 100ms) to check the visiting file of the buffer. On timeout,
if the file exists, revert the file and add a new watch, otherwise, tell user the file doesn’t exist
and fallback to the default polling(5s).
Once the visiting file of the buffer can be accessed, go back to use file-notify again.

On 29 Dec 2017, 6:14 PM +0800, Eli Zaretskii <eliz@gnu.org>, wrote:
From: Michael Albinus <michael.albinus@gmx.de
Date: Fri, 29 Dec 2017 11:02:10 +0100
Cc: "emacs-devel@gnu.org" <emacs-devel@gnu.org

Emacs: 26.0.90
OS: macOS 10.13.2

1. start Emacs with -Q
2. open a test file (like 1.c)
3. turn on auto-revert-mode by run global-auto-revert-mode
4. add auto-revert-use-notify as a watcher variable by
debug-on-variable-change
5. open the test file with vim
6. modify the test file in vim and save it
7. Emacs enter debug with the following msg:

Debugger entered--setting auto-revert-use-notify in buffer 1.c to nil:
debug--implement-debug-watch(auto-revert-use-notify nil set #<buffer
1.c>)
auto-revert-notify-handler((13 stopped "/Users/jun/test/1.c"))
file-notify--rm-descriptor(13)
file-notify-rm-watch(13)
#f(compiled-function (key value) #<bytecode 0x40a952e1>)(13
(#<buffer 1.c>))
maphash(#f(compiled-function (key value) #<bytecode 0x40a952e1>)
#<hash-table equal 7/65 0x40e7896d>)
auto-revert-notify-rm-watch()
auto-revert-notify-handler((13 stopped "/Users/jun/test/1.c"))
file-notify--rm-descriptor(13)
file-notify-rm-watch(13)
file-notify-callback((13 (rename delete) "/Users/jun/test/1.c"))
file-notify-handle-event((file-notify (13 (rename delete)
"/Users/jun/test/1.c") file-notify-callback))
funcall-interactively(file-notify-handle-event (file-notify (13
(rename delete) "/Users/jun/test/1.c") file-notify-callback))
call-interactively(file-notify-handle-event nil [(file-notify (13
(rename delete) "/Users/jun/test/1.c") file-notify-callback)])
command-execute(file-notify-handle-event nil [(file-notify (13
(rename delete) "/Users/jun/test/1.c") file-notify-callback)] t)

Well, this is not an error. The watchdog over "/Users/jun/test/1.c"
receives the `(rename delete)' events from writing the file. Due to
this, the watchdog is removed, as you see in the backtrace.

Exactly, I think this is precisely the expected behavior: step 4
requests the debugger to be entered when auto-revert-use-notify is
modified, and auto-revert-notify-handler does modify it when the
watched file is deleted:

(if (eq action 'stopped)
;; File notification has stopped. Continue with polling.
(cl-dolist (buffer
(if global-auto-revert-mode
(buffer-list) auto-revert-buffer-list))
(with-current-buffer buffer
(when (and (equal descriptor auto-revert-notify-watch-descriptor)
(or
;; A buffer associated with a file.
(and (stringp buffer-file-name)
(string-equal
(file-name-nondirectory file)
(file-name-nondirectory buffer-file-name)))
;; A buffer w/o a file, like dired.
(null buffer-file-name)))
(auto-revert-notify-rm-watch)
(setq-local auto-revert-use-notify nil)))) <<<<<<<<<<<<<