bug#21729: doc-view messed up by early auto-revert

bug#21729: doc-view messed up by early auto-revert

[Stefan Monnier]

Package: Emacs
Version: 25.0.50


When I work on LaTeX documents, I generally have a doc-view-mode buffer
open displaying the PDF output.  I'd like this document to auto-update
after I recompile the .tex file(s), but using M-x auto-revert-mode
doesn't work well for that:

  emacs -Q foo.pdf
  M-x auto-revert-mode RET

then in a shell

  pdflatex foo

The result is that the doc-view buffer ends up displaying the raw file's
content (for me anyway: it's timing dependent, but I see this
behavior on several machines, with very different performance.  the
foo.tex document needs to be large enough, of course).

I think there are two problems:

- Doc-view should inhibit auto-revert when the file was modified too
  recently (on the assumption that it's not complete yet).

- [ Just guessing here: ] We may sometimes fail to revert the file after
  it's complete, maybe because the last reversion finished after the file
  was complete, even though it started before, so the last reversion
  failed to include some changes that were added while the buffer was
  being reverted.


        Stefan




In GNU Emacs 25.0.50.1 (x86_64-unknown-linux-gnu, X toolkit, Xaw scroll bars)
 of 2015-10-16
Repository revision: 26a88b59cd790e9b8ee7656a9c8672079c09ef52
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Debian GNU/Linux 8.2 (jessie)

Configured using:
 'configure -C --enable-checking --enable-check-lisp-object-type
 'CFLAGS=-Wall -g3 -Og -Wno-pointer-sign'
 PKG_CONFIG_PATH=/home/monnier/lib/pkgconfig'

Configured features:
XPM JPEG TIFF GIF PNG SOUND NOTIFY GNUTLS LIBXML2 FREETYPE XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11

Important settings:
  value of $LANG: C
  locale-coding-system: nil

Major mode: InactiveMinibuffer

Minor modes in effect:
  shell-dirtrack-mode: t
  diff-auto-refine-mode: t
  electric-pair-mode: t
  url-handler-mode: t
  global-reveal-mode: t
  reveal-mode: t
  auto-insert-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  global-prettify-symbols-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent messages:
Error during redisplay: (jit-lock-function 12402647) signaled (scan-error "Unbalanced parentheses" 12345873 12345678)
Error during redisplay: (jit-lock-function 12402637) signaled (scan-error "Unbalanced parentheses" 12345793 12345678)
Error during redisplay: (jit-lock-function 12402795) signaled (scan-error "Unbalanced parentheses" 12345781 12345678)
Error during redisplay: (jit-lock-function 12402787) signaled (scan-error "Unbalanced parentheses" 12345701 12345678)
Error during redisplay: (jit-lock-function 12402904) signaled (scan-error "Unbalanced parentheses" 12345688 12345678)
Error during redisplay: (jit-lock-function 12402780) signaled (scan-error "Unbalanced parentheses" 12345830 12345678)
Error during redisplay: (jit-lock-function 12403005) signaled (scan-error "Unbalanced parentheses" 12345753 12345678)
Reverting buffer ‘paper.pdf’.
Type C-c C-c to toggle between editing or viewing the document.
Warning: vc-find-file-hook is obsolete!

Load-path shadows:
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-ref-man hides /home/monnier/src/emacs/elpa/packages/ada-ref-man/ada-ref-man
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-prj hides /home/monnier/src/emacs/work/lisp/progmodes/ada-prj
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-stmt hides /home/monnier/src/emacs/work/lisp/progmodes/ada-stmt
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-mode hides /home/monnier/src/emacs/work/lisp/progmodes/ada-mode
/home/monnier/src/emacs/elpa/packages/ada-mode/ada-xref hides /home/monnier/src/emacs/work/lisp/progmodes/ada-xref
/home/monnier/src/emacs/elpa/packages/crisp/crisp hides /home/monnier/src/emacs/work/lisp/obsolete/crisp
/home/monnier/src/emacs/elpa/packages/landmark/landmark hides /home/monnier/src/emacs/work/lisp/obsolete/landmark
/home/monnier/src/emacs/work/lisp/emacs-lisp/cl-generic hides /home/monnier/src/emacs/elpa/packages/cl-generic/cl-generic

Features:
(mail-extr emacsbug message rfc822 mml mml-sec mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev mailheader sendmail rfc2047
rfc2045 ietf-drums mail-utils eieio-opt speedbar sb-image ezimage dframe
add-log log-view pcvs-util dabbrev autorevert filenotify doc-view
jka-compr image-mode dired format-spec reftex-sel cursor-sensor
reftex-ref reftex-parse reftex-toc reftex-dcr reftex reftex-vars
tex-mode compile shell pcomplete latexenc executable copyright xscheme
warnings unsafep trace testcover shadow scheme re-builder profiler
inf-lisp ielm pp comint ansi-color ring gmm-utils ert find-func ewoc
debug elp edebug cl-indent cus-edit cus-start cus-load wid-edit sort mpc
misearch multi-isearch vc vc-dispatcher vc-git diff-mode filecache
epa-file epa derived epg server noutline outline easy-mmode flyspell
ispell checkdoc thingatpt load-dir elec-pair url-handlers url-parse
auth-source eieio byte-opt bytecomp byte-compile cl-extra cconv
gnus-util time-date mm-util help-fns help-mode mail-prsvr password-cache
url-vars reveal autoinsert proof-site proof-autoloads cl pg-vars
savehist minibuf-eldef finder-inf advice info eieio-core cl-macs gv
cl-loaddefs pcase cl-lib package easymenu epg-config bbdb-autoloads
vm-autoloads disp-table mule-util tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list newcomment
elisp-mode lisp-mode prog-mode register page menu-bar rfn-eshadow timer
select scroll-bar mouse jit-lock font-lock syntax font-core frame
cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan thai
tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek romanian
slovak czech european ethiopic indian cyrillic chinese charscript
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote inotify dynamic-setting
font-render-setting x-toolkit x multi-tty make-network-process emacs)

Memory information:
((conses 16 273500 37468)
 (symbols 48 29159 0) (miscs 40 3792 1044) (strings 32 64808 12475)
 (string-bytes 1 2487240)
 (vectors 16 40418) (vector-slots 8 1637632 116578) (floats 8 425 624)
 (intervals 56 13968 376)
 (buffers 976 35) (heap 1024 392922 6279))

bug#21729: doc-view messed up by early auto-revert

[Tassilo Horn]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

Hi Stefan,

> When I work on LaTeX documents, I generally have a doc-view-mode
> buffer open displaying the PDF output.  I'd like this document to
> auto-update after I recompile the .tex file(s), but using M-x
> auto-revert-mode doesn't work well for that:
>
>   emacs -Q foo.pdf
>   M-x auto-revert-mode RET
>
> then in a shell
>
>   pdflatex foo
>
> The result is that the doc-view buffer ends up displaying the raw
> file's content (for me anyway: it's timing dependent, but I see this
> behavior on several machines, with very different performance.  the
> foo.tex document needs to be large enough, of course).

Yes, I experienced the same problem although with the PDF Tools viewer
when I tried to enable `auto-revert-mode' in the PDF file buffer.  What
I do now instead is

--8<---------------cut here---------------start------------->8---
(defun th/pdf-view-revert-buffer-maybe (file)
  (when-let ((buf (find-buffer-visiting file)))
    (with-current-buffer buf
      (when (derived-mode-p 'pdf-view-mode)
	(pdf-view-revert-buffer nil t)))))

(add-hook 'TeX-after-TeX-LaTeX-command-finished-hook
	  #'th/pdf-view-revert-buffer-maybe)
--8<---------------cut here---------------end--------------->8---

i.e., I revert the buffer from an AUCTeX hook that's run after the
compilation has finished.

> I think there are two problems:
>
> - Doc-view should inhibit auto-revert when the file was modified too
>   recently (on the assumption that it's not complete yet).

Something like checking if the file's MTIME is more than a second ago?

Hm, no, that won't work.  I currently work on some large document (~450
pages).  When I compile that, the file changes several times per second
during the compilation, then it doesn't change for ~15 seconds while
latex (lualatex in this case) is still running, and then starts changing
again for maybe 3 seconds.

I think the long delay is caused by computing the index and
bibliography.  I just checked that during that period, actually no file
changes, e.g., neither pdf, aux, bcf, idx, etc.  The only thing I can
see is that there's a foo.synctex.gz(busy) file which really disappears
when the compilation has finished but of course nobody is required to
use synctex.

Well, one reasonably reliable way to test if a pdf file is correct is
calling pdfinfo (provided by poppler) on it.  A non-zero exit status
indicated that the file is broken/incomplete.  Should we use that?

> - [ Just guessing here: ] We may sometimes fail to revert the file
> after it's complete, maybe because the last reversion finished after
> the file was complete, even though it started before, so the last
> reversion failed to include some changes that were added while the
> buffer was being reverted.

Yeah, that's possible.  And I think there could also be a problem when
people use latexmk which compiles your document several times until all
refs are defined and its stable.  Here, we might revert exactly between
two latex runs.

Bye,
Tassilo

bug#21729: doc-view messed up by early auto-revert

[Stefan Monnier]

> i.e., I revert the buffer from an AUCTeX hook that's run after the
> compilation has finished.

That's a good approach if you run pdflatex from Emacs itself, yes.
[ Hmm... I guess I could also use emacsclient to get the same result,
  but it'd be a personal hack rather than a general functionality like
  auto-revert-mode.  ]

>> - Doc-view should inhibit auto-revert when the file was modified too
>> recently (on the assumption that it's not complete yet).
> Something like checking if the file's MTIME is more than a second ago?

Yes.

> Hm, no, that won't work.  I currently work on some large document (~450
> pages).  When I compile that, the file changes several times per second
> during the compilation, then it doesn't change for ~15 seconds while
> latex (lualatex in this case) is still running, and then starts changing
> again for maybe 3 seconds.

So we could additionally either check that the file is valid, or simply
"fail gracefully" if it isn't (e.g. keep displaying the old image, and
emit a message pointing to C-c C-c to see the raw data).  But it's not
a deal breaker: in your case, we may end up trying to revert the file
too early, which is annoying, but as long as we do end up re-reverting
it when the file is finally complete, I think it's OK.

>> - [ Just guessing here: ] We may sometimes fail to revert the file
>> after it's complete, maybe because the last reversion finished after
>> the file was complete, even though it started before, so the last
>> reversion failed to include some changes that were added while the
>> buffer was being reverted.
> Yeah, that's possible.  And I think there could also be a problem when
> people use latexmk which compiles your document several times until all
> refs are defined and its stable.  Here, we might revert exactly between
> two latex runs.

I don't think reverting between two latex runs is a problem.  It might
be "wasted work", but for a generic functionality like auto-revert-mode,
there's no way to do better since it can't know that another run is
about to come.


        Stefan

bug#21729: doc-view messed up by early auto-revert

[Tassilo Horn]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> Hm, no, that won't work.  I currently work on some large document
>> (~450 pages).  When I compile that, the file changes several times
>> per second during the compilation, then it doesn't change for ~15
>> seconds while latex (lualatex in this case) is still running, and
>> then starts changing again for maybe 3 seconds.
>
> So we could additionally either check that the file is valid, or
> simply "fail gracefully" if it isn't (e.g. keep displaying the old
> image, and emit a message pointing to C-c C-c to see the raw data).
> But it's not a deal breaker: in your case, we may end up trying to
> revert the file too early, which is annoying, but as long as we do end
> up re-reverting it when the file is finally complete, I think it's OK.

In commit 11d1422, I've added a check which omits reverting when the PDF
is corrupted and spits out a message if doc-view-revert-buffer has been
called interactively.  Seems to work fine but it is annoying that
autorevert.el spits out a "Reverting buffer ..." message whenever it
calls revert-buffer.

I think in doc-view-mode it's obvious enough when a file is being
reverted, so maybe we should set auto-revert-verbose to nil
buffer-locally?

Bye,
Tassilo

bug#21729: doc-view messed up by early auto-revert

[Nicolas Richard]

Tassilo Horn <tsdh@gnu.org> writes:
> I think in doc-view-mode it's obvious enough when a file is being
> reverted, so maybe we should set auto-revert-verbose to nil
> buffer-locally?

Sometimes emacs does weird things, and I think it's nice if it explains
what it does, even if it should be obvious. Morever the message is not
only useful while the file is being reverted, but also after the fact.
e.g. if I have foo.pdf open, and that file is controlled by owncloud and
gets updated by the cloud, I like to find this information in
*Messages*.

-- 
Nico.

bug#21729: doc-view messed up by early auto-revert

[Tassilo Horn]

Nicolas Richard <youngfrog@members.fsf.org> writes:

> Tassilo Horn <tsdh@gnu.org> writes:
>> I think in doc-view-mode it's obvious enough when a file is being
>> reverted, so maybe we should set auto-revert-verbose to nil
>> buffer-locally?
>
> Sometimes emacs does weird things, and I think it's nice if it
> explains what it does, even if it should be obvious.  Morever the
> message is not only useful while the file is being reverted, but also
> after the fact.  e.g. if I have foo.pdf open, and that file is
> controlled by owncloud and gets updated by the cloud, I like to find
> this information in *Messages*.

Yes, I see.  But then we could still set `auto-revert-verbose' to nil
buffer-locally and issue that message in `doc-view-revert-buffer'.  The
problem with the message from autorevert.el is that it'll be displayed
even though doc-view decides that it cannot auto-revert right now
because the file is corrupted/not ready yet.

Alternatively, we could of course enhance autorevert.el, e.g., by
introducing some buffer-local variable `auto-revert-predicate' which is
run by auto-revert in order to test if the current buffer can be
reverted right now.  That could then be used by doc-view, and I guess
there are other modes which could make use of such a feature, too.

Stefan, do you think that would be a good idea?

Bye,
Tassilo

bug#21729: doc-view messed up by early auto-revert

[Lars Ingebrigtsen]

It seems the original issue reported here was fixed, but then an
enhancement request is discussed:

Tassilo Horn <tsdh@gnu.org> writes:

> Alternatively, we could of course enhance autorevert.el, e.g., by
> introducing some buffer-local variable `auto-revert-predicate' which is
> run by auto-revert in order to test if the current buffer can be
> reverted right now.  That could then be used by doc-view, and I guess
> there are other modes which could make use of such a feature, too.
>
> Stefan, do you think that would be a good idea?

I'm not Stefan, but I think that sounds like a good idea.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#21729: doc-view messed up by early auto-revert

[Tassilo Horn]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> It seems the original issue reported here was fixed, but then an
> enhancement request is discussed:
>
> Tassilo Horn <tsdh@gnu.org> writes:
>
>> Alternatively, we could of course enhance autorevert.el, e.g., by
>> introducing some buffer-local variable `auto-revert-predicate' which is
>> run by auto-revert in order to test if the current buffer can be
>> reverted right now.  That could then be used by doc-view, and I guess
>> there are other modes which could make use of such a feature, too.
>>
>> Stefan, do you think that would be a good idea?
>
> I'm not Stefan, but I think that sounds like a good idea.

I don't know.  There are not many file types which are regenerated
(where regeneration modifies the file several times and takes some
time), likely to be opened in Emacs, and which can be checked for
validity.

Bye,
Tassilo

bug#21729: doc-view messed up by early auto-revert

[Stefan Monnier]

>>> I think in doc-view-mode it's obvious enough when a file is being
>>> reverted, so maybe we should set auto-revert-verbose to nil
>>> buffer-locally?
>> Sometimes emacs does weird things, and I think it's nice if it
>> explains what it does, even if it should be obvious.  Morever the
>> message is not only useful while the file is being reverted, but also
>> after the fact.  e.g. if I have foo.pdf open, and that file is
>> controlled by owncloud and gets updated by the cloud, I like to find
>> this information in *Messages*.
> Yes, I see.  But then we could still set `auto-revert-verbose' to nil
> buffer-locally and issue that message in `doc-view-revert-buffer'.

IIUC the message is emitted before calling doc-view-revert-buffer, so
doc-view-revert-buffer could just emit its own message (e.g. "Reverting
buffer... aborted!") and that would hide the generic message.


        Stefan

bug#21729: doc-view messed up by early auto-revert

[Lars Ingebrigtsen]

Tassilo Horn <tsdh@gnu.org> writes:

>> I'm not Stefan, but I think that sounds like a good idea.
>
> I don't know.  There are not many file types which are regenerated
> (where regeneration modifies the file several times and takes some
> time), likely to be opened in Emacs, and which can be checked for
> validity.

OK, then I guess (since the original bug was apparently fixed) there's
nothing more to do here, and I'm therefore closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no