bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Hi,

Consider the following

1. emacs -Q
2. M-: (require 'ob-shell)
3. Open the following org file
   #+begin_src sh :results graphics file :file /tmp/colour.png
   convert -size 300x300 xc:#002b36 /tmp/colour.png
   #+end_src
4. Move point to the source block and C-c C-c yes
5. C-c C-x C-v
6. Observe the 300x300 image appearing
7. Edit the source code to 500x300
8. C-c C-c yes
9. Image disappears
10. C-c C-x C-v
11. Observe what appears to be the old 300x300 image
12. Move the cursor to the image
13. Observe the correct 500x300 appearing while the cursor is on it!!

If I step through the code inserting the image overlays, the wrong (not
updated) overlay appears in `org-display-inline-images':

    (let ((ov (make-overlay
	       (org-element-property :begin link)
	       (progn
		 (goto-char
		  (org-element-property :end link))
		 (skip-chars-backward " \t")
		 (point)))))
      (overlay-put ov 'display image) ;; <- the updated image is correctly displayed
      (overlay-put ov 'face 'default) ;; <- un-updated image is displayed until we move cursor to the overlay
      (overlay-put ov 'org-image-overlay t)


In GNU Emacs 30.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version
 3.24.35, cairo version 1.17.6) of 2022-12-07 built on localhost
Repository revision: f0e118d3f676afe39df29663399d4bbe8f53f208
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Gentoo Linux

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Date: Thu, 08 Dec 2022 11:17:01 +0000
> 
> 1. emacs -Q
> 2. M-: (require 'ob-shell)
> 3. Open the following org file
>    #+begin_src sh :results graphics file :file /tmp/colour.png
>    convert -size 300x300 xc:#002b36 /tmp/colour.png
>    #+end_src
> 4. Move point to the source block and C-c C-c yes
> 5. C-c C-x C-v
> 6. Observe the 300x300 image appearing
> 7. Edit the source code to 500x300
> 8. C-c C-c yes
> 9. Image disappears
> 10. C-c C-x C-v
> 11. Observe what appears to be the old 300x300 image
> 12. Move the cursor to the image
> 13. Observe the correct 500x300 appearing while the cursor is on it!!

Is the "convert" part in step 3 necessary to reproduce this?  If so,
it's a huge turn-off for me, because I don't have, and won't have,
ImageMagick installed, or anything similar to it.  So someone else
will have to investigate this in that case, sorry.

> If I step through the code inserting the image overlays, the wrong (not
> updated) overlay appears in `org-display-inline-images':

Where is this code through which you are stepping?

>     (let ((ov (make-overlay
> 	       (org-element-property :begin link)
> 	       (progn
> 		 (goto-char
> 		  (org-element-property :end link))
> 		 (skip-chars-backward " \t")
> 		 (point)))))
>       (overlay-put ov 'display image) ;; <- the updated image is correctly displayed
>       (overlay-put ov 'face 'default) ;; <- un-updated image is displayed until we move cursor to the overlay

And if you switch the order of the above two lines?

Anyway, it is not reliable to try to deduce what happens when stepping
through the code if the problem is relied to cursor motion or
redisplay, because Edebug changes all of that in radical ways.

Finally, a stab in the dark: the fact that the problem disappears when
you move the cursor might mean that a call to 'redisplay' (or some
other trigger for redisplay to know stuff has changed) is missing
somewhere.

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 2151 bytes --]

Eli Zaretskii <eliz@gnu.org> writes:

>> 3. Open the following org file
>>    #+begin_src sh :results graphics file :file /tmp/colour.png
>>    convert -size 300x300 xc:#002b36 /tmp/colour.png
>>    #+end_src
> ...
> Is the "convert" part in step 3 necessary to reproduce this?  If so,
> it's a huge turn-off for me, because I don't have, and won't have,
> ImageMagick installed, or anything similar to it.  So someone else
> will have to investigate this in that case, sorry.

It is not. Anything generating an image of known size will do.
The original report is in
https://list.orgmode.org/87wn7287n2.fsf@localhost/T/#mda89a57883b92c8b48ad6c4aec9e111c131efe92

You can try the following:

#+begin_src sh :results graphics file :file /tmp/colour.png
true
#+end_src

and then change to

#+begin_src sh :results graphics file :file /tmp/colour2.png
true
#+end_src

The images are attached.

>> If I step through the code inserting the image overlays, the wrong (not
>> updated) overlay appears in `org-display-inline-images':
>
> Where is this code through which you are stepping?

`org-display-inline-images'

>>     (let ((ov (make-overlay
>> 	       (org-element-property :begin link)
>> 	       (progn
>> 		 (goto-char
>> 		  (org-element-property :end link))
>> 		 (skip-chars-backward " \t")
>> 		 (point)))))
>>       (overlay-put ov 'display image) ;; <- the updated image is correctly displayed
>>       (overlay-put ov 'face 'default) ;; <- un-updated image is displayed until we move cursor to the overlay
>
> And if you switch the order of the above two lines?

Ain't helping.

> Anyway, it is not reliable to try to deduce what happens when stepping
> through the code if the problem is relied to cursor motion or
> redisplay, because Edebug changes all of that in radical ways.

I use debug-on-entry. Probably not much different though.

> Finally, a stab in the dark: the fact that the problem disappears when
> you move the cursor might mean that a call to 'redisplay' (or some
> other trigger for redisplay to know stuff has changed) is missing
> somewhere.

Nope. When I move the cursor away, the erroneous image is displayed again.


[-- Attachment #2: colour.png --]
[-- Type: image/png, Size: 351 bytes --]

[-- Attachment #3: colour2.png --]
[-- Type: image/png, Size: 358 bytes --]

[-- Attachment #4: Type: text/plain, Size: 225 bytes --]



-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 08 Dec 2022 11:52:49 +0000
> 
> > Finally, a stab in the dark: the fact that the problem disappears when
> > you move the cursor might mean that a call to 'redisplay' (or some
> > other trigger for redisplay to know stuff has changed) is missing
> > somewhere.
> 
> Nope. When I move the cursor away, the erroneous image is displayed again.

?? Really?

And the fact that you report this for Emacs 30, does it mean that
previous versions don't behave like that and this is a recent
regression?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Nope. When I move the cursor away, the erroneous image is displayed again.
>
> ?? Really?

Yup. I hope that it is reproducible on your side as well.

> And the fact that you report this for Emacs 30, does it mean that
> previous versions don't behave like that and this is a recent
> regression?

I was also able to reproduce using Emacs 28 + Org 9.6.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 08 Dec 2022 11:52:49 +0000
> 
> You can try the following:
> 
> #+begin_src sh :results graphics file :file /tmp/colour.png
> true
> #+end_src

Can you help me modify the recipe so that I won't need a 'sh'
executable that is a Posix shell?  I'm not familiar with the Org
source blocks feature very well, so I don't know how to tweak this to
use some other program.  Since we are not really interested in what
the program does here, it could be anything, I think.  Does that have
to be a program that Emacs runs?  Or is that an important part of the
problem?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> Can you help me modify the recipe so that I won't need a 'sh'
> executable that is a Posix shell?

Ok. Try the following:

1. Save colour.png to /tmp/colour.png (300x300)
2. emacs -Q
3. Open an org file with the following contents

   <point>
   
   [[file:/tmp/colour.png]]

4. C-c C-x C-v
5. Observe 300x300 image displayed
6. Overwrite /tmp/colour.png with colour2.png (500x300)
7. C-c C-x C-v
8. Image disappear
9. C-c C-x C-v
10. _Old_ image appears
11. Move point to the image. Observe image being updated to the correct
    500x300 size

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 08 Dec 2022 19:27:39 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Can you help me modify the recipe so that I won't need a 'sh'
> > executable that is a Posix shell?
> 
> Ok. Try the following:
> 
> 1. Save colour.png to /tmp/colour.png (300x300)
> 2. emacs -Q
> 3. Open an org file with the following contents
> 
>    <point>
>    
>    [[file:/tmp/colour.png]]
> 
> 4. C-c C-x C-v
> 5. Observe 300x300 image displayed
> 6. Overwrite /tmp/colour.png with colour2.png (500x300)
> 7. C-c C-x C-v
> 8. Image disappear
> 9. C-c C-x C-v
> 10. _Old_ image appears
> 11. Move point to the image. Observe image being updated to the correct
>     500x300 size

Thanks.  I think when the image is replaced, you need to call
clear-image-cache.

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> Thanks.  I think when the image is replaced, you need to call
> clear-image-cache.

Sure, I can. Or even `image-flush'.
However, it is not obvious at all that it should be necessary to clear
image cache.

The current Emacs' behaviour is unexpected and very confusing here.
Ideally, Emacs should be able to judge by itself whether to update
cached image in memory.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Sat, 10 Dec 2022 09:57:33 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Thanks.  I think when the image is replaced, you need to call
> > clear-image-cache.
> 
> Sure, I can. Or even `image-flush'.
> However, it is not obvious at all that it should be necessary to clear
> image cache.
> 
> The current Emacs' behaviour is unexpected and very confusing here.
> Ideally, Emacs should be able to judge by itself whether to update
> cached image in memory.

You want Emacs to hit the disk and process the file on each redisplay
cycle?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The current Emacs' behaviour is unexpected and very confusing here.
>> Ideally, Emacs should be able to judge by itself whether to update
>> cached image in memory.
>
> You want Emacs to hit the disk and process the file on each redisplay
> cycle?

No. It can be less frequent. Maybe just when a new image view is created
(like in overlay). Or maybe on timer - make images in the image cache
expire after some time.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Sat, 10 Dec 2022 11:10:19 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The current Emacs' behaviour is unexpected and very confusing here.
> >> Ideally, Emacs should be able to judge by itself whether to update
> >> cached image in memory.
> >
> > You want Emacs to hit the disk and process the file on each redisplay
> > cycle?
> 
> No. It can be less frequent. Maybe just when a new image view is created
> (like in overlay). Or maybe on timer - make images in the image cache
> expire after some time.

The image-handling code doesn't necessarily know where the image came
from.

The idea of clearing images from time to time was discussed, but AFAIR
we didn't find a good way that fits all the needs for doing so.

I think this is best left to the application level.  In particular, in
this case, the application _knows_ when it will replace the image
file.

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> > You want Emacs to hit the disk and process the file on each redisplay
>> > cycle?
>> 
>> No. It can be less frequent. Maybe just when a new image view is created
>> (like in overlay). Or maybe on timer - make images in the image cache
>> expire after some time.
>
> The image-handling code doesn't necessarily know where the image came
> from.
>
> The idea of clearing images from time to time was discussed, but AFAIR
> we didn't find a good way that fits all the needs for doing so.

Are you referring to https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33275
?

I do not see anyone proposing cache expiry time there. Expiry is not
really related to this particular bug report, but might be a good idea
to help large memory consumption that is reported from time to time (I
reported once for pdfs and I have seen people report memory issues with
images).

> I think this is best left to the application level.  In particular, in
> this case, the application _knows_ when it will replace the image
> file.

No. Not really. Source block execution does not pass any information
about updating/not updating images to Org image display code. Even if it
did, the same issue could appear when the image file changes on disk
outside Emacs.

I was still able to make some improvements in Org's image code thanks to
your replies.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a12d15df9

Yet, do note that flickering two different image versions when moving
point is unexpected even considering the information you provided.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Sun, 11 Dec 2022 09:19:16 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > The idea of clearing images from time to time was discussed, but AFAIR
> > we didn't find a good way that fits all the needs for doing so.
> 
> Are you referring to https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33275
> ?

That's one of them, yes.  There were others.

> I do not see anyone proposing cache expiry time there. Expiry is not
> really related to this particular bug report, but might be a good idea
> to help large memory consumption that is reported from time to time (I
> reported once for pdfs and I have seen people report memory issues with
> images).
> 
> > I think this is best left to the application level.  In particular, in
> > this case, the application _knows_ when it will replace the image
> > file.
> 
> No. Not really. Source block execution does not pass any information
> about updating/not updating images to Org image display code. Even if it
> did, the same issue could appear when the image file changes on disk
> outside Emacs.

Then I don't think I understand what you expect Emacs to do in these
cases.  We have no idea when the image file is replaced, and cannot
have such an idea without examining the file at high enough frequency.
Doing this "from time to time" is going to miss some changes, or take
note of them too late.  What else is possible?

> I was still able to make some improvements in Org's image code thanks to
> your replies.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=a12d15df9
> 
> Yet, do note that flickering two different image versions when moving
> point is unexpected even considering the information you provided.

Flickering is expected when you do something that affects a large
portion of the Emacs display.  For example, the same will happen if
you change a large overlay at high enough frequency.  There's no way
around that except not doing that.

Why was this implementation chosen for whatever feature that produces
images?  Emacs doesn't react instantly to changes in disk files that
it visits, and here you expect it to do so.  Isn't it possible to
implement this in some other way, like have the program produce its
image data in a temporary Emacs buffer, then use that buffer's
contents for creating an image?  Then I believe the updated image will
have a different hash value, and there will be no cache-related
collisions.

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> No. Not really. Source block execution does not pass any information
>> about updating/not updating images to Org image display code. Even if it
>> did, the same issue could appear when the image file changes on disk
>> outside Emacs.
>
> Then I don't think I understand what you expect Emacs to do in these
> cases.  We have no idea when the image file is replaced, and cannot
> have such an idea without examining the file at high enough frequency.
> Doing this "from time to time" is going to miss some changes, or take
> note of them too late.  What else is possible?

For example, `create-image' could accept optional argument to update the
cached image if the file modification time is newer than it was when the
image was cached.

>> Yet, do note that flickering two different image versions when moving
>> point is unexpected even considering the information you provided.
>
> Flickering is expected when you do something that affects a large
> portion of the Emacs display.  For example, the same will happen if
> you change a large overlay at high enough frequency.  There's no way
> around that except not doing that.

Sorry, I used inaccurate word here.
What I am referring to here is image overlay displayed old and new image
when you move the cursor to the overlay and back repeatedly.

> Why was this implementation chosen for whatever feature that produces
> images?  Emacs doesn't react instantly to changes in disk files that
> it visits, and here you expect it to do so.  Isn't it possible to
> implement this in some other way, like have the program produce its
> image data in a temporary Emacs buffer, then use that buffer's
> contents for creating an image?  Then I believe the updated image will
> have a different hash value, and there will be no cache-related
> collisions.

It could be possible, but looks like unnecessary complication.
Also, creating buffers often creates overheads associated with user
hooks.

I am not sure if something like

(let ((revert-without-query '(".")))
		      (with-current-buffer (find-file-noselect file)
                        ;; Do not crown the buffer list.
                        (unless (string-prefix-p " " (buffer-name))
                          (setf (buffer-name) (concat " " (buffer-name))))
			(buffer-string)))

is not going to cause significant slowdowns.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Mon, 12 Dec 2022 08:35:43 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Then I don't think I understand what you expect Emacs to do in these
> > cases.  We have no idea when the image file is replaced, and cannot
> > have such an idea without examining the file at high enough frequency.
> > Doing this "from time to time" is going to miss some changes, or take
> > note of them too late.  What else is possible?
> 
> For example, `create-image' could accept optional argument to update the
> cached image if the file modification time is newer than it was when the
> image was cached.

Patches welcome.  We currently hash only the spec itself, and don't
pay attention to the file's meta-data.  (I also think that if we go
this way, file modification time alone could be insufficient for 100%
reliability, what with today's fast filesystems.)

> >> Yet, do note that flickering two different image versions when moving
> >> point is unexpected even considering the information you provided.
> >
> > Flickering is expected when you do something that affects a large
> > portion of the Emacs display.  For example, the same will happen if
> > you change a large overlay at high enough frequency.  There's no way
> > around that except not doing that.
> 
> Sorry, I used inaccurate word here.
> What I am referring to here is image overlay displayed old and new image
> when you move the cursor to the overlay and back repeatedly.

Yes, I understood.  My response is still the same.

> > Why was this implementation chosen for whatever feature that produces
> > images?  Emacs doesn't react instantly to changes in disk files that
> > it visits, and here you expect it to do so.  Isn't it possible to
> > implement this in some other way, like have the program produce its
> > image data in a temporary Emacs buffer, then use that buffer's
> > contents for creating an image?  Then I believe the updated image will
> > have a different hash value, and there will be no cache-related
> > collisions.
> 
> It could be possible, but looks like unnecessary complication.
> Also, creating buffers often creates overheads associated with user
> hooks.
> 
> I am not sure if something like
> 
> (let ((revert-without-query '(".")))
> 		      (with-current-buffer (find-file-noselect file)
>                         ;; Do not crown the buffer list.
>                         (unless (string-prefix-p " " (buffer-name))
>                           (setf (buffer-name) (concat " " (buffer-name))))
> 			(buffer-string)))
> 
> is not going to cause significant slowdowns.

Sorry, you lost me here: how is the above snippet relevant to what I
proposed as an alternative implementation?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Why was this implementation chosen for whatever feature that produces
>> > images?  Emacs doesn't react instantly to changes in disk files that
>> > it visits, and here you expect it to do so.  Isn't it possible to
>> > implement this in some other way, like have the program produce its
>> > image data in a temporary Emacs buffer, then use that buffer's
>> > contents for creating an image?  Then I believe the updated image will
>> > have a different hash value, and there will be no cache-related
>> > collisions.
>> 
>> It could be possible, but looks like unnecessary complication.
>> Also, creating buffers often creates overheads associated with user
>> hooks.
>> 
>> I am not sure if something like
>> 
>> (let ((revert-without-query '(".")))
>> 		      (with-current-buffer (find-file-noselect file)
>>                         ;; Do not crown the buffer list.
>>                         (unless (string-prefix-p " " (buffer-name))
>>                           (setf (buffer-name) (concat " " (buffer-name))))
>> 			(buffer-string)))
>> 
>> is not going to cause significant slowdowns.
>
> Sorry, you lost me here: how is the above snippet relevant to what I
> proposed as an alternative implementation?

The above snippet creates a buffer with image FILE, reverts it if
necessary (`find-file-noselect' should do it), and returns the image
data that is guaranteed to be up-to-date with the actual file.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 15 Dec 2022 10:24:16 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> I am not sure if something like
> >> 
> >> (let ((revert-without-query '(".")))
> >> 		      (with-current-buffer (find-file-noselect file)
> >>                         ;; Do not crown the buffer list.
> >>                         (unless (string-prefix-p " " (buffer-name))
> >>                           (setf (buffer-name) (concat " " (buffer-name))))
> >> 			(buffer-string)))
> >> 
> >> is not going to cause significant slowdowns.
> >
> > Sorry, you lost me here: how is the above snippet relevant to what I
> > proposed as an alternative implementation?
> 
> The above snippet creates a buffer with image FILE, reverts it if
> necessary (`find-file-noselect' should do it), and returns the image
> data that is guaranteed to be up-to-date with the actual file.

Then why do you think it will cause slowdowns?  Especially if you
call find-file-noselect with its 3rd argument non-nil, as I think you
should anyway?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> The above snippet creates a buffer with image FILE, reverts it if
>> necessary (`find-file-noselect' should do it), and returns the image
>> data that is guaranteed to be up-to-date with the actual file.
>
> Then why do you think it will cause slowdowns?  Especially if you
> call find-file-noselect with its 3rd argument non-nil, as I think you
> should anyway?

Hooks will be executed first time an image is opened. This may cause
slowdowns when there is a huge number of inline images in the buffer,
first time they are displayed.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 15 Dec 2022 10:53:19 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> The above snippet creates a buffer with image FILE, reverts it if
> >> necessary (`find-file-noselect' should do it), and returns the image
> >> data that is guaranteed to be up-to-date with the actual file.
> >
> > Then why do you think it will cause slowdowns?  Especially if you
> > call find-file-noselect with its 3rd argument non-nil, as I think you
> > should anyway?
> 
> Hooks will be executed first time an image is opened. This may cause
> slowdowns when there is a huge number of inline images in the buffer,
> first time they are displayed.

Which hooks are we talking about here?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Hooks will be executed first time an image is opened. This may cause
>> slowdowns when there is a huge number of inline images in the buffer,
>> first time they are displayed.
>
> Which hooks are we talking about here?

find-file-hook, change-major-mode-after-body-hook,
and after-change-major-mode-hook.

In my setup, these hooks call 29 functions total.
It may be more in heavily customized distributions like Doom.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 15 Dec 2022 11:05:56 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> Hooks will be executed first time an image is opened. This may cause
> >> slowdowns when there is a huge number of inline images in the buffer,
> >> first time they are displayed.
> >
> > Which hooks are we talking about here?
> 
> find-file-hook, change-major-mode-after-body-hook,
> and after-change-major-mode-hook.

You can let-bind them to nil around that snippet, mo?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> find-file-hook, change-major-mode-after-body-hook,
>> and after-change-major-mode-hook.
>
> You can let-bind them to nil around that snippet, mo?

I only know these hooks from reading the code.
How can I make sure that let-binding will reliably work in future Emacs
versions?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 15 Dec 2022 11:20:49 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> find-file-hook, change-major-mode-after-body-hook,
> >> and after-change-major-mode-hook.
> >
> > You can let-bind them to nil around that snippet, mo?
> 
> I only know these hooks from reading the code.
> How can I make sure that let-binding will reliably work in future Emacs
> versions?

I don't think I understand the question.  Do you mean we could
possibly remove these hooks in future versions?  If not, what do you
mean by "reliably"?

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> > You can let-bind them to nil around that snippet, mo?
>> 
>> I only know these hooks from reading the code.
>> How can I make sure that let-binding will reliably work in future Emacs
>> versions?
>
> I don't think I understand the question.  Do you mean we could
> possibly remove these hooks in future versions?  If not, what do you
> mean by "reliably"?

I am mostly afraid about adding new similar hooks in future. I am not
even sure if my list of executed hooks is complete.

Practically, I'd prefer Emacs to provide a macro or extra argument that
binds all the necessary hooks when opening a file buffer. Maybe,
something similar to `delay-mode-hooks'.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: 59902@debbugs.gnu.org
> Date: Thu, 15 Dec 2022 13:23:58 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> > You can let-bind them to nil around that snippet, mo?
> >> 
> >> I only know these hooks from reading the code.
> >> How can I make sure that let-binding will reliably work in future Emacs
> >> versions?
> >
> > I don't think I understand the question.  Do you mean we could
> > possibly remove these hooks in future versions?  If not, what do you
> > mean by "reliably"?
> 
> I am mostly afraid about adding new similar hooks in future. I am not
> even sure if my list of executed hooks is complete.

If and when we'll add new hooks and they prove to be an issue in this
context (i.e. will slow down display of images), they should be added
to those disabled.

> Practically, I'd prefer Emacs to provide a macro or extra argument that
> binds all the necessary hooks when opening a file buffer. Maybe,
> something similar to `delay-mode-hooks'.

I don't think this is possible in practice, since only the application
can know which hooks are or aren't needed.

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Practically, I'd prefer Emacs to provide a macro or extra argument that
>> binds all the necessary hooks when opening a file buffer. Maybe,
>> something similar to `delay-mode-hooks'.
>
> I don't think this is possible in practice, since only the application
> can know which hooks are or aren't needed.

I'd say that only the application can know which hooks _are_ needed.

For example, a common issue with Org + large number of small files is
running hooks. We do know that org-mode-hook may be needed for
correctness, while all other hooks are likely not needed for
non-interactive use (parsing the file contents).

So, it is not about disabling some hooks but rather about
enabling some known-in-advance hooks.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

bug#59902: 30.0.50; Image overlay is not updated until the cursor moves to the overlay

[William Denton]

About not needing a sh executable: I don't know if this helps, but if you have R 
installed this will show the problem:

#+begin_src R :results graphics file :file /tmp/plot.png
plot(1)
#+end_src

C-c C-c to execute, and a small chart will be generated.  Change "plot(1)" to 
"plot(2)" and re-execute.  The file on disk will change, but not the image in 
the buffer.

Move the pointer down to after the colon in #+RESULTS:, move it right once more 
to the next line, and the image will change.  You can toggle back and forth by 
moving the pointer left and right.

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada