Images generated by R code blocks do not display

Images generated by R code blocks do not display

[William Denton]

I have a problem with images made by R code blocks in Org: not being displayed 
or refreshed.  (This has been going on for a while, but I can't remember when it 
started.)  Here's a minimal reproducible example I hope others can check.  It 
requires R on your system and the ESS package.

First, here's my minimal-init.el file:

#----- minimal-init.el
   (setq debug-on-error t
         debug-on-signal nil
         debug-on-quit nil)

   (add-to-list 'load-path (expand-file-name "/usr/local/src/org-mode/lisp"))

   (org-babel-do-load-languages 'org-babel-load-languages '(
        (R . t)
        (shell . t)
        ))

   (add-to-list 'load-path "~/.emacs.d/elpa/ess-20221121.1627") ;;  "/usr/local/src/ESS/lisp"
   (require 'ess-r-mode)
# -----

For me this runs Org from the source tree (up to date today) and ESS from where 
it was installed (package updated yesterday, as it happens).  Adjust paths as 
necessary.

Next, here is r.org, with one R code block that makes a simple chart:

# ----- r.org
   #+begin_src R :results graphics file :file /tmp/plot.png :width 300 :height 300
   plot(c(1, 2, 3, 4), c(1, 8, 27, 64))
   #+end_src
# -----

Run Emacs:

$ emacs -Q -l minimal-init.el

In Emacs:

+ C-x C-f (load r.org wherever it is)
+ C-c C-c (to execute the R code)
+ Say yes to "Evaluate this R code block on your system?"
+ #RESULTS block appears (good) but image does not show
+ C-c C-x C-v (to toggle image display)
+ image appears
+ C-c C-c (to execute R code again)
+ Say yes to "Evaluate this R code block on your system?"
+ image disappears
+ C-c C-x C-v says "Inline image display turned off"
+ C-c C-x C-v says "Inline image display turned off" (and so on)

Expected behaviour:  Images should appear when this code block is run or rerun, 
and C-c C-X C-v (org-toggle-inline-images) should toggle the setting, not keep 
it off.

Thanks,

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

> In Emacs:
>
> + C-x C-f (load r.org wherever it is)
> + C-c C-c (to execute the R code)
> + Say yes to "Evaluate this R code block on your system?"
> + #RESULTS block appears (good) but image does not show
> + C-c C-x C-v (to toggle image display)
> + image appears
> + C-c C-c (to execute R code again)
> + Say yes to "Evaluate this R code block on your system?"
> + image disappears
> + C-c C-x C-v says "Inline image display turned off"
> + C-c C-x C-v says "Inline image display turned off" (and so on)
>
> Expected behaviour:  Images should appear when this code block is run or rerun, 
> and C-c C-X C-v (org-toggle-inline-images) should toggle the setting, not keep 
> it off.

I am currently not able to test third-party packages in clear
environment, but I think I know what could have caused the problem.

I think I fixed the issue on main.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d4522dd4d

Can you try the latest main and see if the problem persists?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

Thanks for looking at this.  You've fixed the org-toggle-inline-images problem, 
but the image generation and display problem is still there for me.

Here's a one-liner shell block that generates an image with Imagemagick, so no 
extra Emacs packages necessary:

# -----

#+begin_src sh :results graphics file :file /tmp/colour.png
convert -size 300x300 xc:#002b36 /tmp/colour.png
#+end_src

# -----

But the image doesn't appear right away:

+ C-x C-f (load this file)
+ C-c C-c (to execute)
+ Say yes to "Evalute this sh code block on your system?"
+ #RESULTS block appears (good) but image does not show
+ But: C-c C-x C-v does work to toggle image display (fixed!)

This is the simplest way I could think of to generate an image with minimal 
dependencies, so I hope this boils the problem down in a helpful way.  The image 
should appear immediately in the buffer, shouldn't it?


Thanks,

Bill


On 24 November 2022, Ihor Radchenko wrote:

> William Denton <wtd@pobox.com> writes:
>
>> In Emacs:
>>
>> + C-x C-f (load r.org wherever it is)
>> + C-c C-c (to execute the R code)
>> + Say yes to "Evaluate this R code block on your system?"
>> + #RESULTS block appears (good) but image does not show
>> + C-c C-x C-v (to toggle image display)
>> + image appears
>> + C-c C-c (to execute R code again)
>> + Say yes to "Evaluate this R code block on your system?"
>> + image disappears
>> + C-c C-x C-v says "Inline image display turned off"
>> + C-c C-x C-v says "Inline image display turned off" (and so on)
>>
>> Expected behaviour:  Images should appear when this code block is run or rerun,
>> and C-c C-X C-v (org-toggle-inline-images) should toggle the setting, not keep
>> it off.
>
> I am currently not able to test third-party packages in clear
> environment, but I think I know what could have caused the problem.
>
> I think I fixed the issue on main.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d4522dd4d
>
> Can you try the latest main and see if the problem persists?
>
>

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

> #+begin_src sh :results graphics file :file /tmp/colour.png
> convert -size 300x300 xc:#002b36 /tmp/colour.png
> #+end_src
>
> # -----
>
> But the image doesn't appear right away:
>
> + C-x C-f (load this file)
> + C-c C-c (to execute)
> + Say yes to "Evalute this sh code block on your system?"
> + #RESULTS block appears (good) but image does not show
> + But: C-c C-x C-v does work to toggle image display (fixed!)
>
> This is the simplest way I could think of to generate an image with minimal 
> dependencies, so I hope this boils the problem down in a helpful way.  The image 
> should appear immediately in the buffer, shouldn't it?

No, it should not.
Though it is often requested.

I personally just do it via hook:

(add-hook! 'org-babel-after-execute-hook
  (unless (eq this-command 'org-babel-tangle)
    (org-display-inline-images nil nil
                               (save-excursion (org-back-to-heading-or-point-min t))
                               (save-excursion (or (outline-next-heading) (point-max))))))

Maybe one day someone™ can submit a patch ;)

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

On 24 November 2022, Ihor Radchenko wrote:

>> The image should appear immediately in the buffer, shouldn't it?
>
> No, it should not.
> Though it is often requested.

Huh, strange.  It's always done it for me, at least with R code blocks, as far 
as I can remember.  But back to my original problem!

Here's another angle on it, with images not being refreshed even though they've 
changed.

Setup:  my minimal-init.el from my first email.

File, r.org:

# -----

#+begin_src R :results graphics file :file /tmp/plot.png :width 300 :height 300
plot(c(1, 2, 3, 4), c(1, 8, 27, 64))
#+end_src

# -----

Steps:

+ C-x C-f (load r.org)
+ C-c C-c (execute R block)
+ answer yes to "Evaluate this?" question
+ #RESULTS appears, but image doesn't show
+ C-c C-x C-v (to see image)
+ look at /tmp/plot.png: it is 300 x 300
+ edit width and height settings so they are each 500 pixels
+ C-c C-c (execute R block again)
+ answer yes to "Evaluate this?" question
+ image disappears
+ C-c C-x C-v (to see image)
+ image is still 300 pixels square! 
+ look at /tmp/plot.png: it is 500 x 500
+ C-x C-s (save file r.org)
+ quit and restart Emacs with minimal init
+ C-x C-f (load r.org)
+ C-c C-x C-v (to see image)
+ image is 500 x 500!

So R is generating a new, larger image, and it's there on disk, but Emacs won't 
show it.

Thanks again,

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

> + C-c C-c (execute R block again)
> + answer yes to "Evaluate this?" question
> + image disappears
> + C-c C-x C-v (to see image)
> + image is still 300 pixels square! 

What if you run M-: (clear-image-cache) before C-c C-x C-v?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

On 24 November 2022, Ihor Radchenko wrote:

> William Denton <wtd@pobox.com> writes:
>
>> + C-c C-c (execute R block again)
>> + answer yes to "Evaluate this?" question
>> + image disappears
>> + C-c C-x C-v (to see image)
>> + image is still 300 pixels square!
>
> What if you run M-: (clear-image-cache) before C-c C-x C-v?

That does it!


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

>> What if you run M-: (clear-image-cache) before C-c C-x C-v?
>
> That does it!

Strange.
I am unable to reproduce the issue on my side (using the earlier bash
code block you provided).

What is your Emacs version?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

On 24 November 2022, Ihor Radchenko wrote:

> William Denton <wtd@pobox.com> writes:
>
>>> What if you run M-: (clear-image-cache) before C-c C-x C-v?
>>
>> That does it!
>
> Strange.
> I am unable to reproduce the issue on my side (using the earlier bash
> code block you provided).
>
> What is your Emacs version?

I refreshed the source just now, and it's at commit 183c66be97c:

GNU Emacs 29.0.50 (build 7, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0) of 2022-11-23

I reran the convert image command, and the (clear-image-cache) problem is still 
there.  If you generate an image that's 300x300, then change the command so it 
makes a 500x500 image, and rerun the command, a 500x500 image appears right 
away?  Or does after you C-c C-x C-v?  I have to run M-: (clear-image-cache) and 
then the image in the buffer updates to the larger one.


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

> I refreshed the source just now, and it's at commit 183c66be97c:
>
> GNU Emacs 29.0.50 (build 7, x86_64-pc-linux-gnu, GTK+ Version 3.24.33, cairo version 1.16.0) of 2022-11-23
>
> I reran the convert image command, and the (clear-image-cache) problem is still 
> there.  If you generate an image that's 300x300, then change the command so it 
> makes a 500x500 image, and rerun the command, a 500x500 image appears right 
> away?  Or does after you C-c C-x C-v?  I have to run M-: (clear-image-cache) and 
> then the image in the buffer updates to the larger one.

I tried again now, and I am able to reproduce.
This is very strange.

What I did is

1. cd /path/to/org/repo
2. git checkout main
3. make repro
4. M-: (require 'ob-shell)
5. Open the following org file

#+begin_src sh :results graphics file :file /tmp/colour.png
convert -size 300x300 xc:#002b36 /tmp/colour.png
#+end_src

6. Move point to the source block and C-c C-c yes
7. C-c C-x C-v
8. Observe the 300x300 image appearing
9. Edit the source code to 500x300
10. C-c C-c yes
11. Image disappears
12. C-c C-x C-v
13. Observe what appears to be the old 300x300 image
14. Move the cursor to the image
15. Observe the correct 500x300 appearing while the cursor is on it!!

Could you please confirm?

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[Jeremie Juste]

Hello Ihor,


On Friday, 25 Nov 2022 at 02:03, Ihor Radchenko wrote:
>
> What I did is
>
> 1. cd /path/to/org/repo
> 2. git checkout main
> 3. make repro
> 4. M-: (require 'ob-shell)
> 5. Open the following org file
>
> #+begin_src sh :results graphics file :file /tmp/colour.png
> convert -size 300x300 xc:#002b36 /tmp/colour.png
> #+end_src
>
> 6. Move point to the source block and C-c C-c yes
> 7. C-c C-x C-v
> 8. Observe the 300x300 image appearing
> 9. Edit the source code to 500x300
> 10. C-c C-c yes
> 11. Image disappears
> 12. C-c C-x C-v
> 13. Observe what appears to be the old 300x300 image
> 14. Move the cursor to the image
> 15. Observe the correct 500x300 appearing while the cursor is on it!!
>
I tried to reproduce your steps on Org mode version 9.6-pre
(release_9.5.5-1096-gf83e45.dirty @ /home/djj/src/org-mode/lisp/).

and I get stuck at step 12, Inline display image cannot be switched on
again.

Hope this helps,

Jeremie

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

Jeremie Juste <jeremiejuste@gmail.com> writes:

> I tried to reproduce your steps on Org mode version 9.6-pre
> (release_9.5.5-1096-gf83e45.dirty @ /home/djj/src/org-mode/lisp/).
>
> and I get stuck at step 12, Inline display image cannot be switched on
> again.

Not switching on again has been fixed in
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d4522dd4d
(see the previous replies in this thread)

You need a newer Org version.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[Jeremie Juste]

Hello Ihor,


On Sunday, 27 Nov 2022 at 13:21, Ihor Radchenko wrote:
> Jeremie Juste <jeremiejuste@gmail.com> writes:
>
>> I tried to reproduce your steps on Org mode version 9.6-pre
>> (release_9.5.5-1096-gf83e45.dirty @ /home/djj/src/org-mode/lisp/).
>>
>> and I get stuck at step 12, Inline display image cannot be switched on
>> again.
>
> Not switching on again has been fixed in
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=d4522dd4d
> (see the previous replies in this thread)
>
> You need a newer Org version.
Sorry, Yes I realised that I hadn't pulled.

I confirm the behaviour with the version

Org mode version 9.6-pre (release_9.5.5-1158-g927621 @
/home/djj/src/org-mode/lisp/)

Best regards,
Jeremie

Re: Images generated by R code blocks do not display

[William Denton]

On 25 November 2022, Ihor Radchenko wrote:

> 1. cd /path/to/org/repo
> 2. git checkout main
> 3. make repro
> 4. M-: (require 'ob-shell)
> 5. Open the following org file
>
> #+begin_src sh :results graphics file :file /tmp/colour.png
> convert -size 300x300 xc:#002b36 /tmp/colour.png
> #+end_src
>
> 6. Move point to the source block and C-c C-c yes
> 7. C-c C-x C-v
> 8. Observe the 300x300 image appearing
> 9. Edit the source code to 500x300
> 10. C-c C-c yes
> 11. Image disappears
> 12. C-c C-x C-v
> 13. Observe what appears to be the old 300x300 image
> 14. Move the cursor to the image
> 15. Observe the correct 500x300 appearing while the cursor is on it!!
>
> Could you please confirm?

Confirmed!  Just as Jeremie did also.  I'd seen this image-switching behaviour 
before and hadn't been able to recreate it with a simple example, but there it 
is.  "This is very strange," indeed.

I don't know if this is useful, but:

16.  Edit source code back to 300x300
17.  C-c C-c yes
18.  Image disappears
19.  C-c C-x C-v
20.  300x300 image appears (a new one?  or the first one?)
21.  Move the cursor to the image
22.  The now incorrect 500x300 appears while the cursor is on it, same as 15

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[William Denton]

I tried git bisect and, if I did it right, the problem may have started here:

commit 46b71f910844c14d8db1feb54c07de26d101cc05
Author: stardiviner <numbchild@gmail.com>
Date:   Tue Oct 4 12:36:32 2022 +0800

     org.el: Support auto display inline images when cycling

     * lisp/org.el (org-toggle-inline-images): Support region.
     (org-display-inline-images): Fix refresh argument logic.
     (org-remove-inline-images): Support region.
     ...

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

William Denton <wtd@pobox.com> writes:

>> Could you please confirm?
>
> Confirmed!

I now reported this to Emacs upstream.
I do not see anything wrong on Org side in this case.
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59902

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

On 8 December 2022, Ihor Radchenko wrote:

> I now reported this to Emacs upstream.
> I do not see anything wrong on Org side in this case.
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59902

Thanks for reporting this there.  I like how you simplified the bug example with 
the two images, so nothing needs to be generated on the fly.

As a reminder for anyone else having this problem, where updated images don't 
appear, this will refresh them:

M-: (clear-image-cache)


Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

Re: Images generated by R code blocks do not display

[Ihor Radchenko]

Ihor Radchenko <yantar92@posteo.net> writes:

> I now reported this to Emacs upstream.
> I do not see anything wrong on Org side in this case.
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59902

Since the bug will affect older Emacs versions, I pushed a workaround to
bugfix.
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=7fefc3027

It will no longer be necessary when Emacs fixes the issue and when we
stop supporting all the older Emacs versions with this bug being present.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>

Re: Images generated by R code blocks do not display

[William Denton]

On 10 December 2022, Ihor Radchenko wrote:

> Ihor Radchenko <yantar92@posteo.net> writes:
>
>> I now reported this to Emacs upstream.
>> I do not see anything wrong on Org side in this case.
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=59902
>
> Since the bug will affect older Emacs versions, I pushed a workaround to
> bugfix.
> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=7fefc3027
>
> It will no longer be necessary when Emacs fixes the issue and when we
> stop supporting all the older Emacs versions with this bug being present.

Thank you!  It's so nice to have this working again.  I have a couple of Org 
files where I'm updating data and images a few times a week, and when this broke 
it was a real pain---particularly because everything had worked so well for so 
long.  Now my life is easy again.  I appreciate your work on this and everything 
else, Ihor.

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada