Generated graphic not included in exported TeX/PDF

Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Hi,

I have just re-exported an org beamer presentation, which I created two
years ago with a version of Org which was fairly up-to-date then.
Everything still works fine, except for the fact that the images
generated by the org file are not included in the exported PDF and
indeed are missing from the generated TeX file.

In the org file I have

  #+NAME: backfill-tikz
  #+HEADER: :fit yes :file ./generated/backfill_tikz.png :imagemagick yes :iminoptions -density 600
  #+HEADER: :headers '("\\usepackage{pgfplots}\\usepackage{sansmath}")
  #+HEADER: :results raw
  #+BEGIN_SRC latex :cache yes
    \usepgfplotslibrary{colorbrewer}
    \begin{tikzpicture}
    ... 
    \end{tikzpicture}
  #+END_SRC

  #+RESULTS[e2d28096fe9dceffb0b63ad4a441160ff2a0d1a7]: backfill-tikz
  [[file:./generated/backfill_tikz.png]]

As I say, the image itself is created, but not included when exported, although it is
displayed if I do org-toggle-inline-images in the org buffer.

My current Org version is

  Org mode version 9.4 (9.4-9-gab0052-elpaplus @ /home/loris/.emacs.d/elpa/org-plus-contrib-20200928/)

Any ideas what might be happening?

Cheers,

Loris

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

"Loris Bennett" <loris.bennett@fu-berlin.de> writes:

> Hi,
>
> I have just re-exported an org beamer presentation, which I created two
> years ago with a version of Org which was fairly up-to-date then.
> Everything still works fine, except for the fact that the images
> generated by the org file are not included in the exported PDF and
> indeed are missing from the generated TeX file.
>
> In the org file I have
>
>   #+NAME: backfill-tikz
>   #+HEADER: :fit yes :file ./generated/backfill_tikz.png :imagemagick yes :iminoptions -density 600
>
>   #+HEADER: :headers '("\\usepackage{pgfplots}\\usepackage{sansmath}")
>   #+HEADER: :results raw
>
>   #+BEGIN_SRC latex :cache yes
>     \usepgfplotslibrary{colorbrewer}
>     \begin{tikzpicture}
>     ... 
>     \end{tikzpicture}
>   #+END_SRC
>
>   #+RESULTS[e2d28096fe9dceffb0b63ad4a441160ff2a0d1a7]: backfill-tikz
>   [[file:./generated/backfill_tikz.png]]
>
> As I say, the image itself is created, but not included when exported, although it is
> displayed if I do org-toggle-inline-images in the org buffer.
>
> My current Org version is
>
>   Org mode version 9.4 (9.4-9-gab0052-elpaplus @ /home/loris/.emacs.d/elpa/org-plus-contrib-20200928/)
>
> Any ideas what might be happening?

I have just noticed that this only affects images generated with tikz.
One image created with ditaa is included correctly.  All images are PNGs. 

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

"Loris Bennett" <loris.bennett@fu-berlin.de> writes:

> "Loris Bennett" <loris.bennett@fu-berlin.de> writes:
>
>> Hi,
>>
>> I have just re-exported an org beamer presentation, which I created two
>> years ago with a version of Org which was fairly up-to-date then.
>> Everything still works fine, except for the fact that the images
>> generated by the org file are not included in the exported PDF and
>> indeed are missing from the generated TeX file.
>>
>> In the org file I have
>>
>>   #+NAME: backfill-tikz
>>   #+HEADER: :fit yes :file ./generated/backfill_tikz.png :imagemagick yes :iminoptions -density 600
>>
>>   #+HEADER: :headers '("\\usepackage{pgfplots}\\usepackage{sansmath}")
>>   #+HEADER: :results raw
>>
>>   #+BEGIN_SRC latex :cache yes
>>     \usepgfplotslibrary{colorbrewer}
>>     \begin{tikzpicture}
>>     ... 
>>     \end{tikzpicture}
>>   #+END_SRC
>>
>>   #+RESULTS[e2d28096fe9dceffb0b63ad4a441160ff2a0d1a7]: backfill-tikz
>>   [[file:./generated/backfill_tikz.png]]
>>
>> As I say, the image itself is created, but not included when exported, although it is
>> displayed if I do org-toggle-inline-images in the org buffer.
>>
>> My current Org version is
>>
>>   Org mode version 9.4 (9.4-9-gab0052-elpaplus @ /home/loris/.emacs.d/elpa/org-plus-contrib-20200928/)
>>
>> Any ideas what might be happening?
>
> I have just noticed that this only affects images generated with tikz.
> One image created with ditaa is included correctly.  All images are PNGs. 

Just updated to 

  Org mode version 9.4.4 (9.4.4-4-g99eafe-elpa @ /home/loris/.emacs.d/elpa/org-20210104/)

but the problem remains.

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

"Loris Bennett" <loris.bennett@fu-berlin.de> writes:

> "Loris Bennett" <loris.bennett@fu-berlin.de> writes:
>
>> "Loris Bennett" <loris.bennett@fu-berlin.de> writes:
>>
>>> Hi,
>>>
>>> I have just re-exported an org beamer presentation, which I created two
>>> years ago with a version of Org which was fairly up-to-date then.
>>> Everything still works fine, except for the fact that the images
>>> generated by the org file are not included in the exported PDF and
>>> indeed are missing from the generated TeX file.
>>>
>>> In the org file I have
>>>
>>>   #+NAME: backfill-tikz
>>>   #+HEADER: :fit yes :file ./generated/backfill_tikz.png :imagemagick yes :iminoptions -density 600
>>>
>>>   #+HEADER: :headers '("\\usepackage{pgfplots}\\usepackage{sansmath}")
>>>   #+HEADER: :results raw
>>>
>>>   #+BEGIN_SRC latex :cache yes
>>>     \usepgfplotslibrary{colorbrewer}
>>>     \begin{tikzpicture}
>>>     ... 
>>>     \end{tikzpicture}
>>>   #+END_SRC
>>>
>>>   #+RESULTS[e2d28096fe9dceffb0b63ad4a441160ff2a0d1a7]: backfill-tikz
>>>   [[file:./generated/backfill_tikz.png]]
>>>
>>> As I say, the image itself is created, but not included when exported, although it is
>>> displayed if I do org-toggle-inline-images in the org buffer.
>>>
>>> My current Org version is
>>>
>>>   Org mode version 9.4 (9.4-9-gab0052-elpaplus @ /home/loris/.emacs.d/elpa/org-plus-contrib-20200928/)
>>>
>>> Any ideas what might be happening?
>>
>> I have just noticed that this only affects images generated with tikz.
>> One image created with ditaa is included correctly.  All images are PNGs. 
>
> Just updated to 
>
>   Org mode version 9.4.4 (9.4.4-4-g99eafe-elpa @ /home/loris/.emacs.d/elpa/org-20210104/)
>
> but the problem remains.

As I was having problems with the caching of the images, I replaced

  #+BEGIN_SRC latex :cache yes

with

  #+BEGIN_SRC latex :cache yes :eval never-export

and now the images are not only not regenerated but are now also
exported properly.

Is this a bug?

Cheers,

Loris

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

I haven't tried your example but maybe add an explicit ":exports
results" to the src block?
-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-166-g291993

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 565 bytes --]

Loris (& the list),

did you get this sorted?  It seems that I am running into the same
problem now.

What seems to be happening is that the LaTeX file is created in
/tmp/babel-XXXXXX/latex-YYYYYY.tex but the log, aux, etc. files are
being created in the current directory (where the file with the LaTeX
src block is).  Although the PDF file is created properly, it is not
where org would expect it to be (maybe)?

The LaTeX file in /tmp/ compiles perfectly fine when I compile it
manually.  Attached is /tmp/ecm.org and here is a screenshot of my /tmp
directory:


[-- Attachment #2: Type: image/png, Size: 54795 bytes --]

[-- Attachment #3: Type: text/plain, Size: 316 bytes --]


Note that latex-iLJhcd.tex is in /tmp/babel-falBv7/ but the
{log,pdf,out} files are in /tmp/.

The backtrace on error is also attached.

This has been done with org from git updated a few minutes ago.  Emacs
from a few days ago.

Thank you,
eric

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: ecm.org --]
[-- Type: text/x-org, Size: 534 bytes --]

title: ECM
* black box model
This is a graphical representation of \(y=f(x)\):

#+header: :exports results
#+header: :results raw
#+header: :file blackboxmodel.png
#+header: :imagemagick yes :iminoptions -density 600 :imoutoptions -geometry 400
#+header: :fit yes
#+begin_src latex
\begin{tikzpicture}[node distance=2cm, shorten >=2pt, shorten <=2pt]
  \node (x) {$x$};
  \node[fill=yellow!10!black,draw=white] (box) [right of=x] {Model} edge [<-] (x);
  \node (f) [right of=box] {$f(x)$} edge [<-] (box);
\end{tikzpicture}
#+end_src

[-- Attachment #5: trace.txt --]
[-- Type: text/plain, Size: 1702 bytes --]

Debugger entered--Lisp error: (error "File \"/tmp/babel-falBv7/latex-iLJhcd.pdf\" wasn’t p...")
  signal(error ("File \"/tmp/babel-falBv7/latex-iLJhcd.pdf\" wasn’t p..."))
  error("File \"/tmp/babel-falBv7/latex-iLJhcd.pdf\" wasn't p...")
  org-compile-file("/tmp/babel-falBv7/latex-iLJhcd.tex" ("pdflatex %f" "bibtex %b" "pdflatex %f" "pdflatex %f") "pdf" "See \"*Org PDF LaTeX Output*\" for details" #<buffer *Org PDF LaTeX Output*> ((66 . "bibtex") (76 . "pdflatex")))
  org-latex-compile("/tmp/babel-falBv7/latex-iLJhcd.tex")
  org-babel-latex-tex-to-pdf("/tmp/babel-falBv7/latex-iLJhcd.tex")
  org-babel-execute:latex("\\begin{tikzpicture}[node distance=2cm, shorten >=2..." ((:colname-names) (:rowname-names) (:result-params "replace" "raw") (:result-type . value) (:results . "replace raw") (:exports . "results") (:session . "none") (:cache . "no") (:noweb . "no") (:hlines . "no") (:tangle . "no") (:fit . "yes") (:imagemagick . "yes") (:iminoptions . "-density 600") (:imoutoptions . "-geometry 400") (:file . "blackboxmodel.png")))
  org-babel-execute-src-block(nil ("latex" "\\begin{tikzpicture}[node distance=2cm, shorten >=2..." ((:colname-names) (:rowname-names) (:result-params "raw" "replace") (:result-type . value) (:results . "raw replace") (:exports . "results") (:file . "blackboxmodel.png") (:imoutoptions . "-geometry 400") (:iminoptions . "-density 600") (:imagemagick . "yes") (:fit . "yes") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no") (:session . "none")) "" nil 265 "(ref:%s)"))
  org-ctrl-c-ctrl-c(nil)
  funcall-interactively(org-ctrl-c-ctrl-c nil)
  call-interactively(org-ctrl-c-ctrl-c nil nil)
  command-execute(org-ctrl-c-ctrl-c)

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

Answering my own email.

Turns out the problem was that I changed the default value of
org-latex-pdf-process and no longer had the "-output-directory %o" part
of the command.  For some reason, this has not been a problem until
recently.  Maybe just pure luck.

So, maybe check your value of org-latex-pdf-process.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Answering my own email.
>
> Turns out the problem was that I changed the default value of
> org-latex-pdf-process and no longer had the "-output-directory %o" part
> of the command.  For some reason, this has not been a problem until
> recently.  Maybe just pure luck.
>
> So, maybe check your value of org-latex-pdf-process.

Interesting.  I have

  latexmk -g -pdf -pdflatex=\"%latex\" -shell-escape -outdir=%o %f

which I don't think has changed recently.

However, since with

  #+BEGIN_SRC latex :cache yes :eval never-export

caching, not exporting, and including the graphics all work, I not sure
how to debug the problem.  It is not at all clear to me how these three
things are connected.

Cheers,

Loris

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

Does my ecm.org (in the email I sent to you & list on Saturday) work for you?
-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> Does my ecm.org (in the email I sent to you & list on Saturday) work for you?

Oh, sorry, I overlooked that.  My org-latex-packages-alist is currently
empty, so if I add

  #+header: :headers '("\\usepackage{tikz}")

I can produce the PNG and PDF files, but the image is still not included in
the PDF.

Despite being subject to a bit of a clean-up when I moved from Emacs 25
to Emacs 26, my init.el is still a bit crufty, so I tried with a fairly
minimal config with no org customization, but still using Org 9.4.4 and
got the same result: no error but image not included in PDF.

Cheers,

Loris

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

But what happens if you try my ecm as it is?  Does it export?
-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> But what happens if you try my ecm as it is?  Does it export?

Yes, it exports.  However although the diagram is genererated, it is not
included in the PDF.

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

On Monday, 11 Jan 2021 at 14:08, Loris Bennett wrote:
> Yes, it exports.  However although the diagram is genererated, it is not
> included in the PDF.

By "included", what do you mean?  If I export that document to HTML, I
see a link to the PDF.  If I ask for a PNG, I see the image.  This
behaviour is expected as browsers do not show PDF documents inline
generally.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Monday, 11 Jan 2021 at 14:08, Loris Bennett wrote:
>> Yes, it exports.  However although the diagram is genererated, it is not
>> included in the PDF.
>
> By "included", what do you mean?  If I export that document to HTML, I
> see a link to the PDF.  If I ask for a PNG, I see the image.  This
> behaviour is expected as browsers do not show PDF documents inline
> generally.

Sorry for not being clear.

I'm only exporting to PDF not HTML and I am creating the image in the
source block as a PNG.  I am then viewing the resulting PDF file either
using a PDF viewer (evince) or directly in Emacs.  The file
blackboxmodel.png is created, but not inserted into the final ecm.pdf.

Adding

 :cache yes :eval never-export

in cargo-cult fashion doesn't solve the problem when using with the ECM
and a minimal Emacs set-up although it does with my complete
presentation and my standard init.el.
 
-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

On Monday, 11 Jan 2021 at 15:03, Loris Bennett wrote:
> I'm only exporting to PDF not HTML and I am creating the image in the
> source block as a PNG.  I am then viewing the resulting PDF file either
> using a PDF viewer (evince) or directly in Emacs.  The file
> blackboxmodel.png is created, but not inserted into the final ecm.pdf.

Okay, thanks for clarifying.  If I export ecm.org to PDF via LaTeX,
asking for a PNG image, it works just fine for me.  Strange.

Hopefully somebody else can chime in and maybe give some clue as to
where to look.  I've attached the ecm.org file again.  This exports to
both article and beamer PDF and, for that matter, to HTML.

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: ecm.org --]
[-- Type: text/x-org, Size: 584 bytes --]

#+title: ECM
* black box model
This is a graphical representation of \(y=f(x)\):

#+header: :exports results
#+header: :results file raw
#+header: :file blackboxmodel.png
#+header: :imagemagick yes :iminoptions -density 600 :imoutoptions -geometry 400
#+header: :fit yes
#+header: :headers '("\\usepackage{tikz}")
#+begin_src latex
\begin{tikzpicture}[node distance=2cm, shorten >=2pt, shorten <=2pt]
  \node (x) {$x$};
  \node[fill=yellow!10!white,draw=white] (box) [right of=x] {Model} edge [<-] (x);
  \node (f) [right of=box] {$f(x)$} edge [<-] (box);
\end{tikzpicture}
#+end_src

Re: Generated graphic not included in exported TeX/PDF

[Loris Bennett]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Monday, 11 Jan 2021 at 15:03, Loris Bennett wrote:
>> I'm only exporting to PDF not HTML and I am creating the image in the
>> source block as a PNG.  I am then viewing the resulting PDF file either
>> using a PDF viewer (evince) or directly in Emacs.  The file
>> blackboxmodel.png is created, but not inserted into the final ecm.pdf.
>
> Okay, thanks for clarifying.  If I export ecm.org to PDF via LaTeX,
> asking for a PNG image, it works just fine for me.  Strange.
>
> Hopefully somebody else can chime in and maybe give some clue as to
> where to look.  I've attached the ecm.org file again.  This exports to
> both article and beamer PDF and, for that matter, to HTML.

With 

  #+header: :results file raw

rather than just

  #+header: :results raw

it works for me too.  Thanks!

The workaround with

  :cache yes :eval never-export

seems just to have prevented an existing 

  file:blackboxmodel.png

from getting clobbered by evaluating the source block without :file.  

Cheers,

Loris

-- 
This signature is currently under construction.

Re: Generated graphic not included in exported TeX/PDF

[Eric S Fraga]

On Monday, 11 Jan 2021 at 16:36, Loris Bennett wrote:
> With 
>   #+header: :results file raw
> it works for me too.  Thanks!

Excellent!  Whew. :-)

-- 
: Eric S Fraga via Emacs 28.0.50, Org release_9.4.4-179-g5b5c42