BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[Emmanuel Charpentier]

I shall report formally the problems mentioned in 
http://lists.gnu.org/archive/html/emacs-orgmode/2019-04/msg00240.html,
trying to isolate them as much as possible. I'll start with the
simplest (and, IMHO, the most annoying).

Whereas the HTML exporter correctly recognize a source block as such,
the LaTeX/PDF exporter does not.

This can be demonstrated in a (minimal) org file using *only* the
built-in tools (no external packages), where a source block is labelled
as such by the HTML exporter but as a figure by the LaTeX exporter.The
generated LaTeX source shows a suspicious
"\captionof{figure}{\label{orgf2d4160}" in the export of the source
block.

Further tests (not shown) show that the proble persists when org-mode
is set up to use the minted package for LaTeX export of source blocks
(minted is indeed used, but the block is s still mislabeled).

This example also exhibits some cosmetic problems (spacing) that will
be ignored for now.

The org source and resulting pdf and html files are available here : 
https://drive.google.com/drive/folders/1bQmaefiztW9ZDRNLcjOD_WbEmTg4d8dG?usp=sharing

Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[Nicolas Goaziou]

Hello,

Emmanuel Charpentier <emm.charpentier@free.fr> writes:

> Whereas the HTML exporter correctly recognize a source block as such,
> the LaTeX/PDF exporter does not.
>
> This can be demonstrated in a (minimal) org file using *only* the
> built-in tools (no external packages), where a source block is labelled
> as such by the HTML exporter but as a figure by the LaTeX exporter.The
> generated LaTeX source shows a suspicious
> "\captionof{figure}{\label{orgf2d4160}" in the export of the source
> block.
>
> Further tests (not shown) show that the proble persists when org-mode
> is set up to use the minted package for LaTeX export of source blocks
> (minted is indeed used, but the block is s still mislabeled).

What LaTeX code do you suggest instead?

> The org source and resulting pdf and html files are available here : 
> https://drive.google.com/drive/folders/1bQmaefiztW9ZDRNLcjOD_WbEmTg4d8dG?usp=sharing

Could you send your ECM in the ML without requiring to access Google
Drive?

Thank you.

Regards,

-- 
Nicolas Goaziou

Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[Emmanuel Charpentier]

[-- Attachment #1: Type: text/plain, Size: 2216 bytes --]

Hi, Nicolas !

Le mardi 30 avril 2019 à 19:05 +0200, Nicolas Goaziou a écrit :
> Hello,
> 
> Emmanuel Charpentier <emm.charpentier@free.fr> writes:
> 
> > Whereas the HTML exporter correctly recognize a source block as
> > such,
> > the LaTeX/PDF exporter does not.
> > 
> > This can be demonstrated in a (minimal) org file using *only* the
> > built-in tools (no external packages), where a source block is
> > labelled
> > as such by the HTML exporter but as a figure by the LaTeX
> > exporter.The
> > generated LaTeX source shows a suspicious
> > "\captionof{figure}{\label{orgf2d4160}" in the export of the source
> > block.
> > 
> > Further tests (not shown) show that the proble persists when org-
> > mode
> > is set up to use the minted package for LaTeX export of source
> > blocks
> > (minted is indeed used, but the block is s still mislabeled).
> 
> What LaTeX code do you suggest instead?

Dunno. Maybe \captionof{listing} ? Or let minted do its thing ? Anyway,
in most cases, the code excerpts will be too long for a float ; the
idea is to send them (unfloated but labeled/captioned) in an appendix
and point to them via a link.

[ BTW : we should stop suggesting to use listings, which is problematic
for anything not strictly ASCII, bloody likely to occur at least on a
comment/prompt for most of mankind... Listingsutf8 is but a patch
working only for (a subset of) European languages. Arabic, Hebrew,
Asian languages : nope. Minted in (xe|lua)tex is probably more usable
(but needs --shell-escape, which can justifiably frowned upon... ].

> > The org source and resulting pdf and html files are available here
> > : 
> > https://drive.google.com/drive/folders/1bQmaefiztW9ZDRNLcjOD_WbEmTg4d8dG?usp=sharing
> 
> Could you send your ECM in the ML without requiring to access Google
> Drive?

Attached.

But please coordinate with Bastien, who asked me just yesterday to
avoid posting fat attachments to a message going to a lot of people.
Which I did (to no avail : the "heavy" mail went through anyway. Go
figure.... ;-).


> Thank you.

You're welcome ;-). Thank YOU for considering scratching *MY* itch...

--
Emmanuel Charpentier


[-- Attachment #2: C1.html --]
[-- Type: text/html, Size: 10176 bytes --]

[-- Attachment #3: C1.org --]
[-- Type: text/plain, Size: 774 bytes --]

# An archi-minimal test of source blocks labelling,
# using only the built-in tools.

#+options: tex:t title:nil author:nil date:nil toc:nil
#+property: header-args:R :session
#+title: Test of source block exports

The R block [[lst:DoFig]] generates the figure [[fig:DoFig]]:

#+name: DoFig
#+begin_src R :exports results :results output graphics :file Fig.png
curve(sin(x),from=-pi, to=pi)
#+end_src

#+name: fig:DoFig
#+caption: Figure generated by R
#+RESULTS: DoFig
[[file:Fig.png]]

#+name: lst:DoFig
#+caption: R code generating a figure
#+begin_src R :eval no :exports code :noweb yes
<<DoFig>>
#+end_src

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it).

[-- Attachment #4: C1.pdf --]
[-- Type: application/pdf, Size: 55457 bytes --]

[-- Attachment #5: C1.tex --]
[-- Type: text/x-tex, Size: 1168 bytes --]

% Created 2019-04-30 mar. 09:10
% Intended LaTeX compiler: pdflatex
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\date{}
\title{Test of source block exports}
\hypersetup{
 pdfauthor={Emmanuel Charpentier},
 pdftitle={Test of source block exports},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.1 (Org mode 9.2.3)}, 
 pdflang={English}}
\begin{document}

The R block \ref{orgf2d4160} generates the figure \ref{fig:org1c384b0}:

\begin{figure}[htbp]
\centering
\includegraphics[width=.9\linewidth]{Fig.png}
\caption{\label{fig:org1c384b0}
Figure generated by R}
\end{figure}

\begin{verbatim}
curve(sin(x),from=-pi, to=pi)
\end{verbatim}
\captionof{figure}{\label{orgf2d4160}
R code generating a figure}

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it).
\end{document}

[-- Attachment #6: Fig.png --]
[-- Type: image/png, Size: 7444 bytes --]

Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[John Kitchin]

[-- Attachment #1: Type: text/plain, Size: 3281 bytes --]

This works as expected in LaTeX for me with org-ref (you could probably use
raw latex if you wanted):

#+caption: The first source block. label:src-1
#+BEGIN_SRC ipython
print('Hello world')
#+END_SRC


Later we refer to the code in Listing ref:src-1 or ref:src-2. That is
different than Figure ref:fig-1.

#+caption: My figure. label:fig-1
[[./test.png]]


#+caption: The first source block. label:src-2
#+BEGIN_SRC ipython
print('Hello world')
#+END_SRC

The downside is that for html the src-block links are not numbered. I don't
know how easy that would be to fix in the export functions.

John

-----------------------------------
Professor John Kitchin
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Tue, Apr 30, 2019 at 1:40 PM Emmanuel Charpentier <
emm.charpentier@free.fr> wrote:

> Hi, Nicolas !
>
> Le mardi 30 avril 2019 à 19:05 +0200, Nicolas Goaziou a écrit :
> > Hello,
> >
> > Emmanuel Charpentier <emm.charpentier@free.fr> writes:
> >
> > > Whereas the HTML exporter correctly recognize a source block as
> > > such,
> > > the LaTeX/PDF exporter does not.
> > >
> > > This can be demonstrated in a (minimal) org file using *only* the
> > > built-in tools (no external packages), where a source block is
> > > labelled
> > > as such by the HTML exporter but as a figure by the LaTeX
> > > exporter.The
> > > generated LaTeX source shows a suspicious
> > > "\captionof{figure}{\label{orgf2d4160}" in the export of the source
> > > block.
> > >
> > > Further tests (not shown) show that the proble persists when org-
> > > mode
> > > is set up to use the minted package for LaTeX export of source
> > > blocks
> > > (minted is indeed used, but the block is s still mislabeled).
> >
> > What LaTeX code do you suggest instead?
>
> Dunno. Maybe \captionof{listing} ? Or let minted do its thing ? Anyway,
> in most cases, the code excerpts will be too long for a float ; the
> idea is to send them (unfloated but labeled/captioned) in an appendix
> and point to them via a link.
>
> [ BTW : we should stop suggesting to use listings, which is problematic
> for anything not strictly ASCII, bloody likely to occur at least on a
> comment/prompt for most of mankind... Listingsutf8 is but a patch
> working only for (a subset of) European languages. Arabic, Hebrew,
> Asian languages : nope. Minted in (xe|lua)tex is probably more usable
> (but needs --shell-escape, which can justifiably frowned upon... ].
>
> > > The org source and resulting pdf and html files are available here
> > > :
> > >
> https://drive.google.com/drive/folders/1bQmaefiztW9ZDRNLcjOD_WbEmTg4d8dG?usp=sharing
> >
> > Could you send your ECM in the ML without requiring to access Google
> > Drive?
>
> Attached.
>
> But please coordinate with Bastien, who asked me just yesterday to
> avoid posting fat attachments to a message going to a lot of people.
> Which I did (to no avail : the "heavy" mail went through anyway. Go
> figure.... ;-).
>
>
> > Thank you.
>
> You're welcome ;-). Thank YOU for considering scratching *MY* itch...
>
> --
> Emmanuel Charpentier
>
>

[-- Attachment #2: Type: text/html, Size: 4552 bytes --]

Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[Emmanuel Charpentier]

[-- Attachment #1.1: Type: text/plain, Size: 4283 bytes --]

Dear John,
This does *not* work for me. See the attached files.
I begin to suspect a very recent bug : I'm following MELPA, which
updates about once a week. My org-version says: Org mode version 9.2.3
(9.2.3-11-g42abf5-elpaplus @ /home/charpent/.emacs.d/elpa/org-plus-
contrib-20190429/).
Cordially,
--Emmanuel Charpentier
Le mardi 30 avril 2019 à 13:46 -0400, John Kitchin a écrit :
> This works as expected in LaTeX for me with org-ref (you could
> probably use raw latex if you wanted):
> #+caption: The first source block. label:src-1#+BEGIN_SRC
> ipythonprint('Hello world')#+END_SRC
> 
> Later we refer to the code in Listing ref:src-1 or ref:src-2. That is
> different than Figure ref:fig-1.
> #+caption: My figure. label:fig-1[[./test.png]]
> 
> #+caption: The first source block. label:src-2#+BEGIN_SRC
> ipythonprint('Hello world')#+END_SRC
> The downside is that for html the src-block links are not numbered. I
> don't know how easy that would be to fix in the export functions.
> John
> 
> -----------------------------------
> Professor John Kitchin 
> Doherty Hall A207F
> Department of Chemical Engineering
> Carnegie Mellon University
> Pittsburgh, PA 15213
> 412-268-7803
> @johnkitchin
> http://kitchingroup.cheme.cmu.edu
> 
> 
> 
> On Tue, Apr 30, 2019 at 1:40 PM Emmanuel Charpentier <
> emm.charpentier@free.fr> wrote:
> > Hi, Nicolas !
> > 
> > 
> > 
> > Le mardi 30 avril 2019 à 19:05 +0200, Nicolas Goaziou a écrit :
> > 
> > > Hello,
> > 
> > > 
> > 
> > > Emmanuel Charpentier <emm.charpentier@free.fr> writes:
> > 
> > > 
> > 
> > > > Whereas the HTML exporter correctly recognize a source block as
> > 
> > > > such,
> > 
> > > > the LaTeX/PDF exporter does not.
> > 
> > > > 
> > 
> > > > This can be demonstrated in a (minimal) org file using *only*
> > the
> > 
> > > > built-in tools (no external packages), where a source block is
> > 
> > > > labelled
> > 
> > > > as such by the HTML exporter but as a figure by the LaTeX
> > 
> > > > exporter.The
> > 
> > > > generated LaTeX source shows a suspicious
> > 
> > > > "\captionof{figure}{\label{orgf2d4160}" in the export of the
> > source
> > 
> > > > block.
> > 
> > > > 
> > 
> > > > Further tests (not shown) show that the proble persists when
> > org-
> > 
> > > > mode
> > 
> > > > is set up to use the minted package for LaTeX export of source
> > 
> > > > blocks
> > 
> > > > (minted is indeed used, but the block is s still mislabeled).
> > 
> > > 
> > 
> > > What LaTeX code do you suggest instead?
> > 
> > 
> > 
> > Dunno. Maybe \captionof{listing} ? Or let minted do its thing ?
> > Anyway,
> > 
> > in most cases, the code excerpts will be too long for a float ; the
> > 
> > idea is to send them (unfloated but labeled/captioned) in an
> > appendix
> > 
> > and point to them via a link.
> > 
> > 
> > 
> > [ BTW : we should stop suggesting to use listings, which is
> > problematic
> > 
> > for anything not strictly ASCII, bloody likely to occur at least on
> > a
> > 
> > comment/prompt for most of mankind... Listingsutf8 is but a patch
> > 
> > working only for (a subset of) European languages. Arabic, Hebrew,
> > 
> > Asian languages : nope. Minted in (xe|lua)tex is probably more
> > usable
> > 
> > (but needs --shell-escape, which can justifiably frowned upon... ].
> > 
> > 
> > 
> > > > The org source and resulting pdf and html files are available
> > here
> > 
> > > > : 
> > 
> > > > 
> > https://drive.google.com/drive/folders/1bQmaefiztW9ZDRNLcjOD_WbEmTg4d8dG?usp=sharing
> > 
> > > 
> > 
> > > Could you send your ECM in the ML without requiring to access
> > Google
> > 
> > > Drive?
> > 
> > 
> > 
> > Attached.
> > 
> > 
> > 
> > But please coordinate with Bastien, who asked me just yesterday to
> > 
> > avoid posting fat attachments to a message going to a lot of
> > people.
> > 
> > Which I did (to no avail : the "heavy" mail went through anyway. Go
> > 
> > figure.... ;-).
> > 
> > 
> > 
> > 
> > 
> > > Thank you.
> > 
> > 
> > 
> > You're welcome ;-). Thank YOU for considering scratching *MY*
> > itch...
> > 
> > 
> > 
> > --
> > 
> > Emmanuel Charpentier
> > 
> > 
> > 

[-- Attachment #1.2: Type: text/html, Size: 5234 bytes --]

[-- Attachment #2: C2.html --]
[-- Type: text/html, Size: 10213 bytes --]

[-- Attachment #3: C2.org --]
[-- Type: text/plain, Size: 770 bytes --]

# An archi-minimal test of source blocks labelling,
# using only the built-in tools.

#+options: tex:t title:nil author:nil date:nil toc:nil
#+property: header-args:R :session
#+title: Test of source block exports

The R block ref:lst:DoFig generates the figure ref:fig:DoFig:

#+name: DoFig
#+begin_src R :exports results :results output graphics :file Fig.png
curve(sin(x),from=-pi, to=pi)
#+end_src

#+caption: Figure generated by R label:fig:DoFig
#+RESULTS: DoFig
[[file:Fig.png]]

#+caption: R code generating a figure label:lst:DoFig
#+begin_src R :eval no :exports code :noweb yes
<<DoFig>>
#+end_src

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it).

[-- Attachment #4: C2.pdf --]
[-- Type: application/pdf, Size: 55454 bytes --]

[-- Attachment #5: C2.tex --]
[-- Type: text/x-tex, Size: 1156 bytes --]

% Created 2019-04-30 mar. 20:01
% Intended LaTeX compiler: pdflatex
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\date{}
\title{Test of source block exports}
\hypersetup{
 pdfauthor={Emmanuel Charpentier},
 pdftitle={Test of source block exports},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.1 (Org mode 9.2.3)}, 
 pdflang={English}}
\begin{document}

The R block \ref{lst:DoFig} generates the figure \ref{fig:DoFig}:

\begin{figure}[htbp]
\centering
\includegraphics[width=.9\linewidth]{Fig.png}
\caption{Figure generated by R \label{fig:DoFig}}
\end{figure}

\begin{verbatim}
curve(sin(x),from=-pi, to=pi)
\end{verbatim}
\captionof{figure}{R code generating a figure \label{lst:DoFig}}

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it).
\end{document}

[-- Attachment #6: Fig.png --]
[-- Type: image/png, Size: 7444 bytes --]

Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.

[Nicolas Goaziou]

Hello,

Emmanuel Charpentier <emm.charpentier@free.fr> writes:

> Dunno. Maybe \captionof{listing} ?

I don't know either, but the above raises an error:

  LaTeX Error: No counter 'listing' defined.

It could work, but the complete solution is probably a bit more
involved. If you can write a setup that produces the desired effect, and
supports lists of listings without any package, please let us know.

> But please coordinate with Bastien, who asked me just yesterday to
> avoid posting fat attachments to a message going to a lot of people.
> Which I did (to no avail : the "heavy" mail went through anyway. Go
> figure.... ;-).

No need for coordination: the two advices are not mutually exclusive.

Regards,

-- 
Nicolas Goaziou

Correction and addition [Re: BUG: the builtin LaTeX/P DF exporter mislabels source blocks as figures.]

[Emmanuel Charpentier]

[-- Attachment #1: Type: text/plain, Size: 385 bytes --]

I reported incorrectly that the bug Ireported was also present when
using the "minted" package to format source blocks.

This is incorrect : I re-checked my trials, and re-wrote another test,
which gives the ecpected result. See attached files.

This might help pinpointing the problem happening when minted is *NOT*
used.

HTH,

--
Emmanuel Charpentier

[ Previous posts : Snip... ]


[-- Attachment #2: C4.html --]
[-- Type: text/html, Size: 11225 bytes --]

[-- Attachment #3: C4.org --]
[-- Type: text/plain, Size: 1555 bytes --]

# An archi-minimal test of source blocks labelling,
# using only the built-in tools.

#+options: tex:t title:nil author:nil date:nil toc:nil
#+title: Test of source block exports

#+begin_src emacs-lisp :exports none :results none
  (setq
   org-latex-pdf-process
   (list "latexmk -pdf -pdflatex='%latex -interaction=nonstopmode -shell-escape' -bibtex -f %f")
   org-latex-listings 'minted)
  (add-to-list 'org-latex-packages-alist '("newfloat" "minted" t))
#+end_src

# #+latex_header: \usepackage[newfloat]{minted}
# #+latex_header: \usepackage{listings}

The Python block [[lst:DoFig]] generates the figure [[fig:DoFig]]

#+name: DoFig
#+begin_src python :results file :exports none
import matplotlib, numpy
matplotlib.use('Agg')
import matplotlib.pyplot as plt
fig=plt.figure(figsize=(4,2))
x=numpy.linspace(-15,15)
plt.plot(numpy.sin(x)/x)
fig.tight_layout()
plt.savefig('Fig2.png')
return 'Fig2.png'
#+end_src

#+RESULTS: DoFig
[[file:Fig2.png]]

#+name: fig:DoFig
#+caption: Figure generated by python
#+ATTR_LATEX: :height 2.5in
[[file:Fig2.png]]

# #+name: fig:DoFig
# #+caption: Figure generated by python
# #+RESULTS: DoFig
# [[file:Fig2.png]]


#+name: lst:DoFig
#+caption: Python code generating a figure
#+begin_src python :eval no :exports code :noweb yes
<<DoFig>>
#+end_src

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it) *when the default ~verbatim~ method is used.* When one bothers to use ~minted~, the result is correct.

[-- Attachment #4: C4.pdf --]
[-- Type: application/pdf, Size: 60828 bytes --]

[-- Attachment #5: C4.tex --]
[-- Type: text/x-tex, Size: 1553 bytes --]

% Created 2019-04-30 mar. 23:15
% Intended LaTeX compiler: pdflatex
\documentclass[11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{graphicx}
\usepackage{grffile}
\usepackage{longtable}
\usepackage{wrapfig}
\usepackage{rotating}
\usepackage[normalem]{ulem}
\usepackage{amsmath}
\usepackage{textcomp}
\usepackage{amssymb}
\usepackage{capt-of}
\usepackage{hyperref}
\usepackage[newfloat]{minted}
\date{}
\title{Test of source block exports}
\hypersetup{
 pdfauthor={Emmanuel Charpentier},
 pdftitle={Test of source block exports},
 pdfkeywords={},
 pdfsubject={},
 pdfcreator={Emacs 26.1 (Org mode 9.2.3)}, 
 pdflang={English}}
\begin{document}

The Python block \ref{org471e7a4} generates the figure \ref{fig:orgb0ae5e3}

\begin{figure}[htbp]
\centering
\includegraphics[height=2.5in]{Fig2.png}
\caption{\label{fig:orgb0ae5e3}
Figure generated by python}
\end{figure}


\begin{listing}[htbp]
\begin{minted}[]{python}
import matplotlib, numpy
matplotlib.use('Agg')
import matplotlib.pyplot as plt
fig=plt.figure(figsize=(4,2))
x=numpy.linspace(-15,15)
plt.plot(numpy.sin(x)/x)
fig.tight_layout()
plt.savefig('Fig2.png')
return 'Fig2.png'
\end{minted}
\caption{\label{org471e7a4}
Python code generating a figure}
\end{listing}

Problem: whereas the HTML export labels the source listing as "Listing 1" and is correctly refers to 1, the PDF export labels it "Figure 2" (and points to it) \textbf{when the default \texttt{verbatim} method is used.} When one bothers to use \texttt{minted}, the result is correct.
\end{document}

[-- Attachment #6: Fig2.png --]
[-- Type: image/png, Size: 11763 bytes --]