Orgmode fails to export specific web-links as latex/pdf

Orgmode fails to export specific web-links as latex/pdf

[Mark Edgington]

Hello all-

When trying to export via latex -> pdf, the following org-mode snippet
fails to compile.  Presumably there's something that should be changed
in the export filters?

Here's the org-mode snippet:
---------------------
*** [[http://www.example.com:90/search~S2?/Xsearchterm&searchscope%3D2&SORT%3DD/Xsearchterm&searchscope%3D2&SORT%3DD&SUBKEY%3Dsearchterm/1%252C742%252C742%252CB/frameset&FF%3DXsearchterm&searchscope%3D2&SORT%3DD&16%252C16%252C][Title
(mediatype) / Publisher]]
---------------------

Relevant generated tex code:
---------------------
\subsubsection{\href{http://www.example.com:90/search~S2?/Xsearchterm&searchscope=2&SORT=D/Xsearchterm&searchscope=2&SORT=D&SUBKEY=searchterm/1%2C742%2C742%2CB/frameset&FF=Xsearchterm&searchscope=2&SORT=D&16%2C16%2C}{Title
(mediatype) / Publisher}}
\label{sec-1-6-1}
(Does this work?)
---------------------

Relevant snippet from the latex log:
---------------------
Runaway argument?
{\href {http://www.example.com:90/search~S2?/Xsearchterm&searchscope=\ETC.
! File ended while scanning use of \@xdblarg.
<inserted text>
                \par
<*> /tmp/test.tex

! Emergency stop.
<*> /tmp/test.tex

!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on .//test.log.
---------------------

I'm using org-mode release_8.0.5-316-g878d6c.  Let me know if you have
any ideas on what is going on, and how to make this work correctly.

Regards,

Mark

Re: Orgmode fails to export specific web-links as latex/pdf

[Bastien]

[-- Attachment #1: Type: text/plain, Size: 241 bytes --]

Hi Mark,

I confirm this bug: "%" in links breaks the export.

The attached patch is a quick fix, but it needs to be double-checked
and surely amended.  Also, I didn't check other link-related functions.

Thanks for reporting this bad bug,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: fix-latex-percent-char-in-links.patch --]
[-- Type: text/x-patch, Size: 626 bytes --]

diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
index aa30cf1..650b7cc 100644
--- a/lisp/ox-latex.el
+++ b/lisp/ox-latex.el
@@ -1748,7 +1748,9 @@ DESC is the description part of the link, or the empty string.
 INFO is a plist holding contextual information.  See
 `org-export-data'."
   (let* ((type (org-element-property :type link))
-	 (raw-path (org-element-property :path link))
+	 (raw-path (org-latex-plain-text
+		    (org-link-escape (org-element-property :path link))
+		    info))
 	 ;; Ensure DESC really exists, or set it to nil.
 	 (desc (and (not (string= desc "")) desc))
 	 (imagep (org-export-inline-image-p

[-- Attachment #3: Type: text/plain, Size: 14 bytes --]


-- 
 Bastien

Re: Orgmode fails to export specific web-links as latex/pdf

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> I confirm this bug: "%" in links breaks the export.
>
> The attached patch is a quick fix, but it needs to be double-checked
> and surely amended.  Also, I didn't check other link-related functions.
>
> Thanks for reporting this bad bug,
>
>
> diff --git a/lisp/ox-latex.el b/lisp/ox-latex.el
> index aa30cf1..650b7cc 100644
> --- a/lisp/ox-latex.el
> +++ b/lisp/ox-latex.el
> @@ -1748,7 +1748,9 @@ DESC is the description part of the link, or the empty string.
>  INFO is a plist holding contextual information.  See
>  `org-export-data'."
>    (let* ((type (org-element-property :type link))
> -	 (raw-path (org-element-property :path link))
> +	 (raw-path (org-latex-plain-text
> +		    (org-link-escape (org-element-property :path link))
> +		    info))
>  	 ;; Ensure DESC really exists, or set it to nil.
>  	 (desc (and (not (string= desc "")) desc))
>  	 (imagep (org-export-inline-image-p

Actually, some characters need to be escaped when used in a URL within
a \href command, but not all of them. `org-latex-plain-text' escapes too
many of them and `org-link-escape' doesn't escape enough (and we don't
need to rely on it).

According to hyperref documentation # and ~ need not be escaped. Though,
%, { and } do. I'm not sure about \, ^ and _.

Also, I don't think \url command has the same problem, so escaping can
happen later in the function, not at the `raw-path' level.


Regards,

-- 
Nicolas Goaziou

Re: Orgmode fails to export specific web-links as latex/pdf

[Bastien]

Hi Nicolas,

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> Actually, some characters need to be escaped when used in a URL within
> a \href command, but not all of them. `org-latex-plain-text' escapes too
> many of them and `org-link-escape' doesn't escape enough (and we don't
> need to rely on it).

Thanks for the details.

> According to hyperref documentation # and ~ need not be escaped. Though,
> %, { and } do. I'm not sure about \, ^ and _.
>
> Also, I don't think \url command has the same problem, so escaping can
> happen later in the function, not at the `raw-path' level.

Please feel free to go ahead with whatever fix is better, I was just
trying to quickly reproduce the bug.

Thanks,

-- 
 Bastien

Re: Orgmode fails to export specific web-links as latex/pdf

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>> According to hyperref documentation # and ~ need not be escaped. Though,
>> %, { and } do. I'm not sure about \, ^ and _.
>>
>> Also, I don't think \url command has the same problem, so escaping can
>> happen later in the function, not at the `raw-path' level.
>
> Please feel free to go ahead with whatever fix is better, I was just
> trying to quickly reproduce the bug.

I will have a look at it, but this is not clear to me either. I still
have to understand what should be escaped, and when.


Regards,

-- 
Nicolas Goaziou

Re: Orgmode fails to export specific web-links as latex/pdf

[Bastien]

Nicolas Goaziou <n.goaziou@gmail.com> writes:

> I will have a look at it, but this is not clear to me either. I still
> have to understand what should be escaped, and when.

All right, no hurry!  And thanks a lot in advance,

-- 
 Bastien

Re: Orgmode fails to export specific web-links as latex/pdf

[Yury Bulka]

A quick related observation. Consider this Org-mode link:
[[https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(%C2%AB%D0%9F%D1%96%D1%81%D0%BD%D1%8F_%D0%BF%D1%80%D0%B8_%D0%BA%D0%BE%D0%BB%D0%B8%D1%81%D1%86%D1%96%C2%BB).ogg][https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg]]

The LaTeX code it generates:
\href{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg}{\url{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby}$_{\text{(«Пісня}_{\text{при}}_{\text{колисці»}}\text{)}}$.ogg}

Or if I add a "#+OPTIONS: ^:nil" line, then LaTeX code will be the following:
\href{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg}{\url{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby}\_(«Пісня\_при\_колисці»).ogg}

I.e. it wraps only part of the description of the link with \url due
to the parentheses in it.

2013/7/10, Bastien <bzg@gnu.org>:
> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> I will have a look at it, but this is not clear to me either. I still
>> have to understand what should be escaped, and when.
>
> All right, no hurry!  And thanks a lot in advance,
>
> --
>  Bastien
>
>

Re: Orgmode fails to export specific web-links as latex/pdf

[Yury Bulka]

One more thing, related to unicode inside the href of the link:

[[https://commons.wikimedia.org/wiki/File:%D0%91%D1%83%D0%B7%D0%BE%D0%BA.ogg][https://commons.wikimedia.org/wiki/File:Бузок.ogg]]

Turns into:
\href{https://commons.wikimedia.org/wiki/File:Бузок.ogg}{\url{https://commons.wikimedia.org/wiki/File:Бузок.ogg}}

I.e. for some reason the href gets url-decoded. Such links don't work in evince.

2013/7/16, Yury Bulka <yurkobb@gmail.com>:
> A quick related observation. Consider this Org-mode link:
> [[https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(%C2%AB%D0%9F%D1%96%D1%81%D0%BD%D1%8F_%D0%BF%D1%80%D0%B8_%D0%BA%D0%BE%D0%BB%D0%B8%D1%81%D1%86%D1%96%C2%BB).ogg][https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg]]
>
> The LaTeX code it generates:
> \href{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg}{\url{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby}$_{\text{(«Пісня}_{\text{при}}_{\text{колисці»}}\text{)}}$.ogg}
>
> Or if I add a "#+OPTIONS: ^:nil" line, then LaTeX code will be the
> following:
> \href{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg}{\url{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby}\_(«Пісня\_при\_колисці»).ogg}
>
> I.e. it wraps only part of the description of the link with \url due
> to the parentheses in it.
>
> 2013/7/10, Bastien <bzg@gnu.org>:
>> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>>
>>> I will have a look at it, but this is not clear to me either. I still
>>> have to understand what should be escaped, and when.
>>
>> All right, no hurry!  And thanks a lot in advance,
>>
>> --
>>  Bastien
>>
>>
>

Re: Orgmode fails to export specific web-links as latex/pdf

[Nicolas Goaziou]

Hello,

Yury Bulka <yurkobb@gmail.com> writes:

> A quick related observation. Consider this Org-mode link:
> [[https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(%C2%AB%D0%9F%D1%96%D1%81%D0%BD%D1%8F_%D0%BF%D1%80%D0%B8_%D0%BA%D0%BE%D0%BB%D0%B8%D1%81%D1%86%D1%96%C2%BB).ogg][https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg]]
>
> The LaTeX code it generates:
> \href{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby_(«Пісня_при_колисці»).ogg}{\url{https://commons.wikimedia.org/wiki/File:Lysenko-Lullaby}$_{\text{(«Пісня}_{\text{при}}_{\text{колисці»}}\text{)}}$.ogg}

This is to be expected. Only the beginning of your description part
matches `org-plain-link-re' and gets converted to a link.


Regards,

-- 
Nicolas Goaziou

Re: Orgmode fails to export specific web-links as latex/pdf

[Nicolas Goaziou]

Yury Bulka <yurkobb@gmail.com> writes:

> One more thing, related to unicode inside the href of the link:
>
> [[https://commons.wikimedia.org/wiki/File:%D0%91%D1%83%D0%B7%D0%BE%D0%BA.ogg][https://commons.wikimedia.org/wiki/File:Бузок.ogg]]
>
> Turns into:
> \href{https://commons.wikimedia.org/wiki/File:Бузок.ogg}{\url{https://commons.wikimedia.org/wiki/File:Бузок.ogg}}
>
> I.e. for some reason the href gets url-decoded. Such links don't work
> in evince.

Indeed, links are url-decoded when parsed (see line 3139 in
org-element.el). I think the reasoning behind it is that url should then
be encoded again to make sure every url is correctly encoded.

One possibility is to use (org-link-escape (org-link-unescape
raw-link)), that is every parsed link will be url-encoded. Another one
is to leave links as-is, but back-end developers will have to deal with
that. I lean towards the former.

Funnily, this only applied to bracket links (i.e., [[link][desc]]),
plain links are not url-decoded (or encoded) at all. This isn't
consistent and also needs to be fixed.

Any opinion?


Regards,

-- 
Nicolas Goaziou

Re: Orgmode fails to export specific web-links as latex/pdf

[Nicolas Goaziou]

Hello,

Bastien <bzg@gnu.org> writes:

> Nicolas Goaziou <n.goaziou@gmail.com> writes:
>
>> I will have a look at it, but this is not clear to me either. I still
>> have to understand what should be escaped, and when.
>
> All right, no hurry!  And thanks a lot in advance,

I cannot reproduce the bug anymore. Can you?


Regards,

-- 
Nicolas Goaziou