[PATCH] Documentation and NEWS for ` org-latex-language-alist'

[PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 347 bytes --]

Hi,

I am attaching a patch with the documentation of the new variable in the
Manual and the updated NEWS.

Best regards,

Juan Manuel

-- 
--
------------------------------------------------------
Juan Manuel Macías 

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-doc-org-manual.org-documentation-for-org-latex-langu.patch --]
[-- Type: text/x-patch, Size: 4540 bytes --]

From 2ec740e4b2f691f619878a7b86e4874fa05d3a82 Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Mon, 8 Aug 2022 16:30:01 +0200
Subject: [PATCH] doc/org-manual.org: documentation for
 `org-latex-language-alist'

* etc/ORG-NEWS: update the news with the new variable.
---
 doc/org-manual.org | 57 ++++++++++++++++++++++++++++++++++++++++++++--
 etc/ORG-NEWS       | 13 +++++++++++
 2 files changed, 68 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 466718e6e..5d0283bf2 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -13370,19 +13370,72 @@ general options (see [[*Export Settings]]).
 - =LANGUAGE= ::
   #+cindex: @samp{LANGUAGE}, keyword
   #+vindex: org-latex-packages-alist
+  #+vindex: org-latex-language-alist
+  #+vindex: org-export-default-language
+  
+  The list of languages supported by Org is stored in the variable
+  ~org-latex-language-alist~.
+
   In order to be effective, the =babel= or =polyglossia=
   packages---according to the LaTeX compiler used---must be loaded
   with the appropriate language as argument.  This can be accomplished
   by modifying the ~org-latex-packages-alist~ variable, e.g., with the
-  following snippet:
+  following snippet (note that =polyglossia= does not work with
+  pdfLaTeX):
 
   #+begin_src emacs-lisp
   (add-to-list 'org-latex-packages-alist
-               '("AUTO" "babel" t ("pdflatex")))
+               '("AUTO" "babel" t ("pdflatex" "xelatex" "lualatex")))
   (add-to-list 'org-latex-packages-alist
                '("AUTO" "polyglossia" t ("xelatex" "lualatex")))
   #+end_src
 
+  LaTeX packages =babel= or =polyglossia= can also be loaded in a
+  document.  The "AUTO" string will be replaced in both cases by the
+  appropiate value for the =LANGUAGE= keyword, if present in the
+  document, or by the value of ~org-export-default-language~.  Let's
+  see some examples in one or another case.
+
+  =Babel= accepts the classic syntax and (in addition) the new syntax
+  with the =\babelprovide= command to load the languages using the new
+  =INI= files procedure.  Keep in mind that there are a number of
+  languages that are only served in babel using =INI= files, so they
+  cannot be declared using the classic syntax, but only using the
+  =\babelprovide= command (see
+  http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf).
+  Valid usage examples could be:
+
+  #+begin_example
+  ,#+LATEX_HEADER: \usepackage[french,italian,AUTO]{babel}
+  #+end_example
+
+  where "AUTO" is the main language.  But it can also be loaded using
+  the =\babelprovide= command:
+
+  #+begin_example
+  ,#+LATEX_HEADER: \usepackage[french,italian]{babel}
+  ,#+LATEX_HEADER: \babelprovide[import, main]{AUTO}
+  #+end_example
+
+  =Polyglossia=, for this procedure to be effective, must be loaded
+  using the same =babel= classic syntax (but note that /this is not/
+  the actual polyglossia syntax).  For example, suppose a document
+  declares Polytonic Greek as the primary language, and French as the
+  secondary language.  In this case, it would be expressed as:
+
+  #+begin_example
+  ,#+LANGUAGE: el-polyton
+  ,#+LATEX_HEADER: \usepackage[french,AUTO]{polyglossia}
+  #+end_example
+
+  This would produce in LaTeX (with the actual =polyglossia= syntax):
+
+  #+begin_example
+  \usepackage{polyglossia}
+  \setmainlanguage[variant=polytonic]{greek}
+  \setotherlanguage{french}
+  #+end_example
+
 - =LATEX_CLASS= ::
 
   #+cindex: @samp{LATEX_CLASS}, keyword
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 00fe101dc..fc9ac688a 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -431,6 +431,19 @@ prompting for a link description.  It can be a string (used as-is) or
 a function (called with the same arguments as
 ~org-make-link-description-function~ to return a string to use).
 
+*** New list of languages for LaTeX export: ~org-latex-language-alist~ 
+
+~org-latex-language-alist~ unifies into a single list the old language
+lists for the =babel= and =polyglossia= LaTeX packages:
+~org-latex-babel-language-alist~ and
+~org-latex-polyglossia-language-alist~, respectively, which are
+declared obsolete.
+
+This new list captures the current state of art regarding language
+support in LaTeX.  The new =babel= syntax for loading languages via
+=ini= files and the new command =\babelprovide= (see:
+http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf)
+are also supported.
 * Version 9.5
 
 ** Important announcements and breaking changes
-- 
2.37.1

Re: [PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Ihor Radchenko]

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> I am attaching a patch with the documentation of the new variable in the
> Manual and the updated NEWS.

Thanks!

I tried to apply your patch and read the relevant section of the manual
from the beginning. I feel that the new wording is awkward.
The patch is adding text to LaTeX export settings section where
individual #+KEYWORDS are discussed. Yet, the patch goes deeply into
setting some variables from the very beginning only providing a single
example where the #+LANGUAGE keyword is actually used.

I attached the screenshot of HTML version of the manual for clarity.


[-- Attachment #2: exported-patch-as-it-reads.png --]
[-- Type: image/png, Size: 220253 bytes --]

[-- Attachment #3: Type: text/plain, Size: 613 bytes --]


I feel like we should first describe what #+LANGUAGE keyword does and
go into the gory details of LaTeX language support a bit later.

Basically, I am asking you to reshuffle paragraphs a bit possibly moving
some parts to more relevant 13.10.3 LaTeX header and sectioning
structure. People have no idea about #+LATEX_HEADER when reading 13.10.2
LaTeX specific export settings (coming before the 13.10.3).

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: [PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Max Nikulin]

On 08/08/2022 21:39, Juan Manuel Macías wrote:
> 
> I am attaching a patch with the documentation of the new variable in the
> Manual and the updated NEWS.

> +  =\babelprovide= command (see
> +  http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf).
> +  Valid usage examples could be:

Firefox may block downloading of a file through unencrypted http: 
protocol if a link is opened from a https: page (HTML version of the Org 
manual served from orgmode.org this case). The problem with CTAN mirrors 
is that not all hosts have TLS certificates including their CTAN names.

Re: [PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Juan Manuel Macías]

[-- Attachment #1: Type: text/plain, Size: 1850 bytes --]

Hi, Ihor. Sorry for my late reply. These last few days have been a bit
complicated for me due to work issues.

Ihor Radchenko writes:

> I feel like we should first describe what #+LANGUAGE keyword does and
> go into the gory details of LaTeX language support a bit later.
>
> Basically, I am asking you to reshuffle paragraphs a bit possibly moving
> some parts to more relevant 13.10.3 LaTeX header and sectioning
> structure. People have no idea about #+LATEX_HEADER when reading 13.10.2
> LaTeX specific export settings (coming before the 13.10.3).

I am attaching a new version of the patch.

Addendum: Just as I was going to send this email, I saw the other email
from Maxim in this thread, who says:

> Firefox may block downloading of a file through unencrypted http:
> protocol if a link is opened from a https: page (HTML version of the
> Org manual served from orgmode.org this case). The problem with CTAN
> mirrors is that not all hosts have TLS certificates including their
> CTAN names.

I can't figure out how to link to the babel documentation in a way other
than a link to CTAN. If this might be an inconvenience (due to https
issues), perhaps we could simply put in the documentation something like
'see the Babel docs', or referencing the command 'texdoc -M
babel', which opens the locally installed babel documentation. The
problem is that not all GNU/Linux distributions include the
documentation in their versions of TeX live. For example, Arch Linux
does not include documentation in the official repos and must be
installed separately via an AUR package.

Best regards,

Juan Manuel 

-- 
--
------------------------------------------------------
Juan Manuel Macías 

https://juanmanuelmacias.com

https://lunotipia.juanmanuelmacias.com

https://gnutas.juanmanuelmacias.com


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-doc-org-manual.org-documentation-for-org-latex-langu.patch --]
[-- Type: text/x-patch, Size: 4628 bytes --]

From 46192df70cd00e2eca2e8e059e9d39723b9d32d8 Mon Sep 17 00:00:00 2001
From: Juan Manuel Macias <maciaschain@posteo.net>
Date: Tue, 16 Aug 2022 15:41:22 +0200
Subject: [PATCH] doc/org-manual.org: documentation for
 `org-latex-language-alist'

* etc/ORG-NEWS: update the news with the new variable.
---
 doc/org-manual.org | 58 ++++++++++++++++++++++++++++++++++++++++++++--
 etc/ORG-NEWS       | 13 +++++++++++
 2 files changed, 69 insertions(+), 2 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 466718e6e..8973f08ce 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -13370,15 +13370,21 @@ general options (see [[*Export Settings]]).
 - =LANGUAGE= ::
   #+cindex: @samp{LANGUAGE}, keyword
   #+vindex: org-latex-packages-alist
+  #+vindex: org-latex-language-alist
+  
+  The list of languages supported by Org is stored in the variable
+  ~org-latex-language-alist~.
+
   In order to be effective, the =babel= or =polyglossia=
   packages---according to the LaTeX compiler used---must be loaded
   with the appropriate language as argument.  This can be accomplished
   by modifying the ~org-latex-packages-alist~ variable, e.g., with the
-  following snippet:
+  following snippet (note that =polyglossia= does not work with
+  pdfLaTeX):
 
   #+begin_src emacs-lisp
   (add-to-list 'org-latex-packages-alist
-               '("AUTO" "babel" t ("pdflatex")))
+               '("AUTO" "babel" t ("pdflatex" "xelatex" "lualatex")))
   (add-to-list 'org-latex-packages-alist
                '("AUTO" "polyglossia" t ("xelatex" "lualatex")))
   #+end_src
@@ -13508,6 +13514,54 @@ A sample Org file with the above headers:
   some more text
 #+end_example
 
+#+cindex: @samp{LANGUAGE}, keyword
+#+vindex: org-export-default-language
+LaTeX packages =babel= or =polyglossia= can also be loaded in a
+document.  The "AUTO" string will be replaced in both cases by the
+appropiate value for the =LANGUAGE= keyword, if present in the
+document, or by the value of ~org-export-default-language~.  Let's see
+some examples in one or another case.
+
+=Babel= accepts the classic syntax and (in addition) the new syntax
+with the =\babelprovide= command to load the languages using the new
+=INI= files procedure.  Keep in mind that there are a number of
+languages that are only served in babel using =INI= files, so they
+cannot be declared using the classic syntax, but only using the
+=\babelprovide= command (see
+http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf).
+Valid usage examples could be:
+
+#+begin_example
+,#+LATEX_HEADER: \usepackage[french,italian,AUTO]{babel}
+#+end_example
+
+where "AUTO" is the main language.  But it can also be loaded using
+the =\babelprovide= command:
+
+#+begin_example
+,#+LATEX_HEADER: \usepackage[french,italian]{babel}
+,#+LATEX_HEADER: \babelprovide[import, main]{AUTO}
+#+end_example
+
+=Polyglossia=, for this procedure to be effective, must be loaded
+using the same =babel= classic syntax (but note that /this is not/
+the actual polyglossia syntax).  For example, suppose a document
+declares Polytonic Greek as the primary language, and French as the
+secondary language.  In this case, it would be expressed as:
+
+#+begin_example
+,#+LANGUAGE: el-polyton
+,#+LATEX_HEADER: \usepackage[french,AUTO]{polyglossia}
+#+end_example
+
+This would produce in LaTeX (with the actual =polyglossia= syntax):
+
+#+begin_example
+\usepackage{polyglossia}
+\setmainlanguage[variant=polytonic]{greek}
+\setotherlanguage{french}
+#+end_example
+
 *** Quoting LaTeX code
 :PROPERTIES:
 :DESCRIPTION: Incorporating literal @LaTeX{} code.
diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 00fe101dc..fc9ac688a 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -431,6 +431,19 @@ prompting for a link description.  It can be a string (used as-is) or
 a function (called with the same arguments as
 ~org-make-link-description-function~ to return a string to use).
 
+*** New list of languages for LaTeX export: ~org-latex-language-alist~ 
+
+~org-latex-language-alist~ unifies into a single list the old language
+lists for the =babel= and =polyglossia= LaTeX packages:
+~org-latex-babel-language-alist~ and
+~org-latex-polyglossia-language-alist~, respectively, which are
+declared obsolete.
+
+This new list captures the current state of art regarding language
+support in LaTeX.  The new =babel= syntax for loading languages via
+=ini= files and the new command =\babelprovide= (see:
+http://mirrors.ctan.org/macros/latex/required/babel/base/babel.pdf)
+are also supported.
 * Version 9.5
 
 ** Important announcements and breaking changes
-- 
2.37.2

Re: [PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Max Nikulin]

On 16/08/2022 21:16, Juan Manuel Macías wrote:
> 
>> Firefox may block downloading of a file through unencrypted http:
>> protocol if a link is opened from a https: page (HTML version of the
>> Org manual served from orgmode.org this case). The problem with CTAN
>> mirrors is that not all hosts have TLS certificates including their
>> CTAN names.
> 
> I can't figure out how to link to the babel documentation in a way other
> than a link to CTAN. If this might be an inconvenience (due to https
> issues), perhaps we could simply put in the documentation something like
> 'see the Babel docs', or referencing the command 'texdoc -M
> babel', which opens the locally installed babel documentation. The
> problem is that not all GNU/Linux distributions include the
> documentation in their versions of TeX live. For example, Arch Linux
> does not include documentation in the official repos and must be
> installed separately via an AUR package.

I would suggest https (TLS) link and title of the document. I believe 
that is should be a requirement for "official" CTAN mirrors to have 
valid TLS certificate.

P.S. Why there is no ol-texdoc.el?

Re: [PATCH] Documentation and NEWS for ` org-latex-language-alist'

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

> I am attaching a new version of the patch.

Thanks!
Applied onto main via 243ded74b adding one sentence to the =LANGUAGE=
keyword description and changing the ctan links to https.

I have added a short description of what LANGUAGE keyword value is:
"Language code of the primary document language."

> Addendum: Just as I was going to send this email, I saw the other email
> from Maxim in this thread, who says:
>
>> Firefox may block downloading of a file through unencrypted http:
>> protocol if a link is opened from a https: page (HTML version of the
>> Org manual served from orgmode.org this case). The problem with CTAN
>> mirrors is that not all hosts have TLS certificates including their
>> CTAN names.
>
> I can't figure out how to link to the babel documentation in a way other
> than a link to CTAN. If this might be an inconvenience (due to https
> issues), perhaps we could simply put in the documentation something like
> 'see the Babel docs', or referencing the command 'texdoc -M
> babel', which opens the locally installed babel documentation. The
> problem is that not all GNU/Linux distributions include the
> documentation in their versions of TeX live. For example, Arch Linux
> does not include documentation in the official repos and must be
> installed separately via an AUR package.

I went with https. It does open at least in some cases. The problem with
CTAN certificates is not our problem. Users may still search the
relevant documents if the link is dead.

Note that we still have a number of http links in the manual. One may
want to fix them.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

http: links in the manual

[Max Nikulin]

On 20/08/2022 12:51, Ihor Radchenko wrote:
> 
> Note that we still have a number of http links in the manual. One may
> want to fix them.

One may got no response trying to fix a link.

Max Nikulin to emacs-orgmode. [PATCH] org-manual.org: Update links to 
MathJax docs. Sun, 3 Oct 2021 23:17:46 +0700. 
https://list.orgmode.org/sjcl3b$gsr$1@ciao.gmane.io

In the particular case of docs.mathjax.org I am unsure if mild 
preference of http: over https: is not a mistake in the server 
configuration. I do not mind "https:" there, any variant is better than 
the old broken link.

Re: http: links in the manual

[Juan Manuel Macías]

Max Nikulin writes:

> One may got no response trying to fix a link.
>
> Max Nikulin to emacs-orgmode. [PATCH] org-manual.org: Update links to
> MathJax docs. Sun, 3 Oct 2021 23:17:46 +0700.
> https://list.orgmode.org/sjcl3b$gsr$1@ciao.gmane.io
>
> In the particular case of docs.mathjax.org I am unsure if mild
> preference of http: over https: is not a mistake in the server
> configuration. I do not mind "https:" there, any variant is better
> than the old broken link.

Maybe, instead of repairing the links manually, we could think of some
code that would do this work periodically, and also check the health of
the links, running a url request on each link and returning a list of
broken links. I don't know if it is possible to do something like that
in Elisp, as I don't have much experience with web and link issues. I
think there are also external tools, like Selenium Web Driver, but my
experience with it is very limited (I use Selenium from time to time
when I want to take a screenshot of a web page).

Best regards,

Juan Manuel 

Auto-checking dead links in the manual (was: http: links in the manual)

[Ihor Radchenko]

Juan Manuel Macías <maciaschain@posteo.net> writes:

>> Max Nikulin to emacs-orgmode. [PATCH] org-manual.org: Update links to
>> MathJax docs. Sun, 3 Oct 2021 23:17:46 +0700.
>> https://list.orgmode.org/sjcl3b$gsr$1@ciao.gmane.io
>>
>> In the particular case of docs.mathjax.org I am unsure if mild
>> preference of http: over https: is not a mistake in the server
>> configuration. I do not mind "https:" there, any variant is better
>> than the old broken link.
>
> Maybe, instead of repairing the links manually, we could think of some
> code that would do this work periodically, and also check the health of
> the links, running a url request on each link and returning a list of
> broken links. I don't know if it is possible to do something like that
> in Elisp, as I don't have much experience with web and link issues. I
> think there are also external tools, like Selenium Web Driver, but my
> experience with it is very limited (I use Selenium from time to time
> when I want to take a screenshot of a web page).

This is a good idea.

Selenium is probably an overkill since we should better not link JS-only
websites from the manual anyway. What we can do instead is a make target
that will use something like wget.

Patches are welcome!

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

Re: Auto-checking dead links in the manual (was: http: links in the manual)

[Max Nikulin]

On 22/08/2022 09:46, Ihor Radchenko wrote:
> Juan Manuel Macías writes:
> 
>> Maybe, instead of repairing the links manually, we could think of some
>> code that would do this work periodically, and also check the health of
>> the links, running a url request on each link and returning a list of
>> broken links. I don't know if it is possible to do something like that
>> in Elisp, as I don't have much experience with web and link issues. I
>> think there are also external tools, like Selenium Web Driver, but my
>> experience with it is very limited (I use Selenium from time to time
>> when I want to take a screenshot of a web page).
> 
> This is a good idea.
> 
> Selenium is probably an overkill since we should better not link JS-only
> websites from the manual anyway. What we can do instead is a make target
> that will use something like wget.
> 
> Patches are welcome!

I hope that selenium is currently overkill, however more sites are 
starting to use anti-DDOS shields like cloudflare and HTTP client may be 
banned just because it does not fetch other resources like JS scripts.

I do not have a patch, just an idea: export backend that ignores 
everything besides link and either send requests from lisp code or 
generate file for another tool.

#+attr_linklint: ...

may be used to specify regexp that target page is expected to contain. 
There are some complications like e.g. "info:" links having special code 
to generate HTML with URL derived from original path. So it may be more 
robust to parse HTML document (without checking of linked document text).

Re: Auto-checking dead links in the manual (was: http: links in the manual)

[Hendursaga]

> I hope that selenium is currently overkill

Me too, although the WebDriver protocol itself is less bloated than Selenium. Personally I use Etaoin[1] for anything WebDriver-related, it's pretty compact, Lisp-y, and you can easily run unit tests with Emacs. As for anything ready-made for cleaning up dead links, I'm not aware of, unfortunately.

[1] https://github.com/clj-commons/etaoin

Re: Auto-checking dead links in the manual (was: http: links in the manual)

[Ihor Radchenko]

Max Nikulin <manikulin@gmail.com> writes:

> I hope that selenium is currently overkill, however more sites are 
> starting to use anti-DDOS shields like cloudflare and HTTP client may be 
> banned just because it does not fetch other resources like JS scripts.

Such links are to be considered dead for the purposes of Org manual.
We must not link websites that cannot be opened without running non-free
JS. It is according to GNU Documentation Standards.

> I do not have a patch, just an idea: export backend that ignores 
> everything besides link and either send requests from lisp code or 
> generate file for another tool.
>
> #+attr_linklint: ...
>
> may be used to specify regexp that target page is expected to contain. 
> There are some complications like e.g. "info:" links having special code 
> to generate HTML with URL derived from original path. So it may be more 
> robust to parse HTML document (without checking of linked document text).

Yes, the most robust way will be simply extracting links from the html
version of the manual and testing them using whatever method is
appropriate.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92