bug#62493: xelatex won't render fonts correctly without full texlive

bug#62493: xelatex won't render fonts correctly without full texlive

[John Kehayias via Bug reports for GNU Guix]

[-- Attachment #1: Type: text/plain, Size: 2418 bytes --]

Hi TeX Guixers,

Ran into a curious problem today of xelatex not properly rendering what I think is a mapping/ligature issue: the usual TeX ``something'' quote syntax should become proper double quotes (for English at least) but don't.

Here is a simple test tex file:

--8<---------------cut here---------------start------------->8---
\documentclass{article}
\begin{document}
``test quote''
\end{document}
--8<---------------cut here---------------end--------------->8---

If you compile it with

--8<---------------cut here---------------start------------->8---
guix shell texlive-base texlive-fontspec -- xelatex test.tex
--8<---------------cut here---------------end--------------->8---

you'll see the same quotes as in the tex file. Full log (test-bad.log) attached but the relevant line looks to me like

--8<---------------cut here---------------start------------->8---
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/gnu/store/npkjqvkgzj3pfhqwbcc504kgp1z6fj8s-profile/share/texmf-dist/tex/latex/base/size10.clo
File: size10.clo 2020/04/10 v1.4m Standard LaTeX file (size option)
Font mapping `tex-text.tec' for font `[lmroman10-regular]:mapping=tex-text;' not found.
)
--8<---------------cut here---------------end--------------->8---

I did try adding various packages, changing fonts and font settings, but to no avail.

Meanwhile, using

--8<---------------cut here---------------start------------->8---
guix shell texlive -- xelatex test.tex
--8<---------------cut here---------------end--------------->8---

gives a nicely rendered double quote mark. The log (also attached as test.log) shows no such similar message.

Some searching the most succinct and relevant summary I think is here <https://tex.stackexchange.com/questions/193412/what-is-happening-to-the-quotes>

Anyway, seems like something is missing when we use the minimal packages, but I can't seem to find what to add to guix shell to change the output of the pdf. Probably it is not captured in a package or is otherwise lost?

Any ideas?

Thanks!
John

PS: I don't think it is related, but xelatex will complain about not finding a fontspec.cfg either. I made one locally from what it used to be, but didn't change the output for me. I think it went missing here:

<https://git.savannah.gnu.org/cgit/guix.git/commit/?id=ad5dddc610fa19e78aaa1885106a419e55f0322b>

[-- Attachment #2: test.log --]
[-- Type: application/octet-stream, Size: 3115 bytes --]

This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021/GNU Guix) (preloaded format=xelatex 2022.5.28)  27 MAR 2023 22:00
entering extended mode
 restricted \write18 enabled.
 %&-line parsing enabled.
**test.tex
(./test.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18>
(/gnu/store/blxbw5ghhzw2bv71mnrf9l62bfh7xa2j-texlive-texmf-20210325/share/texmf
-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class

(/gnu/store/blxbw5ghhzw2bv71mnrf9l62bfh7xa2j-texlive-texmf-20210325/share/texmf
-dist/tex/latex/base/size10.clo
File: size10.clo 2020/04/10 v1.4m Standard LaTeX file (size option)
)
\c@part=\count175
\c@section=\count176
\c@subsection=\count177
\c@subsubsection=\count178
\c@paragraph=\count179
\c@subparagraph=\count180
\c@figure=\count181
\c@table=\count182
\abovecaptionskip=\skip47
\belowcaptionskip=\skip48
\bibindent=\dimen138
)
(/gnu/store/blxbw5ghhzw2bv71mnrf9l62bfh7xa2j-texlive-texmf-20210325/share/texmf
-dist/tex/latex/l3backend/l3backend-xetex.def
File: l3backend-xetex.def 2021-03-18 L3 backend support: XeTeX
 (|extractbb --version)
\c__kernel_sys_dvipdfmx_version_int=\count183
\l__color_backend_stack_int=\count184
\g__color_backend_stack_int=\count185
\g__graphics_track_int=\count186
\l__pdf_internal_box=\box47
\g__pdf_backend_object_int=\count187
\g__pdf_backend_annotation_int=\count188
\g__pdf_backend_link_int=\count189
)
(./test.aux)
\openout1 = `test.aux'.

LaTeX Font Info:    Checking defaults for OML/cmm/m/it on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OMS/cmsy/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OT1/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for T1/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for TS1/cmr/m/n on input line 5.
LaTeX Font Info:    Trying to load font information for TS1+cmr on input line 5
.

(/gnu/store/blxbw5ghhzw2bv71mnrf9l62bfh7xa2j-texlive-texmf-20210325/share/texmf
-dist/tex/latex/base/ts1cmr.fd
File: ts1cmr.fd 2019/12/16 v2.5j Standard LaTeX font definitions
)
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for TU/lmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OMX/cmex/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for U/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
 [1

] (./test.aux) ) 
Here is how much of TeX's memory you used:
 433 strings out of 476919
 9136 string characters out of 5799887
 297109 words of memory out of 5000000
 20511 multiletter control sequences out of 15000+600000
 403438 words of font info for 28 fonts, out of 8000000 for 9000
 1348 hyphenation exceptions out of 8191
 44i,4n,50p,143b,100s stack positions out of 5000i,500n,10000p,200000b,80000s

Output written on test.pdf (1 page).

[-- Attachment #3: test-bad.log --]
[-- Type: application/octet-stream, Size: 3135 bytes --]

This is XeTeX, Version 3.141592653-2.6-0.999993 (TeX Live 2021/GNU Guix) (preloaded format=xelatex 2022.9.14)  27 MAR 2023 21:35
entering extended mode
 restricted \write18 enabled.
 %&-line parsing enabled.
**test.tex
(./test.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18> (/gnu/store/npkjqvkgzj3pfhqwbcc504kgp1z6fj8s-profile/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/gnu/store/npkjqvkgzj3pfhqwbcc504kgp1z6fj8s-profile/share/texmf-dist/tex/latex/base/size10.clo
File: size10.clo 2020/04/10 v1.4m Standard LaTeX file (size option)
Font mapping `tex-text.tec' for font `[lmroman10-regular]:mapping=tex-text;' not found.
)
\c@part=\count175
\c@section=\count176
\c@subsection=\count177
\c@subsubsection=\count178
\c@paragraph=\count179
\c@subparagraph=\count180
\c@figure=\count181
\c@table=\count182
\abovecaptionskip=\skip47
\belowcaptionskip=\skip48
\bibindent=\dimen138
) (/gnu/store/npkjqvkgzj3pfhqwbcc504kgp1z6fj8s-profile/share/texmf-dist/tex/latex/l3backend/l3backend-xetex.def
File: l3backend-xetex.def 2021-03-18 L3 backend support: XeTeX
 (|extractbb --version)
\c__kernel_sys_dvipdfmx_version_int=\count183
\l__color_backend_stack_int=\count184
\g__color_backend_stack_int=\count185
\g__graphics_track_int=\count186
\l__pdf_internal_box=\box47
\g__pdf_backend_object_int=\count187
\g__pdf_backend_annotation_int=\count188
\g__pdf_backend_link_int=\count189
) (./test.aux)
\openout1 = `test.aux'.

LaTeX Font Info:    Checking defaults for OML/cmm/m/it on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OMS/cmsy/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OT1/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for T1/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for TS1/cmr/m/n on input line 5.
LaTeX Font Info:    Trying to load font information for TS1+cmr on input line 5.
 (/gnu/store/npkjqvkgzj3pfhqwbcc504kgp1z6fj8s-profile/share/texmf-dist/tex/latex/base/ts1cmr.fd
File: ts1cmr.fd 2019/12/16 v2.5j Standard LaTeX font definitions
)
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for TU/lmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for OMX/cmex/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
LaTeX Font Info:    Checking defaults for U/cmr/m/n on input line 5.
LaTeX Font Info:    ... okay on input line 5.
 [1

] (./test.aux) ) 
Here is how much of TeX's memory you used:
 431 strings out of 479064
 8935 string characters out of 5868570
 290483 words of memory out of 5000000
 20247 multiletter control sequences out of 15000+600000
 403438 words of font info for 28 fonts, out of 8000000 for 9000
 14 hyphenation exceptions out of 8191
 44i,4n,50p,143b,100s stack positions out of 5000i,500n,10000p,200000b,80000s

Output written on test.pdf (1 page).

bug#62493: xelatex won't render fonts correctly without full texlive

[Ricardo Wurmus]

FWIW, it does work with lualatex:

  guix shell --container texlive-base texlive-fontspec -- lualatex test.tex

-- 
Ricardo

bug#62493: xelatex won't render fonts correctly without full texlive

[Ricardo Wurmus]

We seem to be missing the xetex package that provides files for font mapping:

--8<---------------cut here---------------start------------->8---
$ info tex-text.tec
tlmgr.pl: cannot find package tex-text.tec, searching for other matches:

Packages containing `tex-text.tec' in their title/description:

Packages containing files matching `tex-text.tec':
xepersian:
	texmf-dist/fonts/misc/xetex/fontmapping/xepersian/persian-tex-text.tec
xetex:
	texmf-dist/fonts/misc/xetex/fontmapping/base/tex-text.tec
--8<---------------cut here---------------end--------------->8---

I was under the impression that we did have a xetex package, but I can’t
find it now.

-- 
Ricardo

bug#62493: xelatex won't render fonts correctly without full texlive

[John Kehayias via Bug reports for GNU Guix]

Hi Ricardo,

Note that I didn't get your replies as it was sent just to the debbugs
email (I always find it strange that the bug email alias doesn't send to
the submitter by default). This message from Simon was helpful for me,
at least via Emacs: https://issues.guix.gnu.org/59855#2 (perhaps you
already know or have some other method, but thought I'd mention it and
I'm open to more such tricks!)

Ricardo Wurmus <rekado@elephly.net> writes:

> We seem to be missing the xetex package that provides files for font mapping:
>
> $ info tex-text.tec
> tlmgr.pl: cannot find package tex-text.tec, searching for other matches:
>
> Packages containing `tex-text.tec' in their title/description:
>
> Packages containing files matching `tex-text.tec':
> xepersian:
> 	texmf-dist/fonts/misc/xetex/fontmapping/xepersian/persian-tex-text.tec
> xetex:
> 	texmf-dist/fonts/misc/xetex/fontmapping/base/tex-text.tec
>

Ah, that's good to know, thanks for figuring that out.

> I was under the impression that we did have a xetex package, but I can’t
> find it now.

I also don't see any record of such a package. Trying

--8<---------------cut here---------------start------------->8---
guix shell subversion -- guix import texlive xetex
[...]
Exported revision 59745.
A    /tmp/guix-directory.cKaqWd/doc/xetex/base
A    /tmp/guix-directory.cKaqWd/doc/xetex/base/NEWS
A    /tmp/guix-directory.cKaqWd/doc/xetex/base/README
A    /tmp/guix-directory.cKaqWd/doc/xetex/base/XeTeX-notes.pdf
A    /tmp/guix-directory.cKaqWd/doc/xetex/base/XeTeX-notes.tex
Exported revision 59745.
A    /tmp/guix-directory.cKaqWd/fonts/misc/xetex/fontmapping/base
A    /tmp/guix-directory.cKaqWd/fonts/misc/xetex/fontmapping/base/qx-unicode.map
A    /tmp/guix-directory.cKaqWd/fonts/misc/xetex/fontmapping/base/qx-unicode.tec
A    /tmp/guix-directory.cKaqWd/fonts/misc/xetex/fontmapping/base/tex-text.map
A    /tmp/guix-directory.cKaqWd/fonts/misc/xetex/fontmapping/base/tex-text.tec
Exported revision 59745.
svn: E160013: File not found: revision 66702, path '/tags/texlive-2021.3/Master/texmf-dist/tcode'
command "svn" "export" "--non-interactive" "--trust-server-cert" "-r" "59745" "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/" "/tmp/guix-directory.cKaqWd/tcode/" failed with status 1
guix import: error: failed to download description for package 'xetex'
--8<---------------cut here---------------end--------------->8---

or with a -r as well:

--8<---------------cut here---------------start------------->8---
Exported revision 59745.
A    /tmp/guix-directory.XcAtR6/doc/xetex/base
A    /tmp/guix-directory.XcAtR6/doc/xetex/base/NEWS
A    /tmp/guix-directory.XcAtR6/doc/xetex/base/README
A    /tmp/guix-directory.XcAtR6/doc/xetex/base/XeTeX-notes.pdf
A    /tmp/guix-directory.XcAtR6/doc/xetex/base/XeTeX-notes.tex
Exported revision 59745.
A    /tmp/guix-directory.XcAtR6/fonts/misc/xetex/fontmapping/base
A    /tmp/guix-directory.XcAtR6/fonts/misc/xetex/fontmapping/base/qx-unicode.map
A    /tmp/guix-directory.XcAtR6/fonts/misc/xetex/fontmapping/base/qx-unicode.tec
A    /tmp/guix-directory.XcAtR6/fonts/misc/xetex/fontmapping/base/tex-text.map
A    /tmp/guix-directory.XcAtR6/fonts/misc/xetex/fontmapping/base/tex-text.tec
Exported revision 59745.
svn: E160013: File not found: revision 66702, path '/tags/texlive-2021.3/Master/texmf-dist/tcode'
command "svn" "export" "--non-interactive" "--trust-server-cert" "-r" "59745" "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/" "/tmp/guix-directory.XcAtR6/tcode/" failed with status 1
Backtrace:
           8 (primitive-load "/home/john/.config/guix/current/bin/gu…")
In guix/ui.scm:
   2300:7  7 (run-guix . _)
  2263:10  6 (run-guix-command _ . _)
In guix/scripts/import.scm:
    89:11  5 (guix-import . _)
In ice-9/boot-9.scm:
  1752:10  4 (with-exception-handler _ _ #:unwind? _ # _)
In guix/scripts/import/texlive.scm:
    92:38  3 (_)
In guix/import/utils.scm:
   638:27  2 (recursive-import _ #:repo->guix-package _ #:guix-name . #)
   630:33  1 (lookup-node "xetex" #f)
In ice-9/boot-9.scm:
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
Wrong number of values returned to continuation (expected 2)
--8<---------------cut here---------------end--------------->8---

I'm not up to with texlive packaging, any tips? Or perhaps someone we
can ask for input here?

Thanks!
John

bug#62493: xelatex won't render fonts correctly without full texlive

[Ricardo Wurmus]

John Kehayias <john.kehayias@protonmail.com> writes:

> Exported revision 59745.
> svn: E160013: File not found: revision 66702, path '/tags/texlive-2021.3/Master/texmf-dist/tcode'
> command "svn" "export" "--non-interactive" "--trust-server-cert" "-r" "59745" "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/" "/tmp/guix-directory.XcAtR6/tcode/" failed with status 1

The importer is naive and assumes that any file in tlpdb starts with
“texmf-dist/”.  For the xetex package there’s a file
“tlpkg/tlpostcode/xetex.pl” and if you drop the same number of
characters that you would for “texmf-dist/” you end up with
“tcode/xetex.pl”.

Oops!

-- 
Ricardo

bug#62493: xelatex won't render fonts correctly without full texlive

[John Kehayias via Bug reports for GNU Guix]

"Ricardo Wurmus" <rekado@elephly.net> writes:

> John Kehayias <john.kehayias@protonmail.com> writes:
>
>> Exported revision 59745.
>> svn: E160013: File not found: revision 66702, path
>> '/tags/texlive-2021.3/Master/texmf-dist/tcode'
>> command "svn" "export" "--non-interactive" "--trust-server-cert"
>> "-r" "59745"
>> "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/"
>> "/tmp/guix-directory.XcAtR6/tcode/" failed with status 1
>
> The importer is naive and assumes that any file in tlpdb starts with
> “texmf-dist/”.  For the xetex package there’s a file
> “tlpkg/tlpostcode/xetex.pl” and if you drop the same number of
> characters that you would for “texmf-dist/” you end up with
> “tcode/xetex.pl”.
>
> Oops!

Unfortunately that leaves me at a bit of a loss for making a
texlive-xetex package as I'm not understanding the whole texlive package
structure at first glance. I've tried something similar to other
packages I see but haven't gotten anywhere.

Any pointers or similar package to look at?

Thanks!

bug#62493: xelatex won't render fonts correctly without full texlive

[Ricardo Wurmus]

John Kehayias <john.kehayias@protonmail.com> writes:

> "Ricardo Wurmus" <rekado@elephly.net> writes:
>
>> John Kehayias <john.kehayias@protonmail.com> writes:
>>
>>> Exported revision 59745.
>>> svn: E160013: File not found: revision 66702, path
>>> '/tags/texlive-2021.3/Master/texmf-dist/tcode'
>>> command "svn" "export" "--non-interactive" "--trust-server-cert"
>>> "-r" "59745"
>>> "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/"
>>> "/tmp/guix-directory.XcAtR6/tcode/" failed with status 1
>>
>> The importer is naive and assumes that any file in tlpdb starts with
>> “texmf-dist/”.  For the xetex package there’s a file
>> “tlpkg/tlpostcode/xetex.pl” and if you drop the same number of
>> characters that you would for “texmf-dist/” you end up with
>> “tcode/xetex.pl”.
>>
>> Oops!
>
> Unfortunately that leaves me at a bit of a loss for making a
> texlive-xetex package as I'm not understanding the whole texlive package
> structure at first glance. I've tried something similar to other
> packages I see but haven't gotten anywhere.

Commit cbf731ae481b434e657b05c80b4a32282e5d112b adds texlive-xetex.  Now
the quotes look fine.

I adjusted the importer in a few commits after that, but generally the
recursive texlive importer suffers from the fact that we have a lot of
inconsistent package names.

-- 
Ricardo

bug#62493: xelatex won't render fonts correctly without full texlive

[John Kehayias via Bug reports for GNU Guix]

Hi Ricardo,

On Wed, Mar 29, 2023 at 05:32 PM, Ricardo Wurmus wrote:

> John Kehayias <john.kehayias@protonmail.com> writes:
>
>> Exported revision 59745.
>> svn: E160013: File not found: revision 66702, path
>> '/tags/texlive-2021.3/Master/texmf-dist/tcode'
>> command "svn" "export" "--non-interactive" "--trust-server-cert"
>> "-r" "59745"
>> "svn://www.tug.org/texlive/tags/texlive-2021.3/Master/texmf-dist/tcode/"
>> "/tmp/guix-directory.XcAtR6/tcode/" failed with status 1
>
> The importer is naive and assumes that any file in tlpdb starts with
> “texmf-dist/”.  For the xetex package there’s a file
> “tlpkg/tlpostcode/xetex.pl” and if you drop the same number of
> characters that you would for “texmf-dist/” you end up with
> “tcode/xetex.pl”.
>
> Oops!

I see you pushed some fixes for this (I didn't try with -r as that kept running for a while and might have tried to generate a whole lot of the dependency tree), thanks! I do get an import for xetex now.

And on that note I see cbf731ae481b434e657b05c80b4a32282e5d112b which adds texlive-xetex.

I can confirm with guix shell texlive-base texlive-fontspec texlive-xetex -- xelatex test.tex the output pdf has proper quotation marks and the log looks clean.

Many thanks!
John