bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Jelle Licht]

[-- Attachment #1: Type: text/plain, Size: 4887 bytes --]

I think I found a bug in our amsfonts texlive package. I will describe
my journey in finding this bug, as I still do not have clear picture
on the why/when/what is going on. I think I also saw several other
people running into this issue the last few months, so either way I am
happy to have found something reproducible that at least demonstrates
that I am sane :).

The eror message is:
" ! Math formula deleted: Insufficient extension fonts."

If you, like me, want to use Emacs' org-mode capabilities and export to
pdf using latex, by default you will generate an intermediate .tex file
that uses the ulem package. Using this package leads to the
aforementioned error message.

(Skip everything after this if you do not care about my descent into madness)

I used a profile containing the following (relevant) texlive packages:
--8<---------------cut here---------------start------------->8---
texlive-base	
texlive-latex-preview	
texlive-latex-base	
texlive-latexconfig	
texlive-fonts-ec	
texlive-latex-oberdiek	
texlive-latex-wrapfig	
texlive-generic-ulem	
texlive-latex-capt-of	
texlive-latex-hyperref
texlive-amsfonts	
texlive-fontinst	
texlive-metafont-base	
texlive-unicode-data	
texlive-pstool	
texlive-cm	
texlive-cm-super	
texlive-latex-amscls	
texlive-fonts-latex	
texlive-latex-amsmath	
--8<---------------cut here---------------end--------------->8---


I ran both `strace pdflatex working 2> working-strace.log' and `strace
pdflatex broken 2> broken-strace.log' See the attached `working.tex'
and `broken.tex' for tiny examples that demonstrate this.

The relevant part of the diff between straces:

* Working:
--8<---------------cut here---------------start------------->8---
access("/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/cmex7.tfm", R_OK) = 0
stat("/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/cmex7.tfm", {st_mode=S_IFREG|0444, st_size=940, ...}) = 0
openat(AT_FDCWD, "/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/cmex7.tfm", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0444, st_size=940, ...}) = 0
read(6, "\0\353\0\2\0\0\0\177\0#\0\6\0\16\0\3\0\0\0\0\0\34\0\r\27#\260\255\0p\0\0"..., 4096) = 940
close(6)                                = 0
openat(AT_FDCWD, "working.pdf", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 6
write(1, " [1", 3)                      = 3
--8<---------------cut here---------------end--------------->8---

* Broken:
--8<---------------cut here---------------start------------->8---
access("/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/euler/cmex7.tfm", R_OK) = 0
stat("/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/euler/cmex7.tfm", {st_mode=S_IFREG|0444, st_size=1312, ...}) = 0
openat(AT_FDCWD, "/home/jlicht/.guix-profile/share/texmf-dist/fonts/tfm/public/amsfonts/euler/cmex7.tfm", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0444, st_size=1312, ...}) = 0
read(6, "\1H\0\21\0\0\0\332\0*\0\20\0\20\0\6\0\0\0\0\0\0\0\6d\235jM\0\240\0\0"..., 4096) = 1312
close(6)                                = 0
write(1, "\n", 1)                       = 1
write(1, "! Math formula deleted: Insuffic"..., 54) = 54
--8<---------------cut here---------------end--------------->8---

We see that a different file is used when resolving the same font!
Furthermore, one of these fonts is a totally different size than the
other.


If we run: `guix build --check texlive-amsfonts | grep cmex7', we see:
--8<---------------cut here---------------start------------->8---
Font metrics written on /tmp/guix-build-texlive-amsfonts-49435.drv-0/source/build-fonts/cmex7.tfm.
Output written on /tmp/guix-build-texlive-amsfonts-49435.drv-0/source/build-fonts/cmex7.600gf (128 characters, 30684 bytes).
Transcript written on /tmp/guix-build-texlive-amsfonts-49435.drv-0/source/build-fonts/cmex7.log.
converting afm font cmex7
cmex7 CMEX7
`build-fonts/cmex7.600gf' -> `/gnu/store/hrxlw7s1d8q0z5kipizjr7ib49bw4hjp-texlive-amsfonts-49435/share/texmf-dist/fonts/tfm/public/amsfonts/cmex7.600gf'
`build-fonts/cmex7.tfm' -> `/gnu/store/hrxlw7s1d8q0z5kipizjr7ib49bw4hjp-texlive-amsfonts-49435/share/texmf-dist/fonts/tfm/public/amsfonts/cmex7.tfm'
`build-fonts/euler/cmex7.tfm' -> `/gnu/store/hrxlw7s1d8q0z5kipizjr7ib49bw4hjp-texlive-amsfonts-49435/share/texmf-dist/fonts/tfm/public/amsfonts/euler/cmex7.tfm'
--8<---------------cut here---------------end--------------->8---

AFAIK, and from looking at the full (and correctly working)
texlive-texmf build, the cmex7.tfm in `euler' is not correctly build.
My best guess is that this happens because cmex has both a mf file and a
afm file in `guix build --source texlive-amsfonts'. The one 'built'
using afm2tfm seems to be broken and/or not matching other metadata
generated, as given by this example.

Thanks for reading along, I hope we will find a solution to this, as
non-modular texlive is simply the worst :).

[-- Attachment #2: working.tex --]
[-- Type: application/x-tex, Size: 102 bytes --]

[-- Attachment #3: broken.tex --]
[-- Type: application/x-tex, Size: 120 bytes --]

[-- Attachment #4: Type: text/plain, Size: 10 bytes --]


 - Jelle

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Jelle Licht]

Jelle Licht <jlicht@fsfe.org> writes:

> The eror message is:
> " ! Math formula deleted: Insufficient extension fonts."
[snip]
> AFAIK, and from looking at the full (and correctly working)
> texlive-texmf build, the cmex7.tfm in `euler' is not correctly build.
> My best guess is that this happens because cmex has both a mf file and a
> afm file in `guix build --source texlive-amsfonts'. The one 'built'
> using afm2tfm seems to be broken and/or not matching other metadata
> generated, as given by this example.


I have found a workaround for my immediate problem, but I'm not nearly
enough of a tex guru to foresee any issues my changes might cause.

After some trial and error that took longer than I'm willing to admit, I
have the following snippet:
 
--8<---------------cut here---------------start------------->8---
diff --git a/gnu/packages/tex.scm b/gnu/packages/tex.scm
index cd461314b5..363c7a318c 100644
--- a/gnu/packages/tex.scm
+++ b/gnu/packages/tex.scm
@@ -1108,7 +1108,7 @@ Taco Hoekwater.")
                  ;; convert the afm files instead.
                  (let ((build (string-append (getcwd) "/build-fonts/euler")))
                    (mkdir build)
-                   (with-directory-excursion "fonts/afm/public/amsfonts/"
+                   (with-directory-excursion "fonts/afm/public/amsfonts/euler"
                      (for-each (lambda (font)
                                  (format #t "converting afm font ~a\n" (basename font ".afm"))
                                  (invoke "afm2tfm" font
--8<---------------cut here---------------end--------------->8---

With this patch applied, I can make use of the modular texlive system
from the comfort of Emacs + org. It could be that there are other 'ghost
fonts' haunting up the place. 

The following...
--8<---------------cut here---------------start------------->8---
guix refresh -l texlive-amsfonts
Building the following 1438 packages would ensure 3202 dependent packages are rebuilt
--8<---------------cut here---------------end--------------->8---

makes me think this is very much a disruptive change. I'm not in a hurry
to get this upstreamed, but if anyone could reproduce the problem (and
my fix...), I would be more confident in pushing it.

- Jelle

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Ricardo Wurmus]

I can reproduce this with a simpler document:

--8<---------------cut here---------------start------------->8---
\documentclass{article}
\usepackage{amsfonts}
\begin{document}
\begin{equation}
    x
\end{equation}
\end{document}
--8<---------------cut here---------------end--------------->8---

and this manifest

--8<---------------cut here---------------start------------->8---
(specifications->manifest
 (list
  "texlive-base"
  "texlive-amsfonts-fixed"
;;  "texlive-latex-amsmath"
  ))
--8<---------------cut here---------------end--------------->8---


-- 
Ricardo

bug#40558: (no subject)

[elaexuotee]

[-- Attachment #1.1: Type: text/plain, Size: 1101 bytes --]

I encountered a similar issue while trying to package something with a texlive-union input. Tracking down the issue has killed way too many hours.

FWIW, the `working.tex' minimal example is also giving me similar problems:

    $ guix describe
    Generation 28    5月 07 2020 01:10:02   (current)
      guix bed695a
        repository URL: https://git.savannah.gnu.org/git/guix.git
        branch: master
        commit: bed695aa94cd85800ec2c6296fb2d13e7ac29133
    $ guix environment --pure -e '((@ (gnu packages tex) texlive-union) `(,(@ (gnu packages tex) texlive-amsfonts)))'
    $ pdflatex working
    ...
    ! Math formula deleted: Insufficient symbol fonts.
    \)  ->\relax \ifmmode \ifinner $
                                    \else \@badmath \fi \else \@badmath \fi
    l.4 Hello! \(y = x^2\)
    ...

With the patch to texlive-amsfonts the above typesets just fine; however, metafont ends up generating cmmi10.657pk and cmr10.657pk font files. Is this expected? Typsetting it from the texlive installation of my foreign distro doesn't call out to metafont at all.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 260 bytes --]

bug#40558: (no subject)

[Jelle Licht]

elaexuotee@wilsonb.com writes:

> With the patch to texlive-amsfonts the above typesets just fine; however, metafont ends up generating cmmi10.657pk and cmr10.657pk font files. Is this expected? Typsetting it from the texlive installation of my foreign distro doesn't call out to metafont at all.

As I mentioned earlier, I am not a tex expert at all. I have no clue,
but if my patch makes spooky things happen, we should probably hold off
on applying it.

- Jelle

bug#40558: (no subject)

[Ricardo Wurmus]

elaexuotee@wilsonb.com writes:

> With the patch to texlive-amsfonts the above typesets just fine;
> however, metafont ends up generating cmmi10.657pk and cmr10.657pk font
> files. Is this expected? Typsetting it from the texlive installation
> of my foreign distro doesn't call out to metafont at all.

This is a problem with the modular TeX Live packages.  The pk files are
bitmap fonts.  I found that adding texlive-cm-super to the union helps
in that the bitmap variants of the CM fonts will not be generated any
more.  More font packages may be needed in the union to prevent TeX from
falling back to bitmap fonts in other cases.

In any case, that’s unrelated to Jelle’s patch, which looks fine to me.

-- 
Ricardo

bug#40558: (no subject)

[elaexuotee--- via Bug reports for GNU Guix]

Ricardo Wurmus <rekado@elephly.net> wrote:
> This is a problem with the modular TeX Live packages.  The pk files are
> bitmap fonts.  I found that adding texlive-cm-super to the union helps
> in that the bitmap variants of the CM fonts will not be generated any
> more.  More font packages may be needed in the union to prevent TeX from
> falling back to bitmap fonts in other cases.
> 
> In any case, that’s unrelated to Jelle’s patch, which looks fine to me.

Thank you, Ricardo, for looking into this.

I have lost signficant amounts of hair trying to find a solution. Adding
texlive-cm-super doesn't help for the document I trying to typeset. In fact, I
even grabbed all texlive packages with fonts and threw them in the
texlive-union to no effect. No matter what, pdflatex bails when trying to find
the font to set $~$.

The only way I have gotten it to typeset under a texlive-union so far is by
munging texlive-amsfonts as Jelle mentioned.

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Ricardo Wurmus]

elaexuotee@wilsonb.com writes:

> Ricardo Wurmus <rekado@elephly.net> wrote:
>> This is a problem with the modular TeX Live packages.  The pk files are
>> bitmap fonts.  I found that adding texlive-cm-super to the union helps
>> in that the bitmap variants of the CM fonts will not be generated any
>> more.  More font packages may be needed in the union to prevent TeX from
>> falling back to bitmap fonts in other cases.
>> 
>> In any case, that’s unrelated to Jelle’s patch, which looks fine to me.
>
> Thank you, Ricardo, for looking into this.
>
> I have lost signficant amounts of hair trying to find a solution. Adding
> texlive-cm-super doesn't help for the document I trying to typeset. In fact, I
> even grabbed all texlive packages with fonts and threw them in the
> texlive-union to no effect. No matter what, pdflatex bails when trying to find
> the font to set $~$.
>
> The only way I have gotten it to typeset under a texlive-union so far is by
> munging texlive-amsfonts as Jelle mentioned.

I have since added texlive-amsfonts/fixed, which installs all the files
it is supposed to (according do the tlpdb).  I’ve also since fixed font
search.

Can this issue be closed?

-- 
Ricardo

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[elaexuotee--- via Bug reports for GNU Guix]

Ricardo Wurmus <rekado@elephly.net> wrote:
> 
> elaexuotee@wilsonb.com writes:
> 
> > Ricardo Wurmus <rekado@elephly.net> wrote:
> >> This is a problem with the modular TeX Live packages.  The pk files are
> >> bitmap fonts.  I found that adding texlive-cm-super to the union helps
> >> in that the bitmap variants of the CM fonts will not be generated any
> >> more.  More font packages may be needed in the union to prevent TeX from
> >> falling back to bitmap fonts in other cases.
> >> 
> >> In any case, that’s unrelated to Jelle’s patch, which looks fine to me.
> >
> > Thank you, Ricardo, for looking into this.
> >
> > I have lost signficant amounts of hair trying to find a solution. Adding
> > texlive-cm-super doesn't help for the document I trying to typeset. In fact, I
> > even grabbed all texlive packages with fonts and threw them in the
> > texlive-union to no effect. No matter what, pdflatex bails when trying to find
> > the font to set $~$.
> >
> > The only way I have gotten it to typeset under a texlive-union so far is by
> > munging texlive-amsfonts as Jelle mentioned.
> 
> I have since added texlive-amsfonts/fixed, which installs all the files
> it is supposed to (according do the tlpdb).  I’ve also since fixed font
> search.
> 
> Can this issue be closed?

Are we sure this is fixed? The issue where you added texlive-amsfonts/fixed is
still seeing the original missing fonts error for eufm10:

https://issues.guix.gnu.org/53339#3-lineno36

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Ricardo Wurmus]

elaexuotee@wilsonb.com writes:

> Ricardo Wurmus <rekado@elephly.net> wrote:
>> 
>> elaexuotee@wilsonb.com writes:
>> 
>> > Ricardo Wurmus <rekado@elephly.net> wrote:
>> >> This is a problem with the modular TeX Live packages.  The pk files are
>> >> bitmap fonts.  I found that adding texlive-cm-super to the union helps
>> >> in that the bitmap variants of the CM fonts will not be generated any
>> >> more.  More font packages may be needed in the union to prevent TeX from
>> >> falling back to bitmap fonts in other cases.
>> >> 
>> >> In any case, that’s unrelated to Jelle’s patch, which looks fine to me.
>> >
>> > Thank you, Ricardo, for looking into this.
>> >
>> > I have lost signficant amounts of hair trying to find a solution. Adding
>> > texlive-cm-super doesn't help for the document I trying to typeset. In fact, I
>> > even grabbed all texlive packages with fonts and threw them in the
>> > texlive-union to no effect. No matter what, pdflatex bails when trying to find
>> > the font to set $~$.
>> >
>> > The only way I have gotten it to typeset under a texlive-union so far is by
>> > munging texlive-amsfonts as Jelle mentioned.
>> 
>> I have since added texlive-amsfonts/fixed, which installs all the files
>> it is supposed to (according do the tlpdb).  I’ve also since fixed font
>> search.
>> 
>> Can this issue be closed?
>
> Are we sure this is fixed? The issue where you added texlive-amsfonts/fixed is
> still seeing the original missing fonts error for eufm10:
>
> https://issues.guix.gnu.org/53339#3-lineno36

At least the question “is texlive-amsfonts broken” is definitively
answered.  This was what this issue was about, no?  I’d rather keep the
other issue separate.

-- 
Ricardo

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[elaexuotee--- via Bug reports for GNU Guix]

> At least the question “is texlive-amsfonts broken” is definitively
> answered.  This was what this issue was about, no?  I’d rather keep the
> other issue separate.

Well, it's still broken in the sense that we're not able to typeset with
eufm10, no? That said, I guess it could make sense to define this issue as
"tlpdb non-conformance problems" and close as fixed.

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Ricardo Wurmus]

[-- Attachment #1: Type: text/plain, Size: 496 bytes --]


elaexuotee@wilsonb.com writes:

>> At least the question “is texlive-amsfonts broken” is definitively
>> answered.  This was what this issue was about, no?  I’d rather keep the
>> other issue separate.
>
> Well, it's still broken in the sense that we're not able to typeset with
> eufm10, no?

Something might be broken, but it’s not texlive-amsfonts.

Going back to the first message in this bug report here I can no longer
reproduce the problem.  I used this manifest:


[-- Attachment #2: manifest --]
[-- Type: text/plain, Size: 587 bytes --]

(specifications->manifest
 (list "texlive-amscls"	
       "texlive-amsfonts"	
       "texlive-base"	
       "texlive-cm"	
       "texlive-cm-super"	
       "texlive-fontinst"	
       "texlive-fonts-ec"	
       "texlive-fonts-latex"	
       "texlive-generic-ulem"	
       "texlive-hyperref"
       "texlive-latex-amsmath"
       "texlive-latex-base"	
       "texlive-latex-capt-of"	
       "texlive-latex-preview"	
       "texlive-latex-wrapfig"	
       "texlive-latexconfig"	
       "texlive-metafont"	
       "texlive-oberdiek"	
       "texlive-pstool"	
       "texlive-unicode-data"))

[-- Attachment #3: Type: text/plain, Size: 21 bytes --]


And this TeX file:


[-- Attachment #4: broken.tex --]
[-- Type: application/x-tex, Size: 120 bytes --]

[-- Attachment #5: Type: text/plain, Size: 211 bytes --]


Running pdflatex on the file throws no errors and it produces a PDF file
as expected.

So I’ll close this issue.  I suggest we keep investigating the problem
with eufm10 in issue 53339.

-- 
Ricardo

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[zimoun]

Hi Ricardo,

On Fri, 21 Jan 2022 at 18:21, Ricardo Wurmus <rekado@elephly.net> wrote:

>> The only way I have gotten it to typeset under a texlive-union so far is by
>> munging texlive-amsfonts as Jelle mentioned.
>
> I have since added texlive-amsfonts/fixed, which installs all the files
> it is supposed to (according do the tlpdb).  I’ve also since fixed font
> search.
>
> Can this issue be closed?

I reopen the issue because it appears to be broken for beamer.

--8<---------------cut here---------------start------------->8---
\documentclass{beamer}
\begin{document}
\begin{frame}
  \begin{tabular}{c|c}
    foo & bar
  \end{tabular}
\end{frame}
\end{document}
--8<---------------cut here---------------end--------------->8---

leads to the error:

--8<---------------cut here---------------start------------->8---
! Math formula deleted: Insufficient extension fonts.
\endtabular ->\crcr \egroup \egroup $
                                     \egroup
l.7 \end{frame}

? --8<---------------cut here---------------end--------------->8---

Weird, isn't?  That's because 'tabular' uses some math stuff for
aligning, IIUC.  Otherwise, this example:

--8<---------------cut here---------------start------------->8---
\documentclass{beamer}
\begin{document}
\begin{frame}
  \begin{equation}
    x
  \end{equation}
\end{frame}
\end{document}
--8<---------------cut here---------------end--------------->8---


Using Guix 4dffece, the invokation is:

   guix shell -C -m manifest.scm -- pdflatex foo.tex

where 'manifest.scm' reads:

--8<---------------cut here---------------start------------->8---
(specifications->manifest
 (list
  "rubber"

  "texlive-base"
  "texlive-fonts-ec"
  "texlive-kpfonts"
  "texlive-cm-super"
  "texlive-amsfonts-fixed"

  "texlive-beamer"
  "texlive-translator"
  "texlive-ulem"
  "texlive-capt-of"
  "texlive-hyperref"
  "texlive-carlisle"

  "texlive-latex-geometry"
  "texlive-latex-wrapfig"
  "texlive-latex-amsmath"
  "texlive-babel-french"
  "texlive-latex-listings"
  ))
--8<---------------cut here---------------end--------------->8---

(Maybe I miss a package in the manifest?)


Last, note it works using the BIG 'texlive' package.

Cheers,
simon

bug#40558: Modular TexLive "Insufficient extension fonts" and duplicate fonts

[Ricardo Wurmus]

This is now fixed with commit 64fcf9508af318cc2d71811815cfbe99063867b1.

The cause: we generated the tfm files from afm and pl files in a
misguided attempt to build as many things from “source” as possible.
Turns out that this results in bad tfm files.  These files mere meant to
be installed just as they are included in the upstream bundle.

That’s what we’re doing now.

-- 
Ricardo