[bug#63604] Branch "tex-team-next" in need of review

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

Hello,

I just pushed branch named "tex-team-next", based on current "tex-team"
branch. At its core, it merely modifies the texlive build system.
Consequently, it also rewrites (almost) all texlive-prefixed packages
accordingly, and tweaks the texlive importer to follow suit.

The most obvious differences with the last iteration of this build
system are that `simple-texlive-package' function along with its
`#:trivial?' argument disappear, and the `#:tex-directory' keyword is no
longer used. Package definitions are now more regular, in comparison to
other packages in Guix.

Internally, this build system now:

- tries to build ".dtx" files when no build targets, through
  `#:build-targets keyword', are specified and no ".ins" file are
  present in the source;

- generates font metrics (".tfm") from Metafont files (".mf"), with the
  consequence of adding a new `#:ignore-metafont-files' keyword for
  a few tricky font packages;

- guarantees that no cruft, e.g., ".log" files, is installed in the
  output by comparing generated files with those expected by the TeX
  Live distribution;

Since these changes affected almost all packages in "tex.scm", I took
the opportunity to:

- normalize package names, so that every package is now named after the
  texlive-TEXLIVENAME scheme, where TEXLIVENAME is the name of the
  matching package in the TeX Live distribution;

- add appropriate propagated inputs, which will hopefully drastically
  reduce the number of packages to declare when installing a working,
  minimal, and modular bundle;

- created new packages required as propagated inputs, and a couple
  others to try out the font metrics generation on peculiar setups.

Unfortunately, these changes do not affect the "bootstrap" part of the
texlive packages, e.g., `texlive-bin', `texlive-latex-base', or
`texlive-hyphen-package'… While creating new texlive packages is
certainly, all of this doesn't help in updating packages individually or
as a whole. Neither does it solve the dreaded hyphenation issue. Sorry.

In order to help anyone interested in reviewing this 400+ packages set,
here are some guidelines. The noteworthy patches are:

- patches 1 and 389, about the real changes to the build system;
- patches 3, 4, 5, 6 and 390, about refinements to its relative
  importer.

These ones should be reviewed, since this is where the Guile code lives.
The others are not so interesting and can be merrily skipped. Anyway,

- patches 2, 360, and 388 are the steps taken to ultimately remove
  `simple-texlive-package' function without breaking every build right
  from the beginning;

- everything else (!) is just boring package rewriting, or addition.

Feedback welcome,

Regards,
-- 
Nicolas Goaziou

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Since these changes affected almost all packages in "tex.scm", I took
> the opportunity to:
>

[...]

> - add appropriate propagated inputs, which will hopefully drastically
>   reduce the number of packages to declare when installing a working,
>   minimal, and modular bundle;

On second thought, this may not be a great idea.

OTOH, it requires manual intervention (grepping for RequirePackage
through ".sty" and ".cls" files, and finding out the corresponding TeX
Live packages).

OTOH,

  guix import texlive --recursive collection-whatever

requires only a small fix to produce a useful output.

So, it may be better to remove "invisible" propagated inputs from
packages, keeping only those specified by TeX Live, and start providing
collections.

WDYT?

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

Nicolas Goaziou <mail@nicolasgoaziou.fr> writes:

> Internally, this build system now:

[...]

> - generates font metrics (".tfm") from Metafont files (".mf"), with the
>   consequence of adding a new `#:ignore-metafont-files' keyword for
>   a few tricky font packages;

I improved the font metrics generation, and the new
#:ignore-metafont-files keyword is not necessary anymore. I removed it
from the code base.

Also, the importer now behaves better when importing meta-packages, such
as collection-basic or… scheme-full.

I'm Cc'ing the TeX team.

Regards,

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

I just pushed an update on tex-team-next branch.

The biggest change is about hyphenations. I created one big package
containing all hyphenations, named `texlive-hyphen-complete'.
`texlive-hyphen-base' still exists for low-level packages, such as
`texlive-tex', but every other hyphenation related package has been
deprecated and removed.

Also, the build system now generates TeX formats on demand, with the
#:generate-formats keyword.

At this point, with the following command,

  ./pre-inst-env guix shell --pure texlive-bin texlive-ec texlive-latex-base texlive-babel-french texlive-carlisle -- latex /tmp/test.tex

I can compile the following LaTeX document _without any hyphenation
warning_:

--8<---------------cut here---------------start------------->8---
\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[french]{babel}
\begin{document}
Success!
\end{document}
--8<---------------cut here---------------end--------------->8---

So, what's next?

- Install executables (copied from `texlive-bin') along with the texlive
  package, when appropriate. The more packages you install, the less you
  need `texlive-bin'. Ultimately, I want to get rid of `texlive-bin' as
  a native input for every package. I think it should be limited to the
  first few packages needed to build the rest. If that works, the
  #:texlive-bin keyword can go away.

- Replace `texlive-latex-base', which doesn't exist in TeX Live, with
  `texlive-latex-bin'.  Consequently, `#:texlive-latex-base' would be
  renamed `#:texlive-latex-bin'.

- Limit propagated inputs to the bare minimum, as defined in Tex Live,
  and, at the same time, introduce collections and schemes. For example,
  `texlive-scheme-basic' would replace `texlive-base'.

Regards,

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

Here's another update for tex-team-next branch.

The biggest change is the removal of `texlive-base' and
`texlive-latex-base' packages, which do not exist as TeX Live packages.
They are replaced by, respectively, `texlive-scheme-basic' and
`texlive-latex-bin'.

Consequently, the `#:texlive-latex-base' argument no longer exists. It
is replaced by `#:texlive-latex-bin?'. The importer automatically set it
to #f when necessary.

I also started to introduce collections and schemes from TeX Live.  At
this point, the following command,

  ./pre-inst-env guix shell --pure texlive-scheme-basic texlive-babel-french -- latex /tmp/test.tex

can compile the following LaTeX document without any hyphenation warning:

--8<---------------cut here---------------start------------->8---
\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage[french]{babel}
\begin{document}
Success!
\end{document}
--8<---------------cut here---------------end--------------->8---

> So, what's next?
>
> - Install executables (copied from `texlive-bin') along with the texlive
>   package, when appropriate. The more packages you install, the less you
>   need `texlive-bin'. Ultimately, I want to get rid of `texlive-bin' as
>   a native input for every package. I think it should be limited to the
>   first few packages needed to build the rest. If that works, the
>   #:texlive-bin keyword can go away.

I admit that plan failed. Many binaries are tightly bound to each other.
I couldn't extract each of them and associate them to the right package.

It would be possible, however, to split `texlive-bin' into multiple
independent outputs (as NixOS does) in order to reduce the size of TeX
Live packages. Regarding that size, there's also the issue (at least it
seems to me) that `texlive-bin' retains a reference to GCC.

I won't look into any of the above in the current "tex-team-next"
branch.

> - Replace `texlive-latex-base', which doesn't exist in TeX Live, with
>   `texlive-latex-bin'.  Consequently, `#:texlive-latex-base' would be
>   renamed `#:texlive-latex-bin'.

Done.

> - Limit propagated inputs to the bare minimum, as defined in Tex Live,
>   and, at the same time, introduce collections and schemes. For example,
>   `texlive-scheme-basic' would replace `texlive-base'.

I started this. Since I need to review each package to reduce propagated
inputs, I may as well consider updating TeX Live to a more recent
release first. I'll see if I can do this during the next days.
Unfortunately, building `texlive-bin' takes a long time.

Feedback welcome.

Regards,

[bug#63604] Branch "tex-team-next" in need of review

[Nicolas Goaziou]

Hello,

The "tex-team-next" branch has reached a milestone, so I started the
process of merging it into "master".

In a nutshell, this branch provides:

 - TeX Live 2023 ;

 - more packages, including some collections and schemes ;

 - a beefed-up importer : it handles most native inputs and arguments.
   It can also produce meta-packages ;

 - normalized package names and definitions : no more
   `simple-texlive-package', only a limited set of Guix specific
   packages, described in a comment at the beginning of the module ;

 - a more voluntary build system : it can generate runfiles, build font
   metrics, generate TeX formats, and install scripts ;

 - a lighter `texlive-bin' : it no longer provides every TeX Live script
   right from the start, reducing (!) the number of files added to
   "bin/" from 400 to 200 ;

 - an actually usable modular TeX Live, with no more hyphenation issues,
   and less packages to list thanks to collections.

Adding new packages is very easy, but they still have to be updated
manually every time we bump TeX Live version. It would be nice to buy us
an updater at some point.

I also think we can further modularize `texlive-bin': some binaries may
be built along with their respective texlive package instead. Food for
thoughts.

Regards,
-- 
Nicolas Goaziou

[bug#63604] Closing?

[Andreas Enge]

Hello,

as far as I can tell, the branch has been merged, can this issue
be closed?

Andreas

bug#63604: Closing?

[Nicolas Goaziou]

Hello,

Andreas Enge <andreas@enge.fr> writes:

> as far as I can tell, the branch has been merged, can this issue
> be closed?

Sure. Done.

Regards,
-- 
Nicolas Goaziou