bug#73161: 29.4; Exporting Org Agenda to PDF with Cyrillic Characters

[Eli Zaretskii <eliz@gnu.org>]

> Date: Mon, 09 Sep 2024 22:38:13 +0000
> From:  Serghei Iakovlev via "Bug reports for GNU Emacs,
>  the Swiss army knife of text editors" <bug-gnu-emacs@gnu.org>
> 
> I’m reaching out because I’ve hit a wall after spending many hours
> debugging what seemed like a straightforward issue but turned out to
> be a rabbit hole.  I’ve been working on customizing `org-agenda'
> export to PDF and initially thought my problem was related to
> `ox-texinfo'.  I spent half the night tweaking my Emacs
> configuration, digging through documentation, and adjusting settings
> in hopes of resolving the issue.  I went down a lot of wrong paths
> before realizing that `org-store-agenda-views' doesn't actually call
> `ox-texinfo'.  That was my first mistake - not diving into the source
> code earlier.
> 
> Turns out, the function calls `org-agenda-write', which in turn
> executes something entirely different.  Relevant part of
> org-agenda.el:
> 
> --8<---------------cut here---------------start------------->8---
> (require 'ps-print)
> (ps-print-buffer-with-faces
>   (concat (file-name-sans-extension file) ".ps"))
> (call-process "ps2pdf" nil nil nil
>   (expand-file-name
>     (concat (file-name-sans-extension file) ".ps"))
>   (expand-file-name file))
> (delete-file (concat (file-name-sans-extension file) ".ps"))
> (message "PDF written to %s" file)
> --8<---------------cut here---------------end--------------->8---

IMO, Org should document that org-agenda-write uses ps-print and
ps2pdf, so that users could know where to look for cause of trouble
and which parts to customize to work around these problems.

> So, after realizing this, I thought I could fix the issue by tweaking
> `ps-print' and `ps2pdf' settings.  I’ve tried adjusting my
> `org-agenda-exporter-settings' multiple times, setting various font
> families and print headers, and I’ve seen some changes reflected in
> the output.  For instance, I can see that the header is removed, and
> the font face in the PDF actually changes when I tweak
> `ps-font-family'.  But no matter what I try, Cyrillic characters
> still show up as question marks or garbled symbols.
> 
> Here’s a snippet of my best try:
> 
> --8<---------------cut here---------------start------------->8---
> (setq org-agenda-exporter-settings
>   '((ps-multibyte-buffer 'bdf-font-except-latin)
>     (ps-print-header nil)
>     (ps-print-color-p 'black-white)
>     (ps-font-family 'Helvetica)
>     (htmlize-output-type 'css)))
> --8<---------------cut here---------------end--------------->8---
> 
> I’ve tried changing `ps-multibyte-buffer' to `bdf-font-except-latin',
> thinking it would handle non-Latin characters, but that hasn’t
> worked.  I’ve messed around with various `ps-font-family' values, tried
> specifying paths to BDF fonts, and set up `bdf-directory-list', but
> still no dice.  I’ve also been battling with `ps2pdf', which seems to
> completely ignore the fonts I’m trying to set up, reverting to
> something that just doesn’t support Cyrillic.

I don't know about ps2pdf, but I did at one time, years ago, have a
working setup that was capable of using ps-print to print Cyrillic
text from Emacs to a PostScript printer.  I will try to describe what
I think I did back then, using my old files and archives to recollect
stuff I've long forgotten, in the hope it will help you.

> As far as I can tell, `ps-mule' expects me to install BDF fonts and set
> the path to them in `bdf-directory-list'.

Yes.  Here's what I had back when I printed Cyrillic text using
ps-print:

  (setq ps-multibyte-buffer 'bdf-font-except-latin)
  (setq bdf-directory-list (list (concat invocation-directory "../fonts/bdf")))

(The value of bdf-directory-list is just my personal choice back then:
you can have any directory there, just make sure the BDF fonts arfe
really in that directory.  But it must be a list, even if you just
have a single directory.)

> If this is indeed the reason why I’m seeing garbled text instead of
> proper characters, then my situation might be even worse.  The
> workstation I’m working on is running macOS, and as far as I know,
> there's no native support for pcf/bdf fonts in macOS.  At least,
> I've never heard of anyone using these on macOS.

The BDF fonts I used are from the GNU Intlfonts.  I think they are the
only ones that are guaranteed to work with ps-mule and ps-bdf, or at
least worked in the past (see below).  AFAICT, you can still download
them from here:

  https://ftp.gnu.org/gnu/intlfonts/intlfonts-1.2.1.tar.gz

When you unpack the tarball, chdir to the intlfonts-1.2.1 created by
unpacking, and type:

  $ make bdf INSTALLDIR=/path/to/where/to/install

This should install the BDF fonts in the specified directory, and you
should point bdf-directory-list to it.  (You can delete the
intlfonts-1.2.1 tree after you do the above.)

AFAIR, this was all I needed back then; I hope I don't misremember.

However: I last used this with Emacs 21, which is before Emacs was
converted to Unicode-based treatment of non-ASCII characters.  So it
is possible that the above will not work with the current codebase,
because ps-mule needs to decide which of the BDF fonts to use for each
script (in your case Cyrillic), and I just don't remember if this code
was adjusted to the changes we made in Emacs 23 (I think it was, but I
am not sure).

And finally: the situation with ps-print wrt non-ASCII text is not a
good one.  Even if BDF fonts still work, they are somewhat ugly, and
are nowadays almost unsupported by systems, so it is hard to find nice
BDF fonts.  And ps-mule doesn't support anything else.  There were
plans to write ps-type1 to support Type1 fonts, but it never happened.

Unfortunately, this basically means that any Lisp program that builds
on ps-print will only reasonably support ASCII and Latin-1 text, the
rest of scripts will need to use BDF fonts, and will probably not look
very nice.  IMO, Org should document this limitation as well.

What ps2pdf will do with the PS files created by ps-print under the
above setup, I don't know.  Don't hold your breath.

HTH, and sorry for your journey down this rabbit hole.