[Question] How to change the height of the box for unprintable glyphs?

[Question] How to change the height of the box for unprintable glyphs?

[Rodrigo Morales]

* The context

In some of my use cases, I feel the need to use a bitmap font in a =*terminal*= buffer.

Some days ago, I noticed that some lines in a =*terminal*= buffer were shown with a different height. This caused the tmux status bar when it is shown at the top to be shown out of the =*terminal*= buffer. This is undesired behavior. I learned that this happened because Emacs tries to display some characters using different fonts which might have different heights. For this reason, I started looking for a bitmap font which defines glyphs for all Unicode characters. Thus, by using a single font, the height of all lines would be the same. I found GNU Unifont.

I downloaded the file =unifont-15.1.05.pcf.gz= from https://unifoundry.com/unifont/ . Then, I decompressed the file using =gunzip= and copied the file =unifont-15.1.05.bdf= to =~/.fonts=. Finally, I executed =fc-cache -f -v=.

I launched =emacs -Q /tmp/a.txt= (the contents of =/tmp/a.txt= are shown in the first code block below).

#+HEADER: :tangle /tmp/a.txt
#+BEGIN_SRC text
Line 1: Hello!
Line 2: 你好！
Line 3: 𑂩𑂰𑂧𑂩𑂰𑂧 ("Hello" in Kaithi, retrieved from HELLO buffer in GNU Emacs 29.3)
#+END_SRC

I then tried to make Emacs display all characters only using Unifont. If the glyph is not defined by Unifont, then a box with the hexadecimal notation of the character should be shown. I accomplished this by evaluating the following sexpL

#+HEADER: :results silent
#+BEGIN_SRC elisp
(progn
 (set-fontset-font "fontset-startup" '(#x000000 . #x3FFFFF) "Unifont")
 (set-fontset-font "fontset-default" '(#x000000 . #x3FFFFF) "Unifont"))
#+END_SRC

Upon evaluation of the sexp, all characters in the buffer =a.txt= except characters in line 3 were shown using GNU Unifont, boxes with hexadecimal notation were shown in line 3. See screenshot in http://web.archive.org/web/20240515172349/http://0x0.st/XK5h.png

The boxes shown in line 3 significantly increased the height of that line. =(line-pixel-height= returned 16 in line 1 and line 2, but it returned 37 in line 3.

* The question

How to reduce the height of the rectangle containing hexadecimal notation that Emacs uses to display characters for which no font was found?

* The XY problem

To avoid [[https://www.perlmonks.org/?node_id=542341][the XY problem]], I want to make clear that my main goal is to use a bitmap font in a =*terminal*= buffer and ensure that all lines have the same height (the X). One way that I found to accomplish this is by using a bitmap font that defines glyphs for most characters. However, this raised the problem that undefined glyphs are shown with a larger height (the Y).

Re: [Question] How to change the height of the box for unprintable glyphs?

[Eli Zaretskii]

> Date: Wed, 15 May 2024 12:37:43 -0500
> From: Rodrigo Morales <me@rodrigomorales.site>
> 
> Some days ago, I noticed that some lines in a =*terminal*= buffer were shown with a different height. This caused the tmux status bar when it is shown at the top to be shown out of the =*terminal*= buffer. This is undesired behavior. I learned that this happened because Emacs tries to display some characters using different fonts which might have different heights. For this reason, I started looking for a bitmap font which defines glyphs for all Unicode characters. Thus, by using a single font, the height of all lines would be the same. I found GNU Unifont.

Side note: IMNSHO, Unifont is a very ugly font, so I don't recommend
this solution.

> Upon evaluation of the sexp, all characters in the buffer =a.txt= except characters in line 3 were shown using GNU Unifont, boxes with hexadecimal notation were shown in line 3. See screenshot in http://web.archive.org/web/20240515172349/http://0x0.st/XK5h.png
> 
> The boxes shown in line 3 significantly increased the height of that line. =(line-pixel-height= returned 16 in line 1 and line 2, but it returned 37 in line 3.
> 
> * The question
> 
> How to reduce the height of the rectangle containing hexadecimal notation that Emacs uses to display characters for which no font was found?

You can modify the face used for this display.  It's called
'glyphless-char'; if you make it small enough, the box will become
smaller as well, I think.

But I think a better idea is to customize
glyphless-char-display-control such that one of the other alternative
displays is used for characters for which there's no font.

Re: [Question] How to change the height of the box for unprintable glyphs?

[Rodrigo Morales]

Eli Zaretskii <eliz@gnu.org> writes:

> Side note: IMNSHO, Unifont is a very ugly font, so I don't recommend
> this solution.

Thanks for the suggestion. I'm still looking for a font, Unifont is not the last decision. If you know an alternative, please let me know.

This is my use case: I usually use tmux in a terminal emulator. I display the tmux status bar at the top. In order to show the most number of lines in a window, I prefer using extremely small fonts (to me, getting as much lines/information as possible in the window is more important than aesthetics). I feel that extremely small fonts bitmap fonts are easier to read than extremely small OpenType fonts. In Emacs, some characters have different height, they increase the height of the lines where they are shown. These lines with increased height make the tmux status bar not to be visible in the =*terminal*= window. This is not convenient for me because the tmux status bar contains information that I need to always see.

These are the possible alternatives that I have thought:

Alternative 1: Find a bitmap font that defines a glyph for most characters. [[https://unifoundry.com/unifont/index.html][Unifont]] does that. The problem is that Emacs presents [[https://lists.gnu.org/archive/html/help-gnu-emacs/2024-05/msg00242.html][this issue]] when using that font. In addition to that, the font has a single size: 16x16. I'm looking for a smaller font. Previously, I had used =5x7.bdf= from the [[https://gitlab.freedesktop.org/xorg/font/misc-misc][=misc-misc= font]].

Alternative 2: Find a way to make Emacs force a line height for all lines in a buffer regardless of the fonts that are being used. If I get to do that, I could use =5x7.bdf= in =*terminal*=, and the characters that are not defined by that font would be shown with other fonts. I don't know how to do this but I can investigate.

Alternative 3: If it is not feasible to force a line height for all lines in a buffer, find a way to make Emacs use a single font in a buffer, those characters that are not supported by the font would be shown as empty boxes, these boxes shouldn't increase the line height. Thus, I would be able to use =5x7.bdf= in =*terminal*=. I know that =5x7.bdf= supports a small set of characters, so many characters would be shown as empty boxes. This wouldn't be too much of a problem because I would only use this solution when I work with ASCII characters.

Please let me know your thoughts.

Re: [Question] How to change the height of the box for unprintable glyphs?

[Manuel Giraud via Users list for the GNU Emacs text editor]

Rodrigo Morales <me@rodrigomorales.site> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Side note: IMNSHO, Unifont is a very ugly font, so I don't recommend
>> this solution.
>
> Thanks for the suggestion. I'm still looking for a font, Unifont is
> not the last decision. If you know an alternative, please let me know.
>
> This is my use case: I usually use tmux in a terminal emulator. I
> display the tmux status bar at the top. In order to show the most
> number of lines in a window, I prefer using extremely small fonts (to
> me, getting as much lines/information as possible in the window is
> more important than aesthetics). I feel that extremely small fonts
> bitmap fonts are easier to read than extremely small OpenType
> fonts.

[...]

Hi,

I think you could Ttyp0 a try:
https://people.mpi-inf.mpg.de/~uwe/misc/uw-ttyp0/

It is a bitmap font with an impressive glyph coverage.

Best regards,
-- 
Manuel Giraud

Re: [Question] How to change the height of the box for unprintable glyphs?

[Eli Zaretskii]

> From: Manuel Giraud <manuel@ledu-giraud.fr>
> Cc: Eli Zaretskii <eliz@gnu.org>,  <help-gnu-emacs@gnu.org>
> Date: Thu, 16 May 2024 21:08:39 +0200
> 
> I think you could Ttyp0 a try:
> https://people.mpi-inf.mpg.de/~uwe/misc/uw-ttyp0/
> 
> It is a bitmap font with an impressive glyph coverage.

Why would anyone want to use bitmap fonts these days?

Re: [Question] How to change the height of the box for unprintable glyphs?

[Manuel Giraud via Users list for the GNU Emacs text editor]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Manuel Giraud <manuel@ledu-giraud.fr>
>> Cc: Eli Zaretskii <eliz@gnu.org>,  <help-gnu-emacs@gnu.org>
>> Date: Thu, 16 May 2024 21:08:39 +0200
>> 
>> I think you could Ttyp0 a try:
>> https://people.mpi-inf.mpg.de/~uwe/misc/uw-ttyp0/
>> 
>> It is a bitmap font with an impressive glyph coverage.
>
> Why would anyone want to use bitmap fonts these days?

I sometimes do when I am sick of the visual blur (that I know would go
away with a better hardware) and that is what OP asked for.
-- 
Manuel Giraud

Re: [Question] How to change the height of the box for unprintable glyphs?

[Rodrigo Morales]

I think I have found what I was looking for.

I searched occurences of the word =glyphless= in the source code of Emacs and I found the variable =glyphless-char-display-control=. This variable controls the handling of characters for which there is no font that can display them.

#+BEGIN_SRC text
glyphless-char-display-control is a variable defined in ‘characters.el’.

Its value is
((format-control . thin-space)
 (variation-selectors . thin-space)
 (no-font . hex-code))
#+END_SRC

I changed the cdr of =no-font= from =hex-code= to =empty-box= as shown in the minimal Emacs configuration below. I used =customize-set-variable= because in the docstring of =glyphless-char-display-control=, I read that I should not set its value directly from Lisp because the variable takes effect via a custom =:set=.

#+HEADER: :tangle ~/.config/emacs/init.el
#+BEGIN_SRC elisp
(set-fontset-font "fontset-startup" '(#x000000 . #x3FFFFF) "Unifont")
(set-fontset-font "fontset-default" '(#x000000 . #x3FFFFF) "Unifont")
(customize-set-variable
 'glyphless-char-display-control
 '((format-control . thin-space)
   (variation-selectors . thin-space)
   (no-font . empty-box)))
#+END_SRC

I launched emacs using =emacs /tmp/a.txt=. The content of =/tmp/a.txt= is shown below.

#+HEADER: :tangle /tmp/a.txt
#+BEGIN_SRC text
Line 1: Hello!
Line 2: 你好！
Line 3: 𑂩𑂰𑂧𑂩𑂰𑂧 ("Hello" in Kaithi, retrieved from HELLO buffer in GNU Emacs 29.3)
#+END_SRC

I evaluated =(line-pixel-height)= in the three lines and it returned =16= in every line.

[[http://web.archive.org/web/20240515192102/http://0x0.st/XKRd.png][This screenshot]] shows the results returned by =line-pixel-height= in the three lines with the following configuration:

#+HEADER: :tangle ~/.config/emacs/init.el
#+BEGIN_SRC elisp
(set-fontset-font "fontset-startup" '(#x000000 . #x3FFFFF) "Unifont")
(set-fontset-font "fontset-default" '(#x000000 . #x3FFFFF) "Unifont")
#+END_SRC

[[http://web.archive.org/web/20240515192227/http://0x0.st/XKRn.png][This screenshot]] shows the results returned by =line-pixel-height= in the three lines with the following configuration:

#+HEADER: :tangle ~/.config/emacs/init.el
#+BEGIN_SRC elisp
(set-fontset-font "fontset-startup" '(#x000000 . #x3FFFFF) "Unifont")
(set-fontset-font "fontset-default" '(#x000000 . #x3FFFFF) "Unifont")
(customize-set-variable
 'glyphless-char-display-control
 '((format-control . thin-space)
   (variation-selectors . thin-space)
   (no-font . empty-box)))
#+END_SRC

These are the sexps that I used in the two screenshots above:

#+BEGIN_SRC elisp
(with-selected-window (get-buffer-window "a.txt") (goto-line 1) (line-pixel-height))
(with-selected-window (get-buffer-window "a.txt") (goto-line 2) (line-pixel-height))
(with-selected-window (get-buffer-window "a.txt") (goto-line 3) (line-pixel-height))
#+END_SRC