23.0.60; Heavy display problems with new font backend

23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

[-- Attachment #1: Type: text/plain, Size: 7012 bytes --]


Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:

I use

  Emacs.Font: DejaVu Sans Mono-10

set in my .Xdefaults and get quite a lot display problems.  I update my
copy of emacs nearly daily (with make maintainerclean and all), and my
subjective feeling is that in the last 3 or 4 days I have much more
problems.

The most frequent problem is that individual chars look wrong, for
example:

  - quite often a "w", "k", "S" or "r" looks bold although it shouldn't
  - "m" has a green shine between its legs
  - fonts aren't sharp as if subpixel hinting was disabled

The attached pics fonts{1,2,3,4}.png show some of those minor display
problems.

While trying to find some more of those display problems I opened the
attached UTF-8 testfile and found a major display bug.  The pics
v-with-dot.png and v-with-dot2.png show it.

In v-with-dot.png the text is more or less displayed correctly.  The
dots over the v and r are displayed a bit higher using kwrite with the
same font, which looks a bit more natural.  Point is left of the text on
the same line.  Now I move point to the right and as soon as it passes
the v, a and b they're duplicated to the left, resulting in the
situation in v-with-dot2.png.

Additionally in that file most greek chars aren't antialiased.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
/usr/share/emacs/23.0.60/etc/DEBUG for instructions.


In GNU Emacs 23.0.60.2 (i686-pc-linux-gnu, GTK+ Version 2.12.9)
 of 2008-04-19 on baldur
Windowing system distributor `The X.Org Foundation', version 11.0.10400090
configured using `configure  '--prefix=/usr' '--host=i686-pc-linux-gnu' '--mandir=/usr/share/man' '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc' '--localstatedir=/var/lib' '--program-suffix=-emacs-23' '--infodir=/usr/share/info/emacs-23' '--without-carbon' '--with-sound' '--with-x' '--with-toolkit-scroll-bars' '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff' '--with-xpm' '--enable-font-backend' '--with-freetype' '--with-xft' '--with-libotf' '--with-m17n-flt' '--with-x-toolkit=gtk' '--without-hesiod' '--with-kerberos' '--with-kerberos5' '--with-gpm' '--with-dbus' '--build=i686-pc-linux-gnu' 'build_alias=i686-pc-linux-gnu' 'host_alias=i686-pc-linux-gnu' 'CFLAGS=-march=i386 -O0 -g' 'LDFLAGS=''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  gnus-undo-mode: t
  yas/minor-mode: t
  shell-dirtrack-mode: t
  recentf-mode: t
  icomplete-mode: t
  window-number-meta-mode: t
  window-number-mode: t
  savehist-mode: t
  exec-abbrev-cmd-mode: t
  show-paren-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t

Recent input:
SPC SPC SPC SPC SPC <return> <return> <return> <return> 
<return> <return> <return> <return> <return> <return> 
<return> <return> <return> <return> <return> SPC SPC 
c <return> c <return> SPC c <return> B <backspace> 
y q <return> B <backspace> y q l s C-x C-f U T <return> 
<down-mouse-5> <mouse-5> <double-down-mouse-5> <double-mouse-5> 
<triple-down-mouse-5> <triple-mouse-5> <triple-down-mouse-5> 
<triple-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<down-mouse-5> <mouse-5> <double-down-mouse-5> <double-mouse-5> 
<triple-down-mouse-5> <triple-mouse-5> <triple-down-mouse-5> 
<triple-mouse-5> <down-mouse-5> <mouse-5> <double-down-mouse-5> 
<double-mouse-5> <triple-down-mouse-5> <triple-mouse-5> 
<triple-down-mouse-5> <triple-mouse-5> <down-mouse-4> 
<mouse-4> <double-down-mouse-4> <double-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <triple-down-mouse-4> <triple-mouse-4> 
<triple-down-mouse-4> <triple-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <down-mouse-4> <mouse-4> <double-down-mouse-4> 
<double-mouse-4> <triple-down-mouse-4> <triple-mouse-4> 
<triple-down-mouse-4> <triple-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <triple-down-mouse-4> <triple-mouse-4> 
<down-mouse-4> <mouse-4> <double-down-mouse-4> <double-mouse-4> 
<triple-down-mouse-4> <triple-mouse-4> <triple-down-mouse-4> 
<triple-mouse-4> <triple-down-mouse-4> <triple-mouse-4> 
<down-mouse-1> <mouse-1> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> <down> <down> 
<down> <down> <down> <right> <up> <up> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
C-x k C-g C-x b <return> M-x r e b <return> <retur
n>

Recent messages:
(No changes need to be saved)
20080419T120438.479> Saving /home/heimdall/.newsrc.eld...
Wrote /home/heimdall/.newsrc.eld
20080419T120438.632> Saving /home/heimdall/.newsrc.eld...done
Contacting host: 87.117.229.205:80
EMMS: "Ordo Equitum Solis - Playing with the Fire" submitted to last.fm
Loading /usr/share/emacs/23.0.60/lisp/international/uni-name.el (source)...done
byte-code: Beginning of buffer [6 times]
Contacting host: post.audioscrobbler.com:80
EMMS: Now playing infos submitted to last.fm
Quit


[-- Attachment #2: fonts1.png --]
[-- Type: image/png, Size: 6310 bytes --]

[-- Attachment #3: fonts2.png --]
[-- Type: image/png, Size: 1317 bytes --]

[-- Attachment #4: fonts3.png --]
[-- Type: image/png, Size: 1343 bytes --]

[-- Attachment #5: fonts4.png --]
[-- Type: image/png, Size: 3001 bytes --]

[-- Attachment #6: UTF-8-demo.txt --]
[-- Type: text/plain, Size: 14264 bytes --]


UTF-8 encoded sample plain-text file
‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾

Markus Kuhn [ˈmaʳkʊs kuːn] <http://www.cl.cam.ac.uk/~mgk25/> — 2002-07-25


The ASCII compatible UTF-8 encoding used in this plain-text file
is defined in Unicode, ISO 10646-1, and RFC 2279.


Using Unicode/UTF-8, you can write in emails and source code things such as

Mathematics and sciences:

  ∮ E⋅da = Q,  n → ∞, ∑ f(i) = ∏ g(i),      ⎧⎡⎛┌─────┐⎞⎤⎫
                                            ⎪⎢⎜│a²+b³ ⎟⎥⎪
  ∀x∈ℝ: ⌈x⌉ = −⌊−x⌋, α ∧ ¬β = ¬(¬α ∨ β),    ⎪⎢⎜│───── ⎟⎥⎪
                                            ⎪⎢⎜⎷ c₈   ⎟⎥⎪
  ℕ ⊆ ℕ₀ ⊂ ℤ ⊂ ℚ ⊂ ℝ ⊂ ℂ,                   ⎨⎢⎜       ⎟⎥⎬
                                            ⎪⎢⎜ ∞     ⎟⎥⎪
  ⊥ < a ≠ b ≡ c ≤ d ≪ ⊤ ⇒ (⟦A⟧ ⇔ ⟪B⟫),      ⎪⎢⎜ ⎲     ⎟⎥⎪
                                            ⎪⎢⎜ ⎳aⁱ-bⁱ⎟⎥⎪
  2H₂ + O₂ ⇌ 2H₂O, R = 4.7 kΩ, ⌀ 200 mm     ⎩⎣⎝i=1    ⎠⎦⎭

Linguistics and dictionaries:

  ði ıntəˈnæʃənəl fəˈnɛtık əsoʊsiˈeıʃn
  Y [ˈʏpsilɔn], Yen [jɛn], Yoga [ˈjoːgɑ]

APL:

  ((V⍳V)=⍳⍴V)/V←,V    ⌷←⍳→⍴∆∇⊃‾⍎⍕⌈

Nicer typography in plain text files:

  ╔══════════════════════════════════════════╗
  ║                                          ║
  ║   • ‘single’ and “double” quotes         ║
  ║                                          ║
  ║   • Curly apostrophes: “We’ve been here” ║
  ║                                          ║
  ║   • Latin-1 apostrophe and accents: '´`  ║
  ║                                          ║
  ║   • ‚deutsche‘ „Anführungszeichen“       ║
  ║                                          ║
  ║   • †, ‡, ‰, •, 3–4, —, −5/+5, ™, …      ║
  ║                                          ║
  ║   • ASCII safety test: 1lI|, 0OD, 8B     ║
  ║                      ╭─────────╮         ║
  ║   • the euro symbol: │ 14.95 € │         ║
  ║                      ╰─────────╯         ║
  ╚══════════════════════════════════════════╝

Combining characters:

  STARGΛ̊TE SG-1, a = v̇ = r̈, a⃑ ⊥ b⃑

Greek (in Polytonic):

  The Greek anthem:

  Σὲ γνωρίζω ἀπὸ τὴν κόψη
  τοῦ σπαθιοῦ τὴν τρομερή,
  σὲ γνωρίζω ἀπὸ τὴν ὄψη
  ποὺ μὲ βία μετράει τὴ γῆ.

  ᾿Απ᾿ τὰ κόκκαλα βγαλμένη
  τῶν ῾Ελλήνων τὰ ἱερά
  καὶ σὰν πρῶτα ἀνδρειωμένη
  χαῖρε, ὦ χαῖρε, ᾿Ελευθεριά!

  From a speech of Demosthenes in the 4th century BC:

  Οὐχὶ ταὐτὰ παρίσταταί μοι γιγνώσκειν, ὦ ἄνδρες ᾿Αθηναῖοι,
  ὅταν τ᾿ εἰς τὰ πράγματα ἀποβλέψω καὶ ὅταν πρὸς τοὺς
  λόγους οὓς ἀκούω· τοὺς μὲν γὰρ λόγους περὶ τοῦ
  τιμωρήσασθαι Φίλιππον ὁρῶ γιγνομένους, τὰ δὲ πράγματ᾿
  εἰς τοῦτο προήκοντα,  ὥσθ᾿ ὅπως μὴ πεισόμεθ᾿ αὐτοὶ
  πρότερον κακῶς σκέψασθαι δέον. οὐδέν οὖν ἄλλο μοι δοκοῦσιν
  οἱ τὰ τοιαῦτα λέγοντες ἢ τὴν ὑπόθεσιν, περὶ ἧς βουλεύεσθαι,
  οὐχὶ τὴν οὖσαν παριστάντες ὑμῖν ἁμαρτάνειν. ἐγὼ δέ, ὅτι μέν
  ποτ᾿ ἐξῆν τῇ πόλει καὶ τὰ αὑτῆς ἔχειν ἀσφαλῶς καὶ Φίλιππον
  τιμωρήσασθαι, καὶ μάλ᾿ ἀκριβῶς οἶδα· ἐπ᾿ ἐμοῦ γάρ, οὐ πάλαι
  γέγονεν ταῦτ᾿ ἀμφότερα· νῦν μέντοι πέπεισμαι τοῦθ᾿ ἱκανὸν
  προλαβεῖν ἡμῖν εἶναι τὴν πρώτην, ὅπως τοὺς συμμάχους
  σώσομεν. ἐὰν γὰρ τοῦτο βεβαίως ὑπάρξῃ, τότε καὶ περὶ τοῦ
  τίνα τιμωρήσεταί τις καὶ ὃν τρόπον ἐξέσται σκοπεῖν· πρὶν δὲ
  τὴν ἀρχὴν ὀρθῶς ὑποθέσθαι, μάταιον ἡγοῦμαι περὶ τῆς
  τελευτῆς ὁντινοῦν ποιεῖσθαι λόγον.

  Δημοσθένους, Γ´ ᾿Ολυνθιακὸς

Georgian:

  From a Unicode conference invitation:

  გთხოვთ ახლავე გაიაროთ რეგისტრაცია Unicode-ის მეათე საერთაშორისო
  კონფერენციაზე დასასწრებად, რომელიც გაიმართება 10-12 მარტს,
  ქ. მაინცში, გერმანიაში. კონფერენცია შეჰკრებს ერთად მსოფლიოს
  ექსპერტებს ისეთ დარგებში როგორიცაა ინტერნეტი და Unicode-ი,
  ინტერნაციონალიზაცია და ლოკალიზაცია, Unicode-ის გამოყენება
  ოპერაციულ სისტემებსა, და გამოყენებით პროგრამებში, შრიფტებში,
  ტექსტების დამუშავებასა და მრავალენოვან კომპიუტერულ სისტემებში.

Russian:

  From a Unicode conference invitation:

  Зарегистрируйтесь сейчас на Десятую Международную Конференцию по
  Unicode, которая состоится 10-12 марта 1997 года в Майнце в Германии.
  Конференция соберет широкий круг экспертов по  вопросам глобального
  Интернета и Unicode, локализации и интернационализации, воплощению и
  применению Unicode в различных операционных системах и программных
  приложениях, шрифтах, верстке и многоязычных компьютерных системах.

Thai (UCS Level 2):

  Excerpt from a poetry on The Romance of The Three Kingdoms (a Chinese
  classic 'San Gua'):

  [----------------------------|------------------------]
    ๏ แผ่นดินฮั่นเสื่อมโทรมแสนสังเวช  พระปกเกศกองบู๊กู้ขึ้นใหม่
  สิบสองกษัตริย์ก่อนหน้าแลถัดไป       สององค์ไซร้โง่เขลาเบาปัญญา
    ทรงนับถือขันทีเป็นที่พึ่ง           บ้านเมืองจึงวิปริตเป็นนักหนา
  โฮจิ๋นเรียกทัพทั่วหัวเมืองมา         หมายจะฆ่ามดชั่วตัวสำคัญ
    เหมือนขับไสไล่เสือจากเคหา      รับหมาป่าเข้ามาเลยอาสัญ
  ฝ่ายอ้องอุ้นยุแยกให้แตกกัน          ใช้สาวนั้นเป็นชนวนชื่นชวนใจ
    พลันลิฉุยกุยกีกลับก่อเหตุ          ช่างอาเพศจริงหนาฟ้าร้องไห้
  ต้องรบราฆ่าฟันจนบรรลัย           ฤๅหาใครค้ำชูกู้บรรลังก์ ฯ

  (The above is a two-column text. If combining characters are handled
  correctly, the lines of the second column should be aligned with the
  | character above.)

Ethiopian:

  Proverbs in the Amharic language:

  ሰማይ አይታረስ ንጉሥ አይከሰስ።
  ብላ ካለኝ እንደአባቴ በቆመጠኝ።
  ጌጥ ያለቤቱ ቁምጥና ነው።
  ደሀ በሕልሙ ቅቤ ባይጠጣ ንጣት በገደለው።
  የአፍ ወለምታ በቅቤ አይታሽም።
  አይጥ በበላ ዳዋ ተመታ።
  ሲተረጉሙ ይደረግሙ።
  ቀስ በቀስ፥ ዕንቁላል በእግሩ ይሄዳል።
  ድር ቢያብር አንበሳ ያስር።
  ሰው እንደቤቱ እንጅ እንደ ጉረቤቱ አይተዳደርም።
  እግዜር የከፈተውን ጉሮሮ ሳይዘጋው አይድርም።
  የጎረቤት ሌባ፥ ቢያዩት ይስቅ ባያዩት ያጠልቅ።
  ሥራ ከመፍታት ልጄን ላፋታት።
  ዓባይ ማደሪያ የለው፥ ግንድ ይዞ ይዞራል።
  የእስላም አገሩ መካ የአሞራ አገሩ ዋርካ።
  ተንጋሎ ቢተፉ ተመልሶ ባፉ።
  ወዳጅህ ማር ቢሆን ጨርስህ አትላሰው።
  እግርህን በፍራሽህ ልክ ዘርጋ።

Runes:

  ᚻᛖ ᚳᚹᚫᚦ ᚦᚫᛏ ᚻᛖ ᛒᚢᛞᛖ ᚩᚾ ᚦᚫᛗ ᛚᚪᚾᛞᛖ ᚾᚩᚱᚦᚹᛖᚪᚱᛞᚢᛗ ᚹᛁᚦ ᚦᚪ ᚹᛖᛥᚫ

  (Old English, which transcribed into Latin reads 'He cwaeth that he
  bude thaem lande northweardum with tha Westsae.' and means 'He said
  that he lived in the northern land near the Western Sea.')

Braille:

  ⡌⠁⠧⠑ ⠼⠁⠒  ⡍⠜⠇⠑⠹⠰⠎ ⡣⠕⠌

  ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠙⠑⠁⠙⠒ ⠞⠕ ⠃⠑⠛⠔ ⠺⠊⠹⠲ ⡹⠻⠑ ⠊⠎ ⠝⠕ ⠙⠳⠃⠞
  ⠱⠁⠞⠑⠧⠻ ⠁⠃⠳⠞ ⠹⠁⠞⠲ ⡹⠑ ⠗⠑⠛⠊⠌⠻ ⠕⠋ ⠙⠊⠎ ⠃⠥⠗⠊⠁⠇ ⠺⠁⠎
  ⠎⠊⠛⠝⠫ ⠃⠹ ⠹⠑ ⠊⠇⠻⠛⠹⠍⠁⠝⠂ ⠹⠑ ⠊⠇⠻⠅⠂ ⠹⠑ ⠥⠝⠙⠻⠞⠁⠅⠻⠂
  ⠁⠝⠙ ⠹⠑ ⠡⠊⠑⠋ ⠍⠳⠗⠝⠻⠲ ⡎⠊⠗⠕⠕⠛⠑ ⠎⠊⠛⠝⠫ ⠊⠞⠲ ⡁⠝⠙
  ⡎⠊⠗⠕⠕⠛⠑⠰⠎ ⠝⠁⠍⠑ ⠺⠁⠎ ⠛⠕⠕⠙ ⠥⠏⠕⠝ ⠰⡡⠁⠝⠛⠑⠂ ⠋⠕⠗ ⠁⠝⠹⠹⠔⠛ ⠙⠑
  ⠡⠕⠎⠑ ⠞⠕ ⠏⠥⠞ ⠙⠊⠎ ⠙⠁⠝⠙ ⠞⠕⠲

  ⡕⠇⠙ ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲

  ⡍⠔⠙⠖ ⡊ ⠙⠕⠝⠰⠞ ⠍⠑⠁⠝ ⠞⠕ ⠎⠁⠹ ⠹⠁⠞ ⡊ ⠅⠝⠪⠂ ⠕⠋ ⠍⠹
  ⠪⠝ ⠅⠝⠪⠇⠫⠛⠑⠂ ⠱⠁⠞ ⠹⠻⠑ ⠊⠎ ⠏⠜⠞⠊⠊⠥⠇⠜⠇⠹ ⠙⠑⠁⠙ ⠁⠃⠳⠞
  ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲ ⡊ ⠍⠊⠣⠞ ⠙⠁⠧⠑ ⠃⠑⠲ ⠔⠊⠇⠔⠫⠂ ⠍⠹⠎⠑⠇⠋⠂ ⠞⠕
  ⠗⠑⠛⠜⠙ ⠁ ⠊⠕⠋⠋⠔⠤⠝⠁⠊⠇ ⠁⠎ ⠹⠑ ⠙⠑⠁⠙⠑⠌ ⠏⠊⠑⠊⠑ ⠕⠋ ⠊⠗⠕⠝⠍⠕⠝⠛⠻⠹
  ⠔ ⠹⠑ ⠞⠗⠁⠙⠑⠲ ⡃⠥⠞ ⠹⠑ ⠺⠊⠎⠙⠕⠍ ⠕⠋ ⠳⠗ ⠁⠝⠊⠑⠌⠕⠗⠎
  ⠊⠎ ⠔ ⠹⠑ ⠎⠊⠍⠊⠇⠑⠆ ⠁⠝⠙ ⠍⠹ ⠥⠝⠙⠁⠇⠇⠪⠫ ⠙⠁⠝⠙⠎
  ⠩⠁⠇⠇ ⠝⠕⠞ ⠙⠊⠌⠥⠗⠃ ⠊⠞⠂ ⠕⠗ ⠹⠑ ⡊⠳⠝⠞⠗⠹⠰⠎ ⠙⠕⠝⠑ ⠋⠕⠗⠲ ⡹⠳
  ⠺⠊⠇⠇ ⠹⠻⠑⠋⠕⠗⠑ ⠏⠻⠍⠊⠞ ⠍⠑ ⠞⠕ ⠗⠑⠏⠑⠁⠞⠂ ⠑⠍⠏⠙⠁⠞⠊⠊⠁⠇⠇⠹⠂ ⠹⠁⠞
  ⡍⠜⠇⠑⠹ ⠺⠁⠎ ⠁⠎ ⠙⠑⠁⠙ ⠁⠎ ⠁ ⠙⠕⠕⠗⠤⠝⠁⠊⠇⠲

  (The first couple of paragraphs of "A Christmas Carol" by Dickens)

Compact font selection example text:

  ABCDEFGHIJKLMNOPQRSTUVWXYZ /0123456789
  abcdefghijklmnopqrstuvwxyz £©µÀÆÖÞßéöÿ
  –—‘“”„†•…‰™œŠŸž€ ΑΒΓΔΩαβγδω АБВГДабвгд
  ∀∂∈ℝ∧∪≡∞ ↑↗↨↻⇣ ┐┼╔╘░►☺♀ fi�⑀₂ἠḂӥẄɐː⍎אԱა

Greetings in various languages:

  Hello world, Καλημέρα κόσμε, コンニチハ

Box drawing alignment tests:                                          █
                                                                      ▉
  ╔══╦══╗  ┌──┬──┐  ╭──┬──╮  ╭──┬──╮  ┏━━┳━━┓  ┎┒┏┑   ╷  ╻ ┏┯┓ ┌┰┐    ▊ ╱╲╱╲╳╳╳
  ║┌─╨─┐║  │╔═╧═╗│  │╒═╪═╕│  │╓─╁─╖│  ┃┌─╂─┐┃  ┗╃╄┙  ╶┼╴╺╋╸┠┼┨ ┝╋┥    ▋ ╲╱╲╱╳╳╳
  ║│╲ ╱│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╿ │┃  ┍╅╆┓   ╵  ╹ ┗┷┛ └┸┘    ▌ ╱╲╱╲╳╳╳
  ╠╡ ╳ ╞╣  ├╢   ╟┤  ├┼─┼─┼┤  ├╫─╂─╫┤  ┣┿╾┼╼┿┫  ┕┛┖┚     ┌┄┄┐ ╎ ┏┅┅┓ ┋ ▍ ╲╱╲╱╳╳╳
  ║│╱ ╲│║  │║   ║│  ││ │ ││  │║ ┃ ║│  ┃│ ╽ │┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▎
  ║└─╥─┘║  │╚═╤═╝│  │╘═╪═╛│  │╙─╀─╜│  ┃└─╂─┘┃  ░░▒▒▓▓██ ┊  ┆ ╎ ╏  ┇ ┋ ▏
  ╚══╩══╝  └──┴──┘  ╰──┴──╯  ╰──┴──╯  ┗━━┻━━┛  ▗▄▖▛▀▜   └╌╌┘ ╎ ┗╍╍┛ ┋  ▁▂▃▄▅▆▇█
                                               ▝▀▘▙▄▟

[-- Attachment #7: v-with-dot.png --]
[-- Type: image/png, Size: 2903 bytes --]

[-- Attachment #8: v-with-dot2.png --]
[-- Type: image/png, Size: 2638 bytes --]

Re: 23.0.60; Heavy display problems with new font backend

[Stefan Monnier]

>   - quite often a "w", "k", "S" or "r" looks bold although it shouldn't

This is a case of overwrite: each time you put the cursor next to
a char, it seems that the char gets redrawn over itself, so with
anti-aliasing it gets darker&darker (i.e. bolder and bolder).

>   - "m" has a green shine between its legs

No idea, here.

>   - fonts aren't sharp as if subpixel hinting was disabled

IIUC subpixel anti-aliasing is indeed not enabled.


        Stefan

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Sat, 19 Apr 2008 09:19:33 -0400 Stefan Monnier wrote:

> IIUC subpixel anti-aliasing is indeed not enabled.

It is, at least in the first screenshot (you can see the color artifacts
when you magnify the image).  But I suspect it's set to BGR and not RGB
(as most LCDs require).

David

Re: 23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

David Hansen <david.hansen@gmx.net> writes:

Hi David,

>> IIUC subpixel anti-aliasing is indeed not enabled.
>
> It is, at least in the first screenshot (you can see the color
> artifacts when you magnify the image).  But I suspect it's set to BGR
> and not RGB (as most LCDs require).

No, it is and was set to rgb:

,----
| Xft.hinting:            true
| Xft.hintstyle:          hintfull
| Xft.antialias:          rgba
| Xft.rgba:               rgb
`----

Bye,
Tassilo

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Sun, 20 Apr 2008 11:41:47 +0200 Tassilo Horn wrote:

> David Hansen <david.hansen@gmx.net> writes:
>
> Hi David,
>
>>> IIUC subpixel anti-aliasing is indeed not enabled.
>>
>> It is, at least in the first screenshot (you can see the color
>> artifacts when you magnify the image).  But I suspect it's set to BGR
>> and not RGB (as most LCDs require).
>
> No, it is and was set to rgb:

Ah, ok, my fault when adding colors in my head :)

Re: 23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

David Hansen <david.hansen@gmx.net> writes:

Hi David,

>>>> IIUC subpixel anti-aliasing is indeed not enabled.
>>>
>>> It is, at least in the first screenshot (you can see the color
>>> artifacts when you magnify the image).  But I suspect it's set to
>>> BGR and not RGB (as most LCDs require).
>>
>> No, it is and was set to rgb:
>
> Ah, ok, my fault when adding colors in my head :)

No problem.  What's really strange is that today everything looks much
better.  The normal font looks sharp, there're very few bold chars and I
cannot reproduce the bug with the combined chars in UTF-8-demo.txt.

Looking at the ChangeLogs I cannot find a change that might have fixed
the problems.  Well, yesterday was a bad day anyway -- weather, mood,
work...  Probably I screwed emacs up mentally. ;-)

Bye,
Tassilo

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Sun, 20 Apr 2008 15:52:35 +0200 Tassilo Horn wrote:

> David Hansen <david.hansen@gmx.net> writes:
>
> Hi David,
>
>>>>> IIUC subpixel anti-aliasing is indeed not enabled.
>>>>
>>>> It is, at least in the first screenshot (you can see the color
>>>> artifacts when you magnify the image).  But I suspect it's set to
>>>> BGR and not RGB (as most LCDs require).
>>>
>>> No, it is and was set to rgb:
>>
>> Ah, ok, my fault when adding colors in my head :)
>
> No problem.  What's really strange is that today everything looks much
> better.  The normal font looks sharp, there're very few bold chars and I
> cannot reproduce the bug with the combined chars in UTF-8-demo.txt.
>
> Looking at the ChangeLogs I cannot find a change that might have fixed
> the problems.  Well, yesterday was a bad day anyway -- weather, mood,
> work...  Probably I screwed emacs up mentally. ;-)

IMHO it's great improvement when you disable the autohinter (requires
that you compiled freetype with -DFUCK_PATENTS).  Anyway I played around
a bit with this "new technology" and think it looks incredible ugly
compared to the good old bitmap fonts.

David

Re: 23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>   - quite often a "w", "k", "S" or "r" looks bold although it
>>     shouldn't
>
> This is a case of overwrite: each time you put the cursor next to a
> char, it seems that the char gets redrawn over itself, so with
> anti-aliasing it gets darker&darker (i.e. bolder and bolder).

Hm, but how can the direction in which point moves over the char change
that?  Most of the time (if not always), moving from right to left made
the char regular again.  Moving backwords made it appear boldish.

>>   - fonts aren't sharp as if subpixel hinting was disabled
>
> IIUC subpixel anti-aliasing is indeed not enabled.

I've set it in XFCE for all applications and added duplicate settings to
my .Xdefaults to be sure.

Bye,
Tassilo

Re: 23.0.60; Heavy display problems with new font backend

[Johan Bockgård]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>   - quite often a "w", "k", "S" or "r" looks bold although it shouldn't
>
> This is a case of overwrite: each time you put the cursor next to
> a char, it seems that the char gets redrawn over itself, so with
> anti-aliasing it gets darker&darker (i.e. bolder and bolder).

Also see this thread

http://news.gmane.org/find-root.php?message_id=<yoij4pc8bejp.fsf@remote2.student.chalmers.se>

-- 
Johan Bockgård

Re: 23.0.60; Heavy display problems with new font backend

[Jason Rumney]

Tassilo Horn wrote:
>   - quite often a "w", "k", "S" or "r" looks bold although it shouldn't
>   - "m" has a green shine between its legs
>   - fonts aren't sharp as if subpixel hinting was disabled
>
> The attached pics fonts{1,2,3,4}.png show some of those minor display
> problems.
>   

I only see the first problem. Which suggests that the subpixel 
antialiasing is correct for my display but incorrect for yours, as the 
letters all look sharp for me, without any more color artifacts than 
you'd expect from subpixel antialiasing.

> While trying to find some more of those display problems I opened the
> attached UTF-8 testfile and found a major display bug.  The pics
> v-with-dot.png and v-with-dot2.png show it.
>   

These seem to be problems in the composition rules for those modifiers. 
Things probably don't look much better with the old font code either.

> Additionally in that file most greek chars aren't antialiased.
>   

Only some fonts support antialiasing.  If you try different greek fonts, 
you might find one where it works.

Re: 23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

Jason Rumney <jasonr@gnu.org> writes:

> Tassilo Horn wrote:
>>   - quite often a "w", "k", "S" or "r" looks bold although it shouldn't
>>   - "m" has a green shine between its legs
>>   - fonts aren't sharp as if subpixel hinting was disabled
>>
>> The attached pics fonts{1,2,3,4}.png show some of those minor display
>> problems.
>>   
>
> I only see the first problem. Which suggests that the subpixel
> antialiasing is correct for my display but incorrect for yours, as the
> letters all look sharp for me, without any more color artifacts than
> you'd expect from subpixel antialiasing.

Hm, I use XFCE here which sets subpixel antialiasing for all
applications (I tried setting it with Xresources, too).  If I compare
emacs to some other editor using the same font, the difference is
obvious.

>> While trying to find some more of those display problems I opened the
>> attached UTF-8 testfile and found a major display bug.  The pics
>> v-with-dot.png and v-with-dot2.png show it.
>>   
> These seem to be problems in the composition rules for those
> modifiers. Things probably don't look much better with the old font
> code either.

Hm, with --disable-font-backend using adobe courier font, I cannot see
the dots over v and r, but at least the characters aren't displayed
twice when point moves over them.

>> Additionally in that file most greek chars aren't antialiased.
>>
> Only some fonts support antialiasing.  If you try different greek
> fonts, you might find one where it works.

Hm, at least kwrite does antialiasing for these greek chars using the
same font I've set for emacs.  The results don't look too sharp, though.

Bye,
Tassilo

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Sat, 19 Apr 2008 14:50:36 +0100 Jason Rumney wrote:

> These seem to be problems in the composition rules for those
> modifiers. Things probably don't look much better with the old font
> code either.

Looks far better with disabled font backend:

http://www.alice-dsl.net/david.hansen/comb.png

David

Re: 23.0.60; Heavy display problems with new font backend

[sand]

David Hansen writes:
> Looks far better with disabled font backend:
> 
> http://www.alice-dsl.net/david.hansen/comb.png

A fellow Neep Alt-er!  Have you also tried the Neep Alt ISO10646-1
registry fonts from Debian's xfonts-jmk package?  When I use the
ISO10646 fonts with the new font backend, Emacs seems to think that
the font is missing large quantities of codepoints.  Left and right
quotes, for example, show up using double-width Chinese fonts.  I'm
curious whether other people see the same thing.

Since the whole Neep Alt ISO10646 font collection is a hack (it's
Neep Alt ISO8859-1 superimposed on Misc-Fixed ISO10646), there's
always a chance that there's something wrong in the font itself, but
everything shows up just fine in xterm.  When I have the time I'm
planning on digging into Xft as the next suspect.

Derek

-- 
Derek Upham
sand@blarg.net

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Mon, 21 Apr 2008 20:59:04 -0700 sand@blarg.net wrote:

> David Hansen writes:
>> Looks far better with disabled font backend:
>> 
>> http://www.alice-dsl.net/david.hansen/comb.png
>
> A fellow Neep Alt-er!  Have you also tried the Neep Alt ISO10646-1
> registry fonts from Debian's xfonts-jmk package?  When I use the
> ISO10646 fonts with the new font backend, Emacs seems to think that
> the font is missing large quantities of codepoints.  Left and right
> quotes, for example, show up using double-width Chinese fonts.  I'm
> curious whether other people see the same thing.

I noticed that problem not only with the neep font but also with Monaco
and the fairly complete DejaVu Mono font.  I even reported it as a bug,
but it may be possible that this is some limited understanding of
dealing with fontsets on my side.

It may be worth a look how rxvt selects fonts.  When looking at the
unicode test file it looks far better in urxvt, no matter which font I
specify as the default.

> Since the whole Neep Alt ISO10646 font collection is a hack (it's
> Neep Alt ISO8859-1 superimposed on Misc-Fixed ISO10646), there's
> always a chance that there's something wrong in the font itself, but
> everything shows up just fine in xterm.  When I have the time I'm
> planning on digging into Xft as the next suspect.

Well, if you want to use the good old bitmap fonts anyway it's probably
not worth.  These antialiased fonts look nice if they are huge black
letters on a bright background.  Small characters just look like s...,
especially with white fg on black bg.  I think I'll just use the old
font handling as long as possible.

David

Re: 23.0.60; Heavy display problems with new font backend

[Jason Rumney]

David Hansen wrote:

> Well, if you want to use the good old bitmap fonts anyway it's probably
> not worth.  These antialiased fonts look nice if they are huge black
> letters on a bright background.  Small characters just look like s...,
> especially with white fg on black bg.  I think I'll just use the old
> font handling as long as possible.
>   

There is a new font backend for the "old" X fonts, so you should be able 
to just keep using the old bitmapped fonts without having to resort to 
--disable-font-backend.
If Xft has fonts of the same name that it imposes antialiasing on, then 
you can put the following in .Xresources to only use old X fonts:

Emacs.fontBackend: x

Re: 23.0.60; Heavy display problems with new font backend

[David Hansen]

On Tue, 22 Apr 2008 11:21:01 +0100 Jason Rumney wrote:

> David Hansen wrote:
>
>> Well, if you want to use the good old bitmap fonts anyway it's probably
>> not worth.  These antialiased fonts look nice if they are huge black
>> letters on a bright background.  Small characters just look like s...,
>> especially with white fg on black bg.  I think I'll just use the old
>> font handling as long as possible.
>>   
>
> There is a new font backend for the "old" X fonts, so you should be
> able to just keep using the old bitmapped fonts without having to
> resort to --disable-font-backend.

This doesn't solve the display problems with non latin-1 characters.

> If Xft has fonts of the same name that it imposes antialiasing on,
> then you can put the following in .Xresources to only use old X fonts:
>
> Emacs.fontBackend: x

Why compile in code I don't won't to use anyway?

David

Re: 23.0.60; Heavy display problems with new font backend

[Jason Rumney]

David Hansen wrote:
> Why compile in code I don't won't to use anyway?
>   

Good point, ./configure --without-freetype will disable all the freetype 
related backends.

Re: 23.0.60; Heavy display problems with new font backend

[Tassilo Horn]

Jason Rumney <jasonr@gnu.org> writes:

Hi Jason,

> There is a new font backend for the "old" X fonts, so you should be
> able to just keep using the old bitmapped fonts without having to
> resort to --disable-font-backend.

If I select an X bitmap font using

  Emacs.Font: -misc-fixed-medium-r-*-*-15-*-*-*-*-*-iso10646-1

in my .Xdefaults, I have some font size problems.  For example in gnus
summary buffers, text in gnus-summary-normal-unread face is normal, but
text in gnus-summary-low-read appears smaller and garbles my tabular
layout.  describe-face shows that its Font parameter is nil, Slant is
italic and Foreground is DarkGreen, the rest is unspecefied.

misc-fixed doesn't provide italic chars (according to xfontsel), so I
guess that's the cause of Font being nil.

But emacs with

  Emacs.FontBackend: x
  Emacs.Font: -misc-fixed-medium-r-*-*-15-*-*-*-*-*-iso10646-1

handles that situation much better.  The output of describe-face is the
same (e.g. Font is nil), but instead of using some smaller characters it
simply applies only the applicable parameters to the default face,
i.e. it's DarkGreen but not italic.

Bye,
Tassilo

Re: 23.0.60; Heavy display problems with new font backend

[Juanma Barranquero]

On Tue, Apr 22, 2008 at 11:48 AM, David Hansen <david.hansen@gmx.net> wrote:

>  I noticed that problem not only with the neep font but also with Monaco
>  and the fairly complete DejaVu Mono font.

Using DejaVu Sans Mono, I'm having a weird effect with the euro sign €

 - Emacs 22: chooses DejaVu Sans Mono
 - Emacs 23: chooses DejaVu Sans Mono
 - Emacs 23 with --disable-font-backend: chooses Microsoft Sans Serif

 Juanma

Re: 23.0.60; Heavy display problems with new font backend

[sand]

David Hansen writes:
> > A fellow Neep Alt-er!  Have you also tried the Neep Alt ISO10646-1
> > registry fonts from Debian's xfonts-jmk package?  When I use the
> > ISO10646 fonts with the new font backend, Emacs seems to think that
> > the font is missing large quantities of codepoints.  Left and right
> > quotes, for example, show up using double-width Chinese fonts.  I'm
> > curious whether other people see the same thing.
> 
> I noticed that problem not only with the neep font but also with Monaco
> and the fairly complete DejaVu Mono font.  I even reported it as a bug,
> but it may be possible that this is some limited understanding of
> dealing with fontsets on my side.
> 
> It may be worth a look how rxvt selects fonts.  When looking at the
> unicode test file it looks far better in urxvt, no matter which font I
> specify as the default.

I traced the fonts being sent to ftfont_has_char() and discovered that
for a letter like LATIN SMALL LETTER S ACUTE (#x15b), Emacs was asking
about the Neep Alt family as expected, but only about fonts with the
iso8859-1 registry/encoding within that family.  When I removed the
iso8859-1 fonts entirely from the font path and HUPed the font server,
Emacs started asking about the iso10646-1 registry/encoding fonts, and
those *were* found and displayed correctly.  (On the other hand, the
mode line's file name---and just the mode line's file name---dropped
down to a different font.)  So:

 1. It looks like Emacs has a problem determining the right registry
 to use for certain code points, or the font picking fallback code has
 problems.

 2. The file name display code may not be correctly integrated with
 the new font backend.

At this point I think I need to get more familiar with the contents of
etc/DEBUG...

Derek

-- 
Derek Upham
sand@blarg.net

ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

sand@blarg.net writes:
> David Hansen writes:
> I traced the fonts being sent to ftfont_has_char() and discovered that
> for a letter like LATIN SMALL LETTER S ACUTE (#x15b), Emacs was asking
> about the Neep Alt family as expected, but only about fonts with the
> iso8859-1 registry/encoding within that family.  When I removed the
> iso8859-1 fonts entirely from the font path and HUPed the font server,
> Emacs started asking about the iso10646-1 registry/encoding fonts, and
> those *were* found and displayed correctly.  (On the other hand, the
> mode line's file name---and just the mode line's file name---dropped
> down to a different font.)  So:
> 
>  1. It looks like Emacs has a problem determining the right registry
>  to use for certain code points, or the font picking fallback code has
>  problems.
> 
>  2. The file name display code may not be correctly integrated with
>  the new font backend.
> 
> At this point I think I need to get more familiar with the contents of
> etc/DEBUG...

I think I have found the cause for problem #1.  In ftfont_list(), the
code gathers a list of candidate fonts that match the
foundry/family/... requirements:

    objset = FcObjectSetBuild (FC_FOUNDRY, FC_FAMILY, FC_WEIGHT, FC_SLANT,
                   FC_WIDTH, FC_PIXEL_SIZE, FC_SPACING,
                   FC_CHARSET, FC_FILE,
  #ifdef FC_FONTFORMAT
                   FC_FONTFORMAT,
  #endif    /* FC_FONTFORMAT */
                   NULL);
  /* ... elided ... */
  fontset = FcFontList (NULL, pattern, objset);

Note that this doesn't include any registry restriction.

The code loops across the returned fontsets, calling
ftfont_pattern_entity() to generate font_entity structs.  But at no
point does it attempt to filter the font list by compatible
registries.  We get, for example:

(gdb) frame
#0  ftfont_pattern_entity (p=0x89fce40, frame=148009620, registry=138791553) at /home/upham/src/emacs/Apollo/emacs-cvs/src/ftfont.c:116
(gdb) p file
$151 = (FcChar8 *) 0x8a70688 
"/home/upham/.fonts/jmk/neep-alt-iso8859-1-06x11.pcf.gz"
(gdb) p registry
$152 = 138791553
(gdb) xpr registry
Lisp_Symbol
$153 = (struct Lisp_Symbol *) 0x845ca80
"iso10646-1"

Emacs will think that "neep-alt-iso8859-1-06x11.pcf.gz" is a valid
font for displaying "iso10646-1", but it isn't, and we end up with
missing code points.

This explains why removing the iso8859-1 fonts fixed the problem
(except for the mode line file name): the current code also points
iso8859-1 requesters to iso10646-1 fonts, and those always work.  I
also think this explains why I don't see this consistently across
hosts: depending on how the font list is ordered (maybe due to inode
ordering on disk?), some hosts will get a correct iso10646-1 ->
iso10646-1 mapping first at display time, while others will get an
incorrect iso10646-1 -> iso8859-1 mapping.

Another family that should have the same problem is misc-fixed, as it
also has both iso8859-1 and iso10646-1 registry fonts.  There may be
other families that I'm not aware of.

Derek

-- 
Derek Upham
sand@blarg.net

[PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

sand@blarg.net writes:
> I think I have found the cause for problem #1.  In ftfont_list(), the
> code gathers a list of candidate fonts that match the
> foundry/family/... requirements:
> 
>     objset = FcObjectSetBuild (FC_FOUNDRY, FC_FAMILY, FC_WEIGHT, FC_SLANT,
>                    FC_WIDTH, FC_PIXEL_SIZE, FC_SPACING,
>                    FC_CHARSET, FC_FILE,
>   #ifdef FC_FONTFORMAT
>                    FC_FONTFORMAT,
>   #endif    /* FC_FONTFORMAT */
>                    NULL);
>   /* ... elided ... */
>   fontset = FcFontList (NULL, pattern, objset);
> 
> Note that this doesn't include any registry restriction.
> 
> The code loops across the returned fontsets, calling
> ftfont_pattern_entity() to generate font_entity structs.  But at no
> point does it attempt to filter the font list by compatible
> registries.  We get, for example:
> 
> (gdb) frame
> #0  ftfont_pattern_entity (p=0x89fce40, frame=148009620, registry=138791553) at /home/upham/src/emacs/Apollo/emacs-cvs/src/ftfont.c:116
> (gdb) p file
> $151 = (FcChar8 *) 0x8a70688 
> "/home/upham/.fonts/jmk/neep-alt-iso8859-1-06x11.pcf.gz"
> (gdb) p registry
> $152 = 138791553
> (gdb) xpr registry
> Lisp_Symbol
> $153 = (struct Lisp_Symbol *) 0x845ca80
> "iso10646-1"
> 
> Emacs will think that "neep-alt-iso8859-1-06x11.pcf.gz" is a valid
> font for displaying "iso10646-1", but it isn't, and we end up with
> missing code points.
> 
> This explains why removing the iso8859-1 fonts fixed the problem
> (except for the mode line file name): the current code also points
> iso8859-1 requesters to iso10646-1 fonts, and those always work.  I
> also think this explains why I don't see this consistently across
> hosts: depending on how the font list is ordered (maybe due to inode
> ordering on disk?), some hosts will get a correct iso10646-1 ->
> iso10646-1 mapping first at display time, while others will get an
> incorrect iso10646-1 -> iso8859-1 mapping.
> 
> Another family that should have the same problem is misc-fixed, as it
> also has both iso8859-1 and iso10646-1 registry fonts.  There may be
> other families that I'm not aware of.

Here's a patch against today's CVS HEAD.

The ftfont_spec_pattern() function generates an FcPattern object that
can be used to list only fonts matching the spec.  For the purposes of
this discussion, there are two "interesting" ways of restricting
patterns: via charset (FcCharSet), or via langset (FcLangSet).  The
former requires the font to have each of the codepoints listed in the
FcCharSet.  The latter requires the font to support all the languages
in the FcLangSet.

1. If we pass a font spec with registry ISO-8859 to
ftfont_spec_pattern(), then the code sets up an FcCharSet that has
every ASCII codepoint (but not Latin-1, that's commented out for some
reason).

2. If we pass a font spec with a non-ISO-8859, non-ISO-10646,
non-Unicode-BMP registry, the function immediately returns an empty
pattern.

3. ISO-10646 and Unicode-BMP registries are handled in a more
complicated manner...

If the ISO-10646 font spec has an associated :script parameter (or an
OpenType spec that refers to a script), the code looks in
'script-representative-chars' for codepoints to put into a charset.
If the font spec has an associated language, the code adds the
language to the langset.

However, an ISO-10646 font spec without a special script or language
ends up with neither a charset nor a langset.  The resulting pattern
will match *any* characters and languages.  In partcular, it will let
an ISO-8859 font match the ISO-10646 spec.

The fix below checks for a missing charset and missing langset.  In
that case, we create a charset with at least one ISO-10646 codepoint
outside of ISO-8859.  The charset should be as small as possible,
since a font missing any of the charset's codepoints becomes
completely invalid.  I have chosen LEFT DOUBLE QUOTATION MARK, which
is associated with English and which I believe is pervasive.

With the new charset restriction, ISO-8859 fonts are no longer
considered matches and the font mismatch problem goes away.

(We could add codepoints 32 through 127 and 192 through 255 to the
ISO-10646 charset, but it's unlikely that any font advertising itself
as ISO-10646 will be missing those codepoints.  If we do need those
extra codepoints, we can copy the implementation from
ftfont_build_basic_charsets().)

Derek

--
Derek Upham
sand@blarg.net

------------------------------ cut here ------------------------------

Index: ftfont.c
===================================================================
RCS file: /sources/emacs/emacs/src/ftfont.c,v
retrieving revision 1.9
diff -u -u -r1.9 ftfont.c
--- ftfont.c    3 Apr 2008 08:16:54 -0000    1.9
+++ ftfont.c    6 May 2008 21:08:44 -0000
@@ -38,6 +38,9 @@
 #include "font.h"
 #include "ftfont.h"
 
+/* Codepoint in ISO-10646 that most English fonts will have. */
+#define CODEPOINT_ISO10646_ENGLISH 0x201C /* LEFT DOUBLE QUOTATION MARK */
+
 /* Symbolic type of this font-driver.  */
 Lisp_Object Qfreetype;
 
@@ -521,6 +524,20 @@
 	}
     }
 
+  /* Lack of charset and langset at this point indicates an requested
+     ISO-10646 registry with no special script or language
+     requirement.  We need a charset with some codepoint outside of
+     the ISO-8859-* range that most "English" fonts will have.
+     Otherwise the resulting pattern will also match ISO-8859 fonts.  */
+  if (! charset && ! langset)
+    {
+      charset = FcCharSetCreate ();
+      if (! charset)
+        goto err;
+      if (! FcCharSetAddChar (charset, CODEPOINT_ISO10646_ENGLISH))
+        goto err;
+    }
+
   pattern = FcPatternCreate ();
   if (! pattern)
     goto err;

[PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

Can I please get an ack on this patch, or at least on the code problem
that it's fixing?  I'd like to get some sort of fix into CVS.

Thanks,

Derek

sand@blarg.net writes:
> Here's a patch against today's CVS HEAD.
> 
> The ftfont_spec_pattern() function generates an FcPattern object that
> can be used to list only fonts matching the spec.  For the purposes of
> this discussion, there are two "interesting" ways of restricting
> patterns: via charset (FcCharSet), or via langset (FcLangSet).  The
> former requires the font to have each of the codepoints listed in the
> FcCharSet.  The latter requires the font to support all the languages
> in the FcLangSet.
> 
> 1. If we pass a font spec with registry ISO-8859 to
> ftfont_spec_pattern(), then the code sets up an FcCharSet that has
> every ASCII codepoint (but not Latin-1, that's commented out for some
> reason).
> 
> 2. If we pass a font spec with a non-ISO-8859, non-ISO-10646,
> non-Unicode-BMP registry, the function immediately returns an empty
> pattern.
> 
> 3. ISO-10646 and Unicode-BMP registries are handled in a more
> complicated manner...
> 
> If the ISO-10646 font spec has an associated :script parameter (or an
> OpenType spec that refers to a script), the code looks in
> 'script-representative-chars' for codepoints to put into a charset.
> If the font spec has an associated language, the code adds the
> language to the langset.
> 
> However, an ISO-10646 font spec without a special script or language
> ends up with neither a charset nor a langset.  The resulting pattern
> will match *any* characters and languages.  In partcular, it will let
> an ISO-8859 font match the ISO-10646 spec.
> 
> The fix below checks for a missing charset and missing langset.  In
> that case, we create a charset with at least one ISO-10646 codepoint
> outside of ISO-8859.  The charset should be as small as possible,
> since a font missing any of the charset's codepoints becomes
> completely invalid.  I have chosen LEFT DOUBLE QUOTATION MARK, which
> is associated with English and which I believe is pervasive.
> 
> With the new charset restriction, ISO-8859 fonts are no longer
> considered matches and the font mismatch problem goes away.
> 
> (We could add codepoints 32 through 127 and 192 through 255 to the
> ISO-10646 charset, but it's unlikely that any font advertising itself
> as ISO-10646 will be missing those codepoints.  If we do need those
> extra codepoints, we can copy the implementation from
> ftfont_build_basic_charsets().)
> 
> Derek
> 
> --
> Derek Upham
> sand@blarg.net
> 
> ------------------------------ cut here ------------------------------
> 
> Index: ftfont.c
> ===================================================================
> RCS file: /sources/emacs/emacs/src/ftfont.c,v
> retrieving revision 1.9
> diff -u -u -r1.9 ftfont.c
> --- ftfont.c    3 Apr 2008 08:16:54 -0000    1.9
> +++ ftfont.c    6 May 2008 21:08:44 -0000
> @@ -38,6 +38,9 @@
>  #include "font.h"
>  #include "ftfont.h"
>  
> +/* Codepoint in ISO-10646 that most English fonts will have. */
> +#define CODEPOINT_ISO10646_ENGLISH 0x201C /* LEFT DOUBLE QUOTATION MARK */
> +
>  /* Symbolic type of this font-driver.  */
>  Lisp_Object Qfreetype;
>  
> @@ -521,6 +524,20 @@
>  	}
>      }
>  
> +  /* Lack of charset and langset at this point indicates an requested
> +     ISO-10646 registry with no special script or language
> +     requirement.  We need a charset with some codepoint outside of
> +     the ISO-8859-* range that most "English" fonts will have.
> +     Otherwise the resulting pattern will also match ISO-8859 fonts.  */
> +  if (! charset && ! langset)
> +    {
> +      charset = FcCharSetCreate ();
> +      if (! charset)
> +        goto err;
> +      if (! FcCharSetAddChar (charset, CODEPOINT_ISO10646_ENGLISH))
> +        goto err;
> +    }
> +
>    pattern = FcPatternCreate ();
>    if (! pattern)
>      goto err;

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[Jason Rumney]

sand@blarg.net wrote:
> Can I please get an ack on this patch, or at least on the code problem
> that it's fixing?  I'd like to get some sort of fix into CVS.
>   
Kenichi Handa is probably the only person qualified to comment, and he 
said early this week that he won't have time to spend on Emacs this 
week, so please remain patient.

In the meantime, you might like to check out the font-backend branch, as 
the bug may already be fixed there, or your patch may need adapting to it.

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

Jason Rumney writes:
> Kenichi Handa is probably the only person qualified to comment, and he 
> said early this week that he won't have time to spend on Emacs this 
> week, so please remain patient.
> 
> In the meantime, you might like to check out the font-backend branch, as 
> the bug may already be fixed there, or your patch may need adapting to it.

Thanks.  I'll construct a patch for "font-backend" as well and send it.

Derek

-- 
Derek Upham
sand@blarg.net

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

sand@blarg.net writes:
> Jason Rumney writes:
> > Kenichi Handa is probably the only person qualified to comment, and he 
> > said early this week that he won't have time to spend on Emacs this 
> > week, so please remain patient.
> > 
> > In the meantime, you might like to check out the font-backend branch, as 
> > the bug may already be fixed there, or your patch may need adapting to it.
> 
> Thanks.  I'll construct a patch for "font-backend" as well and send it.

The font-backend branch has the same problem, and essentially the same
patch works for it.  It's not worth constructing a separate patch for
that branch.  Plus, it looks like the patch itself, while useful, is
fixing secondary problems rather than root problems.  I decided to try
attacking the issues at the Lisp level.

Most of the problems seem to be caused by bad configuration in
"fontset.el".  I have appended the Lisp code I came up with to get my
environment (almost) working.  The exception is Korean syllabics,
which show up fine with the old backend (using Daewoo Mincho
KSC5601.1987) but don't show up at all with the new one.  The fontset
declaration looks okay, so I'm going to keep digging.

-- 
Derek Upham
sand@blarg.net

------------------------------ cut here ------------------------------

(eval-after-load "fontset"
  '(progn
     ;; Representative characters for Latin need to include the
     ;; various extension blocks.  This should replace the existing
     ;; definition.
     (setq script-representative-chars
           (cons '(latin ?A ?Z ?a ?z #x00C0 #x0100 #x0180 #x1e00)
                 script-representative-chars))

     ;; Representative characters for symbols were not defined.
     (setq script-representative-chars
           (cons '(symbol #x201C #x2200 #x2500)
                 script-representative-chars))

     ;; Representative characters for phonetics were not defined.
     (setq script-representative-chars
           (cons '(phonetic #x0250 #x0283)
                 script-representative-chars))))

;; No fontset was defined for Armenian.
(set-fontset-font "fontset-default" 'armenian
 (font-spec :registry "iso10646-1" :script 'armenian))

;; The Thai fontset definition assumes that you have OpenType
;; definitions set up properly.  We need an explicit :script
;; declaration as a backup.  I think this should go after the :otf
;; declaration, but I'm doing it as a 'prepend here.  (Perhaps
;; FreeSerif Thai fonts comply with OpenType, but Misc-Fixed
;; ones don't.)
(set-fontset-font "fontset-default" 'thai
 (font-spec :registry "iso10646-1" :script 'thai) nil 'prepend)

;; 'latin script needs a declaration that incorporates the Latin
;; representative characters.  None of the current ones do.
(set-fontset-font "fontset-default" 'latin
 (font-spec :registry "iso10646-1" :script 'latin) nil 'prepend)

;; No 'symbol script declaration exists.  Create one using the new
;; representative characters.
(set-fontset-font "fontset-default" 'symbol
 (font-spec :registry "iso10646-1" :script 'symbol))

;; No 'phonetic declaration exists.  Create one using the new
;; representative characters.  (Note that there is an 'ipa
;; declaration, but 'ipa doesn't exist as a real script.)
(set-fontset-font "fontset-default" 'phonetic
 (font-spec :registry "iso10646-1" :script 'phonetic))

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

sand@blarg.net writes:
> The font-backend branch has the same problem, and essentially the same
> patch works for it.  It's not worth constructing a separate patch for
> that branch.  Plus, it looks like the patch itself, while useful, is
> fixing secondary problems rather than root problems.  I decided to try
> attacking the issues at the Lisp level.
> 
> Most of the problems seem to be caused by bad configuration in
> "fontset.el".  I have appended the Lisp code I came up with to get my
> environment (almost) working.  The exception is Korean syllabics,
> which show up fine with the old backend (using Daewoo Mincho
> KSC5601.1987) but don't show up at all with the new one.  The fontset
> declaration looks okay, so I'm going to keep digging.

I got Hangul working after adding another declaration for ISO-10646
with a :script parameter:

;; 'hangul script has a :language restriction for iso10646-1, but also
;; needs :script restriction as alternative.
(set-fontset-font "fontset-default" 'hangul
                  (font-spec :registry "iso10646-1" :script 'hangul) nil 'prepend)

It looks like any time we have an :otf or a :language declaration for
a script, we need a :script declaration as a backup for safety.

Derek

-- 
Derek Upham
sand@blarg.net

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[Kenichi Handa]

In article <18467.55964.443458.171763@priss.frightenedpiglet.com>, sand@blarg.net writes:

> Can I please get an ack on this patch, or at least on the code problem
> that it's fixing?  I'd like to get some sort of fix into CVS.

Sorry for no response on this matter.  I'm now in Porland,
and don't have a time to consider your patch.  Could you
please wait until I go back to Japan next week?

---
Kenichi Handa
handa@ni.aist.go.jp

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[Kenichi Handa]

Sorry for the late response on this matter.

In article <18467.55964.443458.171763@priss.frightenedpiglet.com>, sand@blarg.net writes:

> > The ftfont_spec_pattern() function generates an FcPattern object that
> > can be used to list only fonts matching the spec.  For the purposes of
> > this discussion, there are two "interesting" ways of restricting
> > patterns: via charset (FcCharSet), or via langset (FcLangSet).  The
> > former requires the font to have each of the codepoints listed in the
> > FcCharSet.  The latter requires the font to support all the languages
> > in the FcLangSet.
> > 
> > 1. If we pass a font spec with registry ISO-8859 to
> > ftfont_spec_pattern(), then the code sets up an FcCharSet that has
> > every ASCII codepoint (but not Latin-1, that's commented out for some
> > reason).
> > 
> > 2. If we pass a font spec with a non-ISO-8859, non-ISO-10646,
> > non-Unicode-BMP registry, the function immediately returns an empty
> > pattern.
> > 
> > 3. ISO-10646 and Unicode-BMP registries are handled in a more
> > complicated manner...
> > 
> > If the ISO-10646 font spec has an associated :script parameter (or an
> > OpenType spec that refers to a script), the code looks in
> > 'script-representative-chars' for codepoints to put into a charset.
> > If the font spec has an associated language, the code adds the
> > language to the langset.
> > 
> > However, an ISO-10646 font spec without a special script or language
> > ends up with neither a charset nor a langset.  The resulting pattern
> > will match *any* characters and languages.  In partcular, it will let
> > an ISO-8859 font match the ISO-10646 spec.
> > 
> > The fix below checks for a missing charset and missing langset.  In
> > that case, we create a charset with at least one ISO-10646 codepoint
> > outside of ISO-8859.  The charset should be as small as possible,
> > since a font missing any of the charset's codepoints becomes
> > completely invalid.  I have chosen LEFT DOUBLE QUOTATION MARK, which
> > is associated with English and which I believe is pervasive.

I think it's very ad-hoc to return only such fonts that has
that specific character in that case.  Many non-English
fonts doesn't support it.

First of all, I have not that much considered about using
PCF fonts via ftfont driver because it can be used by xfont
driver if you put them in X's font-path.  On the contrary, I
was even thinking about rejecting all PCF fonts in xft
backend.

It's very unfortunate that fontconfig doesn't provide a
facility to find PCF fonts by registry.

If we still keep supporting PFC fonts in xft backend, I
think the right way is to create a FcCharSet object for each
registry (other than iso-10646 and unicode-*) that is
sufficient to avoid incorrect match.  The mapping of
registry->chars can be defined by some thing like
`registry-representative-chars' (analogus to
script-representative-chars).

And, for the case that iso10646-1 font is requested, we
should use FT_Get_BDF_Charset_ID to avoid non-iso10646-1 PCF
fonts, perhaps considering a use a cache if that is too
slow.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[sand]

Kenichi Handa writes:
> I think it's very ad-hoc to return only such fonts that has
> that specific character in that case.  Many non-English
> fonts doesn't support it.

Please take a look at my recent mail with the subject "Accumulated
fontset definition tweaks for testing".  I have come up with fixes for
all my display problems in the Emacs Lisp libraries.  The email lists
the various problems I found in "fontset.el".

> First of all, I have not that much considered about using
> PCF fonts via ftfont driver because it can be used by xfont
> driver if you put them in X's font-path.  On the contrary, I
> was even thinking about rejecting all PCF fonts in xft
> backend.

Does the xfont driver continue to work if a person specifies "xft" as
the font backend?  If it does, then I think that will be okay.  If it
doesn't then the ftfont driver should continue to support PCF fonts
alongside TrueType fonts, for consistency with other applications.

Derek

-- 
Derek Upham
sand@blarg.net

Re: [PATCH] Re: ftfont ISO10646-1 font bug found (was Re: 23.0.60; Heavy display problems with new font backend)

[Kenichi Handa]

In article <18477.8958.959847.65162@priss.frightenedpiglet.com>, sand@blarg.net writes:

> Kenichi Handa writes:
> > I think it's very ad-hoc to return only such fonts that has
> > that specific character in that case.  Many non-English
> > fonts doesn't support it.

> Please take a look at my recent mail with the subject "Accumulated
> fontset definition tweaks for testing".  I have come up with fixes for
> all my display problems in the Emacs Lisp libraries.  The email lists
> the various problems I found in "fontset.el".

I'm now reading it, and will respond soon.

> > First of all, I have not that much considered about using
> > PCF fonts via ftfont driver because it can be used by xfont
> > driver if you put them in X's font-path.  On the contrary, I
> > was even thinking about rejecting all PCF fonts in xft
> > backend.

> Does the xfont driver continue to work if a person specifies "xft" as
> the font backend?

No if one specifies ONLY xft backend.  But, why does he do
that if he also wants PCF fonts?

---
Kenichi Handa
handa@ni.aist.go.jp