bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Toshikazu Nakamura]

Hello,

I use Emacs for editing texts that contain Japanese as well as English
and other languages. In order to give characters of each language a
better appearance, I use a fontset that specifies a different font for
ASCII characters and double-width Japanese characters.

Since Emacs 25.1, however, I find that the default ASCII font is
always used for some characters that belongs to some other character
set (e.g. japanese-jisx0208) and for which another font is specified.

(set-language-environment "Japanese")
(set-face-attribute 'default nil :family "DejaVu Sans Mono" :height 96)
(set-fontset-font nil 'japanese-jisx0208
                  (font-spec :family "TakaoGothic"))
(add-to-list 'face-font-rescale-alist '(".*TakaoGothic.*" . 1.25))

When I launch Emacs with the "-Q" option and load the above settings,
the default ASCII font is used for characters like #x2026 and #x25bd,
though the result of describe-char shows that they belong to
japanese-jisx0208:

             position: 80 of 4241 (2%), column: 44
            character: … (displayed as …) (codepoint 8230, #o20046, #x2026)
    preferred charset: japanese-jisx0208 (JISX0208.1983/1990 Japanese Kanji: ISO-IR-87)
code point in charset: 0x2144
               script: symbol
               syntax: _ 	which means: symbol
             category: .:Base, >:Not at bol, c:Chinese, h:Korean, j:Japanese
             to input: type "C-x 8 RET 2026" or "C-x 8 RET HORIZONTAL ELLIPSIS"
          buffer code: #xE2 #x80 #xA6
            file code: #xE2 #x80 #xA6 (encoded by coding system utf-8-unix)
              display: by this font (glyph code)
    xft:-unknown-DejaVu Sans Mono-normal-normal-normal-*-13-*-*-*-m-0-iso10646-1 (#x728)

Character code properties: customize what to show
  name: HORIZONTAL ELLIPSIS
  general-category: Po (Punctuation, Other)
  decomposition: (compat 46 46 46) (compat '.' '.' '.')

Emacs 24 uses for these symbols the font specified for
japanese-jisx0208 and displays them as double-width Japanese symbols.
When displayed with the default ASCII font as in Emacs 25, these
symbols give Japanese texts a rather awkward appearance.

The same issue is discussed in the thread starting with the following
message:

http://lists.gnu.org/archive/html/help-gnu-emacs/2016-10/msg00004.html

I agree with Alex that there should be some way to override the
default ASCII font, especially because in my case it considerably
alters the appearance of the text. I would appreciate it if some
customization could be made possible.

Best regards.



In GNU Emacs 26.0.50.2 (x86_64-unknown-linux-gnu, GTK+ Version 3.14.5)
 of 2016-10-03 built on epson
Repository revision: a7e9d1cce3f935dbe9f242f0bf2dbf34f5919952
Windowing system distributor 'The X.Org Foundation', version 11.0.11604000
System Description:	Kona Linux 3.0 Mate/x86_64

Configured using:
 'configure --without-xim'

Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GCONF GSETTINGS
NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS GTK3 X11

Important settings:
  value of $LANG: ja_JP.UTF-8
  value of $XMODIFIERS: @im=uim
  locale-coding-system: utf-8-unix

Major mode: Folder

Minor modes in effect:
  shell-dirtrack-mode: t
  global-company-mode: t
  global-smart-tab-mode: t
  recentf-mode: t
  delete-selection-mode: t
  show-paren-mode: t
  scroll-mode-line-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/usr/local/share/emacs/site-lisp/flim/hex-util hides /usr/local/share/emacs/26.0.50/lisp/hex-util
/usr/local/share/emacs/site-lisp/flim/md4 hides /usr/local/share/emacs/26.0.50/lisp/md4
/usr/local/share/emacs/site-lisp/flim/hmac-def hides /usr/local/share/emacs/26.0.50/lisp/net/hmac-def
/usr/local/share/emacs/site-lisp/flim/sasl-cram hides /usr/local/share/emacs/26.0.50/lisp/net/sasl-cram
/usr/local/share/emacs/site-lisp/flim/sasl-digest hides /usr/local/share/emacs/26.0.50/lisp/net/sasl-digest
/usr/local/share/emacs/site-lisp/flim/ntlm hides /usr/local/share/emacs/26.0.50/lisp/net/ntlm
/usr/local/share/emacs/site-lisp/flim/sasl hides /usr/local/share/emacs/26.0.50/lisp/net/sasl
/usr/local/share/emacs/site-lisp/flim/hmac-md5 hides /usr/local/share/emacs/26.0.50/lisp/net/hmac-md5
/usr/local/share/emacs/site-lisp/flim/sasl-ntlm hides /usr/local/share/emacs/26.0.50/lisp/net/sasl-ntlm

Features:
(shadow eieio-opt speedbar sb-image ezimage dframe find-func dabbrev
hippie-exp help-fns radix-tree noutline outline rect shr-color color
thingatpt mel-q-ccl skk-cdb cdb descr-text emacsbug message rfc822 mml
mml-sec epa epg gnus-util rmail rmail-loaddefs mm-decode mm-bodies
mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
skk-annotation skk-look skk-num skk-cursor skk-sticky skk-dcomp
skk-comp skk-server skk-busyu skk-tankan skk-cus cus-edit cus-start
cus-load skk skk-emacs skk-macs skk-leim company-ispell ispell solar
cal-dst diary-lib diary-loaddefs pcmpl-x mime-shr shr svg dom
x-face-e21 wl-expire wl-score shell grep misearch multi-isearch
skk-vars network-stream puny seq nsm subr-x starttls navi2ch-thumbnail
navi2ch-p2 navi2ch-auto-modify navi2ch-localfile navi2ch-oyster
navi2ch-mona navi2ch-futaba navi2ch-machibbs navi2ch-jbbs-shitaraba
navi2ch-be2ch navi2ch-http-date navi2ch-multibbs navi2ch-jbbs-net
navi2ch-version navi2ch-splash navi2ch-e21 navi2ch-message
navi2ch-directory navi2ch-search navi2ch-history navi2ch-bookmark
navi2ch-articles navi2ch-board navi2ch-board-misc
navi2ch-popup-article navi2ch-article navi2ch-list navi2ch-net
browse-url navi2ch-util navi2ch-face navi2ch-config navi2ch-vars
navi2ch modb-legacy elmo-pop3 elmo-internal elmo-archive modb-standard
wl-mime mime-play filename mime-edit mime-setup mail-mime-setup
semi-setup mime-image wl-spam elmo-spam elsp-header elsp-generic tls
gnutls wl-demo wl-draft eword-encode wl-template sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils elmo-net elmo-cache
elmo-map elmo-dop wl-news wl-address wl-thread wl-action wl-summary
wl-refile wl-message elmo-mime mmelmo-buffer mmelmo-imap mime-view
mime-conf calist semi-def mmimap mime-parse mmbuffer mmgeneric
wl-highlight elmo-multi wl-folder wl wl-e21 wl-util elmo-flag
elmo-localdir wl-vars epg-config wl-version elmo elmo-signal
elmo-msgdb modb modb-generic modb-entity mime elmo-util emu invisible
inv-23 poem poem-e20 poem-e20_3 eword-decode mel mime-def alist
mcharset mcs-20 mcs-e20 pces pces-e20 pces-20 pcustom elmo-date
timezone elmo-vars path-util elmo-version luna std11 pccl pccl-20 ccl
broken poe pym static apel-ver product disp-table ccc company-oddmuse
company-keywords company-etags etags xref project company-gtags
company-dabbrev-code company-dabbrev company-files company-capf
company-cmake company-xcode company-clang company-semantic
company-eclim company-template company-css company-nxml company-bbdb
pcomplete migemo howm-diary howm-lang-ja howm-lang-en calfw-howm
howm-menu howm-reminder howm-date howm-misc howm-mode howm-view
howm-backend howm honest-report action-lock riffle gfunc illusion
cheat-font-lock howm-common howm-vars howm-version company calfw-cal
calfw japanese-holidays holidays hol-loaddefs cal-menu calendar
cal-loaddefs view smart-tab smart-quotes session redo+ recentf
tree-widget wid-edit minibuf-isearch jaspace whitespace
emms-player-mpv emms-info-libtag emms-librefm-stream xml
emms-librefm-scrobbler emms-playlist-limit emms-volume
emms-volume-amixer emms-i18n emms-history emms-score emms-stream-info
emms-metaplaylist-mode emms-bookmarks emms-cue emms-mode-line-icon
emms-browser sort emms-playlist-sort emms-last-played emms-player-xine
emms-player-mpd tq emms-playing-time emms-lyrics emms-url url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util url-parse auth-source cl-seq eieio byte-opt
bytecomp byte-compile cl-extra cconv eieio-core eieio-loaddefs
password-cache url-vars mailcap emms-streams emms-show-all
emms-tag-editor format-spec emms-mark emms-mode-line emms-cache
emms-info-ogginfo emms-info-mp3info emms-info later-do
emms-playlist-mode emms-player-vlc emms-player-mplayer
emms-player-simple emms-source-playlist emms-source-file locate
emms-setup emms emms-compat bs browse-kill-ring bookmark+
bookmark+-key derived dired-x bookmark+-1 bookmark+-bmu help-mode
easymenu bookmark+-lit bookmark+-mac bookmark pp ack compile comint
ansi-color ring pdf-preview ps-mule ps-print ps-print-loaddefs ps-def
lpr delsel edmacro kmacro dired dired-loaddefs paren avoid cl-macs
easy-mmode cl gv cl-loaddefs pcase cl-lib scroll-mode-line-mode server
japan-util time-date mule-util advice skk-setup skk-autoloads tooltip
eldoc electric uniquify ediff-hook vc-hooks lisp-float-type mwheel
term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment elisp-mode lisp-mode
prog-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core term/tty-colors
frame cl-generic cham georgian utf-8-lang misc-lang vietnamese tibetan
thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
charscript case-table epa-hook jka-cmpr-hook help simple abbrev
obarray minibuffer cl-preloaded nadvice loaddefs button faces cus-face
macroexp files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
dbusbind inotify dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 589477 145280)
 (symbols 48 58877 12)
 (miscs 40 1496 3501)
 (strings 32 173865 10926)
 (string-bytes 1 5945151)
 (vectors 16 56451)
 (vector-slots 8 1870026 203484)
 (floats 8 1230 1238)
 (intervals 56 18117 4643)
 (buffers 976 83)
 (heap 1024 91074 48247))

-- 
Toshikazu Nakamura

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Eli Zaretskii]

> Date: Sat, 08 Oct 2016 23:15:06 +0900
> From: Toshikazu Nakamura <bmcbq234@yahoo.co.jp>
> 
> I use Emacs for editing texts that contain Japanese as well as English
> and other languages. In order to give characters of each language a
> better appearance, I use a fontset that specifies a different font for
> ASCII characters and double-width Japanese characters.
> 
> Since Emacs 25.1, however, I find that the default ASCII font is
> always used for some characters that belongs to some other character
> set (e.g. japanese-jisx0208) and for which another font is specified.

This is a deliberate change in Emacs 25.1.  Preferring the default
face's font is better in the vast majority of cases, because it
doesn't disrupt the text alignment.  The preferred charset shown by
"C-u C-x =" has nothing to do with selecting the font.  (I also wonder
what does a character such as u+2026 has to do with Japanese
characters.)

I've now added a variable that you can set to disable the new behavior
and get back the old pre-Emacs 25.1 behavior, for those users who
want/need that.  The change is on the emacs-25 branch, so Emacs 25.2
will have it.

With that, I'm marking this bug done.

Thanks.

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 441 bytes --]

On 2016-10-08 15:10, Eli Zaretskii wrote:
> I've now added a variable that you can set to disable the new behavior
> and get back the old pre-Emacs 25.1 behavior, for those users who
> want/need that.  The change is on the emacs-25 branch, so Emacs 25.2
> will have it.

How is the new variable called?  Also, what's the way to use a given font for a given charset in Emacs 25.1, without setting that variable?

Thanks!
Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Eli Zaretskii]

> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sat, 8 Oct 2016 20:14:34 -0400
> 
> On 2016-10-08 15:10, Eli Zaretskii wrote:
> > I've now added a variable that you can set to disable the new behavior
> > and get back the old pre-Emacs 25.1 behavior, for those users who
> > want/need that.  The change is on the emacs-25 branch, so Emacs 25.2
> > will have it.
> 
> How is the new variable called?

It's mentioned in NEWS, of course.

> Also, what's the way to use a given font for a given charset in Emacs 25.1, without setting that variable?

You should use set-fontset-font, I think its doc string explains
this.  Also, there's an example in the ELisp manual, where
set-fontset-font is described.

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Clément Pit--Claudel]

[-- Attachment #1.1: Type: text/plain, Size: 1136 bytes --]

On 2016-10-09 03:08, Eli Zaretskii wrote:
>> From: Clément Pit--Claudel <clement.pit@gmail.com>
>> Date: Sat, 8 Oct 2016 20:14:34 -0400
>>
>> On 2016-10-08 15:10, Eli Zaretskii wrote:
>>> I've now added a variable that you can set to disable the new behavior
>>> and get back the old pre-Emacs 25.1 behavior, for those users who
>>> want/need that.  The change is on the emacs-25 branch, so Emacs 25.2
>>> will have it.
>>
>> How is the new variable called?
> 
> It's mentioned in NEWS, of course.
> 
>> Also, what's the way to use a given font for a given charset in Emacs 25.1, without setting that variable?
> 
> You should use set-fontset-font, I think its doc string explains
> this.  Also, there's an example in the ELisp manual, where
> set-fontset-font is described.

Thanks for clarifying.  I don't understand why the original change was needed, though.  What prevented users from adding the default font as the first font in their fontset?  Wouldn't that have achieved exactly the same (pick the character from the default font if available, and otherwise from the next one in the fontset)?

Clément.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Eli Zaretskii]

> Cc: 24644@debbugs.gnu.org
> From: Clément Pit--Claudel <clement.pit@gmail.com>
> Date: Sun, 9 Oct 2016 10:29:44 -0400
> 
> I don't understand why the original change was needed, though.  What prevented users from adding the default font as the first font in their fontset?  Wouldn't that have achieved exactly the same (pick the character from the default font if available, and otherwise from the next one in the fontset)?

The intent was for Emacs to DTRT _by_default_, without asking users to
customize their fontsets.  Doing the latter is somewhat tricky, and
besides it's considered a Good Thing for a program to behave as users
expect out of the box.

bug#24644: 26.0.50; Emacs 25: set-fontset-font does not take effect with some ASCII symbols

[Eli Zaretskii]

> Date: Mon, 10 Oct 2016 18:15:10 +0900
> From: Toshikazu Nakamura <bmcbq234@yahoo.co.jp>
> 
> > This is a deliberate change in Emacs 25.1.  Preferring the default
> > face's font is better in the vast majority of cases, because it
> > doesn't disrupt the text alignment.  The preferred charset shown by
> > "C-u C-x =" has nothing to do with selecting the font.  (I also wonder
> > what does a character such as u+2026 has to do with Japanese
> > characters.)
> 
> Thanks for the explanation.
> 
> When you edit Japanese texts, double-width Japanese characters are
> displayed alongside single-width ASCII characters. u+2026 (HORIZONTAL
> ELLIPSIS) and a bunch of other symbols are frequently used in Japanese
> texts and should be displayed as double-width Japanese symbols, at
> least when they occur in parts of the text written in Japanese.
> 
> > I've now added a variable that you can set to disable the new behavior
> > and get back the old pre-Emacs 25.1 behavior, for those users who
> > want/need that.  The change is on the emacs-25 branch, so Emacs 25.2
> > will have it.
> 
> I have tested the change and it is exactly what I wanted.
> 
> I was not sure how to reply the message delivered from the bug
> tracker, so I deleted the CC field just in case.

You should have kept that CC field.

> Thank you for the quick response.

Thanks for testing the fix.