format width specifier and locale

format width specifier and locale

[Joost Kremers]

[-- Attachment #1: Type: text/plain, Size: 950 bytes --]

Hi list,

While looking into a bug report for a package of mine,[1] it turned out that part
of the bug is caused by a strange interaction between `format` and the system's
locale that looks like a bug, but I'd like to ask here first before I submit a
bug report.

The following two `format` calls produce identical-length strings when issued in
an Emacs with `current-language-environment` set to "English" (or, in my case
"UTF-8"), but when the language environment is set to "Chinese-GBK", the second
call produces a string that is one character shorter:

    (format (concat "%50s") "Boutet de Monvel's calculus and groupoids I")
    (format (concat "%50s") "Boutet de Monvel’s calculus and groupoids I") 

The difference between the two string is the apostrophe character: in the first
string, it's an ASCII apostrophe, in the second string it's a RIGHT SINGLE
QUOTATION MARK.

The effect can be seen in this screen shot:


[-- Attachment #2: Screenshot from 2022-02-23 21.19.14.png --]
[-- Type: image/png, Size: 57261 bytes --]

[-- Attachment #3: Type: text/plain, Size: 375 bytes --]


The font used is DejaVu Sans Mono. I switched language environment with `M-x
set-language-environment`. The second string produced in the Chinese language
environment has one space less.

So is there a reason this is happening, or is it indeed a bug?

TIA

Joost




Footnotes:
[1]  <https://github.com/joostkremers/ebib/issues/243>.

-- 
Joost Kremers
Life has its moments

Re: format width specifier and locale

[Eli Zaretskii]

> From: Joost Kremers <joostkremers@fastmail.fm>
> Date: Wed, 23 Feb 2022 21:02:31 +0100
> 
> While looking into a bug report for a package of mine,[1] it turned out that part
> of the bug is caused by a strange interaction between `format` and the system's
> locale that looks like a bug, but I'd like to ask here first before I submit a
> bug report.
> 
> The following two `format` calls produce identical-length strings when issued in
> an Emacs with `current-language-environment` set to "English" (or, in my case
> "UTF-8"), but when the language environment is set to "Chinese-GBK", the second
> call produces a string that is one character shorter:
> 
>     (format (concat "%50s") "Boutet de Monvel's calculus and groupoids I")
>     (format (concat "%50s") "Boutet de Monvel’s calculus and groupoids I") 
> 
> The difference between the two string is the apostrophe character: in the first
> string, it's an ASCII apostrophe, in the second string it's a RIGHT SINGLE
> QUOTATION MARK.

If you play with the value of text-quoting-style, does that affect the
results in any way?

Re: format width specifier and locale

[Eli Zaretskii]

> Date: Thu, 24 Feb 2022 08:40:23 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> >     (format (concat "%50s") "Boutet de Monvel's calculus and groupoids I")
> >     (format (concat "%50s") "Boutet de Monvel’s calculus and groupoids I") 
> > 
> > The difference between the two string is the apostrophe character: in the first
> > string, it's an ASCII apostrophe, in the second string it's a RIGHT SINGLE
> > QUOTATION MARK.
> 
> If you play with the value of text-quoting-style, does that affect the
> results in any way?

Also, in which Emacs version(s) do you see this?

Re: format width specifier and locale

[Eli Zaretskii]

> Date: Thu, 24 Feb 2022 09:16:23 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: emacs-devel@gnu.org
> 
> > Date: Thu, 24 Feb 2022 08:40:23 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: emacs-devel@gnu.org
> > 
> > >     (format (concat "%50s") "Boutet de Monvel's calculus and groupoids I")
> > >     (format (concat "%50s") "Boutet de Monvel’s calculus and groupoids I") 
> > > 
> > > The difference between the two string is the apostrophe character: in the first
> > > string, it's an ASCII apostrophe, in the second string it's a RIGHT SINGLE
> > > QUOTATION MARK.
> > 
> > If you play with the value of text-quoting-style, does that affect the
> > results in any way?
> 
> Also, in which Emacs version(s) do you see this?

Please ignore these questions.  I see the reason for that behavior:
in the Chinese-GBK language environment, Emacs thinks that the U+2019
character is a double-width character:

  emacs -Q
  C-x RET l Chinese-GBK RET
  M-: (char-width #x2019) RET

This yields 2, not 1.

I'm investigating why we override the default width of U+2019 in this
language-environment (see the function use-cjk-char-width-table in
characters.el and lisp/language/chinese.el that uses it).  Stay tuned
(and maybe file a bug report, so this doesn't get forgotten by any
chance).

Thanks.

Re: format width specifier and locale

[Joost]

On Thu, 24 Feb 2022, at 08:54, Eli Zaretskii wrote:
> (and maybe file a bug report, so this doesn't get forgotten by any
> chance).

I filed a bug report.

-- 
Joost Kremers
Life has its moments