bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Ilya Chernyshov]

Hi. Can't reproduce the issue by myself (I'm on linux), but already have
been sent a couple of issues in my package repo.

Screenshot:

https://user-images.githubusercontent.com/9244946/280170406-64aca531-3542-40b7-9ae3-fc13ab3f1029.png

Link to the issue:

https://github.com/ichernyshovvv/org-timeblock/issues/36

Code to reproduce:

(progn
  (require 'svg)
  (switch-to-buffer "*svg-test*")
  (let ((svg-image (svg-create 200 200))
	(inhibit-read-only t))
    (erase-buffer)
    (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
    (svg-print svg-image)
    (image-mode)))

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Eli Zaretskii]

> From: Ilya Chernyshov <ichernyshovvv@gmail.com>
> Date: Sun, 05 Nov 2023 02:00:18 +0700
> 
> 
> Hi. Can't reproduce the issue by myself (I'm on linux), but already have
> been sent a couple of issues in my package repo.
> 
> Screenshot:
> 
> https://user-images.githubusercontent.com/9244946/280170406-64aca531-3542-40b7-9ae3-fc13ab3f1029.png
> 
> Link to the issue:
> 
> https://github.com/ichernyshovvv/org-timeblock/issues/36
> 
> Code to reproduce:
> 
> (progn
>   (require 'svg)
>   (switch-to-buffer "*svg-test*")
>   (let ((svg-image (svg-create 200 200))
> 	(inhibit-read-only t))
>     (erase-buffer)
>     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
>     (svg-print svg-image)
>     (image-mode)))

I cannot get Emacs on Windows to display _any_ non-ASCII text with the
above recipe, not even if I use Latin-1 characters or characters from
the current system codepage.  Emacs encodes non-ASCII characters as
&#nnnn numbers (you can see that if you type "C-c C-c" in the
*svg-test* buffer), which is probably correct, but the SVG image is
not displayed for some reason.

Alan, any ideas or suggestions?

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Eli Zaretskii]

> Cc: 66941@debbugs.gnu.org
> Date: Sun, 05 Nov 2023 08:47:08 +0200
> From: Eli Zaretskii <eliz@gnu.org>
> 
> > (progn
> >   (require 'svg)
> >   (switch-to-buffer "*svg-test*")
> >   (let ((svg-image (svg-create 200 200))
> > 	(inhibit-read-only t))
> >     (erase-buffer)
> >     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
> >     (svg-print svg-image)
> >     (image-mode)))
> 
> I cannot get Emacs on Windows to display _any_ non-ASCII text with the
> above recipe, not even if I use Latin-1 characters or characters from
> the current system codepage.  Emacs encodes non-ASCII characters as
> &#nnnn numbers (you can see that if you type "C-c C-c" in the
> *svg-test* buffer), which is probably correct, but the SVG image is
> not displayed for some reason.
> 
> Alan, any ideas or suggestions?

Could be a problem with the font used by librsvg to show text?  Maybe
the font doesn't have any non-ASCII characters.

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Alan Third]

On Sun, Nov 05, 2023 at 08:54:27AM +0200, Eli Zaretskii wrote:
> > Cc: 66941@debbugs.gnu.org
> > Date: Sun, 05 Nov 2023 08:47:08 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > 
> > > (progn
> > >   (require 'svg)
> > >   (switch-to-buffer "*svg-test*")
> > >   (let ((svg-image (svg-create 200 200))
> > > 	(inhibit-read-only t))
> > >     (erase-buffer)
> > >     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
> > >     (svg-print svg-image)
> > >     (image-mode)))
> > 
> > I cannot get Emacs on Windows to display _any_ non-ASCII text with the
> > above recipe, not even if I use Latin-1 characters or characters from
> > the current system codepage.  Emacs encodes non-ASCII characters as
> > &#nnnn numbers (you can see that if you type "C-c C-c" in the
> > *svg-test* buffer), which is probably correct, but the SVG image is
> > not displayed for some reason.
> > 
> > Alan, any ideas or suggestions?
> 
> Could be a problem with the font used by librsvg to show text?  Maybe
> the font doesn't have any non-ASCII characters.

This seems plausible. It should be possible to check by over-riding
the CSS with a font you know should work:

(progn
  (require 'svg)
  (switch-to-buffer "*svg-test*")
  (let ((svg-image (svg-create 200 200))
        (inhibit-read-only t))
    (erase-buffer)
    (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
    (insert-image (svg-image svg-image :css "svg{font-family:\"Noto Sans Mono CJK HK\";}"))))

Choose a font you know can show the text you're testing. I've a vague
memory that the Noto CJK fonts behave slightly oddly on Windows, so
maybe something else is a better choice.

I think setting the :css attribute to an empty string should reset
librsvg so it will use its default font settings and that would let
us see if it works right without our customisation.

-- 
Alan Third

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Eli Zaretskii]

> Date: Sun, 5 Nov 2023 23:27:23 +0000
> From: Alan Third <alan@idiocy.org>
> Cc: ichernyshovvv@gmail.com, 66941@debbugs.gnu.org
> 
> On Sun, Nov 05, 2023 at 08:54:27AM +0200, Eli Zaretskii wrote:
> > > Cc: 66941@debbugs.gnu.org
> > > Date: Sun, 05 Nov 2023 08:47:08 +0200
> > > From: Eli Zaretskii <eliz@gnu.org>
> > > 
> > > > (progn
> > > >   (require 'svg)
> > > >   (switch-to-buffer "*svg-test*")
> > > >   (let ((svg-image (svg-create 200 200))
> > > > 	(inhibit-read-only t))
> > > >     (erase-buffer)
> > > >     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
> > > >     (svg-print svg-image)
> > > >     (image-mode)))
> > > 
> > > I cannot get Emacs on Windows to display _any_ non-ASCII text with the
> > > above recipe, not even if I use Latin-1 characters or characters from
> > > the current system codepage.  Emacs encodes non-ASCII characters as
> > > &#nnnn numbers (you can see that if you type "C-c C-c" in the
> > > *svg-test* buffer), which is probably correct, but the SVG image is
> > > not displayed for some reason.
> > > 
> > > Alan, any ideas or suggestions?
> > 
> > Could be a problem with the font used by librsvg to show text?  Maybe
> > the font doesn't have any non-ASCII characters.
> 
> This seems plausible. It should be possible to check by over-riding
> the CSS with a font you know should work:
> 
> (progn
>   (require 'svg)
>   (switch-to-buffer "*svg-test*")
>   (let ((svg-image (svg-create 200 200))
>         (inhibit-read-only t))
>     (erase-buffer)
>     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
>     (insert-image (svg-image svg-image :css "svg{font-family:\"Noto Sans Mono CJK HK\";}"))))
> 
> Choose a font you know can show the text you're testing. I've a vague
> memory that the Noto CJK fonts behave slightly oddly on Windows, so
> maybe something else is a better choice.
> 
> I think setting the :css attribute to an empty string should reset
> librsvg so it will use its default font settings and that would let
> us see if it works right without our customisation.

Thanks.  I tried the above, and it didn't work, at least with (old)
librsvg 2.40.1 I have here.  Moreover, if I use a plain ASCII string,
which does display, the font used for display is identical whether or
not I specify font-family in the :css attribute, so I think somehow
this attribute doesn't have any effect.

Is it possible to specify a font rather than font-family?  Font
families are treated poorly on MS-Windows, so I'd prefer to specify a
particular font, to be sure.

bug#66941: 29.1; Emacs on Windows can't display CJK characters correctly in svg images

[Alan Third]

On Mon, Nov 06, 2023 at 02:21:25PM +0200, Eli Zaretskii wrote:
> > Date: Sun, 5 Nov 2023 23:27:23 +0000
> > From: Alan Third <alan@idiocy.org>
> > Cc: ichernyshovvv@gmail.com, 66941@debbugs.gnu.org
> > 
> > On Sun, Nov 05, 2023 at 08:54:27AM +0200, Eli Zaretskii wrote:
> > > Could be a problem with the font used by librsvg to show text?  Maybe
> > > the font doesn't have any non-ASCII characters.
> > 
> > This seems plausible. It should be possible to check by over-riding
> > the CSS with a font you know should work:
> > 
> > (progn
> >   (require 'svg)
> >   (switch-to-buffer "*svg-test*")
> >   (let ((svg-image (svg-create 200 200))
> >         (inhibit-read-only t))
> >     (erase-buffer)
> >     (svg-text svg-image "我喜欢自由软件" :y 100 :x 50)
> >     (insert-image (svg-image svg-image :css "svg{font-family:\"Noto Sans Mono CJK HK\";}"))))
> > 
> > Choose a font you know can show the text you're testing. I've a vague
> > memory that the Noto CJK fonts behave slightly oddly on Windows, so
> > maybe something else is a better choice.
> > 
> > I think setting the :css attribute to an empty string should reset
> > librsvg so it will use its default font settings and that would let
> > us see if it works right without our customisation.
> 
> Thanks.  I tried the above, and it didn't work, at least with (old)
> librsvg 2.40.1 I have here.  Moreover, if I use a plain ASCII string,
> which does display, the font used for display is identical whether or
> not I specify font-family in the :css attribute, so I think somehow
> this attribute doesn't have any effect.

Ah, OK. CSS only works on librsvg 2.48 and above.

This means that, at least for you, Emacs has no effect on the font
used by librsvg, it will be using its default. I've no idea what that
is and a quick look through the librsvg code hasn't helped.

It should be possible to over-ride it in the svg file itself, but
that's really up to the user.

I don't know how librsvg handles fallbacks for glyphs the font doesn't
support or if there's any way for us to over-ride its behaviour.

> Is it possible to specify a font rather than font-family?  Font
> families are treated poorly on MS-Windows, so I'd prefer to specify a
> particular font, to be sure.

This is how CSS works even in web browsers, I don't think there's a
separate "font" option.
-- 
Alan Third