unofficial mirror of bug-gnu-emacs@gnu.org 
 help / color / mirror / code / Atom feed
* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
@ 2015-02-15  9:05 Sebastien Vauban
  2015-02-15 16:27 ` Eli Zaretskii
                   ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Sebastien Vauban @ 2015-02-15  9:05 UTC (permalink / raw)
  To: 19872-ubl+/3LiMTaZdePnXv/OxA

Hello,

Now that I've changed some Gnus markers to UTF8 "drawings" [1], I don't
have my Gnus summary buffer correctly aligned anymore.

See http://screencast.com/t/s33eYYg7Cn.

Is there a solution to that, to guarantee that the alignment can be
correct?

Worse, it seems that the same UTF8 char can have a "correct" width in
some fonts, and not in others... Not speaking that some UTF8 chars exist
in some fonts, and not in others...

Best regards,
  Seb

[1] Here is my setup:

--8<---------------cut here---------------start------------->8---
(setq gnus-unread-mark ?✉)
(setq gnus-del-mark ?✗)
(setq gnus-read-mark ?✓)
(setq gnus-killed-mark ?☠)
(setq gnus-unseen-mark ?✩)
--8<---------------cut here---------------end--------------->8---

-- 
Sebastien Vauban





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-15  9:05 bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers) Sebastien Vauban
@ 2015-02-15 16:27 ` Eli Zaretskii
       [not found] ` <mailman.138.1424017693.31049.bug-gnu-emacs@gnu.org>
  2016-02-07  6:04 ` Lars Ingebrigtsen
  2 siblings, 0 replies; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-15 16:27 UTC (permalink / raw)
  To: Sebastien Vauban; +Cc: 19872

> From: Sebastien Vauban <sva-news@mygooglest.com>
> Date: Sun, 15 Feb 2015 10:05:58 +0100
> 
> Now that I've changed some Gnus markers to UTF8 "drawings" [1], I don't
> have my Gnus summary buffer correctly aligned anymore.
> 
> See http://screencast.com/t/s33eYYg7Cn.
> 
> Is there a solution to that, to guarantee that the alignment can be
> correct?

Only if Gnus will align text using the :align-to display property,
instead of inserting whitespace characters.

> Worse, it seems that the same UTF8 char can have a "correct" width in
> some fonts, and not in others...

Of course: it depends on the dimensions of the glyphs in each font.





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
       [not found]   ` <mailman.138.1424017693.31049.bug-gnu-emacs-mXXj517/zsQ@public.gmane.org>
@ 2015-02-16  9:45     ` Sebastien Vauban
  2015-02-16 15:51       ` Eli Zaretskii
  0 siblings, 1 reply; 15+ messages in thread
From: Sebastien Vauban @ 2015-02-16  9:45 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: 19872-ubl+/3LiMTaZdePnXv/OxA

Eli Zaretskii wrote:
>> From: Sebastien Vauban <sva-news-D0wtAvR13HarG/iDocfnWg@public.gmane.org>
>> Date: Sun, 15 Feb 2015 10:05:58 +0100
>> 
>> Now that I've changed some Gnus markers to UTF8 "drawings" [1], I don't
>> have my Gnus summary buffer correctly aligned anymore.
>> 
>> See http://screencast.com/t/s33eYYg7Cn.
>> 
>> Is there a solution to that, to guarantee that the alignment can be
>> correct?
>
> Only if Gnus will align text using the :align-to display property,
> instead of inserting whitespace characters.

So, I take it for granted that it doesn't use it yet.  Is this quite
new?

>> Worse, it seems that the same UTF8 char can have a "correct" width in
>> some fonts, and not in others...
>
> Of course: it depends on the dimensions of the glyphs in each font.

Yes, but I was wondering (or hoping) if there was a mechanism in Emacs
to sort of zoom in/out the characters so that they'd take the same space
regardless of their (buggy?) definition (buggy when in
a non-proportional font)...

PS- This problem may occur, maybe, because of automatic font replacement
for characters not found in my default font (Consolas)?

Best regards,
  Seb





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-16  9:45     ` Sebastien Vauban
@ 2015-02-16 15:51       ` Eli Zaretskii
  2015-02-16 20:10         ` Stefan Monnier
  2015-02-17  4:17         ` Lars Ingebrigtsen
  0 siblings, 2 replies; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-16 15:51 UTC (permalink / raw)
  To: Sebastien Vauban; +Cc: 19872

> From: Sebastien Vauban <sva-news@mygooglest.com>
> Cc: 19872@debbugs.gnu.org
> Date: Mon, 16 Feb 2015 10:45:14 +0100
> 
> >> Is there a solution to that, to guarantee that the alignment can be
> >> correct?
> >
> > Only if Gnus will align text using the :align-to display property,
> > instead of inserting whitespace characters.
> 
> So, I take it for granted that it doesn't use it yet.

I guess, judging by your description.  I don't use Gnus, but you can
easily verify that by looking at each character in the offending line
with "C-x =": if all of the whitespace characters are simple blanks or
TABs, then you know.

> Is this quite new?

You mean, :align-to?  It was new 15 years or so ago.

> >> Worse, it seems that the same UTF8 char can have a "correct" width in
> >> some fonts, and not in others...
> >
> > Of course: it depends on the dimensions of the glyphs in each font.
> 
> Yes, but I was wondering (or hoping) if there was a mechanism in Emacs
> to sort of zoom in/out the characters so that they'd take the same space

No, it doesn't.  (Is that at all possible?  I'm not expert on fonts.)
Emacs simply chooses a font whose size matches the best what it needs.

> regardless of their (buggy?) definition (buggy when in
> a non-proportional font)...

They are not buggy.  The font designer(s) decided which size to use
for each glyph.  It's a profession and a discipline.

> PS- This problem may occur, maybe, because of automatic font replacement
> for characters not found in my default font (Consolas)?

Yes, and I'm guessing this is what happened in your case.  You can see
which font each character came from by using "C-u C-x =".





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-16 15:51       ` Eli Zaretskii
@ 2015-02-16 20:10         ` Stefan Monnier
  2015-02-17  4:17         ` Lars Ingebrigtsen
  1 sibling, 0 replies; 15+ messages in thread
From: Stefan Monnier @ 2015-02-16 20:10 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Sebastien Vauban, 19872

> No, it doesn't.  (Is that at all possible?  I'm not expert on fonts.)

Yes, it's definitely possible by scaling the font appropriately (and
not necessarily by the same amount in X as in Y).  To do that we'd first
have to introduce some kind of annotation/variable to indicate that the
text is intended to be fixed width.


        Stefan





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-16 15:51       ` Eli Zaretskii
  2015-02-16 20:10         ` Stefan Monnier
@ 2015-02-17  4:17         ` Lars Ingebrigtsen
  2015-02-17 15:44           ` Eli Zaretskii
  2015-02-18  3:49           ` Stefan Monnier
  1 sibling, 2 replies; 15+ messages in thread
From: Lars Ingebrigtsen @ 2015-02-17  4:17 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: Sebastien Vauban, 19872

>> > Only if Gnus will align text using the :align-to display property,
>> > instead of inserting whitespace characters.
>> 
>> So, I take it for granted that it doesn't use it yet.

Using align-to in the summary buffer wouldn't help much directly,
because Gnus also has to limit the width of the strings.  That can be
done with vertical-motion and stuff, but would make generating the
buffer slow.

If Emacs had a version of align-to that really aligned to the specified
pixel position, even if the text displayed there is wider than that
position, then that would be nice.

(That is, it would have to truncate the text if it's too wide.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-17  4:17         ` Lars Ingebrigtsen
@ 2015-02-17 15:44           ` Eli Zaretskii
  2015-02-18  0:43             ` Lars Ingebrigtsen
  2015-02-18  3:49           ` Stefan Monnier
  1 sibling, 1 reply; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-17 15:44 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: sva-news, 19872

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Sebastien Vauban <sva-news@mygooglest.com>, 19872@debbugs.gnu.org
> Date: Tue, 17 Feb 2015 15:17:28 +1100
> 
> Using align-to in the summary buffer wouldn't help much directly,
> because Gnus also has to limit the width of the strings.

No, you just need to place the display property on the first character
that exceeds the width limit.

> That can be done with vertical-motion and stuff, but would make
> generating the buffer slow.

I don't see why would you need to do all that.  First, you already do
these calculations, to know how many blanks to insert, right?  So you
already know whether a string is too long, at least in terms of
characters, right?  And :align-to can work in character units as well
as in pixels.

And second, AFAIU you are talking about an additional feature.  The OP
presented a use case where no string is too long, AFAICT.  So it would
get you bonus points to handle long strings as well, but that's not
what this bug report is about: the same problem exists with the
current "alignment" using whitespace, right?

> If Emacs had a version of align-to that really aligned to the specified
> pixel position, even if the text displayed there is wider than that
> position, then that would be nice.
> 
> (That is, it would have to truncate the text if it's too wide.)

The things people expect the display engine to do...

I think it would be wrong for the display engine to automatically
truncate text in this case, since there's also the use case where the
alignment is only meant to be in effect when preceding text is not
long enough, similar to minimum width specification in formatted
output.





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-17 15:44           ` Eli Zaretskii
@ 2015-02-18  0:43             ` Lars Ingebrigtsen
  2015-02-18  3:43               ` Eli Zaretskii
  0 siblings, 1 reply; 15+ messages in thread
From: Lars Ingebrigtsen @ 2015-02-18  0:43 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: sva-news, 19872

Eli Zaretskii <eliz@gnu.org> writes:

> I don't see why would you need to do all that.  First, you already do
> these calculations, to know how many blanks to insert, right?  So you
> already know whether a string is too long, at least in terms of
> characters, right?  And :align-to can work in character units as well
> as in pixels.

Well, the problem here is that some fonts are wider than others.  If
Gnus says "this should be 20 characters wide", then if some of the
glyphs are wider than the normal 20 characters, then things won't line
up any more.

You could reserve more space for these instances, but that would leave
too much white space normally.

> And second, AFAIU you are talking about an additional feature.  The OP
> presented a use case where no string is too long, AFAICT.  So it would
> get you bonus points to handle long strings as well, but that's not
> what this bug report is about: the same problem exists with the
> current "alignment" using whitespace, right?

Gnus truncates the strings if they're too long and inserts spaces if
they're too short.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-18  0:43             ` Lars Ingebrigtsen
@ 2015-02-18  3:43               ` Eli Zaretskii
  2015-02-19  5:49                 ` Lars Ingebrigtsen
  0 siblings, 1 reply; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-18  3:43 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: sva-news, 19872

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: sva-news@mygooglest.com,  19872@debbugs.gnu.org
> Date: Wed, 18 Feb 2015 11:43:26 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > I don't see why would you need to do all that.  First, you already do
> > these calculations, to know how many blanks to insert, right?  So you
> > already know whether a string is too long, at least in terms of
> > characters, right?  And :align-to can work in character units as well
> > as in pixels.
> 
> Well, the problem here is that some fonts are wider than others.  If
> Gnus says "this should be 20 characters wide", then if some of the
> glyphs are wider than the normal 20 characters, then things won't line
> up any more.

AFAIR, :align-to works in units of canonical character width, so this
problem does not exist.

> > And second, AFAIU you are talking about an additional feature.  The OP
> > presented a use case where no string is too long, AFAICT.  So it would
> > get you bonus points to handle long strings as well, but that's not
> > what this bug report is about: the same problem exists with the
> > current "alignment" using whitespace, right?
> 
> Gnus truncates the strings if they're too long and inserts spaces if
> they're too short.

Then I think you already have everything in place, just replace
insertion of blanks with a :align-to display property.





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-17  4:17         ` Lars Ingebrigtsen
  2015-02-17 15:44           ` Eli Zaretskii
@ 2015-02-18  3:49           ` Stefan Monnier
  1 sibling, 0 replies; 15+ messages in thread
From: Stefan Monnier @ 2015-02-18  3:49 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: Sebastien Vauban, 19872

> If Emacs had a version of align-to that really aligned to the specified
> pixel position, even if the text displayed there is wider than that
> position, then that would be nice.

Indeed, I already mentioned this need for the redisplay to provide
a feature that makes it truncate data for columns (as in
tabulated-list-mode).


        Stefan





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-18  3:43               ` Eli Zaretskii
@ 2015-02-19  5:49                 ` Lars Ingebrigtsen
  2015-02-19  6:30                   ` Eli Zaretskii
  0 siblings, 1 reply; 15+ messages in thread
From: Lars Ingebrigtsen @ 2015-02-19  5:49 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: sva-news, 19872

Eli Zaretskii <eliz@gnu.org> writes:

> AFAIR, :align-to works in units of canonical character width, so this
> problem does not exist.

I don't know what you mean by "canonical character width".

If we've reserved space for 20 default-width characters, and we have a
strings like

  [12345678901234567890] Foo
  [12345678901234567890] Foo
  [12345678901234567890] Foo
  [廣東話廣東話廣東話廣東話廣東話廣東話廣東] Foo
  [12345678901234567890] Foo
  [12345678901234567890] Foo
  [12345678901234567890] Foo
  [12345678901234567890] Foo

no amount of align-to will make these columns line up.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-19  5:49                 ` Lars Ingebrigtsen
@ 2015-02-19  6:30                   ` Eli Zaretskii
  2015-02-19 13:49                     ` Stefan Monnier
  0 siblings, 1 reply; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-19  6:30 UTC (permalink / raw)
  To: Lars Ingebrigtsen; +Cc: sva-news, 19872

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: sva-news@mygooglest.com,  19872@debbugs.gnu.org
> Date: Thu, 19 Feb 2015 16:49:53 +1100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > AFAIR, :align-to works in units of canonical character width, so this
> > problem does not exist.
> 
> I don't know what you mean by "canonical character width".

It's the average width of the current frame's 'default' face's font.
IOW, the value returned by 'frame-char-width'.  Display-related
functions usually count "columns" in these units.

> If we've reserved space for 20 default-width characters, and we have a
> strings like
> 
>   [12345678901234567890] Foo
>   [12345678901234567890] Foo
>   [12345678901234567890] Foo
>   [廣東話廣東話廣東話廣東話廣東話廣東話廣東] Foo
>   [12345678901234567890] Foo
>   [12345678901234567890] Foo
>   [12345678901234567890] Foo
>   [12345678901234567890] Foo
> 
> no amount of align-to will make these columns line up.

That's a separate problem.  I thought you said that you were
truncating too long strings, so I thought these cases were already
taken care of.

Do you use something like string-width or char-width to measure the
width of strings on display while accounting for wide characters like
the ones above and for zero-width combining characters?  E.g., in this
case string-width says that the string of Kanji characters is
40-column wide, even though it consists of only 20 characters.  And
for a string such as "ẛ̣", string-width returns 1, even though there
are 3 characters there: u+017f, u+0323, and u+0307, because Emacs
composes them on display into a single glyph (a.k.a. "grapheme
cluster").  Since these strings typically use different fonts, the
results are only approximately correct, but they are a much better
approximation than if you count each character as 1 column on display.





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-19  6:30                   ` Eli Zaretskii
@ 2015-02-19 13:49                     ` Stefan Monnier
  2015-02-19 14:03                       ` Eli Zaretskii
  0 siblings, 1 reply; 15+ messages in thread
From: Stefan Monnier @ 2015-02-19 13:49 UTC (permalink / raw)
  To: Eli Zaretskii; +Cc: sva-news, Lars Ingebrigtsen, 19872

>> [12345678901234567890] Foo
>> [12345678901234567890] Foo
>> [12345678901234567890] Foo
>> [廣東話廣東話廣東話廣東話廣東話廣東話廣東] Foo
>> [12345678901234567890] Foo
>> [12345678901234567890] Foo
>> [12345678901234567890] Foo
>> [12345678901234567890] Foo
>> 
>> no amount of align-to will make these columns line up.

> That's a separate problem.  I thought you said that you were
> truncating too long strings, so I thought these cases were already
> taken care of.

The problem is that truncating based on display width is difficult and
costly if we want to handle proportional fonts.

That's why Lars said:

   Using align-to in the summary buffer wouldn't help much directly,
   because Gnus also has to limit the width of the strings.  That can be
   done with vertical-motion and stuff, but would make generating the
   buffer slow.


-- Stefan





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-19 13:49                     ` Stefan Monnier
@ 2015-02-19 14:03                       ` Eli Zaretskii
  0 siblings, 0 replies; 15+ messages in thread
From: Eli Zaretskii @ 2015-02-19 14:03 UTC (permalink / raw)
  To: Stefan Monnier; +Cc: sva-news, larsi, 19872

> From: Stefan Monnier <monnier@IRO.UMontreal.CA>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>, sva-news@mygooglest.com,
>         19872@debbugs.gnu.org
> Date: Thu, 19 Feb 2015 08:49:32 -0500
> 
> >> [12345678901234567890] Foo
> >> [12345678901234567890] Foo
> >> [12345678901234567890] Foo
> >> [廣東話廣東話廣東話廣東話廣東話廣東話廣東] Foo
> >> [12345678901234567890] Foo
> >> [12345678901234567890] Foo
> >> [12345678901234567890] Foo
> >> [12345678901234567890] Foo
> >> 
> >> no amount of align-to will make these columns line up.
> 
> > That's a separate problem.  I thought you said that you were
> > truncating too long strings, so I thought these cases were already
> > taken care of.
> 
> The problem is that truncating based on display width is difficult and
> costly if we want to handle proportional fonts.

I agree in general, but cannot see why this particular case couldn't
be resolved reasonably well, e.g. by adding one more column "just in
case", before the :align-to property.





^ permalink raw reply	[flat|nested] 15+ messages in thread

* bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers)
  2015-02-15  9:05 bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers) Sebastien Vauban
  2015-02-15 16:27 ` Eli Zaretskii
       [not found] ` <mailman.138.1424017693.31049.bug-gnu-emacs@gnu.org>
@ 2016-02-07  6:04 ` Lars Ingebrigtsen
  2 siblings, 0 replies; 15+ messages in thread
From: Lars Ingebrigtsen @ 2016-02-07  6:04 UTC (permalink / raw)
  To: Sebastien Vauban; +Cc: 19872

Emacs now has functions to determine the pixel width of text, so Gnus
could use those to limit the lengths, and use :align-to to do the
padding.

So I think this would finally be possible to fix.  It might make
generating large summary buffers a bit slower, though.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no






^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2016-02-07  6:04 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-02-15  9:05 bug#19872: 24.4; UTF8 characters of unusual width (Gnus markers) Sebastien Vauban
2015-02-15 16:27 ` Eli Zaretskii
     [not found] ` <mailman.138.1424017693.31049.bug-gnu-emacs@gnu.org>
     [not found]   ` <mailman.138.1424017693.31049.bug-gnu-emacs-mXXj517/zsQ@public.gmane.org>
2015-02-16  9:45     ` Sebastien Vauban
2015-02-16 15:51       ` Eli Zaretskii
2015-02-16 20:10         ` Stefan Monnier
2015-02-17  4:17         ` Lars Ingebrigtsen
2015-02-17 15:44           ` Eli Zaretskii
2015-02-18  0:43             ` Lars Ingebrigtsen
2015-02-18  3:43               ` Eli Zaretskii
2015-02-19  5:49                 ` Lars Ingebrigtsen
2015-02-19  6:30                   ` Eli Zaretskii
2015-02-19 13:49                     ` Stefan Monnier
2015-02-19 14:03                       ` Eli Zaretskii
2015-02-18  3:49           ` Stefan Monnier
2016-02-07  6:04 ` Lars Ingebrigtsen

Code repositories for project(s) associated with this public inbox

	https://git.savannah.gnu.org/cgit/emacs.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).