Re: Unicode confusables and reordering characters considered harmful, a simple solution

[Daniel Brooks <db48x@db48x.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> From: Yuri Khan <yuri.v.khan@gmail.com>
>> Date: Thu, 4 Nov 2021 01:45:17 +0700
>> Cc: Daniel Brooks <db48x@db48x.net>, Clément Pit-Claudel <cpitclaudel@gmail.com>, 
>> 	Stefan Kangas <stefan@marxist.se>, Stefan Monnier <monnier@iro.umontreal.ca>, 
>> 	Emacs developers <emacs-devel@gnu.org>
>> 
>> On Thu, 4 Nov 2021 at 00:56, Eli Zaretskii <eliz@gnu.org> wrote:
>> 
>> > The problem with these remappings is that you then get to somehow
>> > discern between the remapped characters and the real characters which
>> > look identically on display.
>> 
>> Real characters are fontified as whichever syntax unit they belong to.
>> Remapped characters are fontified as whitespace-space-face or
>> whitespace-hspace-face depending on whether you add them to
>> whitespace-space-regexp or whitespace-hspace-regexp.
>
> I just used what Daniel posted, and that doesn't display the remapped
> characters in any distinct face.  Gotta tinker?

Yea, I intend to tinker in order to add a new category that has it’s own
face and can be toggled on and off separately and so on. I haven’t
actually started yet though.

> Do you read Hebrew?  Those characters look like line noise there,
> whereas the text with the default display is perfectly readable, and
> most people won't even know these controls are there (as intended).

My suggestion is to only enable it by default in _programming modes_. It
should remain disabled in ordinary prose like a TUTORIAL file.

> What for?  The absolute majority of people won't have any idea what is
> the effect of each of these controls, and how it differs from others.
> Even I many times need to talk myself through their effect on display.
> The UBA spec weighs in at more than 30 pages of highly technical text,
> and I don't expect people to memorize it by heart.

I totally agree, but I think that this is not very relevant. The whole
point is for a programmer who is unaware of BiDi in general to go “WTF‽”
when these characters show up in a source file one day, so that they can
have something to ask questions about.

`what-cursor-position' will show the face, once a face is available, and
it also shows the name of the character. Both are good ways for the user
to find more information, and in principle we could have it show other
information as well. We could pull a description from the Unicode
database perhaps, or just add extra help messages for individual
characters. Now that I think about it, maybe we should just show the
docstring for the face right there next to the name. That would save me
a step from time to time, if nothing else.

db48x