scratch/emoji vs emacs that maybe canʼt display emojis

scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

I have a terminal emulator that can display emojis correctly.

C-x 8 RET 1f606

works fine.

(this is on GNU/Linux)

emacs -Q -nw
 C-x 8 e i ;; emoji-insert
 s	   ;; emoji--command-Emoji > Smileys
 s	   ;; emoji--command-Emoji > Smileys > smiling

Without '-nw', this shows large smileys, but with -nw it shows
nothing, and I then appear to be stuck in the transient until I
C-g. Hitting 'e' gives me:

Unbound suffix: ‘e’ (Use ‘C-g’ to abort, ‘?’ for help) [self-insert-command]

On macos, in gui frames, I see similar issues until I cherry-pick
e3171e7e86, which says to me thereʼs an issue with the 'can I display
this codepoint' code.

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Date: Fri, 05 Nov 2021 09:22:53 +0100
> 
> I have a terminal emulator that can display emojis correctly.

Which one?

Does it behave correctly wrt the display width of the displayed Emoji?
That is, does string-width agree with the number of columns the
displayed Emoji takes on the screen?

> On macos, in gui frames, I see similar issues until I cherry-pick
> e3171e7e86, which says to me thereʼs an issue with the 'can I display
> this codepoint' code.

Indeed, how can we know that a particular TTY frame can display Emoji?
For a single character, we have a more-or-less reliable method, but
for a composable sequence?

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Fri, 05 Nov 2021 13:57:08 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Date: Fri, 05 Nov 2021 09:22:53 +0100
    >> 
    >> I have a terminal emulator that can display emojis correctly.

    Eli> Which one?

iterm2

    Eli> Does it behave correctly wrt the display width of the displayed Emoji?
    Eli> That is, does string-width agree with the number of columns the
    Eli> displayed Emoji takes on the screen?

No, string-width gives 2, it actually takes up 1 column, which is
unexpected for a non-composed character.

    >> On macos, in gui frames, I see similar issues until I cherry-pick
    >> e3171e7e86, which says to me thereʼs an issue with the 'can I display
    >> this codepoint' code.

    Eli> Indeed, how can we know that a particular TTY frame can display Emoji?
    Eli> For a single character, we have a more-or-less reliable method, but
    Eli> for a composable sequence?

Well, this is a single character Iʼm trying to enter. And even if
Emacs canʼt display the codepoints, it shouldn't break the transient.

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Robert Pluim <rpluim@gmail.com>
> Cc: emacs-devel@gnu.org
> Date: Fri, 05 Nov 2021 14:38:41 +0100
> 
>     Eli> Does it behave correctly wrt the display width of the displayed Emoji?
>     Eli> That is, does string-width agree with the number of columns the
>     Eli> displayed Emoji takes on the screen?
> 
> No, string-width gives 2, it actually takes up 1 column, which is
> unexpected for a non-composed character.

This means you will have a messed-up display.

>     >> On macos, in gui frames, I see similar issues until I cherry-pick
>     >> e3171e7e86, which says to me thereʼs an issue with the 'can I display
>     >> this codepoint' code.
> 
>     Eli> Indeed, how can we know that a particular TTY frame can display Emoji?
>     Eli> For a single character, we have a more-or-less reliable method, but
>     Eli> for a composable sequence?
> 
> Well, this is a single character Iʼm trying to enter. And even if
> Emacs canʼt display the codepoints, it shouldn't break the transient.

With a single character it's supposed to work.  Where exactly does it
break? can you show the code?

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Fri, 05 Nov 2021 16:25:16 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Robert Pluim <rpluim@gmail.com>
    >> Cc: emacs-devel@gnu.org
    >> Date: Fri, 05 Nov 2021 14:38:41 +0100
    >> 
    Eli> Does it behave correctly wrt the display width of the displayed Emoji?
    Eli> That is, does string-width agree with the number of columns the
    Eli> displayed Emoji takes on the screen?
    >> 
    >> No, string-width gives 2, it actually takes up 1 column, which is
    >> unexpected for a non-composed character.

    Eli> This means you will have a messed-up display.

Eventually, yes. I can live with that for this specific case.

    >> >> On macos, in gui frames, I see similar issues until I cherry-pick
    >> >> e3171e7e86, which says to me thereʼs an issue with the 'can I display
    >> >> this codepoint' code.
    >> 
    Eli> Indeed, how can we know that a particular TTY frame can display Emoji?
    Eli> For a single character, we have a more-or-less reliable method, but
    Eli> for a composable sequence?
    >> 
    >> Well, this is a single character Iʼm trying to enter. And even if
    >> Emacs canʼt display the codepoints, it shouldn't break the transient.

    Eli> With a single character it's supposed to work.  Where exactly does it
    Eli> break? can you show the code?

I know nothing about transient. I suspect that when I get to the third
level of the transient, because the entries are all single codepoints
or grapheme clusters the generated transient is empty, since none of
them can be displayed (unlike the first 2 levels, which contain
categories of emojis).

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Benjamin Riefenstahl]

Eli Zaretskii writes:
> Does it behave correctly wrt the display width of the displayed Emoji?
> That is, does string-width agree with the number of columns the
> displayed Emoji takes on the screen?

Works fine here with Gnome-Terminal.  Font is set to "Monospace", which
is an alias for "DejaVu Sans Mono", if fc-match is to be believed.

benny

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> On macos, in gui frames, I see similar issues until I cherry-pick
> e3171e7e86, which says to me thereʼs an issue with the 'can I display
> this codepoint' code.

I think it's perhaps best to wait until this has landed on the trunk to
debug it...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Fri, 05 Nov 2021 15:30:18 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> On macos, in gui frames, I see similar issues until I cherry-pick
    >> e3171e7e86, which says to me thereʼs an issue with the 'can I display
    >> this codepoint' code.

    Lars> I think it's perhaps best to wait until this has landed on the trunk to
    Lars> debug it...

Not emacs-28? I guess itʼs a feature, so itʼs too late for that.

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Not emacs-28? I guess itʼs a feature, so itʼs too late for that.

It'll be on ELPA (as a built-in package from the trunk), so it can be
used in emacs-28, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 374 bytes --]

Robert Pluim <rpluim@gmail.com> writes:

>     Lars> I think it's perhaps best to wait until this has landed on
>     Lars> the trunk to
>     Lars> debug it...

Since it's now on the trunk, I've now had a look.  There seems to be
problems in general with emojis in Emacs on the terminal.

1) The string widths seem to be off (1 instead of 2), as Robert said
earlier.  So:


[-- Attachment #2: Type: image/png, Size: 122173 bytes --]

[-- Attachment #3: Type: text/plain, Size: 128 bytes --]


The width calculations are totally off.

2) We don't know whether a grapheme cluster displays or not.  So we get
things like:


[-- Attachment #4: Type: image/png, Size: 14188 bytes --]

[-- Attachment #5: Type: text/plain, Size: 127 bytes --]


for the ones that don't combine in the terminal.  But some do combine in
the terminal, but then Emacs doesn't know that, so:


[-- Attachment #6: Type: image/png, Size: 908 bytes --]

[-- Attachment #7: Type: text/plain, Size: 371 bytes --]


The cursor is in "the middle" of the cluster, covering the last half.

I'm surprised that emojis work as well as they do on the terminal.  🙀
But it seems that there's some work to be done on the display side if we
want this to be more...  consistent...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> 1) The string widths seem to be off (1 instead of 2), as Robert said
> earlier.  So:

Not for all of them.  Normal single-char glyphs are correct:

(string-width "😀")
=> 2

But grapheme clusters are wrong:

(string-width "☺️")
=> 1

Presumably because it doesn't know that it's a cluster?  Because it
computes the width of just the first code point without the variation
selector:

(string-width "☺")
=> 1

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 07 Nov 2021 17:12:07 +0100
> Cc: emacs-devel@gnu.org
> 
> Lars Ingebrigtsen <larsi@gnus.org> writes:
> 
> > 1) The string widths seem to be off (1 instead of 2), as Robert said
> > earlier.  So:
> 
> Not for all of them.  Normal single-char glyphs are correct:
> 
> (string-width "😀")
> => 2
> 
> But grapheme clusters are wrong:
> 
> (string-width "☺️")
> => 1
> 
> Presumably because it doesn't know that it's a cluster?

No, because Emacs has no way of knowing how wide is the cluster
produced by the terminal.  string-width with single characters
accesses our own data (in char-width-table), so we are okay.  But for
compositions on TTY frames we simply assume the result takes one
column, because what else can we do?

This problem has no solution, especially since different terminals
produce results of different sizes.  We could perhaps introduce some
heuristics based on the first character of the composition, for
example that the result of composing CH cannot be narrower than CH
itself.  But that will only work up to a point.  E.g., AFAIR some
Emoji has the width of 1.

And then there are terminal emulators that do some preposterous stuff,
assuming that their tail can wag the dog.  See PROBLEMS and search for
"kitty", for some fun.

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> We could perhaps introduce some heuristics based on the first
> character of the composition, for example that the result of composing
> CH cannot be narrower than CH itself.  But that will only work up to a
> point.  E.g., AFAIR some Emoji has the width of 1.

Perhaps it depends on the font, too?  I'm pretty sure that none of the
composed emojis in Noto Color Emoji have a width of 1, but that's
basically the only emoji font I've been futzing around with.

> And then there are terminal emulators that do some preposterous stuff,
> assuming that their tail can wag the dog.  See PROBLEMS and search for
> "kitty", for some fun.

Heh.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Stefan Monnier]

Lars Ingebrigtsen [2021-11-07 21:48:50] wrote:
> Eli Zaretskii <eliz@gnu.org> writes:
>> We could perhaps introduce some heuristics based on the first
>> character of the composition, for example that the result of composing
>> CH cannot be narrower than CH itself.  But that will only work up to a
>> point.  E.g., AFAIR some Emoji has the width of 1.
> Perhaps it depends on the font, too?  I'm pretty sure that none of the
> composed emojis in Noto Color Emoji have a width of 1, but that's
> basically the only emoji font I've been futzing around with.
>> And then there are terminal emulators that do some preposterous stuff,
>> assuming that their tail can wag the dog.  See PROBLEMS and search for
>> "kitty", for some fun.
> Heh.

How 'bout we add a simple little hack to Emacs: when running in a tty
and we need to display something like an emoji, use a local DNN (deep
neural network) to guess the width that will be used for the char, then
contact an fsf.org service that connects to a nearby camera, grabs a few
frames, passes them to text recognition DNN to estimate the number of
text columns that the terminal used when displaying this char, send the
result back to Emacs and if the initial guess was wrong adjust the
local DNN and redisplay.

WDYT?


        Stefan


PS: OK, we could replace the fsf.org service + remote DNN with say, the
user's brain, but what would be the fun in that?

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> then contact an fsf.org service that connects to a nearby camera

I like the plan, but surely there must be a way to also involve a
service from Google here somehow, so that we can get special
personalised ad emojis, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I like the plan, but surely there must be a way to also involve a
  > service from Google here somehow, so that we can get special
  > personalised ad emojis, too.

It's not acceptable to make Emacs depend on a network service in order
to do its job.  It must be able to work on a computer with no net
connention -- not only so that it works for you when you don't have a
net connection, but also so that users have full control over its
behavior.

In the specific case of Google, its dis-services are likely to have
unjust requirements in their terms and conditions.  If so, it would
be wrong for us to suggest that users accept those conditions.
We will not do that.

The only acceptable way to do this is to write or find free code to do
the job, and include that in or with Emacs.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> It's not acceptable to make Emacs depend on a network service in order
> to do its job. 

It was a joke, following up on other jokes.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > It's not acceptable to make Emacs depend on a network service in order
  > > to do its job. 

  > It was a joke, following up on other jokes.

That's a relief.  Alas, it wasn't labeled as a joke.  How about using
suitable smileys to indicate when you're joking?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> That's a relief.  Alas, it wasn't labeled as a joke.  How about using
> suitable smileys to indicate when you're joking?

What about this one: 🤐

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > What about this one: 🤐

That displays as a diamond on my Linux tty.
I did C-u C-x = now, and it said ZIPPER-MOUTH FACE.
I can guess what it is supposed to look like,
but I would not think of it as meaning "This is a joke."
;-) or ;-} are clear ways of saying that.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[David Masterson]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>> That's a relief.  Alas, it wasn't labeled as a joke.  How about using
>> suitable smileys to indicate when you're joking?
>
> What about this one: 🤐

Doesn't show as an emoji on my Emacs ("GNU Emacs 27.1 (build 1,
x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2020-12-14").
What's needed?

-- 
David Masterson

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

David Masterson <dsmasterson92630@outlook.com> writes:

> Doesn't show as an emoji on my Emacs ("GNU Emacs 27.1 (build 1,
> x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2020-12-14").
> What's needed?

An emoji font (like font-noto-color-emoji) and Emacs 28.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[David Masterson]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> David Masterson <dsmasterson92630@outlook.com> writes:
>
>> Doesn't show as an emoji on my Emacs ("GNU Emacs 27.1 (build 1,
>> x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll bars) of 2020-12-14").
>> What's needed?
>
> An emoji font (like font-noto-color-emoji) and Emacs 28.

Thanks.  I'll look at it.

-- 
David Masterson

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Po Lu]

Richard Stallman <rms@gnu.org> writes:

>   > > It's not acceptable to make Emacs depend on a network service in order
>   > > to do its job. 
>
>   > It was a joke, following up on other jokes.
>
> That's a relief.  Alas, it wasn't labeled as a joke.  How about using
> suitable smileys to indicate when you're joking?

I found it funny, FWIW.

It seems to be a good candidate for etc/DEVEL.HUMOR.  Would it be OK if
I added it there?

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Po Lu <luangruo@yahoo.com>
> Cc: Lars Ingebrigtsen <larsi@gnus.org>,  eliz@gnu.org,  rpluim@gmail.com,
>  monnier@iro.umontreal.ca,  emacs-devel@gnu.org
> Date: Thu, 11 Nov 2021 21:26:48 +0800
> 
> Richard Stallman <rms@gnu.org> writes:
> 
> >   > > It's not acceptable to make Emacs depend on a network service in order
> >   > > to do its job. 
> >
> >   > It was a joke, following up on other jokes.
> >
> > That's a relief.  Alas, it wasn't labeled as a joke.  How about using
> > suitable smileys to indicate when you're joking?
> 
> I found it funny, FWIW.
> 
> It seems to be a good candidate for etc/DEVEL.HUMOR.  Would it be OK if
> I added it there?

Alas, humor is deprecated these days.  Ever wondered why DEVEL.HUMOR
is so short?

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Stefan Monnier]

> Alas, humor is deprecated these days.  Ever wondered why DEVEL.HUMOR
> is so short?

Should we move it to `obsolete`?


        Stefan

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: Po Lu <luangruo@yahoo.com>,  rms@gnu.org,  larsi@gnus.org,
>   rpluim@gmail.com,  emacs-devel@gnu.org
> Date: Thu, 11 Nov 2021 14:22:06 -0500
> 
> > Alas, humor is deprecated these days.  Ever wondered why DEVEL.HUMOR
> > is so short?
> 
> Should we move it to `obsolete`?

Yes, good idea.  But better leave behind an obsolete-alias (like an
empty file which just says the previous content is obsolete).

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Po Lu]

Eli Zaretskii <eliz@gnu.org> writes:

> Yes, good idea.  But better leave behind an obsolete-alias (like an
> empty file which just says the previous content is obsolete).

Are the two of you joking?  I can't tell, FWIW.

But if you are, it would be yet another entry for DEVEL.HUMOR ;)

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Tim Cross]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Lars Ingebrigtsen [2021-11-07 21:48:50] wrote:
>> Eli Zaretskii <eliz@gnu.org> writes:
>>> We could perhaps introduce some heuristics based on the first
>>> character of the composition, for example that the result of composing
>>> CH cannot be narrower than CH itself.  But that will only work up to a
>>> point.  E.g., AFAIR some Emoji has the width of 1.
>> Perhaps it depends on the font, too?  I'm pretty sure that none of the
>> composed emojis in Noto Color Emoji have a width of 1, but that's
>> basically the only emoji font I've been futzing around with.
>>> And then there are terminal emulators that do some preposterous stuff,
>>> assuming that their tail can wag the dog.  See PROBLEMS and search for
>>> "kitty", for some fun.
>> Heh.
>
> How 'bout we add a simple little hack to Emacs: when running in a tty
> and we need to display something like an emoji, use a local DNN (deep
> neural network) to guess the width that will be used for the char, then
> contact an fsf.org service that connects to a nearby camera, grabs a few
> frames, passes them to text recognition DNN to estimate the number of
> text columns that the terminal used when displaying this char, send the
> result back to Emacs and if the initial guess was wrong adjust the
> local DNN and redisplay.
>
> WDYT?
>
Brilliant. However, I think you also need to wrap that communication
with the fsf.org service in a blockchain to secure the captured frames.
We may also be able to improve processing of the frame images using a
systolic array and uniform recurrence equations (URE)- I suspect this could
be slightly faster than the DNN. If the UREs are extended with
additional terms, we could even have an N-dimensional systolic array,
which could process each dimension on a separate core in parallel to
squeeze even more out of the system. 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Benjamin Riefenstahl]

I note that the display in my Gnome-Terminal follows the output of
string-width as you show it.  So this terminal emulator and Emacs at
least are in agreement ;-).

Lars Ingebrigtsen writes:
> Not for all of them.  Normal single-char glyphs are correct:
>
> (string-width "😀")
> => 2

This is shown correctly, the emoji occupies two columns.

> But grapheme clusters are wrong:
>
> (string-width "☺️")
> => 1

Here the display of the emoji also occupies two columns, IOW the glyph
is correctly display by the font.  But the closing quote overlaps the
emoji, probably because the terminal thinks it is just one column wide.

> Presumably because it doesn't know that it's a cluster?  Because it
> computes the width of just the first code point without the variation
> selector:
>
> (string-width "☺")
> => 1

Here the emoji has only one column, and this is shown correctly.


Another note: In my own work with this issue, using NCurses, I have
noticed that NCurses mostly relies on the C library's wcwidth function.
Which seems reasonable for NCurses.  I am not saying that Emacs should
do the same, but I expect NCurses and terminal emulators to develop in a
certain dependency, because it seem that NCurses is the most-used
full-screen terminal library these days.

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Benjamin Riefenstahl <b.riefenstahl@turtle-trading.net>
> Date: Sun, 07 Nov 2021 17:34:53 +0100
> 
> Another note: In my own work with this issue, using NCurses, I have
> noticed that NCurses mostly relies on the C library's wcwidth function.
> Which seems reasonable for NCurses.  I am not saying that Emacs should
> do the same, but I expect NCurses and terminal emulators to develop in a
> certain dependency, because it seem that NCurses is the most-used
> full-screen terminal library these days.

That already happens, to a degree.  Both wcwidth and char-width-table
in Emacs are supposed to be based on the same data whose source is the
Unicode Character Database (UCD).  Of course, whatever libc you are
using and Emacs are not necessarily in sync when a new version of
Unicode comes out.  But other than that, the data should be the same.

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 07 Nov 2021 17:05:35 +0100
> Cc: emacs-devel@gnu.org
> 
> I'm surprised that emojis work as well as they do on the terminal.  🙀
> But it seems that there's some work to be done on the display side if we
> want this to be more...  consistent...

I'm not sure how to make it better.  For starters, each Unicode
version adds gobs of new Emoji sequences, and the terminal emulators
need to keep up.  They don't always, or not as fast as we do.

People who want a good display of Emoji shouldn't do that on TTY
frames, that's all I can say.  With some TTYs, just disabling
auto-composition-mode is enough, or almost so.  Maybe we should have a
special minor mode that disables only compositions of Emoji.

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Sun, 07 Nov 2021 18:25:14 +0200, Eli Zaretskii <eliz@gnu.org> said:

    >> From: Lars Ingebrigtsen <larsi@gnus.org>
    >> Date: Sun, 07 Nov 2021 17:05:35 +0100
    >> Cc: emacs-devel@gnu.org
    >> 
    >> I'm surprised that emojis work as well as they do on the terminal.  🙀
    >> But it seems that there's some work to be done on the display side if we
    >> want this to be more...  consistent...

    Eli> I'm not sure how to make it better.  For starters, each Unicode
    Eli> version adds gobs of new Emoji sequences, and the terminal emulators
    Eli> need to keep up.  They don't always, or not as fast as we do.

    Eli> People who want a good display of Emoji shouldn't do that on TTY
    Eli> frames, that's all I can say.  With some TTYs, just disabling
    Eli> auto-composition-mode is enough, or almost so.  Maybe we should have a
    Eli> special minor mode that disables only compositions of Emoji.

Iʼm not necessarily looking for good display of emoji's on a TTY, but
Iʼd like the entering of them to work correctly (and it currently
doesnʼt, as I said in the very first message of this thread :-) )

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Iʼm not necessarily looking for good display of emoji's on a TTY, but
> Iʼd like the entering of them to work correctly (and it currently
> doesnʼt, as I said in the very first message of this thread :-) )

Have you tried updating?  🤔

Anyway, this reminds me of a question I had but forgot to ask -- is
there a way to query terminals about these things?  I know that we can
query them about other things, but have they perhaps grown a way to
query them about grapheme clusters?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Mon, 08 Nov 2021 11:33:26 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Iʼm not necessarily looking for good display of emoji's on a TTY, but
    >> Iʼd like the entering of them to work correctly (and it currently
    >> doesnʼt, as I said in the very first message of this thread :-) )

    Lars> Have you tried updating?  🤔

I have now. Much better, thanks.

Of course, now I have a feature request: In 'C-x 8 e l', I can hit 'h'
to get the description of the emoji under point. Is there a way to do
that when entering emoji via 'C-x 8 e i'? (I know I can use 'C-x 8 e
d' after the fact, but Iʼm lazy, and if I choose the wrong emoji I
have to re-traverse the transient)

Hmm, '? a' when Iʼm at a leaf in the transient will give me the
documentation of the generated transient function. Perhaps you could
add the emoji description to the docstring?

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Lars Ingebrigtsen]

Robert Pluim <rpluim@gmail.com> writes:

> Of course, now I have a feature request: In 'C-x 8 e l', I can hit 'h'
> to get the description of the emoji under point. Is there a way to do
> that when entering emoji via 'C-x 8 e i'? (I know I can use 'C-x 8 e
> d' after the fact, but Iʼm lazy, and if I choose the wrong emoji I
> have to re-traverse the transient)

We'd discussed activating the cursor movement thing in these
transients -- in that case, adding a help command when point is over an
emoji would be natural.

> Hmm, '? a' when Iʼm at a leaf in the transient will give me the
> documentation of the generated transient function. Perhaps you could
> add the emoji description to the docstring?

I'd rather not put the transients in the obarray at all.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Robert Pluim]

>>>>> On Tue, 09 Nov 2021 04:31:19 +0100, Lars Ingebrigtsen <larsi@gnus.org> said:

    Lars> Robert Pluim <rpluim@gmail.com> writes:
    >> Of course, now I have a feature request: In 'C-x 8 e l', I can hit 'h'
    >> to get the description of the emoji under point. Is there a way to do
    >> that when entering emoji via 'C-x 8 e i'? (I know I can use 'C-x 8 e
    >> d' after the fact, but Iʼm lazy, and if I choose the wrong emoji I
    >> have to re-traverse the transient)

    Lars> We'd discussed activating the cursor movement thing in these
    Lars> transients -- in that case, adding a help command when point is over an
    Lars> emoji would be natural.

Yes, tht would be nice. Or even a dynamic echo-area display (if that
doesnʼt interfere with the transient)

Robert
-- 

Re: scratch/emoji vs emacs that maybe canʼt display emojis

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: Eli Zaretskii <eliz@gnu.org>,  emacs-devel@gnu.org
> Date: Mon, 08 Nov 2021 11:33:26 +0100
> 
> Anyway, this reminds me of a question I had but forgot to ask -- is
> there a way to query terminals about these things?  I know that we can
> query them about other things, but have they perhaps grown a way to
> query them about grapheme clusters?

I don't think such a feature exists.  But the right person to ask is
Thomas Dickey, the developer of xterm and ncurses.