From: Philipp Stephani <p.stephani2@gmail.com>
To: Eli Zaretskii <eliz@gnu.org>
Cc: phst@google.com, alan@idiocy.org, emacs-devel@gnu.org
Subject: Re: [PATCH] Allow inserting non-BMP characters
Date: Fri, 29 Dec 2017 20:14:41 +0000 [thread overview]
Message-ID: <CAArVCkTqGP-=3VQem3LtyTFjsv=cVrPFOiQY91fxSp3AJLBEtw@mail.gmail.com> (raw)
In-Reply-To: <834loahl4j.fsf@gnu.org>
[-- Attachment #1.1: Type: text/plain, Size: 1229 bytes --]
Eli Zaretskii <eliz@gnu.org> schrieb am Do., 28. Dez. 2017 um 17:29 Uhr:
> > From: Philipp Stephani <p.stephani2@gmail.com>
> > Date: Thu, 28 Dec 2017 12:31:39 +0000
> > Cc: phst@google.com, Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> >
> > Alan Third <alan@idiocy.org> schrieb am Do., 28. Dez. 2017 um 12:38 Uhr:
> >
> > On Wed, Dec 27, 2017 at 05:41:23AM +0200, Eli Zaretskii wrote:
> > > What about the possibility that SAFE_NALLOCA could signal an error and
> > > longjmp to top level? Does this code always run in the main thread,
> > > and if so, can it allow such longjmp's?
> >
> > I think it does always run in the main thread, however since it runs
> > within the NSApplication run loop I’ve no idea what would happen if we
> > did a longjmp.
> >
> > We should probably avoid longjmps here. This message is invoked by the
> window manager, which most likely
> > can't deal with longjmps.
>
> Then maybe process the input in chunks using a fixed-size buffer?
>
Maybe, but for now I've reverted to just call characterAtIndex repeatedly.
I guess in 99% of cases we insert a single code unit anyway, and if we plan
to make the code more complex we should benchmark it before.
[-- Attachment #1.2: Type: text/html, Size: 1891 bytes --]
[-- Attachment #2: 0001-Allow-inserting-non-BMP-characters.txt --]
[-- Type: text/plain, Size: 4797 bytes --]
From 383e6fe7c50c1e3a96720284fa5fc2c00610959f Mon Sep 17 00:00:00 2001
From: Philipp Stephani <phst@google.com>
Date: Mon, 25 Dec 2017 22:00:00 +0100
Subject: [PATCH] Allow inserting non-BMP characters
* src/coding.h (char_low_surrogate_p, char_high_surrogate_p)
(surrogates_to_codepoint): New functions.
* src/coding.c (decode_coding_utf_16): Use new inline functions;
remove old macros.
* src/nsterm.m (insertText:): Properly handle surrogate pairs.
---
src/coding.c | 13 +++----------
src/coding.h | 26 ++++++++++++++++++++++++++
src/nsterm.m | 25 +++++++++++++++++++------
3 files changed, 48 insertions(+), 16 deletions(-)
diff --git a/src/coding.c b/src/coding.c
index 1705838ffa..9903d87b92 100644
--- a/src/coding.c
+++ b/src/coding.c
@@ -1515,13 +1515,6 @@ encode_coding_utf_8 (struct coding_system *coding)
/* See the above "GENERAL NOTES on `detect_coding_XXX ()' functions".
Return true if a text is encoded in one of UTF-16 based coding systems. */
-#define UTF_16_HIGH_SURROGATE_P(val) \
- (((val) & 0xFC00) == 0xD800)
-
-#define UTF_16_LOW_SURROGATE_P(val) \
- (((val) & 0xFC00) == 0xDC00)
-
-
static bool
detect_coding_utf_16 (struct coding_system *coding,
struct coding_detection_info *detect_info)
@@ -1686,7 +1679,7 @@ decode_coding_utf_16 (struct coding_system *coding)
if (surrogate)
{
- if (! UTF_16_LOW_SURROGATE_P (c))
+ if (! char_low_surrogate_p (c))
{
if (endian == utf_16_big_endian)
c1 = surrogate >> 8, c2 = surrogate & 0xFF;
@@ -1694,7 +1687,7 @@ decode_coding_utf_16 (struct coding_system *coding)
c1 = surrogate & 0xFF, c2 = surrogate >> 8;
*charbuf++ = c1;
*charbuf++ = c2;
- if (UTF_16_HIGH_SURROGATE_P (c))
+ if (char_high_surrogate_p (c))
CODING_UTF_16_SURROGATE (coding) = surrogate = c;
else
*charbuf++ = c;
@@ -1708,7 +1701,7 @@ decode_coding_utf_16 (struct coding_system *coding)
}
else
{
- if (UTF_16_HIGH_SURROGATE_P (c))
+ if (char_high_surrogate_p (c))
CODING_UTF_16_SURROGATE (coding) = surrogate = c;
else
{
diff --git a/src/coding.h b/src/coding.h
index 66d125b07e..b111ce70c6 100644
--- a/src/coding.h
+++ b/src/coding.h
@@ -662,6 +662,32 @@ struct coding_system
/* Note that this encodes utf-8, not utf-8-emacs, so it's not a no-op. */
#define ENCODE_UTF_8(str) code_convert_string_norecord (str, Qutf_8, true)
+/* Return true if C is a low surrogate. */
+
+INLINE bool
+char_low_surrogate_p (int c)
+{
+ return 0xDC00 <= c && c <= 0xDFFF;
+}
+
+/* Return true if C is a high surrogate. */
+
+INLINE bool
+char_high_surrogate_p (int c)
+{
+ return 0xD800 <= c && c <= 0xDBFF;
+}
+
+/* Return the Unicode code point for the given UTF-16 surrogates. */
+
+INLINE int
+surrogates_to_codepoint (int low, int high)
+{
+ eassert (char_low_surrogate_p (low));
+ eassert (char_high_surrogate_p (high));
+ return 0x10000 + (low - 0xDC00) + ((high - 0xD800) * 0x400);
+}
+
/* Extern declarations. */
extern Lisp_Object code_conversion_save (bool, bool);
extern bool encode_coding_utf_8 (struct coding_system *);
diff --git a/src/nsterm.m b/src/nsterm.m
index 07ac8f978f..0c0b3fd9b5 100644
--- a/src/nsterm.m
+++ b/src/nsterm.m
@@ -6283,14 +6283,13 @@ flag set (this is probably a bug in the OS).
by doCommandBySelector: deleteBackward: */
- (void)insertText: (id)aString
{
- int code;
- int len = [(NSString *)aString length];
- int i;
+ NSString *s = aString;
+ NSUInteger len = [s length];
NSTRACE ("[EmacsView insertText:]");
if (NS_KEYLOG)
- NSLog (@"insertText '%@'\tlen = %d", aString, len);
+ NSLog (@"insertText '%@'\tlen = %lu", aString, (unsigned long) len);
processingCompose = NO;
if (!emacs_event)
@@ -6300,10 +6299,24 @@ - (void)insertText: (id)aString
if (workingText != nil)
[self deleteWorkingText];
+ /* It might be preferable to use getCharacters:range: below,
+ cf. https://developer.apple.com/library/content/documentation/Cocoa/Conceptual/CocoaPerformance/Articles/StringDrawing.html#//apple_ref/doc/uid/TP40001445-112378.
+ However, we probably can't use SAFE_NALLOCA here because it might
+ exit nonlocally. */
+
/* now insert the string as keystrokes */
- for (i =0; i<len; i++)
+ for (NSUInteger i = 0; i < len; i++)
{
- code = [aString characterAtIndex: i];
+ NSUInteger code = [s characterAtIndex:i];
+ if (char_high_surrogate_p (code) && i < len - 1)
+ {
+ unichar low = [s characterAtIndex:i + 1];
+ if (char_low_surrogate_p (low))
+ {
+ code = surrogates_to_codepoint (low, code);
+ ++i;
+ }
+ }
/* TODO: still need this? */
if (code == 0x2DC)
code = '~'; /* 0x7E */
--
2.15.1
next prev parent reply other threads:[~2017-12-29 20:14 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-24 16:00 bug#29837: UTF-16 char display problems and the macOS "character palette" Alan Third
2017-12-24 16:56 ` Eli Zaretskii
2017-12-24 18:23 ` Alan Third
2017-12-24 18:57 ` Eli Zaretskii
2017-12-24 19:28 ` Alan Third
2017-12-24 19:34 ` Eli Zaretskii
2017-12-25 20:13 ` Philipp Stephani
2017-12-25 21:01 ` [PATCH] Allow inserting non-BMP characters Philipp Stephani
2017-12-26 1:26 ` Alan Third
2017-12-26 4:46 ` Eli Zaretskii
2017-12-26 10:35 ` Philipp Stephani
2017-12-26 16:11 ` Eli Zaretskii
2017-12-26 18:50 ` Philipp Stephani
2017-12-26 20:22 ` Eli Zaretskii
2017-12-26 21:36 ` Alan Third
2017-12-27 3:41 ` Eli Zaretskii
2017-12-28 11:38 ` Alan Third
2017-12-28 12:31 ` Philipp Stephani
2017-12-28 16:29 ` Eli Zaretskii
2017-12-29 20:14 ` Philipp Stephani [this message]
2017-12-29 20:27 ` Eli Zaretskii
2018-01-07 15:51 ` Philipp Stephani
2018-01-07 17:40 ` Eli Zaretskii
2018-01-07 18:44 ` Philipp Stephani
2017-12-25 21:07 ` bug#29837: UTF-16 char display problems and the macOS "character palette" Philipp Stephani
2017-12-26 1:34 ` Alan Third
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAArVCkTqGP-=3VQem3LtyTFjsv=cVrPFOiQY91fxSp3AJLBEtw@mail.gmail.com' \
--to=p.stephani2@gmail.com \
--cc=alan@idiocy.org \
--cc=eliz@gnu.org \
--cc=emacs-devel@gnu.org \
--cc=phst@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/emacs.git
https://git.savannah.gnu.org/cgit/emacs/org-mode.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.