bug#2957: 23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input

bug#2957: 23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input

[Ashutosh Mehra]

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:

BUG DESCRIPTION:
When I type the following:
C-x 8 RET greek letter alpha
and press TAB, I get several choices for the greek letter alpha
variants. But I'm unable to type anything into the minibuffer (neither
SPC nor any character) -- Emacs just doesn't accept the input. If I
press RET at this point, I get the error message "ucs-insert: Not a
Unicode character code: nil".
I tried with "emacs -Q" (no init files), and I still get this
incorrect behavior.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
d:/Progs/emacs/etc/DEBUG for instructions.


In GNU Emacs 23.0.92.1 (i386-mingw-nt5.1.2600)
 of 2009-03-31 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x C-g C-x 8 <return> g r e e k SPC l e <tab> <tab>
a l p <tab> <tab> SPC SPC SPC w <tab> <tab> <backspace>
<tab> <tab> <return> M-x r e p o r t - e m <tab> <
return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit
Loading Unicode character names...done
Making completion list... [6 times]
ucs-insert: Not a Unicode character code: nil

bug#2957: 23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input

[Andreas Schwab]

Ashutosh Mehra <ashutoshmehra@gmail.com> writes:

> BUG DESCRIPTION:
> When I type the following:
> C-x 8 RET greek letter alpha
> and press TAB, I get several choices for the greek letter alpha
> variants. But I'm unable to type anything into the minibuffer (neither
> SPC nor any character) -- Emacs just doesn't accept the input. If I
> press RET at this point, I get the error message "ucs-insert: Not a
> Unicode character code: nil".

This is a problem with partial-completion.  You can continue with C-q
SPC, or move back before "letter" and insert either "capital" or
"small".

Here is a reduced testcase:

(completing-read "Type a SPC b TAB: " '("a 1 b" "a 1 b c" "a 1 b d" "a 2 b" "a 2 b c" "a 2 b d"))

The minibuffer contents become "a SPC SPC b", indicating that a word is
missing between "a" and "b".  Either partial-completion should move
point back to where the word is missing or accept SPC as input.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

bug#2957: 23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input

[Stefan Monnier]

>> BUG DESCRIPTION:
>> When I type the following:
>> C-x 8 RET greek letter alpha
>> and press TAB, I get several choices for the greek letter alpha
>> variants.  But I'm unable to type anything into the minibuffer (neither
>> SPC nor any character) -- Emacs just doesn't accept the input.

I understand that SPC would signal an error, but any normal character
should be inserted in the minibuffer just fine.  Do you really mean that
you can't type anything in the minibuffer, or just that anything you
type will later lead to a completion failure or to the error you mention:

>> If I press RET at this point, I get the error message "ucs-insert:
>> Not a Unicode character code: nil".

That looks like a (minor) bug indeed.

> This is a problem with partial-completion.
> (completing-read "Type a SPC b TAB: "
>                  '("a 1 b" "a 1 b c" "a 1 b d" "a 2 b" "a 2 b c" "a 2 b d"))
> The minibuffer contents become "a SPC SPC b", indicating that a word is
> missing between "a" and "b".  Either partial-completion should move
> point back to where the word is missing or accept SPC as input.

Actually, it's a problem with minibuffer-complete-word which sometimes
wants partial completion and sometimes doesn't.
The patch below provides an alternative way to choose between using
partial-completion and not, which is finer-grained and should fix the
above problem without reintroducing the problems it tried to fix,


        Stefan


=== modified file 'lisp/minibuffer.el'
--- lisp/minibuffer.el	2009-03-19 04:24:15 +0000
+++ lisp/minibuffer.el	2009-04-11 13:51:07 +0000
@@ -780,13 +780,13 @@
       ;; If completion finds next char not unique,
       ;; consider adding a space or a hyphen.
       (when (= (length string) (length (car comp)))
-        (let ((exts '(" " "-"))
+        ;; Mark the added char with the `completion-word' property, so it
+        ;; can be handled specially by completion styles such as
+        ;; partial-completion.
+        (let ((exts (mapcar (lambda (str) (propertize str 'completion-word t))
+                            '(" " "-")))
               (before (substring string 0 point))
               (after (substring string point))
-	      ;; Disable partial-completion for this.
-	      (completion-styles
-	       (or (remove 'partial-completion completion-styles)
-		   completion-styles))
 	      tem)
 	  (while (and exts (not (consp tem)))
             (setq tem (completion-try-completion
@@ -1598,7 +1598,13 @@
            (p 0)
            (p0 p))
 
-      (while (setq p (string-match completion-pcm--delim-wild-regex string p))
+      (while (and (setq p (string-match completion-pcm--delim-wild-regex
+                                        string p))
+                  ;; If the char was added by minibuffer-complete-word, then
+                  ;; don't treat it as a delimiter, otherwise "M-x SPC"
+                  ;; ends up inserting a "-" rather than listing
+                  ;; all completions.
+                  (not (get-text-property p 'completion-word string)))
         ;; Usually, completion-pcm--delim-wild-regex matches a delimiter,
         ;; meaning that something can be added *before* it, but it can also
         ;; match a prefix and postfix, in which case something can be added

bug#2957: 23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input

[Ashutosh Mehra]

>>> BUG DESCRIPTION:
>>> When I type the following:
>>> C-x 8 RET greek letter alpha
>>> and press TAB, I get several choices for the greek letter alpha
>>> variants.  But I'm unable to type anything into the minibuffer (neither
>>> SPC nor any character) -- Emacs just doesn't accept the input.
>
> I understand that SPC would signal an error, but any normal character
> should be inserted in the minibuffer just fine.  Do you really mean that
> you can't type anything in the minibuffer, or just that anything you
> type will later lead to a completion failure or to the error you mention:

Yes, any normal character would get inserted (but not SPC). Sorry for
the ambiguity.

Also, I have to confess, I never noticed that partial completion
inserted an extra space in the middle of "GREEK LETTER ALPHA" for me
to type in CAPITAL/SMALL.

> Actually, it's a problem with minibuffer-complete-word which sometimes
> wants partial completion and sometimes doesn't.
> The patch below provides an alternative way to choose between using
> partial-completion and not, which is finer-grained and should fix the
> above problem without reintroducing the problems it tried to fix,

Stefan, I have emacs pretest 23.0.92.1, and could not apply your patch
cleanly (~170 lines appear to have been added in your
lisp/minibuffer.el before the first hunk, and the "string-match" was
"string-match-p" in my version of minibuffer.el). So, I applied the
changes manually, but the original behavior persisted. Emacs doesn't
let me type a SPC (unless I use C-q SPC) after the partial input
"GREEK LETTER ALPHA". I will anyway try out the coming pretests.

Thanks Andreas and Stefan for looking into this so quickly.

Ashutosh

bug#2957: marked as done (23.0.92; ucs-insert: Completion does not work correctly with Unicode Character Name Input)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 962 bytes --]


Your message dated Mon, 13 Apr 2009 22:39:07 -0400
with message-id <jwvfxgcufla.fsf-monnier+emacsbugreports@gnu.org>
and subject line Re: bug#2957: 23.0.92; ucs-insert: Completion does not work correctly
has caused the Emacs bug report #2957,
regarding 23.0.92; ucs-insert: Completion does not work correctly with Unicode  Character Name Input
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
2957: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=2957
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 5243 bytes --]

From: Ashutosh Mehra <ashutoshmehra@gmail.com>
To: emacs-pretest-bug@gnu.org
Subject: 23.0.92; ucs-insert: Completion does not work correctly with Unicode  Character Name Input
Date: Sat, 11 Apr 2009 08:22:13 +0530
Message-ID: <a7204c010904101952r7ababc1bne1a0264ddd893dcd@mail.gmail.com>

Please write in English if possible, because the Emacs maintainers
usually do not have translators to read other languages for them.

Your bug report will be posted to the emacs-pretest-bug@gnu.org mailing list.

Please describe exactly what actions triggered the bug
and the precise symptoms of the bug:

BUG DESCRIPTION:
When I type the following:
C-x 8 RET greek letter alpha
and press TAB, I get several choices for the greek letter alpha
variants. But I'm unable to type anything into the minibuffer (neither
SPC nor any character) -- Emacs just doesn't accept the input. If I
press RET at this point, I get the error message "ucs-insert: Not a
Unicode character code: nil".
I tried with "emacs -Q" (no init files), and I still get this
incorrect behavior.

If Emacs crashed, and you have the Emacs process in the gdb debugger,
please include the output from the following gdb commands:
    `bt full' and `xbacktrace'.
If you would like to further debug the crash, please read the file
d:/Progs/emacs/etc/DEBUG for instructions.


In GNU Emacs 23.0.92.1 (i386-mingw-nt5.1.2600)
 of 2009-03-31 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ENU
  value of $XMODIFIERS: nil
  locale-coding-system: cp1252
  default-enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
M-x C-g C-x 8 <return> g r e e k SPC l e <tab> <tab>
a l p <tab> <tab> SPC SPC SPC w <tab> <tab> <backspace>
<tab> <tab> <return> M-x r e p o r t - e m <tab> <
return>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Quit
Loading Unicode character names...done
Making completion list... [6 times]
ucs-insert: Not a Unicode character code: nil



[-- Attachment #3: Type: message/rfc822, Size: 3030 bytes --]

From: Stefan Monnier <monnier@iro.umontreal.ca>
To: Ashutosh Mehra <ashutoshmehra@gmail.com>
Cc: Andreas Schwab <schwab@linux-m68k.org>, 2957-done@emacsbugs.donarmstrong.com
Subject: Re: bug#2957: 23.0.92; ucs-insert: Completion does not work correctly
Date: Mon, 13 Apr 2009 22:39:07 -0400
Message-ID: <jwvfxgcufla.fsf-monnier+emacsbugreports@gnu.org>

> Yes, any normal character would get inserted (but not SPC). Sorry for
> the ambiguity.

Thanks.

> Also, I have to confess, I never noticed that partial completion
> inserted an extra space in the middle of "GREEK LETTER ALPHA" for me
> to type in CAPITAL/SMALL.

No need to confess.

> Stefan, I have emacs pretest 23.0.92.1, and could not apply your patch
> cleanly (~170 lines appear to have been added in your
> lisp/minibuffer.el before the first hunk, and the "string-match" was
> "string-match-p" in my version of minibuffer.el).  So, I applied the
> changes manually, but the original behavior persisted.  Emacs doesn't
> let me type a SPC (unless I use C-q SPC) after the partial input
> "GREEK LETTER ALPHA".  I will anyway try out the coming pretests.

Thanks for going through the trouble.
The file is precompiled, so you'd need to rebuild Emacs (or to use
M-C-x to reevaluate the definition) to see the difference.
I've installed my patch, so it should be fixed in the next pretest.
If not, please reopen the bug.


        Stefan