describe-char should display the character's Unicode name

describe-char should display the character's Unicode name

[Ted Zlatanov]

`describe-char' currently displays the character, but in cases where the
font is not capable of that, just the codepoint number is not very
useful for the user.  The character's Unicode name would be nice, but
the user has to customize `describe-char-unidata-list'.  At least the
Unicode name ('name) should be in `describe-char-unidata-list' by
default.  Since `describe-char' is used rarely and the user really wants
to know about the character, I think this is sensible.  It could also be
used for `what-cursor-position' which is similarly rarely used.

Ted

Re: describe-char should display the character's Unicode name

[Juri Linkov]

> `describe-char' currently displays the character, but in cases where the
> font is not capable of that, just the codepoint number is not very
> useful for the user.  The character's Unicode name would be nice, but
> the user has to customize `describe-char-unidata-list'.  At least the
> Unicode name ('name) should be in `describe-char-unidata-list' by
> default.  Since `describe-char' is used rarely and the user really wants
> to know about the character, I think this is sensible.  It could also be
> used for `what-cursor-position' which is similarly rarely used.

Displaying the full Unicode information was turned off by default,
because licence conditions of the Unicode data file prohibited adding
it to Emacs, so the user had to download the data file and point the
variable to the location of this file to be able to see the full
information from it.

But since now it is available in Emacs without the need to download it,
it makes sense to display its full Unicode information by default.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Kenichi Handa]

In article <87myki6fqp.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> > `describe-char' currently displays the character, but in cases where the
> > font is not capable of that, just the codepoint number is not very
> > useful for the user.  The character's Unicode name would be nice, but
> > the user has to customize `describe-char-unidata-list'.  At least the
> > Unicode name ('name) should be in `describe-char-unidata-list' by
> > default.  Since `describe-char' is used rarely and the user really wants
> > to know about the character, I think this is sensible.  It could also be
> > used for `what-cursor-position' which is similarly rarely used.

> Displaying the full Unicode information was turned off by default,
> because licence conditions of the Unicode data file prohibited adding
> it to Emacs, so the user had to download the data file and point the
> variable to the location of this file to be able to see the full
> information from it.

> But since now it is available in Emacs without the need to download it,
> it makes sense to display its full Unicode information by default.

I think displaying all available Unicode informaiton is too
much (see the list shown when you customize
describe-char-unidata-list).  Perhaps, name,
general-cateogory, and decompostion are the best for the
default of that variable.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: describe-char should display the character's Unicode name

[Juri Linkov]

> I think displaying all available Unicode informaiton is too
> much (see the list shown when you customize
> describe-char-unidata-list).  Perhaps, name,
> general-cateogory, and decompostion are the best for the
> default of that variable.

I agree, and please also add `old-name' because it often helps
to recognize the character by its old name.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Kenichi Handa]

In article <87mykhz6tf.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> > I think displaying all available Unicode informaiton is too
> > much (see the list shown when you customize
> > describe-char-unidata-list).  Perhaps, name,
> > general-cateogory, and decompostion are the best for the
> > default of that variable.

> I agree, and please also add `old-name' because it often helps
> to recognize the character by its old name.

Ok, I've just installed this change.

*** descr-text.el.~1.66.~	2008-06-30 09:57:09.000000000 +0900
--- descr-text.el	2008-07-17 10:23:54.000000000 +0900
***************
*** 179,185 ****
  	(insert "There are text properties here:\n")
  	(describe-property-list properties)))))
  \f
! (defcustom describe-char-unidata-list nil
    "List of Unicode-based character property names shown by `describe-char'."
    :group 'mule
    :version "23.1"
--- 179,186 ----
  	(insert "There are text properties here:\n")
  	(describe-property-list properties)))))
  \f
! (defcustom describe-char-unidata-list
!   '(name general-category decomposition old-name)
    "List of Unicode-based character property names shown by `describe-char'."
    :group 'mule
    :version "23.1"

---
Kenichi Handa
handa@ni.aist.go.jp

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Thu, 17 Jul 2008 10:26:08 +0900 Kenichi Handa <handa@m17n.org> wrote: 

KH> In article <87mykhz6tf.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:
>> > I think displaying all available Unicode informaiton is too
>> > much (see the list shown when you customize
>> > describe-char-unidata-list).  Perhaps, name,
>> > general-cateogory, and decompostion are the best for the
>> > default of that variable.

>> I agree, and please also add `old-name' because it often helps
>> to recognize the character by its old name.

KH> Ok, I've just installed this change.

Thank you, I've been too busy to be useful with this discussion or
anything ese, but I agree with your patch, those basic properties are
what a user would generally want.  I customized mine to t to see all
properties, of course :)

Ted

Re: describe-char should display the character's Unicode name

[Juri Linkov]

>> > I think displaying all available Unicode informaiton is too
>> > much (see the list shown when you customize
>> > describe-char-unidata-list).  Perhaps, name,
>> > general-cateogory, and decompostion are the best for the
>> > default of that variable.
>
>> I agree, and please also add `old-name' because it often helps
>> to recognize the character by its old name.
>
> Ok, I've just installed this change.

Thank you.

BTW, I remember that we once discussed about an ability to input
characters by their Unicode names and codepoints.  But currently
I can't find such a command, maybe I miss it.  I see only the old command
`ucs-insert'.  Is `ucs-insert' already obsolete?  As I understand,
hex values of Unicode characters can be now entered by `C-q'?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Miles Bader]

Juri Linkov <juri@jurta.org> writes:
> Is `ucs-insert' already obsolete?  As I understand, hex values of
> Unicode characters can be now entered by `C-q'?

Why do you think ucs-insert is obsolete?

I guess C-q _could_ be used to insert unicode chars, but it's not
compatible with previous releases, and a bit weird anyway -- notably,
C-q uses octal by default, whereas ucs-insert uses hex, which seems to
be standard for specifying unicode chars.  While C-q can be changed to
use hex chars, that's a change to its historical behavior.

-Miles

-- 
Infancy, n. The period of our lives when, according to Wordsworth,
'Heaven lies about us.' The world begins lying about us pretty soon
afterward.

inputting characters by hexadigit

[Kenichi Handa]

In article <87tzeogih6.fsf@catnip.gol.com>, Miles Bader <miles@gnu.org> writes:

> While C-q can be changed to use hex chars, that's a change
> to its historical behavior.

I think people who want to input Unicode characters by
typing code point should customize read-quoted-char-radix to
16.

I once proposed C-q 8 and C-9 to change
read-quoted-char-radix to 10 and 16 temporarily so that one
can type C-q 9 c 0 RET to input U+00C0.  I still think it's
very convenient.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Kenichi Handa wrote:
> I once proposed C-q 8 and C-9 to change
> read-quoted-char-radix to 10 and 16 temporarily so that one
> can type C-q 9 c 0 RET to input U+00C0.  I still think it's
> very convenient.


Maybe it is better mnemonic to switch C-q 8 and C-q 9 to 16 resp 10?

Re: inputting characters by hexadigit

[Miles Bader]

Kenichi Handa <handa@m17n.org> writes:
>> While C-q can be changed to use hex chars, that's a change
>> to its historical behavior.
>
> I think people who want to input Unicode characters by
> typing code point should customize read-quoted-char-radix to
> 16.

That presumes that unicode characters are only input by people who want
to do it a lot.  However, for _occasional_ users, it's silly to require
them to customize read-quoted-char-radix just to input a character code
they read on some webpage somewhere.

> I once proposed C-q 8 and C-9 to change
> read-quoted-char-radix to 10 and 16 temporarily so that one
> can type C-q 9 c 0 RET to input U+00C0.  I still think it's
> very convenient.

That's pretty convoluted...

-Miles

-- 
Monday, n. In Christian countries, the day after the baseball game.

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Kenichi Handa wrote:
> 
> I once proposed C-q 8 and C-9 to change
> read-quoted-char-radix to 10 and 16 temporarily so that one
> can type C-q 9 c 0 RET to input U+00C0.  I still think it's
> very convenient.
> 

Well, convenient, but maybe supporting emacs lisp hex
literal syntax might be more memorable? e.g. C-q #x2618

Re: inputting characters by hexadigit

[Miles Bader]

David De La Harpe Golden <david@harpegolden.net> writes:
>> I once proposed C-q 8 and C-9 to change read-quoted-char-radix to 10
>> and 16 temporarily so that one can type C-q 9 c 0 RET to input
>> U+00C0.  I still think it's very convenient.
>
> Well, convenient, but maybe supporting emacs lisp hex
> literal syntax might be more memorable? e.g. C-q #x2618

Yes, that sounds like a good syntax to me -- I think it's important that
it be easy to remember, because many people will use it only
occasionally.

-Miles

-- 
Mayonnaise, n. One of the sauces that serve the French in place of a state
religion.

Re: inputting characters by hexadigit

[Stephen J. Turnbull]

Miles Bader writes:
 > David De La Harpe Golden <david@harpegolden.net> writes:
 > >> I once proposed C-q 8 and C-9 to change read-quoted-char-radix to 10
 > >> and 16 temporarily so that one can type C-q 9 c 0 RET to input
 > >> U+00C0.  I still think it's very convenient.
 > >
 > > Well, convenient, but maybe supporting emacs lisp hex
 > > literal syntax might be more memorable? e.g. C-q #x2618
 > 
 > Yes, that sounds like a good syntax to me -- I think it's important that
 > it be easy to remember, because many people will use it only
 > occasionally.

Agreed, but how about trying the syntax everybody actually uses: `C-q
U+00C0'?  Are there situations where C-q U really needs to insert a U?

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stephen J. Turnbull wrote:

>  > Yes, that sounds like a good syntax to me -- I think it's important that
>  > it be easy to remember, because many people will use it only
>  > occasionally.
> 
> Agreed, but how about trying the syntax everybody actually uses: `C-q
> U+00C0'? 

Well, quibbling, but not sure that everybody uses that for _input_.
Gtk+/GOME apps, firefox  use C-s-U 00C0 for example (yes I know that's
not a runner for emacs, where C-u, sometimes indistinguishable from
C-s-U, is already an important keypress).

More importantly:   Does C-q still do language-environment based
translations for octal input #o200 -> #o377 ?  Manual node "Inserting
Text" says it does, but it doesn't seem to be doing quite what it says
I'm not 100% on what it is doing - embedding the raw byte?
Funny thing is e.g. C-q 23030 embeds U+2618 as expected.

I was, not necessarily for any particularly good reason,  thinking that
entering C-q #xnumber-in-hexadecimal would always do _exactly_ what the
same C-q number-in-octal does ( in turn same as C-q
number-in-hexadecimal  with a hex read-quoted-char-radix). => If
translations as documented in the manual are happening, that would mean
that number is _not_ necessarily simply a unicode code and C-q U+ could
be misleading.

OTOH,if it departed from that C-q octal behaviour and _always_ inserted
the _unicode_ char, well, then C-q U+ would seem fairly appropriate.

Re: inputting characters by hexadigit

[James Cloos]

[Choosing a random message on the thread...]

In the UCS age I would just make read-quoted-char-radix deafult to 16.

No one really uses octal any more.  I can't remember wanting it for
anything other than chmod in the last two decades.

Doing <C-q> <hex>+ <return> is /much/ easier than anything else.

And there have been enough other changes to defaults that one more isn't
such a big deal.  :^/

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: inputting characters by hexadigit

[Stephen J. Turnbull]

James Cloos writes:
 > [Choosing a random message on the thread...]
 > 
 > In the UCS age I would just make read-quoted-char-radix deafult to 16.
 > 
 > No one really uses octal any more.

<sigh /> ":-" C-q "376" RET (:-þ) to you.  (It means "d'oh, I get it
now" in Japanese, I don't use it in English.)

Re: inputting characters by hexadigit

[James Cloos]

>>>>> "Stephen" == Stephen J Turnbull <turnbull@sk.tsukuba.ac.jp> writes:

Stephen> <sigh /> ":-" C-q "376" RET (:-þ) to you.  (It means "d'oh, I
Stephen> get it now" in Japanese, I don't use it in English.)

OK, no one but some (fellow) old-timers.  :^)

But that doesn't work for me.  I get \376 rather than þ.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: inputting characters by hexadigit

[Juri Linkov]

> Stephen> <sigh /> ":-" C-q "376" RET (:-þ) to you.  (It means "d'oh, I
> Stephen> get it now" in Japanese, I don't use it in English.)
>
> OK, no one but some (fellow) old-timers.  :^)
>
> But that doesn't work for me.  I get \376 rather than þ.

It inserts \376 rather than þ due to the following code in `quoted-insert':

    ;; Assume character codes 0240 - 0377 stand for characters in some
    ;; single-byte character set, and convert them to Emacs
    ;; characters.
    (if (and enable-multibyte-characters
	     (>= char ?\240)
	     (<= char ?\377))
	(setq char (unibyte-char-to-multibyte char)))

I don't know if this code is still valid now?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

>     ;; Assume character codes 0240 - 0377 stand for characters in some
>     ;; single-byte character set, and convert them to Emacs
>     ;; characters.
>     (if (and enable-multibyte-characters
> 	     (>= char ?\240)
> 	     (<= char ?\377))
> 	(setq char (unibyte-char-to-multibyte char)))

> I don't know if this code is still valid now?

Indeed, it's not valid any more.
It should probably try something like

  (aref (decode-coding-string (unibyte-string char) locale-coding-system) 0)

instead, tho I'd argue that we should drop this feature and insert the
corresponding Unicode char instead (which would give the latin-1
equivalent as well).


        Stefan

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:
>>     ;; Assume character codes 0240 - 0377 stand for characters in some

> Indeed, it's not valid any more.
> It should probably try something like
> 
>   (aref (decode-coding-string (unibyte-string char) locale-coding-system) 0)
> 
> instead, tho I'd argue that we should drop this feature and insert the
> corresponding Unicode char instead (which would give the latin-1
> equivalent as well).
> 


While I agree that having it just be unicode simplifies matters,
couldn't doing the translation again pretty easily be made a
customisable "read-quoted-char-use-locale-coding-system" boolean ?

... but should that be
read-quoted-char-use-buffer-file-coding-system  ?
- i.e. if you open an iso8859-15 file, and "C-q a4" (assuming hex radix)
, you get a euro sign inserted?

- it just strikes me people with non unicode locales (or files)
are  some of the most likely to want to enter characters by number from
the relevant non-unicode charset...

Re: inputting characters by hexadigit

[Juri Linkov]

>>>     ;; Assume character codes 0240 - 0377 stand for characters in some
>
>> Indeed, it's not valid any more.
>> It should probably try something like
>>
>>   (aref (decode-coding-string (unibyte-string char) locale-coding-system) 0)
>>
>> instead, tho I'd argue that we should drop this feature and insert the
>> corresponding Unicode char instead (which would give the latin-1
>> equivalent as well).
>
> While I agree that having it just be unicode simplifies matters,
> couldn't doing the translation again pretty easily be made a
> customisable "read-quoted-char-use-locale-coding-system" boolean ?
>
> ... but should that be
> read-quoted-char-use-buffer-file-coding-system  ?
> - i.e. if you open an iso8859-15 file, and "C-q a4" (assuming hex radix)
> , you get a euro sign inserted?
>
> - it just strikes me people with non unicode locales (or files)
> are  some of the most likely to want to enter characters by number from
> the relevant non-unicode charset...

Couldn't they do that using the `C-x 8' prefix?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

> Couldn't they do that using the `C-x 8' prefix?

Funny, I just noticed that

   (info "(elisp) Key Binding Conventions")

does not mention C-x at all ...

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Juri Linkov wrote:

>> - it just strikes me people with non unicode locales (or files)
>> are  some of the most likely to want to enter characters by number from
>> the relevant non-unicode charset...
> 
> Couldn't they do that using the `C-x 8' prefix?
> 

By number? Maybe I missed it- can they already?  Otherwise entry would
be dependent on there being a composition defined for the character,
which is not a given. And if you're working with some
once-in-a-blue-moon legacy charset, it might be much easier to look up
the char code in a code table in front of you than learn the
compositions, probably depending on how visually-minded you are.

Of course they could if C-x 8 was expanded with a means to enter
numeric codes. :-)

Re: inputting characters by hexadigit

[Juri Linkov]

>>     ;; Assume character codes 0240 - 0377 stand for characters in some
>>     ;; single-byte character set, and convert them to Emacs
>>     ;; characters.
>>     (if (and enable-multibyte-characters
>> 	     (>= char ?\240)
>> 	     (<= char ?\377))
>> 	(setq char (unibyte-char-to-multibyte char)))
>
>> I don't know if this code is still valid now?
>
> Indeed, it's not valid any more.
> It should probably try something like
>
>   (aref (decode-coding-string (unibyte-string char) locale-coding-system) 0)
>
> instead, tho I'd argue that we should drop this feature and insert the
> corresponding Unicode char instead (which would give the latin-1
> equivalent as well).

So is it ok to remove this code or it's better to comment it out?

BTW, there are also warnings in the same function (and few other
places):

    In quoted-insert:
    simple.el:685:22:Warning: `translation-table-for-input' is an obsolete
        variable (as of Emacs 23.1).

Could you tell what changes this requires?

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

>> Indeed, it's not valid any more.
>> It should probably try something like
>> 
>> (aref (decode-coding-string (unibyte-string char) locale-coding-system) 0)
>> 
>> instead, tho I'd argue that we should drop this feature and insert the
>> corresponding Unicode char instead (which would give the latin-1
>> equivalent as well).

> So is it ok to remove this code or it's better to comment it out?

You can comment it out.

> BTW, there are also warnings in the same function (and few other
> places):

>     In quoted-insert:
>     simple.el:685:22:Warning: `translation-table-for-input' is an obsolete
>         variable (as of Emacs 23.1).

> Could you tell what changes this requires?

None.  This is part of the code that provides the
translation-table-for-input feature, so the obsolescence of that feature
is not directly relevant (except for the fact that that code should
hopefully never be used and will disappear in the future).


        Stefan

Re: inputting characters by hexadigit

[Stephen J. Turnbull]

James Cloos writes:
 > >>>>> "Stephen" == Stephen J Turnbull <stephen@xemacs.org> writes:
 > 
 > Stephen> <sigh /> ":-" C-q "376" RET (:-þ) to you.  (It means "d'oh, I
 > Stephen> get it now" in Japanese, I don't use it in English.)
 > 
 > OK, no one but some (fellow) old-timers.  :^)
 > 
 > But that doesn't work for me.  I get \376 rather than þ.

Interesting.  I haven't tried with a recent (GNU) Emacs.

Re: inputting characters by hexadigit

[Miles Bader]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:
> Agreed, but how about trying the syntax everybody actually uses: `C-q
> U+00C0'?  Are there situations where C-q U really needs to insert a U?

"Everybody?"  I've never used that before and yet I do every once in a
while insert unicode chars by hex code.

It would be more useful to keep commonality with elisp syntax I think.

-Miles

-- 
Abstainer, n. A weak person who yields to the temptation of denying himself a
pleasure. A total abstainer is one who abstains from everything but
abstention, and especially from inactivity in the affairs of others.

Re: inputting characters by hexadigit

[Juri Linkov]

>  > >> I once proposed C-q 8 and C-9 to change read-quoted-char-radix
>  > >> to 10 and 16 temporarily so that one can type C-q 9 c 0 RET to
>  > >> input U+00C0.  I still think it's very convenient.
>  > >
>  > > Well, convenient, but maybe supporting emacs lisp hex
>  > > literal syntax might be more memorable? e.g. C-q #x2618
>  > 
>  > Yes, that sounds like a good syntax to me -- I think it's important
>  > that it be easy to remember, because many people will use it
>  > only occasionally.
>
> Agreed, but how about trying the syntax everybody actually uses:
> `C-q U+00C0'?

It seems more convenient to omit the plus sign and accept `C-q U00C0'.

> Are there situations where C-q U really needs to insert a U?

Yes, there are rare situations where `C-q U' is necessary to insert
`a' without overwriting the underlying character in overwrite-mode.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Juri Linkov]

> David De La Harpe Golden <david@harpegolden.net> writes:
>>> I once proposed C-q 8 and C-9 to change read-quoted-char-radix to 10
>>> and 16 temporarily so that one can type C-q 9 c 0 RET to input
>>> U+00C0.  I still think it's very convenient.
>>
>> Well, convenient, but maybe supporting emacs lisp hex
>> literal syntax might be more memorable? e.g. C-q #x2618
>
> Yes, that sounds like a good syntax to me -- I think it's important that
> it be easy to remember, because many people will use it only
> occasionally.

In pre-23 days I once submitted a patch that used `C-q x00B6' syntax.
But I agree that hash notation is better because it is easy to remember
and consistent with standard Lisp syntax.

The patch below now provides the following variants:

C-q #x2323
C-q #o21443
C-q #b0010001100100011
C-q #d8995

However, I'm not sure about #d8995.  It is non-standard syntax where read
fails with (invalid-read-syntax "#").  The standard syntax for decimal is
without hash notation.  This suggests using `C-q 8995'.  But this means
changing read-quoted-char-radix default to 10!

Anyway, this patch also provides `C-q U2323' syntax that is mnemonic
for everyone accustomed to U+ syntax.

And another convenience is `C-q # RET' where after typing RET it reads a
code point or Unicode name in the minibuffer.

Index: lisp/subr.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/subr.el,v
retrieving revision 1.601
diff -c -r1.601 subr.el
*** lisp/subr.el	27 Jun 2008 02:13:36 -0000	1.601
--- lisp/subr.el	19 Jul 2008 00:27:36 -0000
***************
*** 1721,1727 ****
  The optional argument PROMPT specifies a string to use to prompt the user.
  The variable `read-quoted-char-radix' controls which radix to use
  for numeric input."
!   (let ((message-log-max nil) done (first t) (code 0) char translated)
      (while (not done)
        (let ((inhibit-quit first)
  	    ;; Don't let C-h get the help message--only help function keys.
--- 1721,1728 ----
  The optional argument PROMPT specifies a string to use to prompt the user.
  The variable `read-quoted-char-radix' controls which radix to use
  for numeric input."
!   (let ((message-log-max nil) done (first t) (code 0) char translated
! 	(radix read-quoted-char-radix) hash)
      (while (not done)
        (let ((inhibit-quit first)
  	    ;; Don't let C-h get the help message--only help function keys.
***************
*** 1753,1765 ****
  	     ;; Turn a meta-character into a character with the 0200 bit set.
  	     (setq code (logior (logand translated (lognot ?\M-\^@)) 128)
  		   done t))
! 	    ((and (<= ?0 translated) (< translated (+ ?0 (min 10 read-quoted-char-radix))))
! 	     (setq code (+ (* code read-quoted-char-radix) (- translated ?0)))
  	     (and prompt (setq prompt (message "%s %c" prompt translated))))
  	    ((and (<= ?a (downcase translated))
! 		  (< (downcase translated) (+ ?a -10 (min 36 read-quoted-char-radix))))
! 	     (setq code (+ (* code read-quoted-char-radix)
! 			   (+ 10 (- (downcase translated) ?a))))
  	     (and prompt (setq prompt (message "%s %c" prompt translated))))
  	    ((and (not first) (eq translated ?\C-m))
  	     (setq done t))
--- 1754,1789 ----
  	     ;; Turn a meta-character into a character with the 0200 bit set.
  	     (setq code (logior (logand translated (lognot ?\M-\^@)) 128)
  		   done t))
! 	    ((and first (= ?u (downcase translated)))
! 	     (setq radix 16)
! 	     (and prompt (setq prompt (message "%s U+" prompt))))
! 	    ((and first (= ?# translated))
! 	     (setq hash t)
! 	     (and prompt (setq prompt (message "%s #" prompt))))
! 	    (hash
! 	     (cond
! 	      ((= ?d (downcase translated))
! 	       (setq radix 10)
! 	       (and prompt (setq prompt (message "%s D" prompt))))
! 	      ((= ?b (downcase translated))
! 	       (setq radix 2)
! 	       (and prompt (setq prompt (message "%s B" prompt))))
! 	      ((= ?o (downcase translated))
! 	       (setq radix 8)
! 	       (and prompt (setq prompt (message "%s O" prompt))))
! 	      ((= ?x (downcase translated))
! 	       (setq radix 16)
! 	       (and prompt (setq prompt (message "%s X" prompt))))
! 	      ((= ?\015 (downcase translated))
! 	       (setq done t code (read-char-by-name "Unicode (hex or name): "))
! 	       (and prompt (setq prompt (message "%s X" prompt)))))
! 	     (setq hash nil))
! 	    ((and (<= ?0 translated) (< translated (+ ?0 (min 10 radix))))
! 	     (setq code (+ (* code radix) (- translated ?0)))
  	     (and prompt (setq prompt (message "%s %c" prompt translated))))
  	    ((and (<= ?a (downcase translated))
! 		  (< (downcase translated) (+ ?a -10 (min 36 radix))))
! 	     (setq code (+ (* code radix) (+ 10 (- (downcase translated) ?a))))
  	     (and prompt (setq prompt (message "%s %c" prompt translated))))
  	    ((and (not first) (eq translated ?\C-m))
  	     (setq done t))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Juri Linkov]

> The patch below now provides the following variants:
>
> C-q #x2323
> C-q #o21443
> C-q #b0010001100100011
> C-q #d8995
>
> However, I'm not sure about #d8995.  It is non-standard syntax where read
> fails with (invalid-read-syntax "#").  The standard syntax for decimal is
> without hash notation.  This suggests using `C-q 8995'.  But this means
> changing read-quoted-char-radix default to 10!
>
> Anyway, this patch also provides `C-q U2323' syntax that is mnemonic
> for everyone accustomed to U+ syntax.
>
> And another convenience is `C-q # RET' where after typing RET it reads a
> code point or Unicode name in the minibuffer.

It seems better to support all Emacs Lisp notations including both
leading characters `#' and `?':

      #b101100 => 44
      #o54 => 44
      #x2c => 44
      #24r1k => 44

      ?\012 => 10         ?\t => 9         ?\C-j => 10
      ?\101 => 65         ?A => 65         ?\v => 11
      ?\f => 12           ?\r => 13        ?\e => 27
      ?\s => 32           ?\\ => 92        ?\d => 127

      ?\uNNNN

The patch below simply uses `read' to read a character from the entered
string.  It also prepends `?' when the string begins with `\' thus allowing
to input Unicode code points using `C-q \uNNNN'.

Index: lisp/subr.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/subr.el,v
retrieving revision 1.601
diff -u -r1.601 subr.el
--- lisp/subr.el	27 Jun 2008 02:13:36 -0000	1.601
+++ lisp/subr.el	20 Jul 2008 20:28:29 -0000
@@ -1721,7 +1721,7 @@
 The optional argument PROMPT specifies a string to use to prompt the user.
 The variable `read-quoted-char-radix' controls which radix to use
 for numeric input."
-  (let ((message-log-max nil) done (first t) (code 0) char translated)
+  (let ((message-log-max nil) done (first t) (code 0) char translated string)
     (while (not done)
       (let ((inhibit-quit first)
 	    ;; Don't let C-h get the help message--only help function keys.
@@ -1753,6 +1753,19 @@
 	     ;; Turn a meta-character into a character with the 0200 bit set.
 	     (setq code (logior (logand translated (lognot ?\M-\^@)) 128)
 		   done t))
+	    ((and first (memq translated '(?# ?? ?\\)))
+	     (setq string (char-to-string translated))
+	     (and prompt (setq prompt (message "%s %c" prompt translated))))
+	    (string
+	     (if (eq translated ?\C-m)
+		 (setq done t code
+		       (if (eq (length string) 1)
+			   (read-char-by-name "Unicode (hex or name): ")
+			 (setq code (read (if (eq (aref string 0) ?\\)
+					      (concat "?" string)
+					    string)))))
+	       (setq string (concat string (char-to-string translated)))
+	       (and prompt (setq prompt (message "%s %c" prompt translated)))))
 	    ((and (<= ?0 translated) (< translated (+ ?0 (min 10 read-quoted-char-radix))))
 	     (setq code (+ (* code read-quoted-char-radix) (- translated ?0)))
 	     (and prompt (setq prompt (message "%s %c" prompt translated))))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

AFAIK, the main use of C-q is to insert a char by hitting the
corresponding key, handy when that key is bound to something else than
self-insert-command.

So by using chars ? # or \ to mean something else, we get into the
situation where the user might not know how to insert #, ?, or \ when
those keys are bound to non-self-inserting commands.  I think this is
a serious problem.


        Stefan

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:
> AFAIK, the main use of C-q is to insert a char by hitting the
> corresponding key, handy when that key is bound to something else than
> self-insert-command.
> 
> So by using chars ? # or \ to mean something else, we get into the
> situation where the user might not know how to insert #, ?, or \ when
> those keys are bound to non-self-inserting commands.  I think this is
> a serious problem.
> 

Well, that's true enough, though of course already applies to 0-7 or
0-9A-F depending on read-quoted-char-radix

Maybe just providing read-quoted-char-radix is enough :-), especially if
the #o240 -> #o377 encoding breakage is resolved.

(Though one thing that springs to mind is to allow C-q ## to insert
a literal #...  Also, couldn't #?\ pretty easily be made another
customize option? read-quoted-char-accept-elisp-literals)

Re: inputting characters by hexadigit

[Stefan Monnier]

>> So by using chars ? # or \ to mean something else, we get into the
>> situation where the user might not know how to insert #, ?, or \ when
>> those keys are bound to non-self-inserting commands.  I think this is
>> a serious problem.

> Well, that's true enough, though of course already applies to 0-7 or
> 0-9A-F depending on read-quoted-char-radix

That's true, and it bit me already several times.  I'd really prefer not
to make it worse.

> (Though one thing that springs to mind is to allow C-q ## to insert
> a literal #...

Maybe C-q # SPC (and C-q ? SPC, ...) could be used, indeed.  But it's
still yet-another-rule to learn.

> Also, couldn't #?\ pretty easily be made another
> customize option? read-quoted-char-accept-elisp-literals)

I'm only worried about the default behavior, here, but providing this
functionality via an option doesn't seem very satisfactory either.

I think a good solution should start by scrapping the C-q <octal>
corner case, and replace it with something that does not interfere as
much with C-q <char> and which additionally accepts hexadecimal (and
maybe more such as char names, ...).


        Stefan

Re: inputting characters by hexadigit

[Juri Linkov]

>> (Though one thing that springs to mind is to allow C-q ## to insert
>> a literal #...
>
> Maybe C-q # SPC (and C-q ? SPC, ...) could be used, indeed.  But it's
> still yet-another-rule to learn.

The beauty of using standard Emacs Lisp notations is that we don't have
to invent another escaping syntax.  So with the latest patch it is
possible to insert them with

    C-q ?#
    C-q ??
    C-q \#
    C-q \?
    C-q \\

All these notations are well documented in Emacs manuals.

> I think a good solution should start by scrapping the C-q <octal>
> corner case, and replace it with something that does not interfere as
> much with C-q <char> and which additionally accepts hexadecimal (and
> maybe more such as char names, ...).

I agree with your idea of separating input by number.  To do this
we should find only one special key to activate input by number.
I propose `RET'.  Currently `C-q RET' inserts "^M".  This character
is rarely needed to insert since more useful line-feed is on `C-q C-j'.
And it is more natural to insert ^M by using `C-q C-m'.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Johan Bockgård]

Juri Linkov <juri@jurta.org> writes:

> To do this we should find only one special key to activate input by
> number. I propose `RET'.

I suggest C-q.

> Currently `C-q RET' inserts "^M". This character is rarely needed to
> insert since more useful line-feed is on `C-q C-j'. And it is more
> natural to insert ^M by using `C-q C-m'.

RET is C-m.

-- 
Johan Bockgård

Re: inputting characters by hexadigit

[David Kastrup]

bojohan+news@dd.chalmers.se (Johan Bockgård) writes:

> Juri Linkov <juri@jurta.org> writes:
>
>> To do this we should find only one special key to activate input by
>> number. I propose `RET'.
>
> I suggest C-q.

That leaves the user without a reasonable or obvious key combination to
insert C-q.

>> Currently `C-q RET' inserts "^M". This character is rarely needed to
>> insert since more useful line-feed is on `C-q C-j'. And it is more
>> natural to insert ^M by using `C-q C-m'.
>
> RET is C-m.

Yes.

-- 
David Kastrup

Re: inputting characters by hexadigit

[Johan Bockgård]

David Kastrup <dak@gnu.org> writes:

> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>
>> Juri Linkov <juri@jurta.org> writes:
>>
>>> To do this we should find only one special key to activate input by
>>> number. I propose `RET'.
>>
>> I suggest C-q.
>
> That leaves the user without a reasonable or obvious key combination to
> insert C-q.

C-q C-q C-q

-- 
Johan Bockgård

Re: inputting characters by hexadigit

[David Kastrup]

bojohan+news@dd.chalmers.se (Johan Bockgård) writes:

> David Kastrup <dak@gnu.org> writes:
>
>> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>>
>>> Juri Linkov <juri@jurta.org> writes:
>>>
>>>> To do this we should find only one special key to activate input by
>>>> number. I propose `RET'.
>>>
>>> I suggest C-q.
>>
>> That leaves the user without a reasonable or obvious key combination to
>> insert C-q.
>
> C-q C-q C-q

Where is the reasoning or obviousness?

-- 
David Kastrup

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

David Kastrup wrote:
> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
> 
>> David Kastrup <dak@gnu.org> writes:
>>
>>> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>>>
>>>> Juri Linkov <juri@jurta.org> writes:
>>>>
>>>>> To do this we should find only one special key to activate input by
>>>>> number. I propose `RET'.
>>>> I suggest C-q.
>>> That leaves the user without a reasonable or obvious key combination to
>>> insert C-q.
>> C-q C-q C-q
> 
> Where is the reasoning or obviousness?

Maybe: easy to type, easy to remember + a little message after C-q C-q

Re: inputting characters by hexadigit

[David Kastrup]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:

> David Kastrup wrote:
>> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>>
>>> David Kastrup <dak@gnu.org> writes:
>>>
>>>> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>>>>
>>>>> Juri Linkov <juri@jurta.org> writes:
>>>>>
>>>>>> To do this we should find only one special key to activate input by
>>>>>> number. I propose `RET'.
>>>>> I suggest C-q.
>>>> That leaves the user without a reasonable or obvious key combination to
>>>> insert C-q.
>>> C-q C-q C-q
>>
>> Where is the reasoning or obviousness?
>
> Maybe: easy to type, easy to remember + a little message after C-q C-q

I don't like it.  I think that _if_ we are going to overload C-q, we
should use characters that are _not_ in the normal use set of C-q.

U is tolerable in that regard, and so would be x and o.  But C-q or RET
or # (some recent suggestions) are not in that set in my opinion.

-- 
David Kastrup

Re: inputting characters by hexadigit

[James Cloos]

>>>>> "Juri" == Juri Linkov <juri@jurta.org> writes:

Juri> I propose `RET'.  Currently `C-q RET' inserts "^M".
Juri> And it is more natural to insert ^M by using `C-q C-m'.

RET and C-m are the same on most terminals.  And it does need to be
entered by way of C-q on occasion.  

I do like Your idea of using the standard number schemes as an option,
but I also want something super simple like C-q <hex>+ RET.

And if C-q shouldn't get the overloading, what key should cover this?

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: inputting characters by hexadigit

[Stefan Monnier]

> I do like Your idea of using the standard number schemes as an option,
> but I also want something super simple like C-q <hex>+ RET.

If we consider that C-q <char> should stick to "insert <char>", then
one way to still use C-q + <charcode> is to use non-chars in <charcode>,
e.g. use the meta modifier.

This said, C-q M-<something> 3b5 RET doesn't sound very appealing.


        Stefan

Re: inputting characters by hexadigit

[Miles Bader]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
> If we consider that C-q <char> should stick to "insert <char>", then
> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
> e.g. use the meta modifier.

While that's a nice goal, I'm not sure the alternatives to C-q + MAGIC
are any better...

It seems to me that the having a single magic character would be
reasonable, it's having 27 different magic characters that starts to get
annoying.

So how about "#" as the single magic char, and have "##" to insert a #
character?

Then than # followed by other characers can be used for lots of cool
stuff... # DIGS, #xDIGS, #oDIGS, ...

-Miles

-- 
Once, adj. Enough.

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Miles Bader wrote:
> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>> If we consider that C-q <char> should stick to "insert <char>", then
>> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
>> e.g. use the meta modifier.
> 
> While that's a nice goal, I'm not sure the alternatives to C-q + MAGIC
> are any better...
> 
> It seems to me that the having a single magic character would be
> reasonable, it's having 27 different magic characters that starts to get
> annoying.
> 
> So how about "#" as the single magic char, and have "##" to insert a #
> character?

I am not sure if I misunderstand you, but having a printable character 
would IMO be very disturbing since Emacs can be used for so many things.

How about C-M-q as the magig key?

Re: inputting characters by hexadigit

[Johan Bockgård]

Miles Bader <miles@gnu.org> writes:

> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>> If we consider that C-q <char> should stick to "insert <char>", then
>> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
>> e.g. use the meta modifier.
>
> While that's a nice goal, I'm not sure the alternatives to C-q + MAGIC
> are any better...

C-q with a negative prefix argument currently does nothing...

-- 
Johan Bockgård

Re: inputting characters by hexadigit

[David Kastrup]

bojohan+news@dd.chalmers.se (Johan Bockgård) writes:

> Miles Bader <miles@gnu.org> writes:
>
>> Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>>> If we consider that C-q <char> should stick to "insert <char>", then
>>> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
>>> e.g. use the meta modifier.
>>
>> While that's a nice goal, I'm not sure the alternatives to C-q + MAGIC
>> are any better...
>
> C-q with a negative prefix argument currently does nothing...

C-- C-u C-u C-q e3 RET for base 16 entry?  Not the most pretty thing in
sight, but probably easier to type than the equivalent C-- 16 C-q e3
RET.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: inputting characters by hexadigit

[Johan Bockgård]

David Kastrup <dak@gnu.org> writes:

> bojohan+news@dd.chalmers.se (Johan Bockgård) writes:
>
>> C-q with a negative prefix argument currently does nothing...
>
> C-- C-u C-u C-q e3 RET for base 16 entry?  Not the most pretty thing in
> sight, but probably easier to type than the equivalent C-- 16 C-q e3
> RET.

I was thinking C-- C-q #xe3 etc (the prefix is still a repeat count).

-- 
Johan Bockgård

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Johan Bockgård wrote:

> I was thinking C-- C-q #xe3 etc (the prefix is still a repeat count).
> 

☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆☆
Repeating should definitely be kept possible... :-)
☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘☘

Re: inputting characters by hexadigit

[Juri Linkov]

>>> C-q with a negative prefix argument currently does nothing...
>>
>> C-- C-u C-u C-q e3 RET for base 16 entry?  Not the most pretty thing in
>> sight, but probably easier to type than the equivalent C-- 16 C-q e3
>> RET.
>
> I was thinking C-- C-q #xe3 etc (the prefix is still a repeat count).

This is easy to type, but using a negative prefix to input a number
is not too intuitive.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[David Kastrup]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

>> I do like Your idea of using the standard number schemes as an option,
>> but I also want something super simple like C-q <hex>+ RET.
>
> If we consider that C-q <char> should stick to "insert <char>", then
> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
> e.g. use the meta modifier.
>
> This said, C-q M-<something> 3b5 RET doesn't sound very appealing.

Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: inputting characters by hexadigit

[David De La Harpe Golden]

David Kastrup wrote:

> Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?
> 

C-M-q == indent-pp-sexp ...

Re: inputting characters by hexadigit

[David Kastrup]

David De La Harpe Golden <david@harpegolden.net> writes:

> David Kastrup wrote:
>
>> Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?
>> 
>
> C-M-q == indent-pp-sexp ...

Hm?

indent-pp-sexp is not on any key

GNU Emacs 23.0.60.3 (i686-pc-linux-gnu, GTK+ Version 2.12.9) of
2008-06-01 on lola

So if this has changed as a global binding, it must have been rather
recently and thus should not be cast into stone.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

David Kastrup wrote:
> David De La Harpe Golden <david@harpegolden.net> writes:
> 
>> David Kastrup wrote:
>>
>>> Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?
>>>
>> C-M-q == indent-pp-sexp ...
> 
> Hm?
> 
> indent-pp-sexp is not on any key
> 
> GNU Emacs 23.0.60.3 (i686-pc-linux-gnu, GTK+ Version 2.12.9) of
> 2008-06-01 on lola


Unfortunately it is there at least 2006-12-10 (which is the oldest 
version I have available here).

Re: inputting characters by hexadigit

[David Kastrup]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:

> David Kastrup wrote:
>> David De La Harpe Golden <david@harpegolden.net> writes:
>>
>>> David Kastrup wrote:
>>>
>>>> Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?
>>>>
>>> C-M-q == indent-pp-sexp ...
>>
>> Hm?
>>
>> indent-pp-sexp is not on any key
>>
>> GNU Emacs 23.0.60.3 (i686-pc-linux-gnu, GTK+ Version 2.12.9) of
>> 2008-06-01 on lola
>
>
> Unfortunately it is there at least 2006-12-10 (which is the oldest
> version I have available here).

It's not there in message mode for me, and not in fundamental mode.

Any idea what gives?

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: inputting characters by hexadigit

[Alfred M. Szmidt]

C-M-q is a terrible keybinding to choose, it tends to mean "fill
current defun" in whatever language specific mode you are in; and is
used in many modes making it quite pointless to use it for inserting
characters by hex.  Inserting chars. by hex is also not the most
common thing to have, so having a single chord for it seems silly.

I liked the proposal of using M-#, one could allow other input formats
as well:

C-q M-# xff
C-q M-# b01010101
C-q M-# o157

and would be a good mnemonic...

RE: inputting characters by hexadigit

[Drew Adams]

> C-M-q is a terrible keybinding to choose, it tends to mean "fill
> current defun" in whatever language specific mode you are in; and is
> used in many modes making it quite pointless to use it for inserting
> characters by hex.  Inserting chars. by hex is also not the most
> common thing to have, so having a single chord for it seems silly.

I was going to stay quiet on this topic, but I agree. Those are both good
arguments: (1) don't conflict with defun filling, (2) don't waste key chords.

Re: inputting characters by hexadigit

[Miles Bader]

"Alfred M. Szmidt" <ams@gnu.org> writes:
> I liked the proposal of using M-#, one could allow other input formats
> as well:
>
> C-q M-# xff
> C-q M-# b01010101
> C-q M-# o157
>
> and would be a good mnemonic...

Bindings that swap modifiers (control, then meta) are inevitably a pain
in the ass to type though...

Also many environments don't have real meta key available, and
presumably `ESC' is out as an alternative, because using C-q to quote
escape is probably rather common.

-Miles

-- 
Fast, small, soon; pick any 2.

Re: inputting characters by hexadigit

[David De La Harpe Golden]

David Kastrup wrote:
> David De La Harpe Golden <david@harpegolden.net> writes:
> 
>> David Kastrup wrote:
>>
>>> Why not C-M-q 3b5 RET ?  That one is not taken in general, it seems?
>>>
>> C-M-q == indent-pp-sexp ...
> 
> Hm?
> 
> indent-pp-sexp is not on any key
> 
> GNU Emacs 23.0.60.3 (i686-pc-linux-gnu, GTK+ Version 2.12.9) of
> 2008-06-01 on lola
> 
> So if this has changed as a global binding, it must have been rather
> recently and thus should not be cast into stone.

Hmm. actually, turns out it's "only" bound in the various lisps and
other programming languages major modes (including the scratch buffer
and c-mode, btw), so you're right that it's not a _true_ global binding
(I just spend most of my emacs time in such modes...).  But it's very
long-standing as "indent expression" AFAIK, was in emacs 19.

It's  one of those "pseudo global" bindings where the keypress does a
conceptually similar thing in many modes - in lisp mode,
indent-pp-sexp, in c mode, c-indent-exp, in perl mode, perl-indent-exp,
and so on.

Re: inputting characters by hexadigit

[Miles Bader]

David Kastrup <dak@gnu.org> writes:
>> C-M-q == indent-pp-sexp ...
>
> Hm?
>
> indent-pp-sexp is not on any key

It's bound in programming language modes (and has been for just about
forever).

It's quite a useful binding actually, I use it a lot...

-Miles

-- 
Optimist, n. A proponent of the doctrine that black is white.

Re: inputting characters by hexadigit

[James Cloos]

>>>>> "Stefan" == Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

Stefan> This said, C-q M-<something> 3b5 RET doesn't sound very appealing.

Agreed.  It does not.

Hard problem.

-JimC
-- 
James Cloos <cloos@jhcloos.com>         OpenPGP: 1024D/ED7DAEA6

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:
>> I do like Your idea of using the standard number schemes as an option,
>> but I also want something super simple like C-q <hex>+ RET.
> 
> If we consider that C-q <char> should stick to "insert <char>", then
> one way to still use C-q + <charcode> is to use non-chars in <charcode>,
> e.g. use the meta modifier.
> 
> This said, C-q M-<something> 3b5 RET doesn't sound very appealing.
> 

Could be made work, probably, but implementation a bit messy, since,
say, C-q M-u  ==> C-q ESC u on text terminals, which presently then
inserts an ESC literal, followed by a self-inserted u ...


On the whole, I'm now really wishing there was a separate binding. Are
there _any_ C-x «somethings» left???   C-x %  ?

Re: inputting characters by hexadigit

[Juri Linkov]

> Juri> I propose `RET'.  Currently `C-q RET' inserts "^M".
> Juri> And it is more natural to insert ^M by using `C-q C-m'.
>
> RET and C-m are the same on most terminals.  And it does need to be
> entered by way of C-q on occasion.

Ah, of course, I was confused by the case of M-TAB and M-C-i.
They are the same for Emacs but window managers steal one of them.

Maybe then `C-q M-RET'?

> I do like Your idea of using the standard number schemes as an option,
> but I also want something super simple like C-q <hex>+ RET.

Isn't `C-q M-RET #x<hex>+ RET' still easy to type?

Or even `C-q M-: #x<hex>+ RET'?  This is very similar to eval-expression
such as `C-u M-: #x<hex>+ RET' with only one difference that the latter
inserts it as a number and the former will insert it as a character.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Miles Bader]

Juri Linkov <juri@jurta.org> writes:
> Maybe then `C-q M-RET'?

(1) bouncing modifier keys are annoying to type

(2) many environments don't have a real meta key

-miles

-- 
Hers, pron. His.

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:

> I think a good solution should start by scrapping the C-q <octal>
> corner case.

Seems like something people'd get used to having pretty rapidly, even if
it was unfortunately positioned in retrospect.

One advantage of hanging the char-by-code stuff off C-q is (obviously
enough) that it doesn't eat another default global binding.

But maybe a separate global binding needs eating, if the goal is to
avoid further C-q pollution.  Inserting a character by code (or unicode
name) is perhaps something genuinely important enough? That would
basically come down to deciding a more convenient default binding for a
souped-up M-x ucs-insert.

I dunno what's left at this stage. Heh. C-x#  might be very nice. Funny,
I don't _recall_ ever using it (cue zillions of people pointing out what
it is used for)

Re: inputting characters by hexadigit

[David De La Harpe Golden]

David De La Harpe Golden wrote:

> I dunno what's left at this stage. Heh. C-x#  might be very nice. Funny,
> I don't _recall_ ever using it (cue zillions of people pointing out what
> it is used for)

Including me - that'd be emacslient server-edit, then.  Duh.

Re: inputting characters by hexadigit

[David Kastrup]

David De La Harpe Golden <david@harpegolden.net> writes:

> Stefan Monnier wrote:
>> AFAIK, the main use of C-q is to insert a char by hitting the
>> corresponding key, handy when that key is bound to something else than
>> self-insert-command.
>> 
>> So by using chars ? # or \ to mean something else, we get into the
>> situation where the user might not know how to insert #, ?, or \ when
>> those keys are bound to non-self-inserting commands.  I think this is
>> a serious problem.
>> 
>
> Well, that's true enough, though of course already applies to 0-7 or
> 0-9A-F depending on read-quoted-char-radix

But those are not likely to be rebound to electric functionality.  In
particular # is typically electric in cc-mode IIRC.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum

Re: inputting characters by hexadigit

[Kenichi Handa]

In article <jwv8wvwdnb5.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@iro.umontreal.ca> writes:

> AFAIK, the main use of C-q is to insert a char by hitting the
> corresponding key, handy when that key is bound to something else than
> self-insert-command.

> So by using chars ? # or \ to mean something else, we get into the
> situation where the user might not know how to insert #, ?, or \ when
> those keys are bound to non-self-inserting commands.  I think this is
> a serious problem.

One funny idea.  How about using C-u (u for Unicode) as this way?

  C-u 0 x X X X X RET  for U+XXXX   (and C-0 x X X X X RET)

C-u 0 x and C-0 x are noop now.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Juanma Barranquero]

On Tue, Jul 22, 2008 at 02:51, Kenichi Handa <handa@m17n.org> wrote:

> One funny idea.  How about using C-u (u for Unicode) as this way?
>
>  C-u 0 x X X X X RET  for U+XXXX   (and C-0 x X X X X RET)
>
> C-u 0 x and C-0 x are noop now.

FWIW, I like this one much, much more than the convoluted C-q alternatives...


  Juanma

Re: inputting characters by hexadigit

[Miles Bader]

"Juanma Barranquero" <lekktu@gmail.com> writes:
>> One funny idea.  How about using C-u (u for Unicode) as this way?
>>
>>  C-u 0 x X X X X RET  for U+XXXX   (and C-0 x X X X X RET)
>>
>> C-u 0 x and C-0 x are noop now.
>
> FWIW, I like this one much, much more than the convoluted C-q alternatives...

This behavior seems much, much, weirder, since it doesn't hook into any
existing command in a way that make any sense...

Where exactly does this get implemented anyway?   `self-insert-command'?

-Miles

-- 
Neighbor, n. One whom we are commanded to love as ourselves, and who does all
he knows how to make us disobedient.

Re: inputting characters by hexadigit

[Juanma Barranquero]

On Tue, Jul 22, 2008 at 10:33, Miles Bader <miles.bader@necel.com> wrote:

> This behavior seems much, much, weirder, since it doesn't hook into any
> existing command in a way that make any sense...

That's true. But, honestly, when I want to type a Unicode codepoint,
I'm not interested in the command that implements it, just that it is
convenient and easy to remember. Switching to a Unicode input method
would be a no-no; C-q seems most logical, but I don't much like the
idea of suddenly adding escape chars to it, so C-q # XXXX inserts
Unicode and then you need C-q # # to insert a simple #, etc.

That said, I currently have super-i bound to ucs-insert and I'm quite
happy with it, so any way we go it's fine by me.

  Juanma

Re: inputting characters by hexadigit

[Kenichi Handa]

In article <buosku2b9mw.fsf@dhapc248.dev.necel.com>, Miles Bader <miles.bader@necel.com> writes:

> "Juanma Barranquero" <lekktu@gmail.com> writes:
>>> One funny idea.  How about using C-u (u for Unicode) as this way?
>>> 
>>> C-u 0 x X X X X RET  for U+XXXX   (and C-0 x X X X X RET)
>>> 
>>> C-u 0 x and C-0 x are noop now.
> >
> > FWIW, I like this one much, much more than the convoluted C-q alternatives...

> This behavior seems much, much, weirder, since it doesn't hook into any
> existing command in a way that make any sense...

We can think of it as a new key-binding for a new command.

The other ideas are:

  C-x # x X ... RET (this also allows C-x # b ..., C-x # o ...)
  
  C-c # x X ... RET (likewise)

  C-c 0 x X ... RET  (or C-c 0 X ... RET, C-c 8 X ... RET)

It seems that C-c 8 X ... RET is the easiest to type.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Kenichi Handa wrote:
> The other ideas are:
> 
>   C-x # x X ... RET (this also allows C-x # b ..., C-x # o ...)


Is not `server-edit' (a misleading name!) for finishing buffers with 
Emacs clients waiting more important? That is currently on `C-x #' and 
at least I use it a lot.

Re: inputting characters by hexadigit

[Miles Bader]

"Lennart Borgman (gmail)" <lennart.borgman@gmail.com> writes:
>> The other ideas are:
>>
>>   C-x # x X ... RET (this also allows C-x # b ..., C-x # o ...)
>
> Is not `server-edit' (a misleading name!) for finishing buffers with
> Emacs clients waiting more important? That is currently on `C-x #' and
> at least I use it a lot.

How about "C-x :"?  It seems to be unused.

-Miles

-- 
Hers, pron. His.

Re: inputting characters by hexadigit

[Stefan Monnier]

> It seems that C-c 8 X ... RET is the easiest to type.

I think C-c 8 is reserved already, but indeed C-x 8 might be a very good
choice, since it's already used to enter other non-ascii chars.
I think it would make a lot of sense to extend it with other
combinations for various Unicode chars, as well as hex-codes for
arbitrary chars.


        Stefan

RE: inputting characters by hexadigit

[Drew Adams]

> C-x 8 might be a very good choice, since it's already used to
> enter other non-ascii chars. I think it would make a lot of
> sense to extend it with other combinations for various Unicode
> chars, as well as hex-codes for arbitrary chars.

This sounds like a good idea to me.

It would of course require some changes to the doc - e.g. the Emacs manual, node
Unibyte Mode, which describes this (as being for Latin-1 only). 

(I also suggest adding some mention of `iso-transl-ctl-x-8-map' to the Elisp
manual.)

Re: inputting characters by hexadigit

[Alfred M. Szmidt]

What about C-\ and having a input method (maybe that already exists?),
would be a lovley feature to use to input multiple unicode characters,
and then drop back to whatever input method you were using.

Inserting unicode in hex cannot be _that_ common that it requires a
short keybinding...

Re: inputting characters by hexadigit

[Juri Linkov]

> What about C-\ and having a input method (maybe that already exists?),
> would be a lovley feature to use to input multiple unicode characters,
> and then drop back to whatever input method you were using.

Such an input method exists already: `ucs' in uni-input.el.

But it can't read Unicode character by name or use other notations.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Alfred M. Szmidt wrote:
> What about C-\ and having a input method (maybe that already exists?),

The ucs input method _does_ exist. However, especially if you're already
using some other input method, it's a bit of a pain to switch to ucs and
back again just for the odd character. It needs a little expansion (it
only covers the BMP (4-hexit codes) and maybe should handle unicode
names if that's now legally feasible)

Er. Though I guess I should confess that I tend to have the ucs IM a C-\
away - I tend to use system-level input methods (scim/skim) for languages...

M-x ucs-insert (or M-x uc TAB RET) is shorter than switching input
method with C-x RET C-\ ucs RET , though presently doesn't allow C-u for
repetition, which is annoying (though that's presumably easily fixed).

> Inserting unicode in hex cannot be _that_ common that it requires a
> short keybinding...

If you're doing a lot of i18n/m17n/l10n work it can just be handy - and
you're quite likely to be in another input method, too, so the ucs input
method is less handy.

[following is mostly just thread recap...]

☆ The fact remains, there is already an existing short-keybinding
facility in emacs for numeric char code entry ‽ - C-q supports entering
by numeric code. It used to be okay on unibyte emacs , but is a bit
unsuitable for unicode emacs, at least with its default octal (icky
octal) radix.  Despite that, N.B., _it already works for entering
unicode_, except that right now, the existing facility does the wrong
thing for codes between #o240 and #o377.  At the very least, that's a
bug that needs to be resolved, even if nothing else changes as a result
of this thread...

The facility does damage the "purity" of C-q - If the real point of C-q
is to literally insert the next keypress, it obviously blocks that for
0-7 or 0-9 or 0-9a-fA-F depending on your setting of
read-quoted-char-radix.  But if it were to be dropped from C-q, then
it'd be IMO reasonable for people to expect a similarly easy alternative.


http://en.wikipedia.org/wiki/Unicode_input

Emacs _clashes with_ the short keybinding for unicode input typical on
gtk/GNOME desktops - C-s-U (which emacs can't treat as different to
C-u in general due to terminal restrictions).

(... Hmm. Is M-+ taken? would be a bit like the windoze keybinding...)

It's one thing for emacs to provide different short keybindings for
typically-short-keybinding-accessible functionality (as in C-w/M-w/C-y
vs. C-x/C-c/C-v), it's another to not provide a short keybinding at all...

[cheap shot] Vim provides quite a short keybinding - C-v u - for it.

There is some fondness for unicode symbols in some circles, and some
people DO just learn off a few hex codes. I myself tend to do ☘
/ ☠ / ☢ / ☺ / ♬ etc. from memory (or at least remember such symbols
are mostly blocks #x2600 #x2700 ...)

Re: inputting characters by hexadigit

[Miles Bader]

"Alfred M. Szmidt" <ams@gnu.org> writes:
> What about C-\ and having a input method (maybe that already exists?),
> would be a lovley feature to use to input multiple unicode characters,
> and then drop back to whatever input method you were using.

There is one already, but it's a pain to use in practice.

There really needs to be a one-shot input command, for the same reason
we need (the current) C-x 8 in addition to the various latin input
methods.

-Miles

-- 
"Whatever you do will be insignificant, but it is very important that
 you do it."  Mahatma Gandhi

Re: inputting characters by hexadigit

[Juri Linkov]

>> It seems that C-c 8 X ... RET is the easiest to type.
>
> I think C-c 8 is reserved already, but indeed C-x 8 might be a very good
> choice, since it's already used to enter other non-ascii chars.
> I think it would make a lot of sense to extend it with other
> combinations for various Unicode chars, as well as hex-codes for
> arbitrary chars.

It seems C-x 8 is a good key prefix for this thing, and the simplest
change we could make now is to bind `C-x 8 RET' to `ucs-insert'.

Or maybe also bind a new key to the `C-x RET' prefix, e.g.
bind `C-x RET C-q' to `ucs-insert'.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Miles Bader]

Stefan Monnier <monnier@IRO.UMontreal.CA> writes:
>> It seems that C-c 8 X ... RET is the easiest to type.
>
> I think C-c 8 is reserved already, but indeed C-x 8 might be a very
> good choice, since it's already used to enter other non-ascii chars.
> I think it would make a lot of sense to extend it with other
> combinations for various Unicode chars, as well as hex-codes for
> arbitrary chars.

That sounds like a great plan!

It's perhaps the first suggested binding in this long thread that
actually seems as if it stands a chance of being remembered by all those
people who only use hex entry twice a year (by my thinking, the vast
majority of users) ... :-)

-Miles

-- 
Twice, adv. Once too often.

Re: inputting characters by hexadigit

[Kenichi Handa]

In article <jwvr69l7v7b.fsf-monnier+emacs@gnu.org>, Stefan Monnier <monnier@IRO.UMontreal.CA> writes:

> > It seems that C-c 8 X ... RET is the easiest to type.
> I think C-c 8 is reserved already, but indeed C-x 8 might be a very good
> choice, since it's already used to enter other non-ascii chars.

C-x 8 1 and C-x 8 3 are already used.  That's why I proposed
C-x # ... instead.  Or, do you mean this?

  C-x 8 _SOME_NON_USED_KEY_ X X X X RET

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Stefan Monnier]

>> > It seems that C-c 8 X ... RET is the easiest to type.
>> I think C-c 8 is reserved already, but indeed C-x 8 might be a very good
>> choice, since it's already used to enter other non-ascii chars.

> C-x 8 1 and C-x 8 3 are already used.  That's why I proposed
> C-x # ... instead.  Or, do you mean this?

>   C-x 8 _SOME_NON_USED_KEY_ X X X X RET

I'd drop the C-x 8 1 / 2, C-x 8 1 / 4, and C-x 8 3 / 4, replace them
with something else, and then use C-x 8 NNNN RET.  Note that I'd also
imagine the NNNN to use hex, so it would also conflict with other
existing bindings, but I think we can force the use to use C-x 8 0 NNNN
for those cases.


        Stefan

Re: inputting characters by hexadigit

[Miles Bader]

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> I'd drop the C-x 8 1 / 2, C-x 8 1 / 4, and C-x 8 3 / 4, replace them
> with something else

Heh, I always try to type "C-x 8 / 1 2" etc for those chars anyway ... :-)
(somehow it makes more sense to me, given C-x 8's general "accent
prefix" style, and of course looks more like lisp :-).

> and then use C-x 8 NNNN RET.  Note that I'd also imagine the NNNN to
> use hex, so it would also conflict with other existing bindings, but I
> think we can force the use to use C-x 8 0 NNNN for those cases.

The only conflict, afaict, is "C-x 8 c", which currently gives "¢".

That binding could just be moved somewhere else; I don't imagine the
cent-sign is used very heavily...

-Miles

-- 
Any man who is a triangle, has thee right, when in Cartesian Space,
to have angles, which when summed, come to know more, nor no less,
than nine score degrees, should he so wish.  [TEMPLE OV THEE LEMUR]

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:

> I'd drop the C-x 8 1 / 2, C-x 8 1 / 4, and C-x 8 3 / 4, replace them

Was going to suggest C-x 8 h - half C-x 8 q - quarter C-x 8 t - three
quarters  ... but then noticed iso-transl-set-language, some languages
use C-x 8 h...

But there's also the issue that unicode includes a number of other
common fractions, so it might be nice to allow for supporting 'em as C-x
8 1/3 for ⅓ and similar for ⅝ ⅗ etc. in future.

> with something else, and then use C-x 8 NNNN RET. 

Should allow for N to NNNNNN, not just NNNN  .

> Note that I'd also imagine the NNNN to use hex, so it would also conflict with other
> existing bindings, 

Also N.B. conflicts change depending on M-x iso-transl-set-language   !

IMO might be better to just require a single character prefix, rather
than eating so much of C-x 8 space.

C-x 8 # 263a seems nice to me, though, my bad,
C-x 8 0 263a is perhaps easier to type for people who
aren't using british keyboards (# is an unshifted key for british/irish).

Is it a goal to eventually deprecate C-q entry?
Just wondering... C-q 0x263a  doesn't seem so bad...

Re: inputting characters by hexadigit

[Stefan Monnier]

> But there's also the issue that unicode includes a number of other
> common fractions, so it might be nice to allow for supporting 'em as C-x
> 8 1/3 for ⅓ and similar for ⅝ ⅗ etc. in future.

That's OK.  They don't need to use "1 / 3" and can use "/ 1 3" instead,
for example.

>> with something else, and then use C-x 8 NNNN RET. 
> Should allow for N to NNNNNN, not just NNNN  .

That's what the RET is for.

>> Note that I'd also imagine the NNNN to use hex, so it would also
>> conflict with other existing bindings, 

> Also N.B. conflicts change depending on M-x iso-transl-set-language!

Thanks for pointing it out.

> IMO might be better to just require a single character prefix, rather
> than eating so much of C-x 8 space.

> C-x 8 # 263a seems nice to me, though, my bad,
> C-x 8 0 263a is perhaps easier to type for people who
> aren't using british keyboards (# is an unshifted key for british/irish).

I think sticking to digits is the best choice since it's the only one
that guarantees there won't be any need to dance around with modifiers.

> Is it a goal to eventually deprecate C-q entry?

Very much so, yes.

> Just wondering... C-q 0x263a  doesn't seem so bad...

C-x 8 0 2 6 3 a is just as short.  And if we can drop the 0, it's
even shorter.


        Stefan

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:

> I think sticking to digits is the best choice since it's the only one
> that guarantees there won't be any need to dance around with modifiers.

[Well, except for those accursed french laptop keyboards, where numbers
need shift and there's not even a numeric keypad.]

Re: inputting characters by hexadigit

[Stefan Monnier]

>> I think sticking to digits is the best choice since it's the only one
>> that guarantees there won't be any need to dance around with modifiers.

> [Well, except for those accursed french laptop keyboards, where numbers
> need shift and there's not even a numeric keypad.]

Still no particular dancing: since the code has to be entered in hex
(that's the whole point), they'll have to press shift, but they don't
need to switch between pressed/unpressed to hit the preceding 8.
So it's still pretty much optimal for those keyboards.


        Stefan

Re: inputting characters by hexadigit

[David De La Harpe Golden]

Stefan Monnier wrote:

> Still no particular dancing: 

Okay, you're right.

C-x 8 does seem sensible:  It presently only does iso-8859-1, the
iso-transl-set-language "language" choices just select mildly different
schemes for entering iso-8859-1 characters - and of course, unicode
starts with iso-8859-1, so it's pretty purely expanding existing
functionality if you squint.

No need to even rename things I guess, just reencode and expand -
unicode is iso/iec 10646 after all, so iso-transl.el ,
iso-transl-ctl-x-8-map  etc. are all arguably still fine names.  :-)

Re: inputting characters by hexadigit

[Juri Linkov]

>> Just wondering... C-q 0x263a  doesn't seem so bad...
>
> C-x 8 0 2 6 3 a is just as short.  And if we can drop the 0, it's
> even shorter.

Using digits for the prefix key makes input too confusing.  In the middle
of the input sequence it is difficult to see what the current input number
is.  Is it "8 0 2 6" or "0 2 6 3" or, after dropping the 0, is it "8 2 6 3".
It is difficult for the user to visually parse such a key sequence and to
complete input.

Another drawback (that C-q currently has too) is an inability to edit
incomplete input.  This pushes us back to the era of dumb calculators
without the backspace key.

A more convenient method is to use the minibuffer to input the number.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Miles Bader]

Juri Linkov <juri@jurta.org> writes:
> Another drawback (that C-q currently has too) is an inability to edit
> incomplete input.  This pushes us back to the era of dumb calculators
> without the backspace key.
>
> A more convenient method is to use the minibuffer to input the number.

We could have C-x 8 <DIGIT> prompt using the minibuffer,
with <DIGIT> pre-stuffed into it...

   (defun read-and-insert-numeric-char (initial-input)
     (interactive (list (string last-command-char)))
     (insert 
      (string-to-number 
       (read-from-minibuffer "Character code (hex): " initial-input)
       16)))

   (dolist (ch '(?0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9 ?a ?b ?c ?d ?e ?f))
     (global-set-key (vector ?\C-x ?8 ch) 'read-and-insert-numeric-char))

[This is not entirely correct, since I'm not sure where the C-x 8 keymap
is located, so I'm just using the global map...]

-Miles

-- 
Somebody has to do something, and it's just incredibly pathetic that it
has to be us.  -- Jerry Garcia

Re: inputting characters by hexadigit

[Juri Linkov]

>> Another drawback (that C-q currently has too) is an inability to edit
>> incomplete input.  This pushes us back to the era of dumb calculators
>> without the backspace key.
>>
>> A more convenient method is to use the minibuffer to input the number.
>
> We could have C-x 8 <DIGIT> prompt using the minibuffer,
> with <DIGIT> pre-stuffed into it...
>
>    (defun read-and-insert-numeric-char (initial-input)
>      (interactive (list (string last-command-char)))
>      (insert 
>       (string-to-number 
>        (read-from-minibuffer "Character code (hex): " initial-input)
>        16)))
>
>    (dolist (ch '(?0 ?1 ?2 ?3 ?4 ?5 ?6 ?7 ?8 ?9 ?a ?b ?c ?d ?e ?f))
>      (global-set-key (vector ?\C-x ?8 ch) 'read-and-insert-numeric-char))
>
> [This is not entirely correct, since I'm not sure where the C-x 8 keymap
> is located, so I'm just using the global map...]

Such complexity has a tendency to cause a pain in the buffer.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Miles Bader]

Juri Linkov <juri@jurta.org> writes:
> Such complexity has a tendency to cause a pain in the buffer.

You didn't actually try it, did you?

-Miles

-- 
Alliance, n. In international politics, the union of two thieves who have
their hands so deeply inserted in each other's pockets that they cannot
separately plunder a third.

RE: inputting characters by hexadigit

[Drew Adams]

> How about using C-u (u for Unicode) as this way?
>
> C-u 0 x X X X X RET  for U+XXXX   (and C-0 x X X X X RET)
>
> C-u 0 x and C-0 x are noop now.

Please keep `C-u' for its intended purpose, which is already quite general. A
command might well use a zero prefix arg value for a specific behavior, and that
command might be bound to anything (even `x').

It is not just because some key binding (e.g. `C-u 0 x') is not predefined that
it should be predefined for Unicode entry.

This thread got off the ground because some thought that using an input method
could be too heavy for inputting Unicode chars. OK, keep looking for a better
(non "input method") input method, if you must. But please stay away from key
chords and existing key bindings.

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Tue, 22 Jul 2008 07:06:36 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> This thread got off the ground because some thought that using an input method
DA> could be too heavy for inputting Unicode chars. OK, keep looking for a better
DA> (non "input method") input method, if you must. But please stay away from key
DA> chords and existing key bindings.

I don't understand why we don't just provide an interactive function and
let the users pick the keybinding they like.  IMHO this is completely
useless for many users (considering input methods exist for particular
languages and we've talked about stacking multiple input methods), and
essential for much fewer people (who need to enter arbitrary Unicode
characters by number).

Ted

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Ted Zlatanov wrote:
> On Tue, 22 Jul 2008 07:06:36 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 
> 
> DA> This thread got off the ground because some thought that using an input method
> DA> could be too heavy for inputting Unicode chars. OK, keep looking for a better
> DA> (non "input method") input method, if you must. But please stay away from key
> DA> chords and existing key bindings.
> 
> I don't understand why we don't just provide an interactive function and
> let the users pick the keybinding they like.  IMHO this is completely
> useless for many users (considering input methods exist for particular
> languages and we've talked about stacking multiple input methods), and
> essential for much fewer people (who need to enter arbitrary Unicode
> characters by number).

Seems like a good suggestion to me if combined with Juri's latest 
suggestion.

RE: inputting characters by hexadigit

[Drew Adams]

> > DA> This thread got off the ground because some thought 
> > DA> that using an input method could be too heavy for
> > DA> inputting Unicode chars. OK, keep looking for a better
> > DA> (non "input method") input method, if you must. But 
> > DA> please stay away from key chords and existing key bindings.
> > 
> > I don't understand why we don't just provide an interactive 
> > function and let the users pick the keybinding they like.
> > IMHO this is completely useless for many users (considering
> > input methods exist for particular languages and we've talked
> > about stacking multiple input methods), and essential for much
> > fewer people (who need to enter arbitrary Unicode characters
> > by number).
> 
> Seems like a good suggestion to me if combined with Juri's
> latest suggestion.

+1

Re: inputting characters by hexadigit

[Juri Linkov]

>> DA> This thread got off the ground because some thought that using an input method
>> DA> could be too heavy for inputting Unicode chars. OK, keep looking for a better
>> DA> (non "input method") input method, if you must. But please stay away from key
>> DA> chords and existing key bindings.
>>
>> I don't understand why we don't just provide an interactive function and
>> let the users pick the keybinding they like.  IMHO this is completely
>> useless for many users (considering input methods exist for particular
>> languages and we've talked about stacking multiple input methods), and
>> essential for much fewer people (who need to enter arbitrary Unicode
>> characters by number).
>
> Seems like a good suggestion to me if combined with Juri's
> latest suggestion.

Do you mean the one based on Stefan's suggestion to use `C-x 8'?

It seems this is the simplest thing to do.  And one additional key
after `C-x 8' to activate the minibuffer doesn't make it worse on
usually quite long numbers.

After binding `C-x 8 RET' to `ucs-insert' we could improve `ucs-insert'
to parse more notations supported by Emacs like "#x..." and "?\x..."
and to input characters by Unicode name with completion.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

>>> I don't understand why we don't just provide an interactive function and
>>> let the users pick the keybinding they like.  IMHO this is completely

We have that already.  What I want is to deprecate/remove the C-q NNN
method of entry, and move it elsewhere.  I think that C-x 8 is the best
place for it.  We can also provide a default binding for ucs-insert on
C-x 8 RET so users don't have to choose one.

> After binding `C-x 8 RET' to `ucs-insert' we could improve `ucs-insert'
> to parse more notations supported by Emacs like "#x..." and "?\x..."
> and to input characters by Unicode name with completion.

Yes, that's the idea.


        Stefan

Re: inputting characters by hexadigit

[Juri Linkov]

>>>> I don't understand why we don't just provide an interactive function and
>>>> let the users pick the keybinding they like.  IMHO this is completely
>
> We have that already.  What I want is to deprecate/remove the C-q NNN
> method of entry, and move it elsewhere.  I think that C-x 8 is the best
> place for it.

From my experience using C-q to input a number was a pain in the echo area.
IMHO, using the minibuffer is more convenient that reading a string
with a sequence of `read-event'.  That's why I have doubts about moving
C-q NNN without changes to C-x 8.  Using `read-event' was necessary on
C-q because it has to read a character, but no more necessary on C-x 8.
So C-x 8 reading a number in the minibuffer seems more attractive.

However, if an exact equivalent of C-q is necessary on another key
then perhaps `C-x 8 C-q' is a better variant for that.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

>> From my experience using C-q to input a number was a pain in the echo area.
> IMHO, using the minibuffer is more convenient that reading a string
> with a sequence of `read-event'.  That's why I have doubts about moving

Then use C-x 8 RET.  No need for us to choose.


        Stefan

Re: inputting characters by hexadigit

[Juri Linkov]

>>> From my experience using C-q to input a number was a pain in the echo area.
>> IMHO, using the minibuffer is more convenient that reading a string
>> with a sequence of `read-event'.  That's why I have doubts about moving
>
> Then use C-x 8 RET.  No need for us to choose.

I installed a patch that changes `ucs-insert' to read Unicode names.
Anyone could improve it further with a discussed multi-level
completion mechanism.

A patch below binds `ucs-insert' to `C-x 8 RET'.  I tried to bind RET
to iso-transl-ctl-x-8-map in iso-transl.el, but then `C-x 8 RET' calls
`ucs-insert' as a non-interactive function (i.e. ignoring its `interactive'
declaration).  However, `ucs-insert' bound directly to `ctl-x-map' as below
works without problems.

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.335
diff -u -r1.335 mule-cmds.el
--- lisp/international/mule-cmds.el	29 Jul 2008 14:45:50 -0000	1.335
+++ lisp/international/mule-cmds.el	29 Jul 2008 15:50:47 -0000
@@ -2878,6 +2878,7 @@
       (error "Not a Unicode character code: 0x%X" arg))
   (insert-and-inherit arg))
 
+(define-key ctl-x-map "8\r" 'ucs-insert)
 
 ;; arch-tag: b382c432-4b36-460e-bf4c-05efd0bb18dc
 ;;; mule-cmds.el ends here

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Chong Yidong]

Juri Linkov <juri@jurta.org> writes:

>>>> From my experience using C-q to input a number was a pain in the echo area.
>>> IMHO, using the minibuffer is more convenient that reading a string
>>> with a sequence of `read-event'.  That's why I have doubts about moving
>>
>> Then use C-x 8 RET.  No need for us to choose.
>
> I installed a patch that changes `ucs-insert' to read Unicode names.
> Anyone could improve it further with a discussed multi-level
> completion mechanism.
>
> A patch below binds `ucs-insert' to `C-x 8 RET'.  I tried to bind RET
> to iso-transl-ctl-x-8-map in iso-transl.el, but then `C-x 8 RET' calls
> `ucs-insert' as a non-interactive function (i.e. ignoring its `interactive'
> declaration).  However, `ucs-insert' bound directly to `ctl-x-map' as below
> works without problems.

Look OK to me.

I'm curious, though: by default, when you use C-q to input a number, the
radix is octal; by changing read-quoted-char-radix, you can change this
to decimal or hexadecimal.  In contrast, ucs-insert always reads the
code in hexadecimal.  I don't use these commands myself, so I'm
wondering which is the preferred behavior (based on the defaults in
other applications, and so forth)?

Re: inputting characters by hexadigit

[Juri Linkov]

> I'm curious, though: by default, when you use C-q to input a number, the
> radix is octal; by changing read-quoted-char-radix, you can change this
> to decimal or hexadecimal.  In contrast, ucs-insert always reads the
> code in hexadecimal.  I don't use these commands myself, so I'm
> wondering which is the preferred behavior (based on the defaults in
> other applications, and so forth)?

I think we should improve `ucs-insert' to accept input in standard
Emacs Lisp notations: #o... for octal and #x... for hex.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Stefan Monnier]

> I think we should improve `ucs-insert' to accept input in standard
> Emacs Lisp notations: #o... for octal and #x... for hex.

Agreed.


        Stefan

Re: inputting characters by hexadigit

[Chong Yidong]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> I think we should improve `ucs-insert' to accept input in standard
>> Emacs Lisp notations: #o... for octal and #x... for hex.
>
> Agreed.

Sure, though it should be noted that this is an incompatible editing
change: with standing Lisp notation, 123 would be decimal, not octal.

Re: inputting characters by hexadigit

[Juri Linkov]

>>> I think we should improve `ucs-insert' to accept input in standard
>>> Emacs Lisp notations: #o... for octal and #x... for hex.
>>
>> Agreed.
>
> Sure, though it should be noted that this is an incompatible editing
> change: with standing Lisp notation, 123 would be decimal, not octal.

Hexadecimal numbers is the primary notation for Unicode code points
so it would be better to keep the current default.  In addition we can
support the hash notation that allows entering decimal numbers (though,
in less convenient way) as e.g. #10r182.  I propose the following patch
that keeps `ucs-insert' backward-compatible in regard of treating
numbers as hex by default, adds reading numbers with hash notations,
and improves error-handling of `ucs-insert'.

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.335
diff -c -r1.335 mule-cmds.el
*** lisp/international/mule-cmds.el	29 Jul 2008 14:45:50 -0000	1.335
--- lisp/international/mule-cmds.el	30 Jul 2008 15:29:11 -0000
***************
*** 2864,2883 ****
  as a number."
    (let* ((completion-ignore-case t)
  	 (input (completing-read prompt ucs-completions)))
!     (or (and (string-match "^[0-9a-fA-F]+$" input)
! 	     (string-to-number input 16))
! 	(cdr (assoc input (ucs-names))))))
  
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
    (interactive (list (read-char-by-name "Unicode (name or hex): ")))
!   (or (integerp arg)
        (setq arg (string-to-number arg 16)))
!   (if (or (< arg 0) (> arg #x10FFFF))
!       (error "Not a Unicode character code: 0x%X" arg))
    (insert-and-inherit arg))
  
  
  ;; arch-tag: b382c432-4b36-460e-bf4c-05efd0bb18dc
  ;;; mule-cmds.el ends here
--- 2864,2891 ----
  as a number."
    (let* ((completion-ignore-case t)
  	 (input (completing-read prompt ucs-completions)))
!     (cond
!      ((string-match "^[0-9a-fA-F]+$" input)
!       (string-to-number input 16))
!      ((string-match "^#" input)
!       (read input))
!      (t
!       (cdr (assoc input (ucs-names)))))))
  
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
    (interactive (list (read-char-by-name "Unicode (name or hex): ")))
!   (if (stringp arg)
        (setq arg (string-to-number arg 16)))
!   (cond
!    ((not (integerp arg))
!     (error "Not a Unicode character code: %S" arg))
!    ((< arg 0) (> arg #x10FFFF)
!     (error "Not a Unicode character code: 0x%X" arg)))
    (insert-and-inherit arg))
  
+ (define-key ctl-x-map "8\r" 'ucs-insert)
  
  ;; arch-tag: b382c432-4b36-460e-bf4c-05efd0bb18dc
  ;;; mule-cmds.el ends here

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Wed, 30 Jul 2008 18:29:19 +0300 Juri Linkov <juri@jurta.org> wrote: 

JL> Hexadecimal numbers is the primary notation for Unicode code points
JL> so it would be better to keep the current default.  In addition we can
JL> support the hash notation that allows entering decimal numbers (though,
JL> in less convenient way) as e.g. #10r182.  I propose the following patch
JL> that keeps `ucs-insert' backward-compatible in regard of treating
JL> numbers as hex by default, adds reading numbers with hash notations,
JL> and improves error-handling of `ucs-insert'.

How about these separate functions

ucs-insert-hex
ucs-insert-standard

as customization choices for ucs-insert?  Obviously one will call the
other, but the name will clearly tell the user what he's calling.  The
confusion will mostly happen the first time he uses the wrong input, at
which time the user will check the docs and understand the choices.

Ted

Re: describe-char should display the character's Unicode name

[Kenichi Handa]

In article <87tzeokrku.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> BTW, I remember that we once discussed about an ability to input
> characters by their Unicode names and codepoints.  But currently
> I can't find such a command, maybe I miss it.

You can use the input method "ucs" to input any Unicode
character by typing its codepoint.  A method by typing
Unicode name is not yet implemented.

> I see only the old command
> `ucs-insert'.  Is `ucs-insert' already obsolete?

We can make it obsolete now.

> As I understand,
> hex values of Unicode characters can be now entered by `C-q'?

Yes.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: describe-char should display the character's Unicode name

[Miles Bader]

Kenichi Handa <handa@m17n.org> writes:
> You can use the input method "ucs" to input any Unicode
> character by typing its codepoint.  A method by typing
> Unicode name is not yet implemented.

An input method is not a good solution because while input methods are
good for heavy use, they're quite annoying to juggle for occasional inputs.

>> `ucs-insert'.  Is `ucs-insert' already obsolete?
>
> We can make it obsolete now.

Please don't, until there's a better replacement.

[One possibility would be an addition to C-q to make it accept "# hex-digits" 
or something.]

-Miles

-- 
The trouble with most people is that they think with their hopes or
fears or wishes rather than with their minds.  -- Will Durant

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Fri, 18 Jul 2008 09:58:46 +0900 Miles Bader <miles@gnu.org> wrote: 

MB> Kenichi Handa <handa@m17n.org> writes:
>> You can use the input method "ucs" to input any Unicode
>> character by typing its codepoint.  A method by typing
>> Unicode name is not yet implemented.

MB> An input method is not a good solution because while input methods are
MB> good for heavy use, they're quite annoying to juggle for occasional inputs.

Agreed, there should be a function that gives all the Unicode characters
as input candidates and the user can select the one he wants quickly (I
don't see one currently).  An input method wouldn't provide completion
candidates, so the user has to remember all the names he needs.

Would a few thousand candidates be a problem for completion?  I've only
seen it with hundreds and it's not slow, but for this maybe maybe we can
break it down by block name and then character name.

Ted

RE: describe-char should display the character's Unicode name

[Drew Adams]

> Would a few thousand candidates be a problem for completion?  

No.

At least I see no problem on Windows XP with an average laptop, even with tens
of thousands of candidates.

Re: describe-char should display the character's Unicode name

[Stefan Monnier]

>>> You can use the input method "ucs" to input any Unicode
>>> character by typing its codepoint.  A method by typing
>>> Unicode name is not yet implemented.

MB> An input method is not a good solution because while input methods are
MB> good for heavy use, they're quite annoying to juggle for occasional inputs.

> Agreed, there should be a function that gives all the Unicode characters
> as input candidates and the user can select the one he wants quickly (I
> don't see one currently).  An input method wouldn't provide completion
> candidates, so the user has to remember all the names he needs.

Actually, we should have completion for input methods.  It would be handy
for things like the TeX input method.  Currently if I hit TAB after \a
I do get some sort of "list of possible completions", but it's displayed
as a tree which is kind of hard to read and I can select one of them
from the list.  Also TAB after \rig doesn't complete to \right as normal
completion would.

> Would a few thousand candidates be a problem for completion?  I've only
> seen it with hundreds and it's not slow, but for this maybe maybe we can
> break it down by block name and then character name.

I don't forsee any serious problem, except when when displaying the
complete list of completions (e.g. if you hit TAB before entering
any input).


        Stefan

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Fri, 18 Jul 2008 10:31:43 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Actually, we should have completion for input methods.  It would be handy
SM> for things like the TeX input method.  Currently if I hit TAB after \a
SM> I do get some sort of "list of possible completions", but it's displayed
SM> as a tree which is kind of hard to read and I can select one of them
SM> from the list.  Also TAB after \rig doesn't complete to \right as normal
SM> completion would.

It doesn't work for cyrillic-translit, for example, so I'd guess it
requires core changes to the quail code.  The other issue is that TAB
has a meaning while you're typing, so if you type /'TAB in
cyrillic-translit, you'd get two completion candidates (/' and /'')
instead of ’ followed by the TAB character.  I think that's less useful
to the user, since hitting TAB is so common in all Emacs modes.

Regardless I agree that it's useful to do this, because so many quail
input methods have more mappings than an user can easily remember.

Maybe it could be bound to M-TAB.  I don't know if it can be a callable
function, because it needs to be incorporated into the quail keystroke
flow.  

I also think Unicode characters in particular can use an
`ucs-insert-by-name' function instead of or in addition to an input
method, because it's such a useful operation and there are so many of
them.  The user wouldn't have to customize anything or call anything
complex, either, he'd just call that function and work with the
completion candidate list in the standard Emacs way--no special
keystrokes.  Input methods are a bit harder to customize and use (but of
course, they have many advantages over functions).

Ted

Re: describe-char should display the character's Unicode name

[Juri Linkov]

> SM> Actually, we should have completion for input methods.  It would be handy
> SM> for things like the TeX input method.  Currently if I hit TAB after \a
> SM> I do get some sort of "list of possible completions", but it's displayed
> SM> as a tree which is kind of hard to read and I can select one of them
> SM> from the list.  Also TAB after \rig doesn't complete to \right as normal
> SM> completion would.
>
> It doesn't work for cyrillic-translit, for example, so I'd guess it
> requires core changes to the quail code.

It works for the TeX input method because its arg `translation-keys'
of `quail-define-package' contains explicit key bindings:

    '(("\t" . quail-completion))

> The other issue is that TAB has a meaning while you're typing, so if
> you type /'TAB in cyrillic-translit, you'd get two completion
> candidates (/' and /'') instead of ’ followed by the TAB character.
> I think that's less useful to the user, since hitting TAB is so common
> in all Emacs modes.

It is difficult to find a completion key other than TAB.  But at least
quail-completion on TAB is not an unexpected key while input is not finished.

> Regardless I agree that it's useful to do this, because so many quail
> input methods have more mappings than an user can easily remember.
>
> Maybe it could be bound to M-TAB.  I don't know if it can be a callable
> function, because it needs to be incorporated into the quail keystroke
> flow.  

M-TAB is not a reliable key since often it is configured to switch windows.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Sat, 19 Jul 2008 03:45:23 +0300 Juri Linkov <juri@jurta.org> wrote: 

>> Maybe it could be bound to M-TAB.  I don't know if it can be a callable
>> function, because it needs to be incorporated into the quail keystroke
>> flow.  

JL> M-TAB is not a reliable key since often it is configured to switch windows.

Not when it's invoked with `ESC Tab' :)

Anyhow, I'll let you make this decision, I am OK with any key choice.

Ted

Re: describe-char should display the character's Unicode name

[Lennart Borgman (gmail)]

Ted Zlatanov wrote:
> On Sat, 19 Jul 2008 03:45:23 +0300 Juri Linkov <juri@jurta.org> wrote: 
> 
>>> Maybe it could be bound to M-TAB.  I don't know if it can be a callable
>>> function, because it needs to be incorporated into the quail keystroke
>>> flow.  
> 
> JL> M-TAB is not a reliable key since often it is configured to switch windows.
> 
> Not when it's invoked with `ESC Tab' :)


Which can't be used if you are using Viper.

Re: describe-char should display the character's Unicode name

[David De La Harpe Golden]

Lennart Borgman (gmail) wrote:

>> JL> M-TAB is not a reliable key since often it is configured to switch
>> windows.
>>
>> Not when it's invoked with `ESC Tab' :)
> 
> 
> Which can't be used if you are using Viper.
> 

cough C-M-i cough

Re: describe-char should display the character's Unicode name

[Juri Linkov]

>>> Maybe it could be bound to M-TAB.  I don't know if it can be a callable
>>> function, because it needs to be incorporated into the quail keystroke
>>> flow.
>
> JL> M-TAB is not a reliable key since often it is configured to switch windows.
>
> Not when it's invoked with `ESC Tab' :)
>
> Anyhow, I'll let you make this decision, I am OK with any key choice.

Hmm, it seems impossible to use `M-TAB' for quail-completion.

I tried to add '(("\M-\t" . quail-completion)) to the `translation-keys' arg
of `quail-define-package' but it fails with

   (error "Key sequence M-TAB starts with non-prefix key ")

and according to its docstring, only single key character is allowed
in additional key bindings:

    translation-keys specifies additional key bindings used while translation
    region is active.  It is an alist of single key character vs. corresponding
    command to be called.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Wed, 23 Jul 2008 01:57:27 +0300 Juri Linkov <juri@jurta.org> wrote: 

>>>> Maybe it could be bound to M-TAB.  I don't know if it can be a callable
>>>> function, because it needs to be incorporated into the quail keystroke
>>>> flow.
>> 
JL> M-TAB is not a reliable key since often it is configured to switch windows.
>> 
>> Not when it's invoked with `ESC Tab' :)
>> 
>> Anyhow, I'll let you make this decision, I am OK with any key choice.

JL> Hmm, it seems impossible to use `M-TAB' for quail-completion.

JL> I tried to add '(("\M-\t" . quail-completion)) to the `translation-keys' arg
JL> of `quail-define-package' but it fails with

JL>    (error "Key sequence M-TAB starts with non-prefix key ")

JL> and according to its docstring, only single key character is allowed
JL> in additional key bindings:

JL>     translation-keys specifies additional key bindings used while translation
JL>     region is active.  It is an alist of single key character vs. corresponding
JL>     command to be called.

I wish APL keyboards were still in use :)

Well, if it can't be done, then maybe TAB should be allowed, but only if
the user turns it on (globally, not per input method, since multiple
input methods will need to use it).  It should be off by default.  That
way TAB is still useful with Quail to do indentation, but if the user
knows he won't need TAB when in a Quail input method, he can customize
it to do `quail-completion'.  That seems like a decent compromise to me,
what do you think?

Ted

Re: describe-char should display the character's Unicode name

[Juri Linkov]

> Well, if it can't be done, then maybe TAB should be allowed, but only if
> the user turns it on (globally, not per input method, since multiple
> input methods will need to use it).  It should be off by default.  That
> way TAB is still useful with Quail to do indentation, but if the user
> knows he won't need TAB when in a Quail input method, he can customize
> it to do `quail-completion'.  That seems like a decent compromise to me,
> what do you think?

I agree but currently have no idea about implementation.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Lennart Borgman (gmail)]

[-- Attachment #1: Type: text/plain, Size: 1041 bytes --]

Juri Linkov wrote:
>> Well, if it can't be done, then maybe TAB should be allowed, but only if
>> the user turns it on (globally, not per input method, since multiple
>> input methods will need to use it).  It should be off by default.  That
>> way TAB is still useful with Quail to do indentation, but if the user
>> knows he won't need TAB when in a Quail input method, he can customize
>> it to do `quail-completion'.  That seems like a decent compromise to me,
>> what do you think?
> 
> I agree but currently have no idea about implementation.

I am not sure I understand the details here (or rather "I am sure I do 
not understand all the details here"), but things like this is what 
tabkey2.el tries to solve.

I have done some significant changes since I last proposed it. I have 
attached a new version.

Could you please see if what you want can fit in tabkey2.el? (The idea 
there is that normally the first Tab does indentation while the second 
Tab key can do some kind of completion - and S-Tab can choose what to 
complete.)

[-- Attachment #2: tabkey2.el --]
[-- Type: text/plain, Size: 56269 bytes --]

;;; tabkey2.el --- Use second tab key pressed for what you want
;;
;; Author: Lennart Borgman (lennart O borgman A gmail O com)
;; Created: 2008-03-15T14:40:28+0100 Sat
(defconst tabkey2:version "1.31")
;; Last-Updated: 2008-07-21T22:24:55+0200 Mon
;; URL: http://www.emacswiki.org/cgi-bin/wiki/tabkey2.el
;; Keywords:
;; Compatibility:
;;
;; Features that might be required by this library:
;;
;;   `appmenu', `cl'.
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;;; Commentary:
;;
;; The tab key is in Emacs often used for indentation.  However if you
;; press the tab key a second time and Emacs tries to do indentation
;; again, then usually nothing exciting will happen.  Then why not use
;; second tab key in a row for something else?
;;
;; Commonly used completion functions in Emacs is often bound to
;; something corresponding to Alt-Tab.  Unfortunately this is unusable
;; if you have a window manager that have an apetite for it (like that
;; on MS Windows for example, and several on GNU/Linux).
;;
;; Then using the second tab key press for completion might be a good
;; choice and perhaps also easy to remember.
;;
;; This little library tries to make it easy to do use the second tab
;; press for completion.  Or you can see this library as a swizz army
;; knife for the tab key ;-)
;;
;; See `tabkey2-mode' for more information.
;;
;;
;; This is a generalization of an idea Sebastien Rocca Serra once
;; presented on Emacs Wiki and called "Smart Tab".  (It seems like
;; many others have also been using Tab for completion in one way or
;; another for years.)
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;;; Change log:
;;
;; Version 1.04:
;; - Add overlay to display state after first tab.
;;
;; Version 1.05:
;; - Fix remove overlay problem.
;;
;; Version 1.06:
;; - Add completion function choice.
;; - Add support for popcmp popup completion.
;;
;; Version 1.07:
;; - Add informational message after first tab.
;;
;; Version 1.08:
;; - Give better informational message after first tab.
;;
;; Version 1.09:
;; - Put flyspell first.
;;
;; Version 1.09:
;; - Give the overlay higher priority.
;;
;; Version 1.10:
;; - Correct tabkey2-completion-functions.
;; - Add double-tab for modes where tab can not be typed again.
;; - Use better condition for when completion can be done, so that it
;;   can be done later while still on the same line.
;; - Add a better message handling for the "Tab completion state".
;; - Add C-g break out of the "Tab completion state".
;; - Add faces for highlight.
;; - Make it work in custom mode buffers.
;; - Fix documentation for `tabkey2-first'
;;
;; Version 1.11:
;; - Don't call chosen completion function directly.  Instead make it
;;   default for current buffer.
;;
;; Version 1.12:
;; - Simplify code.
;; - Add help to C-f1 during "Tab completion state".
;; - Fix documentation basics.
;; - Add customization of state message and line marking.
;; - Fix handling of double-Tab modes.
;; - Make user interaction better.
;; - Handle read-only in custom buffers better.
;; - Add more flexible check for if completion function is active.
;; - Support predictive mode.
;; - Reorder and simplify.
;;
;; Version 1.13:
;; - Add org-mode to the double-tab gang.
;; - Make it possible to use double-tab in normal buffers.
;; - Add cycling through completion functions to S-tab.
;;
;; Version 1.14:
;; - Fix bug in handling of read-only.
;; - Show completion binding in help message.
;; - Add binding to make current choice buffer local when cycling.
;;
;; Version 1.15:
;; - Fix problem at buffer end.
;; - Add S-tab to enter completion state without indentation.
;; - Add backtab bindings too for this.
;; - Remove double-tab, S-tab is better.
;; - Add list of modes that uses more tabs.
;; - Add list of modes that uses tab only for completion.
;; - Move first overlay when indentation changes.
;; - Make mark at line beginning 1 char long.
;;
;; Version 1.16:
;; - Don't call tab function when alternate key is pressed.
;;
;; Version 1.17:
;; - Let alternate key cycle completion functions instead of complete.
;; - Bind backtab.
;; - Fix bug when only one completion funciton was available.
;; - Fix bug when alt key and major without fix indent.
;;
;; Version 1.18:
;; - Add popup style messages.
;; - Add delay to first message.
;; - Use different face for indicator on line and message.
;; - Use different face for echo area and popup messages.
;; - Add anything to completion functions.
;; - Put help funciton on f1.
;; - Always bind alternate key to cycle.
;; - Change defcustoms to simplify (excuse me).
;; - Work around end of buffer problems.
;; - Work around start of buffer problems.
;; - Assure popup messages are visible.
;; - Reorder code in more logical order.
;;
;; Version 1.19:
;; - Make overlay keymap end advance.
;; - Remove overlay keymap parent.
;;
;; Version 1.20:
;; - Fix bug on emtpy line.
;; - Fix some text problems.
;; - Make f1 c/k work in tab completion state.
;;
;; Version 1.20:
;; - Fixed bug in overlay removal.
;;
;; Version 1.21:
;; - Fixed bug in minibuffer setup.
;;
;; Version 1.22:
;; - Honour widget-forward, button-forward.
;;
;; Version 1.23:
;; - Remove binding of shift tab.
;; - Check if use-region-p is defined.
;;
;; Version 1.24:
;; - Add option for completion state mode line marker.
;; - Fix bug in tabkey2-show-completion-functions.
;; - Move off completion point cancels completion state.
;; - Fix bugs in help.
;; - Try to fix some problems with invisible text, at least in
;;   org-mode.
;; - Restore window config, completions often leaves without.
;;
;; Version 1.25:
;; - Fix bug in tabkey2-completion-state-p.
;;
;; Version 1.26:
;; - Make tabkey2-mode a buffer local mode.
;; - Add tabkey2-global-mode.
;; - Fix some bugs.
;;
;; Version 1.27:
;; - Fix some bugs in customization.
;;
;; Version 1.28:
;; - Use invisible-p.
;;
;; Version 1.29:
;; - Remove tabkey2-global-mode because of problem with minibuffers.
;;
;; Version 1.30:
;; - Add Semantic's smart completion to completion functions.
;;   (Thanks Eric.)
;;
;; Version 1.31:
;; - Added yasnippet and pabbrev completion functions. (Thanks Eric.)
;; - Reorded completion functions.
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;;; Known bugs
;;
;; - Maybe problems with comint shell.
;; - Does not check visibility very carefully.
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License as
;; published by the Free Software Foundation; either version 2, or
;; (at your option) any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with this program; see the file COPYING.  If not, write to
;; the Free Software Foundation, Inc., 51 Franklin Street, Fifth
;; Floor, Boston, MA 02110-1301, USA.
;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;;; Code:

(eval-when-compile (require 'cl))
;;(require 'popcmp nil t)
(require 'appmenu nil t)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Custom

(defgroup tabkey2 nil
  "Customization of second tab key press."
  :group 'nxhtml
  :group 'convenience)

(defface tabkey2-highlight-line
  '((t :inherit highlight))
  "Face for marker on current line."
  :group 'tabkey2)

(defface tabkey2-highlight-message
  '((t :inherit tabkey2-highlight-line))
  "Face for messages in echo area."
  :group 'tabkey2)

(defface tabkey2-highlight-popup
  '((default :box t :inherit tabkey2-highlight-message)
    (((class color) (background light)) :foreground "black")
    (((class color) (background dark)) :foreground "yellow"))
  "Face for popup messages."
  :group 'tabkey2)

(defcustom tabkey2-show-mark-on-active-line t
  "Show mark on active line if non-nil.
This mark is shown during 'Tab completion state'."
  :type 'boolean
  :group 'tabkey2)

(defvar tabkey2-completion-lighter nil)
(defcustom tabkey2-completion-lighter-on nil
  "Mode line lighter for function `tabkey2-completion-state-mode'."
  :type 'boolean
  :set (lambda (symbol value)
         (set-default symbol value)
         (setq tabkey2-completion-lighter (if value " Tab2" nil))
         (setq minor-mode-alist
               (assq-delete-all 'tabkey2-completion-state-mode
                                minor-mode-alist)))
  :group 'tabkey2)

(defcustom tabkey2-show-message-on-enter 2.0
  "If non-nil show message when entering 'Tab completion state'.
If value is a number then delay message that number of seconds."
  :type '(choice (const :tag "Don't show" nil)
                 (const :tag "Show at once" t)
                 (float :tag "Show, but delayed (seconds)"))
  :group 'tabkey2)


;; (setq tabkey2-message-style 'popup)
;; (setq tabkey2-message-style 'echo-area)
(defcustom tabkey2-message-style 'popup
  "How to show messages."
  :type '(choice (const :tag "Popup" popup)
                 (const :tag "Echo area" echo-area))
  :group 'tabkey2)

(defcustom tabkey2-in-minibuffer nil
  "If non-nil use command `tabkey2-mode' also in minibuffer."
  :type 'boolean
  :group 'tabkey2)

(defcustom tabkey2-in-appmenu t
  "Show a completion menu in command `appmenu-mode' if t."
  :type 'boolean
  :set (lambda (sym val)
         (set-default sym val)
         (when (fboundp 'appmenu-add)
           (if val
               (appmenu-add 'tabkey2 nil t "Completion" 'tabkey2-appmenu)
             (appmenu-remove 'tabkey2))))
  :group 'tabkey2)

(defcustom tabkey2-completion-functions
  '(
    ;; Temporary things
    ("Spell check word" flyspell-correct-word-before-point)
    ;; Main mode related, often used
    ("Semantic Smart Completion" senator-complete-symbol senator-minor-mode)
    ("nXML completion" nxml-complete)
    ("Complete Emacs symbol" lisp-complete-symbol)
    ("Widget complete" widget-complete)
    ("Comint Dynamic Complete" comint-dynamic-complete)
    ("PHP completion" php-complete-function)
    ("Tags completion" complete-symbol)
    ;; General word completion
    ("Predictive word" complete-word-at-point predictive-mode)
    ("Predictive abbreviations" pabbrev-expand-maybe)
    ("Dynamic word expansion" dabbrev-expand)
    ("Ispell complete word" ispell-complete-word)
    ;; Snippets
    ("Yasnippet" yas/expand)
    ;; The catch all
    ("Anything" anything (commandp 'anything))
    )
  "List of completion functions.
The first 'active' entry in this list is normally used during the
'Tab completion state' by `tabkey2-complete'.  An entry in the
list should have either of these forms

  \(DESCRIPTION FUNCTION)
  \(DESCRIPTION FUNCTION EXPRESSION)

The entry is considered active if:

-  The symbol FUNCTION is bound to a function

and

- this function has a key binding at point,

  or

  the elisp expression EXPRESSION evaluates to non-nil.
  If it is a single symbol then its variable value is used,
  otherwise the elisp form is evaled.

When choosing with `tabkey2-cycle-completion-functions'
only the currently active entry in this list are shown."
  :type '(repeat (list string (choice (command :tag "Currently known command")
                                      (symbol  :tag "Command not known yet"))
                       (choice (const :tag "Active only if it has a key binding at point" nil)
                               (sexp :tag "Elisp, if evals to non-nil then active"))))
  :group 'tabkey2)

;; Use emulation mode map for first Tab key
(defconst tabkey2-mode-emul-map (make-sparse-keymap)
  "This keymap just binds tab and alternate key all the time.
By default this binds Tab to `tabkey2-first'.  The actual keys
bound are in `tabkey2-first-key' and `tabkey2-alternate-key'.")

(defvar tabkey2--emul-keymap-alist nil)

;; (setq tabkey2-keymap-overlay nil)
(defconst tabkey2-completion-state-emul-map
  (let ((map (make-sparse-keymap)))
    (define-key map [(control ?c) tab]    'tabkey2-make-current-sticky)

    ;;(define-key map tabkey2-alternate-key 'tabkey2-cycle-completion-functions)
    (define-key map [backtab]             'tabkey2-cycle-completion-functions)

    (define-key map [(control f1)]        'tabkey2-completion-function-help)
    (define-key map [(meta f1)]           'tabkey2-show-completion-functions)
    (define-key map [f1]                  'tabkey2-completion-state-help)

    (define-key map [(control ?g)]        'tabkey2-completion-state-off)
    (define-key map [tab]                 'tabkey2-complete)
    map)
  "This keymap is for `tabkey2-keymap-overlay'.")

(defun tabkey2-bind-keys (first-key alternate-key)
  (let ((mode-map tabkey2-mode-emul-map)
        (comp-map tabkey2-completion-state-emul-map))
    ;; First key
    (when (and (boundp 'tabkey2-first-key)
	       tabkey2-first-key)
      (define-key mode-map tabkey2-first-key nil))
    (when first-key
      (define-key mode-map first-key 'tabkey2-first))
    ;; Alternate key
    (when (and (boundp 'tabkey2-alternate-key)
	       tabkey2-alternate-key)
      (define-key mode-map tabkey2-alternate-key nil)
      (define-key comp-map tabkey2-alternate-key nil))
    (when alternate-key
      (define-key mode-map alternate-key 'tabkey2-cycle-completion-functions)
      (define-key comp-map alternate-key 'tabkey2-cycle-completion-functions))
    (when (and (boundp 'tabkey2-completion-state-mode)
	       tabkey2-completion-state-mode)
      (tabkey2-completion-state-mode -1)
      (tabkey2-completion-state-mode 1))))

(defcustom tabkey2-first-key [tab]
  "First key, first time indents, more invocations completes.
This key is always bound to `tabkey2-first'."
  :set (lambda (sym val)
         (tabkey2-bind-keys val (when (boundp 'tabkey2-alternate-key) tabkey2-alternate-key))
         (set-default sym val))
  :type 'key-sequence
  :group 'tabkey2)

(defcustom tabkey2-alternate-key [f8]
  "Alternate key, bound to cycle and show completion functions.
This key is always bound to `tabkey2-cycle-completion-functions'."
  :set (lambda (sym val)
         (tabkey2-bind-keys (when (boundp 'tabkey2-first-key) tabkey2-first-key) val)
         (set-default sym val))
  :type 'key-sequence
  :group 'tabkey2)

(tabkey2-bind-keys tabkey2-first-key tabkey2-alternate-key)

(defcustom tabkey2-modes-that-use-more-tabs
  '(python-mode
    haskell-mode
    makefile-mode
    org-mode
    Custom-mode
    ;; other
    cmd-mode
    )
  "In those modes use must use S-Tab to start completion state.
In those modes pressing Tab several types may make sense so you
can not go into 'Tab completion state' just because one Tab has
been pressed.  Instead you use S-Tab to go into that state.
After that Tab does completion.

You can do use S-Tab in other modes too if you want too."
  :type '(repeat (choice (command :tag "Currently known command")
                         (symbol  :tag "Command not known yet")))
  :group 'tabkey2)

(defcustom tabkey2-modes-that-just-complete
  '(shell-mode)
  "Tab is only used for completion in these modes.
Therefore `tabkey2-first' just calls the function on Tab."
  :type '(repeat (choice (command :tag "Currently known command")
                         (symbol  :tag "Command not known yet")))
  :group 'tabkey2)

;;(setq tabkey2-use-popup-menus nil)
;; (defcustom tabkey2-use-popup-menus (when (featurep 'popcmp) t)
;;   "Use pop menus if available."
;;   :type 'boolean
;;   :group 'tabkey2)

(defvar tabkey2-preferred nil
  "Preferred function for second tab key press.")
(make-variable-buffer-local 'tabkey2-preferred)
(put 'tabkey2-preferred 'permanent-local t)

(defvar tabkey2-fallback nil
  "Fallback function for second tab key press.")
(make-variable-buffer-local 'tabkey2-fallback)
(put 'tabkey2-fallback 'permanent-local t)


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; State

(defvar tabkey2-overlay nil
  "Show when tab key 2 action is to be done.")
(defvar tabkey2-keymap-overlay nil
  "Hold the keymap for tab key 2.")

(defun tabkey2-completion-state-p ()
  "Return t if Tab completion state should continue.
Otherwise return nil."
  (when (and (eq (current-buffer) (overlay-buffer tabkey2-keymap-overlay))
             (eq (overlay-get tabkey2-keymap-overlay 'window) (selected-window)))
    (let* ((start (overlay-start tabkey2-keymap-overlay))
           (end   (overlay-end   tabkey2-keymap-overlay))
           (chars (append (buffer-substring-no-properties start end) nil)))
      (and (not (memq ?\n chars))
           (<= start (point))
           (<= (point) end)))))

(defvar tabkey2-current-tab-info nil
  "Saved information message for Tab completion state.")
(defvar tabkey2-current-tab-function nil
  "Tab completion state current completion function.")

(defun tabkey2-read-only-p ()
  "Return non-nil if buffer seems to be read-only at point."
  (or buffer-read-only
      (get-char-property (min (+ 0 (point)) (point-max)) 'read-only)
      (let ((remap (command-remapping 'self-insert-command (point))))
        (memq remap '(Custom-no-edit)))))

;;;; Minor mode active after first tab

(defun tabkey2-move-overlays ()
  "Move overlays that mark the state and carries the state keymap."
  (let* ((beg (let ((inhibit-field-text-motion t))
                (line-beginning-position)))
         (ind (current-indentation))
         (end (+ beg 1)) ;(if (> ind 0) ind 1)))
         (inhibit-read-only t))
    (unless tabkey2-overlay
      (setq tabkey2-overlay (make-overlay beg end)))
    ;; Fix-me: gets some strange errors, try avoid moving:
    (unless (and (eq (current-buffer) (overlay-buffer tabkey2-overlay))
                 (= beg (overlay-start tabkey2-overlay))
                 (= end (overlay-end   tabkey2-overlay)))
      (move-overlay tabkey2-overlay beg end (current-buffer)))
    ;; Give it a high priority, it is very temporary
    (overlay-put tabkey2-overlay 'priority 1000)
    (if tabkey2-show-mark-on-active-line
        (progn
          (overlay-put tabkey2-overlay 'face 'tabkey2-highlight-line)
          (overlay-put tabkey2-overlay 'help-echo
                       "This highlight shows that Tab completion state is on"))
      (overlay-put tabkey2-overlay 'face nil)
      (overlay-put tabkey2-overlay 'help-echo nil)))
  ;; The keymap overlay
  (let ((beg (line-beginning-position))
        (end (line-end-position)))
    ;;(when (= end (point-max)) (setq end (1+ end)))
    (setq beg (point))
    (setq end (point))

    (unless tabkey2-keymap-overlay
      ;; Make the rear of the overlay advance so that the keymap works
      ;; at the end of a line and the end of the buffer.
      (setq tabkey2-keymap-overlay (make-overlay 0 0 nil nil t)))
    (overlay-put tabkey2-keymap-overlay 'priority 1000)
    ;;(overlay-put tabkey2-keymap-overlay 'face 'secondary-selection)
    (overlay-put tabkey2-keymap-overlay 'keymap
                 tabkey2-completion-state-emul-map)
    (overlay-put tabkey2-keymap-overlay 'window (selected-window))
    (move-overlay tabkey2-keymap-overlay beg end (current-buffer))))

(defun tabkey2-is-active (fun chk)
  "Return t FUN is active.
Return t if CHK is a symbol with non-nil value or a form that
evals to non-nil.

Otherwise return t if FUN has a key binding at point."
  (or (if (symbolp chk)
          (when (boundp chk) (symbol-value chk))
        (eval chk))
      (let* ((emulation-mode-map-alists
              ;; Remove keymaps from tabkey2 in this copy:
              (delq 'tabkey2--emul-keymap-alist
                    (copy-sequence emulation-mode-map-alists)))
             (keys (tabkey2-symbol-keys fun))
             kb-bound)
        (dolist (key keys)
          (unless (memq (car (append key nil))
                        '(menu-bar))
            (setq kb-bound t)))
        kb-bound)))

(defun tabkey2-is-active-p (fun)
  "Return FUN is active.
Look it up in `tabkey2-completion-functions' to find out what to
check and return the value from `tabkey2-is-active'."
  (let ((chk (catch 'chk
               (dolist (rec tabkey2-completion-functions)
                 (when (eq fun (nth 1 rec))
                   (throw 'chk (nth 2 rec)))))))
    (tabkey2-is-active fun chk)))

(defun tabkey2-first-active-from-completion-functions ()
  "Return first active completion function.
Look in `tabkey2-completion-functions' for the first function
that has an active key binding."
  (catch 'active-fun
    (dolist (rec tabkey2-completion-functions)
      (let ((fun (nth 1 rec))
            (chk (nth 2 rec)))
        (when (tabkey2-is-active fun chk)
          (throw 'active-fun fun))))))

(defun tabkey2-get-default-completion-fun ()
  "Return the default completion function.
See `tabkey2-first' for the list considered."
  (or (when (and tabkey2-chosen-completion-function
                 (tabkey2-is-active-p
                  tabkey2-chosen-completion-function))
        tabkey2-chosen-completion-function)
      tabkey2-preferred
      (tabkey2-first-active-from-completion-functions)
      tabkey2-fallback))

(defvar tabkey2-completion-state-mode nil)
;;(make-variable-buffer-local 'tabkey2-completion-state-mode)
(defun tabkey2-completion-state-mode (arg)
  "Tab completion state minor mode.
This pseudo-minor mode holds the 'Tab completion state'.  When this
minor mode is on completion key bindings are available.

With ARG a positive number turn on, otherwise turn off this minor
mode.

See `tabkey2-first' for more information."
  ;;(assq-delete-all 'tabkey2-completion-state-mode minor-mode-alist)
  (unless (assoc 'tabkey2-completion-state-mode minor-mode-alist)
    ;;(setq minor-mode-alist (cons '(tabkey2-completion-state-mode " Tab2")
    (setq minor-mode-alist (cons (list 'tabkey2-completion-state-mode
                                       tabkey2-completion-lighter)
                                 minor-mode-alist)))
  (let ((emul-map (cdr (car tabkey2--emul-keymap-alist)))
        (old-wincfg tabkey2-completion-state-mode))
    (setq tabkey2-completion-state-mode (when (and (numberp arg)
                                                   (> arg 0))
                                          ;;t
                                          (current-window-configuration)
                                          ))
    (if tabkey2-completion-state-mode
        (progn
          ;; Set default completion function
          (tabkey2-make-message-and-set-fun
           (tabkey2-get-default-completion-fun))
          ;; Message
          ;;(setq tabkey2-message-is-shown nil)
          (when tabkey2-show-message-on-enter
            (tabkey2-show-current-message
             (when (numberp tabkey2-show-message-on-enter)
               tabkey2-show-message-on-enter)))
          ;; Move overlays
          (tabkey2-move-overlays)
          ;; Work around eob keymap problem ...
          ;;(set-keymap-parent emul-map (overlay-get tabkey2-keymap-overlay
          ;;                                         'keymap))
          ;; Set up for pre/post-command-hook
          (add-hook 'pre-command-hook 'tabkey2-pre-command)
          (add-hook 'post-command-hook 'tabkey2-post-command))
      ;;(set-keymap-parent emul-map nil)
      (when (and old-wincfg
                 tabkey2-keymap-overlay
                 (eq (overlay-get tabkey2-keymap-overlay 'window) (selected-window))
                 (not (active-minibuffer-window)))
        (set-window-configuration old-wincfg))
      (let ((inhibit-read-only t))
        (when tabkey2-keymap-overlay
          (delete-overlay tabkey2-keymap-overlay))
        (when tabkey2-overlay
          (delete-overlay tabkey2-overlay)))
      (remove-hook 'pre-command-hook 'tabkey2-pre-command)
      (remove-hook 'post-command-hook 'tabkey2-post-command)
      (tabkey2-overlay-message nil)
      ;;(message "")
      )))

(defun tabkey2-completion-state-off ()
  "Quit Tab completion state."
  (interactive)
  (tabkey2-completion-state-mode -1)
  (message "Quit"))

(defvar tabkey2-message-is-shown nil)
(defun tabkey2-message-is-shown ()
  (case tabkey2-message-style
    ('popup
     (when tabkey2-overlay-message
       (overlay-buffer tabkey2-overlay-message)))
    ('echo-area
     (get (current-message) 'tabkey2))))

(defun tabkey2-pre-command ()
  "Run this in `pre-command-hook'.
Check if message is shown.
Remove overlay message.
Cancel delayed message."
  ;;(message "=====> tabkey2-pre-command")
  (condition-case err
      (progn
        (setq tabkey2-message-is-shown (tabkey2-message-is-shown))
        ;;(message "tabkey2-overlay-message=%s, is-shown=%s" tabkey2-overlay-message tabkey2-message-is-shown)
        (tabkey2-overlay-message nil)
        (tabkey2-cancel-delayed-message)
        ;;(message "here buffer=%s, this-command=%s" (current-buffer) this-command)
        )
    (error (message "tabkey2 pre: %s" (error-message-string err)))))

(defun tabkey2-post-command ()
  "Turn off Tab completion state if not feasable any more.
This is run in `post-command-hook' after each command."
  (condition-case err
      (save-match-data
        ;; Delayd messages
        (if (not (tabkey2-completion-state-p))
            (tabkey2-completion-state-mode -1)
          (tabkey2-move-overlays)))
    (error (message "tabkey2 post: %s" (error-message-string err)))))

(defun tabkey2-minibuffer-setup ()
  "Activate/deactivate function `tabkey2-mode' in minibuffer."
  (set (make-local-variable 'tabkey2-mode)
       (and tabkey2-mode
            tabkey2-in-minibuffer))
  (unless tabkey2-mode
    (set (make-local-variable 'emulation-mode-map-alists)
         (delq 'tabkey2--emul-keymap-alist
               (copy-sequence emulation-mode-map-alists)))))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Message functions

;; Fix-me: is something like this included in Emacs now? reveal.el
;; gets it wrong.
(unless (fboundp 'invisible-p)
  (defun invisible-p (pos)
    "Return non-nil if the character after POS is currently invisible."
    (let ((prop
           (get-char-property pos 'invisible)))
      (if (eq buffer-invisibility-spec t)
          prop
        (if (listp prop)
            (catch 'invis
              (dolist (p prop)
                (when (or (memq p buffer-invisibility-spec)
                          (assq p buffer-invisibility-spec))
                  (throw 'invis t))))
          (or (memq prop buffer-invisibility-spec)
              (assq prop buffer-invisibility-spec)))))))

(defvar tabkey2-overlay-message nil)

;; (defun test-scroll ()
;;   (interactive)
;;   (setq debug-on-error t)
;;   (let* ((buffer-name "test-scroll")
;;         (buffer (get-buffer buffer-name)))
;;     (when buffer (kill-buffer buffer))
;;     (setq buffer (get-buffer-create buffer-name))
;;     (switch-to-buffer buffer)
;;     (message "here 1") (sit-for 1)
;;     (condition-case err
;;         (scroll-up 1)
;;       (error (message "scroll-up error: %s" err)
;;              (sit-for 1)))
;;     (message "here 2") (sit-for 1)
;;     (scroll-up 1)
;;     (message "here 3") (sit-for 1)
;;     ))

(defun tabkey2-overlay-message (txt)
  "Display TXT below or above current line using an overlay."
  ;;(setq tabkey2-message-is-shown txt)
  (if (not txt)
      (when tabkey2-overlay-message
        (delete-overlay tabkey2-overlay-message)
        (setq tabkey2-overlay-message nil))
    (let ((ovl tabkey2-overlay-message)
          (column (current-column))
          (txt-len (length txt))
          (here (point))
          beg end
          (before "")
          (after "")
          ovl-str too-much
          (is-eob (eobp))
          (direction 1))
      (unless ovl (setq ovl (make-overlay 0 0)))
      (when tabkey2-overlay-message
        (delete-overlay tabkey2-overlay-message))
      (setq tabkey2-overlay-message ovl)

      (when is-eob
        (setq direction -1))
      (when (and (/= (point-min) (window-start))
                 (not (pos-visible-in-window-p (min (point-max) (1+ (line-end-position))))))
        ;; Go back inside window to avoid aggressive scrolling:
        (forward-line -1)
        (scroll-up 1)
        (forward-line 1))
      (forward-line direction)
      ;; Fix-me: Emacs bug workaround
      (if (when (< 1 (point))
            (invisible-p (1- (line-end-position))))
          (progn
            (goto-char here)
            (tabkey2-echo-area-message txt))
        ;; Fix-me: Does this really do anything now:
        (when (invisible-p (point))
          (while (invisible-p (point))
            (forward-line direction)))
        (setq beg (line-beginning-position))
        (setq end (line-end-position))

        (if (or (invisible-p beg) (invisible-p end))
          ;; Give up, do not fight invisibility:
            (progn
              (tabkey2-overlay-message nil)
              (tabkey2-echo-area-message txt))

          ;; string before
          (move-to-column column)
          (setq before (buffer-substring beg (point)))
          (when (< (current-column) column)
            (setq before
                  (concat before
                          (make-string (- column (current-column)) ? ))))
          (setq too-much (- (+ 1 txt-len (length before))
                            (window-width)))
          (when (> too-much 0)
            (setq before (substring before 0 (- too-much))))

          (unless (> too-much 0)
            (move-to-column (+ txt-len (length before)))
            (setq after (buffer-substring (point) end)))

          (setq ovl-str (concat before
                                (propertize txt 'face 'tabkey2-highlight-popup)
                                after
                                ))

          (overlay-put ovl 'after-string ovl-str)
          (overlay-put ovl 'display "")
          (overlay-put ovl 'window (selected-window))
          (move-overlay ovl beg end (current-buffer)))

        (goto-char here)
        ))))

;; Fix-me: This was not usable IMO. Too much flickering.
;; (defun tabkey2-tooltip (txt)
;;   (let* ((params tooltip-frame-parameters)
;;          (coord (car (point-to-coord (point))))
;;          (left (car coord))
;;          (top  (cadr coord))
;;          tooltip-frame-parameters
;;          )
;;     ;; Fix-me: how do you get char height??
;;     (setq top (+ top 50))
;;     (setq params (tooltip-set-param params 'left left))
;;     (setq params (tooltip-set-param params 'top top))
;;     (setq params (tooltip-set-param params 'top top))
;;     (setq tooltip-frame-parameters params)
;;     (tooltip-hide)
;;     (tooltip-show txt nil)))

(defun tabkey2-echo-area-message (txt)
  "Show TXT in the echo area with a special face.
Shown with the face `tabkey2-highlight-message'."
  (message "%s" (propertize txt
                            'face 'tabkey2-highlight-message
                            'tabkey2 t)))

(defun tabkey2-deliver-message (txt)
  "Show message TXT to user."
  (case tabkey2-message-style
    (popup (tabkey2-overlay-message txt))
    (t (tabkey2-echo-area-message txt))))

(defun tabkey2-timer-deliver-message (txt where)
  "Show message TXT to user.
Protect from errors cause this is run during a timer."
  (when (and tabkey2-completion-state-mode
             (equal (point-marker) where))
    (condition-case err
        (tabkey2-deliver-message txt)
      (error (message "tabkey2-timer-deliver-message: %s"
                      (error-message-string err))))))

(defvar tabkey2-delayed-timer nil)

(defun tabkey2-cancel-delayed-message ()
  "Cancel delayed message."
  (when tabkey2-delayed-timer
    (cancel-timer tabkey2-delayed-timer)
    (setq tabkey2-delayed-timer)))

(defun tabkey2-maybe-delayed-message (txt delay)
  "Show message TXT, delay it if DELAY is non-nil."
  (if delay
      (setq tabkey2-delayed-timer
            (run-with-idle-timer
             delay nil
             'tabkey2-timer-deliver-message txt (point-marker)))
    (tabkey2-deliver-message txt)))

(defun tabkey2-message (delay format-string &rest args)
  "Show, if DELAY delayed, otherwise immediately message.
FORMAT-STRING and ARGS are like for `message'."
  (let ((txt (apply 'format format-string args)))
    (tabkey2-maybe-delayed-message txt delay)))

(defun tabkey2-show-current-message (&optional delay)
  "Show current completion message, delayed if DELAY is non-nil."
  (tabkey2-cancel-delayed-message)
  (tabkey2-message delay "%s" tabkey2-current-tab-info))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Completion function selection etc

(defun tabkey2-symbol-keys (comp-fun)
  "Get a list of all key bindings for COMP-FUN."
  (let* ((remapped (command-remapping comp-fun)))
    (where-is-internal comp-fun
                       nil ;;overriding-local-map
                       nil nil remapped)))

(defun tabkey2-get-active-completion-functions ()
  "Get a list of active completion functions.
Consider only those in `tabkey2-completion-functions'."
  (delq nil
        (mapcar (lambda (rec)
                  (let ((fun (nth 1 rec))
                        (chk (nth 2 rec)))
                    (when (tabkey2-is-active fun chk) rec)))
                tabkey2-completion-functions)))

(defvar tabkey2-chosen-completion-function nil)
(make-variable-buffer-local 'tabkey2-chosen-completion-function)
(put 'tabkey2-chosen-completion-function 'permanent-local t)

(defun tabkey2-make-current-sticky ()
  "Make current Tab completion function sticky.
Set the current Tab completion function at point as default for
the current buffer."
  (interactive)
  (let ((set-it
         (y-or-n-p
          (format
           "Make %s default for Tab completion in current buffer? "
           tabkey2-current-tab-function))))
    (when set-it
      (setq tabkey2-chosen-completion-function
            tabkey2-current-tab-function))
    (unless set-it
      (when (local-variable-p 'tabkey2-chosen-completion-function)
        (when (y-or-n-p "Use default Tab completion selection in buffer? ")
          (setq set-it t))
        (kill-local-variable 'tabkey2-chosen-completion-function)))
    (when (tabkey2-completion-state-p)
      (tabkey2-message nil "%s%s" tabkey2-current-tab-info
                       (if set-it " - Done" "")))))

(defun tabkey2-cycle-completion-functions (prefix)
  "Cycle through cnd display ompletion functions.
If 'Tab completion state' is not on then turn it on.

If PREFIX is given just show what this command will do."
  (interactive "P")
  (if (tabkey2-read-only-p)
      (message "Buffer is read only at point")
    (unless tabkey2-completion-state-mode (tabkey2-completion-state-mode 1))
    (save-match-data
      (if prefix
          ;; fix-me
          (message "(TabKey2) %s: show/cycle completion function"
                   last-input-event)
        (let* ((active (mapcar (lambda (rec)
                                 (nth 1 rec))
                               (tabkey2-get-active-completion-functions)))
               (first (car active))
               next)
          ;;(message "is-shown=%s current=%s active=%s overlay=%s" tabkey2-message-is-shown tabkey2-current-tab-function active tabkey2-overlay)
          (when (or tabkey2-message-is-shown
                    ;;(tabkey2-message-is-shown)
                    )
            ;; Message is shown currently so change
            (when tabkey2-current-tab-function
              (while (and active (not next))
                (when (eq (car active) tabkey2-current-tab-function)
                  (setq next (cadr active)))
                (setq active (cdr active))))
            (unless next (setq next first))
            ;;(message "next=%s" next)
            (tabkey2-make-message-and-set-fun next)))
        (tabkey2-show-current-message)))))


;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Handling of Tab and alternate key

(defun tabkey2-first (prefix)
  "Do something else after first Tab.
This function is bound to the Tab key \(or whatever key
`tabkey2-first-key' is) when minor mode command `tabkey2-mode' is
on.  It works like this:

1. The first time Tab is pressed do whatever Tab would have done
   if minor mode command `tabkey2-mode' was off.

   Then before next command enter a new temporary 'Tab completion
   state' for just the next command.  Show this by a highlight on
   the indentation and a marker \"Tab2\" in the mode line.

   However if either
   - the minibuffer is active and `tabkey2-in-minibuffer' is nil
   - `major-mode' is in `tabkey2-modes-that-use-more-tabs' then
     do not enter this temporary 'Tab completion state'.

   For major modes where it make sense to press Tab several times
   you can use `tabkey2-alternate-key' to enter 'Tab completion
   state'.


2. As long as point is not move do completion when Tab is pressed
   again.  Show that this state is active with a highlighting at
   the line beginning, a marker on the mode line (Tab2) and a
   message in the echo area which tells what kind of completion
   will be done.

   When deciding what kind of completion to do look in the table
   below and do whatever it found first that is not nil:

   - `tabkey2-preferred'
   - `tabkey2-completion-functions'
   - `tabkey2-fallback'

3. Of course, there must be some way for you to easily determine
   what kind of completion because there are many in Emacs. If
   you do not turn it off this function will show that to you.
   And if you turn it off you can still display it, see the key
   bindings below.

   If this function is used with a PREFIX argument then it just
   shows what Tab will do.

   If the default kind of completion is not what you want then
   you can choose completion function from any of the candidates
   in `tabkey2-completion-functions'.  During the 'Tab completion
   state' the following extra key bindings are available:

\\{tabkey2-completion-state-emul-map}

Of course, some languages does not have a fixed indent as is
assumed above. You can put major modes for those in
`tabkey2-modes-that-just-complete'.

Some major modes uses tab for something else already. Those are
in `tabkey2-modes-that-use-more-tabs'.  There is an alternate
key, `tabkey2-alternate-key' if you want to do completion
there. Note that this key does not do completion. It however
enters 'Tab completion state' in which you have access to the
keys above for completion etc. \(This key also lets you cycle
through the completion functions too choose which one to use.)

-----
NOTE: This uses `emulation-mode-map-alists' and it supposes that
nothing else is bound to Tab there."
  (interactive "P")
  (if (and tabkey2-keymap-overlay
           (eq (overlay-buffer tabkey2-keymap-overlay) (current-buffer))
           (eq (overlay-get tabkey2-keymap-overlay 'window) (selected-window))
           (>= (point) (overlay-start tabkey2-keymap-overlay))
           (<= (point) (overlay-end   tabkey2-keymap-overlay)))
      ;; We should maybe not be here, but the keymap does not work at
      ;; the end of the buffer so we call the second tab function from
      ;; here:
      (if (memq 'shift (event-modifiers last-input-event))
          (call-interactively 'tabkey2-cycle-completion-functions)
        (call-interactively 'tabkey2-complete prefix))
    (let* ((emma-without-tabkey2
            ;; Remove keymaps from tabkey2 in this copy:
            (delq 'tabkey2--emul-keymap-alist
                  (copy-sequence emulation-mode-map-alists)))
           (just-complete (memq major-mode tabkey2-modes-that-just-complete))
           (what (if just-complete
                     'complete
                   (if (or (unless tabkey2-in-minibuffer
                             (active-minibuffer-window))
                           (when (fboundp 'use-region-p) (use-region-p))
                           (memq major-mode tabkey2-modes-that-use-more-tabs))
                       'indent
                     'indent-complete
                     )))
           (to-do-1 (unless (or
                             ;; Skip action on tab if shift tab,
                             ;; backtab or a mode in the "just
                             ;; complete" list
                             (memq 'shift (event-modifiers last-input-event))
                             (equal [backtab] (this-command-keys-vector))
                             )
                      (let ((emulation-mode-map-alists emma-without-tabkey2))
                        (key-binding [?\t] t))))
           (to-do-2 (unless (or
                             ;;(memq what '(complete))
                             (memq what '(indent))
                             (memq to-do-1 '(widget-forward button-forward)))
                      (tabkey2-get-default-completion-fun))))
      (if prefix
          (if (memq 'shift (event-modifiers last-input-event))
              (message
               "(TabKey2) First shift %s: turn on 'Tab completion state'"
               last-input-event)
            (message "(TabKey2) First %s: %s, next: maybe %s"
                     last-input-event to-do-1
                     (if to-do-2 to-do-2 "(same)")))
        (when to-do-1
          (let ((last-command to-do-1)
                mumamo-multi-major-mode)
            (call-interactively to-do-1)))
        (unless (tabkey2-read-only-p)
          (when to-do-2
            (tabkey2-completion-state-mode 1)))))))


(defun tabkey2-complete (prefix)
  "Call current completion function.
If used with a PREFIX argument then just show what Tab will do."
  (interactive "P")
  (if prefix
      (message "(TabKey2) %s: %s"
               last-input-event tabkey2-current-tab-function)
    (call-interactively tabkey2-current-tab-function)))

(define-minor-mode tabkey2-mode
  "More fun with Tab key number two (completion etc).
This global minor mode by default binds Tab in a way that let you
do completion with Tab in all buffers \(where it is possible).

The Tab key is easy to type on your keyboard.  Then why not use
it for completion, something that is very useful?  Shells usually
use Tab for completion so many are used to it.  This was the idea
of Smart Tabs and this is a generalization of that idea.

However in Emacs the Tab key is usually used for indentation.
The idea here is that if Tab has been pressed once for
indentation, then as long as point stays further Tab keys might
as well do completion.

So you kind of do Tab-Tab for first completion \(and then just
Tab for further completions as long as point is not moved).

See `tabkey2-first' for more information about usage.

Note: If you do not want the Tab-Tab behaviour above, but still
want an easy way to reach the available completion functions,
then you can instead of turning on tabkey2-mode enter this in
your .emacs:

 \(global-set-key [f8] 'tabkey2-cycle-completion-functions)

After hitting f8 you will then be in the same state as after the
first in tabkey2-mode."
  :keymap nil
  :global t
  :group 'tabkey2
  (if tabkey2-mode
      (progn
        (add-hook 'minibuffer-setup-hook 'tabkey2-minibuffer-setup)
        ;; Update emul here if keymap have changed
        (setq tabkey2--emul-keymap-alist
              (list (cons 'tabkey2-mode
                          tabkey2-mode-emul-map)))
        (add-to-list 'emulation-mode-map-alists 'tabkey2--emul-keymap-alist))
    (tabkey2-completion-state-mode -1)
    (remove-hook 'minibuffer-setup-hook 'tabkey2-minibuffer-setup)
    (setq emulation-mode-map-alists (delq 'tabkey2--emul-keymap-alist
                                          emulation-mode-map-alists))))

;; Fix-me: I am not sure that it really is useful with a globalized
;; minor mode here because there are so many other ways to control
;; what happens in a specific buffer. Maybe it would just be
;; confusing?
;;
;; If found another problem with making it globalized: tabkey2-mode
;; uses emulation-mode-map-alist.  I decided to remove this therefore.
;;
;; (defun tabkey2-turn-on ()
;;   "Turn on `tabkey2-mode' in current buffer."
;;   (tabkey2-mode 1))

;; (defvar tabkey2-turn-on-function 'tabkey2-turn-on
;;   "Function used to mabye turn on `tabkey2-mode' in current-buffer.
;; This function is used by `tabkey2-global-mode' to turn on
;; `tabkey2-mode'.")

;; (defun tabkey2-turn-on-in-buffer ()
;;   (funcall tabkey2-turn-on-function))

;; (define-globalized-minor-mode tabkey2-global-mode
;;   tabkey2-mode tabkey2-turn-on-in-buffer)

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;; Help functions

(defun tabkey2-show-completion-state-help ()
  "Help for 'Tab completion state'.
To get out of this state you can move out of the current line.

During this state the keymap below is active.  This state stops
as soon as you leave the current row.

\\{tabkey2-completion-state-emul-map}
See function `tabkey2-mode' for more information.

If you want to use Emacs normal help function then press F1
again.")

(defun tabkey2-completion-state-help ()
  "Show help for 'Tab completion state'."
  (interactive)
  ;;(message "tckv=%s" (this-command-keys-vector)) ;;(sit-for 1)
  ;; Fix-me: There seems to be an Emacs bug lurking here. Sometimes
  ;; invoked-by-f1 is not [f1].
  (let ((invoked-by-f1 (equal (this-command-keys-vector) [f1]))
        normal-help)
    ;;(message "invoked-by-f1=%s" invoked-by-f1) ;; fix-me
    (if (not invoked-by-f1)
        (describe-function 'tabkey2-show-completion-state-help)
      (setq normal-help
            (read-event
             (propertize
              (concat "Type a key for Emacs help."
                      " Or, wait for Tab completion state help: ")
              'face 'highlight)
             nil
             4))
      (case normal-help
        ((nil)
         ;;(message "Tab completion state help")
         (describe-function 'tabkey2-show-completion-state-help))
        (?c
         (call-interactively 'describe-key-briefly))
        (?k
         (call-interactively 'describe-key))
        (t
         (tabkey2-completion-state-mode -1)
         (setq unread-command-events
               (reverse
                (cons
                 normal-help
                 (append (this-command-keys) nil)))))))))

(defun tabkey2-completion-function-help ()
  "Show help for current completion function."
  (interactive)
  (describe-function tabkey2-current-tab-function))




(defun tabkey2-get-key-binding (fun)
  "Get key binding for FUN during 'Tab completion state'."
  (let* ((remapped (command-remapping fun))
         (key (where-is-internal fun
                                 tabkey2-completion-state-emul-map
                                 t
                                 nil
                                 remapped)))
    key))

(defun tabkey2-make-message-and-set-fun (comp-fun)
  "Set current completion function to COMP-FUN.
Build message but don't show it."
  (setq tabkey2-current-tab-function comp-fun)
  (let* ((chs-fun 'tabkey2-cycle-completion-functions)
         (key (tabkey2-get-key-binding chs-fun))
         (active-funs (tabkey2-get-active-completion-functions))
         what)
    (dolist (rec tabkey2-completion-functions)
      (let ((fun (nth 1 rec))
            (txt (nth 0 rec)))
        (when (eq fun comp-fun) (setq what txt))))
    (let ((info (concat (format "Tab: %s" what)
                        (if (cdr (tabkey2-get-active-completion-functions))
                            (format ", other %s, help F1"
                                    (key-description key))
                          ""))))
      (setq tabkey2-current-tab-info info))))

(defun tabkey2-get-active-string (bnd fun buf)
  "Get string to show for state.
BND: means active
FUN: function
BUF: buffer"
  (if bnd
      (if (with-current-buffer buf (tabkey2-read-only-p))
          (propertize "active, but read-only" 'face '( :foreground "red"))
        (propertize "active" 'face '( :foreground "green")))
    (if (fboundp fun)
        (propertize "not active" 'face '( :foreground "red"))
      (propertize "not defined" 'face '( :foreground "gray")))))

(defun tabkey2-show-completion-functions ()
  "Show what currently may be used for completion."
  (interactive)
  (let ((orig-buf (current-buffer))
        (orig-mn  mode-name)
        (active-mark (concat " "
                             (propertize " <= "
                                         'face '( :background "yellow"))))
        (act-found nil)
        (chosen-fun tabkey2-chosen-completion-function)
        what
        chosen)
    (when chosen-fun
      (dolist (rec tabkey2-completion-functions)
        (let ((fun (nth 1 rec))
              (txt (nth 0 rec)))
          (when (eq fun chosen-fun) (setq what txt))))
      (setq chosen (list what chosen-fun)))
    (with-output-to-temp-buffer (help-buffer)
      (help-setup-xref (list #'tabkey2-show-completion-functions)
                       (interactive-p))
      (with-current-buffer (help-buffer)
        (insert (concat "The completion functions available for"
                        " 'Tab completion' in buffer\n'"
                (buffer-name orig-buf)
                "' at point with mode " orig-mn " are shown below.\n"
                "The first active function is used.\n\n"))
        (if (not chosen)
            (insert "  None is chosen")
          (let* ((txt (nth 0 chosen))
                 (fun (nth 1 chosen))
                 (chk (nth 2 chosen))
                 (bnd (with-current-buffer orig-buf
                        (tabkey2-is-active fun chk)))
                 (act (tabkey2-get-active-string bnd fun orig-buf)))
            (insert (format "  Chosen currently is %s (%s): %s"
                            txt fun act))
            (when bnd (insert active-mark) (setq act-found t))))
        (insert "\n\n")
        (if (not tabkey2-preferred)
            (insert "  None is preferred")
          (let* ((txt (nth 0 tabkey2-preferred))
                 (fun (nth 1 tabkey2-preferred))
                 (chk (nth 2 chosen))
                 (bnd (with-current-buffer orig-buf
                        (tabkey2-is-active fun chk)))
                 (act (tabkey2-get-active-string bnd fun orig-buf)))
            (insert (format "  Preferred is %s (`%s')': %s"
                            txt fun act))
            (when bnd (insert active-mark) (setq act-found t))))
        (insert "\n\n")
        (dolist (comp-fun tabkey2-completion-functions)
          (let* ((txt (nth 0 comp-fun))
                 (fun (nth 1 comp-fun))
                 (chk (nth 2 comp-fun))
                 (bnd (with-current-buffer orig-buf
                        (tabkey2-is-active fun chk)))
                 (act (tabkey2-get-active-string bnd fun orig-buf)))
            (insert
             (format
              "  %s (`%s'): %s"
              txt fun act))
            (when (and (not act-found) bnd)
              (insert active-mark) (setq act-found t))
            (insert "\n")))
        (insert "\n")
        (if (not tabkey2-fallback)
            (insert "  There is no fallback")
          (let* ((txt (nth 0 tabkey2-fallback))
                 (fun (nth 1 tabkey2-fallback))
                 (chk (nth 2 tabkey2-fallback))
                 (bnd (with-current-buffer orig-buf
                        (tabkey2-is-active fun chk)))
                 (act (tabkey2-get-active-string bnd fun orig-buf)))
            (insert (format "  Fallback is %s (`%s'): %s"
                            txt fun act))
            (when (and (not act-found) bnd)
              (insert active-mark)
              (setq act-found t))))
        (insert "\n\nSee function `tabkey2-mode' for more information.")
        (print-help-return-message)))))

(defvar tabkey2-completing-read 'completing-read)

(defun tabkey2-set-fun (fun)
  "Use function FUN for Tab in 'Tab completion state'."
  (setq tabkey2-chosen-completion-function fun)
  (unless fun
    (setq fun (tabkey2-first-active-from-completion-functions)))
  (tabkey2-make-message-and-set-fun fun)
  (when (tabkey2-completion-state-p)
    (message "%s" tabkey2-current-tab-info)))

(defun tabkey2-appmenu ()
  "Make a menu for minor mode command `appmenu-mode'."
  (unless (tabkey2-read-only-p)
    (let* ((cf-r (reverse (tabkey2-get-active-completion-functions)))
           (tit "Complete")
           (map (make-sparse-keymap tit)))
      (define-key map [tabkey2-usage]
        (list 'menu-item "Show available completion functions"
              'tabkey2-show-completion-functions))
      (define-key map [tabkey2-divider-1] (list 'menu-item "--"))
      (let ((set-map (make-sparse-keymap "Set completion")))
        (define-key map [tabkey2-choose]
          (list 'menu-item "Set Primary Tab completion for buffer" set-map))
        (dolist (cf-rec cf-r)
          (let ((dsc (nth 0 cf-rec))
                (fun (nth 1 cf-rec)))
            (define-key set-map
              (vector (intern (format "tabkey2-set-%s" fun)))
              (list 'menu-item dsc
                    `(lambda ()
                       (interactive)
                       (tabkey2-set-fun ',fun))
                    :button
                    `(:radio
                      . (eq ',fun tabkey2-chosen-completion-function))))))
        (define-key set-map [tabkey2-set-div] (list 'menu-item "--"))
        (define-key set-map [tabkey2-set-default]
          (list 'menu-item "Default Tab completion"
                (lambda ()
                  (interactive)
                  (tabkey2-set-fun nil))
                :button
                '(:radio . (null tabkey2-chosen-completion-function))))
        (define-key set-map [tabkey2-set-header-div] (list 'menu-item "--"))
        (define-key set-map [tabkey2-set-header]
          (list 'menu-item "Set Primary Tab completion for buffer"))
        )
      (define-key map [tabkey2-divider] (list 'menu-item "--"))
      (dolist (cf-rec cf-r)
        (let ((dsc (nth 0 cf-rec))
              (fun (nth 1 cf-rec)))
          (define-key map
            (vector (intern (format "tabkey2-call-%s" fun)))
            (list 'menu-item dsc fun
                  :button
                  `(:toggle
                    . (eq ',fun tabkey2-chosen-completion-function))
                  ))))
      map)))

;; (defun tabkey2-completion-menu-popup ()
;;   "Pop up a menu with completion alternatives."
;;   (interactive)
;;   (let ((menu (tabkey2-appmenu)))
;;     (popup-menu-at-point menu)))

;; (defun tabkey2-choose-completion-function ()
;;   "Set current completion function.
;; Let user choose completion function from those in
;; `tabkey2-completion-functions' that have some key binding at
;; point.

;; Let the chosen completion function be the default for subsequent
;; completions in the current buffer."
;;   ;; Fix-me: adjust to mumamo.
;;   (interactive)
;;   (save-match-data
;;     (if (and (featurep 'popcmp)
;;              tabkey2-use-popup-menus)
;;         (tabkey2-completion-menu-popup)
;;       (when (eq 'completing-read tabkey2-completing-read) (isearch-unread 'tab))
;;       (let* ((cf-r (reverse (tabkey2-get-active-completion-functions)))
;;              (cf (cons '("- Use default Tab completion" nil) cf-r))
;;              (hist (mapcar (lambda (rec)
;;                              (car rec))
;;                            cf))
;;              (tit (funcall tabkey2-completing-read "Set current completion function: " cf
;;                            nil ;; predicate
;;                            t   ;; require-match
;;                            nil ;; initial-input
;;                            'hist ;; hist
;;                            ))
;;              (fun-rec (assoc-string tit cf))
;;              (fun (cadr fun-rec)))
;;         (setq tabkey2-chosen-completion-function fun)
;;         (unless fun
;;           (setq fun (tabkey2-first-active-from-completion-functions)))
;;         (tabkey2-make-message-and-set-fun fun)
;;         (when (tabkey2-completion-state-p)
;;           (tabkey2-show-current-message))))))

;; (defun tabkey2-add-to-appmenu ()
;;   "Add a menu to function `appmenu-mode'."
;;   (appmenu-add 'tabkey2 nil t "Completion" 'tabkey2-appmenu))


(provide 'tabkey2)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;; tabkey2.el ends here

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Thu, 24 Jul 2008 00:52:08 +0200 "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> wrote: 

LB(> I am not sure I understand the details here (or rather "I am sure I do
LB(> not understand all the details here"), but things like this is what
LB(> tabkey2.el tries to solve.

LB(> I have done some significant changes since I last proposed it. I have
LB(> attached a new version.

LB(> Could you please see if what you want can fit in tabkey2.el? (The idea
LB(> there is that normally the first Tab does indentation while the second
LB(> Tab key can do some kind of completion - and S-Tab can choose what to
LB(> complete.)

That seems like a good idea, if it can work with Quail input methods.
They require an actual key name, so the changes would not be minor.
Otherwise I like it: I can't remember any time I've need to press TAB
more than once in any setting.

Ted

Re: describe-char should display the character's Unicode name

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1032 bytes --]

On 7/24/08, Ted Zlatanov <tzz@lifelogs.com> wrote:
>
> On Thu, 24 Jul 2008 00:52:08 +0200 "Lennart Borgman (gmail)" <
> lennart.borgman@gmail.com> wrote:
>
> LB(> I am not sure I understand the details here (or rather "I am sure I do
> LB(> not understand all the details here"), but things like this is what
> LB(> tabkey2.el tries to solve.
>
> LB(> I have done some significant changes since I last proposed it. I have
> LB(> attached a new version.
>
> LB(> Could you please see if what you want can fit in tabkey2.el? (The idea
> LB(> there is that normally the first Tab does indentation while the second
> LB(> Tab key can do some kind of completion - and S-Tab can choose what to
> LB(> complete.)
>
> That seems like a good idea, if it can work with Quail input methods.
> They require an actual key name, so the changes would not be minor.
> Otherwise I like it: I can't remember any time I've need to press TAB
> more than once in any setting.
>
> Ted


Can you please explain that thing with "an actual key name" above?

[-- Attachment #2: Type: text/html, Size: 1455 bytes --]

Re: describe-char should display the character's Unicode name

[Ted Zlatanov]

On Thu, 24 Jul 2008 15:39:42 +0200 "Lennart Borgman" <lennart.borgman@gmail.com> wrote: 

LB> Can you please explain that thing with "an actual key name" above?

Juri mentioned that M-TAB, for example, won't work as a Quail method
invocation key.  You need to use a single key, not a key sequence
apparently (but realize I don't know the internals and am just repeating
what Juri found).

Ted

Re: describe-char should display the character's Unicode name

[Lennart Borgman (gmail)]

Ted Zlatanov wrote:
> On Thu, 24 Jul 2008 15:39:42 +0200 "Lennart Borgman" <lennart.borgman@gmail.com> wrote: 
> 
> LB> Can you please explain that thing with "an actual key name" above?
> 
> Juri mentioned that M-TAB, for example, won't work as a Quail method
> invocation key.  You need to use a single key, not a key sequence
> apparently (but realize I don't know the internals and am just repeating
> what Juri found).


Thanks Ted. Juri, could you please explain a bit? Would it be possible 
to use tabkey2.el for Quyail in its current incarnation where I do 
something like

   (funcall original-command-on-tab)

on the 2nd Tab press.

Re: describe-char should display the character's Unicode name

[Juri Linkov]

>> LB> Can you please explain that thing with "an actual key name" above?
>>
>> Juri mentioned that M-TAB, for example, won't work as a Quail method
>> invocation key.  You need to use a single key, not a key sequence
>> apparently (but realize I don't know the internals and am just repeating
>> what Juri found).
>
> Thanks Ted. Juri, could you please explain a bit? Would it be possible to
> use tabkey2.el for Quyail in its current incarnation where I do something
> like
>
>   (funcall original-command-on-tab)
>
> on the 2nd Tab press.

You need to dig deeply into the Quail internals to find the answer.
Sorry, I can't do this in the next few days.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Kenichi Handa]

In article <87prp294o0.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> You need to dig deeply into the Quail internals to find the answer.
> Sorry, I can't do this in the next few days.

Me neither, sorry.  I'm now working on the automatic
composition without using a text property.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: describe-char should display the character's Unicode name

[Juri Linkov]

> MB> An input method is not a good solution because while input methods are
> MB> good for heavy use, they're quite annoying to juggle for occasional inputs.
>
> Agreed, there should be a function that gives all the Unicode characters
> as input candidates and the user can select the one he wants quickly (I
> don't see one currently).  An input method wouldn't provide completion
> candidates, so the user has to remember all the names he needs.
>
> Would a few thousand candidates be a problem for completion?  I've only
> seen it with hundreds and it's not slow, but for this maybe maybe we can
> break it down by block name and then character name.

There is no problem with completion in a new function below created
to read a character by its Unicode name or hex number string.  There
is only a small delay when it creates the completion list, and a
longer delay when it displays the *Completions* buffer with all
Unicode names:

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.333
diff -c -r1.333 mule-cmds.el
*** lisp/international/mule-cmds.el	15 Jul 2008 18:15:03 -0000	1.333
--- lisp/international/mule-cmds.el	19 Jul 2008 00:35:23 -0000
***************
*** 2846,2855 ****
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive "sUnicode (hex): ")
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))
--- 2849,2873 ----
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
+ (defun read-char-by-name (prompt)
+   "Read a character by its Unicode name or hex number string.
+ Display PROMPT and read a string that represent a character
+ by its Unicode property `name' or `old-name'.  It also accepts
+ a hexadecimal number of Unicode code point.  Returns a character
+ as a number."
+   (let (name names)
+     (dotimes (c #x10FFFF)
+       (if (setq name (get-char-code-property c 'name))
+ 	  (setq names (cons (cons name c) names)))
+       (if (setq name (get-char-code-property c 'old-name))
+ 	  (setq names (cons (cons name c) names))))
+     (or (cdr (assoc (setq name (completing-read prompt names)) names))
+ 	(string-to-number name 16))))
+ 
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive (list (read-char-by-name "Unicode (hex or name): ")))
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: describe-char should display the character's Unicode name

[Kenichi Handa]

In article <873am6n21q.fsf@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> There is no problem with completion in a new function below created
> to read a character by its Unicode name or hex number string.  There
> is only a small delay when it creates the completion list, and a
> longer delay when it displays the *Completions* buffer with all
> Unicode names:

> Index: lisp/international/mule-cmds.el
[...]
> + (defun read-char-by-name (prompt)
> +   "Read a character by its Unicode name or hex number string.
> + Display PROMPT and read a string that represent a character
> + by its Unicode property `name' or `old-name'.  It also accepts
> + a hexadecimal number of Unicode code point.  Returns a character
> + as a number."
> +   (let (name names)
> +     (dotimes (c #x10FFFF)
> +       (if (setq name (get-char-code-property c 'name))
> + 	  (setq names (cons (cons name c) names)))
> +       (if (setq name (get-char-code-property c 'old-name))
> + 	  (setq names (cons (cons name c) names))))
> +     (or (cdr (assoc (setq name (completing-read prompt names)) names))
> + 	(string-to-number name 16))))
> + 

I think it is better to skip these ranges:
  #x3400..#x4dbf   -- CJK Ideograph Extension A
  #x4e00..#x9fff   -- CJK Ideograph
  #xd800..#xfaFF   -- surroage-pair, private use, CJK COMPATIBILITY IDEOGRAPH
  #x20000..#x2ffff -- CJK Ideograph Extension B
and end the loop at #xeffff (#xf0000.. are for private use)

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Juri Linkov]

>> + (defun read-char-by-name (prompt)
>> +   "Read a character by its Unicode name or hex number string.
>> + Display PROMPT and read a string that represent a character
>> + by its Unicode property `name' or `old-name'.  It also accepts
>> + a hexadecimal number of Unicode code point.  Returns a character
>> + as a number."
>> +   (let (name names)
>> +     (dotimes (c #x10FFFF)
>> +       (if (setq name (get-char-code-property c 'name))
>> + 	  (setq names (cons (cons name c) names)))
>> +       (if (setq name (get-char-code-property c 'old-name))
>> + 	  (setq names (cons (cons name c) names))))
>> +     (or (cdr (assoc (setq name (completing-read prompt names)) names))
>> + 	(string-to-number name 16))))
>> +
>
> I think it is better to skip these ranges:
>   #x3400..#x4dbf   -- CJK Ideograph Extension A
>   #x4e00..#x9fff   -- CJK Ideograph
>   #xd800..#xfaFF   -- surroage-pair, private use, CJK COMPATIBILITY IDEOGRAPH
>   #x20000..#x2ffff -- CJK Ideograph Extension B
> and end the loop at #xeffff (#xf0000.. are for private use)

Actually there are no Unicode names in these ranges in UnicodeData.txt.
It has only lines for the first and the last character in these ranges:

3400;<CJK Ideograph Extension A, First>;Lo;0;L;;;;;N;;;;;
4DB5;<CJK Ideograph Extension A, Last>;Lo;0;L;;;;;N;;;;;
4E00;<CJK Ideograph, First>;Lo;0;L;;;;;N;;;;;
9FC3;<CJK Ideograph, Last>;Lo;0;L;;;;;N;;;;;
D800;<Non Private Use High Surrogate, First>;Cs;0;L;;;;;N;;;;;
DB7F;<Non Private Use High Surrogate, Last>;Cs;0;L;;;;;N;;;;;
DB80;<Private Use High Surrogate, First>;Cs;0;L;;;;;N;;;;;
DBFF;<Private Use High Surrogate, Last>;Cs;0;L;;;;;N;;;;;
DC00;<Low Surrogate, First>;Cs;0;L;;;;;N;;;;;
DFFF;<Low Surrogate, Last>;Cs;0;L;;;;;N;;;;;
E000;<Private Use, First>;Co;0;L;;;;;N;;;;;
F8FF;<Private Use, Last>;Co;0;L;;;;;N;;;;;
20000;<CJK Ideograph Extension B, First>;Lo;0;L;;;;;N;;;;;
2A6D6;<CJK Ideograph Extension B, Last>;Lo;0;L;;;;;N;;;;;
F0000;<Plane 15 Private Use, First>;Co;0;L;;;;;N;;;;;
FFFFD;<Plane 15 Private Use, Last>;Co;0;L;;;;;N;;;;;
100000;<Plane 16 Private Use, First>;Co;0;L;;;;;N;;;;;
10FFFD;<Plane 16 Private Use, Last>;Co;0;L;;;;;N;;;;;

If it would be possible to loop over names instead of loop over all
characters to check for their names, then this code would be more fast,
but I don't see how it would be possible to loop over all defined names
in UnicodeData.txt.

If this is not possible then we could optimize the loop over all
characters in the chartable to skip these useless ranges.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Kenichi Handa]

In article <87sku5if8t.fsf_-_@jurta.org>, Juri Linkov <juri@jurta.org> writes:

> > I think it is better to skip these ranges:
> >   #x3400..#x4dbf   -- CJK Ideograph Extension A
> >   #x4e00..#x9fff   -- CJK Ideograph
> >   #xd800..#xfaFF   -- surroage-pair, private use, CJK COMPATIBILITY IDEOGRAPH
> >   #x20000..#x2ffff -- CJK Ideograph Extension B
> > and end the loop at #xeffff (#xf0000.. are for private use)

> Actually there are no Unicode names in these ranges in UnicodeData.txt.
> It has only lines for the first and the last character in these ranges:

Yes.  But, for CJK chars:

   (get-char-code-property CHAR 'name)

returns a valid name something like "CJK IDEOGRAPH-3400"(*)
because get-char-code-property not only looks up
UnicodeData.txt but also compute a proper value if
necessary.

> If it would be possible to loop over names instead of loop over all
> characters to check for their names, then this code would be more fast,
> but I don't see how it would be possible to loop over all defined names
> in UnicodeData.txt.

> If this is not possible then we could optimize the loop over all
> characters in the chartable to skip these useless ranges.

I think it doesn't work because Hangul syllabic character
names must also be computed algorithmically(*).   I think
just doing somethink like this is good:

 (dotimes (c #xEFFFF)
    (unless (CHAR-IS-IN-A-RANGE-TO-SKIP-P c)
       ...))


(*): "The Unicode Standard 5.1" has this section.

4.8 Name—Normative
[...]
Ideographs and Hangul Syllables. Names for ideographs and
Hangul syllables are derived algorithmically. Unified CJK
ideographs are named CJK UNIFIED IDEOGRAPH-x, where x is
replaced with the hexadecimal Unicode code point—for
example, cjk unified ideograph-4E00. Similarly,
compatibility CJK ideographs are named “CJK COMPATIBILITY
IDEOGRAPH-x”. The names of Hangul syllables are generated as
described in “Hangul Syllable Names” in Section 3.12,
Conjoining Jamo Behavior.

---
Kenichi Handa
handa@ni.aist.go.jp

Re: inputting characters by hexadigit

[Juri Linkov]

>> > I think it is better to skip these ranges:
>> >   #x3400..#x4dbf   -- CJK Ideograph Extension A
>> >   #x4e00..#x9fff   -- CJK Ideograph
>> >   #xd800..#xfaFF   -- surroage-pair, private use, CJK COMPATIBILITY IDEOGRAPH
>> >   #x20000..#x2ffff -- CJK Ideograph Extension B
>> > and end the loop at #xeffff (#xf0000.. are for private use)
>
>> Actually there are no Unicode names in these ranges in UnicodeData.txt.
>> It has only lines for the first and the last character in these ranges:
>
> Yes.  But, for CJK chars:
>
>    (get-char-code-property CHAR 'name)
>
> returns a valid name something like "CJK IDEOGRAPH-3400"(*)
> because get-char-code-property not only looks up
> UnicodeData.txt but also compute a proper value if
> necessary.

Thanks, I see now why it is necessary to skip these ranges.

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.333
diff -c -r1.333 mule-cmds.el
*** lisp/international/mule-cmds.el	15 Jul 2008 18:15:03 -0000	1.333
--- lisp/international/mule-cmds.el	20 Jul 2008 20:27:21 -0000
***************
*** 2846,2855 ****
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive "sUnicode (hex): ")
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))
--- 2849,2879 ----
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
+ (defun read-char-by-name (prompt)
+   "Read a character by its Unicode name or hex number string.
+ Display PROMPT and read a string that represents a character
+ by its Unicode property `name' or `old-name'.  It also accepts
+ a hexadecimal number of Unicode code point.  Returns a character
+ as a number."
+   (let (name names)
+     (dotimes (c #xEFFFF)
+       (unless (or
+ 	       (and (>= c #x3400 ) (<= c #x4dbf )) ; CJK Ideograph Extension A
+ 	       (and (>= c #x4e00 ) (<= c #x9fff )) ; CJK Ideograph
+ 	       (and (>= c #xd800 ) (<= c #xfaff )) ; Private/Surrogate
+ 	       (and (>= c #x20000) (<= c #x2ffff)) ; CJK Ideograph Extension B
+ 	       )
+ 	(if (setq name (get-char-code-property c 'name))
+ 	    (setq names (cons (cons name c) names)))
+ 	(if (setq name (get-char-code-property c 'old-name))
+ 	    (setq names (cons (cons name c) names)))))
+     (or (cdr (assoc (setq name (completing-read prompt names)) names))
+ 	(string-to-number name 16))))
+ 
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive (list (read-char-by-name "Unicode (hex or name): ")))
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Sun, 20 Jul 2008 23:27:47 +0300 Juri Linkov <juri@jurta.org> wrote: 

JL> Thanks, I see now why it is necessary to skip these ranges.
[patch to ucs-insert that includes read-char-by-name omitted]

On Wed, 23 Jul 2008 06:44:51 -0700 "Drew Adams" <drew.adams@oracle.com> wrote:

>> > DA> This thread got off the ground because some thought 
>> > DA> that using an input method could be too heavy for
>> > DA> inputting Unicode chars. OK, keep looking for a better
>> > DA> (non "input method") input method, if you must. But 
>> > DA> please stay away from key chords and existing key bindings.
>> > 
>> > I don't understand why we don't just provide an interactive 
>> > function and let the users pick the keybinding they like.
>> > IMHO this is completely useless for many users (considering
>> > input methods exist for particular languages and we've talked
>> > about stacking multiple input methods), and essential for much
>> > fewer people (who need to enter arbitrary Unicode characters
>> > by number).
>> 
>> Seems like a good suggestion to me if combined with Juri's
>> latest suggestion.

DA> +1

I tried Juri's patch.  It works fine, but on my system it was pretty
slow (and I have a decent, fast workstation).  It took 3-5 seconds to
display the list without byte-compiling the code (which I generally do
when testing to be sure functions are not too slow).  I worry that on a
slow machine hitting TAB to see the completions will be painful.

Can completions be cascaded somehow?  The first tier would show all the
common first words, e.g. ... AEGEAN APL GREEK ... and then selecting
something from the first tier would cascade down to the second tier.
This might actually be a sensible suggestion for any completion list
with more than 100 or so members, and the tier separation can be done
with the word syntax of the current mode.

Anyhow, besides the speed issue, Juri's patch works fine for me and I
can now insert ANKHs and GERMAN PENNY SYMBOLs anywhere I like :)

Ted

Re: inputting characters by hexadigit

[Stefan Monnier]

> Can completions be cascaded somehow?  The first tier would show all the
> common first words, e.g. ... AEGEAN APL GREEK ... and then selecting
> something from the first tier would cascade down to the second tier.

The slow display should only happen when the list is really long,
i.e. basically if you hit TAB with an empty minibuffer.  So we could
indeed easily use a different completion behavior in this case.

> This might actually be a sensible suggestion for any completion list
> with more than 100 or so members, and the tier separation can be done
> with the word syntax of the current mode.

You mean show the list as a tree?  That cannot be done with the current
default completio code.  But some variant of it can be done.  E.g.  we
can use a completion similar to file names, so additionally to
<charname> we could accept input of the form <category>/<charname> and
provide completion on this (and if the completion list on <charname> is too
long, don't display it: only display the completion on
<category>/<charname>).


        Stefan

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Wed, 23 Jul 2008 15:31:36 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> Can completions be cascaded somehow?  The first tier would show all the
>> common first words, e.g. ... AEGEAN APL GREEK ... and then selecting
>> something from the first tier would cascade down to the second tier.

SM> The slow display should only happen when the list is really long,
SM> i.e. basically if you hit TAB with an empty minibuffer.  So we could
SM> indeed easily use a different completion behavior in this case.

>> This might actually be a sensible suggestion for any completion list
>> with more than 100 or so members, and the tier separation can be done
>> with the word syntax of the current mode.

SM> You mean show the list as a tree?  That cannot be done with the current
SM> default completio code.  But some variant of it can be done.  E.g.  we
SM> can use a completion similar to file names, so additionally to
SM> <charname> we could accept input of the form <category>/<charname> and
SM> provide completion on this (and if the completion list on <charname> is too
SM> long, don't display it: only display the completion on
SM> <category>/<charname>).

To clarify my proposal:

Split on the word syntax, so (taking the Unicode example) hitting TAB
initially gives

APL (200 candidates)
AEGEAN (20 candidates)
GREEK (50 candidtates)

then as the user types A, the list becomes

APL (200 candidates)
AEGEAN (20 candidates)

with the A highlighted, and so on.  As soon as less than X candidates
are available the usual list is displayed; X is customizable but 40 by
default.

The word syntax can depend on the current mode, or be overridden by the
function being called.

The advantage over categories, which you suggested, is that the
completion code won't need categories supplied--it automatically gets
them based on the word syntax, so it's less work for the programmer and
thus more likely to get used everywhere and be suitable as a drop-in
replacement for the current completion code.  The current behavior
(e.g. M-x TAB) is nearly unusable without either apropos or icicles
(that I know of) to search through the list.

Ted

Re: inputting characters by hexadigit

[Stefan Monnier]

> The advantage over categories, which you suggested, is that the
> completion code won't need categories supplied--it automatically gets
> them based on the word syntax, so it's less work for the programmer and

Maybe I misunderstand what you mean by word syntax.
Are you referring to something more like partial-completion-mode?


        Stefan

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Wed, 23 Jul 2008 22:08:29 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

>> The advantage over categories, which you suggested, is that the
>> completion code won't need categories supplied--it automatically gets
>> them based on the word syntax, so it's less work for the programmer and

SM> Maybe I misunderstand what you mean by word syntax.
SM> Are you referring to something more like partial-completion-mode?

That's close, but it looks for abbreviations rather than categories, and
is nicest when you know the word you want in advance.  When you don't,
you still have to walk through the list of candidates.

The essential problem is that more than 40 or so candidates are simply
unusable--it's not just slow, it's unmanageable.  You don't know what's
at the end of the list until you go there, and by then you've forgotten
what was at the beginning.  With categories you end up with meaningful
prefixes instead, as in my example.  The category name is meaningful
because the name of the thing you are completing is split in an
informational way, not (as p-c-m does) by first letter, and then the
first thing from the split string is used as the category.

If this doesn't sound useful I'll drop it.

Ted

Re: inputting characters by hexadigit

[Lennart Borgman]

[-- Attachment #1: Type: text/plain, Size: 1356 bytes --]

On 7/24/08, Ted Zlatanov <tzz@lifelogs.com> wrote:
>
> On Wed, 23 Jul 2008 22:08:29 -0400 Stefan Monnier <
> monnier@iro.umontreal.ca> wrote:
>
> >> The advantage over categories, which you suggested, is that the
> >> completion code won't need categories supplied--it automatically gets
> >> them based on the word syntax, so it's less work for the programmer and
>
> SM> Maybe I misunderstand what you mean by word syntax.
> SM> Are you referring to something more like partial-completion-mode?
>
> That's close, but it looks for abbreviations rather than categories, and
> is nicest when you know the word you want in advance.  When you don't,
> you still have to walk through the list of candidates.
>
> The essential problem is that more than 40 or so candidates are simply
> unusable--it's not just slow, it's unmanageable.  You don't know what's
> at the end of the list until you go there, and by then you've forgotten
> what was at the beginning.  With categories you end up with meaningful
> prefixes instead, as in my example.  The category name is meaningful
> because the name of the thing you are completing is split in an
> informational way, not (as p-c-m does) by first letter, and then the
> first thing from the split string is used as the category.
>
> If this doesn't sound useful I'll drop it.
>
> Ted


To me it sounded very useful.

[-- Attachment #2: Type: text/html, Size: 1972 bytes --]

Re: inputting characters by hexadigit

[Stefan Monnier]

SM> Maybe I misunderstand what you mean by word syntax.
SM> Are you referring to something more like partial-completion-mode?

> That's close, but it looks for abbreviations rather than categories, and
> is nicest when you know the word you want in advance.  When you don't,
> you still have to walk through the list of candidates.

So I indeed do not know what you mean by word syntax.  I.e. were do your
categories come from?


        Stefan

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Thu, 24 Jul 2008 10:03:55 -0400 Stefan Monnier <monnier@IRO.UMontreal.CA> wrote: 

SM> Maybe I misunderstand what you mean by word syntax.
SM> Are you referring to something more like partial-completion-mode?

>> That's close, but it looks for abbreviations rather than categories, and
>> is nicest when you know the word you want in advance.  When you don't,
>> you still have to walk through the list of candidates.

SM> So I indeed do not know what you mean by word syntax.  I.e. were do your
SM> categories come from?

Sorry if I'm confusing you or myself.

The categories are what you get when you split all the candidate strings
based on the word syntax.  partial-completion-mode does the same thing,
the difference is that it expects you to know that you can use z-x-y
because there's a zeta-xray-yogurt variable.  My proposal tries to
display just the categories to condense the list of candidates, refining
and digging deeper into the categories whenever possible until there's a
small enough list of candidates.

Ted

Re: inputting characters by hexadigit

[Stefan Monnier]

> The categories are what you get when you split all the candidate strings
> based on the word syntax.  partial-completion-mode does the same thing,
> the difference is that it expects you to know that you can use z-x-y
> because there's a zeta-xray-yogurt variable.  My proposal tries to
> display just the categories to condense the list of candidates, refining
> and digging deeper into the categories whenever possible until there's a
> small enough list of candidates.

So, it's like a substring search (like what you get with * A P L TAB),
except that the substring is anchored at the beginning of words, and
that *Completions* only lists the words rather than actual completion
candidates (on the assumption that there is a fair bit of redundancy in
the list of words).  It sounds interesting.

The "anchored substring search" should be fairly easy to implement as
a new completion-style.  The *Completions* change would require less
"modular/configurable" changes, I suspect, but it's worth trying it out.


        Stefan

RE: inputting characters by hexadigit

[Drew Adams]

> So, it's like a substring search (like what you get with * A P L TAB),
> except that the substring is anchored at the beginning of words, and
> that *Completions* only lists the words rather than actual completion
> candidates (on the assumption that there is a fair bit of 
> redundancy in the list of words).  It sounds interesting.
> 
> The "anchored substring search" should be fairly easy to implement as
> a new completion-style.  The *Completions* change would require less
> "modular/configurable" changes, I suspect, but it's worth trying it out.

I can't believe this. So now we're going to muck with existing completion
mechanisms even more, trying out new features, just for this Unicode character
input thing?

How about finishing what you were doing to move the completion code to Lisp, and
adding the `completing-read-function' variable we've been waiting for? Pings
about this get no reply, but you are apparently considering trying out new
completion features.

The so-called translation to Lisp has already resulted in adding new "features"
with no discussion here. The addition of `completing-read-function', unlike
those changes, will not require users to change their code in any way. How about
it? Can you please bring `completing-read' into line with `read-file-name' by
giving it a `completing-read-function' variable? Thanks.

-------

> > >> > >> How about having `completing-read' just call a
> > >> > >> `completing-read-function' variable if non-nil?
> > >> > >> This is the same thing that `read-file-name' does, with
> > >> > >> `read-file-name-function'.
> > >> > >
> > >> > > It sounds useful to authors of Emacs extensions and 
> > >> > > might even be useful directly for end users, letting
> > >> > > them decide what completing-read function they feel
> > >> > > like using on any particular day.
> > >> >
> > >> > I agree. Is there any reason not to add 
> > >> > completing-read-function?
> > >>
> > >> No one has objected to the idea.
> > >> Could someone please implement this?
> > >
> > > Any news on this?
> > 
> > Let's wait when Stefan moves completing-read to 
> > minibuffer.el and then just add
> > (if completing-read-function
> >     (funcall completing-read-function ;-)
> 
> Any news?

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Thu, 24 Jul 2008 08:55:03 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> Speaking only for Icicles, what you type in the minibuffer automatically
DA> (incrementally) filters the candidates (by default, at least), so I don't see
DA> the problem you mention.

DA> It sounds like you want to be able to also filter on category, that is, filter
DA> on the pair: category + name-in-category. For that, in Icicles, you can use a
DA> multi-completion that includes the category.

DA> What you described was actually, IIUC, to *first* filter on category, *then*
DA> filter on name-in-category. That too is possible in Icicles, but filtering on
DA> either or both, as I mentioned above, is better - quicker and more general.

I don't know how all that maps to Icicles and your terminology, but what
I'm proposing is really very simple.  I'll explain it differently:

-> user types `M-x TAB'
level = 0 (this is the deepest category level)
candidates = ankle-beta-copy, ankle-boring-call, 200 others with 5 common first words
display: ankle(2) right(100) ring(100) rung(100) wrong(100) zeta(100)

-> user types `a TAB'
level = 1 (but we don't need to display categories!)
candidates = ankle-beta-copy, ankle-boring-call
display: ankle-beta-copy, ankle-boring-call (with `a' highlighted)

-> user types `DEL TAB'
(back to first state)

-> user types `r TAB'
level = 1
candidates = 300 beginning with `r'
display: right(100) ring(100) rung(100)

-> user types `ing-r TAB'
level = 2 (we need to display categories, too many candidates)
candidates = 85 beginning with `ring-r'
display: ring-rock(20) ring-ring(25) ring-roar(40)

Basically at any point if there are less than N candidates, just show
the list.  Otherwise, generate the categories for N candidates by
constructing a tree of words in the current candidates' names (split the
names according to the word syntax).  Now, prune the top node (making
its child the top) until the top has more than 1 child.  Then the top's
children are the available categories (but they must be displayed with
the prefixes we pruned earlier).  I think it's a very simple algorithm.

DA> In Icicles, you can also filter progressively, using multiple patterns, so you
DA> can easily filter first with category, then with name-in-category, if you like,
DA> using separate patterns. But you need not do things in that order, and you need
DA> not even treat the two separately at all.

I don't think Icicles is the right way to implement what I'm describing,
because it does a lot of other things which are fairly intrusive to the
user who like the normal completion behavior.

On Thu, 24 Jul 2008 10:00:15 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> So now we're going to muck with existing completion mechanisms even
DA> more, trying out new features, just for this Unicode character input
DA> thing?

It's useful, as I've explained, for any list with lots of members,
e.g. the candidates shown for M-x TAB.  I have no opinion on the other
issues you brought up, and I don't need this feature in Emacs itself
unless it requires changes at the C level.  But I do think it would be
useful, especially on a text terminal, where scrolling through the list
can be pretty painful.

Ted

RE: inputting characters by hexadigit

[Drew Adams]

> I don't know how all that maps to Icicles and your 
> terminology, but what I'm proposing is really very simple.
> I'll explain it differently:
...

Yes, I understood you. I replied because you mentioned Icicles, and with Icicles
your proposal is not needed (it offers no added benefit). I tried to explain
that, but I can explain it better off-list, if you are interested.

Wrt vanilla Emacs, I have nothing particular to say about your proposal.

BTW, what you describe is similar to what Imenu does: use submenus when
appropriate. In my own code (for emacs-lisp buffers), the Imenu submenus are
categories in the sense you use: User Options, Variables, Functions, Macros,
Keys in Maps... (http://www.emacswiki.org/cgi-bin/wiki/imenu%2b.el)

> DA> In Icicles, you can also filter progressively, using 
> DA> multiple patterns, so you can easily filter first with
> DA> category, then with name-in-category, if you like,
> DA> using separate patterns. But you need not do things in 
> DA> that order, and you need not even treat the two separately at all.
> 
> I don't think Icicles is the right way to implement what I'm 
> describing, because it does a lot of other things which are
> fairly intrusive to the user who like the normal completion behavior.

I didn't suggest that Icicles be added to Emacs or that its implementation be
copied. I was letting you know that you can do what you want with Icicles, in
case you're interested. You brought up Icicles; I did not:

> >> When you know what's in the list, isearch,
> >> partial-completion-mode, icicles, etc. work great.

Even when you do not know what is in the list, Icicles works great in your use
case. That was my point. 

That, and letting you know that with Icicles you can add categories (or any
other info) to completion candidates, and you can then filter on (complete
against) the category as well as the rest of the candidate text.

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Thu, 24 Jul 2008 11:37:00 -0700 "Drew Adams" <drew.adams@oracle.com> wrote: 

DA> Yes, I understood you. I replied because you mentioned Icicles, and with Icicles
DA> your proposal is not needed (it offers no added benefit). I tried to explain
DA> that, but I can explain it better off-list, if you are interested.

I understand, and thanks for explaining.

Ted

Re: inputting characters by hexadigit

[Stefan Monnier]

> I can't believe this. So now we're going to muck with existing
> completion mechanisms even more, trying out new features, just for
> this Unicode character input thing?

Looks like a sensitive subject, huh?

> How about finishing what you were doing to move the completion code to
> Lisp,

It's finished for now.

> and adding the `completing-read-function' variable we've been
> waiting for?

Talking like a king?  I'm not opposed to such a change, but I have other
things to do.  So if someone else wants to install such a thing, he
should go right ahead.  In the mean time, defadvice works well for
such things.

> The so-called translation to Lisp has already resulted in adding new
> "features" with no discussion here.

The purpose of the translation was to then facilitate the introduction
of new features.  So, yes, there are new features, of course.  Indeed,
I have no discussed them but instead imposed them by fiat, waiting for
the reactions.  Surprisingly enough, I basically haven't heard any
complaint about the new behavior(s), even though it enables
the equivalent of partial-completion-mode by default.

> The addition of `completing-read-function', unlike those changes, will
> not require users to change their code in any way.

The new completion code is about as compatible to the old as I could
make it.  I can't remember the problems it introduced, but IIRC they
were all unrelated to the translation itself but instead related to
other bugfixes that were applied at the same time.

BTW, I regret having forgotten about the incompatibilities.  It would be
good to keep track of them somewhere, to help other package developpers.


        Stefan

RE: inputting characters by hexadigit

[Drew Adams]

> > >> > >> How about having `completing-read' just call a
> > >> > >> `completing-read-function' variable if non-nil?
> > >> > >> This is the same thing that `read-file-name' does,
> > >> > >> with `read-file-name-function'.
> > >> > >
> > >> > > It sounds useful to authors of Emacs extensions and 
> > >> > > might even be useful directly for end users, letting
> > >> > > them decide what completing-read function they feel
> > >> > > like using on any particular day.
> > >> >
> > >> > I agree. Is there any reason not to add 
> > >> > completing-read-function?
> > >>
> > >> No one has objected to the idea.
> > >> Could someone please implement this?
> > 
> > Let's wait when Stefan moves completing-read to 
> > minibuffer.el and then just add
> > (if completing-read-function
> >     (funcall completing-read-function ;-)


SM> I'm not opposed to such a change, but I have other things
SM> to do. So if someone else wants to install such a thing,
SM> he should go right ahead.

I hope someone will please do that. Thanks.

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Drew Adams wrote:
>>>>>>>> How about having `completing-read' just call a
>>>>>>>> `completing-read-function' variable if non-nil?
>>>>>>>> This is the same thing that `read-file-name' does,
>>>>>>>> with `read-file-name-function'.
>>>>>>> It sounds useful to authors of Emacs extensions and 
>>>>>>> might even be useful directly for end users, letting
>>>>>>> them decide what completing-read function they feel
>>>>>>> like using on any particular day.
>>>>>> I agree. Is there any reason not to add 
>>>>>> completing-read-function?
>>>>> No one has objected to the idea.
>>>>> Could someone please implement this?
>>> Let's wait when Stefan moves completing-read to 
>>> minibuffer.el and then just add
>>> (if completing-read-function
>>>     (funcall completing-read-function ;-)
> 
> 
> SM> I'm not opposed to such a change, but I have other things
> SM> to do. So if someone else wants to install such a thing,
> SM> he should go right ahead.
> 
> I hope someone will please do that. Thanks.

Could not this be done by just renaming the C function completing-read 
to completing-read-1 and then adding

(defun completing-read (prompt
                         collection
                         &optional predicate require-match
                         initial-input hist def
                         inherit-input-method)
   ... documentation ...
   (if completing-read-function
       (funcall completing-read-function ...)
     (completing-read-1 ...)))

to minibuffer.el?

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Lennart Borgman (gmail) wrote:
> Drew Adams wrote:
>>>>>>>>> How about having `completing-read' just call a
>>>>>>>>> `completing-read-function' variable if non-nil?
>>>>>>>>> This is the same thing that `read-file-name' does,
>>>>>>>>> with `read-file-name-function'.
>>>>>>>> It sounds useful to authors of Emacs extensions and might even 
>>>>>>>> be useful directly for end users, letting
>>>>>>>> them decide what completing-read function they feel
>>>>>>>> like using on any particular day.
>>>>>>> I agree. Is there any reason not to add completing-read-function?
>>>>>> No one has objected to the idea.
>>>>>> Could someone please implement this?
>>>> Let's wait when Stefan moves completing-read to minibuffer.el and 
>>>> then just add
>>>> (if completing-read-function
>>>>     (funcall completing-read-function ;-)
>>
>>
>> SM> I'm not opposed to such a change, but I have other things
>> SM> to do. So if someone else wants to install such a thing,
>> SM> he should go right ahead.
>>
>> I hope someone will please do that. Thanks.
> 
> Could not this be done by just renaming the C function completing-read 
> to completing-read-1 and then adding
> 
> (defun completing-read (prompt
>                         collection
>                         &optional predicate require-match
>                         initial-input hist def
>                         inherit-input-method)
>   ... documentation ...
>   (if completing-read-function
>       (funcall completing-read-function ...)
>     (completing-read-1 ...)))
> 
> to minibuffer.el?


If implemented like this then completing-read-1 and completing-read will 
have identical argument lists. How should the doc strings for those 
functions be handled in a case like this?

Is it ok to refer from completing-read to completing-read-1 and keep the 
full doc in the C code? (I guess it is very useful in the C code as a 
comment to the code.)

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Lennart Borgman (gmail) wrote:
>> Could not this be done by just renaming the C function completing-read 
>> to completing-read-1 and then adding
>>
>> (defun completing-read (prompt
>>                         collection
>>                         &optional predicate require-match
>>                         initial-input hist def
>>                         inherit-input-method)
>>   ... documentation ...
>>   (if completing-read-function
>>       (funcall completing-read-function ...)
>>     (completing-read-1 ...)))
>>
>> to minibuffer.el?
> 
> 
> If implemented like this then completing-read-1 and completing-read will 
> have identical argument lists. How should the doc strings for those 
> functions be handled in a case like this?
> 
> Is it ok to refer from completing-read to completing-read-1 and keep the 
> full doc in the C code? (I guess it is very useful in the C code as a 
> comment to the code.)

And it can not be done that simple ;-(

There are some calls to Fcompleting_read (the C imlementation of 
completing-read). This makes is necessary to write a new C function 
taking care of these calls. This new function must be able to call the 
lisp function pointed to by completing-read-function.

I do not know how to do that.

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Thu, 24 Jul 2008 22:37:45 +0200 "Lennart Borgman (gmail)" <lennart.borgman@gmail.com> wrote: 

LB> Could not this be done by just renaming the C function completing-read
LB> to completing-read-1 and then adding

LB> (defun completing-read (prompt
LB>                         collection
LB>                         &optional predicate require-match
LB>                         initial-input hist def
LB>                         inherit-input-method)
LB>   ... documentation ...
LB>   (if completing-read-function
LB>       (funcall completing-read-function ...)
LB>     (completing-read-1 ...)))

LB> to minibuffer.el?

I've seen a lot of this monkey-patching in Gnus and I can't say I like
it much.  I forget who said that "if a function has more than 7
arguments you forgot some" but it's especially true with complicated
functions like this.  So rather than renaming the function to get more
functionality, we should rethink the API and find ways to simplify it.

Are you going to try to implement this?  It seems like you're
interested; my time is very limited currently so I won't be able to do
it soon but I can help with testing and some code.

Ted

Re: inputting characters by hexadigit

[Juri Linkov]

>> Can completions be cascaded somehow?  The first tier would show all the
>> common first words, e.g. ... AEGEAN APL GREEK ... and then selecting
>> something from the first tier would cascade down to the second tier.
>
> The slow display should only happen when the list is really long,
> i.e. basically if you hit TAB with an empty minibuffer.  So we could
> indeed easily use a different completion behavior in this case.

When I type TAB in the empty minibuffer, I really want to see all
completions even when the list is really long, to be able to use isearch
to find a completion candidate etc.  Everyone who wants to narrow the
completion list can type the first word like "latin TAB", so I think the
current completion behavior is satisfactory.

I'm now trying to improve the performance of Unicode name completion.
To cache a list of (CHAR-NAME . CHAR-CODE) pairs I created a new
variable `ucs-names' and a function that returns its value or
creates a new list (time-consuming operation).  The intention is
to fill this list only when the user tries to complete or enters
a string that doesn't look like a hex number.

I also tried to use `lazy-completion-table' but it seems slower
than giving a ready alist in the `collection' arg of `completing-read'
though I didn't make measurements.  The current patch is below.
Can you see any problems with it?

Index: lisp/international/mule-cmds.el
===================================================================
RCS file: /sources/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.333
diff -c -r1.333 mule-cmds.el
*** lisp/international/mule-cmds.el	15 Jul 2008 18:15:03 -0000	1.333
--- lisp/international/mule-cmds.el	23 Jul 2008 22:34:03 -0000
***************
*** 2846,2855 ****
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive "sUnicode (hex): ")
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))
--- 2849,2894 ----
  (defvar nonascii-insert-offset 0 "This variable is obsolete.")
  (defvar nonascii-translation-table nil "This variable is obsolete.")
  
+ (defvar ucs-names nil
+   "Alist of cached (CHAR-NAME . CHAR-CODE) pairs.")
+ 
+ (defun ucs-names ()
+   "Return alist of (CHAR-NAME . CHAR-CODE) pairs cached in `ucs-names'."
+   (or ucs-names
+       (setq ucs-names
+ 	    (let (name names)
+ 	      (dotimes (c #xEFFFF)
+ 		(unless (or
+ 			 (and (>= c #x3400 ) (<= c #x4dbf )) ; CJK Ideograph Extension A
+ 			 (and (>= c #x4e00 ) (<= c #x9fff )) ; CJK Ideograph
+ 			 (and (>= c #xd800 ) (<= c #xfaff )) ; Private/Surrogate
+ 			 (and (>= c #x20000) (<= c #x2ffff)) ; CJK Ideograph Extension B
+ 			 )
+ 		  (if (setq name (get-char-code-property c 'name))
+ 		      (setq names (cons (cons name c) names)))
+ 		  (if (setq name (get-char-code-property c 'old-name))
+ 		      (setq names (cons (cons name c) names)))))
+ 	      names))))
+ 
+ (defvar ucs-completions (lazy-completion-table ucs-completions ucs-names)
+   "Lazy completion table for completing on Unicode character names.")
+ 
+ (defun read-char-by-name (prompt)
+   "Read a character by its Unicode name or hex number string.
+ Display PROMPT and read a string that represents a character
+ by its Unicode property `name' or `old-name'.  It also accepts
+ a hexadecimal number of Unicode code point.  Returns a character
+ as a number."
+   (let* ((completion-ignore-case t)
+ 	 (input (completing-read prompt ucs-completions)))
+     (or (and (string-match "^[0-9a-fA-F]+$" input)
+ 	     (string-to-number input 16))
+ 	(cdr (assoc input (ucs-names))))))
+ 
  (defun ucs-insert (arg)
    "Insert a character of the given Unicode code point.
  Interactively, prompts for a hex string giving the code."
!   (interactive (list (read-char-by-name "Unicode (hex or name): ")))
    (or (integerp arg)
        (setq arg (string-to-number arg 16)))
    (if (or (< arg 0) (> arg #x10FFFF))

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Lennart Borgman (gmail)]

Juri Linkov wrote:
>>> Can completions be cascaded somehow?  The first tier would show all the
>>> common first words, e.g. ... AEGEAN APL GREEK ... and then selecting
>>> something from the first tier would cascade down to the second tier.
>> The slow display should only happen when the list is really long,
>> i.e. basically if you hit TAB with an empty minibuffer.  So we could
>> indeed easily use a different completion behavior in this case.
> 
> When I type TAB in the empty minibuffer, I really want to see all
> completions even when the list is really long, to be able to use isearch
> to find a completion candidate etc.  Everyone who wants to narrow the
> completion list can type the first word like "latin TAB", so I think the
> current completion behavior is satisfactory.


To me the suggestion that someone (sorry forgot) sent to complete word 
by word looks very useful here.

And I would even suggest it as a general completion feature.

Re: inputting characters by hexadigit

[Stefan Monnier]

> And I would even suggest it as a general completion feature.

Try * A TAB: the completion list will show the complete names, but only
for those char names that contain an A, so * A P L TAB will list all the
chars that contain "APL" in their name.


        Stefan

Re: inputting characters by hexadigit

[Juri Linkov]

>> And I would even suggest it as a general completion feature.
>
> Try * A TAB: the completion list will show the complete names, but only
> for those char names that contain an A, so * A P L TAB will list all the
> chars that contain "APL" in their name.

BTW, I encountered one problem with this functionality.

For instance, when there is a buffer with the name "*grep*", typing
`C-x b *g TAB' completes to this name, good.  But when I do the same
and the *grep* buffer doesn't exist for various reasons then the
completion list is confusing, it displays a large list of irrelevant
completions.

This is a minor problem but it would be better to improve this somehow.

-- 
Juri Linkov
http://www.jurta.org/emacs/

Re: inputting characters by hexadigit

[Ted Zlatanov]

On Thu, 24 Jul 2008 01:35:38 +0300 Juri Linkov <juri@jurta.org> wrote: 

JL> When I type TAB in the empty minibuffer, I really want to see all
JL> completions even when the list is really long, to be able to use isearch
JL> to find a completion candidate etc.  Everyone who wants to narrow the
JL> completion list can type the first word like "latin TAB", so I think the
JL> current completion behavior is satisfactory.

On Wed, 23 Jul 2008 22:18:08 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Try * A TAB: the completion list will show the complete names, but only
SM> for those char names that contain an A, so * A P L TAB will list all the
SM> chars that contain "APL" in their name.

The problem is that you don't know that something at the end begins with
ZETA until you go to the end of the list.  When you know what's in the
list, isearch, partial-completion-mode, icicles, etc. work great.  I
don't think they should be disabled in favor of categorization, only
that when there's too much information for a screenful it should be
condensed (maybe something like folding-mode can be used to hide the
things in a category?)

Ted

RE: inputting characters by hexadigit

[Drew Adams]

> The problem is that you don't know that something at the end 
> begins with ZETA until you go to the end of the list.  When you know
> what's in the list, isearch, partial-completion-mode, icicles, etc.
> work great.  I don't think they should be disabled in favor of
> categorization, only that when there's too much information for a
> screenful it should be condensed (maybe something like folding-mode
> can be used to hide the things in a category?)

A bit OT:

Speaking only for Icicles, what you type in the minibuffer automatically
(incrementally) filters the candidates (by default, at least), so I don't see
the problem you mention.

It sounds like you want to be able to also filter on category, that is, filter
on the pair: category + name-in-category. For that, in Icicles, you can use a
multi-completion that includes the category.

What you described was actually, IIUC, to *first* filter on category, *then*
filter on name-in-category. That too is possible in Icicles, but filtering on
either or both, as I mentioned above, is better - quicker and more general.

In Icicles, you can also filter progressively, using multiple patterns, so you
can easily filter first with category, then with name-in-category, if you like,
using separate patterns. But you need not do things in that order, and you need
not even treat the two separately at all.

Re: describe-char should display the character's Unicode name

[Juri Linkov]

>> You can use the input method "ucs" to input any Unicode
>> character by typing its codepoint.  A method by typing
>> Unicode name is not yet implemented.
>
> An input method is not a good solution because while input methods are
> good for heavy use, they're quite annoying to juggle for occasional inputs.

Yes, it is inconvenient to activate an input method to insert just one
Unicode character.

>>> `ucs-insert'.  Is `ucs-insert' already obsolete?
>>
>> We can make it obsolete now.
>
> Please don't, until there's a better replacement.
>
> [One possibility would be an addition to C-q to make it accept "# hex-digits"
> or something.]

By the word "obsolete" I meant developing a better method to input
Unicode characters than `ucs-insert'.  Of course, there is no sense
to remove `ucs-insert'.

-- 
Juri Linkov
http://www.jurta.org/emacs/