bug#3526: 23.0.94; `list-character-sets' display

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

What is a "supplementary character set"? Why are such sets listed
separately? What sets are in the first, non "supplementary" list. IOW,
why two lists, and what is the difference?

In GNU Emacs 23.0.94.1 (i386-mingw-nt5.1.2600)
 of 2009-05-24 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'
 

bug#3526: 23.0.94; `list-character-sets' display

[Kenichi Handa]

In article <CF216C40E93948CE89C4D4BEC2702E86@us.oracle.com>, "Drew Adams" <drew.adams@oracle.com> writes:

> What is a "supplementary character set"? Why are such sets listed
> separately? What sets are in the first, non "supplementary" list. IOW,
> why two lists, and what is the difference?

A supplementary character set is a character set defined
with `:supplementary-p' flag.  The meaning of this flag is
(in the docstring of define-charset):

VALUE must be nil or t.  If the VALUE is t, the charset is
supplementary, which means it is used only as a parent or a
subset of some other charset, or it is provided just for backward
compatibility.

For instance, gb18030-2-byte, gb18030-4-byte-bmp,
gb18030-4-byte-ext-1, gb18030-4-byte-ext-2, and
gb18030-4-byte-smp are all supplementary character sets to
define gb18030 which is a superset of the above all.

arabic-1-column, arabic-2-column, arabic-digit are also
supplementary character sets that are provided for backward
compatiblity.

Priorities of supplementary character sets are always lower
than the other character set, so, for instance (char-charset
CHAR) never return a supplementary character set.

---
Kenichi Handa
handa@m17n.org

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

Thanks for this info. Maybe we could have a link from the `list-character-sets'
display to the Info node that explains this?

And maybe we could include a one-liner that explains (most of) what it means?
E.g., as the doc string says: "used only as a parent or a subset of some other
charset"

These things would help, I think: one-line reminder, and link to obtain more
info. I expect that many users are not too familiar with these things.

bug#3526: 23.0.94; `list-character-sets' display

[Kenichi Handa]

In article <8385521F908145EE8D60682CEAB9D7D1@us.oracle.com>, "Drew Adams" <drew.adams@oracle.com> writes:

> Thanks for this info. Maybe we could have a link from the
> `list-character-sets' display to the Info node that
> explains this?

> And maybe we could include a one-liner that explains (most of) what it means?
> E.g., as the doc string says: "used only as a parent or a subset of some other
> charset"

I updated the Info node (emacs)Charsets for the new way of
charsets handling and added @findex for list-character-sets
(in doc/emacs/mule.texi), and modified list-character-sets-1
to do above (in lisp/international/mule-diag.el).  Please
check the English wording of the new text and fix/improve
it.

---
Kenichi Handa
handa@m17n.org

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

> > Thanks for this info. Maybe we could have a link from the
> > `list-character-sets' display to the Info node that
> > explains this?
> 
> > And maybe we could include a one-liner that explains (most 
> of) what it means?
> > E.g., as the doc string says: "used only as a parent or a 
> subset of some other
> > charset"
> 
> I updated the Info node (emacs)Charsets for the new way of
> charsets handling and added @findex for list-character-sets
> (in doc/emacs/mule.texi), and modified list-character-sets-1
> to do above (in lisp/international/mule-diag.el).  Please
> check the English wording of the new text and fix/improve
> it.

Thanks for the fix.

If you can send me the new text or point me to a URL that has it, I'll take a
look at the English. If not, someone else can perhaps help with this. Thx.

bug#3526: 23.0.94; `list-character-sets' display

[Kenichi Handa]

In article <3057BD2C673B48079D31AC58DBF72C08@us.oracle.com>, "Drew Adams" <drew.adams@oracle.com> writes:
> > I updated the Info node (emacs)Charsets for the new way of
> > charsets handling and added @findex for list-character-sets
> > (in doc/emacs/mule.texi), and modified list-character-sets-1
> > to do above (in lisp/international/mule-diag.el).  Please
> > check the English wording of the new text and fix/improve
> > it.

> Thanks for the fix.

> If you can send me the new text or point me to a URL that has it, I'll take a
> look at the English. If not, someone else can perhaps help with this. Thx.

This is the one-line help in *Character Set List* buffer,
------------------------------------------------------------
Character sets for defining another charset or obsolete now
------------------------------------------------------------

and this is the "Charsets" node of mule.texi.

------------------------------------------------------------
@node Charsets
@section Charsets
@cindex charsets

  Emacs defines most of popular character sets (e.g. ascii,
iso-8859-1, cp1250, big5, unicode) as @dfn{charsets} and a few of its
own charsets (e.g. emacs, unicode-bmp, eight-bit).  All supported
characters belong to one or more charsets.  Usually you don't have to
take care of ``charset'', but knowing about it may help understanding
the behavior of Emacs in some cases.

  One example is a font selection.  In each language environment,
charsets have different priorities.  Emacs, at first, tries to use a
font that matches with charsets of higher priority.  For instance, in
Japanese language environment, the charset @code{japanese-jisx0208}
has the highest priority (@xref{describe-language-environment}).  So,
Emacs tries to use a font whose @code{registry} property is
``JISX0208.1983-0'' for characters belonging to that charset.

  Another example is a use of @code{charset} text property.  When
Emacs reads a file encoded in a coding systems that uses escape
sequences to switch charsets (e.g. iso-2022-int-1), the buffer text
keep the information of the original charset by @code{charset} text
property.  By using this information, Emacs can write the file with
the same byte sequence as the original.

@findex list-charset-chars
@cindex characters in a certain charset
@findex describe-character-set
  There are two commands for obtaining information about Emacs
charsets.  The command @kbd{M-x list-charset-chars} prompts for a
charset name, and displays all the characters in that character set.
The command @kbd{M-x describe-character-set} prompts for a charset
name and displays information about that charset, including its
internal representation within Emacs.

@findex list-character-sets
  To display a list of all the supported charsets, type @kbd{M-x
list-character-sets}.  The list gives the names of charsets and
additional information to identity each charset (see ISO/IEC's this
page <http://www.itscj.ipsj.or.jp/ISO-IR/> for the detail).  In the
list, charsets are categorized into two; the normal charsets are
listed first, and the supplementary charsets are listed last.  A
charset in the latter category is used for defining another charset
(as a parent or a subset), or was used only in Emacs of the older
versions.

  To find out which charset a character in the buffer belongs to,
put point before it and type @kbd{C-u C-x =}.
------------------------------------------------------------

---
Kenichi Handa
handa@m17n.org

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

> This is the one-line help in *Character Set List* buffer,
> ------------------------------------------------------------
> Character sets for defining another charset or obsolete now
> ------------------------------------------------------------

Did you mean "are" instead of "or"? If so, the English (with "are") is correct.

If this is a statement about something being obsolete that was previously
supported, then it belongs in NEWS, not here. The text here should just help
with what is displayed - it should not try to explain why something that was
previously explained is no longer explained.

(If that's not when this statement means, then I don't know what it is saying.)

> and this is the "Charsets" node of mule.texi.
> 
> ------------------------------------------------------------
> @node Charsets
> @section Charsets
> @cindex charsets
> 
>   Emacs defines most of popular character sets (e.g. ascii,
> iso-8859-1, cp1250, big5, unicode) as @dfn{charsets} and a few of its
> own charsets (e.g. emacs, unicode-bmp, eight-bit).

Emacs defines most popular character sets (e.g. ascii, iso-8859-1, cp1250, big5,
unicode), as well as a few of its own (e.g. emacs, unicode-bmp, eight-bit), as
@dfn{charsets}.

[Or better: End the sentence at the comma, and use a separate sentence to say
that Emacs refers to character sets as @dfn{charsets}.]

> All supported characters 

All characters supported by Emacs

[As opposed to all chars supported by one or more character sets.]

> belong to one or more charsets.  Usually you don't have to
> take care of ``charset'', but knowing about it may help understanding
> the behavior of Emacs in some cases.

I think something like this is what you want:

You generally do not need to pay attention to charsets as you use Emacs.
However, it can help to know a little about them.

>   One example is a font selection.  

One example is font selection.

[or "One example is selecting a font"]

> In each language environment,
> charsets have different priorities.  Emacs, at first, tries to use a
> font that matches with charsets of higher priority.  

Emacs first tries to use a font that matches charsets of higher priority.

[But why "first"? I don't see anything that counteracts this. So just "Emacs
tries".]

> For instance, in

a

> Japanese language environment, the charset @code{japanese-jisx0208}
> has the highest priority (@xref{describe-language-environment}).
> So,

So

> Emacs tries to use a font whose @code{registry} property is
> ``JISX0208.1983-0''
> for 

whose [not for]

> characters belonging to that charset.
> 
>   Another example is a use of @code{charset} text property.  

Another example is use of the @code{charset} text property.

> When Emacs reads a file encoded in a coding
> systems 

system

> that uses escape sequences to switch charsets (e.g. iso-2022-int-1),
> the buffer text keep the information of the original charset
> by @code{charset} text property.  
> By using this information, Emacs can write the file with
> the same byte sequence as the original.

that uses escape sequences to switch charsets (e.g. iso-2022-int-1),
text property @code{charset} is attached to the buffer text, to record the
charset in which the file is encoded. Emacs uses this information to be able to
write the file again using the original byte sequence.

> @findex list-charset-chars
> @cindex characters in a certain charset
> @findex describe-character-set
>   There are two commands for obtaining information about Emacs
> charsets.  The command @kbd{M-x list-charset-chars} prompts for a
> charset name, and displays all the characters in that character set.

charset name and displays all the characters in that character set.

> The command @kbd{M-x describe-character-set} prompts for a charset
> name and displays information about that charset, including its
> internal representation within Emacs.
> 
> @findex list-character-sets
>   To display a list of all the supported charsets, 

To display a list of all supported charsets,

> type @kbd{M-x list-character-sets}. 

> The list gives the names of charsets and
> additional information to identity each charset (see ISO/IEC's this
> page <http://www.itscj.ipsj.or.jp/ISO-IR/> for the detail).  

The list shows charset names and other identifying ISO/IEC information
(see <http://www.itscj.ipsj.or.jp/ISO-IR/> for details).

> In the
> list, charsets are categorized into two; the normal charsets are
> listed first, and the supplementary charsets are listed last. 

Charsets are listed in two groups: normal and supplementary charsets.

> A
> charset in the latter category is used for defining another charset
> (as a parent or a subset), or was used only in Emacs of the older
> versions.

A @dfn{supplementary charset} is used to define another charset as its parent or
subset. 

[parent or child? ancestor or descendent? superset or subset? Not sure what's
correct here, but parent and subset are not parallel.]

Some of the charsets listed as supplementary are obsolete; they were used in
Emacs versions prior to version 23.

[Is that correct? If so, why not get rid of these here? Why show stuff that is
no longer in Emacs?]

>   To find out which charset a character in the buffer belongs to,

To find out which charset a character belongs to,

> put point before it and type @kbd{C-u C-x =}.

[But this last sentence has nothing to do with `list-character-sets' display, so
it should be moved elsewhere (e.g. earlier in the node). You might also want to
cross-ref another node that mentions `C-x =' (and vice versa).

HTH. Someone else should check that the content makes sense using the edits I
suggested. I don't want to mess up the meaning by guessing wrong.

Thx - Drew

bug#3526: 23.0.94; `list-character-sets' display

[Jason Rumney]

Drew Adams wrote:

>> Emacs tries to use a font whose @code{registry} property is
>> ``JISX0208.1983-0''
>> for 
>>     
>
> whose [not for]
>
>   
>> characters belonging to that charset.
>>     

I'd be more inclined to remove a `whose' than add another one to that 
sentence.

So Emacs tries to use a font with a @code{registry} property of ``JISX0208.1983-0'' for characters belonging to that charset.


> A @dfn{supplementary charset} is used to define another charset as its parent or
> subset. 
>
> [parent or child? ancestor or descendent? superset or subset? Not sure what's
> correct here, but parent and subset are not parallel.]
>   

I think the parent is the "another charset", and subset is the 
"supplementary charset". But as worded above, it isn't clear which 
charset "its parent or subset" is referring to anyway, so I'd reword it 
further:

A @dfn{supplementary charset} is used to define another charset. It does 
not form a complete charset on its own, so should not generally be used 
directly.

> Some of the charsets listed as supplementary are obsolete; they were used in
> Emacs versions prior to version 23.
>
> [Is that correct? If so, why not get rid of these here? Why show stuff that is
> no longer in Emacs?]
>   
I think charsets like mule-unicode-0100-23ff, which existed in previous 
versions because of technical restrictions on the size of charsets, may 
be referred to by old lisp code, so we need to keep them around for 
compatibility.

>>   To find out which charset a character in the buffer belongs to,
>>     
>
> To find out which charset a character belongs to,
>   
Since the described function acts on a character in a buffer, I think 
the original wording is correct.

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

Your changes look good to me, Jason. 

> > Some of the charsets listed as supplementary are obsolete; 
> > they were used in Emacs versions prior to version 23.
> >
> > [Is that correct? If so, why not get rid of these here? Why 
> > show stuff that is no longer in Emacs?]
>  
> I think charsets like mule-unicode-0100-23ff, which existed 
> in previous versions because of technical restrictions on the
> size of charsets, may be referred to by old lisp code, so we
> need to keep them around for compatibility.

That we need, for backward compatibility, to keep around code that is not used
by the current version is one thing. That doesn't mean, however, that we should
show those charsets to the Emacs 23 user in a help display.

I can't judge whether that display could be useful to Emacs 23 users or not -
it's your call. Typically, only the current release is described in user doc
(and in the UI). Compatibility issues are sometimes called out in the doc, but
more often that is done in release notes (e.g. NEWS).

> >>   To find out which charset a character in the buffer belongs to,
> >
> > To find out which charset a character belongs to,
>   
> Since the described function acts on a character in a buffer, I think 
> the original wording is correct.

"the buffer" confuses more than it helps here, IMO. It could easily be taken to
refer to the list-charsets display buffer, given the context so far. And "put
point before it and type @kbd{C-u C-x =}" can only apply to a character in a
(displayed) buffer anyway, so "in the buffer" is redundant at best and confusing
at worst.

But if this sentence is moved, as I suggested, so that the context is no longer
the list charsets display, then that possible confusion can be removed.
Personally, I would say just this:

"To show the charset of the character at point, use @kbd{C-u C-x =}."

There is no need to describe, in addition, how to make a given character become
the character at point.

bug#3526: 23.0.94; `list-character-sets' display

[Kenichi Handa]

In article <0F745F1932EF4338B8D737D20FB2B94F@us.oracle.com>, "Drew Adams" <drew.adams@oracle.com> writes:

> > This is the one-line help in *Character Set List* buffer,
> > ------------------------------------------------------------
> > Character sets for defining another charset or obsolete now
> > ------------------------------------------------------------

> Did you mean "are" instead of "or"? If so, the English (with "are") is correct.

I mean "or".  What I want to say is:

"Character sets listed below exist:
  * for defining another charset, or
  * for backward compatibility."

> If this is a statement about something being obsolete that was previously
> supported, then it belongs in NEWS, not here. The text here should just help
> with what is displayed - it should not try to explain why something that was
> previously explained is no longer explained.

> (If that's not when this statement means, then I don't know what it is saying.)

> > and this is the "Charsets" node of mule.texi.
> > 
> > ------------------------------------------------------------
> > @node Charsets
> > @section Charsets
> > @cindex charsets
> > 
> >   Emacs defines most of popular character sets (e.g. ascii,
> > iso-8859-1, cp1250, big5, unicode) as @dfn{charsets} and a few of its
> > own charsets (e.g. emacs, unicode-bmp, eight-bit).

> Emacs defines most popular character sets (e.g. ascii, iso-8859-1, cp1250, big5,
> unicode), as well as a few of its own (e.g. emacs, unicode-bmp, eight-bit), as
> @dfn{charsets}.

> [Or better: End the sentence at the comma, and use a separate sentence to say
> that Emacs refers to character sets as @dfn{charsets}.]

Thank you for the correction for mule.texi.  But, it's not
efficient that I hear you (and Jason's) comments in mail,
and commit changes.  Could you please directly commit the
changes?   Of course, I'll answer questions.

> > All supported characters 

> All characters supported by Emacs

> [As opposed to all chars supported by one or more character sets.]

> > belong to one or more charsets.  Usually you don't have to
> > take care of ``charset'', but knowing about it may help understanding
> > the behavior of Emacs in some cases.

> I think something like this is what you want:

> You generally do not need to pay attention to charsets as you use Emacs.
> However, it can help to know a little about them.

Yes.

> >   One example is a font selection.  

> One example is font selection.

> [or "One example is selecting a font"]

> > In each language environment,
> > charsets have different priorities.  Emacs, at first, tries to use a
> > font that matches with charsets of higher priority.  

> Emacs first tries to use a font that matches charsets of higher priority.

> [But why "first"? I don't see anything that counteracts this. So just "Emacs
> tries".]

Ok for deleting that "at first".

---
Kenichi Handa
handa@m17n.org

bug#3526: 23.0.94; `list-character-sets' display

[Drew Adams]

> I mean "or".  What I want to say is:
> "Character sets listed below exist:
>   * for defining another charset, or
>   * for backward compatibility."

I would not try to say that in an intro sentence - I would drop the sentence
altogether.

That is said more completely further down, where you define supplementary
charsets and explain that some of the supplementary charsets listed are
obsolete. That is exactly where that info belongs, IMO.

> Thank you for the correction for mule.texi.  But, it's not
> efficient that I hear you (and Jason's) comments in mail,
> and commit changes.  Could you please directly commit the
> changes?   Of course, I'll answer questions.

Not I, but perhaps Jason will do that.

bug#3526: marked as done (23.0.94; `list-character-sets' display)

[Emacs bug Tracking System]

[-- Attachment #1: Type: text/plain, Size: 876 bytes --]

Your message dated Sat, 31 Oct 2009 15:32:32 -0400
with message-id <87k4yb2wbz.fsf@stupidchicken.com>
and subject line Re: bug#3526: 23.0.94; `list-character-sets' display
has caused the Emacs bug report #3526,
regarding 23.0.94; `list-character-sets' display
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner@emacsbugs.donarmstrong.com
immediately.)


-- 
3526: http://emacsbugs.donarmstrong.com/cgi-bin/bugreport.cgi?bug=3526
Emacs Bug Tracking System
Contact owner@emacsbugs.donarmstrong.com with problems

[-- Attachment #2: Type: message/rfc822, Size: 3042 bytes --]

From: "Drew Adams" <drew.adams@oracle.com>
To: <emacs-pretest-bug@gnu.org>
Subject: 23.0.94; `list-character-sets' display
Date: Wed, 10 Jun 2009 13:27:21 -0700
Message-ID: <CF216C40E93948CE89C4D4BEC2702E86@us.oracle.com>

What is a "supplementary character set"? Why are such sets listed
separately? What sets are in the first, non "supplementary" list. IOW,
why two lists, and what is the difference?

In GNU Emacs 23.0.94.1 (i386-mingw-nt5.1.2600)
 of 2009-05-24 on SOFT-MJASON
Windowing system distributor `Microsoft Corp.', version 5.1.2600
configured using `configure --with-gcc (3.4)'
 



[-- Attachment #3: Type: message/rfc822, Size: 1692 bytes --]

From: Chong Yidong <cyd@stupidchicken.com>
To: "Drew Adams" <drew.adams@oracle.com>
Cc: "Kenichi Handa" <handa@m17n.org>, jasonr@gnu.org, 3526-done@emacsbugs.donarmstrong.com
Subject: Re: bug#3526: 23.0.94; `list-character-sets' display
Date: Sat, 31 Oct 2009 15:32:32 -0400
Message-ID: <87k4yb2wbz.fsf@stupidchicken.com>

I have copyedited the Charsets node in the Emacs manual, and fixed the
message in list-character-sets-1.  I think this resolves all the issued
raised in this thread.