fontsets/charsets documentation suggestion

fontsets/charsets documentation suggestion

[Eric Abrahamsen]

Hi,

Pursuant to this [1] thread on the carbon Emacs mailing list, I'm  
writing in with a small request regarding the documentation for  
fontsets in Emacs 23. I know that fonts, fontsets and charsets are one  
of the more complicated aspects of Emacs, but I think for this exact  
reason the documentation for fontsets and charsets should be beefed  
up. Everything in the current docs is correct, but it is such a  
complicated issue that I think a more basic, ramped-up (almost  
tutorial-style) approach would be very helpful, particularly for  
readers unfamiliar with the issue to begin with. In detail:

1. Clearer separation of Linux, Windows and Mac font issues – they  
seem to be very different.

2. A glossary. What is a charset, what is a font registry, what is a  
script name symbol, what is encoding and how are these things related?  
A ground-up explanation of the ecology of bytes, encodings, charsets  
and fonts would be great. Even, dare I say it, a diagram. This stuff  
is very confusing.

3. What are the relevant functions and variables? As a non-expert user  
of middling programming ability, I was totally baffled by set-fontset- 
font and all the possible permutations of its arguments. Likewise for  
create-fontset-from-fontset-spec. Since those two functions seem to be  
the weapons of choice for font manipulation, their descriptions could  
stand to be about four times as long as they are, with LOTS more  
examples. And when you're trying to get your fontsets right  
(particularly with multiple languages), you need to see lists of  
possible charsets and registries. These lists are in variables like  
charset-list and charset-script-alist which *aren't mentioned in the  
docs*.

4. I think describe-char is the main tool people use for examining  
characters. If that's the case, it would be nice to have clearer  
correspondences between the information that describe-char produces,  
and the information that set-fontset-font et al consumes. For  
instance, describe-char now seems to report all characters as either  
ascii or unicode-bmp -- this is less helpful when you're trying to  
target a font to latin or han, for instance.

That was it. Emacs 23 is enormously better at handling multilingual  
environments, but it would be even easier with more in-depth  
documentation!

Thanks,
Eric

[1] http://groups.google.com/group/carbon-emacs/browse_thread/thread/a919ee739fe7c9fd

Re: fontsets/charsets documentation suggestion

[Eli Zaretskii]

> From: Eric Abrahamsen <girzel@gmail.com>
> Date: Wed, 1 Jul 2009 23:19:51 -0700
> 
> 2. A glossary. What is a charset, what is a font registry, what is a  
> script name symbol, what is encoding and how are these things related?  
> A ground-up explanation of the ecology of bytes, encodings, charsets  
> and fonts would be great. Even, dare I say it, a diagram. This stuff  
> is very confusing.

This is in the manual, see the node "Character Sets" and maybe the
other nodes from its parent "Non-ASCII Characters", if you need more
background.

If something is missing or unclear, please post more specific gripes.

> 3. What are the relevant functions and variables? As a non-expert user  
> of middling programming ability, I was totally baffled by set-fontset- 
> font and all the possible permutations of its arguments. Likewise for  
> create-fontset-from-fontset-spec. Since those two functions seem to be  
> the weapons of choice for font manipulation, their descriptions could  
> stand to be about four times as long as they are, with LOTS more  
> examples. And when you're trying to get your fontsets right  
> (particularly with multiple languages), you need to see lists of  
> possible charsets and registries. These lists are in variables like  
> charset-list and charset-script-alist which *aren't mentioned in the  
> docs*.

Can you give a list of use-cases for which you'd like to have
examples, and/or point out what is unclear or insufficiently detailed
in the "Fontsets" node of the manual?

Re: fontsets/charsets documentation suggestion

[Eric Abrahamsen]

On Jul 2, 2009, at 12:15 PM, Eli Zaretskii wrote:

>> From: Eric Abrahamsen <girzel@gmail.com>
>> Date: Wed, 1 Jul 2009 23:19:51 -0700
>>
>> 2. A glossary. What is a charset, what is a font registry, what is a
>> script name symbol, what is encoding and how are these things  
>> related?
>> A ground-up explanation of the ecology of bytes, encodings, charsets
>> and fonts would be great. Even, dare I say it, a diagram. This stuff
>> is very confusing.
>
> This is in the manual, see the node "Character Sets" and maybe the
> other nodes from its parent "Non-ASCII Characters", if you need more
> background.
>
> If something is missing or unclear, please post more specific gripes.

Hmm, I wonder if this is part of the problem. Section 26 of my Emacs  
Info file is 'International Character Set Support', and everything  
relevant seems to be under this heading. I don't see a 'Character  
Sets' or a 'Non-ASCII Characters' node. There's no index entry for  
'registry', neither does a grep of the info files for 'registry'  
produce any definition of the word as related to fonts. Am I looking  
in the wrong place? (I'm using a CVS build on Mac OSX, the ns  
Emacs.app version.)

>
>> 3. What are the relevant functions and variables? As a non-expert  
>> user
>> of middling programming ability, I was totally baffled by set- 
>> fontset-
>> font and all the possible permutations of its arguments. Likewise for
>> create-fontset-from-fontset-spec. Since those two functions seem to  
>> be
>> the weapons of choice for font manipulation, their descriptions could
>> stand to be about four times as long as they are, with LOTS more
>> examples. And when you're trying to get your fontsets right
>> (particularly with multiple languages), you need to see lists of
>> possible charsets and registries. These lists are in variables like
>> charset-list and charset-script-alist which *aren't mentioned in the
>> docs*.
>
> Can you give a list of use-cases for which you'd like to have
> examples, and/or point out what is unclear or insufficiently detailed
> in the "Fontsets" node of the manual?

The existing examples would be clear if I'd gone into it actually  
understanding the difference between a charset, a script and a  
registry, and knowing which variables contained the possible values  
for each of these three concepts. My particular use-case was defining  
specific fonts for English/Latin, Chinese and Thai, and I did not  
understand where 'han' came from, and its relationship to 'chinese- 
gb2312' (I have since been directed to charset-script-alist). If I'm  
actually missing a chunk of the documentation this would be better.

There aren't any examples of using create-fontset-from-fontset-spec,  
and it would be nice to see one or two. The documentation for the  
function is technically correct, I'm sure, but is the kind of  
documentation that's only useful if you already understand how it  
works. Neither 26.15 Defining Fontsets nor the documentation say how  
you actually employ a fontset once you've made it. This has got to be  
one of the most complex areas of Emacs customization, and it would be  
nice to err on the side of too much hand-holding.

Thanks for your response, and I do hope that I'm just missing  
something...

Eric

Re: fontsets/charsets documentation suggestion

[Eli Zaretskii]

> Cc: emacs-devel@gnu.org
> From: Eric Abrahamsen <girzel@gmail.com>
> Date: Fri, 3 Jul 2009 18:40:56 -0700
> 
> > This is in the manual, see the node "Character Sets" and maybe the
> > other nodes from its parent "Non-ASCII Characters", if you need more
> > background.
> >
> > If something is missing or unclear, please post more specific gripes.
> 
> Hmm, I wonder if this is part of the problem. Section 26 of my Emacs  
> Info file is 'International Character Set Support', and everything  
> relevant seems to be under this heading. I don't see a 'Character  
> Sets' or a 'Non-ASCII Characters' node. There's no index entry for  
> 'registry', neither does a grep of the info files for 'registry'  
> produce any definition of the word as related to fonts. Am I looking  
> in the wrong place? (I'm using a CVS build on Mac OSX, the ns  
> Emacs.app version.)

I meant the Emacs Lisp manual, not the User manual.  Sorry, I should
have made that clear, but for some reason (perhaps because you
mentioned create-fontset-from-fontset-spec) I thought you were already
looking there, and that you need this information for writing Lisp
code.

> The existing examples would be clear if I'd gone into it actually  
> understanding the difference between a charset, a script and a  
> registry, and knowing which variables contained the possible values  
> for each of these three concepts. My particular use-case was defining  
> specific fonts for English/Latin, Chinese and Thai, and I did not  
> understand where 'han' came from, and its relationship to 'chinese- 
> gb2312' (I have since been directed to charset-script-alist). If I'm  
> actually missing a chunk of the documentation this would be better.
> 
> There aren't any examples of using create-fontset-from-fontset-spec,  
> and it would be nice to see one or two. The documentation for the  
> function is technically correct, I'm sure, but is the kind of  
> documentation that's only useful if you already understand how it  
> works. Neither 26.15 Defining Fontsets nor the documentation say how  
> you actually employ a fontset once you've made it. This has got to be  
> one of the most complex areas of Emacs customization, and it would be  
> nice to err on the side of too much hand-holding.

I agree that the description of fontsets in the User manual is barely
minimal, especially if the user (a) needs to create complex fontsets
dealing with several different character sets, and (b) does not
already know quite a lot about character sets.  The problem is that
most people who work on the manual know very little about subtleties
and intricacies of fontsets, and it clearly shows in the quality of
that section.

I wonder if the stuff in the ELisp manual helps in any way.  If it
does, we could copy some of it, with suitable editing, into the User
manual.

Re: fontsets/charsets documentation suggestion

[Eric Abrahamsen]

On Jul 4, 2009, at 1:07 AM, Eli Zaretskii wrote:

>> Cc: emacs-devel@gnu.org
>> From: Eric Abrahamsen <girzel@gmail.com>
>> Date: Fri, 3 Jul 2009 18:40:56 -0700
>>
>>> This is in the manual, see the node "Character Sets" and maybe the
>>> other nodes from its parent "Non-ASCII Characters", if you need more
>>> background.
>>>
>>> If something is missing or unclear, please post more specific  
>>> gripes.
>>
>> Hmm, I wonder if this is part of the problem. Section 26 of my Emacs
>> Info file is 'International Character Set Support', and everything
>> relevant seems to be under this heading. I don't see a 'Character
>> Sets' or a 'Non-ASCII Characters' node. There's no index entry for
>> 'registry', neither does a grep of the info files for 'registry'
>> produce any definition of the word as related to fonts. Am I looking
>> in the wrong place? (I'm using a CVS build on Mac OSX, the ns
>> Emacs.app version.)
>
> I meant the Emacs Lisp manual, not the User manual.  Sorry, I should
> have made that clear, but for some reason (perhaps because you
> mentioned create-fontset-from-fontset-spec) I thought you were already
> looking there, and that you need this information for writing Lisp
> code.

Good lord yes, that makes much more sense. I'm approaching this purely  
as a user trying to define a multi-lingual environment, not an Elisp  
coder, and some of the Elisp manual stuff is necessary to make that  
work.

>
>> The existing examples would be clear if I'd gone into it actually
>> understanding the difference between a charset, a script and a
>> registry, and knowing which variables contained the possible values
>> for each of these three concepts. My particular use-case was defining
>> specific fonts for English/Latin, Chinese and Thai, and I did not
>> understand where 'han' came from, and its relationship to 'chinese-
>> gb2312' (I have since been directed to charset-script-alist). If I'm
>> actually missing a chunk of the documentation this would be better.
>>
>> There aren't any examples of using create-fontset-from-fontset-spec,
>> and it would be nice to see one or two. The documentation for the
>> function is technically correct, I'm sure, but is the kind of
>> documentation that's only useful if you already understand how it
>> works. Neither 26.15 Defining Fontsets nor the documentation say how
>> you actually employ a fontset once you've made it. This has got to be
>> one of the most complex areas of Emacs customization, and it would be
>> nice to err on the side of too much hand-holding.
>
> I agree that the description of fontsets in the User manual is barely
> minimal, especially if the user (a) needs to create complex fontsets
> dealing with several different character sets, and (b) does not
> already know quite a lot about character sets.  The problem is that
> most people who work on the manual know very little about subtleties
> and intricacies of fontsets, and it clearly shows in the quality of
> that section.
>
> I wonder if the stuff in the ELisp manual helps in any way.  If it
> does, we could copy some of it, with suitable editing, into the User
> manual.

I think that's exactly what should be done. Emacs 23 is far better at  
providing multilingual support, which means more and more users are  
going to want to do it, and they're going to need more help.

So maybe a step-by-step tutorial with the following:

1. A really quick version of 33.1 Text Representations
2. How to find what fonts are on your machine and available to Emacs  
(with system-dependent variations).
3. The difference between charset, registry and script, and the  
relevant functions and variables.
4. How to read a font string.
5. What a fontset is, and what it looks like in its various guises.
6. How to find currently defined fontsets (ie list-fontsets), and how  
they relate to each other (I've got fontset-default, fontset-standard,  
and fontset-startup and I have no idea which of those I should be  
working with).
7. How to use set-fontset-font and create-fontset-from-fontset-spec to  
create and modify fontsets. Describe the argument variations within  
the manual, not just in the function docs.
8. How to examine existing characters in buffers (ie C-u C-x =), and  
to use that information to inform your usage of the above two functions.

I'd be happy to assist in whatever way possible – I could suggest more  
additions to the manual or read through new docs (I'm an editor, among  
other things). I know squat about fontsets, obviously, but maybe that  
could be useful.

Thanks for your responses on this!

Eric