Re: On language-dependent defaults for character-folding

[Eli Zaretskii <eliz@gnu.org>]

> From: Richard Stallman <rms@gnu.org>
> CC: eliz@gnu.org, larsi@gnus.org, emacs-devel@gnu.org
> Date: Mon, 22 Feb 2016 12:59:00 -0500
> 
>   > Right. I guess I'm getting ahead of myself. As you know, I'm advocating
>   > choosing a default language based on the locale of the user.
> 
> We need:
> 
> * A per-buffer language preference variable.
> * A global value which becomes the default for new buffers.

That's unnecessarily restrictive; we can do better with the current
infrastructure.  Some encodings provide us with charset information,
which can be used to deduce the language of the text.  Some characters
belong to Unicode blocks that allow identification of the language, or
maybe a small group of languages.  In some cases, the text itself
comes with metadata which describes the language.  And there might be
other sources of information about the language.

It would be silly to disregard this information where it exists.

There are other aspects of this that need to be considered, if we want
for language-specific searching to be solid.  E.g., what happens with
text copied to another buffer which might have a different per-buffer
language preference? does it suddenly behave differently when
searched?

But the most basic issue is that any significant development in these
directions require to re-implement the feature on the C level, and use
char-tables for folding, like we do with case-mapping.  So until
someone steps forward for the job, all we can do is small corrections
to the existing implementation.  For example, the default state of
character-folding might depend on the locale's language -- we could
turn it off by default for languages whose users expressed
dissatisfaction with the feature.  We could also augment the regular
expressions created for folding the search string by filtering out
variants that users of a particular language don't want.  If people
think these ideas will make more users happy, we can work on that.