Re: On language-dependent defaults for character-folding

[Richard Stallman <rms@gnu.org>]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Some minimal amount of folding will nevertheless be necessary even in
  > asymmetric mode, in order to find character sequences produced by
  > decomposing characters like ö into o and the combining mark ̈.  That's
  > because these two characters when juxtaposed (ö) look identical to the
  > precomposed character on most displays, so we should by default find
  > such decomposed sequences even when the search string includes the
  > precomposed character.

That is interesting.  It means we need several levels of folding:

* Different appearances of the same letter+decorations:
  as a single code point, or as a composition.

* Identical-looking distinct code points (Latin a and Cyrillic a).

* The same letter with different decorations (o and ö in English).

* Equivalent letters (ö and ø in Swedish).

* Non-equivalent letters modified from a common base (o and ö in
  Swedish).

The first level is language-independent and should be handled
symmetrically, with each folding group as an equivalence class.

Is there any need, ever, to disable the first level?
Perhaps it would be good to enable that all the time.

The second level is also language-independent.  Does anyone ever want
to turn it off?

The other levels are language-specific, and the user might want to
enable or disable them.  When enabled, the user might want them
handled symmetrically or asymmetrically.

-- 
Dr Richard Stallman
President, Free Software Foundation (gnu.org, fsf.org)
Internet Hall-of-Famer (internethalloffame.org)
Skype: No way! See stallman.org/skype.html.