character folding future [was: Questions about isearch]

[Drew Adams <drew.adams@oracle.com>]

> Ok. I'm going to work on the char-folding a little bit more today to
> implement support for multi-char matches and to combine it with
> case-folding. Hopefully that will iron out the final inconsistencies.

Thanks for working on this, Artur.  I invite you to also
take a look at some code I wrote for this, which I've put
in `character-fold+.el'.  It follows a previous discussion.

Any of that, or similar, that gets added to vanilla Emacs
will mean one less thing for me to bother with. ;-)

A description is here:
http://www.emacswiki.org/emacs/CharacterFoldPlus.
The code is here:
http://www.emacswiki.org/emacs/download/character-fold%2b.el

The additions are essentially these:

1. An option, `char-fold-ad-hoc', for the ad hoc char foldings.
   Default value: the same ad hoc foldings as vanilla Emacs
   (quotation marks).

2. A Boolean option, `char-fold-symmetric', which when non-nil
   means that all members of a folding equivalence class are
   treated equivalently, whether base char, compositions, or
   other strings of chars.  This lets you search for e' or é
   and find e and any of the other members of its class
   (including composition strings).  The default value is nil
   (off).

3. A general workhorse function, `update-char-fold-table',
   that updates the value of variable `character-fold-table'
   (from which it was derived).  It is used when option
   `char-fold-symmetric' is toggled, and it makes use of
   options `char-fold-ad-hoc' and `char-fold-symmetric'.

4. `character-fold-to-regexp' is advised, to reflect whether
   char folding is currently symmetric.

Library Isearch+ provides a toggle for `char-fold-symmetric',
bound by default to `M-s =' during Isearch.

Another Isearch toggle can be useful when char folding is
symmetric: `M-s h L', which toggles lazy highlighting, which
can slow things down when using symmetric char folding.

The code for `isearch+.el' is here:
http://www.emacswiki.org/emacs/download/isearch%2b.el


Earlier, I invited a discussion about future customization
of character folding (and folding in general).  That hasn't
happened, so far.  But `char-fold-ad-hoc' could be a start.

One possibility is for an alist option, whose entries would
each be a list (MODES CLASSES), where CLASSES is a list of
char-folding classes such as that of `char-fold-ad-hoc'.
When any of the MODES is current, those CLASSES would be
used by `update-char-fold-table'.

Users could thus:

1. Add their own equivalence classes.

2. Associate any number of such classes with particular modes.

3. Customize the ad hoc classes used by default.

In addition, we could provide the class that abstracts
from diacriticals explicitly, as another, non-customizable
(?) class, so that users could include or exclude it too
wrt specific modes.  (Currently it is implicit in char
folding, i.e., hard-coded.)

Letting users exclude the broad diacritical class and
include their own classes would accomodate wanting some
diacritical foldings but not others.  With symmetric
folding it should offer considerable flexibility.

Utility functions that do some of the work currently done
by `update-char-fold-table' could be created, to be used
by users to easily create their own diacritical classes.
Currently, that part is still hard-coded (only ad hoc
foldings are open to user customization, so far).