bug#22043: 25.0.50; search-forward and char folding

[Eli Zaretskii <eliz@gnu.org>]

> Date: Sun, 29 Nov 2015 12:41:16 -0800 (PST)
> From: Drew Adams <drew.adams@oracle.com>
> Cc: m.kupfer@acm.org, 22043@debbugs.gnu.org
> 
> > > IOW, isn't this default behavior true for all incremental
> > > search commands except regexp search, and only for those
> > > commands (no non-incremental search commands)?
> > 
> > No.  Nonincremental vs incremental is not the issue.  The issue is
> > whether the search function that the command employs uses regexps or
> > not.  It is a limitation of how these features are implemented that
> > they absolutely require regexp search.
> 
> OK, but from a user point of view, is this not the case:
> 
> 1. S?he invokes search using `C-M-s' or `C-s', which are
> advertised as regexp and plain (non regexp) search.  IOW,
> regardless of what might go on under the covers (and a
> lot already does, for lax whitespace searching), s?he
> thinks of `C-s' as performing a non-regexp search.

Perhaps so, but (a) I see that you've dropped the incremental vs
nonincremental distinction, which agrees with what I say above; and
(b) we cannot really say in the manual something like "commands
invoked by `C-s' do character folding", because the reader might not
yet know/remember enough for such a distinction to be useful for her.

> 2. There is no character folding with the "regexp" commands
> (`C-M-s'), because char folding substitutes its own regexp
> for the user input, and char folding does not currently
> parse regexp-pattern user input.

That's implementation.  From the user POV, typing "C-M-s M-s '"
magically does support character folding, but it also switches the
search to a non-regexp one!  So is it a regexp search or isn't it?

> Perhaps, to be more precise, the difference is search that
> does or does not accept general regexp patterns as _input_.
> Those that do have "regexp" (or "-re-"?) in their name;
> those that do not do not have it.  The former do not
> support char folding; the latter do.  Is that correct (and
> complete)?

I think neither, because of the subtle behind-the-scenes effect of the
"M-s C" toggles (where C is a character like ' or _ or w).

> I guess I was mistaken in thinking that non-incremental
> search commands, such as `nonincremental-search-forward',
> do not support char folding (regardless of whether they
> include "-re" in their name).  Which ones support it, and
> under what circumstances?

Each command should tell in its doc string whether it does or doesn't,
and the manual should also document that where it describes the
command itself.  AFAICS, this is currently (as of today ;-) so.  I
don't think we can do any better.  The real criterion is "where it's
easy to provide given the limitations of the implementation we have",
but that's not useful for user documentation.

Anyway, if I return to the original issue, the section with the
offending "Search commands in Emacs by default perform character
folding" sentence has its main focus on explaining what is character
folding and how to enable/disable it; it does not focus on the
specific commands.  So it uses some vague definitions of the default
behavior, which is later described more accurately for each particular
command.  This is standard practice in user-level documentation, when
describing complex issues: you first provide an overview that might
not be 100% accurate, but should give the reader a clear and simple
enough idea of the subject, leaving the more accurate details for
later in-depth coverage.  I don't see how we can do significantly
better; all of the proposals till now make the description much more
complicated and thus confusing, especially upon first reading.

If people think that saying something more vague like

  Some search commands by default perform character folding (whether
  it does or doesn't is documented for each specific command)

will do a better job, maybe we could use that.  To me, this sounds
worse, because it immediately raises the question: which commands do
and which don't.  That question will interfere with the reader's
attention to the issue at hand, which isn't the particular commands,
but the folding in general and how to toggle/disable it.