Letter-case conversions in network protocols

[Eli Zaretskii <eliz@gnu.org>]

For the immediate trigger, see bug#44604.

The general problem that I'd like us to discuss is how to deal with
letter-case conversions in code that deals with protocols, such as
network-related protocols, that need to recognize certain keywords.

The problem here is that when Emacs starts in certain locales, or
changes to the corresponding language-environments, we modify the case
tables to comply with the rules of those locales.  An example (though
not the only one) is the Turkish locale; see
turkish-case-conversion-enable.  As result of calling that, downcasing
'I' no longer produces 'i', and code which attempts to match keywords
including 'i' case-insensitively fails.

Since we generally use the same text-search and matching APIs both for
implementing the keyword-based protocols and for more general
processing of human-readable code, there's no easy solutions when we
need to ignore language-specific case-conversion rules in some of the
code.  For example, let-binding case-table cannot be done on a too
high level, because it will then affect any text processing below that
level, and a high-level function has no way of knowing what kind of
text processing will be needed by the code it calls, directly or
indirectly.

So what would be the best/easiest solution to this class of problems?

An immediate, but not necessarily easy, candidate is to use

  (with-case-table ascii-case-table

everywhere where we use text-search facilities for keyword processing.
However, this means we will have to go over all the places which do
this, and manually change the code there, and so will developers of
any 3rd-party packages.

Are there better solutions?  Ideas are welcome.