unencodable-char-position [Re: Several serious problems]

[Kenichi Handa <handa@etl.go.jp>]

In article <200207250312.g6P3C9J06653@aztec.santafe.edu>, Richard Stallman <rms@gnu.org> writes:

>     If the specified coding system is totally inappropriate for
>     the buffer, highlighting them will results in huge amount of
>     overlays and also it takes long time to finish the job.

> That is true.

>       If
>     we limit the number of highlighting, it may give users
>     incorrect information (i.e. non-highlighted characters seems
>     to be encodable).

> It could highlight the first N runs of such characters, and display a
> message saying "Many more unencodable characters found--type WHATEVER
> to view them".  WHATEVER could be the same command with a prefix
> argument.

I implemented that and tried on several files.  But, it
seems that such kind of feature is not that helpful.

In the case that the buffer contains many unencodable chars,
usually the specified coding system is wrong, and we must
use a different coding system.  So, it is not that
interesting to know where are the other unencodable
characters.

In the case that the buffer contains a few unencodable
chars, as it's seldam that more than one of them appear in
one window, highlighting the other unencodable chars is not
that useful.


By the way, I've just noticed that Dave has already
installed the function `unencodable-char-position' in
mule-cmds.el and used it in select-safe-coding-system.

That function resembles to check-coding-system-region on
which we are currently discussing.

But, as the docstring says, it's slow.

So, I commited these changes.

(1) Re-implementation of unencodable-char-position in C
    while adding two optional arguments.
----------------------------------------------------------------------
unencodable-char-position is a built-in function.
(unencodable-char-position START END CODING-SYSTEM &optional COUNT STRING)

Return position of first un-encodable character in a region.
START and END specfiy the region and CODING-SYSTEM specifies the
encoding to check.  Return nil if CODING-SYSTEM does encode the region.

If optional 4th argument COUNT is non-nil, it specifies at most how
many un-encodable characters to search.  In this case, the value is a
list of positions.

If optional 5th argument STRING is non-nil, it is a string to search
for un-encodable characters.  In that case, START and END are indexes
to the string.
----------------------------------------------------------------------


(2) New function `search-unencodable-char' for interactive
    use.  It utilizes `unencodable-char-position'.

----------------------------------------------------------------------
(search-unencodable-char CODING-SYSTEM)

Search forward from point for a character that is not encodable.
It asks which coding system to check.
If such a character is found, set point after that character.
Otherwise, don't move point.

When called from a program, the value is a position of the found character,
or nil if all characters are encodable.
----------------------------------------------------------------------

It may be good to bind C-x RET s to this command.

Could someone make this command more user friendly
(e.g. improving messages)?  It is also easy to modify this
funciton to highlight a few more (or windowful) unencodable
characters if you think that is surely helpful.

(3) Make select-safe-coding-system to show (at most 10)
    unencodable characters for each default coding systems
    tried.

Now, if any unencodable chars are found, one can type C-g to
cancel further saving.  As C-g doesn't hide *Warning*
buffer, one can clik on the displayed unencodable chars to
jump to the corresponding position in a buffer.


---
Ken'ichi HANDA
handa@etl.go.jp