Re: Creating a coding system

[David Kastrup <dak@gnu.org>]

handa@gnu.org (K. Handa) writes:

> Hi, sorry for the late response.
>
> In article <87ppbeitcs.fsf@fencepost.gnu.org>, David Kastrup <dak@gnu.org> writes:
>> Ok, what am I doing wrong here?  Why does decode-coding-string not do
>> anything here?
>
>> (define-translation-table 'midi-decode-table
>>   (make-translation-table-from-alist
>>    (mapcar
>>     (lambda (p)
>>       (cons (car p) (string-to-vector (cdr p))))
>>     '(([144 0] . "c,,,,")
> [...]
>> (define-coding-system 'midi
>>   "This converts Midi note-on events to note names"
>>   :mnemonic ?M
>>   :coding-type 'charset
>>   :eol-type 'unix
>>   :decode-translation-table 'midi-decode-table
>>   :mime-text-unsuitable t)
>
> Please add
>   :charset-list '(iso-8859-1)
> to the arguments of define-ccoding-system.
>
> The translation table of coding system works AFTER byte
> sequences are decoded into char sequences by the basic
> decoding routine which is specified by :coding-type (and the
> other additional attributes).  As it seems that you are
> expecting that the basic decoding routine decodes the byte
> 144 to the character 144, using the following set is good:
>   :coding-type 'charset
>   :charset-list '(iso-8859-1)

It's one of the things I got to work.

> The other method is to use CCL (i.e :coding-type 'ccl), but,
> if the combination of the charset decoding and translation
> table works, it's faster than running CCL code.

The translation table is not happy about translating things to nothing.
Apparently that makes the calculation of the reverse translation go
wrong.

> If you need arithmetic or conditional operation, you have to use CCL,
> or :post-read-conversion.

> PS.  Should I read the other mails of this thread?  I'm very
> sorry for this lazy attitude, but I don't have a time to
> read all emacs-devel mails.

At the current point it would appear that I manage working with CCL.  It
was a big puzzler that data gets lost unless the CCL program is written
as a _loop_, and it's annoying that the documentation just mentions
(loop statement ...)
as a construct without bothering to point out that the loop will not
actually loop.  "loop" apparently just places a label you can jump to
using (repeat) or its ilk.

So my first approaches lost data when it was arriving fast, and it took
about a day to figure out why that was.

If you have time to spare on that topic, I'd rather you spend it on
putting some more info in the Elisp manual or at least the
define-ccl-program and define-coding-system doc strings.  At the current
point of time, it is quite opaque what the :coding-type specification in
define-coding-system does, it is not clear how CCL code is being run
under which conditions, the attribute :valids (apparently part of ccl
charset-type) is not documented at all.

And the following afterthought in define-ccl-program is quite opaque as
well:

    TRANSLATE :=
            (translate-character REG(table) REG(charset) REG(codepoint))
            | (translate-character SYMBOL REG(charset) REG(codepoint))
            ;; SYMBOL must refer to a table defined by `define-translation-table'.
    LOOKUP :=
            (lookup-character SYMBOL REG(charset) REG(codepoint))
            | (lookup-integer SYMBOL REG(integer))
            ;; SYMBOL refers to a table defined by `define-translation-hash-table'.
    MAP :=
         (iterate-multiple-map REG REG MAP-IDs)
         | (map-multiple REG REG (MAP-SET))
         | (map-single REG REG MAP-ID)
    MAP-IDs := MAP-ID ...
    MAP-SET := MAP-IDs | (MAP-IDs) MAP-SET
    MAP-ID := integer

It's not clear what is input and output for TRANSLATE and LOOKUP, the
operation itself can only be guessed by looking at the _data_ structures
given to `define-translation-hash-table' and `define-translation-table',
and there is absolutely no guessing what the MAP operations are.  And
looking at the code in src/ccl.c is creating rather more than less
confusion as the mapping stuff is really complex and only the mechanisms
(if at all) are documented rather than the purpose.

There is also nothing in the DOC string of `define-coding-system' or the
Elisp manual that would help in guessing what kind of options to choose
for what kind of task.

The purpose/definition of coding-type emacs-mule (particularly post
Emacs-23) is not given, or what the various options with coding-type
iso-2022 are.  It is not clear when one would use coding-type raw-text
and when utf-8 (and how does utf-8 relate to emacs-mule?).

-- 
David Kastrup