Re: Need some help with Rmail/mbox

[Eli Zaretskii <eliz@gnu.org>]

> Date: Fri, 19 Sep 2008 01:35:12 -0400
> From: Paul Michael Reilly <pmr@pajato.com>
> Cc: emacs-devel@gnu.org
> 
> I first copy the relevant headers to the view buffer by collecting
> them from the PMAIL buffer into a string and insert the string into
> the view buffer.

I hope you use insert-buffer-substring instead of actually making a
string and inserting it.  Consing large strings is not a good idea, if
all you need is copy text from one buffer to another.

> Then I basically copy the message body into a string and insert it
> into the view buffer.

Same comment as above.

> But when I started to work on the decoding it seemed that decoding
> the string before inserting it seemed like a good idea.

Actually, it isn't: in Emacs, whenever you can work on a buffer
instead of a string, you should generally prefer a buffer.
Specifically, decoding of strings uses scratch buffers behind your
back, and you don't gain anything in efficiency.

So just copy the text to the view buffer, then decode it in-place.

> (Pardon my Elisp rustiness ... is it better to use buffer to
> buffer copying than insert string?)

Yes.

> If you had said content type and content encoding I would have said
> "yup" and that is what led to my request for help.  Except for the
> case of quoted-printable and base64 I'm not sure how to parse those
> two headers (Content-Type and Content-Transfer-Encoding) into a coding
> system so that I can then do the decoding.

You should parse them separately, and use them separately, just like
Rmail/Babyl does: first decode qp or b64 into 8-bit encoded bytes,
then decode the rest using the charset gleaned from the Content-Type
header.

> I'm assuming the coding system guesswork becomes relevant for
> combinations of the two headers that Rmail does not grok.

This should not happen, in general; but for more robust code, you
could try `undecided' if all else fails; this is what Rmail/Babyl
does.  See rmail-decode-region.

> And I now see that there is a strong relationship between charset
> and coding system.

Yes; they are mostly the same.  Emacs defines an alias coding-system
for every MIME charset, IIRC.

> OK, this is helpful.  I assume that for all other type/subtype cases
> we punt for now and use guessing or just raw text?

It's not raw text, it should be plain ASCII (before you qp- or
b64-decode them; I suggest not to decode their original qp or b64
encoding until you support those additional types).  Rmail/Babyl uses
`undecided' for those, and so can you.

> But certainly
> there are some that we want to process/decode in some fashion,
> e.g. text/html or text/xml.

Eventually, yes.

> Is there another Emacs package/library
> that you are aware of that provides a good model for where we want to
> take Rmail so that it handles more type/subtype cases seamlessly in
> the view buffer? Even perhaps audio and video (not pure MIME,
> i.e. multipart ... yet).

Gnus, of course.  But again, I suggest not to bother about these
extensions for now: just make Rmail/mbox be no worse than Rmail/Babyl,
so that people could start using it.  Extensions can come later.

> >         Wash body for presentation, eg:
> >             Highlight and activate url-like substrings
> >             Highlight quoted material
> 
> I don't believe Rmail does either of these operations now.

Right, it doesn't.  We have ffap and similar features to do that
without highlighting, although highlighting would be nice (again, as
an extension of what Rmail does now).