Hi all

Regarding the RMAIL mbox branch I've had a conversation with Richard and Venkatesh recently which I'm appending below. I've also added Venkatesh to the CC.

The MIME code we had did not share anything with Gnus, unfortunately.

Cheers
Alex


On Sun, Mar 9, 2008 at 3:44 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> Sadly I haven't eard from Enrik and Alex for ages. I know there
> is an embryon of a working mbox branch 8but* it is far from
> complete.

> What is not quite clear to me is what do people expect from this
> "rewrite effort" exactly ?

To me the first goal should be just to change the underlying format from
BABYL to standard mbox so that it can interact with other programs.

> I guess the primary goal is to switch to a more "popular" file
> storage but Alex and Enrik have also worked on the MIME part to,
> at last, have a decent MIME support (even basic) for us, Rmail users.

MIME additions would be good.  Ideally, this should share a lot of code
with the Gnus MIME code (some of which is also used by MH-E IIUC).



Forwarded conversation
Subject: about working on rmail-inbox
------------------------

From: Venkatesh Pitta <venkateshpitta@gmail.com>
Date: Wed, Jan 2, 2008 at 7:36 PM
To: alex@emacswiki.org


Dear Alex,

My name is Venkatesh.  I found your contact on throu /whois on
freenode.  I usually remain silent in #emacs for most of the time.  I
saw your message that RMS is looking for people to work on
rmail-inbox.  I was not able to respond to you before you went offline
because I was not at the desk that time.

I like to know what skills would it take to work on rmail-inbox.  I
know knowledge of elisp is instrumental.  I have very little knowledge
of elisp.  Would it be possible to learn while working?  I have been
using GNU Emacs for over a year now on and off.  I use most of its
facilities such as dired and erc with relative ease.  If it is
possible to learn while working, I'd work on rmail-inbox.

About me:  25 old male from India.  I was working as a programmer
until December - 2006.  I'll be joining University of Sydney from
March - 2008.

--
yours,
Venkatesh
----------
From: Alex <kensanata@gmail.com>
Date: Thu, Jan 3, 2008 at 12:20 AM
To: Venkatesh Pitta <venkateshpitta@gmail.com>
Cc: "Richard M. Stallman" <rms@gnu.org>


Hi Venkatesh

I learnt Elisp while trying to write a better SQL mode, so learning on the job seems to be how we all do it. :)

I'm sure it will take you quite some time to read through the rmail code and understanding how it all works. As you know I'm usually available on IRC and willing to help, so you would not on your own.

As for the problem itself: Rmail is the Emacs' simplest mail reader. Unfortunately it doesn't use the mbox format for its mail files. It uses its own very similar but slightly incompatible BABYL format. The Rmail mbox branch is supposed to fix that. At the same time, all developers that were interested in the issue realized that Rmail comes without MIME support out of the box, and all existing MIME support extensions to Rmail depend on the BABYL format. Thus, switching to mbox requires the implementation of decent MIME handling: Messages need to be decoded for searches and for reading. This is tricky because the original Rmail mode worked by narrowing: It just opened the entire mailbox file and limited display to the mail body and selected headers. With MIME encoding this is difficult because if you decode the buffer content you're modifying the original file, which is something you don't want to do. Some of the existing extensions decoded mails and saved those decoded mails instead of the original in the mailbox file.

The rmail mbox branch contains a number of changes to the Rmail files (documented in the ChangeLog file of that branch). The way I implemented it, Rmail uses a temporary buffer for display of the current message. This requires that a lot of keybindings be moved to that new mode used for this buffer, and it requires some fiddling to get summary, mailbox file, and message display to by in sync.

Mail encoding and sending is not part of Rmail, so you don't need to worry about that. This is handled by Mail or Message Mode.

As far as I'm concerned, all MIME parts that are not plain text can just be saved to disk. Emacs doesn't really have to know how to handle them. As long as we have a mechanism by which to extend it later, then we can postpone image handling and so on.

I guess the first thing you should do is get yourself a mail account for testing, and use Rmail as-is with it. Get used to reading some mail using Rmail. Switching to the next and the previous mail, assign keywords (tags), switch to views showing only a particular tag, moving to the next and previous mail within this list, etc. Understand how the code works. Then check out the rmail mbox branch, look at the ChangeLog file and skim it to see whether this starts to make sense in light of what I wrote above. And then debug it, or finish it, or do whatever is necessary to get it to work. Or junk it and start with a totally different implementation. RMS mentioned that he thinks using an external tool like metamail is the way to go and he mailed me one of the existing extensions called rmailmime.el by Masanobu UMEDA. At the time I thought that this solution was no good, but I cannot remember the exact reasons. Perhaps I was already using a mix of Windows, OSX, and GNU/Linux, and not all systems had the required metamail utiltity available. Or I felt that the overhead for searches was too big. Anyway, once you decide to start working on it, the decision is up to you.

If you're still interested, let me (and RMS) know!

Thanks for your interest in contributing to Free Software. :)

Cheers
Alex
----------
From: Venkatesh Pitta <venkateshpitta@gmail.com>
Date: Fri, Jan 4, 2008 at 4:03 AM
To: Alex <kensanata@gmail.com>
Cc: "Richard M. Stallman" <rms@gnu.org>


Dear Alex,

I'm following the first steps that you have pointed out.  I'm using
rmail to read mails on the my account. Hopefully, soon, I'd be staring
at the code and begin making sense of it.

I'll work on rmail.
--
yours,
Venkatesh
----------
From: Richard Stallman <rms@gnu.org>
Date: Fri, Jan 4, 2008 at 6:29 AM
To: Alex <kensanata@gmail.com>
Cc: venkateshpitta@gmail.com


Venkatesh, are you interested in finishing up the rmail-mbox code?
I would appreciate that a lot.
I don't think the version I worked on does that.
Is this something that you subsequently changed?
Or are you talking about a previous version?

Using a temporary buffer has the big disadvantage that you cannot edit
the message "in place" any more.  Does it have an advantage to
outweigh that disadvantage?
That seems like a nonsequitur to me.  I do not see why switching formats
_requires_ an _increase_ in the level of MIME support.

It would be nice to do more to support MIME, but that's a separate issue.
My version of the mbox-based Rmail works on narrowing too.
Why don't we want to do this?  It works in Babyl-based Rmail, so why
shouldn't rmail-mbox do it this way too.
We can still do it that way.  Why not?
----------
From: Alex <kensanata@gmail.com>
Date: Fri, Jan 4, 2008 at 8:49 AM
To: rms@gnu.org
Cc: venkateshpitta@gmail.com


You asked whether using a separate buffer to display a message was
code I had written myself or not, and we talked about the use of
narrowing to display a message.

I just checked out the rmail-mbox-branch and I'm referring to the
following code in rmail rmail-show-message:

     (let ((beg (rmail-desc-get-start n))
           (end (rmail-desc-get-end n)))
       (rmail-header-show-headers)
       (widen)
       (narrow-to-region beg end)
       (goto-char (point-min))

Displaying messages therefore still works by narrowing, allows editing
in place, and so on.

The code also has the following:

       (if (eq rmail-enable-mime t)
           (funcall rmail-show-mime-function)
         (setq rmail-view-buffer rmail-buffer))

If I remember correctly what you had to do is set rmail-enable-mime to
t and rmail-show-mime-function to rmail-mime and load rmailmm.el. Then
the message you're viewing gets copied to a separate buffer and a MIME
handler is called to display it. At the time a separate modifiable
buffer seemed like the only sane way of MIME handling.

It seems possible to finish the Rmail code without having to work on
the MIME stuff with the current setup. That's good news.

You said that switching formats does not require an increase in MIME
support. That is technically true. At the time I wanted to use RMAIL
for my regular mail, however. One did not require the other, but both
were required by the kind of mail I was getting. (And not being able
to read my mail with Rmail has caused me to switch to other mail
programs, and that in turn has drained my dedication to work on
Rmail.)

You also asked why modifying the original file was something we don't
want to do. Some modification is ok: If the message is an encoded
plain text message, recoding it and changing the MIME headers to
reflect that is no problem because the mail file is still usable by
other mail readers. That's the only limitation on modifications as far
as I know. One MIME handling extension would take HTML attachments,
transform them into plain text, and replace the attachment with the
plain text. This kind of modification would not be ok, I think.
----------
From: Venkatesh Pitta <venkateshpitta@gmail.com>
Date: Fri, Jan 4, 2008 at 12:31 PM
To: rms@gnu.org
Cc: Alex <kensanata@gmail.com>


Yes, Richard.  I am interested in finishing the rmail-mbox work.



--
yours,
Venkatesh