Re: Recognize mbox files?

Re: Recognize mbox files?

[Alfred M. Szmidt]

    > What would happen if you have a legal non-mbox file, that
    > contains X-RMAIL-ATTRIBUTES?

   In any case, surely this check would be optional.

I hope not, then it makes rmail not usable for most users.

What is wrong with using real local variables?  Either by adding it to
the first message, or always adding "-*- rmail -*-" to X-RMAIL-...?
That way a user could disable rmail explictly for a file, or enable it
by adding -*- rmail -*- to it.  This would then work exactly as it did
for RMAIL/BABYL.  Without adding specifc rmail hacks to other parts of
emacs.

Re: Recognize mbox files?

[Stefan Monnier]

> What is wrong with using real local variables?  Either by adding it to
> the first message, or always adding "-*- rmail -*-" to X-RMAIL-...?
> That way a user could disable rmail explictly for a file, or enable it
> by adding -*- rmail -*- to it.  This would then work exactly as it did
> for RMAIL/BABYL.  Without adding specifc rmail hacks to other parts of
> emacs.

That's my idea as well: try to use the -*-rmail-*- tag.
It might even be possible to place it on the first ("From ") line.
It probably means to lose/garble that first line's info, so we may want
to first copy that info into an "Envelope-From:" header.

Another issue is to make sure those files get opened into a unibyte
buffer (i.e. use a `binary' coding-system).  So maybe the tag should
really be -*-mode:rmail;coding:binary-*-.


        Stefan

Re: Recognize mbox files?

[Stephen J. Turnbull]

Stefan Monnier writes:
 > Alfred Szmidt writes:

 > > What is wrong with using real local variables?

Nothing.  The main issue is that you may not want to use the
-*- stuff -*- hack since you do not want to touch the From_ line.

Doesn't Emacs recognize local variables in the last page?  Why not
simply have a last "message" from Rmail that simply says

    This mailbox is an explicit Rmail mbox.  That means that Emacs
    will recognize it and automatically start Rmail mode, and you can
    set local variables here.  Keep the local variables at the end of
    this message.

    Local Variables:
    mode: rmail
    coding: binary
    End:

Rmail could suppress display of that message ordinarily.

 > That's my idea as well: try to use the -*-rmail-*- tag.
 > It might even be possible to place it on the first ("From ") line.

Nope, that loses immediately because it breaks the mbox format (FSVO
"mbox format" of course, but it certainly will be different from what
the system MDA generates).  Why bother changing from BABYL if you're
just going to break the format in a different way?  Doesn't emacs
recognize -*- stuff -*- in the second line?  In fact, why not simply
have a real local variables section as above?

 > Another issue is to make sure those files get opened into a unibyte
 > buffer (i.e. use a `binary' coding-system).  So maybe the tag should
 > really be -*-mode:rmail;coding:binary-*-.

A good reason to have a verbose Local Variables section.

Re: Recognize mbox files?

[Miles Bader]

"Stephen J. Turnbull" <turnbull@sk.tsukuba.ac.jp> writes:
>  > > What is wrong with using real local variables?
>
> Nothing.  The main issue is that you may not want to use the
> -*- stuff -*- hack since you do not want to touch the From_ line.

I see no reason _not_ to use the From_ line... it rarely contains
anything particularly valuable (other than the "From_" prefix which
marks it as a separator), and since its syntax isn't well-defined, any
information that _is_ there is only ever used for human perusal -- and
appending a -*- variable doesn't cause problems for humans.

-Miles

-- 
Education, n. That which discloses to the wise and disguises from the foolish
their lack of understanding.

Re: Recognize mbox files?

[Stephen J. Turnbull]

Miles Bader writes:

 > I see no reason _not_ to use the From_ line...

People *do* write tools that parse stuff out of the From_ line.  Even
though that's almost always a bad idea, it usually works OK because
there's usually only one MTA they deal with.  Why screw them up,
purely gratuitously?

Re: Recognize mbox files?

[Richard M Stallman]

    I see no reason _not_ to use the From_ line... it rarely contains
    anything particularly valuable (other than the "From_" prefix which
    marks it as a separator), and since its syntax isn't well-defined, any
    information that _is_ there is only ever used for human perusal -- and
    appending a -*- variable doesn't cause problems for humans.

We would need to verify that this does not cause problems in any other
often-used mail reader program.

Re: Recognize mbox files?

[Richard M Stallman]

    That's my idea as well: try to use the -*-rmail-*- tag.

To make this really work, it is necessary to reinsert that tag
after expunging or sorting.  Also, it may cause trouble for
other mail-reading programs.

Re: Recognize mbox files?

[Alfred M. Szmidt]

   Doesn't Emacs recognize local variables in the last page?  Why not
   simply have a last "message" from Rmail that simply says

       This mailbox is an explicit Rmail mbox.  That means that Emacs
       will recognize it and automatically start Rmail mode, and you can
       set local variables here.  Keep the local variables at the end of
       this message.

       Local Variables:
       mode: rmail
       coding: binary
       End:

   Rmail could suppress display of that message ordinarily.

I like this.

Re: Recognize mbox files?

[Xavier Maillard]

             A better idea would be for Rmail to detect whether it has
             already munged the file (a quick grep for "^X-RMAIL-.*"
             headers should do) and if not make the buffer read-only.

         That sounds like a good idea.  It should be enough to check the
         first message for that header, since a previous visit with Rmail
         will necessarily have displayed the first message and put that
         header into it.

      This mbox detection problem sounds like what we are trying to solve
      with bug#2222 (Labels). Launching rmail would automatically add a
      message on top of the mbox file, adding special rmail headers and
      thus making this mbox file detection really easy (if grepping for
      rmail headers on the first message). We would have solved two
      problems that way. I am strongly in favour of this approach. Still
      we need to find a consensus to the question: to hide or not to
      hide.

   How would thiis work for "C-x C-f ~/RMAIL.foo"?  Is the suggestion to
   do this check on all files one opens?  I think this would be very
   bad...

See below. But I guess emacs is already doing such check each
time it opens a file, no ?

   Using file variables is the right solution.

This is what we have discussed using normal local variables, so
C-x C-f would work or I did not understand all the discussion :/
The alternative would be to use a special suffix for the rmail
files and add it to the auto-mode-alist variable but I find it
less useful since we cannot offer user the possibility not to
open a mbox file in rmail-mode.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Recognize mbox files?

[Xavier Maillard]

       > What would happen if you have a legal non-mbox file, that
       > contains X-RMAIL-ATTRIBUTES?

      In any case, surely this check would be optional.

   I hope not, then it makes rmail not usable for most users.

   What is wrong with using real local variables?  Either by adding it to
   the first message, or always adding "-*- rmail -*-" to X-RMAIL-...?
   That way a user could disable rmail explictly for a file, or enable it
   by adding -*- rmail -*- to it.  This would then work exactly as it did
   for RMAIL/BABYL.  Without adding specifc rmail hacks to other parts of
   emacs.

I like this approach.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Recognize mbox files?

[Alfred M. Szmidt]

       I still do not understand how this will work for find-file on a
       RMAIL file.  Before you could just do C-x C-f on a RMAIL file,
       and it would open in rmail-mode.  Could someone explain how
       this would work by using X-RMAIL-ATTRIBUTES?

   Emacs would check for X-RMAIL-ATTRIBUTES: in the first message,
   find it, and visit the file using Rmail.

So such a check would be needed for _every_ file one does a find-file
on.  What would happen if you have a legal non-mbox file, that
contains X-RMAIL-ATTRIBUTES?  

Re: Recognize mbox files?

[Stephen J. Turnbull]

Alfred M\. Szmidt writes:

 > So such a check would be needed for _every_ file one does a find-file
 > on.

Are you worried this will be a terrible performance problem?  Emacs
does this kind of thing on every file anyway, looking for local
variables.

 > What would happen if you have a legal non-mbox file, that
 > contains X-RMAIL-ATTRIBUTES?  

In any case, surely this check would be optional.

Re: Recognize mbox files?

[Richard M Stallman]

    So such a check would be needed for _every_ file one does a find-file
    on.

Several such checks are already being done.

	 What would happen if you have a legal non-mbox file, that
    contains X-RMAIL-ATTRIBUTES?  

If it starts with a "From ..." line and has an X-RMAIL-ATTRIBUTES: line
sortly after, it would be treated as an Rmail file by default.
That ought to be rather rare, I think.

Re: Recognize mbox files?

[Alfred M. Szmidt]

I still do not understand how this will work for find-file on a RMAIL
file.  Before you could just do C-x C-f on a RMAIL file, and it would
open in rmail-mode.  Could someone explain how this would work by
using X-RMAIL-ATTRIBUTES?

Re: Recognize mbox files?

[Richard M Stallman]

    I still do not understand how this will work for find-file on a RMAIL
    file.  Before you could just do C-x C-f on a RMAIL file, and it would
    open in rmail-mode.  Could someone explain how this would work by
    using X-RMAIL-ATTRIBUTES?

Emacs would check for X-RMAIL-ATTRIBUTES: in the first message,
find it, and visit the file using Rmail.

Re: Recognize mbox files?

[Alfred M. Szmidt]

	  A better idea would be for Rmail to detect whether it has
	  already munged the file (a quick grep for "^X-RMAIL-.*"
	  headers should do) and if not make the buffer read-only.

      That sounds like a good idea.  It should be enough to check the
      first message for that header, since a previous visit with Rmail
      will necessarily have displayed the first message and put that
      header into it.

   This mbox detection problem sounds like what we are trying to solve
   with bug#2222 (Labels). Launching rmail would automatically add a
   message on top of the mbox file, adding special rmail headers and
   thus making this mbox file detection really easy (if grepping for
   rmail headers on the first message). We would have solved two
   problems that way. I am strongly in favour of this approach. Still
   we need to find a consensus to the question: to hide or not to
   hide.

How would thiis work for "C-x C-f ~/RMAIL.foo"?  Is the suggestion to
do this check on all files one opens?  I think this would be very
bad...

Using file variables is the right solution.

Re: Recognize mbox files?

[Xavier Maillard]

       A better idea would be for Rmail to detect whether it has already
       munged the file (a quick grep for "^X-RMAIL-.*" headers should do) and
       if not make the buffer read-only.

   That sounds like a good idea.
   It should be enough to check the first message for that header,
   since a previous visit with Rmail will necessarily have displayed the
   first message and put that header into it.

This mbox detection problem sounds like what we are trying to
solve with bug#2222 (Labels). Launching rmail would automatically
add a message on top of the mbox file, adding special rmail
headers and thus making this mbox file detection really easy (if
grepping for rmail headers on the first message). We would have
solved two problems that way. I am strongly in favour of this
approach. Still we need to find a consensus to the question: to
hide or not to hide.

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Recognize mbox files?

[Stefan Monnier]

> approach. Still we need to find a consensus to the question: to
> hide or not to hide.

Hide, without a hesitation.  Is it really a question?


        Stefan

Re: Recognize mbox files?

[Richard M Stallman]

    This mbox detection problem sounds like what we are trying to
    solve with bug#2222 (Labels). Launching rmail would automatically
    add a message on top of the mbox file, adding special rmail
    headers and thus making this mbox file detection really easy (if
    grepping for rmail headers on the first message).

It is already easy to detect an Rmail file.
Just look for the X-RMAIL-ATTRIBUTES header in the first message.

Re: Recognize mbox files?

[Xavier Maillard]

          When a buffer was converted to BABYL, the file was never saved
          automatically.

      Rmail still does not save automatically.

   That is very good.

I second that (saving a big file is really slow :/)

	Xavier
-- 
http://www.gnu.org
http://www.april.org
http://www.lolica.org

Re: Recognize mbox files?

[Alfred M. Szmidt]

       Infact, a solution to the whole problem is simple.  Upon adding
       headers, add the following header to the first message:

       X-RMAIL-MODE: -*- rmail -*-

   I see two problems with this:

   * The first line is supposed to start with `From ' followed by
     name, date, etc.

The header doesn't need to be on the first line though.

   * If you put that header into just the first message, and you
     delete the first message, the header will no longer be present.

Very true, this did not occur to me.  rmail could check, and add the
header when saving the file.  Then it would work for all cases.

       When a buffer was converted to BABYL, the file was never saved
       automatically.

   Rmail still does not save automatically.

That is very good.

Re: Recognize mbox files?

[Alfred M. Szmidt]

   >> >> Otherwise, Rmail will have to be considered risky for use by mail
   >> >> admins.
   >> > Heh, what it does nowadays is nothing compared with what it did before
   >> > the mbox merge: it would convert the file to Babyl!  How's that for
   >> > ``risky''?
   >> 
   >> But *when* did it do it?  Would it do it, just by C-x C-f <somemboxfile> RET?

   > The conversion happened when you turned on rmail-mode.  If C-x C-f did
   > that automatically, the conversion happened automatically just by
   > typing C-x C-f.

   But could C-x C-f do that "by accident"?  Anyway, even if it could, we
   want to be more careful now.  I think the idea of checking the presence
   of RMAIL tags before allowing to modify the buffer is a good plan.

There seems to be confusion about how rmail work(s/ed).

A RMAIL/BABYL file contains a header that specifies the mode of the
file, so when you open such files (C-x C-f), you will get the file in
rmail-mode.

If you opened a random mbox file, no conversion occurs. 

If you type M-x rmail, rmail will read the mail spool, and save the
content to ~/RMAIL, converting the result to BABYL.

When a buffer was converted to BABYL, the file was never saved
automatically.  This was, I found anyway, useful for looking at mbox
files where you could then just use rmail to navigate th file,
disgarding the conversion later.

rmail would _never_ convert a file to BABYL unless the user explicitly
commanded emacs to do so, by invoking M-x rmail, or C-u M-x rmail.


Infact, a solution to the whole problem is simple.  Upon adding
headers, add the following header to the first message:

X-RMAIL-MODE: -*- rmail -*-

The file will then be handled by Emacs correctly, when the user does a
C-x C-f.

Re: Recognize mbox files?

[Richard M Stallman]

    Infact, a solution to the whole problem is simple.  Upon adding
    headers, add the following header to the first message:

    X-RMAIL-MODE: -*- rmail -*-

I see two problems with this:

* The first line is supposed to start with `From ' followed by name, date, etc.

* If you put that header into just the first message, and you delete
the first message, the header will no longer be present.

    When a buffer was converted to BABYL, the file was never saved
    automatically.

Rmail still does not save automatically.

Re: Recognize mbox files?

[Alfred M. Szmidt]

   > Otherwise, Rmail will have to be considered risky for use by mail
   > admins.

   Heh, what it does nowadays is nothing compared with what it did
   before the mbox merge: it would convert the file to Babyl!  How's
   that for ``risky''?

But only if you explicitly did "C-u M-x rmail FILE", it would do
nothing if you just opened a mbox file.

Re: Recognize mbox files?

[Alfred M. Szmidt]

   OTOH, the format of that From line is somewhat free, so we could
   arrange to place some kind of cookie in it (maybe even
   "-*-rmail-*-" could fit).

This sounds like a good solution as well.

   > What about assuming that rmail files always start witj "RMAIL" or
   > "XMAIL" (and ignore case-sensitivity)? Then RMAIL.foo would be a
   > RMAIL file, and xmail.123 would also be a rmail file.

   Yes, a naming convention seems easier.  a ".mbox" extension seems
   like an obvious choice as well.

I dislike this choice, we use the .mbox extention for spool files that
are nott put into the normal mail spool.  Having them open up using
rmail, and consequently get modified wouldn't be good.

   > That would work for most normal cases.  That way if you open
   > /com/mail/ams or some other mail spool file, RMAIL will not do
   > anything funny with it.

   It's important that if opened on a non-mbox file, Rmail shouldn't
   do anything bad (e.g. it shouldn't modify the file).

I do not think that mbox files should be opened using rmail either,
only files that have been explictly OKed by the user (modeline
{X,R}MAIL naming scheme, ...).

Re: Recognize mbox files?

[Alfred M. Szmidt]

   > Is it reliable enough to recognize mbox files that we could visit
   > them automatically in Rmail mode?

   I don't think so.  Mbox files are lines that begin with "From ",
   separated by arbitrary data.  There's not enough information to
   autodetect them.

What about assuming that rmail files always start witj "RMAIL" or
"XMAIL" (and ignore case-sensitivity)? Then RMAIL.foo would be a RMAIL
file, and xmail.123 would also be a rmail file.

That would work for most normal cases.  That way if you open
/com/mail/ams or some other mail spool file, RMAIL will not do
anything funny with it.

Re: Recognize mbox files?

[Stefan Monnier]

>> Is it reliable enough to recognize mbox files that we could visit
>> them automatically in Rmail mode?

>    I don't think so.  Mbox files are lines that begin with "From ",
>    separated by arbitrary data.  There's not enough information to
>    autodetect them.

Agreed.
OTOH, the format of that From line is somewhat free, so we could
arrange to place some kind of cookie in it (maybe even "-*-rmail-*-"
could fit).

Note that we need to set not only the major-mode, but also the
coding-system (binary).

> What about assuming that rmail files always start witj "RMAIL" or
> "XMAIL" (and ignore case-sensitivity)? Then RMAIL.foo would be a RMAIL
> file, and xmail.123 would also be a rmail file.

Yes, a naming convention seems easier.  a ".mbox" extension seems like
an obvious choice as well.

> That would work for most normal cases.  That way if you open
> /com/mail/ams or some other mail spool file, RMAIL will not do
> anything funny with it.

It's important that if opened on a non-mbox file, Rmail shouldn't do
anything bad (e.g. it shouldn't modify the file).


        Stefan

Re: Recognize mbox files?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Date: Mon, 09 Feb 2009 20:34:27 -0500
> Cc: Chong Yidong <cyd@stupidchicken.com>, rms@gnu.org, emacs-devel@gnu.org
> 
> It's important that if opened on a non-mbox file, Rmail shouldn't do
> anything bad (e.g. it shouldn't modify the file).

This could be impossible: it could be a while before Rmail detects
that a file isn't an mbox file.  By that time, it could have already
modified some of it.

Re: Recognize mbox files?

[Richard M Stallman]

    What about assuming that rmail files always start witj "RMAIL" or
    "XMAIL" (and ignore case-sensitivity)? Then RMAIL.foo would be a RMAIL
    file, and xmail.123 would also be a rmail file.

I normally use *.xmail for these files.
I think others use *.mbox.
But the general idea would work.

Re: Recognize mbox files?

[Chetan Pandya]

-----------------
From: Richard M Stallman <rms@gnu.org>


    What about assuming that rmail files always start witj "RMAIL" or
    "XMAIL" (and ignore case-sensitivity)? Then RMAIL.foo would be a RMAIL
    file, and xmail.123 would also be a rmail file.

I normally use *.xmail for these files.
I think others use *.mbox.
But the general idea would work.
-------------------
If the intent is to work with other mail readers, assuming any such extension will be difficult.  As long as the files are used from within rmail, no such convention is needed, as rmail can expect them to be formatted as expected.

Personally I am against C-x C-f trying to interpret these files as rmail files automatically, but it may still be useful to specify binary coding for these files when editing these files.  The users, of course, have the option of setting auto-mode-alist and auto-coding-alist as they like.

-Chetan

Re: Recognize mbox files?

[Richard M Stallman]

    If the intent is to work with other mail readers, assuming any
    such extension will be difficult.

Why so?

Re: Recognize mbox files?

[Stephen J. Turnbull]

Richard M Stallman writes:

 > I normally use *.xmail for these files.
 > I think others use *.mbox.
 > But the general idea would work.

As long as Rmail munges the file without asking, autodetecting by
filename is a bad idea, because far too many applications use mbox
files as an interchange format, and there's no fixed convention for
naming them, although "mbox" often appears somewhere in the name.  You
could use something unlikely to mean "I'm an mbox file", although that
sorta defeats the purpose.  Eg, something like
"auto-open-me-in-rmail-mode.$SERIAL_NO" would be OK.

A better idea would be for Rmail to detect whether it has already
munged the file (a quick grep for "^X-RMAIL-.*" headers should do) and
if not make the buffer read-only.  When it needs to edit the buffer,
it can ask the user whether that's OK.  That should be the default
(ie, failsafe) behavior.  If you want to have an option to turn off
that protection, it would be OK.

Otherwise, Rmail will have to be considered risky for use by mail
admins.

Re: Recognize mbox files?

[Eli Zaretskii]

> From: "Stephen J. Turnbull" <stephen@xemacs.org>
> Date: Wed, 11 Feb 2009 13:51:51 +0900
> Cc: ams@gnu.org, cyd@stupidchicken.com, emacs-devel@gnu.org
> 
> Richard M Stallman writes:
> 
>  > I normally use *.xmail for these files.
>  > I think others use *.mbox.
>  > But the general idea would work.
> 
> As long as Rmail munges the file without asking

It just adds one or two X-SOMETHING headers, that's all.

(I'm not saying that I'm in favor of autodetection, but "munges" is
greatly exaggerated, IMO.)

> Otherwise, Rmail will have to be considered risky for use by mail
> admins.

Heh, what it does nowadays is nothing compared with what it did before
the mbox merge: it would convert the file to Babyl!  How's that for
``risky''?

Re: Recognize mbox files?

[Stefan Monnier]

>> > I normally use *.xmail for these files.
>> > I think others use *.mbox.
>> > But the general idea would work.
>> 
>> As long as Rmail munges the file without asking

> It just adds one or two X-SOMETHING headers, that's all.

The problem is if this detection is not 100% reliable, this may actually
have a serious consequence, especially if you don't notice it.

>> Otherwise, Rmail will have to be considered risky for use by mail
>> admins.
> Heh, what it does nowadays is nothing compared with what it did before
> the mbox merge: it would convert the file to Babyl!  How's that for
> ``risky''?

But *when* did it do it?  Would it do it, just by C-x C-f <somemboxfile> RET?


        Stefan

Re: Recognize mbox files?

[Eli Zaretskii]

> From: Stefan Monnier <monnier@iro.umontreal.ca>
> Cc: "Stephen J. Turnbull" <stephen@xemacs.org>,  cyd@stupidchicken.com,  ams@gnu.org,  rms@gnu.org,  emacs-devel@gnu.org
> Date: Wed, 11 Feb 2009 20:59:20 -0500
> 
> >> Otherwise, Rmail will have to be considered risky for use by mail
> >> admins.
> > Heh, what it does nowadays is nothing compared with what it did before
> > the mbox merge: it would convert the file to Babyl!  How's that for
> > ``risky''?
> 
> But *when* did it do it?  Would it do it, just by C-x C-f <somemboxfile> RET?

The conversion happened when you turned on rmail-mode.  If C-x C-f did
that automatically, the conversion happened automatically just by
typing C-x C-f.

Re: Recognize mbox files?

[Stefan Monnier]

>> >> Otherwise, Rmail will have to be considered risky for use by mail
>> >> admins.
>> > Heh, what it does nowadays is nothing compared with what it did before
>> > the mbox merge: it would convert the file to Babyl!  How's that for
>> > ``risky''?
>> 
>> But *when* did it do it?  Would it do it, just by C-x C-f <somemboxfile> RET?

> The conversion happened when you turned on rmail-mode.  If C-x C-f did
> that automatically, the conversion happened automatically just by
> typing C-x C-f.

But could C-x C-f do that "by accident"?  Anyway, even if it could, we
want to be more careful now.  I think the idea of checking the presence
of RMAIL tags before allowing to modify the buffer is a good plan.


        Stefan

Re: Recognize mbox files?

[Stephen J. Turnbull]

Eli Zaretskii writes:

 > It just adds one or two X-SOMETHING headers, that's all.

But it changes the mtime, which may incite an automatic process,
depending on the system, and certainly destroys evidence about when
the most recent mail was delivered to it, etc.  Rmail should save the
file when it *needs* to record information about how it changed the
file contents, and not before.

 > > Otherwise, Rmail will have to be considered risky for use by mail
 > > admins.
 > 
 > Heh, what it does nowadays is nothing compared with what it did before
 > the mbox merge: it would convert the file to Babyl!  How's that for
 > ``risky''?

It's totally unacceptable, and one of the reasons why I switched away
from Rmail two decades ago.

Re: Recognize mbox files?

[Richard M Stallman]

    A better idea would be for Rmail to detect whether it has already
    munged the file (a quick grep for "^X-RMAIL-.*" headers should do) and
    if not make the buffer read-only.

That sounds like a good idea.
It should be enough to check the first message for that header,
since a previous visit with Rmail will necessarily have displayed the
first message and put that header into it.

Recognize mbox files?

[Richard M Stallman]

Is it reliable enough to recognize mbox files
that we could visit them automatically in Rmail mode?

Re: Recognize mbox files?

[Chong Yidong]

Richard M Stallman <rms@gnu.org> writes:

> Is it reliable enough to recognize mbox files
> that we could visit them automatically in Rmail mode?

I don't think so.  Mbox files are lines that begin with "From ",
separated by arbitrary data.  There's not enough information to
autodetect them.

Re: Recognize mbox files?

[Harald Hanche-Olsen]

+ Chong Yidong <cyd@stupidchicken.com>:

> Richard M Stallman <rms@gnu.org> writes:
> 
> > Is it reliable enough to recognize mbox files
> > that we could visit them automatically in Rmail mode?
> 
> I don't think so.  Mbox files are lines that begin with "From ",
> separated by arbitrary data.  There's not enough information to
> autodetect them.

But there is more: There should be an RFC 822 header behind the "From
" line. Checking for complete RFC 822 compliance may be too much, but
at least, if you look for a bunch of lines matching '^[a-z][-a-z]*:',
each followed by zero or more lines beginning with white space, and
the whole bunch followed by an empty line, you're getting close. Bonus
points for detecting specific headers such as Subject: From: and
Message-ID:. (Beware that some mail programs like to stuff a bogus
message at the start of the mbox file in which they can store their
own data. This message might miss a Message-ID:, but otherwise it
should look normal.)

- Harald

Re: Recognize mbox files?

[Reiner Steib]

On Mon, Feb 09 2009, Chong Yidong wrote:

> Richard M Stallman <rms@gnu.org> writes:
>
>> Is it reliable enough to recognize mbox files
>> that we could visit them automatically in Rmail mode?

Please don't.

> I don't think so.  Mbox files are lines that begin with "From ",
> separated by arbitrary data.  There's not enough information to
> autodetect them.

More important, rmail-mode seems to modify the file.  Dunno if this is
a bug or a feature of rmail-mode.

$ cat [...]/Incoming* > mbox-test.orig 
$ cp -p mbox-test.orig mbox-test.rmail
$ emacs -Q mbox-test.rmail -f rmail-mode # C-x C-s   C-x C-c
$ diff -U1 mbox-test.orig mbox-test.rmail > mbox-test.diff

,----[ mbox-test.diff ]
| --- mbox-test.orig	2009-02-09 22:13:31.000000000 +0100
| +++ mbox-test.rmail	2009-02-09 22:14:16.000000000 +0100
| @@ -1001,12 +1001,12 @@
| [...]
| -> Viele Grüße
| +> Viele Grüße
|  > 
| [...]
| @@ -4434,2 +4434,3 @@
|  X-Loop: reiner.steib@...
| +X-RMAIL-ATTRIBUTES: --------
`----

(As I wont use Rmail, I don't care much, but ...) it seems that Rmail
recoded iso-8859-1 encoded messages to utf-8 without adjusting the
Content-Type header so the resulting mbox file is no longer correct.
(Modifying the Content-Type would also be a no-no IMHO.)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Recognize mbox files?

[Eli Zaretskii]

> From: Reiner Steib <reinersteib+gmane@imap.cc>
> Date: Mon, 09 Feb 2009 22:26:37 +0100
> Cc: rms@gnu.org, emacs-devel@gnu.org
> 
> More important, rmail-mode seems to modify the file.  Dunno if this is
> a bug or a feature of rmail-mode.

The addition of X-RMAIL-ATTRIBUTES is a feature, but not the other
change.

I think it happened because you first visited the mbox file normally,
and then switched on rmail-mode.  The ``usual'' way to invoke Rmail on
a file is to type

   C-u M-x rmail mbox-test.org RET

from within Emacs.  One difference is that "M-x rmail" visits the file
in a unibyte buffer, unlike "C-x C-f" or "emacs mbox-test.orig".  I'm
quite sure the latter is not really supported by the current Rmail
code.  (The Babyl Rmail worked in that case because Babyl files are
always visited as unibyte, due to auto-coding-regexp-alist.)

> it seems that Rmail recoded iso-8859-1 encoded messages to utf-8

No, I think it's result of turning on Rmail in a multibyte buffer,
because rmail-mode switches the buffer into unibyte mode, and guess
what that does to decoded Latin-1 characters...

Re: Recognize mbox files?

[Reiner Steib]

On Mon, Feb 09 2009, Eli Zaretskii wrote:

> I think it happened because you first visited the mbox file normally,
> and then switched on rmail-mode.  The ``usual'' way to invoke Rmail on
> a file is to type
>
>    C-u M-x rmail mbox-test.org RET
>
> from within Emacs.  One difference is that "M-x rmail" visits the file
> in a unibyte buffer, unlike "C-x C-f" or "emacs mbox-test.orig".  

I repeated the test (with a two days old Emacs from CVS):
`C-u M-x rmail RET mbox-test.rmail2 RET' (then `C-x C-s', `C-x C-c').
The diff between the original file and mbox-test.rmail2 is the same.

After cvs update and make, I only get the additional
X-RMAIL-ATTRIBUTES headers like you described.

Even if rmail-mode only adds these headers, we should *not* enable the
mode by default for mbox files.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Recognize mbox files?

[Eli Zaretskii]

> From: Reiner Steib <reinersteib+gmane@imap.cc>
> Cc: cyd@stupidchicken.com, rms@gnu.org, emacs-devel@gnu.org
> Date: Mon, 09 Feb 2009 23:35:49 +0100
> 
> Even if rmail-mode only adds these headers, we should *not* enable the
> mode by default for mbox files.

That's fine with me, I think mbox files don't have enough
distinguished features to do that anyway.