OSX Mail app → Emacs ?

OSX Mail app → Emacs ?

[Jean-Christophe Helary]

If I wanted to transition my mailing activity from OSX's Mail app to Emacs, what would be the best path to do that ?

I see plenty of tutorials on how to *start* using mail in Emacs, but not on how to move one's thousands of mails to a structure that Emacs can access...

Any hints ?

Jean-Christophe 

Re: OSX Mail app → Emacs ?

[Emanuel Berg]

Jean-Christophe Helary wrote:

> If I wanted to transition my mailing activity
> from OSX's Mail app to Emacs

Good call :)

> I see plenty of tutorials on how to *start*
> using mail in Emacs, but not on how to move
> one's thousands of mails to a structure that
> Emacs can access...

I would not worry about the past. No one can do
anything about the past and the future will
come, inevitably. So just start using Gnus is
step one.

Some of the old mails are possibly still on
some server, in what case Gnus can fetch them
all at once with no problems.

As for your really old mails, not on the
server, how are they presently stored? It is
possible there is a parser that will respool
them in a format used by a Gnus backend. If so,
you can create a group, say old-mails, and then
populate that with the old material.

If there isn't a parser (respooler), it'll be
an uphill battle. Perhaps you can write it
yourself. Ask at gmane.emacs.gnus.user where
they should know.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: OSX Mail app → Emacs ?

[Jean-Christophe Helary]

> On Jun 3, 2017, at 12:39, Emanuel Berg <moasen@zoho.com> wrote:
> 
>> I see plenty of tutorials on how to *start*
>> using mail in Emacs, but not on how to move
>> one's thousands of mails to a structure that
>> Emacs can access...
> 
> I would not worry about the past. No one can do
> anything about the past and the future will
> come, inevitably.

In the meanwhile I have ongoing projects that require me to use the old system...

> As for your really old mails, not on the
> server, how are they presently stored?

Not "really old" I never used IMAP. Everything I do is over POP.
The Mail structure is tons of subfolders that contain .mbox files.

> Ask at gmane.emacs.gnus.user where they should know.

Thank you. I'll do.

Jean-Christophe

> 
> -- 
> underground experts united
> http://user.it.uu.se/~embe8573
> 
> 

Re: OSX Mail app → Emacs ?

[Emanuel Berg]

Jean-Christophe Helary wrote:

>> I would not worry about the past. No one can
>> do anything about the past and the future
>> will come, inevitably.
>
> In the meanwhile I have ongoing projects that
> require me to use the old system...

OK, so now you are even more motivated to
finish them off ASAP :)

> Not "really old" I never used IMAP.
> Everything I do is over POP. The Mail
> structure is tons of subfolders that contain
> .mbox files.

mbox should be very possibly to import directly
into Gnus and/or respool into whatever you
switch to! How should be in the Gnus manual.

I would recommend a backend which stores mails
single files (one file per mail), e.g. nnml.
That way you can use the shell tools on the
mails much easier than those
many-mails-in-one-file methods.

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: OSX Mail app → Emacs ?

[Jean-Christophe Helary]

Thank you those were very helpful hints.

Jean-Christophe

> On Jun 3, 2017, at 13:11, Emanuel Berg <moasen@zoho.com> wrote:
> 
> Jean-Christophe Helary wrote:
> 
>>> I would not worry about the past. No one can
>>> do anything about the past and the future
>>> will come, inevitably.
>> 
>> In the meanwhile I have ongoing projects that
>> require me to use the old system...
> 
> OK, so now you are even more motivated to
> finish them off ASAP :)
> 
>> Not "really old" I never used IMAP.
>> Everything I do is over POP. The Mail
>> structure is tons of subfolders that contain
>> .mbox files.
> 
> mbox should be very possibly to import directly
> into Gnus and/or respool into whatever you
> switch to! How should be in the Gnus manual.
> 
> I would recommend a backend which stores mails
> single files (one file per mail), e.g. nnml.
> That way you can use the shell tools on the
> mails much easier than those
> many-mails-in-one-file methods.
> 
> -- 
> underground experts united
> http://user.it.uu.se/~embe8573
> 
> 

Re: OSX Mail app → Emacs ?

[Joost Kremers]

On Sat, Jun 03 2017, Jean-Christophe Helary wrote:
> If I wanted to transition my mailing activity from OSX's Mail 
> app to Emacs, what would be the best path to do that ?
>
> I see plenty of tutorials on how to *start* using mail in Emacs, 
> but not on how to move one's thousands of mails to a structure 
> that Emacs can access...
>
> Any hints ?

That probably depends on what mailer you want to use. Personally, 
I never got the hang of Gnus, but I'm very happy with mu4e. Mu4e 
only reads maildir format, so you'd have to convert your mbox 
files to maildir. A quick Google search reveals there's a script 
mb2md.pl that can take care of that. (I also needed to convert 
from mbox to maildir when I started using mu4e, but I was using 
Mutt at the time, which handles maildir as well as mbox, so I used 
that for the conversion.)

Since mu/mu4e doesn't talk to any remote servers, you'll need to 
set up something to retrieve your email. In the case of IMAP, that 
would be OfflineIMAP or mbsync. If POP is all you need, you'd be 
looking at fetchmail and perhaps procmail (but don't take my word 
for it, since I haven't used POP in well over a decade.)

If you want to use notmuch, the procedure is pretty much the same. 
Gnus, IIUC, can handle more mailbox formats, so you may get away 
without converting anything. There are other mailers for Emacs 
(Mew and Wanderlust come to mind), but I know even less about 
those than Gnus.

HTH

-- 
Joost Kremers
Life has its moments

Re: OSX Mail app → Emacs ?

[Jean-Christophe Helary]

> On 2017/06/03, at 16:07, Joost Kremers <joostkremers@fastmail.fm> wrote:
> 
> 
> On Sat, Jun 03 2017, Jean-Christophe Helary wrote:
>> If I wanted to transition my mailing activity from OSX's Mail app to Emacs, what would be the best path to do that ?
>> I see plenty of tutorials on how to *start* using mail in Emacs, but not on how to move one's thousands of mails to a structure that Emacs can access...
>> 
>> Any hints ?
> 
> That probably depends on what mailer you want to use.

Thank you very much for the hints.

Before moving to Mac, and before UTF-8 was widely used, I was using a Debian machine with Mutt. So I had a setting for French mails, and another setting for Japanese mails, English mails were handled in either setting. I had fetchmail to get my pop mail and procmail for the filtering. It was a bit complex but I managed. That was 20 years ago. I hope things have evolved since then :)

What I need though beyond a retrieval/display/sending system, is a way to access the selected mail from the OS side. I have a workflow that involves getting the selected mail, saving it separately and its attachments too, marking it with a given tag etc.

In the Emacs based system, I'd need to have a variable that points at that file, maybe a variable that points at its attachments, a way to "tag" the mail (I have half a dozen different tags), and I guess that's it for now...

Jean-Christophe 

Re: OSX Mail app → Emacs ?

[Yuri Khan]

On Sat, Jun 3, 2017 at 11:11 AM, Emanuel Berg <moasen@zoho.com> wrote:

> I would recommend a backend which stores mails
> single files (one file per mail), e.g. nnml.
> That way you can use the shell tools on the
> mails much easier than those
> many-mails-in-one-file methods.

This is an overpromise. It *may* make using shell tools easier *if*
your mail is all in English, and even more so if it’s all plain text.

I am using the single-message-per-file format for my mail archive at
work. Here’s a typical line of text out of that archive:

<td colspan=3D"3" color=3D"white"><span style=3D"color: white; =
font-family:georgia,times,times new roman,serif; font-size:24px; font-style=
: italic; display: block; text-align: center; margin: 0;"><br>=D0=9F=D1=80=
=D0=B8=D0=B2=D0=B5=D1=82, =D0=AE=D1=80=D0=B8=D0=B9! </span></td>

As you can see, the only greppable thing in there is the irrelevant
HTML markup, and all the actual text (which is often UTF-8 but still
sometimes KOI8-R or windows-1251) is safely encoded in
quoted-printable.

Re: OSX Mail app → Emacs ?

[Jean-Christophe Helary]

> On 2017/06/03, at 17:38, Yuri Khan <yuri.v.khan@gmail.com> wrote:
> 
> On Sat, Jun 3, 2017 at 11:11 AM, Emanuel Berg <moasen@zoho.com> wrote:
> 
>> I would recommend a backend which stores mails
>> single files (one file per mail), e.g. nnml.
>> That way you can use the shell tools on the
>> mails much easier than those
>> many-mails-in-one-file methods.
> 
> This is an overpromise. It *may* make using shell tools easier *if*
> your mail is all in English, and even more so if it’s all plain text.

Which is absolutely not the case. I have both plain text and html, all in 3 different languages that use different character sets.

Jean-Christophe 

Re: OSX Mail app → Emacs ?

[Joost Kremers]

On Sat, Jun 03 2017, Jean-Christophe Helary wrote:
> Before moving to Mac, and before UTF-8 was widely used, I was 
> using a Debian machine with Mutt. So I had a setting for French 
> mails, and another setting for Japanese mails, English mails 
> were handled in either setting. I had fetchmail to get my pop 
> mail and procmail for the filtering. It was a bit complex but I 
> managed. That was 20 years ago. I hope things have evolved since 
> then :)

It should definitely be possible to handle mail in different 
languages in the same environment nowadays. :-)

> What I need though beyond a retrieval/display/sending system, is 
> a way to access the selected mail from the OS side. I have a 
> workflow that involves getting the selected mail, saving it 
> separately and its attachments too, marking it with a given tag 
> etc.
>
> In the Emacs based system, I'd need to have a variable that 
> points at that file, maybe a variable that points at its 
> attachments, a way to "tag" the mail (I have half a dozen 
> different tags), and I guess that's it for now...

I can't tell whether that work flow can be copied directly into 
Emacs, but it sounds like you might benefit from integrating your 
mail with Org. Mu4e allows you to create Org links to specific 
messages, which you can then use in capture templates or insert 
directly into an Org buffer. I suspect similar functionality 
exists for Gnus.

IIUC both mu4e and notmuch also support `X-Label:' headers, which 
you may be able to use for tagging. I don't know the details 
though, so you'd have to Google that a bit. But if you integrate 
Org into your mail work flow, you may be able to forego saving and 
tagging email, and just do all the tagging in your Org file.

HTH


-- 
Joost Kremers
Life has its moments

Re: OSX Mail app → Emacs ?

[Jean-Christophe Helary]

> On 2017/06/03, at 17:51, Joost Kremers <joostkremers@fastmail.fm> wrote:

> I can't tell whether that work flow can be copied directly into Emacs, but it sounds like you might benefit from integrating your mail with Org. Mu4e allows you to create Org links to specific messages, which you can then use in capture templates or insert directly into an Org buffer. I suspect similar functionality exists for Gnus.
> 
> IIUC both mu4e and notmuch also support `X-Label:' headers, which you may be able to use for tagging. I don't know the details though, so you'd have to Google that a bit. But if you integrate Org into your mail work flow, you may be able to forego saving and tagging email, and just do all the tagging in your Org file.

What I'll do is check with one account and see what that implies. Thank you very much for the help.

Jean-Christophe 

Re: OSX Mail app → Emacs ?

[Emanuel Berg]

Jean-Christophe Helary
<jean.christophe.helary@gmail.com> writes:

>>> I would recommend a backend which stores mails
>>> single files (one file per mail), e.g. nnml.
>>> That way you can use the shell tools on the mails
>>> much easier than those
>>> many-mails-in-one-file methods.
>>>
>>  This is an overpromise. It *may* make using shell
>> tools easier *if* your mail is all in English, and
>> even more so if it’s all plain text.
>
> Which is absolutely not the case. I have both plain
> text and html, all in 3 different languages that use
> different character sets.

Well, maybe you French and Russian guys should stop
living in the past, "man"!

Ha ha - but I always thought HTML mails should be
avoided anyway and at least computer communication on
lists/groups such as this is in the old
Anglo-American...

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: OSX Mail app → Emacs ?

[Eric Abrahamsen]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> On Sat, Jun 3, 2017 at 11:11 AM, Emanuel Berg <moasen@zoho.com> wrote:
>
>> I would recommend a backend which stores mails
>> single files (one file per mail), e.g. nnml.
>> That way you can use the shell tools on the
>> mails much easier than those
>> many-mails-in-one-file methods.
>
> This is an overpromise. It *may* make using shell tools easier *if*
> your mail is all in English, and even more so if it’s all plain text.
>
> I am using the single-message-per-file format for my mail archive at
> work. Here’s a typical line of text out of that archive:
>
> <td colspan=3D"3" color=3D"white"><span style=3D"color: white; =
> font-family:georgia,times,times new roman,serif; font-size:24px; font-style=
> : italic; display: block; text-align: center; margin: 0;"><br>=D0=9F=D1=80=
> =D0=B8=D0=B2=D0=B5=D1=82, =D0=AE=D1=80=D0=B8=D0=B9! </span></td>
>
> As you can see, the only greppable thing in there is the irrelevant
> HTML markup, and all the actual text (which is often UTF-8 but still
> sometimes KOI8-R or windows-1251) is safely encoded in
> quoted-printable.

Assuming that we're talking about searching, you can use an external
program like notmuch or mairix to index your messages, and Gnus will
happy interface with those programs.

Re: OSX Mail app → Emacs ?

[Emanuel Berg]

Eric Abrahamsen wrote:

> Assuming that we're talking about searching,
> you can use an external program like notmuch
> or mairix to index your messages, and Gnus
> will happy interface with those programs.

See? It is so good it's a joke. You don't even
have to drop your native language!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Re: OSX Mail app → Emacs ?

[Robert Thorpe]

Jean-Christophe Helary <jean.christophe.helary@gmail.com> writes:

>> On 2017/06/03, at 17:38, Yuri Khan <yuri.v.khan@gmail.com> wrote:
>> 
>> On Sat, Jun 3, 2017 at 11:11 AM, Emanuel Berg <moasen@zoho.com> wrote:
>> 
>>> I would recommend a backend which stores mails
>>> single files (one file per mail), e.g. nnml.
>>> That way you can use the shell tools on the
>>> mails much easier than those
>>> many-mails-in-one-file methods.
>> 
>> This is an overpromise. It *may* make using shell tools easier *if*
>> your mail is all in English, and even more so if it’s all plain text.
>
> Which is absolutely not the case. I have both plain text and html, all
> in 3 different languages that use different character sets.

Usually, an email contains a header that describes how it's encoded.

An email client that uses mbox will copy the email and header into the
mbox file.  Then when the email is viewed it can look at the header and
select the correct coding system.

That's how Rmail works, and I think it's how GNUS works too.  So, if the
mbox files contain this information they will be usable despite the fact
that several different languages and character sets are in use.

I'm not sure though if older emails follow these conventions.  If they
don't then there may be problems.  In that case some reprocessing may be required.

As other have said, you can search many mbox files using Mairix.  Or,
you can search one at a time using the search feature of the mail
client.  I do the latter, since I usually have a good idea which mbox
file to look in.

BR,
Robert Thorpe

Re: Re: OSX Mail app → Emacs ?

[Yuri Khan]

On Sun, Jun 4, 2017 at 5:34 AM, Robert Thorpe
<rt@robertthorpeconsulting.com> wrote:

>>> On Sat, Jun 3, 2017 at 11:11 AM, Emanuel Berg <moasen@zoho.com> wrote:
>>>
>>>> I would recommend a backend which stores mails
>>>> single files (one file per mail), e.g. nnml.
>>>> That way you can use the shell tools on the
>>>> mails much easier than those
>>>> many-mails-in-one-file methods.
>>>
>>> This is an overpromise. It *may* make using shell tools easier *if*
>>> your mail is all in English, and even more so if it’s all plain text.
>
> Usually, an email contains a header that describes how it's encoded.
>
> An email client that uses mbox will copy the email and header into the
> mbox file.  Then when the email is viewed it can look at the header and
> select the correct coding system.

Yes, and that is what makes mail *work at all*, that any well-written
specialized software can apply the correct decoding algorithms and
access the text within.

However, the original point was that single-message-per-file storage
allows using *general-purpose UNIX tools* on messages. That is not the
case, because they do not have the knowledge of what decoding to
apply.

I am aware that there are specialized tools for mail indexing which
will require setting up and maintenance. Point is, if you have a mail
archive in any format and no tools beyond GNU Coreutils, you are not
guaranteed to be able to do anything useful with it.

Re: OSX Mail app → Emacs ?

[Emanuel Berg]

Yuri Khan <yuri.v.khan@gmail.com> writes:

> However, the original point was that
> single-message-per-file storage allows using
> *general-purpose UNIX tools* on messages. That is
> not the case, because they do not have the knowledge
> of what decoding to apply.

There is no "decoding" needed for plain text mails in
the latin alphabet. That's the code everything is in
to begin with and the more you keep it that way on
a computer the simpler it gets to do whatever.

I can cd to ~/Mail/mail/misc and grep 'decode' and
see that there are five mails containing that word.
It takes an instant and there is no need to install
additional software or index any material.

-- 
underground experts united
http://user.it.uu.se/~embe8573