tutorial for rmail and IMAP

tutorial for rmail and IMAP

[Sam Halliday]

Hi all,

I'm seriously considering using emacs as my mail client and I should like to give rmail a try.

However, despite several google searches and having a look at the rmail website, I cannot find any tutorials for an rmail beginner.

Does anybody know of an rmail tuturial that will:

1. get me set up with IMAP (specifically gmail)
2. auto-complete recipients using my google address book
3. typical use-cases (receiving mail, sorting mail, responding, etc)

If so, I'd be really very pleased.

Best regards,
Sam

Re: tutorial for rmail and IMAP

[David Hume]

Sam Halliday <sam.halliday@gmail.com> writes:

> Hi all,
>
> I'm seriously considering using emacs as my mail client and I should like to give rmail a try.
>
> However, despite several google searches and having a look at the rmail
> website, I cannot find any tutorials for an rmail beginner.
>
> Does anybody know of an rmail tuturial that will:
>
> 1. get me set up with IMAP (specifically gmail)
> 2. auto-complete recipients using my google address book
> 3. typical use-cases (receiving mail, sorting mail, responding, etc)
>
> If so, I'd be really very pleased.
>
> Best regards,
> Sam

I have just followed these instructions:

http://www.emacswiki.org/emacs/GnusGmail#toc4

And got gmail and yahoo working with imap, at least for reading. I had
to set up application passwords for both as I have second factor
authentication.

Re: tutorial for rmail and IMAP

[Eli Zaretskii]

> Date: Wed, 23 Apr 2014 12:11:12 -0700 (PDT)
> From: Sam Halliday <sam.halliday@gmail.com>
> 
> I'm seriously considering using emacs as my mail client and I should like to give rmail a try.
> 
> However, despite several google searches and having a look at the rmail website, I cannot find any tutorials for an rmail beginner.
> 
> Does anybody know of an rmail tuturial that will:
> 
> 1. get me set up with IMAP (specifically gmail)
> 2. auto-complete recipients using my google address book
> 3. typical use-cases (receiving mail, sorting mail, responding, etc)

Rmail is documented in the Emacs User manual.  For questions not
covered there, ask here, but be more specific than the above.

Re: tutorial for rmail and IMAP

[Sam Halliday]

On Wednesday, 23 April 2014 20:23:04 UTC+1, David Hume  wrote:
> 
> I have just followed these instructions:
> 
> http://www.emacswiki.org/emacs/GnusGmail#toc4

Thanks David. I have already seen that page... but I am specifically interested in using rmail, not gnus. I have tried gnus and I don't like it.

Re: tutorial for rmail and IMAP

[Robert Thorpe]

Sam Halliday <sam.halliday@gmail.com> writes:

> I'm seriously considering using emacs as my mail client and I should like to give rmail a try.
>
> However, despite several google searches and having a look at the rmail website, I cannot find any tutorials for an rmail beginner.

As Eli mentions the Emacs manual describes rmail in detail.

> Does anybody know of an rmail tuturial that will:
>
> 1. get me set up with IMAP (specifically gmail)

It's quite simple.  First you have to install "Movemail" from GNU
Mailutils.  You can use the package manager to do that.  The movemail
that comes with Emacs doesn't handle IMAP.

Then put in.
(setq rmail-primary-inbox-list ("imap://username:password@yourimapserver.com"))

You also have to setup outgoing mail.  I use internal Emacs smtp:
(setq smtpmail-smtp-server "smtp.yoursmtpserver.com")

My username and password are so long that this spans several lines, so
I've used concat to split it across lines.

There seems to be a little bug if your username has a "@" in it.  I can
tell you how to fix that if it happens to you.
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16946

> 2. auto-complete recipients using my google address book

I don't know how to do that, it might not be possible.

> 3. typical use-cases (receiving mail, sorting mail, responding, etc)

They're all described well in the manual.


A couple of other things worth mentioning....  

* Gmail.
People have told me that gmail has some wierd limitation on the number
of IMAP logins you can do with them per day.  I don't use it so I don't
know.

* IMAP.
Rmail is a mode for viewing and editing mbox files.  Movemail does the
mail receiving, it always copies everything to a local mbox file
("RMAIL") and rmail views that.  The IMAP system where all the mail sits
on the IMAP server doesn't work well with this.

You can still use rmail from several PCs though.  What I do is I have
one PC where all mail is kept.  I have a set of rules that move
everything out of the local inbox to subject specific mbox
files.  On other machines I set: (setq rmail-preserve-inbox t).  That
means that movemail doesn't delete email in the IMAP inbox, so it can be
picked up later by my main PC which stores mail.  On those other
machines I delete everything in the local inbox after reading.

* Rules.
I have lots of simple rules that copy things into other mbox files:
(setq rmail-delete-after-output t
      rmail-output-file-alist
      '(("contains this regexp" . "~/Mail/Goes_In_This_File.mbox")
        ("another regexp" . "~/Mail/Another_File.mbox")))

* Sorting.
Rmail sorts actually resort the whole mbox file.  If you save it then
it's saved in the new order, so it's best to sort look for what you need
then revert.  You can also open mbox files in fundamental mode and use
occur on them, which is sometimes useful.

BR,
Robert Thorpe

Re: tutorial for rmail and IMAP

[Sam Halliday]

On 23 Apr 2014 23:45, "Robert Thorpe" <rt@robertthorpeconsulting.com> wrote:
> There seems to be a little bug if your username has a "@" in it.  I can
> tell you how to fix that if it happens to you.
> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=16946

Thanks Robert!

Yes, Gmail usernames have the @ symbol, so I would greatly appreciate your
workaround.

I'd really like to avoid putting passwords in plain text files, especially
emacs configuration. Can I set up rmail to prompt me for the password when
I first open the mode?

Re: tutorial for rmail and IMAP

[Robert Thorpe]

Sam Halliday <sam.halliday@gmail.com> writes:
> Yes, Gmail usernames have the @ symbol, so I would greatly appreciate your
> workaround.

First try things as they are and see if it works.  There maybe something
wrong with my setup that causes the workaround to be needed.

If you need it this is what I did....  First specify your username using
%40 instead of @.
(setq rmail-primary-inbox-list ("imap://username%40yourserver.com@yourimapserver.com"))

Secondly, fish the function rmail-insert-inbox-text out of rmail.el and
put it in a separate .el file.  Then after the bit:

      ;; At this point, TOFILE contains the name to read:
      ;; Either the alternate name (if we renamed)
      ;; or the actual inbox (if not renaming).

I added:
      (with-temp-buffer
	(insert tofile)
	(goto-char (point-min))
	(replace-string "%40" "@")
	(setq tofile (buffer-string)))

I require rmail in my .emacs file and load the .el file containing the
modified version of rmail-insert-inbox-text.  You could modify Emacs
directly, I don't like doing that though.

> I'd really like to avoid putting passwords in plain text files, especially
> emacs configuration. Can I set up rmail to prompt me for the password when
> I first open the mode?

If you omit ":password" in rmail-primary-inbox-list then rmail will ask
you for it every time you start it.  It stores the password for the
session AFAIK.

BR,
Robert Thorpe

Re: tutorial for rmail and IMAP

[Eli Zaretskii]

> From: Robert Thorpe <rt@robertthorpeconsulting.com>
> Date: Wed, 23 Apr 2014 23:45:02 +0100
> Cc: help-gnu-emacs@gnu.org
> 
> > 2. auto-complete recipients using my google address book
> 
> I don't know how to do that, it might not be possible.

One way is to somehow convert the address book into a ~/.mailrc file
of aliases.

Re: tutorial for rmail and IMAP

[Hans BKK]

BBDB should work with rmail, and there are several import/export/sync tools for bbdb <--> gMail contacts.

Re: tutorial for rmail and IMAP

[James Freer]

On 4/23/14, Robert Thorpe <rt@robertthorpeconsulting.com> wrote:
> Sam Halliday <sam.halliday@gmail.com> writes:
>
> * Gmail.
> People have told me that gmail has some wierd limitation on the number
> of IMAP logins you can do with them per day.  I don't use it so I don't
> know.
I have used gmail since 2007 and have had no problems logging in. I
use google docs as well and Chromium - so I login many times per day.

I couldn't get rmail or gnus to work for me and use Alpine.
james

Re: tutorial for rmail and IMAP

[Robert Thorpe]

James Freer <jessejazza3.uk@gmail.com> writes:

> On 4/23/14, Robert Thorpe <rt@robertthorpeconsulting.com> wrote:
>> Sam Halliday <sam.halliday@gmail.com> writes:
>>
>> * Gmail.
>> People have told me that gmail has some wierd limitation on the number
>> of IMAP logins you can do with them per day.  I don't use it so I don't
>> know.
> I have used gmail since 2007 and have had no problems logging in. I
> use google docs as well and Chromium - so I login many times per day.
>
> I couldn't get rmail or gnus to work for me and use Alpine.

There are limitations though.  Google describe them here:
https://support.google.com/a/answer/1071518?hl=en
https://support.google.com/a/answer/2751577

You may hit them if you configure Emacs tools in certain ways.

BR,
Robert Thorpe

Re: tutorial for rmail and IMAP

[James Freer]

On 4/26/14, Robert Thorpe <rt@robertthorpeconsulting.com> wrote:
> James Freer <jessejazza3.uk@gmail.com> writes:
>
>> On 4/23/14, Robert Thorpe <rt@robertthorpeconsulting.com> wrote:
>>> Sam Halliday <sam.halliday@gmail.com> writes:
>>>
>>> * Gmail.
>>> People have told me that gmail has some wierd limitation on the number
>>> of IMAP logins you can do with them per day.  I don't use it so I don't
>>> know.
>> I have used gmail since 2007 and have had no problems logging in. I
>> use google docs as well and Chromium - so I login many times per day.
>>
>> I couldn't get rmail or gnus to work for me and use Alpine.
>
> There are limitations though.  Google describe them here:
> https://support.google.com/a/answer/1071518?hl=en
> https://support.google.com/a/answer/2751577
>
> You may hit them if you configure Emacs tools in certain ways.
>
> BR,
> Robert Thorpe
>
Thanks you are right... I've not noticed any problems. I must say
though that I use gmail's web UI and Alpine remotely. So with the size
of my mailbox I should have had problems.... but none. Using xubuntu
and installing each 6 month release i have found using alpine remotely
saves downloading all mail headers for an install and yet each time I
use it that's exactly what it does.

james

Re: tutorial for rmail and IMAP

[Sam Halliday]

On Saturday, 26 April 2014 10:40:22 UTC+1, Robert Thorpe  wrote:
> There are limitations though.  Google describe them here:
> https://support.google.com/a/answer/1071518?hl=en
> https://support.google.com/a/answer/2751577
> 
> You may hit them if you configure Emacs tools in certain ways.


I don't think this will be a problem for me, as I don't get that much mail (although the archive is large and will take some time on first use).

I gave up trying to use movemail for IMAP access as it just doesn't seem to work at all for me, even on the command line. Instead, I'm investigating using offlineimap which has created me a bunch of Maildir folders under ~/Mail. 

This setup means that I can send mail, and I can use M-x google-contacts to look up my contact list (I've not yet integrated it with M-x m).

However, I still can't get rmail to see the Maildir folders.

I keep most of my config here https://github.com/fommil/unix I'd really appreciate it if an experienced Maildir rmail user could point out what I am doing wrong.

Incidentally, from reading the documentation it would appear that rmail doesn't really support mail folders or browsing of email. Is that correct? If so, I don't think it's going to work for me.

I'm really looking for a simple mail client that allows me to search mail, browse folders, and "tag" or "untag" messages (which is how gmail moves mails between folders). Much as how this person does things: http://chrisdone.com/posts/emacs-mail

I'm going to experiment with the notmuch client and I might give gnus another try.


Thanks all so far, I've not given up on rmail but I'd really like to know if it is worth pursuing or if notmuch is a better fit for my needs.

Re: tutorial for rmail and IMAP

[Robert Thorpe]

Sam Halliday <sam.halliday@gmail.com> writes:
> I gave up trying to use movemail for IMAP access as it just doesn't
> seem to work at all for me, even on the command line.

Sam and discussed this off-list.  I can't access my gmail account using
movemail either.  I enabled IMAP in google settings first.  After that
the command below should work, as far as I can see.

movemail -p 'imap://mygmail.address%40gmail.com@imap.gmail.com'
         Test.mbox password

The connection times out.  I tried adding "--tls" and I tried converting
the dots in my email address to %2s.  I also tried putting the port at
the end of the imap URL, ":993", if I do that I get "Input/output error"
instead of a timeout.  Sam gets a different error "Invalid reply from
the remote host".

Does anyone know what the problem is here?

BR,
Robert Thorpe

Re: tutorial for rmail and IMAP

[Robert Thorpe]

Sam Halliday <sam.halliday@gmail.com> writes:

...
> I don't think this will be a problem for me, as I don't get that much mail (although the archive is large and will take some time on first use).

If you have other devices that access a lot you may hit them.  If you
have to download your entire inbox a few times to get things working you
may hit them.

> I gave up trying to use movemail for IMAP access as it just doesn't seem to work at all for me, even on the command line. Instead, I'm investigating using offlineimap which has created me a bunch of Maildir folders under ~/Mail. 
>
> This setup means that I can send mail, and I can use M-x google-contacts to look up my contact list (I've not yet integrated it with M-x m).
>
> However, I still can't get rmail to see the Maildir folders.

Rmail isn't compatible with the maildir format, as far as I know.


> Incidentally, from reading the documentation it would appear that rmail doesn't really support mail folders or browsing of email. Is that correct? If so, I don't think it's going to work for me.

Not really, you can file email and browse it.

In Thunderbird or Outlook you have "folders".  Whether these are
implemented using actually directories is a different thing.  Mostly
they're not.

In Emacs it's common to talk about things more directly.  MH-E and
Maildir are systems for storing email.  A directory structure is used
and each email is put in a separate file.  Several types of mailer can
use these formats, but not Rmail.

In Rmail you have mboxes rather than directories.  A user can file
emails by subject into separate mboxes.  That's what the
"rmail-output-file-alist" stuff I mentioned before does.  You search and
browse other mboxes in the same way as the inbox.  To the user this
isn't much different to the mailers that use actual directories.  There
are a few pros and cons which you can read about online.

> I'm really looking for a simple mail client that allows me to search mail, browse folders, and "tag" or "untag" messages (which is how gmail moves mails between folders). Much as how this person does things: http://chrisdone.com/posts/emacs-mail

You can label messages in rmail too.  There are inbuilt labels and you
can create your own.  Seperate mbox files and labels are different
things though.

> I'm going to experiment with the notmuch client and I might give gnus another try.
>
>
> Thanks all so far, I've not given up on rmail but I'd really like to know if it is worth pursuing or if notmuch is a better fit for my needs.

Since movemail doesn't seem to work with gmail pursuing other options is
a good idea.

Re: tutorial for rmail and IMAP

[Sam Halliday]

On 26 April 2014 16:11, Robert Thorpe wrote:
> Sam and discussed this off-list.

Oh, apologies... I didn't realise I'd taken it off list.

I'm actually really enjoying notmuch, so I'll give it a longer trial
period before looking at rmail again.

Re: tutorial for rmail and IMAP

[Robert Thorpe]

Earlier this week Sam Halliday was trying to access his gmail account
using movemail.  He couldn't do it and I couldn't access mine when I
tried it.  I got some advice off-list from Herbert Skuhra and tried it
out today.  It worked.  Oddly enough a command I tried out earlier that
failed started to work.

I can download my gmail email with:
movemail -p 'imaps://MyEmail%40gmail.com@imap.gmail.com' gmail.mbox
mypassword

As Herbert Skuhra told me "imaps" must be used instead of "imap" to
activate SSL.  %40 replaces the @ in the username.  The password can be
on the command line, if it isn't then movemail asks for it.  The "-p"
preserves the source inbox, making it a copy rather than a move - it's
not needed if you want a move.

I think the problem here was that gmail takes some time to enable IMAP.
Perhaps when you click on "Enable IMAP" in the settings it only takes
effect a few hours later or the next day.  That's the best explanation I
can come up with anyway.

BR,
Robert Thorpe

Re: tutorial for rmail and IMAP

[Sam Halliday]

[-- Attachment #1: Type: text/plain, Size: 491 bytes --]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:
> Earlier this week Sam Halliday was trying to access his gmail account
> using movemail.
> ...
>
> As Herbert Skuhra told me "imaps" must be used instead of "imap" to
> activate SSL.  %40 replaces the @ in the username.

Thanks Robert!

I will try rmail at some point out of curiosity, but I am really
enjoying `notmuch` which is a very minimal emailing system focused
on search and tag support.

-- 
Best regards,
Sam

[-- Attachment #2: Type: application/pgp-signature, Size: 180 bytes --]

Re: tutorial for rmail and IMAP

[James Freer]

On Wed, 23 Apr 2014, David Hume wrote:

> Sam Halliday <sam.halliday@gmail.com> writes:
>
>> Hi all,
>>
>> I'm seriously considering using emacs as my mail client and I should like to give rmail a try.
>>
>> However, despite several google searches and having a look at the rmail
>> website, I cannot find any tutorials for an rmail beginner.
>>
>> Does anybody know of an rmail tuturial that will:
>>
>> 1. get me set up with IMAP (specifically gmail)
>> 2. auto-complete recipients using my google address book
>> 3. typical use-cases (receiving mail, sorting mail, responding, etc)
>>
>> If so, I'd be really very pleased.
>>
>> Best regards,
>> Sam
>
> I have just followed these instructions:
>
> http://www.emacswiki.org/emacs/GnusGmail#toc4


You managed to get yahoo working with imap. I've been trying to do that without 
success.

> And got gmail and yahoo working with imap, at least for reading. I had
> to set up application passwords for both as I have second factor
> authentication.

What imap and smtp codes did you use? From what I can gather yahoo has imap 
codes for reading mail on mobile phone apps but not PC email.

james

Re: tutorial for rmail and IMAP

[David Hume]

> You managed to get yahoo working with imap. I've been trying to do that
> without success.
>
>> And got gmail and yahoo working with imap, at least for reading. I had
>> to set up application passwords for both as I have second factor
>> authentication.
>
> What imap and smtp codes did you use? From what I can gather yahoo has imap
> codes for reading mail on mobile phone apps but not PC email.
>

Notice that this is gnus, not rmail.

I am not sure what you mean by codes.

I also have a .auth file. I have this in .gnus:

(setq user-full-name "????????")
(setq user-mail-address "????????@yahoo.co.uk")

(setq rmail-preserve-inbox t)
(setq gnus-select-method '(nnimap "gmail"
                                  (nnimap-address "imap.gmail.com")
                                  (nnimap-server-port 993)
                                  (nnimap-stream ssl)))

(add-to-list 'gnus-secondary-select-methods '(nnimap "yahoo"
                                  (nnimap-address "imap.mail.yahoo.com")
                                  (nnimap-server-port 993)
                                  (nnimap-stream ssl)))

(setq message-send-mail-function 'smtpmail-send-it
      smtpmail-starttls-credentials '(("smtp.mail.yahoo.com" 587 nil nil))
      smtpmail-auth-credentials '(("smtp.mail.yahoo.com" 587 "????????@yahoo.co.uk" nil))
      smtpmail-default-smtp-server "smtp.mail.yahoo.com"
      smtpmail-smtp-server "smtp.mail.yahoo.com"
      smtpmail-smtp-service 587)

Re: tutorial for rmail and IMAP

[James Freer]

On Thu, 1 May 2014, David Hume wrote:
>
>> You managed to get yahoo working with imap. I've been trying to do that
>> without success.
>>
>>> And got gmail and yahoo working with imap, at least for reading. I had
>>> to set up application passwords for both as I have second factor
>>> authentication.
>>
>> What imap and smtp codes did you use? From what I can gather yahoo has imap
>> codes for reading mail on mobile phone apps but not PC email.
>>
>
> Notice that this is gnus, not rmail.
>
> I am not sure what you mean by codes.
>
> I also have a .auth file. I have this in .gnus:
>
> (setq user-full-name "????????")
> (setq user-mail-address "????????@yahoo.co.uk")
>
> (setq rmail-preserve-inbox t)
> (setq gnus-select-method '(nnimap "gmail"
>                                  (nnimap-address "imap.gmail.com")
>                                  (nnimap-server-port 993)
>                                  (nnimap-stream ssl)))
>
> (add-to-list 'gnus-secondary-select-methods '(nnimap "yahoo"
>                                  (nnimap-address "imap.mail.yahoo.com")
>                                  (nnimap-server-port 993)
>                                  (nnimap-stream ssl)))
>
> (setq message-send-mail-function 'smtpmail-send-it
>      smtpmail-starttls-credentials '(("smtp.mail.yahoo.com" 587 nil nil))
>      smtpmail-auth-credentials '(("smtp.mail.yahoo.com" 587 "????????@yahoo.co.uk" nil))
>      smtpmail-default-smtp-server "smtp.mail.yahoo.com"
>      smtpmail-smtp-server "smtp.mail.yahoo.com"
>      smtpmail-smtp-service 587)

Ports I meant sorry. I see you use the tls 587 - I tried the ssl 465 which may 
be where I've gone wrong.

james

Re: tutorial for rmail and IMAP

[James Freer]

On Thu, 1 May 2014, David Hume wrote:
>
>> You managed to get yahoo working with imap. I've been trying to do that
>> without success.
>>
>>> And got gmail and yahoo working with imap, at least for reading. I had
>>> to set up application passwords for both as I have second factor
>>> authentication.
>>
>> What imap and smtp codes did you use? From what I can gather yahoo has imap
>> codes for reading mail on mobile phone apps but not PC email.
>>
>
> Notice that this is gnus, not rmail.
>
> I am not sure what you mean by codes.
>
> I also have a .auth file. I have this in .gnus:
>
> (setq user-full-name "????????")
> (setq user-mail-address "????????@yahoo.co.uk")
>
> (setq rmail-preserve-inbox t)
> (setq gnus-select-method '(nnimap "gmail"
>                                  (nnimap-address "imap.gmail.com")
>                                  (nnimap-server-port 993)
>                                  (nnimap-stream ssl)))
>
> (add-to-list 'gnus-secondary-select-methods '(nnimap "yahoo"
>                                  (nnimap-address "imap.mail.yahoo.com")
>                                  (nnimap-server-port 993)
>                                  (nnimap-stream ssl)))
>
> (setq message-send-mail-function 'smtpmail-send-it
>      smtpmail-starttls-credentials '(("smtp.mail.yahoo.com" 587 nil nil))
>      smtpmail-auth-credentials '(("smtp.mail.yahoo.com" 587 "????????@yahoo.co.uk" nil))
>      smtpmail-default-smtp-server "smtp.mail.yahoo.com"
>      smtpmail-smtp-server "smtp.mail.yahoo.com"
>      smtpmail-smtp-service 587)

Sorted now - many thanks David
I used ssl 465 and all is fine. Perhaps yahoo have changed to imap... the last 
time I tried I'm sure one could only use pop.

james