Emacs Manual: Mail sending via SMTP

Emacs Manual: Mail sending via SMTP

[Simon Josefsson]

The number of questions about SMTP in various emacs newsgroups
suggests that it might be useful to at least document it.  I intend
the commit the following to man/sending.texi unless someone objects.

I would appreciate if a native speaker (or anyone that knows english),
and/or someone with familiarity with the typographic conventions (and
whatnot) used in the Emacs manual could proofread this.  Of course,
technical corrections are also appreciated.  Thanks.



Sending via SMTP
================

   Emacs includes a package for sending your mail to a SMTP server and
have it take care of delivering it to the final destination, rather
than letting the MTA on your local system take care of it.  This can be
useful if you don't have a MTA set up on your host, or if your machine
is often disconnected from the Internet.

   Sending mail via SMTP requires configuring your mail user agent
(see *note Mail Methods::) to use the SMTP library.  How to do this should be
described for each mail user agent; for the Message and Gnus user
agents the variable `message-send-mail-function' (see *note Mail Variables: (message)Mail Variables.)
is used.

   The variable `send-mail-function' controls how the default mail user
agent sends mail.  It should be set to a function.  The default is
`sendmail-send-it', but must be set to `smtpmail-send-it' in order to
use the SMTP library.  `feedmail-send-it' is another option.

   Before using SMTP you must find out the hostname of the SMTP server
to use.  Your system administrator should provide you with this
information, but often it is the same as the server you receive mail
from.

   The variable `smtpmail-smtp-server' controls the hostname of the
server to use.  It is a string with an IP address or hostname.  It
defaults to the contents of the `SMTPSERVER' environment variable, or,
if empty, the contents of `smtpmail-default-smtp-server'.

   The variable `smtpmail-default-smtp-server' controls the default
hostname of the server to use.  It is a string with an IP address or
hostname.  It must be set before the SMTP library is loaded.  It has no
effect if set after the SMTP library has been loaded, or if
`smtpmail-smtp-server' is defined.  It is usually set by system
administrators in a site wide initialization file.

   SMTP is normally used on the registered "smtp" TCP service port 25.
Some environments use SMTP in "Mail Submission" mode, which uses port
587.  Using other ports is not uncommon, either for security by
obscurity purposes, port forwarding, or otherwise.

   The variable `smtpmail-smtp-service' controls the port on the server
to contact.  It is either a string, in which case it will be translated
into an integer using system calls, or an integer.

   Many environments require SMTP clients to authenticate themselves
before they are allowed to route mail via a server.  The two following
variables contains the authentication information needed for this.  The
first variable, `smtpmail-auth-credentials', instructs the SMTP library
to use a SASL authentication step, currently only the CRAM-MD5, PLAIN
and LOGIN-MD5 mechanisms are supported and will be selected in that
order if the server supports them.  The second variable,
`smtpmail-starttls-credentials', instructs the SMTP library to connect
to the server using STARTTLS.  This means the protocol exchange can be
integrity protected and confidential by using TLS, and optionally also
authentication of the client.  It is common to use both these
mechanisms, e.g. to use STARTTLS to achieve integrity and
confidentiality and then use SASL for client authentication.

   The variable `smtpmail-auth-credentials' contains a list of
hostname, port, username and password tuples.  When the SMTP library
connects to a host on a certain port, this variable is searched to find
a matching entry for that hostname and port.  If an entry is found, the
authentication process is invoked and the credentials are used.  The
hostname field follows the same format as `smtpmail-smtp-server' (i.e.,
a string) and the port field the same format as `smtpmail-smtp-service'
(i.e., a string or an integer).  The username and password fields,
which either can be `nil' to indicate that the user is queried for the
value interactively, should be strings with the username and password,
respectively, information that is normally provided by system
administrators.

   The variable `smtpmail-starttls-credentials' contains a list of
tuples with hostname, port, name of file containing client key, and
name of file containing client certificate.  The processing is similar
to the previous variable.  The client key and certificate may be `nil'
if you do not wish to use client authentication.  The use of this
variable requires the `starttls' external program to be installed, you
can get it from `ftp://ftp.opaopa.org/pub/elisp/starttls-*.tar.gz'.

   The remaining variables are more esoteric and is normally not needed.

   The variable `smtpmail-debug-info' controls whether to print the
SMTP protocol exchange in the minibuffer, and retain the entire
exchange in a buffer `*trace of SMTP session to mail.example.org*'.

   The variable `smtpmail-debug-verb' controls whether to send the VERB
token to the server.  The VERB server instructs the server to be more
verbose, and often also to attempt final delivery while your SMTP
session is still running.  It is usually only useful together with
`smtpmail-debug-info'.  Note that this may cause mail delivery to take
considerable time if the final destination cannot accept mail.

   The variable `smtpmail-local-domain' controls the hostname sent in
the first EHLO or HELO command sent to the server.  It should only be
set if the `system-name' function returns a name that isn't accepted by
the server.  Do not set this variable unless your server complains.

   The variable `smtpmail-sendto-domain' makes the SMTP library add `@'
and the specified value to recipients specified in the message when
they are sent using the RCPT TO command.  Some configurations of
sendmail requires this behaviour.  Don't bother to set this unless you
have get an error like:

     	Sending failed; SMTP protocol error

   when sending mail, and the *trace of SMTP session to <somewhere>*
buffer (enabled via `smtpmail-debug-info') includes an exchange like:

     	RCPT TO: <someone>
     	501 <someone>: recipient address must contain a domain

   The variable `smtpmail-queue-mail' controls whether a simple off
line mail sender is active.  This variable is a boolean, and defaults
to `nil' (disabled).  If this is non-nil, mail is not sent immediately
but rather queued in the directory `smtpmail-queue-dir' and can be
later sent manually by invoking `smtpmail-send-queued-mail' (typically
when you connect to the Internet).

   The variable `smtpmail-queue-dir' specifies the name of the
directory to hold queued messages.  It defaults to
`~/Mail/queued-mail/'.

   The function `smtpmail-send-queued-mail' can be used to send any
queued mail when `smtpmail-queue-mail' is enabled.  It is typically
invoked interactively with `M-x RET smtpmail-send-queued-mail RET' when
you are connected to the Internet.

Re: Emacs Manual: Mail sending via SMTP

[Simon Josefsson]

I committed it (man/sending.texi), in case someone wants to proofread
the texinfo source.  It can always be removed if it is inappropriate
somehow.

Re: Emacs Manual: Mail sending via SMTP

[Kai Großjohann]

Simon Josefsson <jas@extundo.com> writes:

>    Sending mail via SMTP requires configuring your mail user agent
> (see *note Mail Methods::) to use the SMTP library.  How to do this should be
> described for each mail user agent; for the Message and Gnus user
> agents the variable `message-send-mail-function' (see *note Mail Variables: (message)Mail Variables.)
> is used.
>
>    The variable `send-mail-function' controls how the default mail user
> agent sends mail.  It should be set to a function.  The default is
> `sendmail-send-it', but must be set to `smtpmail-send-it' in order to
> use the SMTP library.  `feedmail-send-it' is another option.

How about putting some example code there?  Something like this:

    If you use RMAIL, the first line will enable the SMTP library; if
    you use Gnus, the second will do:

        (setq send-mail-function 'smtpmail-send-it)
        (setq message-send-mail-function 'smtpmail-send-it)

That way, people can copy and paste easily.

Also, I wonder if it would be useful to give it a little bit more
structure, visually?  For example, you could talk about the concepts
in normal paragraphs, but then use a description list for explaining
each variable.  Also, maybe it would be useful to have a subheading
for the part that talks about authentication.  That would make it
easier for people to find it.

And finally, I'm not sure about the policy regarding `esoteric'
variables.  It's possible that these should normally be documented in
the manual.  Then the description list format would be ideal for
doing that (possibly under an "esoteric variables" heading).

-- 
A preposition is not a good thing to end a sentence with.

Re: Emacs Manual: Mail sending via SMTP

[Alex Schroeder]

Simon Josefsson <jas@extundo.com> writes:

>    Emacs includes a package for sending your mail to a SMTP server and
> have it take care of delivering it to the final destination, rather
> than letting the MTA on your local system take care of it.  This can be
> useful if you don't have a MTA set up on your host, or if your machine
> is often disconnected from the Internet.

When I think back to the days when I started using Emacs, then SMTP
and MTA would not have made much sense to me.  How about this
introduction:

   On the Internet, mail is sent from host to host using the simple
mail transfer protocol (SMTP).  When you read and write mail you are
using a mail program that does not use SMTP -- it just reads mails
from files.  This is called a mail user agent (MUA).  The mail
transfer agent (MTA) is the program that accepts mails via SMTP and
stores them in files.  You also need a mail transfer agent when you
send mails.  You mail program has to send its mail to a MTA that can
pass it on using SMTP.

   Emacs includes a package for sending your mail to a SMTP server and
have it take care of delivering it to the final destination, rather
than letting the MTA on your local system take care of it.  ...

Alex.

Re: Emacs Manual: Mail sending via SMTP

[Robert J. Chassell]

I am confused.  jas@extundo.com says that 

    Emacs includes a package for sending your mail to a SMTP server
    [an MTA] ...

but Alex Schroeder <alex@emacswiki.org> writes also of needing an MTA
to receive email.  It sounds that Emacs provides an MTA only for
sending email.

Does Emacs always need to use the non-Emacs MTA on your local system
to receive email, but provides you with the choice of using either the
non-Emacs MTA on your local system to send email or of using the MTA
that come with Emacs?

-- 
    Robert J. Chassell                         Rattlesnake Enterprises
    http://www.rattlesnake.com                  GnuPG Key ID: 004B4AC8
    http://www.teak.cc                             bob@rattlesnake.com

Re: Emacs Manual: Mail sending via SMTP

[Alex Schroeder]

"Robert J. Chassell" <bob@rattlesnake.com> writes:

> Does Emacs always need to use the non-Emacs MTA on your local system
> to receive email, but provides you with the choice of using either the
> non-Emacs MTA on your local system to send email or of using the MTA
> that come with Emacs?

Yes, because the receiving part of the MTA is the one that receices
SMTP data from the Internet -- it must listen on the SMTP port act
like a server.  Emacs does not do that.

Alex.

Re: Emacs Manual: Mail sending via SMTP

[Benjamin Riefenstahl]

Hi Robert,


"Robert J. Chassell" <bob@rattlesnake.com> writes:
> Does Emacs always need to use the non-Emacs MTA on your local system
> to receive email, but provides you with the choice of using either
> the non-Emacs MTA on your local system to send email or of using the
> MTA that come with Emacs?

Emacs can not act as a receiving MTA (it is not a mail server).  But
the Emacs mail clients (at least some of them) do provide the ability
to actively query a non-local MTA (an MTA on a mail server) via the
POP or IMAP protocols.  So you don't need a *local* MTA for receiving
mail.


Hope this helps, benny

Re: Emacs Manual: Mail sending via SMTP

[Simon Josefsson]

Alex Schroeder <alex@emacswiki.org> writes:

> Simon Josefsson <jas@extundo.com> writes:
>
>>    Emacs includes a package for sending your mail to a SMTP server and
>> have it take care of delivering it to the final destination, rather
>> than letting the MTA on your local system take care of it.  This can be
>> useful if you don't have a MTA set up on your host, or if your machine
>> is often disconnected from the Internet.
>
> When I think back to the days when I started using Emacs, then SMTP
> and MTA would not have made much sense to me.  How about this
> introduction:

Good idea, I added it.

Re: Emacs Manual: Mail sending via SMTP

[Simon Josefsson]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> Simon Josefsson <jas@extundo.com> writes:
>
>>    Sending mail via SMTP requires configuring your mail user agent
>> (see *note Mail Methods::) to use the SMTP library.  How to do this should be
>> described for each mail user agent; for the Message and Gnus user
>> agents the variable `message-send-mail-function' (see *note Mail Variables: (message)Mail Variables.)
>> is used.
>>
>>    The variable `send-mail-function' controls how the default mail user
>> agent sends mail.  It should be set to a function.  The default is
>> `sendmail-send-it', but must be set to `smtpmail-send-it' in order to
>> use the SMTP library.  `feedmail-send-it' is another option.
>
> How about putting some example code there?  Something like this:
>
>     If you use RMAIL, the first line will enable the SMTP library; if
>     you use Gnus, the second will do:
>
>         (setq send-mail-function 'smtpmail-send-it)
>         (setq message-send-mail-function 'smtpmail-send-it)
>
> That way, people can copy and paste easily.

Good idea, added.  Also, it was suggested to split out the SMTP
section into a separate file, so I have done that.  I made some other
changes too.

> Also, I wonder if it would be useful to give it a little bit more
> structure, visually?  For example, you could talk about the concepts
> in normal paragraphs, but then use a description list for explaining
> each variable.  Also, maybe it would be useful to have a subheading
> for the part that talks about authentication.  That would make it
> easier for people to find it.

Yup.  Done.

> And finally, I'm not sure about the policy regarding `esoteric'
> variables.  It's possible that these should normally be documented in
> the manual.  Then the description list format would be ideal for
> doing that (possibly under an "esoteric variables" heading).

Since it is now a standalone manual, I think everything that we can
say about the library is appropriate.

Re: Emacs Manual: Mail sending via SMTP

[Alex Schroeder]

Simon Josefsson <jas@extundo.com> writes:

> Since it is now a standalone manual, I think everything that we can
> say about the library is appropriate.

Yay!  Maybe we can expand it a bit more?  :)

Russel's answer to my original intro-paragraph made it clear that a
separate section might be appropriate -- explain MTA, MUA, SMTP (we
have these three already), POP3, IMAP (the two major players only),
spool, procmail (mail delivery agent, MDA?).

Here is an expanded version.

Cheers,

Alex "Document Mode" Schroeder.  :)




How Mail Works

   On the Internet, mail is sent from mail host to mail host using the
simple mail transfer protocol (SMTP).  To send and receive mail, you
must get it from and send it to a mail host.  Every mail host runs a
mail transger agent (MTA) such as sendmail that accepts mails and
passes them on.  The communication between a mail host and other
clients does not necessarily involve SMTP, however.  Here is short
overview of what is involved.

   The mail program -- also called a mail user agent (MUA) -- usually
sends outgoing mail to a mail host.  When your computer is permanently
connected to the internet, it might even be a mail host itself.  In
this case, the MUA will send mail to the SMTP port on the same
computer.  A local mail transfer agent (MTA) such as exim or sendmail
will take care of your mail and pass it on to the next mail host.

   When you are only connected to the internet from time to time, your
internet service provider (ISP) has probably told you which mail host
to use.  You must configure your MUA to use that mail host.  Since you
are reading this manual, you probably want to configure Emacs to use
SMTP to send mail to that mail host.  More on that in the next
section.

   Things are different when reading mail.  The mail host responsible
for your mail keeps it in a file somewhere.  When your computer is
that mail host, this file is called a spool, and sometimes located in
the directory /var/spool/mail/.  All your MUA has to do is read mail
from the spool, then.

   When your computer is not always connected to the Internet, you
must get the mail from the remote mail host using a protocol such as
POP3 or IMAP.  POP3 essentially downloads all your mail from the mail
host to your computer.  The mail is stored in some file on your
computer, and again, all your MUA has to do is read mail from the
spool.  Sometimes mail is passed to yet another program, a mail
delivery agent (MDA) such as procmail.  These delivery agents often
allow you to filter and munge your mails before you get to see it.

   When you read mail from various machines, downloading mail from the
mail host to your current machine is not convenient.  In that case,
you will probably want to use the IMAP protocol.  Your mail is kept on
the mail host, and you can read it while you are connected via IMAP to
the mail host.

   So how does reading mail via the web work, you ask.  In that case,
the web interface just allows you to remote-control a MUA on the web
host.  Whether the web host is also a mail host, and how all the
pieces interact is completely irrelevant.  You usually cannot use
Emacs to read mail via the web, unless you use software that parses
the ever-changing HTML of the web interface.

Emacs speaks SMTP 

   Emacs includes a package for sending your mail to a SMTP server and
have it take care of delivering it to the final destination, rather
than letting the MTA on your local system take care of it.  ...

Re: Emacs Manual: Mail sending via SMTP

[Simon Josefsson]

Alex Schroeder <alex@emacswiki.org> writes:

> Russel's answer to my original intro-paragraph made it clear that a
> separate section might be appropriate -- explain MTA, MUA, SMTP (we
> have these three already), POP3, IMAP (the two major players only),
> spool, procmail (mail delivery agent, MDA?).
>
> Here is an expanded version.

Thanks, I added it.  I changed two things (conceptually an MDA is
always used to deliver mail from the MTA into local spool files, and
SMTP is rarely used to connect the MTA on the local host, instead mail
is piped into /usr/lib/sendmail).