Changing the default for `send-mail-function'

Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

As previously discussed, the default value of `send-mail-function' would
change to `smtpmail-send-it' (on all architectures) after the smtpmail.el
rework was complete.

This has now been done, so unless anybody has good objections to this
change, I'll be doing it tonight.

(This also means that `message-send-mail-function' will change if you're
running Emacs 24.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 26 Jun 2011 11:14:13 +0200
> Cc: ding@gnus.org
> 
> As previously discussed, the default value of `send-mail-function' would
> change to `smtpmail-send-it' (on all architectures) after the smtpmail.el
> rework was complete.

Could you please summarize how will this work?  smtpmail-send-it needs
the address of the SMTP server (and other parameters, like the port
and the credentials) to be configured, before it will work, doesn't
it?  Or is there some magic in the new code that somehow gleans that
automatically?

If the SMTP server address is to be supplied by the user, then
send-mail-function should remain set to mailclient-send-it on
MS-Windows.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If the SMTP server address is to be supplied by the user, then
> send-mail-function should remain set to mailclient-send-it on
> MS-Windows.

The server address is supplied by the user, just like it is on all other
mail clients (Mail.app, Thunderbird, Outlook, etc).  smtpmail.el will
query the user for the SMTP server if it isn't set already.

`mailclient-send-it' pops up the system mailer, and you have to press
"Send" there too, as I understand it?  (I haven't used it myself.)  I
don't think that's a very user-friendly (default) interface.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Reiner Steib]

On Sun, Jun 26 2011, Lars Magne Ingebrigtsen wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> If the SMTP server address is to be supplied by the user, then
>> send-mail-function should remain set to mailclient-send-it on
>> MS-Windows.
>
> The server address is supplied by the user, just like it is on all other
> mail clients (Mail.app, Thunderbird, Outlook, etc).  

It would be great if Emacs (and Gnus) could do like Thunderbird 3.
Thunderbird uses a list of email provider configuration information
(Mozilla ISP database[1]) as well as well known hosts (and MX records
from DNS?).  E.g. for provider.tld it checks imap.provider.tld,
smtp.provider.tld, mail.provider.tld ...

[1] http://kb.mozillazine.org/Creating_accounts_in_Thunderbird_for_popular_email_providers#ISP_Database
    http://ispdb.mozillamessaging.com/
    http://ispdb.mozillamessaging.com/list

> smtpmail.el will query the user for the SMTP server if it isn't set
> already.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> It would be great if Emacs (and Gnus) could do like Thunderbird 3.
> Thunderbird uses a list of email provider configuration information
> (Mozilla ISP database[1]) as well as well known hosts (and MX records
> from DNS?).  E.g. for provider.tld it checks imap.provider.tld,
> smtp.provider.tld, mail.provider.tld ...

How does that work?  Typically users are running behind a NAT firewall
these days, aren't they?  So you can use the local IP address to find
out what ISP you're in.

And doesn't everybody just use smtp.gmail.com, anyway?  :-)

But, yes, that does seem like it would be nice to have.  I looked up my
own IP address there, and it said that it didn't have any info, so
perhaps it's US centric?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Reiner Steib]

On Sun, Jun 26 2011, Lars Magne Ingebrigtsen wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>> It would be great if Emacs (and Gnus) could do like Thunderbird 3.
>> Thunderbird uses a list of email provider configuration information
>> (Mozilla ISP database[1]) as well as well known hosts (and MX records
>> from DNS?).  E.g. for provider.tld it checks imap.provider.tld,
>> smtp.provider.tld, mail.provider.tld ...
>
> How does that work?  Typically users are running behind a NAT firewall
> these days, aren't they?  So you can use the local IP address to find
> out what ISP you're in.

(you meant "can't"?)

I should have been more precise: After you enter your mail address,
say foobar<at>provider.tld, it checks for provider.tld in the ISP
database and for the hosts names mentioned above to find a suitable
POP/IMAP and SMTP server.

> And doesn't everybody just use smtp.gmail.com, anyway?  :-)
>
> But, yes, that does seem like it would be nice to have.  I looked up my
> own IP address there, and it said that it didn't have any info, so
> perhaps it's US centric?

It worked fine here for gmx.de, t-online.de, fastmail.fm (all listed
in the Mozilla ISP DB) but also for my address at uni-ulm.de (not
listed -> found imap and mail (smtp)).

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

>> How does that work?  Typically users are running behind a NAT firewall
>> these days, aren't they?  So you can use the local IP address to find
>> out what ISP you're in.
>
> (you meant "can't"?)

Yeah.  :-)

> I should have been more precise: After you enter your mail address,
> say foobar<at>provider.tld, it checks for provider.tld in the ISP
> database and for the hosts names mentioned above to find a suitable
> POP/IMAP and SMTP server.

Oh, right.  Hm.  Isn't that even less useful?  :-)  My ISP is get.no,
but I know nobody that uses that ISP's incoming mail server.  So if I
query the Mozilla DB for @gnus.org, it probably won't be very helpful.

I'm just wondering whether the mapping from mail address to outgoing
SMTP address from a database is useful beyond background noise.  I'd
guess it would work for some people, but my gut feeling is that it would
not work for the majority of users.

And using a third-party service is a bit iffy.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Jason Rumney]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> How does that work?  Typically users are running behind a NAT firewall
> these days, aren't they?  So you can use the local IP address to find
> out what ISP you're in.

I think it uses the domain part of the email address, not the IP address
of the local machine.  Users generally know their email address, even if
they do not know their SMTP server.

Re: Changing the default for `send-mail-function'

[Jochen Hein]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>
>> It would be great if Emacs (and Gnus) could do like Thunderbird 3.
>> Thunderbird uses a list of email provider configuration information
>> (Mozilla ISP database[1]) as well as well known hosts (and MX records
>> from DNS?).  E.g. for provider.tld it checks imap.provider.tld,
>> smtp.provider.tld, mail.provider.tld ...
>
> How does that work?

I have no idea how that really works, but I tried thunderbird recently.
I do have a local DNS for jochen.org with aliases imap.jochen.org and
smtp.jochen.org for the right systems - and it just guessed correctly.
I think thunderbird looked into the local domain somehow (search in
/etc/resolv.conf or something). 

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 26 Jun 2011 12:56:03 +0200
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > If the SMTP server address is to be supplied by the user, then
> > send-mail-function should remain set to mailclient-send-it on
> > MS-Windows.
> 
> The server address is supplied by the user, just like it is on all other
> mail clients (Mail.app, Thunderbird, Outlook, etc).  smtpmail.el will
> query the user for the SMTP server if it isn't set already.

Emacs until now didn't require the user to provide the server address.
At least on Windows, the system mailer (Outlook or whatever) is
normally already configured, and many times by a person other than the
one who invoked Emacs.  The latter might not know the address or the
credentials, or both.  We don't want that to get in the way of an
Emacs user who wants to report a bug, for example.

So please leave the value mailclient-send-it on Windows.

> `mailclient-send-it' pops up the system mailer, and you have to press
> "Send" there too, as I understand it?

Yes, and also paste the body of the message (Emacs places it in the
clipboard).  But they don't need to configure Outlook.

> I don't think that's a very user-friendly (default) interface.

Maybe not, but it won't block a bug report on technicalities.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> Emacs until now didn't require the user to provide the server address.
> At least on Windows, the system mailer (Outlook or whatever) is
> normally already configured, and many times by a person other than the
> one who invoked Emacs.  The latter might not know the address or the
> credentials, or both.  We don't want that to get in the way of an
> Emacs user who wants to report a bug, for example.

It's the same situation on Linux, really.  Today, if you want to send
email on Linux, you will have configured exim to do the right thing.
(Or it will be configured by somebody else for you.)

So the situation is the same on Linux and Windows: People today
necessarily have configured their mail setup, or had it configured for
them.

The problem that switching the default from `sendmail-send-it' (on
Linux) and `mailclient-send-it' (on Windows) is to make Emacs work as a
mail client out of the box.

If you install Ubuntu on a new machine, it will install exim, but leave
exim in local delivery mode only by default.  If you then send email
from your brand new fresly installed Emacs, it will fail silently.  It
will call `sendmail-send-it', and exim will take it, and deliver a
bounce locally.  Which probably won't be seen by the user.

I think that's pretty unacceptable behaviour.

On a freshly installed Windows, the situation is similar, but at least
it won't fail silently.  `mailclient-send-it' will send the message,
which will then open Outlook (or something), which then won't be able to
send the message.  But at least it'll tell you so.  I still think it's
pretty yucky mail behaviour, and one that no other common mail clients
will emulate.

Defaulting to `smtpmail-send-it' will make Emacs behave exactly like all
other mail clients: It'll ask you for the outgoing server name.  If you
give it the wrong name, or you don't know it, you will get a proper
error message saying what the problem is exactly.

While this default change will annoy 95% of the current users (since
they have to type the SMTP server name once (and possible credentials if
the SMTP server requires them)), it'll make Emacs mail work for new
users reliably.

However, I think there's a third possibility here, beyond keeping the
current defaults and "hard-swapping" them.  :-)

What about if we default to `query-user', and then the first time you
use it, it'll ask "Use the built-in SMTP support, or use {the Windows
mailer,exim} to send the mail?", and then save the response.

(It probably needs to top up a help buffer explaining the choices in
more detail, though.)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Philipp Haselwarter]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> Emacs until now didn't require the user to provide the server address.
>> At least on Windows, the system mailer (Outlook or whatever) is
>> normally already configured, and many times by a person other than the
>> one who invoked Emacs.  The latter might not know the address or the
>> credentials, or both.  We don't want that to get in the way of an
>> Emacs user who wants to report a bug, for example.
>
> It's the same situation on Linux, really.  Today, if you want to send
> email on Linux, you will have configured exim to do the right thing.
> (Or it will be configured by somebody else for you.)
>
> So the situation is the same on Linux and Windows: People today
> necessarily have configured their mail setup, or had it configured for
> them.
>
fwiw, most people I know just use the web interface (gmail, their uni…)

> The problem that switching the default from `sendmail-send-it' (on
> Linux) and `mailclient-send-it' (on Windows) is to make Emacs work as a
> mail client out of the box.
>
> If you install Ubuntu on a new machine, it will install exim, but leave
> exim in local delivery mode only by default.  If you then send email
> from your brand new fresly installed Emacs, it will fail silently.  It
> will call `sendmail-send-it', and exim will take it, and deliver a
> bounce locally.  Which probably won't be seen by the user.
>
> I think that's pretty unacceptable behaviour.
>
Very much. And most people won't bother to interrogate how to make it
work if it fails them the first time.

> On a freshly installed Windows, the situation is similar, but at least
> it won't fail silently.  `mailclient-send-it' will send the message,
> which will then open Outlook (or something), which then won't be able to
> send the message.  But at least it'll tell you so.  I still think it's
> pretty yucky mail behaviour, and one that no other common mail clients
> will emulate.
>
> Defaulting to `smtpmail-send-it' will make Emacs behave exactly like all
> other mail clients: It'll ask you for the outgoing server name.  If you
> give it the wrong name, or you don't know it, you will get a proper
> error message saying what the problem is exactly.
>
> While this default change will annoy 95% of the current users (since
> they have to type the SMTP server name once (and possible credentials if
> the SMTP server requires them)), it'll make Emacs mail work for new
> users reliably.
>
> However, I think there's a third possibility here, beyond keeping the
> current defaults and "hard-swapping" them.  :-)
>
> What about if we default to `query-user', and then the first time you
> use it, it'll ask "Use the built-in SMTP support, or use {the Windows
> mailer,exim} to send the mail?", and then save the response.
>
> (It probably needs to top up a help buffer explaining the choices in
> more detail, though.)

This sounds like the right thing to do, leaving the decision between a
emacs built-in mailer and an external one to the user.

I guess offering to query the Mozilla database and/or suggesting
smtp.example.com for a user@example.com address with a few hints on how
to find the right server would be seen as convenient by most.


-- 
Philipp Haselwarter

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 26 Jun 2011 15:08:35 +0200
> 
> If you install Ubuntu on a new machine, it will install exim, but leave
> exim in local delivery mode only by default.  If you then send email
> from your brand new fresly installed Emacs, it will fail silently.  It
> will call `sendmail-send-it', and exim will take it, and deliver a
> bounce locally.  Which probably won't be seen by the user.
> 
> I think that's pretty unacceptable behaviour.

Failing silently is indeed bad.

> On a freshly installed Windows, the situation is similar, but at least
> it won't fail silently.

Exactly.

> What about if we default to `query-user', and then the first time you
> use it, it'll ask "Use the built-in SMTP support, or use {the Windows
> mailer,exim} to send the mail?", and then save the response.

That'd be fine, I think.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> What about if we default to `query-user', and then the first time you
>> use it, it'll ask "Use the built-in SMTP support, or use {the Windows
>> mailer,exim} to send the mail?", and then save the response.
>
> That'd be fine, I think.

I can hack that up pretty easily, but we won't be able to change the
default again afterwards, which kinda makes me nervous.  :-)

That is, if we change this to:

(defcustom send-mail-function 'query)

Then the next time a user sends email, they'll get queried, and will
choose one value, which will then be stored in .emacs (via customize).

So, basically, all Emacs users will have a setting for this variable
stored after a while.

We then can't change the default of `send-mail-function' to a new,
fabulous function in the future that we have determined will solve all
out problem, and we should default to instead.  :-)

So perhaps we should think about a `query' solution a bit more first
before we actually do it.

Hm...  another possible solution here would be to have a sort of
multi-sending solution.

It might work as follows.  On Windows, this would default to

(defcustom send-mail-function '(smtpmail-send-it mailclient-send-it))

This would mean that the user, when sending email, would first be
prompted with

"Outgoing SMTP server?"

and if the user just responds RET, or the sending fails in any other
protocol-related way, it just goes to the second function in the list...

On Linux, of course, it wouldn't make sense to fall back to
`sendmail-sent-it', since that will fail silently if it's not
configured.

So...  eh...  doesn't seem all that attractive, either.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

> That is, if we change this to:

> (defcustom send-mail-function 'query)

> Then the next time a user sends email, they'll get queried, and will
> choose one value, which will then be stored in .emacs (via customize).

> So, basically, all Emacs users will have a setting for this variable
> stored after a while.

> We then can't change the default of `send-mail-function' to a new,
> fabulous function in the future that we have determined will solve all
> out problem, and we should default to instead.  :-)

Good point.  Here's a solution:

- set the default to smtpmail-send-it.
- If smtpmail is not configured, it can then start by prompting whether
  the user actually wants to use something else (either his MUA or his
  MTA), and if not then goes on prompting for smtpmail configuration.

BTW, we should push the fd.o or gconf guys to provide some centralized
configuration of email, so that all MUAs (including Gnus, Thunderbird,
...) can share info about how to send email, where to fetch it, ...


        Stefan

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Good point.  Here's a solution:
>
> - set the default to smtpmail-send-it.
> - If smtpmail is not configured, it can then start by prompting whether
>   the user actually wants to use something else (either his MUA or his
>   MTA), and if not then goes on prompting for smtpmail configuration.

This is basically what I've implemented with `sendmail-query-once' later
in the thread.  :-)

Could you take a look at whether what it does is reasonable, and if so,
I can check in a change that makes it the default.

> BTW, we should push the fd.o or gconf guys to provide some centralized
> configuration of email, so that all MUAs (including Gnus, Thunderbird,
> ...) can share info about how to send email, where to fetch it, ...

Yes.  That would be a good long-term solution for GNU/Linux-ey
distributions. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> If you install Ubuntu on a new machine, it will install exim, but leave
> exim in local delivery mode only by default.  If you then send email
> from your brand new fresly installed Emacs, it will fail silently.  It
> will call `sendmail-send-it', and exim will take it, and deliver a
> bounce locally.  Which probably won't be seen by the user.
>
> I think that's pretty unacceptable behaviour.

Agreed, though personally I haven't been able to get smtpmail to work
for my university's mail server.  Using smtpmail-send-it with no other
customizations, attempting to send mail returns a "sending failed:
relaying denied" error.

The mail server requires authenticated smtp, which is pretty common
nowadays; for sendmail-send-it, I've got postfix set up for sasl auth.
I see that smtpmail is supposed to support authenticated smtp, but
changing smtpmail-stream-type to `starttls' didn't work, either:

Debugger entered--Lisp error: (error "No process")
  signal(error ("No process"))
  error("No process")
  set-buffer-process-coding-system(raw-text-unix raw-text-unix)
  byte-code(...)
  smtpmail-via-smtp(("cyd@stupidchicken.com") #<buffer  smtpmail temp>)
  smtpmail-send-it()
  message-smtpmail-send-it()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)

It would be nice to get these issues ironed out if we're gonna be using
smtpmail as the default.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Agreed, though personally I haven't been able to get smtpmail to work
> for my university's mail server.  Using smtpmail-send-it with no other
> customizations, attempting to send mail returns a "sending failed:
> relaying denied" error.

Is that the complete error message you get?

> The mail server requires authenticated smtp, which is pretty common
> nowadays; for sendmail-send-it, I've got postfix set up for sasl auth.

smtpmail.el only supports PLAIN and CRAM-MD5 -- I didn't know that any
SMTP servers used SASL.  That should be easy enough to add, I think, but
I don't have access to any SMTP servers with SASL I can test with, which
makes things more difficult.

> I see that smtpmail is supposed to support authenticated smtp, but
> changing smtpmail-stream-type to `starttls' didn't work, either:
>
> Debugger entered--Lisp error: (error "No process")
>   signal(error ("No process"))
>   error("No process")
>   set-buffer-process-coding-system(raw-text-unix raw-text-unix)

Hm.  That's not a very nice error message.  :-)

Could you

(setq smtpmail-debug-info t)

and then look in the "*trace ...*" buffer after trying to send a message
and see what's in there?

> It would be nice to get these issues ironed out if we're gonna be using
> smtpmail as the default.

Yes, definitely.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>> Agreed, though personally I haven't been able to get smtpmail to work
>> for my university's mail server.  Using smtpmail-send-it with no other
>> customizations, attempting to send mail returns a "sending failed:
>> relaying denied" error.
>
> Is that the complete error message you get?

Backtrace below.  The issue is that the mail server is set up to reject
plain SMTP connections not originating from their own network.  External
connections are supposed to use authenticated SMTP.

Debugger entered--Lisp error: (error "Sending failed: 550 5.7.1 <cyd@stupidchicken.com>... Relaying denied.See http://www.yale.edu/email/norelay.html")
  signal(error ("Sending failed: 550 5.7.1 <cyd@stupidchicken.com>... Relaying denied.See http://www.yale.edu/email/norelay.html"))
  error("Sending failed: %s" "550 5.7.1 <cyd@stupidchicken.com>... Relaying denied.See http://www.yale.edu/email/norelay.html")
  smtpmail-send-it()
  message-smtpmail-send-it()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)
  call-interactively(message-send-and-exit nil nil)

> Could you
>
> (setq smtpmail-debug-info t)
>
> and then look in the "*trace ...*" buffer after trying to send a
> message and see what's in there?

Here it is.

220 vm-emlprdomr-05.its.yale.edu ESMTP server ready at Sun, 26 Jun 2011 15:46:37 -0400^M
250-vm-emlprdomr-05.its.yale.edu Hello c-71-192-165-84.hsd1.ct.comcast.net [71.192.165.84], pleased to meet you^M
250-ENHANCEDSTATUSCODES^M
250-PIPELINING^M
250-8BITMIME^M
250-SIZE 20971520^M
250-DSN^M
250-STARTTLS^M
250-DELIVERBY^M
250 HELP^M

Process smtpmail deleted

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Backtrace below.  The issue is that the mail server is set up to reject
> plain SMTP connections not originating from their own network.  External
> connections are supposed to use authenticated SMTP.
>
> Debugger entered--Lisp error: (error "Sending failed: 550 5.7.1
> <cyd@stupidchicken.com>... Relaying denied.See

Right; it's getting the expected error message, but:

[...]

> 250-ENHANCEDSTATUSCODES^M
> 250-PIPELINING^M
> 250-8BITMIME^M
> 250-SIZE 20971520^M
> 250-DSN^M
> 250-STARTTLS^M
> 250-DELIVERBY^M
> 250 HELP^M

It hasn't announced AUTH, so it's not tried.  And it's not offered AUTH
because it hasn't switched on STARTTLS.  But it's clearly announcing
STARTTLS, so it should be switching on STARTTLS.

So what could me making it not trying STARTTLS?  Do you have neither
compiled-in gnutls support nor gnutls-cli installed?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> It hasn't announced AUTH, so it's not tried.  And it's not offered AUTH
> because it hasn't switched on STARTTLS.  But it's clearly announcing
> STARTTLS, so it should be switching on STARTTLS.
>
> So what could me making it not trying STARTTLS?  Do you have neither
> compiled-in gnutls support nor gnutls-cli installed?

Ah yes, that was the problem.  I had inadvertently reconfigured without
gnutls support.  Rebuilding with gnutls solved the problem; thanks for
the pointer.  Probably smtpmail-send-it needs some code to handle that
failure case in a less cryptic way.

Also, I think `smtpmail-stream-type' should default to `starttls'.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Ah yes, that was the problem.  I had inadvertently reconfigured without
> gnutls support.  Rebuilding with gnutls solved the problem; thanks for
> the pointer.  Probably smtpmail-send-it needs some code to handle that
> failure case in a less cryptic way.

Yes, it should.  I'll see where I can most easily put the test and error
message... 

> Also, I think `smtpmail-stream-type' should default to `starttls'.

No, if you say `starttls', then the connection will fail if the server
doesn't offer (or require) STARTTLS, which isn't what we want, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Yes, it should.  I'll see where I can most easily put the test and error
> message... 

I've now made `open-network-stream' report back with an :error slot what
the error actually was, and made smtpmail.el report it.

It always seems that good error reporting is the most difficult thing to
do.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> smtpmail.el only supports PLAIN and CRAM-MD5 -- I didn't know that any
> SMTP servers used SASL. 

Now I'm even more confused than normal.  :-)  I though SASL was a
specific password/encryption thing (like CRAM-MD5), but it seems like it
a framework for specifying what password passing mechanisms to use.  And
smtpmail.el supports the two vastly most commonly used ones already --
PLAIN and CRAM-MD5, so I think we should be good there.

So we just need to find out why smtpmail.el doesn't try to send
credentials to your server, or why it doesn't do STARTTLS, or both.  The
trace buffer should help there.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

On 6/26/2011 12:35 PM, Chong Yidong wrote:

> Agreed, though personally I haven't been able to get smtpmail to work

I am trying on W32 with no success.

I have compiled with gnutls support. (gnutls-available-p) returns t and 
the following works also:

(require 'gnutls)
(open-gnutls-stream "tls" "tls-buffer" "imap.gmail.com" "imaps")

My configuration looks like this:

(require 'smtpmail)
(setq user-mail-address "myaddress@googlemail.com")
(setq send-mail-function 'smtpmail-send-it)
(setq message-send-mail-function 'smtpmail-send-it)
(setq smtpmail-smtp-server "smtp.gmail.com")
(setq smtpmail-smtp-service 587)
(setq smtpmail-stream-type 'starttls)
(setq smtpmail-debug-info t)
(setq smtpmail-debug-verb t)

Sending an email results in Emacs hanging. Ctrl-g aborts, then the trace 
buffer shows:

220 mx.google.com ESMTP a9sm4719085icy.18

Am I missing anything?

Christoph

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Christoph Scholtes <cschol2112@googlemail.com> writes:

> Sending an email results in Emacs hanging. Ctrl-g aborts, then the
> trace buffer shows:
>
> 220 mx.google.com ESMTP a9sm4719085icy.18

And that's all in the buffer?  Then that must mean that
`open-network-stream' doesn't even recognise the end of the greeting,
for some reason or other.

Could you try

(setq debug-on-quit t)

and then `C-g' when it hangs and mail me the resulting backtrace?

Sending via smtp.gmail.com works for me, for what it's worth.  But I
haven't tried from Windows.  But I do have Windows virtual machines set
up a work that I could try tomorrow.  Are there any nightly Windows
builds that I could install easily?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

On 6/26/2011 3:29 PM, Lars Magne Ingebrigtsen wrote:
> (setq debug-on-quit t)

Debugger entered--Lisp error: (quit)
   accept-process-output(#<process smtpmail> 0 50)
   network-stream-get-response(#<process smtpmail> 1 "^[0-9]+ .*\n")
   network-stream-open-starttls("smtpmail" #<buffer *trace of SMTP 
session to smtp.gmail.com*> "smtp.gmail.com" 587 (:type starttls 
:return-list t :capability-command "EHLO MARVIN\n" :end-of-command 
"^[0-9]+ .*\n" :success "^2.*\n" :always-query-capabilities t 
:starttls-function #[(capabilities) ".........byte code removed)
open-network-stream("smtpmail" #<buffer *trace of SMTP session to 
smtp.gmail.com*> "smtp.gmail.com" 587 :type starttls :return-list t 
:capability-command "EHLO MARVIN\n" :end-of-command "^[0-9]+ .*\n" 
:success "^2.*\n" :always-query-capabilities t :starttls-function 
#[(capabilities) ".............byte code removed)
[byte-code removed]
   smtpmail-via-smtp(("myaddress@gmail.com") #<buffer  smtpmail temp>)
   smtpmail-send-it()
   message-send-mail(nil)
   message-send-via-mail(nil)
   message-send(nil)
   message-send-and-exit(nil)
   call-interactively(message-send-and-exit nil nil)

Some byte code was removed from the trace.

Christoph

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Christoph Scholtes <cschol2112@googlemail.com> writes:

> On 6/26/2011 3:29 PM, Lars Magne Ingebrigtsen wrote:
>> (setq debug-on-quit t)
>
> Debugger entered--Lisp error: (quit)
>   accept-process-output(#<process smtpmail> 0 50)
>   network-stream-get-response(#<process smtpmail> 1 "^[0-9]+ .*\n")

Pretty much what I expected, and somewhat bewildering.  It's waiting for
"^[0-9]+ .*\n" in the buffer, and you said that the buffer contained:

220 mx.google.com ESMTP a9sm4719085icy.18

Was there a newline after that line?

This is the loop that's looking for the end of the greeting: 

    (with-current-buffer (process-buffer stream)
      (save-excursion
	(goto-char start)
	(while (and (memq (process-status stream) '(open run))
		    (not (re-search-forward end-of-command nil t)))
	  (accept-process-output stream 0 50)
	  (goto-char start))

I wonder whether you're hitting the never-chased-down obscure error
where `accept-process-output' changes the current buffer, which I've had
reports about once in a while from some users, but have never
understood.

Does the following hack make smtpmail.el work for you?

=== modified file 'lisp/net/network-stream.el'
*** lisp/net/network-stream.el	2011-06-26 21:05:06 +0000
--- lisp/net/network-stream.el	2011-06-26 22:04:17 +0000
***************
*** 305,310 ****
--- 305,311 ----
  	(while (and (memq (process-status stream) '(open run))
  		    (not (re-search-forward end-of-command nil t)))
  	  (accept-process-output stream 0 50)
+ 	  (set-buffer (process-buffer stream))
  	  (goto-char start))
  	;; Return the data we got back, or nil if the process died.
  	(unless (= start (point))

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

On 6/26/2011 4:05 PM, Lars Magne Ingebrigtsen wrote:

 > Does the following hack make smtpmail.el work for you?
 >
 > === modified file 'lisp/net/network-stream.el'
 > *** lisp/net/network-stream.el	2011-06-26 21:05:06 +0000
 > --- lisp/net/network-stream.el	2011-06-26 22:04:17 +0000
 > ***************
 > *** 305,310 ****
 > --- 305,311 ----
 >    	(while (and (memq (process-status stream) '(open run))
 >    		    (not (re-search-forward end-of-command nil t)))
 >    	  (accept-process-output stream 0 50)
 > + 	  (set-buffer (process-buffer stream))
 >    	  (goto-char start))
 >    	;; Return the data we got back, or nil if the process died.
 >    	(unless (= start (point))
 >

No, but I get past this specific error with this patch:

=== modified file 'lisp/mail/smtpmail.el'
*** lisp/mail/smtpmail.el	2011-06-22 19:24:51 +0000
--- lisp/mail/smtpmail.el	2011-06-26 22:33:50 +0000
***************
*** 639,645 ****
   		 :type smtpmail-stream-type
   		 :return-list t
   		 :capability-command (format "EHLO %s\r\n" (smtpmail-fqdn))
! 		 :end-of-command "^[0-9]+ .*\r\n"
   		 :success "^2.*\n"
   		 :always-query-capabilities t
   		 :starttls-function
--- 639,645 ----
   		 :type smtpmail-stream-type
   		 :return-list t
   		 :capability-command (format "EHLO %s\r\n" (smtpmail-fqdn))
! 		 :end-of-command "^[0-9]+ .*\n"
   		 :success "^2.*\n"
   		 :always-query-capabilities t
   		 :starttls-function


It seems to be a problem with line-endings on Windows.

It still does not send the mail though. Trace buffer:

220 mx.google.com ESMTP d6sm4780041icx.13
250-mx.google.com at your service, [71.208.187.152]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250 ENHANCEDSTATUSCODES
250-mx.google.com at your service, [71.208.187.152]
250-SIZE 35882577
250-8BITMIME
250-STARTTLS
250 ENHANCEDSTATUSCODES
220 2.0.0 Ready to start TLS
250-mx.google.com at your service, [71.208.187.152]
250-SIZE 35882577
250-8BITMIME
250-AUTH LOGIN PLAIN XOAUTH
250 ENHANCEDSTATUSCODES

Process smtpmail deleted
MAIL FROM:<myaddress@googlemail.com>

530-5.5.1 Authentication Required. Learn more at 


530 5.5.1 http://mail.google.com/support/bin/answer.py?answer=14257 
d6sm4780041icx.13

QUIT

221 2.0.0 closing connection d6sm4780041icx.13

 From the *Messages* buffer:

Sending...
Sending via mail...
gnutls.c: [1] (Emacs) GnuTLS library loaded: libgnutls-26.dll
530-5.5.1 Authentication Required. Learn more at 

530 5.5.1 http://mail.google.com/support/bin/answer.py?answer=14257 
d6sm4780041icx.13
221 2.0.0 closing connection d6sm4780041icx.13
smtpmail-send-it: Sending failed: 530-5.5.1 Authentication Required. 
Learn more at
530 5.5.1 http://mail.google.com/support/bin/answer.py?answer=14257 
d6sm4780041icx.13

It seems it loads the gnutls dll correctly. If I don't specify the 
authentication parameters, shouldn't it ask me to supply them?

Christoph

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Christoph Scholtes <cschol2112@googlemail.com> writes:

> ! 		 :end-of-command "^[0-9]+ .*\r\n"

[...]

> It seems to be a problem with line-endings on Windows.

Huh.  Hm.  Could it be an encoding problem?  That is, would binding

    (let* ((coding-system-for-read 'binary)
	   (coding-system-for-write 'binary)

before opening the connection, and/or

  (set-buffer-multibyte nil)

in the buffer be the right thing?  smtpmail.el didn't use to do that, so
I didn't do that either, but I think that would make sense.

Could you try the following patch?

=== modified file 'lisp/mail/smtpmail.el'
*** lisp/mail/smtpmail.el	2011-06-26 21:05:06 +0000
--- lisp/mail/smtpmail.el	2011-06-26 23:32:53 +0000
***************
*** 615,620 ****
--- 615,622 ----
                             (and mail-specify-envelope-from
                                  (mail-envelope-from))
                             user-mail-address))
+ 	(coding-system-for-read 'binary)
+ 	(coding-system-for-write 'binary)
  	response-code
  	process-buffer
  	result
***************
*** 626,631 ****
--- 628,634 ----
  	  (setq process-buffer
  		(get-buffer-create
  		 (format "*trace of SMTP session to %s*" host)))
+ 	  (set-buffer-multibyte nil)
  
  	  ;; clear the trace buffer of old output
  	  (with-current-buffer process-buffer

> It still does not send the mail though. Trace buffer:
>
> 220 mx.google.com ESMTP d6sm4780041icx.13

[...]

> 250-AUTH LOGIN PLAIN XOAUTH

It's announcing AUTH as it should...

> 250 ENHANCEDSTATUSCODES
>
> Process smtpmail deleted
> MAIL FROM:<myaddress@googlemail.com>
>
> 530-5.5.1 Authentication Required. Learn more at 

We're getting the right error message, so at this point, smtpmail should
be asking you for a password, which it didn't.

When I try the same, it does, so perhaps it's more line-ending encoding
issues?  Seems awfully strange though, but there's a slight possibility
that the patch above fixes this too.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Could you try the following patch?

Sorry, try this one:

=== modified file 'lisp/mail/smtpmail.el'
*** lisp/mail/smtpmail.el	2011-06-26 21:05:06 +0000
--- lisp/mail/smtpmail.el	2011-06-26 23:48:29 +0000
***************
*** 615,620 ****
--- 615,622 ----
                             (and mail-specify-envelope-from
                                  (mail-envelope-from))
                             user-mail-address))
+ 	(coding-system-for-read 'binary)
+ 	(coding-system-for-write 'binary)
  	response-code
  	process-buffer
  	result
***************
*** 629,634 ****
--- 631,637 ----
  
  	  ;; clear the trace buffer of old output
  	  (with-current-buffer process-buffer
+ 	    (set-buffer-multibyte nil)
  	    (setq buffer-undo-list t)
  	    (erase-buffer))
  

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

On 6/26/2011 5:49 PM, Lars Magne Ingebrigtsen wrote:

> Could you try the following patch?
>
> Sorry, try this one:

It works!

There must have been other instances where the line endings were messing 
things up. This fixes all problems on W32 and the mails are sent.

For testing, I applied the patch to a Debian built Emacs and it also 
works fine.

Christoph

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Christoph Scholtes <cschol2112@googlemail.com> writes:

> There must have been other instances where the line endings were
> messing things up. This fixes all problems on W32 and the mails are
> sent.

Great!  I checked in the same patch to bzr Emacs last night.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 27 Jun 2011 01:49:33 +0200
> 
>   	  ;; clear the trace buffer of old output
>   	  (with-current-buffer process-buffer
> + 	    (set-buffer-multibyte nil)
>   	    (setq buffer-undo-list t)
>   	    (erase-buffer))

Why?  What purpose would or could making the buffer unibyte possible
serve in this case?

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

Eli Zaretskii <eliz@gnu.org> writes:

> Why?  What purpose would or could making the buffer unibyte possible
> serve in this case?

Without this change, sending smtp mails still works for me.

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Christoph Scholtes <cschol2112@googlemail.com>
> Cc: Lars Magne Ingebrigtsen <larsi@gnus.org>,  emacs-devel@gnu.org
> Date: Sun, 26 Jun 2011 21:34:56 -0600
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Why?  What purpose would or could making the buffer unibyte possible
> > serve in this case?
> 
> Without this change, sending smtp mails still works for me.

Of course!  That buffer is pure ASCII anyway, right?

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>>   	  (with-current-buffer process-buffer
>> + 	    (set-buffer-multibyte nil)
>>   	    (setq buffer-undo-list t)
>>   	    (erase-buffer))
>
> Why?  What purpose would or could making the buffer unibyte possible
> serve in this case?

Pure cargo-culting.  :-)  I looked at what nnimap did differently with
encoding/charsets, and it was `coding-system-for-*', and
`set-buffer-multibyte'.

If somebody on Windows could try removing the `set-buffer-multibyte'
again and post via Gmail, that would be nice...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Mon, 27 Jun 2011 11:39:13 +0200
> Mail-Copies-To: never
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >>   	  (with-current-buffer process-buffer
> >> + 	    (set-buffer-multibyte nil)
> >>   	    (setq buffer-undo-list t)
> >>   	    (erase-buffer))
> >
> > Why?  What purpose would or could making the buffer unibyte possible
> > serve in this case?
> 
> Pure cargo-culting.  :-)  I looked at what nnimap did differently with
> encoding/charsets, and it was `coding-system-for-*', and
> `set-buffer-multibyte'.

Whenever you see that a buffer is set to unibyte, you should first
suspect that the code is wrong.

> If somebody on Windows could try removing the `set-buffer-multibyte'
> again and post via Gmail, that would be nice...

Cristoph already did, and said it still worked for him.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>> If somebody on Windows could try removing the `set-buffer-multibyte'
>> again and post via Gmail, that would be nice...
>
> Cristoph already did, and said it still worked for him.

The patch that he applied included the `set-buffer-multibyte' thing,
didn't it?

I agree that it's almost certain that it's not needed, but I'd rather
get explicit confirmation before possibly screwing up sending email
again.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Christoph Scholtes]

On 6/27/2011 3:39 AM, Lars Magne Ingebrigtsen wrote:

> If somebody on Windows could try removing the `set-buffer-multibyte'
> again and post via Gmail, that would be nice...

I did and it works also. Thank you for fixing this.

To Eli's point: is it safe to assume that the buffer content will always 
be pure ASCII? If not always, then this could lead to more bugs like the 
one we just fixed.

Christoph

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> Date: Mon, 27 Jun 2011 06:00:08 -0600
> From: Christoph Scholtes <cschol2112@googlemail.com>
> 
> To Eli's point: is it safe to assume that the buffer content will always 
> be pure ASCII?

That's the buffer through which we communicate with the SMTP server,
right?  If so, AFAIK the whole protocol is pure ASCII.

Re: Changing the default for `send-mail-function'

[Bill Wohler]

Eli Zaretskii <eliz@gnu.org> writes:

>> Date: Mon, 27 Jun 2011 06:00:08 -0600
>> From: Christoph Scholtes <cschol2112@googlemail.com>
>> 
>> To Eli's point: is it safe to assume that the buffer content will always 
>> be pure ASCII?
>
> That's the buffer through which we communicate with the SMTP server,
> right?  If so, AFAIK the whole protocol is pure ASCII.

Not if the binary extensions in RFC-3030 are in effect. However, the
gnu.org MX server eggs.gnu.org does not appear to have this extension
enabled. If you assume ASCII, you need to ensure that the mail server
never enables those options.

-- 
Bill Wohler <wohler@newt.com> aka <Bill.Wohler@nasa.gov>
http://www.newt.com/wohler/
GnuPG ID:610BD9AD

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Bill Wohler <wohler@newt.com>
> Date: Mon, 27 Jun 2011 17:03:41 -0700
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > That's the buffer through which we communicate with the SMTP server,
> > right?  If so, AFAIK the whole protocol is pure ASCII.
> 
> Not if the binary extensions in RFC-3030 are in effect. However, the
> gnu.org MX server eggs.gnu.org does not appear to have this extension
> enabled. If you assume ASCII, you need to ensure that the mail server
> never enables those options.

Are you talking about BINARYMIME?  If so, isn't the sender (i.e. Emacs
and specifically smtpmail.el) supposed to request that extension?

If I'm right, and Emacs needs to request that feature, then this has
no relevance on the issue at hand, because smtpmail doesn't request
BINARYMIME and in fact doesn't even support it.

Or am I missing something?

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If I'm right, and Emacs needs to request that feature, then this has
> no relevance on the issue at hand, because smtpmail doesn't request
> BINARYMIME and in fact doesn't even support it.
>
> Or am I missing something?

No, and it isn't really relevant in any case.  :-)

We're not using the stuff in the smtpmail.el buffer for anything (except
error reporting), so it's immaterial whether the buffer is single byte
or not, isn't it?  It's not like we insert the email into the buffer,
and then use `process-send-region' (or something) on it.

I'm removing that piece of the cargo cult now.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Tue, 28 Jun 2011 17:29:13 +0200
> 
> We're not using the stuff in the smtpmail.el buffer for anything (except
> error reporting), so it's immaterial whether the buffer is single byte
> or not, isn't it?

If it's immaterial (and I agree it is), the buffer should remain
multibyte.  We should IMO set buffers unibyte only if we have a very
good reason for that, because doing that opens subtle loopholes we are
better off without.  Lately, there are less and less "good reasons"
for using unibyte buffers, which is a Good Thing.

> I'm removing that piece of the cargo cult now.

Thank you.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

I've now implemented the query-the-user-and-then-save-the-default
idea.

The default of `send-mail-function' changes to `sendmail-query-once',
which will query and then save and set the
`sendmail-query-once-function' variable via customize.

=== modified file 'lisp/mail/sendmail.el'
*** lisp/mail/sendmail.el	2011-06-22 23:39:29 +0000
--- lisp/mail/sendmail.el	2011-06-28 18:45:08 +0000
***************
*** 142,162 ****
  ;; when loaddefs.el is loaded. `custom-reevaluate-setting' needs the
  ;; standard value.
  ;;;###autoload
! (put 'send-mail-function 'standard-value
!      ;; MS-Windows can access the clipboard even under -nw.
!      '((if (or (and window-system (eq system-type 'darwin))
! 	       (eq system-type 'windows-nt))
! 	   'mailclient-send-it
! 	 'sendmail-send-it)))
  
  ;; Useful to set in site-init.el
  ;;;###autoload
! (defcustom send-mail-function
!   (if (or (and window-system (eq system-type 'darwin))
! 	  ;; MS-Windows can access the clipboard even under -nw.
! 	  (eq system-type 'windows-nt))
!       'mailclient-send-it
!     'sendmail-send-it)
    "Function to call to send the current buffer as mail.
  The headers should be delimited by a line which is
  not a valid RFC822 header or continuation line,
--- 142,152 ----
  ;; when loaddefs.el is loaded. `custom-reevaluate-setting' needs the
  ;; standard value.
  ;;;###autoload
! (put 'send-mail-function 'standard-value 'sendmail-query-once)
  
  ;; Useful to set in site-init.el
  ;;;###autoload
! (defcustom send-mail-function 'sendmail-query-once
    "Function to call to send the current buffer as mail.
  The headers should be delimited by a line which is
  not a valid RFC822 header or continuation line,
***************
*** 164,169 ****
--- 154,160 ----
  This is used by the default mail-sending commands.  See also
  `message-send-mail-function' for use with the Message package."
    :type '(radio (function-item sendmail-send-it :tag "Use Sendmail package")
+ 		(function-item sendmail-query-once :tag "Query the user")
  		(function-item smtpmail-send-it :tag "Use SMTPmail package")
  		(function-item feedmail-send-it :tag "Use Feedmail package")
  		(function-item mailclient-send-it :tag "Use Mailclient package")
***************
*** 171,176 ****
--- 162,206 ----
    :initialize 'custom-initialize-delay
    :group 'sendmail)
  
+ (defvar sendmail-query-once-function 'query
+   "Either a function to send email, or the symbol `query'.")
+ 
+ (defun sendmail-query-once ()
+   "Send an email via `sendmail-query-once-function'.
+ If `sendmail-query-once-function' is `query', ask the user what
+ function to use, and then save that choice."
+   (when (equal sendmail-query-once-function 'query)
+     (let ((default
+ 	    (cond
+ 	     ((or (and window-system (eq system-type 'darwin))
+ 		  (eq system-type 'windows-nt))
+ 	      'mailclient-send-it)
+ 	     ((and sendmail-program
+ 		   (executable-find sendmail-program))
+ 	      'sendmail-send-it))))
+       (customize-save-variable
+        'sendmail-query-once-function
+        (if (or (not default)
+ 	       ;; We have detected no OS-level mail senders, or we
+ 	       ;; have already configured smtpmail, so we use the
+ 	       ;; internal SMTP service.
+ 	       smtpmail-smtp-server)
+ 	   'smtpmail-send-it
+ 	 ;; Query the user.
+ 	 (unwind-protect
+ 	     (progn
+ 	       (pop-to-buffer "*Mail Help*")
+ 	       (erase-buffer)
+ 	       (insert "Sending mail from Emacs hasn't been set up yet.\n\n"
+ 		       "Type `y' to configure outgoing SMTP, or `n' to use\n"
+ 		       "the default mail sender on your system.\n")
+ 	       (goto-char (point-min))
+ 	       (if (y-or-n-p "Configure outgoing SMTP in Emacs? ")
+ 		   'smtpmail-send-it
+ 		 default))
+ 	   (kill-buffer (current-buffer)))))))
+   (funcall sendmail-query-once-function))
+ 
  ;;;###autoload(custom-initialize-delay 'send-mail-function nil)
  
  ;;;###autoload


-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I've now implemented the query-the-user-and-then-save-the-default
> idea.

I've checked in the function.  To test it:

(setq send-mail-function 'sendmail-query-once)

Comments are welcome before changing the default.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Bill Wohler]

Eli Zaretskii <eliz@gnu.org> wrote:

> > From: Bill Wohler <wohler@newt.com>
> > Date: Mon, 27 Jun 2011 17:03:41 -0700
> > 
> > Eli Zaretskii <eliz@gnu.org> writes:
> > 
> > > That's the buffer through which we communicate with the SMTP server,
> > > right?  If so, AFAIK the whole protocol is pure ASCII.
> > 
> > Not if the binary extensions in RFC-3030 are in effect. However, the
> > gnu.org MX server eggs.gnu.org does not appear to have this extension
> > enabled. If you assume ASCII, you need to ensure that the mail server
> > never enables those options.
> 
> Are you talking about BINARYMIME?  If so, isn't the sender (i.e. Emacs
> and specifically smtpmail.el) supposed to request that extension?
> 
> If I'm right, and Emacs needs to request that feature, then this has
> no relevance on the issue at hand, because smtpmail doesn't request
> BINARYMIME and in fact doesn't even support it.
> 
> Or am I missing something?

I believe 8BITMIME would also be a problem too.

You are correct, as long as you control the MTA and don't request the
8BIT extensions, you'll be fine. Just good to be aware of that.

-- 
Bill Wohler <wohler@newt.com> aka <Bill.Wohler@nasa.gov>
http://www.newt.com/wohler/
GnuPG ID:610BD9AD

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>     (let* ((coding-system-for-read 'binary)
> 	   (coding-system-for-write 'binary)

I've now applied this to smtpmail.el.  It should be the right thing to
do in any case, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 26/06/11 14:08, Lars Magne Ingebrigtsen wrote:

> It's the same situation on Linux, really.  Today, if you want to send
> email on Linux, you will have configured exim to do the right thing.
> (Or it will be configured by somebody else for you.)

What? Hold on there. On fd.o desktop systems, one uses "xdg-email" to 
use the user's preferred email composer to pop up a mail ready for sending.

Some trouble(tm) arises if an advanced the user has actually gone and 
configured emacsclient as their preferred email composer.

> Defaulting to `smtpmail-send-it' will make Emacs behave exactly like all
> other mail clients: It'll ask you for the outgoing server name.

Pretty annoying as I'd have already configured that in my usual mail 
client that apps should be using, grr.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

David De La Harpe Golden <david@harpegolden.net> writes:

> What? Hold on there. On fd.o desktop systems, one uses "xdg-email" to
> use the user's preferred email composer to pop up a mail ready for
> sending.

So you configure xdg-email to have the correct outgoing SMTP address?

> Pretty annoying as I'd have already configured that in my usual mail
> client that apps should be using, grr.

Uhm.  If your email client is sending the email, then your email client
will continue to send the email.

This discussion is about sending email from within Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 26/06/11 22:57, Lars Magne Ingebrigtsen wrote:
> David De La Harpe Golden<david@harpegolden.net>  writes:
>
>> What? Hold on there. On fd.o desktop systems, one uses "xdg-email" to
>> use the user's preferred email composer to pop up a mail ready for
>> sending.
>
> So you configure xdg-email to have the correct outgoing SMTP address?
>

???

xdg-email is a command intended mostly for automated use that launches 
the user's preferred (in their desktop environment's settings) email 
client, such as thunderbird or evolution or kmail (or even emacs) with a 
message open and ready to be sent (possibly after further review and 
editing).

As it opens the desktop user's preferred email client, it's most likely 
to be actually already configured with correct outgoing settings.

http://portland.freedesktop.org/xdg-utils-1.0/xdg-email.html
http://portland.freedesktop.org/wiki/EmailConfig

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

David De La Harpe Golden <david@harpegolden.net> writes:

> xdg-email is a command intended mostly for automated use that launches
> the user's preferred (in their desktop environment's settings) email
> client, such as thunderbird or evolution or kmail (or even emacs) with
> a message open and ready to be sent (possibly after further review and
> editing).

Ok...  so what does that have to do with smtpmail.el or anything?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 27/06/11 00:26, Lars Magne Ingebrigtsen wrote:


> Ok...  so what does that have to do with smtpmail.el or anything?
>

You are talking about changing send-mail-function (see subject line).  A 
function that calls xdg-email to open an email for sending via the 
user's preferred email client on an x11 desktop is a possible value, 
with the advantage that said preferred email client is likely already 
configured to send somewhere useful.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

David De La Harpe Golden <david@harpegolden.net> writes:

> You are talking about changing send-mail-function (see subject line).
> A function that calls xdg-email to open an email for sending via the
> user's preferred email client on an x11 desktop is a possible value,
> with the advantage that said preferred email client is likely already
> configured to send somewhere useful.

I see.  That certainly sounds like an option.  Is there a way to query
xdg-email before calling it to find out whether you'll end up in an
email client that's already configured for outgoing SMTP?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 27/06/11 01:00, Lars Magne Ingebrigtsen wrote:
> David De La Harpe Golden<david@harpegolden.net>  writes:
>
>> You are talking about changing send-mail-function (see subject line).
>> A function that calls xdg-email to open an email for sending via the
>> user's preferred email client on an x11 desktop is a possible value,
>> with the advantage that said preferred email client is likely already
>> configured to send somewhere useful.
>
> I see.  That certainly sounds like an option.  Is there a way to query
> xdg-email before calling it to find out whether you'll end up in an
> email client that's already configured for outgoing SMTP?

Nope, not that I can see anyway.  It has some return codes, but they're 
gross "no mail client whatsoever" problems.  emacs could try for it, and 
if it fails totally (either missing completely or nonzero return) try 
something else, but there is still a possibility of successfully opening 
the mail in a mail client that isn't properly configured.

There are also users who use emacs _as_ their preferred email client in 
the xdg-email sense e.g. [1] (Come to think of it I could start doing 
so, not supporting people using thunderbird as part of my job anymore). 
  Then there is or was some risk of deadlock if emacsclient is used, and 
even if that is avoided, then it could have the peculiar/annoying result 
of opening in emacs again. Could set an env var when forking off 
xdg-email and test for it.

[1]
http://lists.gnu.org/archive/html/emacs-devel/2008-04/msg00606.html
From: 	Manoj Srivastava
Subject: 	Re: Default of send-mail-function
Date: 	Tue, 08 Apr 2008 06:40:57 -0500

Re: Changing the default for `send-mail-function'

[Jan Djärv]

Hi.

David De La Harpe Golden skrev 2011-06-27 01.55:
> On 27/06/11 00:26, Lars Magne Ingebrigtsen wrote:
>
>
>> Ok... so what does that have to do with smtpmail.el or anything?
>>
>
> You are talking about changing send-mail-function (see subject line). A
> function that calls xdg-email to open an email for sending via the user's
> preferred email client on an x11 desktop is a possible value, with the
> advantage that said preferred email client is likely already configured to
> send somewhere useful.

We have that option in report-emacs-bug. C-c m starts you preferred mailer. 
It uses xdg-email where available.

	Jan D.

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 27/06/11 06:59, Jan Djärv wrote:
> Hi.
>
> David De La Harpe Golden skrev 2011-06-27 01.55:
>> On 27/06/11 00:26, Lars Magne Ingebrigtsen wrote:
>>
>>
>>> Ok... so what does that have to do with smtpmail.el or anything?
>>>
>>
>> You are talking about changing send-mail-function (see subject line). A
>> function that calls xdg-email to open an email for sending via the user's
>> preferred email client on an x11 desktop is a possible value, with the
>> advantage that said preferred email client is likely already
>> configured to
>> send somewhere useful.
>
> We have that option in report-emacs-bug. C-c m starts you preferred
> mailer. It uses xdg-email where available.
>

Oh. Er. And there's an implementation for macosx too.  But not w32? 
Taking Eli's point about separating bug reporting needs from 
emacs-as-a-mail-client, can C-c m be made work on w32 (if I'm not 
misreading the code and it doesn't already)?  I don't have a w32 box to 
test, but I had a vague notion just start.exe will open the default mail 
client when given a mailto: url as its arg (might be misremembering), 
similar to the macosx "open" case (gnustep equiv is "openapp", though I 
don't know what it does when passed mailto: urls)

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> Date: Mon, 27 Jun 2011 08:12:52 +0100
> From: David De La Harpe Golden <david@harpegolden.net>
> Cc: emacs-devel@gnu.org
> 
> > We have that option in report-emacs-bug. C-c m starts you preferred
> > mailer. It uses xdg-email where available.
> >
> 
> Oh. Er. And there's an implementation for macosx too.  But not w32? 
> Taking Eli's point about separating bug reporting needs from 
> emacs-as-a-mail-client, can C-c m be made work on w32 (if I'm not 
> misreading the code and it doesn't already)?

It should simply call mailclient-send-it, or do the same, that's all.

> I don't have a w32 box to 
> test, but I had a vague notion just start.exe will open the default mail 
> client when given a mailto: url as its arg (might be misremembering), 
> similar to the macosx "open" case (gnustep equiv is "openapp", though I 
> don't know what it does when passed mailto: urls)

No need to invent the wheel, it is already there in
mailclient-send-it.

But if we want to solve the issue with report-emacs-bug this way
(which is fine with me), then I think C-c C-c should do this
automatically, if we detect that smtpmail is not configured.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    It's the same situation on Linux, really.

Linux has no email facilities -- it is just a kernel.  You must mean
the GNU/Linux system, which is likely to include Exim as well as GNU
Emacs.  Would you please call it "GNU/Linux", not "Linux"?

See http://www.gnu.org/gnu/why-gnu-linux.html for why this makes a difference.

    If you install Ubuntu on a new machine, it will install exim, but leave
    exim in local delivery mode only by default.  If you then send email
    from your brand new fresly installed Emacs, it will fail silently.  It
    will call `sendmail-send-it', and exim will take it, and deliver a
    bounce locally.  Which probably won't be seen by the user.

Now it's clear you're talking about GNU/Linux, because Ubuntu is a
nonfree variant of GNU/Linux.

Emacs now has `mail-interactive' default to t.  Why does `sendmail'
fail silently if it cannot deliver the message?  Why doesn't
`sendmail-send-it' detect the failure and report it?

Why won't the user see the bounce message?  If the bounce message is
delivered locally to the user, why won't Emacs mail readers show it?

Anyway, if things are as you describe, it seems to be asking for
trouble.  I think this needs to be fixed in `sendmail' -- not in each
app that might invole it.

For instance, when exim is configured for local delivery only, we
should make `sendmail' fail quickly for any address with an @ in it,
thus giving Emacs the chance to report the error properly.

One thing Emacs could do in response to such an error is offer
the user the chance to switch to smtpmail instead.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> Emacs now has `mail-interactive' default to t.  Why does `sendmail'
> fail silently if it cannot deliver the message?  Why doesn't
> `sendmail-send-it' detect the failure and report it?

Emacs forks off the "sendmail" program, which takes the message, and
feeds it to exim, which says "ok", and then generates a bounce message.

> Why won't the user see the bounce message?  If the bounce message is
> delivered locally to the user, why won't Emacs mail readers show it?

Most people don't get mail delivered to their local machine, so there's
no reason for them to look for mail locally.

> For instance, when exim is configured for local delivery only, we
> should make `sendmail' fail quickly for any address with an @ in it,
> thus giving Emacs the chance to report the error properly.

If you know of a way to make most of the common GNU/Linux distributions
work in this manner, I'm all ears.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Emacs forks off the "sendmail" program, which takes the message, and
    feeds it to exim, which says "ok", and then generates a bounce message.

This seems like a definite bug.  Fixing it will take time, but it
should be fixed.

    > Why won't the user see the bounce message?  If the bounce message is
    > delivered locally to the user, why won't Emacs mail readers show it?

    Most people don't get mail delivered to their local machine, so there's
    no reason for them to look for mail locally.

Emacs could look for it.

Sending mail could set up a timer to check in 15 seconds for a bounce
message.  If one arrives, and it indicates that there is no outgoing
mail service, Emacs could set up a buffer for sending that message
again, then invite you to configure smtpmail before resending it.

I think this is a perfect solution, since it will ask the user to
configure smtp only if that is really needed, and it won't bother
people who are happily sending their mail with sendmail.

A second step might be: instead of configuring Emacs to send via smtp,
help the user configure Exim to do it.  That way, scripts that want to
send a message will work too.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> Sending mail could set up a timer to check in 15 seconds for a bounce
> message.  If one arrives, and it indicates that there is no outgoing
> mail service, Emacs could set up a buffer for sending that message
> again, then invite you to configure smtpmail before resending it.

I do not know, in general, how to check for a bounce message on a system
that has not been configured for email reading.  If you're thinking of
just checking /var/mail/<user> -- yes, that will work for some systems,
but not systems where the spool is somewhere else.

It sounds very fragile, and fragility is what we don't want.

> A second step might be: instead of configuring Emacs to send via smtp,
> help the user configure Exim to do it.  That way, scripts that want to
> send a message will work too.

I think that's a bad idea.

Times have moved on.  All other modern mail readers have moved on from
trying to use the local MTA to send email out, because the world has
changed.  These days, it's way more likely that you need to talk to your
ISPs local servers directly, and you need to fill out some credentials
tied to your account.

This isn't practical on an MTA-based solution, which is why all modern
mail readers, on all architectures, have independently come to the
conclusion that they have to have the SMTP configuration inside the
application. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    I do not know, in general, how to check for a bounce message on a system
    that has not been configured for email reading.  If you're thinking of
    just checking /var/mail/<user> -- yes, that will work for some systems,
    but not systems where the spool is somewhere else.

Which GNU/Linux distros keep it somewhere else?
How many different places is it normally kept?
(I would guess, not more than 5.)

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Times have moved on.  All other modern mail readers have moved on from
    trying to use the local MTA to send email out, because the world has
    changed.  These days, it's way more likely that you need to talk to your
    ISPs local servers directly, and you need to fill out some credentials
    tied to your account.

If a client program can do that, why can't Exim do that?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Teemu Likonen]

* 2011-06-29T06:58:03-04:00 * Richard Stallman wrote:

>> Times have moved on. All other modern mail readers have moved on from
>> trying to use the local MTA to send email out, because the world has
>> changed. These days, it's way more likely that you need to talk to
>> your ISPs local servers directly, and you need to fill out some
>> credentials tied to your account.
>
> If a client program can do that, why can't Exim do that?

Exim can, but (1) root account is needed to configure that and (2) it's
convenient if user can do all his email configuration from his email
client. I think configuring Exim or Postfix or other MTA is too
low-level for a normal user.

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Wed, 29 Jun 2011 06:58:03 -0400 Richard Stallman <rms@gnu.org> wrote: 

RS>     Times have moved on.  All other modern mail readers have moved on from
RS>     trying to use the local MTA to send email out, because the world has
RS>     changed.  These days, it's way more likely that you need to talk to your
RS>     ISPs local servers directly, and you need to fill out some credentials
RS>     tied to your account.

RS> If a client program can do that, why can't Exim do that?

On a multiuser system, I don't think that would work.

Ted

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

>     Times have moved on.  All other modern mail readers have moved on from
>     trying to use the local MTA to send email out, because the world has
>     changed.  These days, it's way more likely that you need to talk to your
>     ISPs local servers directly, and you need to fill out some credentials
>     tied to your account.
>
> If a client program can do that, why can't Exim do that?

Because SMTP is slowly moving towards a regime of per-user
authentication.

That is, to send someone email these days, you often have to provide a
user name and a password.  If you're the only one sending mail from your
machine, it's theoretically possible to configure exim to provide these
credentials to the upstream MTA.  (At least I think it's possible -- I
haven't actually checked.)  If there are two people sending email from
your machine, that obviously isn't what you want to do, what with the
password then being shared and stuff.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Riley]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Richard Stallman <rms@gnu.org> writes:
>
>>     Times have moved on.  All other modern mail readers have moved on from
>>     trying to use the local MTA to send email out, because the world has
>>     changed.  These days, it's way more likely that you need to talk to your
>>     ISPs local servers directly, and you need to fill out some credentials
>>     tied to your account.
>>
>> If a client program can do that, why can't Exim do that?
>
> Because SMTP is slowly moving towards a regime of per-user
> authentication.
>
> That is, to send someone email these days, you often have to provide a
> user name and a password.  If you're the only one sending mail from your
> machine, it's theoretically possible to configure exim to provide these
> credentials to the upstream MTA.  (At least I think it's possible -- I
> haven't actually checked.)  If there are two people sending email from
> your machine, that obviously isn't what you want to do, what with the
> password then being shared and stuff.

Assuming I havent got completely lost here, exim4 "smart hosts" are a
doddle to configure. But only for one user. For more than one email
account/smtp smarthost there are a plethora of google hits some of which
might work - matching email and/or user to the corect authentication
details needed for that user/email specific smarthost. If thats not
relevant excuse the intrusion.

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Wed, Jun 29, 2011 at 11:42 PM, Richard Riley <rileyrg@googlemail.com> wrote:
> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> Richard Stallman <rms@gnu.org> writes:
>>
>>>     Times have moved on.  All other modern mail readers have moved on from
>>>     trying to use the local MTA to send email out, because the world has
>>>     changed.  These days, it's way more likely that you need to talk to your
>>>     ISPs local servers directly, and you need to fill out some credentials
>>>     tied to your account.
>>>
>>> If a client program can do that, why can't Exim do that?
>>
>> Because SMTP is slowly moving towards a regime of per-user
>> authentication.
>>
>> That is, to send someone email these days, you often have to provide a
>> user name and a password.  If you're the only one sending mail from your
>> machine, it's theoretically possible to configure exim to provide these
>> credentials to the upstream MTA.  (At least I think it's possible -- I
>> haven't actually checked.)  If there are two people sending email from
>> your machine, that obviously isn't what you want to do, what with the
>> password then being shared and stuff.
>
> Assuming I havent got completely lost here, exim4 "smart hosts" are a
> doddle to configure. But only for one user. For more than one email
> account/smtp smarthost there are a plethora of google hits some of which
> might work - matching email and/or user to the corect authentication
> details needed for that user/email specific smarthost. If thats not
> relevant excuse the intrusion.
>

I think this whole argument is still going around in circles and lots
of 'arguments are being put forward which appear to lack any real
facts or are theoretical musings about problems that may occur and are
not a good reflection of reality.

1. There are very few Linux distros that do not place their mail spool
in /var/mail. Those that I've seen have been deliberately configured
that way and it would be reasonable to assume that anyone who has
deliberately changed the default/standard settings has also configured
mail. However, testing for /var/mail is not sufficient to determine if
a local MTA has been configured/setup. I do believe there are tests
that could be done, but at best they would be heuristic in nature and
could not guarantee 100% accuracy. Whether they would be sufficient,
I'm not sure.

2. The whole multi-user, exim and smarthost thing is just a complete
misdirection and misunderstanding regarding mail server configuration.
You need an SMTP host to dispatch mail. That dispatching has nothing
to do with your mail address or your recipient address. If you have
multiple users on a system and provide an MTA on that system, that MTA
either has the necessary access to communicate with other mail servers
or it has sufficient authority to pass the mail onto another MTA for
dispatch (a smarthost).  If authentication is involved, it is only
involved at the local MTA. (it is theoretically possible for a local
MTA to be configured to relay mail to specific SMTP servers based on
the credentials of the sender, but in nearly 20 years of dealing with
mail servers, I've never ever seen this configuration).  The main
reason you don't have this sort of multiple smarthost configuration is
because the management overhead is too high compared to the benefits.
From a client perspective, the SMTP server which dispatches the
message is largely irrelevant.

3. The majority of ISPs use IP address to control access to their SMTP
server for their clients connecting from within their network (i.e.
DSL, modem users) and only use authenticated SMTP when the client
wants to send mail from outside the network. However, it is common to
configure your mail to use authenticated SMTP even when on the ISPs
network simply because of convenience. Many people these days use
laptops and multiple networks. Having your default SMTP configuration
setup using authentication means you will be able to use your ISPs
SMTP server even when not on their network and avoids the need to
maintain multiple profiels and manage switching of profiles depending
on the network you are using.

The major 'issue' I can see with this proposed change of default is
the possible impact to users who use emacs, but don't use it as a MUA
and therefore don't configure it for sending mail, who may want
to/need to submit a bug report using the report-emacs-bug
functionality. Lars' proposed solution of initially prompting for the
necessary information may be sufficient to address this issue - we
will need to look at it ans see if it needs refinement or not etc. It
is also possible that perhaps using mail to automate (or
semi-automate) lodging of bugs needs to be reviewed?

What I'm still not clear about is why this decision to change the
default to smtpmail? I've seen references to arguments concerning the
world marching on and this being more in line with how other software
does it etc, but that seems a little weak to me - there are lots of
things that have not changed for the better, so the world moving on is
a poor argument without more explination and there are plenty of
things other software does which emacs does not do and is a
significant reason why I use emacs and not that other software. This
is not to say I disagree with the proposal, but rather that it is not
clear why the change is required/desired. A better understanding of
that might help provide better suggestions for solutions that work for
more people.

Tim

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    1. There are very few Linux distros that do not place their mail spool
    in /var/mail.

There are none at all, because there are no Linux distros.  The
distros you are thinking of are GNU/Linux distros.  Please give us
credit for our work, by calling them "GNU/Linux distros".

    However, testing for /var/mail is not sufficient to determine if
    a local MTA has been configured/setup.

I believe you, but what I suggested wasn't testing for /var/mail.
It was testing for a bounce message in /var/mail.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Andreas Schwab]

Tim Cross <theophilusx@gmail.com> writes:

> 2. The whole multi-user, exim and smarthost thing is just a complete
> misdirection and misunderstanding regarding mail server configuration.
> You need an SMTP host to dispatch mail. That dispatching has nothing
> to do with your mail address or your recipient address.

Actually it does.  I use different SMTP relays depending on the mail
address I use.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Thu, 30 Jun 2011 18:34:44 +0200 Andreas Schwab <schwab@linux-m68k.org> wrote: 

AS> Tim Cross <theophilusx@gmail.com> writes:
>> 2. The whole multi-user, exim and smarthost thing is just a complete
>> misdirection and misunderstanding regarding mail server configuration.
>> You need an SMTP host to dispatch mail. That dispatching has nothing
>> to do with your mail address or your recipient address.

AS> Actually it does.  I use different SMTP relays depending on the mail
AS> address I use.

Seconded.  For me it also depends on the current Gnus group I'm
visiting.

Ted

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 26/06/11 14:08, Lars Magne Ingebrigtsen wrote:


> What about if we default to `query-user', and then the first time you
> use it, it'll ask "Use the built-in SMTP support, or use {the Windows
> mailer,exim} to send the mail?", and then save the response.
>

Note my point elsemail is essenially that similar can apply to x11 
desktop envs - i.e. the choice is between

(i) the desktop preferred mail client (implementable via the xdg-email 
command line command that exists to indirect to the user's preferred MUA 
i.e. thunderbird, kmail, etc), quite likely to be configured to send 
somewhere sensible (this is highly similar to the point Eli just made 
about how "At least on Windows, the system mailer (Outlook or whatever) 
is normally already configured,")

(ii) the local system's mail daemon (via the standard 
sendmail-compatible command line interface, as provided by exim, 
postfix, true sendmail, etc.), unfortunately nowadays often set up for 
local-only delivery if that.

(iii) builtin smtp.

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>> Emacs until now didn't require the user to provide the server address.
>> At least on Windows, the system mailer (Outlook or whatever) is
>> normally already configured, and many times by a person other than the
>> one who invoked Emacs.  The latter might not know the address or the
>> credentials, or both.  We don't want that to get in the way of an
>> Emacs user who wants to report a bug, for example.

Maybe we should set it not to smtpmail-send-it nor to
mailclient-send-it, but to a new function that prompts the user for
which method to use (mailclient/smtpmail/sendmail) and then saves the
result via customize.

Of course, maybe we can spice up that function to try and guess which
choice to make without prompting the user (e.g. maybe gconf already hods
the answer?), but at least the fallback should be to prompt the user
because in general there's no safe answer (mailclient won't work on most
of my machines since I rarely bother to configure another MUA).


        Stefan

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Maybe we should set it not to smtpmail-send-it nor to
> mailclient-send-it, but to a new function that prompts the user for
> which method to use (mailclient/smtpmail/sendmail) and then saves the
> result via customize.

Yes, but like I said, we would then, in the future, be unable to change
the default again, since all users would by then have a setting
explicitly in .emacs.

Which may be fine (and may be exactly what we want), but is something we
should consider the ramifications of.

> Of course, maybe we can spice up that function to try and guess which
> choice to make without prompting the user (e.g. maybe gconf already hods
> the answer?), but at least the fallback should be to prompt the user
> because in general there's no safe answer (mailclient won't work on most
> of my machines since I rarely bother to configure another MUA).

I think mailclient, in general, is a very bad default option to offer
the users.  It's fine for random programs (that don't have native SMTP
capabilities) to pop up a strange foreign mail program, but Emacs is
more akin to an OS in this area.

Emacs has a lot of mail (composition and reading) programs running
within it (Wanderlust, Gnus, mew, rmail, VM, etc), and smtpmail.el is
the obvious place to configure mail centrally in Emacs.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Sun, 26 Jun 2011 21:47:57 -0400 Stefan Monnier <monnier@iro.umontreal.ca> wrote: 

SM> Maybe we should set it not to smtpmail-send-it nor to
SM> mailclient-send-it, but to a new function that prompts the user for
SM> which method to use (mailclient/smtpmail/sendmail) and then saves the
SM> result via customize.

SM> Of course, maybe we can spice up that function to try and guess which
SM> choice to make without prompting the user (e.g. maybe gconf already hods
SM> the answer?), but at least the fallback should be to prompt the user
SM> because in general there's no safe answer (mailclient won't work on most
SM> of my machines since I rarely bother to configure another MUA).

There is no right answer for everyone nowadays; let's start with that
assumption and work backwards to "we need a good multi-step
configuration interface and outbound mail is the right place to use it".

Customize is not the right interface for the step-by-step configuration
we need to set up outbound mail.  Please consider assistant.el in Gnus;
it or something like it is IMHO the right way to solve this problem.  A
simple `y-or-n-p' sequence or Yet Another Defcustom is not.

Ted

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> We don't want that to get in the way of an Emacs user who wants to
> report a bug, for example.

Another random idea: If that's the main problem, then if the gnu.org MTA
would open a listening port on port 587, and accept mail to gnu.org on
that port, then we could just have the bug reports use
`smtpmail-send-it' to eggs.gnu.org:587 directly...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> From: Lars Magne Ingebrigtsen <larsi@gnus.org>
> Date: Sun, 26 Jun 2011 15:39:56 +0200
> 
> Another random idea: If that's the main problem, then if the gnu.org MTA
> would open a listening port on port 587, and accept mail to gnu.org on
> that port, then we could just have the bug reports use
> `smtpmail-send-it' to eggs.gnu.org:587 directly...

If that's possible, it's even better, yes.  However, I wonder if the
GNU sysadmins would really like the idea of millions of Emacs users
suddenly hitting their mail server...

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> If that's possible, it's even better, yes.  However, I wonder if the
> GNU sysadmins would really like the idea of millions of Emacs users
> suddenly hitting their mail server...

It would just be for sending bug reports to @gnu.org.  It wouldn't be an
open relay to the world.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 26/06/11 14:39, Lars Magne Ingebrigtsen wrote:

> Another random idea: If that's the main problem, then if the gnu.org MTA
> would open a listening port on port 587, and accept mail to gnu.org on
> that port, then we could just have the bug reports use
> `smtpmail-send-it' to eggs.gnu.org:587 directly...
>

AFAIK there _has_ to be auth on 587 per rfc. Yes, emacs could hardcode 
an easily discovered username/password pair (or client cert) and 
"authenticate" with them, but that's a bit of a dubious thing to do.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

David De La Harpe Golden <david@harpegolden.net> writes:

> AFAIK there _has_ to be auth on 587 per rfc.

If you're relaying; yes.  But if you're just accepting mail to local
users, there should be no need for AUTH.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[David De La Harpe Golden]

On 26/06/11 21:37, Lars Magne Ingebrigtsen wrote:
> David De La Harpe Golden<david@harpegolden.net>  writes:
>
>> AFAIK there _has_ to be auth on 587 per rfc.
>
> If you're relaying; yes.  But if you're just accepting mail to local
> users, there should be no need for AUTH.
>

http://tools.ietf.org/html/rfc5068

...

    Submission Authentication:

       MSAs MUST perform authentication on the identity asserted during
       all mail transactions on the SUBMISSION port, even for a message
       having a RCPT TO address that would not cause the message to be
       relayed outside of the local administrative domain.

...

Re: Changing the default for `send-mail-function'

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 3608 bytes --]

On Mon, Jun 27, 2011 at 6:37 AM, Lars Magne Ingebrigtsen <larsi@gnus.org>wrote:

> David De La Harpe Golden <david@harpegolden.net> writes:
>
> > AFAIK there _has_ to be auth on 587 per rfc.
>
> If you're relaying; yes.  But if you're just accepting mail to local
> users, there should be no need for AUTH.
>
> You would open up a terrible exposure to potential spamming! Anyone,
anywhere could post messages.

I'm not sure we are not just creating problems here. Perhaps we just need to
stop, think about how it use to work, think about how it should work and
then work out how to migrate from the old to the new.

There was nothing magic about the old default behavior as far as I am aware.
From memory, if I started emacs and sent a message, it used my local MTA. It
assumed that was configured and working. If there was no local MTA or if it
was misconfigured, then mail failed. It may fail silently or it may alert me
that delivery failed - that all depended on the level of configuration of
the local MTA.

As I understand it, the proposal is to change the default to use
smtpmail-send-it as the default. That is what I currently use (in fact, I
swapped to it because I got sick of my Linux distro upgrading my exim4 and
silently breaking its config - I still run a local version of exim4, which
is used by everything else, such as scripts etc. I also like to use my
organisations smtp server directly as it alerts me to issues with their mail
server, which has been a little unreliable. Using my local MTA tended to hid
such problems).

So, for me, the change in default will have no effect. However, I'm not sure
how it would work for those who are using a local MTA - especailly one which
is not configured to listen on port 25 i.e. just dispatches mail via calls
to sendmail or something similar. If smtpmail-send-it can be setup to
default to the local MTA and handle local MTAs which don't bind to
localhost:25, when no other configuration information is provided, then
wouldn't that change in default be invisible and therefore a non-issue? (I'm
ignoring the win32 issues as I don't run that platform and have no idea what
strangeness it does).

If the default can be changed, but the change has no 'real' impact on the
user, then its not a problem. However, if changing the default means the
user needs to do additional configuration, we need to minimise that pain to
help them migrate.

I think the change in default is more in lines with how other mail clients
handle it these days. In fact, I was surprised by the statement that Ubuntu
installs exim4 by default. I distinctly remember having to install exim4 on
both of the last two ubuntu systems I setup. On one, I don't think anything
was installed by default and on the other, I think it may have been postfix.
Most ISPs now block mail relaying, so any local MTA needs to be configured
as a smarthost anyway and as has been pointed out by others, most Linux mail
clients now require you enter the smtp server, port and (optionally) auth
info when configuring the client. So, I think the change is in the right
direction, but as it is changing a default, extra care needs to be taken to
ensure we don't screw over existing users who are use to the old default of
using the local MTA.

Many of the other suggestions in this thread have some merit, but I think
are largely a distraction from the main issue - how to change the default
with minimal impact on established user expectations. If we cannot make the
change largely 'invisible', then it needs more change
management/communication so that fewer people are caught by surprise.

Tim

[-- Attachment #2: Type: text/html, Size: 4252 bytes --]

Re: Changing the default for `send-mail-function'

[Eli Zaretskii]

> Date: Mon, 27 Jun 2011 14:20:11 +1000
> From: Tim Cross <theophilusx@gmail.com>
> 
> If smtpmail-send-it can be setup to
> default to the local MTA and handle local MTAs which don't bind to
> localhost:25

That might be possible on systems that normally have an MTA.  Windows
systems generally don't, they only have an MUA (or MUA+MSA, to be
precise) which is configured correctly in 99% of cases I saw.

> If the default can be changed, but the change has no 'real' impact on the
> user, then its not a problem. However, if changing the default means the
> user needs to do additional configuration, we need to minimise that pain to
> help them migrate.

On many Windows systems, the MUA is configured by some tech guy, or by
the user that follows written instructions to the letter.  In the
latter case, the instructions are only valid for whatever mailer is
installed, and won't help in configuring smtpmail, unless the user
understands at least a little about SMTP and its authentication
options.  People who don't normally use Emacs as their MUA might not
know all this and in fact could be oblivious to the possibility that
Emacs can serve as a MUA.  When such users want to report a bug, we
don't want to annoy them with configuring smtpmail.

> Many of the other suggestions in this thread have some merit, but I think
> are largely a distraction from the main issue - how to change the default
> with minimal impact on established user expectations.

I think we need to consider 2 use cases here:

 . using Emacs as MUA

 . reporting a bug via "M-x report-emacs-bug"

Both of these are important, but the considerations and the target
audiences are different.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

> I think we need to consider 2 use cases here:
>
>  . using Emacs as MUA
>
>  . reporting a bug via "M-x report-emacs-bug"
>
> Both of these are important, but the considerations and the target
> audiences are different.

Good point.

I think the easiest way to handle the second case is to have a
submission server on the gnu.org machines.  The easiest way to do that
would be to just open up the gnu.org MTA on port 587, and point a CNAME
to it, like bug-submission.gnu.org, and have `M-x report-emacs-bug' try
that first.  (The MTA can, for instance, accept email to
bug-gnu-emacs@gnu.org only.)

It'll work in most (almost all?) cases, and we can try to fall back on
mailclient/etc if it doesn't (if, for instance, the gnu.org MTA is down,
or no longer exists in the future).

This requires the support of the gnu.org mail admins, of course.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Mon, 27 Jun 2011 13:07:22 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> I think the easiest way to handle the second case is to have a
LMI> submission server on the gnu.org machines.  The easiest way to do that
LMI> would be to just open up the gnu.org MTA on port 587, and point a CNAME
LMI> to it, like bug-submission.gnu.org, and have `M-x report-emacs-bug' try
LMI> that first.  (The MTA can, for instance, accept email to
LMI> bug-gnu-emacs@gnu.org only.)

LMI> It'll work in most (almost all?) cases, and we can try to fall back on
LMI> mailclient/etc if it doesn't (if, for instance, the gnu.org MTA is down,
LMI> or no longer exists in the future).

LMI> This requires the support of the gnu.org mail admins, of course.  :-)

I think a web server submission is the right way to solve this problem.
An e-mail bug submission does not work for everyone as we've discussed.
It's still a good submission channel, but doesn't have to be the only one.

Emacs, through url.el, can submit the form and perform some trivial work
to assure the server it's not spam.  It could use hashcash, for
instance.

Ted

Re: Changing the default for `send-mail-function'

[Tim Cross]

[-- Attachment #1: Type: text/plain, Size: 2898 bytes --]

On Mon, Jun 27, 2011 at 9:07 PM, Lars Magne Ingebrigtsen <larsi@gnus.org>wrote:

> Eli Zaretskii <eliz@gnu.org> writes:
>
> > I think we need to consider 2 use cases here:
> >
> >  . using Emacs as MUA
> >
> >  . reporting a bug via "M-x report-emacs-bug"
> >
> > Both of these are important, but the considerations and the target
> > audiences are different.
>
> Good point.
>
> I think the easiest way to handle the second case is to have a
> submission server on the gnu.org machines.  The easiest way to do that
> would be to just open up the gnu.org MTA on port 587, and point a CNAME
> to it, like bug-submission.gnu.org, and have `M-x report-emacs-bug' try
> that first.  (The MTA can, for instance, accept email to
> bug-gnu-emacs@gnu.org only.)
>
> It'll work in most (almost all?) cases, and we can try to fall back on
> mailclient/etc if it doesn't (if, for instance, the gnu.org MTA is down,
> or no longer exists in the future).
>
> This requires the support of the gnu.org mail admins, of course.  :-)
>
> I think there are two weaknesses with this idea.

Firstly, what happens if the gnu-org mail server or software is down? One of
the nice things with mail is it has a little robustness built into it. If
you send a message to an address, but the smtp server for that address is
not responding, the MTA trying to deliver a message to that server will
re-queue the message and try again. It will do this for a number of times
before giving up and bouncing back the message. In many cases, it will even
notify the sender that the message has been delayed. This functionality
would either be lost or we would need to add some additional queueing/retry
facilities in the bug submission functionality.

Secondly, I would be very surprised if the gnu.org admins will be agreeable
to this idea. Essentially, you are talking about an open mail server (even
though it will only accept/deliver to a single address) and as such, a
server that would be open to abuse. To make matters worse, it is a setup
that would make many anti-spam techniques difficult/impossible to implement
because many of the messages would be coming form IPs without DNS entries,
IPs that ISPs often put in blacklists to protect against their systems being
abused by spammers, etc.

We need a good default value which does not depend/require any configuration
by the user to addresss the use case highlighted by Eli i.e. emacs users who
do not use emacs as a MUA and who will not/do not want to configure MUA
components just to be able to send a bug report (most likely won't even
realise this need until they want to submit the report).

For those who do use emacs  as a MUA, it is probably reasonable to expect
they are willing to do some configuration work to get it working and
requiring them to set mail server and possibly auth info is probably
reasonable - it is the first group that are more problematic.

Tim

[-- Attachment #2: Type: text/html, Size: 3875 bytes --]

Re: Changing the default for `send-mail-function'

[Glenn Morris]

>> I think the easiest way to handle the second case is to have a
>> submission server on the gnu.org machines.  The easiest way to do that
>> would be to just open up the gnu.org MTA on port 587, and point a CNAME
>> to it, like bug-submission.gnu.org, and have `M-x report-emacs-bug' try
>> that first.  (The MTA can, for instance, accept email to
>> bug-gnu-emacs@gnu.org only.)

To report an Emacs bug, you just need to be able to send an email.
It does not have to come from Emacs, you can use your normal mail client.
This is written in the manual.

The quoted issue above is only relevant if you believe that there are a
significant population using M-x report-emacs-bug whose reports do not
get sent correctly and who do not realize this has happened. I do not
believe this population exists. Of course, I cannot prove this, but
there have been over 1600 different people submitting Emacs bug reports
in the past 3 years (http://debbugs.gnu.org/rrd/emacs.html).

So personally I don't think it's worth spending time on this, and I
would not bother the GNU sysadmins with it. (It also doesn't really have
much to do with the subject IMO).

Re: Changing the default for `send-mail-function'

[Glenn Morris]

BTW, everyone who reports a bug gets an ack (unless they explicitly say
that they don't want one), so if you really think silently failing
reports are a problem, I suggest just popping up a dialogue box (subject
to X-Debbugs-No-Ack and report-emacs-bug-no-confirmation) after a bug
report is sent with C-c C-c that says something like "Thanks for your
report. After a moderation delay of up to a day or so, you should get an
acknowledgement at <email address that was in report's From:> and see
your report appear on http://debbugs.gnu.org. If not, send your report
to bug-gnu-emacs using your normal mail client."

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Glenn Morris <rgm@gnu.org> writes:

> BTW, everyone who reports a bug gets an ack (unless they explicitly say
> that they don't want one), so if you really think silently failing
> reports are a problem, I suggest just popping up a dialogue box (subject
> to X-Debbugs-No-Ack and report-emacs-bug-no-confirmation) after a bug
> report is sent with C-c C-c that says something like "Thanks for your
> report. After a moderation delay of up to a day or so, you should get an
> acknowledgement at <email address that was in report's From:> and see
> your report appear on http://debbugs.gnu.org. If not, send your report
> to bug-gnu-emacs using your normal mail client."

I added something like this to the report-emacs-bug preamble.  I think
that should suffice.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    swapped to it because I got sick of my Linux distro upgrading my exim4 and
    silently breaking its config

Please don't call it a "Linux distro".  These distributions are
variants of the GNU/Linux system, which is the GNU system with Linux
added.  I wrote GNU Emacs as part of developing the GNU system.  What
we do here is improving the GNU system.  Please call them "GNU/Linux
distros" so as to give us equal mention for the system that we
launched.

See http://www.gnu.org/gnu/linux-and-gnu.html, plus history in
http://www.gnu.org/gnu/the-gnu-project.html.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

> So, for me, the change in default will have no effect. However, I'm not sure
> how it would work for those who are using a local MTA - especailly one which
> is not configured to listen on port 25 i.e. just dispatches mail via calls
> to sendmail or something similar. If smtpmail-send-it can be setup to
> default to the local MTA and handle local MTAs which don't bind to
> localhost:25, when no other configuration information is provided, then
> wouldn't that change in default be invisible and therefore a non-issue? (I'm
> ignoring the win32 issues as I don't run that platform and have no idea what
> strangeness it does).

The main reason for the change is that using a misconfigured MTA leads
to silently dropped mail which is *really bad*.  Furthermore, it's
becoming every time more challenging to properly configure a local MTA,
so it's pretty safe to assume that if an MTA is available it's
probably misconfigured.

I.e. sendmail-send-it should only be used if the user asks for it
explicitly (yes, this is very sad).

> Many of the other suggestions in this thread have some merit, but I think
> are largely a distraction from the main issue - how to change the default
> with minimal impact on established user expectations. If we cannot make the
> change largely 'invisible', then it needs more change
> management/communication so that fewer people are caught by surprise.

Exactly, which is why I think we should use a new mailsetup-send-it
default which first checks if there's a local MTA, then asks the user if
Emacs should use it (i.e. if it's properly configured) and if not
whether to use smtpmail-send-it (and then potentially prompts to
configure it) or mailclient-send-it.


        Stefan

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    The main reason for the change is that using a misconfigured MTA leads
    to silently dropped mail which is *really bad*.

Why doesn't it deliver a bounce message?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[chad]

On Jun 30, 2011, at 5:18 PM, Richard Stallman wrote:

>    The main reason for the change is that using a misconfigured MTA leads
>    to silently dropped mail which is *really bad*.
> 
> Why doesn't it deliver a bounce message?

How would it do that?  It doesn't know about the user's (actual) external 
mail service, and the local system isn't configured to accept mail.

Back when I had reason to look at such systems regularly, it was not 
unusual to find hundreds of such bounces in root's mbox.

*Chad

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    > Why doesn't it deliver a bounce message?

    How would it do that?  It doesn't know about the user's (actual) external 
    mail service, and the local system isn't configured to accept mail.

Someone else said that the default is that Exim is configured to
deliver mail locally, but that's all.  If so, remote mail
would deliver a bounce message locally.

If Exim can't even deliver local mail, then why doesn't sendmail fail
and inform Emacs of this?

    Back when I had reason to look at such systems regularly, it was not 
    unusual to find hundreds of such bounces in root's mbox.

Apparently it could deliver bounces locally.  I think that
means the problem you are talking about did NOT occur,
and my solution would work.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>     The main reason for the change is that using a misconfigured MTA leads
>     to silently dropped mail which is *really bad*.
> Why doesn't it deliver a bounce message?

Where would it deliver it?


        Stefan

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    >     The main reason for the change is that using a misconfigured MTA leads
    >     to silently dropped mail which is *really bad*.
    > Why doesn't it deliver a bounce message?

    Where would it deliver it?

To your local mailbox, wouldn't it?  Isn't that the default for Exim
in GNU/Linux distros?

Are we talking about the case where Exim has not been configured and
it can only do local delivery?  Or are we talking about the case where
the sysadmin made a mistake in configuring it?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Frank Schmitt]

Richard Stallman <rms@gnu.org> writes:

>     >     The main reason for the change is that using a misconfigured MTA leads
>     >     to silently dropped mail which is *really bad*.
>     > Why doesn't it deliver a bounce message?
>
>     Where would it deliver it?
>
> To your local mailbox, wouldn't it?  Isn't that the default for Exim
> in GNU/Linux distros?

But no user looks into the local mailboxes any more as most modern mail user
agents like Thunderbird, Sylpheed, KMail, Evolution don't show the user
mail in the local mailboy by default.

-- 
Have you ever considered how much text can fit in eighty columns?  Given that a
signature typically contains up to four lines of text, this space allows you to
attach a tremendous amount of valuable information to your messages.  Seize the
opportunity and don't waste your signature on bullshit that nobody cares about.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    > To your local mailbox, wouldn't it?  Isn't that the default for Exim
    > in GNU/Linux distros?

    But no user looks into the local mailboxes any more as most modern mail user
    agents like Thunderbird, Sylpheed, KMail, Evolution don't show the user
    mail in the local mailboy by default.

My proposal is that Emacs should check whether a bounce message
arrives in the local mailbox.  For this purpose, it doesn't matter
whether the human user thinks of looking in that local mailbox.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Richard Stallman writes:

 > My proposal is that Emacs should check whether a bounce message
 > arrives in the local mailbox.  For this purpose, it doesn't matter
 > whether the human user thinks of looking in that local mailbox.

However, one case described earlier (which is a common case) is that
the bounce messages end up in root's mailbox (eg via the postmaster
alias).  In that case, Emacs won't be able to read the mail.

There are many other reasons why the bounces won't end up where the
user *can* find them, even if they (or Emacs) is looking for them.
They add up to a fairly high probability of failure to deliver,
assuming that the user has not configured their local MTA for Internet
mail, and only reads mail on another host.

This unreliability is unavoidable because mail is a store-and-forward
channel, lacking a mandatory mechanism for confirmation of receipt by
the addressee.  Neither the MTA nor the MUA can determine if the mail
has failed for quite a while (many days, in the case where the MTA
fails because the network connection is unavailable due to firewalling
and the like, and in the typical default configuration will keep
retrying in hopes that the link will come back up).

Of course there are many reasons to prefer mail for normal use by even
slightly experienced users.

But as Ted Z says, for the fallback channel, it's better to use a
direct connect protocol like HTTP, which can reliably give immediate
success or failure feedback.

Regards,
Steve

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    However, one case described earlier (which is a common case) is that
    the bounce messages end up in root's mailbox (eg via the postmaster
    alias).  In that case, Emacs won't be able to read the mail.

Indeed it won't, but what is the situation where this occurs?
What causes it?

    There are many other reasons why the bounces won't end up where the
    user *can* find them, even if they (or Emacs) is looking for them.

What are some?

There is no inherent reason why local mail delivery of bounces should
be hard, so I think this indicates a bug or bugs in some software.

    This unreliability is unavoidable because mail is a store-and-forward
    channel, lacking a mandatory mechanism for confirmation of receipt by
    the addressee.  Neither the MTA nor the MUA can determine if the mail
    has failed for quite a while

That's true but it is a different question, not germane to this issue.
For this purpose, we don't need to tell whether any real message
succeeded in reaching its addressees.  We only need to tell whether
the local MTA has been configured and is meant to be used.

There is no fundamental reason why it should be hard to determine that.

Here's another approach to determining that:

1. Send mail to a nonexistent local address.
2. See if a bounce message arrives in your local mailbox in a reasonable time.
   If it doesn't, the MTA is not configured enough to deliver bounces.
3. Send mail to a remote address with a nonexistent host.
4. Look at the bounce message.
   The kind of error should tell you whether remote mail is configured.

However, the clean solution is to establish a conventional way for
programs to determine whether a machine has an MTA that should be
used.  sendmail should fail in a specific recognizable way when there
is not one.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Richard Stallman writes:

 >     However, one case described earlier (which is a common case) is that
 >     the bounce messages end up in root's mailbox (eg via the postmaster
 >     alias).  In that case, Emacs won't be able to read the mail.
 > 
 > Indeed it won't, but what is the situation where this occurs?
 > What causes it?

and

 > However, the clean solution is to establish a conventional way for
 > programs to determine whether a machine has an MTA that should be
 > used.  sendmail should fail in a specific recognizable way when there
 > is not one.

Richard, you're not the only one who is busy.  I don't have time for
general discussion of failure modes of the mail protocol, with a view
to repairing sendmail.  It's too difficult.[1]  OTOH, several
proposals have been made for 99.9999% reliable[2] submission mechanisms,
whose protocols already exist and the mechanisms themselves are easy
to implement in Emacs, while requiring only a very small amount of
scripting at the other end.

Perhaps we can make progress if you permit me to ask you a question.
What's wrong with a two-protocol system?  I am thinking of one that
uses the novice mechanism or similar, that presumes that most users
will have Emacs configured to send mail reliably already, and starts
the submission process by asking:

    The preferred and most convenient method to communicate with the
    Emacs issue tracker is by email.  But if you don't already use
    Emacs to send mail, I can use another mechanism (HTTP) for the
    initial submission of this report.

    Would you like me to send by mail now?  (y/n)

etc, etc

If the user elects HTTP, prompt for a mail address where they can be
reached, and store it (maybe in user-mail-address, maybe in a
report-emacs-bug-specific variable, haven't thought about that).

Then if the bug report was sent, ask if the same method should be used
in the future for initial submissions.  In the confirmation message
make it clear that HTTP is *only* for initial submissions (as is M-x
report-emacs-bug itself).

It would be easy to implement using url.el and a very simple CGI at
gnu.org that simply pipes the already-formatted mail message into a
known-good MTA (ie, at gnu.org).

Note that there is no need to worry about future correspondence; that
can be done in the usual way via mailing lists and personal mail.  It
is only the *initial* submission which would use this mechanism.  This
completely sidesteps all of the issues with unconfigured mail systems,
because followup correspondence can be conducted via the user's usual
mail system.  (Note that the initial submission doesn't need to use
HTTP, although IMO Ted Z is right, that HTTP is most reliable for
non-technical reasons, specifically, the fact that if the firewall
won't pass an outgoing HTTP POST, there's no way to guess what port
might be open.  But the Emacs project could use an MTA on a
non-standard port at a gnu.org host that Emacs could connect to
directly, etc.)

How about it?  AFAICS Ted or maybe Lars would be very happy to
implement such a thing.

Footnotes: 
[1]  Nor would such a facility would be generally useful IMO.  Users
who want to send mail to other hosts using sendmail will notice it's
broken quickly, and get expert help quickly.  They want it fixed, not
to be told "don't do that".  Most programs that automate remote
communications do not use email for the purpose any more.  Email
*cannot* provide reliability[2] to programs, while other protocols can
and already do.  It makes no sense to start with a protocol that is
likely to fail, and fallback to a reliable one, in automated reporting.

[2]  In the sense of a very high probability of success, while
providing a nearly absolute guarantee that failure will be detected.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Perhaps we can make progress if you permit me to ask you a question.
    What's wrong with a two-protocol system?

I don't know whether something is wrong with that, because the
question is not entirely clear.

I have nothing against making Emacs able to use multiple
protocols for sending mail.  Having those capabilities
can't hurt anyone.  The question is how Emacs decides which
method to use.

I want to avoid asking users to configure SMTP on systems where
sendmail works.  Some of them won't know that sendmail works, or what
sendmail is, they just send mail and it works.  If Emacs tells them,
"Please configure SMTP to send mail," they won't know this is not
really necessary.  They may waste a lot of time looking for a way to
do that.  If Emacs asks them whether configuring SMTP is necessary,
they won't know the answer.

      I don't have time for
    general discussion of failure modes of the mail protocol, with a view
    to repairing sendmail.  It's too difficult.[1]

You don't have to discuss it, but that doesn't convince me that
making sendmail reliably fail when the MTA is not really set up
is too difficult.

Also, your footnote [1] suggests we are not talking about the same
question.  What I have in mind does not match what you say there.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Mon, Jul 4, 2011 at 9:39 PM, Richard Stallman <rms@gnu.org> wrote:
>    Perhaps we can make progress if you permit me to ask you a question.
>    What's wrong with a two-protocol system?
>
> I don't know whether something is wrong with that, because the
> question is not entirely clear.
>
> I have nothing against making Emacs able to use multiple
> protocols for sending mail.  Having those capabilities
> can't hurt anyone.  The question is how Emacs decides which
> method to use.
>
> I want to avoid asking users to configure SMTP on systems where
> sendmail works.  Some of them won't know that sendmail works, or what
> sendmail is, they just send mail and it works.  If Emacs tells them,
> "Please configure SMTP to send mail," they won't know this is not
> really necessary.  They may waste a lot of time looking for a way to
> do that.  If Emacs asks them whether configuring SMTP is necessary,
> they won't know the answer.
>
>      I don't have time for
>    general discussion of failure modes of the mail protocol, with a view
>    to repairing sendmail.  It's too difficult.[1]
>
> You don't have to discuss it, but that doesn't convince me that
> making sendmail reliably fail when the MTA is not really set up
> is too difficult.
>
> Also, your footnote [1] suggests we are not talking about the same
> question.  What I have in mind does not match what you say there.
>
> --
> Dr Richard Stallman
> President, Free Software Foundation
> 51 Franklin St
> Boston MA 02110
> USA
> www.fsf.org  www.gnu.org
> Skype: No way! That's nonfree (freedom-denying) software.
>  Use free telephony http://directory.fsf.org/category/tel/
>
>

Wow, amazing how something so simple can get so complex so quickly!

I think Stefan's suggestion is a good one for handling the situation
of being able to reliably submit a bug report for emacs users who do
not use emacs for mail and therefore may or may not have mail
configured (and may or may not know if it is).  However, it should
also be noted that in some situations, mail is a more reliable
submission method than http. The weakness of http is that the http
server must be up and running at the point where I submit the bug
report. With mail, I can actually send a message which most mail
servers will repeatedly attempt to deliver for up to (frequently)_ 5
days. So, if the gnu.org mail system is unavailable for a few hours or
possibly a couple of days, my message will still get through.

I think Richard's points are also very valid, but address a slightly
more general issue of email and emacs. It is probably more in line
with the original suggestion of changing the default behaviour for the
send-mail-function. This is a more complex issue. It would seem

     - Not all platforms are equal. For example, Eli pointed out that
win32 should probably use the existing/default    mail client as the
default. This seems reasonable and in-line with user expectations on
that platform.

     - Someone mentioned that there may be information defined by xdg
that could be useful in determining the appropriate default for
systems that support it. Probably worth further investigation

     - Lars has written an interactive 'assistant' function. This
seems like a good idea on many levels. It may need
enhancing/expansion, but seems to be flexible and has the potential to
meet the widest range of individual requirements/expectations.
However, it does not address Richard's point of users who may not be
aware of how the underlying mail mechanism works, that they already
have a functioning MTA or what the right answers is and who will
become frustrated and waste time trying to solve a non-problem

Part of the problem here seems to be a lack of any real facts
concerning the current situation. It has been argued that MTAs are
often misconfigured and that the 'modern' approach is to configure a
specific server (often with authentication information) so that mail
will work form multiple networks etc. Taken at face value, this seems
reasonable for users running emacs on a laptop. However, is this just
users running emacs on a GNU/Linux laptop or does it hold for all
platforms, including OSX and Win32? Does this represent a significant
number of users or is this only a minority group and are the majority
doing fine with the existing defaults? Do we see lots of bug reports
concerning failed bug submission as a result of MTA issues or do we
just not see bugs from this group because they fail to submit? Is
there any real evidence that a majority of GNU/Linux systems that have
an MTA installed are misconfigured or is this just speculation? Should
we be considering different defaults for GNU/Linux laptops compared to
desktops? Can we assume an MTA on a GNU/Linux desktop is more likely
to be in a working state than one on a laptop that may be connected to
a foreign network? What is the breakdown of emacs user platforms? Are
the majority of users on wind32, OSX, GNU/Linux or something else?

Whatever the answer is, I do tend to agree with Stefan and others that
having emacs try to determine if an MTA is available AND correctly
configured is something that would take a lot of work and is unlikely
to ever be that reliable. Of course, a lot depends on your
interpretation of 'correctly configured', but at the base level, it
would have to at least mean that it could dispatch a message to
another mail server which could directly or indirectly deliver the
message or at least bounce it back to the originating server (possibly
not to the originating sender). Given that a mail message can sit with
a server for up to 5 days before it is returned, testing for bounces
may not be feasible. It certainly isn't something that could be done
reliably in 'real-time'. However, it may be possible to extend Lars'
'assistant' idea with some heuristics that could be used to try and
make life easier for the  user. Perhaps what we should focus on is in
making it easier for the user to get things setup and less on
automation?

Tim

Re: Changing the default for `send-mail-function'

[Richard Stallman]

Since there are problems with having Emacs check at run time
whether there is a configured MTA, I propose doing it inside
`make install'.

It could run a program that tries sending messages to certain chosen
addresses, and detects the results.  This program could afford to wait
a certain amount of time for the results to show up, with no danger it
will be terminated by a user who does not realize it is still working
on the problem.

Since `make install' is run as root, it could look for a bounce
message in root's local inbox too.

Whatever it finds, it could report by creating a suitable file
that Emacs will check at run time.  If there is a local MTA,
Emacs would do what it does now.  If not, it would ask the user
to configure sending via SMTP, as has been proposed.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Jan Djärv]

Richard Stallman skrev 2011-07-05 14.45:
> Since there are problems with having Emacs check at run time
> whether there is a configured MTA, I propose doing it inside
> `make install'.
>
> It could run a program that tries sending messages to certain chosen
> addresses, and detects the results.  This program could afford to wait
> a certain amount of time for the results to show up, with no danger it
> will be terminated by a user who does not realize it is still working
> on the problem.
>
> Since `make install' is run as root, it could look for a bounce
> message in root's local inbox too.

FWIW, I almost never run make install as root for Emacs.

	Jan D.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    FWIW, I almost never run make install as root for Emacs.

That susprises me -- could you explain why not?

Anyway, the crucial case here is a multi-user machine where the person
using Emacs does not understand system maintenance and did not install
Emacs himself.  This is the case where there is probably a functioning
MTA plus users who use it without knowing what it is.  In that case,
`make install' was run as root.

Thus, when `make install' is run not as root, it would be ok
not to decide, and let Emacs ask the user whatr to do.

The other way Emacs gets installed is from a package, via apt-get
or something like it.  That always gets done as root.

Thus, we can assume this test is done running as root.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Antoine Levitt]

05/07/11 23:05, Richard Stallman
>     FWIW, I almost never run make install as root for Emacs.
>
> That susprises me -- could you explain why not?

For instance, I used to use school servers that use an antiquated
version of emacs, and I wanted the last one, so I just reinstalled emacs
+ the libs it needed in ~/new_root, with PREFIX=$HOME/new_root
./configure.

> Anyway, the crucial case here is a multi-user machine where the person
> using Emacs does not understand system maintenance and did not install
> Emacs himself.  This is the case where there is probably a functioning
> MTA plus users who use it without knowing what it is.  In that case,
> `make install' was run as root.
>
> Thus, when `make install' is run not as root, it would be ok
> not to decide, and let Emacs ask the user whatr to do.
>
> The other way Emacs gets installed is from a package, via apt-get
> or something like it.  That always gets done as root.
>
> Thus, we can assume this test is done running as root.

Except apt-get doesn't call make install, so it's a package maintainer
gotcha.

Re: Changing the default for `send-mail-function'

[chad]

On Jul 5, 2011, at 2:43 PM, Antoine Levitt wrote:
>> Thus, we can assume this test is done running as root.
> 
> Except apt-get doesn't call make install, so it's a package maintainer
> gotcha.

In fact, I will be surprised if 5% of the used emacs installations
involve `make install' being run as root on anything strongly
resembling the user's configuration, and absolutely shocked if it's as
high as 10%.

In the end, Richard has an idea he wants to try, that covers
relatively rare edge cases.  Multiple people have suggested that this
*might* not be a good idea, but the edge cases are so rare and varied
that no one will be persuaded by theoretical discussion.  We might as
well try it and see.

It would still probably be worth-while to try an http-based
bug-reporting system, if the relevant people have the free
cycles. This probably includes gnu.org maintainers, though, making
that possibility seem slight.

I hope this is helpful; my apologies if not.
*Chad

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

chad writes:

 > It would still probably be worth-while to try an http-based
 > bug-reporting system, if the relevant people have the free
 > cycles.

My cycles are no longer on offer, but I did do a quick estimate of
the work involved.  The changes required to report-emacs-bug are
probably under 30 lines including prompt and status report strings,
and the CGI will be about the same size, maybe somewhat longer if you
provide a "nothing to see here, folks" page to return to sightseers.
It's all straightforward scripting, too, no difficult algorithms to
tune.

 > This probably includes gnu.org maintainers,

True.  However, the CGI will be very simple, merely forwarding the
content of an HTTP POST to the system MTA verbatim.  Any web framework
will have a function for doing this, and if it's already enabled at
www.gnu.org, the security inspection should be very little work.
Note that a similar CGI would probably be useful to other debbugs
users, at low cost.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    In the end, Richard has an idea he wants to try, that covers
    relatively rare edge cases.

Multi-user machines that various people log in on are not "edge cases".

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[chad]

On Jul 6, 2011, at 7:33 AM, Richard Stallman wrote:

>    In the end, Richard has an idea he wants to try, that covers
>    relatively rare edge cases.
> 
> Multi-user machines that various people log in on are not "edge cases".

Maybe.  They are certainly vastly more rare than they were when I
first met you, ~20 years ago, at LCS. Since then, basically all of the
universities, companies, projects, and organizations that I know about
have moved such multi-user machines to niche legacy support - dial-up
systems and the like. Instead of multiple people sharing a large
machine, far more common is to have multiple machines sharing a user:
work machine, home machine, laptop, tablet, cell phone - all capable
of and regularly used for sending email.  In research and corporate
environments, individual users running programs on hundreds or
thousands of machines is far more common than individual machines
running hundreds of users.  I wouldn't be surprised if the gnu.org
login servers were in the top 100 by usage of such machines in the
world.

Once you get past the preponderance of more-machines-than-users, and
narrow it down to machines where emacs is supported, desired, and
installed, you're still going to find that most of them intentionally
disable local mail delivery; it provides little service and has a high
potential nuisance factor. Most installations both large and small
want to channel mail through centralized systems; most of their users
want exactly the same thing.

In the end, the cases where users log into multi-user machines to run
emacs and want to send mail but don't have either a valid user mail
configuration or a working system mail configuration are definitely,
in my perhaps flawed opinion, `edge cases'.

I hope that helps.
*Chad

Re: Changing the default for `send-mail-function'

[Richard Stallman]

      I wouldn't be surprised if the gnu.org
    login servers were in the top 100 by usage of such machines in the
    world.

If it is useful for the FSF, it is probably useful for many other
organizations.

You consider this kind of system a "niche legacy", so you will tend
not to notice them, but that doesn't mean they are few in number.
Each one has many users, and its sysadmins may not be Emacs users.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[chad]

On Jul 6, 2011, at 5:34 PM, Richard Stallman wrote:

>      I wouldn't be surprised if the gnu.org
>    login servers were in the top 100 by usage of such machines in the
>    world.
> 
> If it is useful for the FSF, it is probably useful for many other
> organizations.

I can think of very few organizations that have both the FSF's focus
on computing use and our lack of resources.

> You consider this kind of system a "niche legacy", so you will tend
> not to notice them, but that doesn't mean they are few in number.

That is certainly possible.  What I was trying to convey is simply this:
Such systems used to be common, but are far less so now.  A little
exploration tells me that at least a few tens of thousands of users of
such systems *to my direct knowledge* have moved entirely away 
from that model.  At this point, I am aware of more people that no
longer use such systems than use emacs at all.  Of course, my
personal awareness is a poor over-all metric.

> Each one has many users, and its sysadmins may not be Emacs users.

Indeed, which is why I was trying to suggest that we should simply try 
your experiment and see how it works.

I assume that you replied because you took umbrage at the phrase
`rare edge case'.  I meant no offense; covering edge cases is a 
valiant and necessary part of good software engineering. I replied
in detail because I thought that it might be helpful, since you so
famously ``appear to be well outside the solar system''.

Thanks,
*Chad

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    That is certainly possible.  What I was trying to convey is simply this:
    Such systems used to be common, but are far less so now.

I do not disagree with that.

							      A little
    exploration tells me that at least a few tens of thousands of users of
    such systems *to my direct knowledge* have moved entirely away 
    from that model.  At this point, I am aware of more people that no
    longer use such systems than use emacs at all.

I won't dispute your observations, but is the comparison of those two
quantities germane to the issue?

I'm ready to believe that most Emacs users need to use SMTP now if
they are to send mail.  I agree we should make this easy for them.

However, machines with MTAs that work remain an important case, even
if they are no longer the majority, and their users shouldn't find
that Emacs breaks, or asks them questions they don't know how to
answer, merely in order to do what formerly "just worked".

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Michael Welsh Duggan]

Richard Stallman <rms@gnu.org> writes:

[...]

> I'm ready to believe that most Emacs users need to use SMTP now if
> they are to send mail.  I agree we should make this easy for them.
>
> However, machines with MTAs that work remain an important case, even
> if they are no longer the majority, and their users shouldn't find
> that Emacs breaks, or asks them questions they don't know how to
> answer, merely in order to do what formerly "just worked".

I have a system with a working MTA.  When I updated to the latest emacs
and started gnus I got the following message:

  Sending mail from Emacs hasn't been set up yet.
  Type `y' to configure outgoing SMTP, or `n' to use
  the default mail sender on your system.

  To change this again at a later date, customize the
  `send-mail-function' variable.

  Configure outgoing SMTP in Emacs? 

I typed `n', and everything continued working with sendmail in the way I
was used to.  This may not be the case you are worried about, but I
thought I'd let you know that I, at least, was not inconvenienced by
this change.

-- 
Michael Welsh Duggan
(md5i@md5i.com)

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    I typed `n', and everything continued working with sendmail in the way I
    was used to.  This may not be the case you are worried about, but I
    thought I'd let you know that I, at least, was not inconvenienced by
    this change.

That's because you understood what the right answer was for you to give.
This question won't be a big hassle for anyone who understands what
to answer.



-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Thu, Jul 7, 2011 at 10:34 AM, Richard Stallman <rms@gnu.org> wrote:
>      I wouldn't be surprised if the gnu.org
>    login servers were in the top 100 by usage of such machines in the
>    world.
>
> If it is useful for the FSF, it is probably useful for many other
> organizations.
>
> You consider this kind of system a "niche legacy", so you will tend
> not to notice them, but that doesn't mean they are few in number.
> Each one has many users, and its sysadmins may not be Emacs users.
>
> --
> Dr Richard Stallman
> President, Free Software Foundation
> 51 Franklin St
> Boston MA 02110
> USA
> www.fsf.org  www.gnu.org
> Skype: No way! That's nonfree (freedom-denying) software.
>  Use free telephony http://directory.fsf.org/category/tel/
>
>

I'm not sure the multi-user issue is worth considering. Every
multi-user system I've ever seen has a working local MTA and I suspect
it is reasonable to assume this is the normal situation for multi-user
platforms due to the nature of the platform and the fact such systems
tend to make more extensive use of mail for monitoring and
administration purposes etc.  To what extent they represent a 'niche'
or edge-case etc is largely irrelevant to the question of changing the
default for the send-mail-function.

The whole question still seems to come down to two opposing opinions -
on one side, we have those who believe that the MAJORITY of local MTA
configurations are broken and therefore, we should move to a default
of smtpmail, plus those who feel this is the more modern approach
adopted by most other mail clients and one that tends to fit better
with a model of computing that focuses on mobility and connectivity
from multiple networks (with different firewalls, smtp servers and
authentication requirements etc)  versus an alternative perspective
who question this assumption and who are concerned that many people
who currently don't have to worry about issues associated with
configuring email will now be forced to and in some cases, this could
be difficult as such information may not be readily available - at the
very least, it is a change which could force people to do additional
tasks requiring additional information which they previously did not
have to do.

Changing default settings is something that needs to be supported by
evidence and not just opinion. So far, there does not seem to be any
evidence to justify the change, only opinions. I have not observed an
increase in bugs relating to mail configuration or requests for help
to configure mail or frustration regarding the current default.
Perhaps it is too difficult to acquire the necessary evidence?

I do think the mobile computing issue is possibly the most compelling
of the arguments. However, I'm  not sure what percentage of emacs
users are on GNU/Linux laptops and as I understand the proposal, the
default for Windows is going to stay with the mailclient version
(don't know about OSX). but I believe few OSX installations use a
local MTA anyway.

At any rate, without some facts, I doubt this can be resolved. We
really need to know some numbers. What is the percentage of emacs
users on GNU/Linux and how many are running on laptops? If the current
default needs change, why are we not seeing an increase in bug reports
or requests for help or complaints regarding the existing default?

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Tim Cross <theophilusx@gmail.com> writes:

> The whole question still seems to come down to two opposing opinions -
> on one side, we have those who believe that the MAJORITY of local MTA
> configurations are broken and therefore, we should move to a default
> of smtpmail, plus those who feel this is the more modern approach
> adopted by most other mail clients and one that tends to fit better
> with a model of computing that focuses on mobility and connectivity
> from multiple networks (with different firewalls, smtp servers and
> authentication requirements etc)  versus an alternative perspective
> who question this assumption and who are concerned that many people
> who currently don't have to worry about issues associated with
> configuring email will now be forced to and in some cases, this could
> be difficult as such information may not be readily available - at the
> very least, it is a change which could force people to do additional
> tasks requiring additional information which they previously did not
> have to do.

Even if that may be the longest sentence I've read outside of Kant, and
though the verb didn't end the paragraph, I think that's a very cogent
summary of the issues.

> At any rate, without some facts, I doubt this can be resolved. We
> really need to know some numbers.

I just have one data point -- I installed Ubuntu, and sending mail from
Emacs didn't work.

You'd think that would be pretty typical.  But perhaps I pushed the
wrong button somewhere?  I can't remember doing so, but somebody (else)
should just try to install a default Ubuntu on a machine and see what
happens and report back.

> I have not observed an increase in bugs relating to mail configuration
> or requests for help to configure mail or frustration regarding the
> current default.

Perhaps they weren't able to report the bug?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[chad]

[-- Attachment #1: Type: text/plain, Size: 741 bytes --]


On Jul 6, 2011, at 6:19 PM, Lars Magne Ingebrigtsen wrote:
> 
> I just have one data point -- I installed Ubuntu, and sending mail from
> Emacs didn't work.

This was also my experience, around February of this year.

>> I have not observed an increase in bugs relating to mail configuration
>> or requests for help to configure mail or frustration regarding the
>> current default.
> 
> Perhaps they weren't able to report the bug?  :-)

You laugh, but I saw this pretty frequently when I last looked;
root's mbox was filled with errors, bounces, and bug reports.  At 
the time, I was maintaining up-to-date emacs for a large distributed
infrastructure (MIT), so I cared when I found `lost' bug reports
stuck in never-read location.

*Chad

[-- Attachment #2: Type: text/html, Size: 1298 bytes --]

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    You laugh, but I saw this pretty frequently when I last looked;
    root's mbox was filled with errors, bounces, and bug reports.

I think we need to understand more about how that failure occurred.
How did those messages get into root's inbox?

Was there no MTA installed?

Was an MTA installed and not configured?
Was an MTA installed and configured wrong?
If so, which MTA was it?

If this was due to an MTA installed and not configured, we need to be
concerned about it, but we should also ask whether the distro in
question has fixed its problem since.

if it was due to an error in configuration, I think that was
the sysadmin's fault, so it isn't our responsibility to make
things work despite his mistake.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[chad]

On Jul 7, 2011, at 9:31 AM, Richard Stallman wrote:

>    You laugh, but I saw this pretty frequently when I last looked;
>    root's mbox was filled with errors, bounces, and bug reports.
> 
> I think we need to understand more about how that failure occurred.
> How did those messages get into root's inbox?
> 
> Was there no MTA installed?
> 
> Was an MTA installed and not configured?
> Was an MTA installed and configured wrong?
> If so, which MTA was it?

I've seen it with sendmail, postfix, and exim, but that was
several years ago.  I cannot provide more recent data on such
large installations, unfortunately.

For modern, individual installations, I can tell you that for
GNU/Linux systems like Ubuntu, there typically won't be an MTA
installed. If there is an MTA installed, I suspect that it won't
be configured usefully - I'm not sure that there's a clear line
between `not configured' and `configured wrong' in many of these
cases.  Distributed systems often either don't want local
delivery at all or want it for a tiny subset of cases, that might
include local services just usually do not include interactive
users.  I'm not sure if you'd consider this `configured wrong' or
not, but it does include the case where root@localhost would
work, but non-root-user would not.

It is entirely possible that I've become too pessimistic on this
front; all I can really usefully say at this point is this: My
Personal experience running from about 20 years ago to about 10 years
ago revealed a steadily increasing trend of local delivery problems,
errors, and gotchas.  The email environment since then has moved even
further along the trajectory that seemed to have caused these
problems, so I'm dubious (but not expert) about their chances of
having improved since then.

I hope that helps,
*Chad

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    > Was an MTA installed and not configured?
    > Was an MTA installed and configured wrong?
    > If so, which MTA was it?

    I've seen it with sendmail, postfix, and exim, but that was
    several years ago.  I cannot provide more recent data on such
    large installations, unfortunately.

On a multi-user machine, if mail is totally broken, the sysadmins
will get a bug report.  We can expect that mail on such machines
more or less works.

    For modern, individual installations, I can tell you that for
    GNU/Linux systems like Ubuntu, there typically won't be an MTA
    installed.

It should be easy to determine that no MTA is installed.

	       If there is an MTA installed, I suspect that it won't
    be configured usefully - I'm not sure that there's a clear line
    between `not configured' and `configured wrong' in many of these
    cases.

Even though you aren't sure there is a clear line,
we might find one if we look.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[chad]

[-- Attachment #1: Type: text/plain, Size: 913 bytes --]

On Jul 9, 2011, at 7:38 AM, Richard Stallman wrote:

> On a multi-user machine, if mail is totally broken, the sysadmins
> will get a bug report.  We can expect that mail on such machines
> more or less works.

On most of the multi-user machines of which I'm currently aware 
(again, many fewer than ~10 years ago), mail often `works', but 
local delivery for users does not exist and is not desired. That is, 
users can send mail, but do/can not receive it on the local machine.

> 	       If there is an MTA installed, I suspect that it won't
>    be configured usefully - I'm not sure that there's a clear line
>    between `not configured' and `configured wrong' in many of these
>    cases.
> 
> Even though you aren't sure there is a clear line,
> we might find one if we look.

Absolutely.  I hope my memories and recent experiences have been
at least somewhat helpful in figuring out where to look.

*Chad

[-- Attachment #2: Type: text/html, Size: 1668 bytes --]

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Thu, Jul 7, 2011 at 11:19 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Tim Cross <theophilusx@gmail.com> writes:
>
>> The whole question still seems to come down to two opposing opinions -
>> on one side, we have those who believe that the MAJORITY of local MTA
>> configurations are broken and therefore, we should move to a default
>> of smtpmail, plus those who feel this is the more modern approach
>> adopted by most other mail clients and one that tends to fit better
>> with a model of computing that focuses on mobility and connectivity
>> from multiple networks (with different firewalls, smtp servers and
>> authentication requirements etc)  versus an alternative perspective
>> who question this assumption and who are concerned that many people
>> who currently don't have to worry about issues associated with
>> configuring email will now be forced to and in some cases, this could
>> be difficult as such information may not be readily available - at the
>> very least, it is a change which could force people to do additional
>> tasks requiring additional information which they previously did not
>> have to do.
>
> Even if that may be the longest sentence I've read outside of Kant, and
> though the verb didn't end the paragraph, I think that's a very cogent
> summary of the issues.
>

Very generous and I apologise for the length/grammar. I'm posting from
gmail and due to the poor interface, tend not to edit as much as I
should. Not an excuse, just a statement of my laziness.

>> At any rate, without some facts, I doubt this can be resolved. We
>> really need to know some numbers.
>
> I just have one data point -- I installed Ubuntu, and sending mail from
> Emacs didn't work.
>
> You'd think that would be pretty typical.  But perhaps I pushed the
> wrong button somewhere?  I can't remember doing so, but somebody (else)
> should just try to install a default Ubuntu on a machine and see what
> happens and report back.

I have installed Ubuntu on one fresh system and upgraded an old system
in the last 12 months. The only issue I had was Ubuntu installed
postfix rather than exim. As part of the installation, I was asked a
couple of very simple questions and mail worked fine.

However, I recently helped someone install ubuntu (11.04) and did
notice that no local MTA was installed at all. I suspect a lot depends
on the set of packages  you install. It isn't at all intuitive either.
For example, the reason that one ubuntu install I did got a local MTA
and the other didn't seems to be down to the fact I have a RAID config
and an MTA is a recommended package for mdadm while the other system
had no RAID and there is no MTA listed in the ubuntu-desktop package
(at least, not at the top level - possibly packages in that
meta-package do require/recommend an MTA). Of course, this is just one
distribution out of many. I think this is partially why this is such a
difficult issue to sort out. This also means that we can see reports
from people installing the same distribution with varying outcomes.
Some get a local MTA and some do not. It has been suggested that even
if they do get an MTA, it will be misconfigured. This certainly hasn't
been my experience. When an MTA has been installed, the configuration
has been very straight-forward - no more difficult than setting up an
individual mail client. Then again, I don't run emacs on a laptop, so
I don't need authenticated SMTP support. My local MTA can use my ISPs
server as a smarthost and all is fine.
>
>> I have not observed an increase in bugs relating to mail configuration
>> or requests for help to configure mail or frustration regarding the
>> current default.
>
> Perhaps they weren't able to report the bug?  :-)
>

With only a little irony and a fair slice of humor, I can only smile and nod!

I would like to say that despite the lack of any clear progress and
the likely high level of frustration that many may be feeling over
this question, I do think you need to be commended for both the work
you have done to improve smtpmail and showing the courage to take on
the frequently touchy topic of changing default settings. It is
important that such questions are raised from time to time to ensure
emacs progresses and I think such discussions/debates are valuable and
worthwhile.

Tim

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    >> on one side, we have those who believe that the MAJORITY of local MTA
    >> configurations are broken and therefore, we should move to a default
    >> of smtpmail, plus those who feel this is the more modern approach
    >> adopted by most other mail clients and one that tends to fit better
    >> with a model of computing that focuses on mobility and connectivity
    >> from multiple networks (with different firewalls, smtp servers and
    >> authentication requirements etc)  versus an alternative perspective
    >> who question this assumption and who are concerned that many people
    >> who currently don't have to worry about issues associated with
    >> configuring email will now be forced to and in some cases, this could
    >> be difficult as such information may not be readily available

[I cut off some text at the beginning and end to make it fit this window.]

The description of the second group (which includes me) makes us seem
more opposed than we really are.

I am not at all opposed to making things easy for people for whom the
right solution is smtp.  What I disagree with is the implicit plan to
deprecate machines with an MTA.  We should support both groups of
users.  I am looking for ways to do that.

How many different MTAs are offered as options in the widely used
distros?  Maybe it is not many; maybe for each one we could find a way
to see if it really is configured by looking at files in /etc.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Richard Riley]

Tim Cross <theophilusx@gmail.com> writes:

> Changing default settings is something that needs to be supported by
> evidence and not just opinion. So far, there does not seem to be any
> evidence to justify the change, only opinions. I have not observed an
> increase in bugs relating to mail configuration or requests for help
> to configure mail or frustration regarding the current default.
> Perhaps it is too difficult to acquire the necessary evidence?

From old config files on the 'net or changes to the wiki, I have
received many Qs about mail setup : the wiki, for example, is a hotch
potch of conflicting approaches - many "almost correct". That in itself
is evidence enough of confusion. In the #emacs irc channel there is
frequently a "rolling of eyes" type response whenever someone asks emacs
mail questions... not fair necessarily but there you go. Gnus is, in
fairness, not for the feinthearted as it is. In short there are many
many many people who have issues with mail configs in emacs. Many of the
people having the issues are new to emacs and soon give up and move on -
they certainly are not competent enough or , and more importantly,
confident enough to raise bugs. So I feel There is an issue and it does
need tackling.

The reluctance to change legacy defaults is meant well I am sure, but
people who have used emacs long term and know it backwards can soon
figure out how to change their setup to the "old way" : *new users*
should not be handed difficult to use defaults because of the reluctance
of a tiny minority.

  

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Thu, Jul 7, 2011 at 5:14 PM, Richard Riley <rileyrg@googlemail.com> wrote:
> Tim Cross <theophilusx@gmail.com> writes:
>
>> Changing default settings is something that needs to be supported by
>> evidence and not just opinion. So far, there does not seem to be any
>> evidence to justify the change, only opinions. I have not observed an
>> increase in bugs relating to mail configuration or requests for help
>> to configure mail or frustration regarding the current default.
>> Perhaps it is too difficult to acquire the necessary evidence?
>
> From old config files on the 'net or changes to the wiki, I have
> received many Qs about mail setup : the wiki, for example, is a hotch
> potch of conflicting approaches - many "almost correct". That in itself
> is evidence enough of confusion. In the #emacs irc channel there is
> frequently a "rolling of eyes" type response whenever someone asks emacs
> mail questions... not fair necessarily but there you go. Gnus is, in
> fairness, not for the feinthearted as it is. In short there are many
> many many people who have issues with mail configs in emacs. Many of the
> people having the issues are new to emacs and soon give up and move on -
> they certainly are not competent enough or , and more importantly,
> confident enough to raise bugs. So I feel There is an issue and it does
> need tackling.
>
> The reluctance to change legacy defaults is meant well I am sure, but
> people who have used emacs long term and know it backwards can soon
> figure out how to change their setup to the "old way" : *new users*
> should not be handed difficult to use defaults because of the reluctance
> of a tiny minority.
>

We must see things differently. Yes, I do see questions regarding mail
setup, but rarely see questions that relate to the setting of the
send-mail-function. The questions and most of the information on the
wiki is about configuring gnus, mew, wonderlust or VM etc. Some of
these external mail client packages don't even use this variable.
Changing the default for this specific variable is unlikely to have
any relevance for these types of issues.

I think it is also unreasonable to characterize those who question the
change of a long standing default as a tiny minority who are resisting
changes to a difficult to use default. Such changes need to be
questioned and so far, the only 'resistance' has been to ask for steps
to be taken to minimize the impact on those who have not had an issue
and for whom mail has worked without any need for additional effort.
For these users, additional effort would be required and the change
would represent a more difficult to use default.  I support such
requests and believe efforts are being made to address them that would
not have occurred without questioning the change.

What I find is still unclear is whether those who will benefit will
outweigh those who will be disadvantaged. We know the change will not
affect windows users as I believe the default for that platform is not
changing. It probably won't affect OSX users either. Of the remaining
users, it will likely be beneficial to those running GNU Liinux on a
laptop and I suspect 50/50 for the rest. However, the reality is that
we don't really know - changing defaults when you don't really know
what its impact wil be or how many it will impact is never something
that should be taken lightly.

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Tim Cross <theophilusx@gmail.com> writes:

> We know the change will not affect windows users as I believe the
> default for that platform is not changing. It probably won't affect
> OSX users either.

No, the default changed for all architectures. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    No, the default changed for all architectures. 

You should not have changed it without adequately addressing
the problems I have told you about.



-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>     No, the default changed for all architectures. 
> You should not have changed it without adequately addressing
> the problems I have told you about.

I told him to change it a few days ago.  And I made this decision a long
time ago.


        Stefan

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Fri, Jul 8, 2011 at 1:17 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Tim Cross <theophilusx@gmail.com> writes:
>
>> We know the change will not affect windows users as I believe the
>> default for that platform is not changing. It probably won't affect
>> OSX users either.
>
> No, the default changed for all architectures.
>
> --
> (domestic pets only, the antidote for overdose, milk.)
>  bloggy blog http://lars.ingebrigtsen.no/
>

Oh, OK. I thought it was going to be configured on Windows systems to
use the user's default mail client. This was the previous default for
win32 wasn't it? If so, I don't see why it should be changed for that
platform. Those who want to use emacs for mail can change the default
to smtpmail and for those who do not, emacs will use the native
configured mail client the user is use to (i.e. when sending a bug
report or some other mail related action).

While I can understand the argument to change the default on systems
wherre it had previously been the local MTA because of misconfigured
MTAs and the need to support a mobile computing paradigm, I find it
much harder to see why the default needs to be changed on wind32,
where the default mail client is far less likely to be misconfigured
and more likely to be configured to support a mobile environment if
required. Changing the default also forces users who don't use emacs
as their mail client to now configure it as a client for infrequently
used actions, such as submitting a bug report and use a client
interface they may not be as comfortable with compared to the previous
experience of having the text of the bug report placed within the mail
client you were familiar with and being able to use mail related
functionality without requiring additional effort to configure.

If the only reason to change this default on this platform is to be
consistent with other platforms, I think that is a mistake.

There are lots of people who do not use emacs as their mail client. I
suspect this number has increased as the number of users have
increased that use imap/pop to manage their mail. When I've asked
users why they don't use emacs as their mail client, the frequent
response is because they don't like the 'freezes' that occur when
emacs is checking large remote pop/imap servers etc.

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Tim Cross <theophilusx@gmail.com> writes:

> I thought it was going to be configured on Windows systems to
> use the user's default mail client. This was the previous default for
> win32 wasn't it?

The problem of silently failing is much less of an issue on Windows,
since if you haven't configured your email client there, Windows will
guide you through that.

On the other hand, I think the user experience of sending mail using
`mailclient-send-it' is pretty awful.  It pops up a different mail
client, and then you have to click buttons in that to actually send the
email.

So while changing the default on Windows isn't that vital, I think it's
better (overall) for people to use the built-in SMTP on Windows, too.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Sun, Jul 10, 2011 at 10:27 PM, Lars Magne Ingebrigtsen
<larsi@gnus.org> wrote:
> Tim Cross <theophilusx@gmail.com> writes:
>
>> I thought it was going to be configured on Windows systems to
>> use the user's default mail client. This was the previous default for
>> win32 wasn't it?
>
> The problem of silently failing is much less of an issue on Windows,
> since if you haven't configured your email client there, Windows will
> guide you through that.
>
> On the other hand, I think the user experience of sending mail using
> `mailclient-send-it' is pretty awful.  It pops up a different mail
> client, and then you have to click buttons in that to actually send the
> email.
>
> So while changing the default on Windows isn't that vital, I think it's
> better (overall) for people to use the built-in SMTP on Windows, too.
>

Your argument appears to be from a standpoint of someone who uses
emacs as their mail client. I don't know if they are the ones we
should worry about as much as the occasional user who is more likely
to be affected by the default setting. Someone who wants to use emacs
as their client will probably customize the setup anyway. Functions
etc that make this easier, such as the one you have written are great
for this. However, it is that group who don't use emacs as their mail
client, who may not be that familiar with mail client configuration
and who really don't want to configure emacs for mail that the default
is really relevant for. These users just want mail to work for those
rare instances where emacs initiates an email message, such as bug
reports.

 Anyone on a win32 platform who does not use emacs for mail (and
therefore, is probably uninterested in configuring emacs for mail)
will have either the default mail client configured or will have
installed their preferred mail client and have that configured. These
users are more likely to find the behavior of emacs using the client
they are use to much more convenient for occasional mail messages
initiated from emacs than an emacs based client they are unfamiliar
with.

those who use emacs for their mail will have the incentive and
expectation to configure emacs and therefore less likely to be
affected by the change in default. They will benefit from the helper
function you have written or by any customize interface which helps
them customize their mail setup. These users are motivated to make it
wor.

Tim

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>> I thought it was going to be configured on Windows systems to
>> use the user's default mail client. This was the previous default for
>> win32 wasn't it?
> The problem of silently failing is much less of an issue on Windows,
> since if you haven't configured your email client there, Windows will
> guide you through that.

> On the other hand, I think the user experience of sending mail using
> `mailclient-send-it' is pretty awful.  It pops up a different mail
> client, and then you have to click buttons in that to actually send the
> email.

For one-time use of Emacs's mailer (e.g. for very occasional bug
reports), mailclient-send-it is really not a bad option.  We really
should offer it as an option in the interactive prompting.


        Stefan

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> For one-time use of Emacs's mailer (e.g. for very occasional bug
> reports), mailclient-send-it is really not a bad option.  We really
> should offer it as an option in the interactive prompting.

The `n' choice on Windows/OS X is the `mailclient-send-it' choice...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 7/11/2011 9:27 AM, Lars Magne Ingebrigtsen wrote:
> Stefan Monnier<monnier@iro.umontreal.ca>  writes:
>
>> For one-time use of Emacs's mailer (e.g. for very occasional bug
>> reports), mailclient-send-it is really not a bad option.  We really
>> should offer it as an option in the interactive prompting.
>
> The `n' choice on Windows/OS X is the `mailclient-send-it' choice...

Why limit it to Windows/OS X?  Shouldn't it be offered as an option on 
all systems?

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Ken Brown <kbrown@cornell.edu> writes:

> Why limit it to Windows/OS X?  Shouldn't it be offered as an option on
> all systems?

I guess it works on other systems, too?  If sendmail can't be found,
then perhaps defaulting to mailclient in the `n' case would be a good
default on all systems?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 7/11/2011 12:34 PM, Lars Magne Ingebrigtsen wrote:
> Ken Brown<kbrown@cornell.edu>  writes:
>
>> Why limit it to Windows/OS X?  Shouldn't it be offered as an option on
>> all systems?
>
> I guess it works on other systems, too?  If sendmail can't be found,
> then perhaps defaulting to mailclient in the `n' case would be a good
> default on all systems?

I think so.  It's an easy option for people who don't want to bother 
setting up SMTP and just want to send a quick email.  And I think it 
should work on many systems, since it's based on browse-url.

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 7/11/2011 12:34 PM, Lars Magne Ingebrigtsen wrote:
> Ken Brown<kbrown@cornell.edu>  writes:
>
>> Why limit it to Windows/OS X?  Shouldn't it be offered as an option on
>> all systems?
>
> I guess it works on other systems, too?  If sendmail can't be found,
> then perhaps defaulting to mailclient in the `n' case would be a good
> default on all systems?

I'm not sure if the feature freeze would prevent this change from being 
made, but I'm sending this as a reminder in case you've forgotten about it.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Ken Brown <kbrown@cornell.edu> writes:

>> I guess it works on other systems, too?  If sendmail can't be found,
>> then perhaps defaulting to mailclient in the `n' case would be a good
>> default on all systems?
>
> I'm not sure if the feature freeze would prevent this change from
> being made, but I'm sending this as a reminder in case you've
> forgotten about it.

I don't think anybody had any comments to make.  :-)

How common is it that non-Windows people have a client set up (reachable
via mailclient) that can send mail?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 8/2/2011 6:09 PM, Lars Magne Ingebrigtsen wrote:
> Ken Brown<kbrown@cornell.edu>  writes:
>
>>> I guess it works on other systems, too?  If sendmail can't be found,
>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>> default on all systems?
>>
>> I'm not sure if the feature freeze would prevent this change from
>> being made, but I'm sending this as a reminder in case you've
>> forgotten about it.
>
> I don't think anybody had any comments to make.  :-)
>
> How common is it that non-Windows people have a client set up (reachable
> via mailclient) that can send mail?

I do (under Cygwin).  If I understand mailclient correctly, all that's 
required is that browse-url be able to handle a mailto link.  So I would 
guess that a lot of people have a client set up whether they know it or not.

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>>> I guess it works on other systems, too?  If sendmail can't be found,
>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>> default on all systems?
>> I'm not sure if the feature freeze would prevent this change from
>> being made, but I'm sending this as a reminder in case you've
>> forgotten about it.

Thanks, yes it's something we want to do.

> I don't think anybody had any comments to make.  :-)
> How common is it that non-Windows people have a client set up (reachable
> via mailclient) that can send mail?

Should be the case on most systems nowadays, except for those users who
use Emacs as their MUA.


        Stefan

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 8/2/2011 9:20 PM, Stefan Monnier wrote:
>>>> I guess it works on other systems, too?  If sendmail can't be found,
>>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>>> default on all systems?
>>> I'm not sure if the feature freeze would prevent this change from
>>> being made, but I'm sending this as a reminder in case you've
>>> forgotten about it.
>
> Thanks, yes it's something we want to do.

Good.  There's still the question of exactly what sendmail-query-once 
should do.  I think it should query all users who haven't already set up 
smtpmail.  And, when it queries, mailclient should always be one of the 
choices, even if sendmail exists.  After all, there's no reason for 
emacs to assume that a user prefers sendmail to mailclient.  In fact, 
many users might not even be aware that mailclient is an option.

Ken

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>>>>> I guess it works on other systems, too?  If sendmail can't be found,
>>>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>>>> default on all systems?
>>>> I'm not sure if the feature freeze would prevent this change from
>>>> being made, but I'm sending this as a reminder in case you've
>>>> forgotten about it.
>> Thanks, yes it's something we want to do.
> Good.  There's still the question of exactly what sendmail-query-once should
> do.  I think it should query all users who haven't already set up smtpmail.

I don't understand what you mean.  Clearly it should only query if
sendmail-query-once-function is set to `query', otherwise it would keep
querying forever those users who don't want to use&setup smtpmail.

> And, when it queries, mailclient should always be one of the choices, even
> if sendmail exists.

Yes, of course.  After all, the whole issue that prompted this querying
is that the presence of sendmail is not sufficient to guarantee that
sendmail can be used.


        Stefan

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 8/4/2011 4:15 PM, Stefan Monnier wrote:
>>>>>> I guess it works on other systems, too?  If sendmail can't be found,
>>>>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>>>>> default on all systems?
>>>>> I'm not sure if the feature freeze would prevent this change from
>>>>> being made, but I'm sending this as a reminder in case you've
>>>>> forgotten about it.
>>> Thanks, yes it's something we want to do.
>> Good.  There's still the question of exactly what sendmail-query-once should
>> do.  I think it should query all users who haven't already set up smtpmail.
>
> I don't understand what you mean.  Clearly it should only query if
> sendmail-query-once-function is set to `query', otherwise it would keep
> querying forever those users who don't want to use&setup smtpmail.

Sorry, I wasn't clear.  I meant to refer only to the case where 
sendmail-query-once-function is set to `query'.  The existing code does 
not always query in that situation, even though the documentation says 
it does.

>> And, when it queries, mailclient should always be one of the choices, even
>> if sendmail exists.
>
> Yes, of course.  After all, the whole issue that prompted this querying
> is that the presence of sendmail is not sufficient to guarantee that
> sendmail can be used.

I'm glad you agree.  Lars seemed to have something else in mind.  (See 
the quote from him at the top of this mail.)  He proposed only offering 
emacsclient as an option on systems that don't have sendmail.

Ken

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Fri, Aug 5, 2011 at 7:57 AM, Ken Brown <kbrown@cornell.edu> wrote:
> On 8/4/2011 4:15 PM, Stefan Monnier wrote:
>>>>>>>
>>>>>>> I guess it works on other systems, too?  If sendmail can't be found,
>>>>>>> then perhaps defaulting to mailclient in the `n' case would be a good
>>>>>>> default on all systems?
>>>>>>
>>>>>> I'm not sure if the feature freeze would prevent this change from
>>>>>> being made, but I'm sending this as a reminder in case you've
>>>>>> forgotten about it.
>>>>
>>>> Thanks, yes it's something we want to do.
>>>
>>> Good.  There's still the question of exactly what sendmail-query-once
>>> should
>>> do.  I think it should query all users who haven't already set up
>>> smtpmail.
>>
>> I don't understand what you mean.  Clearly it should only query if
>> sendmail-query-once-function is set to `query', otherwise it would keep
>> querying forever those users who don't want to use&setup smtpmail.
>
> Sorry, I wasn't clear.  I meant to refer only to the case where
> sendmail-query-once-function is set to `query'.  The existing code does not
> always query in that situation, even though the documentation says it does.
>
>>> And, when it queries, mailclient should always be one of the choices,
>>> even
>>> if sendmail exists.
>>
>> Yes, of course.  After all, the whole issue that prompted this querying
>> is that the presence of sendmail is not sufficient to guarantee that
>> sendmail can be used.
>
> I'm glad you agree.  Lars seemed to have something else in mind.  (See the
> quote from him at the top of this mail.)  He proposed only offering
> emacsclient as an option on systems that don't have sendmail.
>

This all still feels like over engineering to me. I wonder if we
really need to re-think and re-evaluate some of the assumptions. After
re-reading many of the posts here, I now suspect that the default
should really be mailclient and not smtpmail or sendmail.

- Sendmail is a good choice for people who have a working and
configured sendmail and are not mobile i.e. can either access the net
directly or a smart relay host all the time.

- Mailclient seems appropriate for those who don't use emacs for mail
by choice at all and prefer some other client. I suspect, but it is
only a guess, there are a lot of emacs users out there who do not use
emacs for mail - especially with the growth in web based clients, such
as gmail etc. Nearly everyone on windows would also have a separate
client configured and I suspect it is similar for mac. This is likely
the largest user base.

- smtpmail. This is probably a large chunk of those who do use emacs
as their mail client. A lot of users don't want to bother with the
configuration of a local MTA and prefer to leave that up to someone
else. This paradigm also works better for mobile devices as you can
have authenticated connections that will work regardless of the
network you are currently connected to.

There are numerous other possible configurations, but I suspect they
represent edge cases compared to the three main ones above. These
days, the mailclient and smtpmail users are probably the biggest
numbers.

I would expect that anyone who wants to run emacs as their mail client
is comfortable with configuring it - most likely using smtpmail. They
also probably do this quite early in their install/setup.

Anyone who does not use emacs as their mail client probably gets
frustrated at being forced to configure their mail settings just to do
something like send a bug report. They would probably be more
comfortable and happier with emacs using their normal client,
especially if emacs is able to open the other mail client and setup
the draft message and appropriate recipient headers etc.

I suspect the suggested default of smtpmail is being driven from a
perspective that sees emacs as the mail client for a majority of
users. I'm unconvinced this is actually the case. Looking at mail
header information in emacs related lists certainly gives the
impression that many emacs users do not use emacs as their mail
client.

1. Set the default to mailclient. This will address the concerns
raised regarding misconfigured local MTAs and will work fine for
windows users and probably mac users. It could also work for many GNU
Liinux users if we can honor things like xdg onfig info.

2. Provide a helper function which makes it easier to select and
configure smtpmail. This will help those who want to use smtpmail. On
GNU Linux, we could perhaps default to this setting if we were unable
to identify appropriate mailclient settings.

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Ken Brown <kbrown@cornell.edu> writes:

> Sorry, I wasn't clear.  I meant to refer only to the case where
> sendmail-query-once-function is set to `query'.  The existing code
> does not always query in that situation, even though the documentation
> says it does.

If Emacs can't find sendmail at all, it won't offer that as an option,
no. 

> I'm glad you agree.  Lars seemed to have something else in mind.  (See
> the quote from him at the top of this mail.)  He proposed only
> offering emacsclient as an option on systems that don't have sendmail.

"mailclient" I guess.  :-)

If we are to offer mailclient as an option also on non-Windows systems,
I have no objection to changing the question to something like

Do you want to use the built-in SMTP, the system mailer or a different
user mailer?  (b, s, d)

or something.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

> Do you want to use the built-in SMTP, the system mailer or a different
> user mailer?  (b, s, d)

That would be perfect, although getting a single-line prompt that's
understandable by random users is a challenge.


        Stefan

Re: Changing the default for `send-mail-function'

[Lars Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That would be perfect, although getting a single-line prompt that's
> understandable by random users is a challenge.

Which reminds me.

emacsbug itself has a lot of internal mail stuff.  See
`report-emacs-bug-can-use-xdg-email',
`report-emacs-bug-can-use-osx-open' and friends.  Shouldn't this be
factored out to allow Emacs to send mail via xdg/"open" in non-bug
cases, too?

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 8/6/2011 1:27 PM, Stefan Monnier wrote:
>> Do you want to use the built-in SMTP, the system mailer or a different
>> user mailer?  (b, s, d)
>
> That would be perfect, although getting a single-line prompt that's
> understandable by random users is a challenge.

I would go one step further and actually offer mailclient as the 
default, with an option to choose something different if desired.  After 
all, we're targeting users who are trying to use emacs to send mail but 
have not configured it.  I would guess that the most common case for 
this is sending a bug report.  Mailclient has the advantage that it 
should just work OOTB for most people, without any configuration.

But it's not clear what's the best way to refer to mailclient in the 
prompt.  How about "your default mailer", perhaps with some help text 
that explains that it is the same mailer the user would get by clicking 
on an email link in their web browser?

Ken

Re: Changing the default for `send-mail-function'

[Lennart Borgman]

On Sat, Aug 6, 2011 at 20:33, Ken Brown <kbrown@cornell.edu> wrote:
> On 8/6/2011 1:27 PM, Stefan Monnier wrote:
>>>
>>> Do you want to use the built-in SMTP, the system mailer or a different
>>> user mailer?  (b, s, d)
>>
>> That would be perfect, although getting a single-line prompt that's
>> understandable by random users is a challenge.
>
> I would go one step further and actually offer mailclient as the default,
> with an option to choose something different if desired.  After all, we're
> targeting users who are trying to use emacs to send mail but have not
> configured it.  I would guess that the most common case for this is sending
> a bug report.  Mailclient has the advantage that it should just work OOTB
> for most people, without any configuration.
>
> But it's not clear what's the best way to refer to mailclient in the prompt.
>  How about "your default mailer", perhaps with some help text that explains
> that it is the same mailer the user would get by clicking on an email link
> in their web browser?

I agree.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Ken Brown <kbrown@cornell.edu> writes:

> After all, we're targeting users who are trying to use emacs to send
> mail but have not configured it.  I would guess that the most common
> case for this is sending a bug report.

No, I think the most common case is people sending email from Emacs 24
for the first time, and who want to continue doing that.

> Mailclient has the advantage that it should just work OOTB for most
> people, without any configuration.

Well, I'm not sure how true that is.  (The "most people" part...)  I
certainly don't have any other mail client set up reachable by
mailclient.el.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Lennart Borgman]

On Sun, Aug 7, 2011 at 01:20, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Ken Brown <kbrown@cornell.edu> writes:
>
>> After all, we're targeting users who are trying to use emacs to send
>> mail but have not configured it.  I would guess that the most common
>> case for this is sending a bug report.
>
> No, I think the most common case is people sending email from Emacs 24
> for the first time, and who want to continue doing that.

I doubt that, but it is not very important. Why not distinguish those
cases, i.e. bug reports and other mail messages from Emacs and treat
them differently in this respect?

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Sun, Aug 7, 2011 at 9:20 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Ken Brown <kbrown@cornell.edu> writes:
>
>> After all, we're targeting users who are trying to use emacs to send
>> mail but have not configured it.  I would guess that the most common
>> case for this is sending a bug report.
>
> No, I think the most common case is people sending email from Emacs 24
> for the first time, and who want to continue doing that.
>

You are talking outside of report-emacs-bug yes? If so, I would agree
that most people who try to send mail other than a bug report are
likely to want to continue sending mail via emacs, if only for a short
period to try it out. However, what proportion of all emacs users
these users are is unknown.

Ken's comment to perhaps treat emacs-bug-report completely separately
is probably correct. Have it default to mailclient if no other mail
settings have been made. What we want to avoid is the user having to
go through any significant additional configuration steps to send a
bug report. For other areas, they are configuring mail, so asking them
to do configuration work/decisions would be reasonable and if they
never want to use emacs for mail, they will never get bothered.

Whatever the default selected, it needs to add value to the largest
group of users possible.

>> Mailclient has the advantage that it should just work OOTB for most
>> people, without any configuration.
>
> Well, I'm not sure how true that is.  (The "most people" part...)  I
> certainly don't have any other mail client set up reachable by
> mailclient.el.  :-)
>

What platform are you on and what percentage of the user base do you
think it would represent?

This is the root of the problem. We all have different opinions about
the user base, the most common case etc. To really get the right
default for this setting, we need a better understanding of the user
base. We need to know what really is the most common user pattern and
need to stop guessing based on our own perceptions.

This is hard to do. One possible metric we could try would be to
analyze the mail headers in some of the emacs related mail lists, such
as this one. We would not want to use the bug list as it will likely
be somewhat biased towards emacs clients due to report-emacs-bug.
However, loooking at the others should give some indication of users
who use emacs for general mail and those who don't.

Another alternative would be to try and run a survey. However, it is
quite difficult to structure good surveys, hard to get people to
participate and when you do, those that do answer are rarely a true
cross section of users.

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Tim Cross <theophilusx@gmail.com> writes:

> Whatever the default selected, it needs to add value to the largest
> group of users possible.

On the other hand, we are in a feature freeze.  Emacs has never
defaulted to using mailclient.el for non-Windows users.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Sun, Aug 7, 2011 at 10:28 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Tim Cross <theophilusx@gmail.com> writes:
>
>> Whatever the default selected, it needs to add value to the largest
>> group of users possible.
>
> On the other hand, we are in a feature freeze.  Emacs has never
> defaulted to using mailclient.el for non-Windows users.
>

If the feature freeze is that critical, I would argue that this
default should not be changed from its default as set in the last
release.

Changing a default is something that requires considerable evaluation,
refinement and ideally, consensus.   If we are to be constrained in
finding the optimum new default settings simply due to a somewhat
arbitrary feature freeze date, we should leave changing this default
until the next release or feature development phase.

Changing of any default setting is not something that should be
'rushed through' simply because of a decision to have a feature
freeze.  If in doubt, we should be conservative and stick with the
default used in the last release and re-visit the new default
proposals in the next development cycle.

Tim

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> That would be perfect, although getting a single-line prompt that's
> understandable by random users is a challenge.

We're displaying a help screen, so we can blather on at length there and
keep the prompt shortish...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>> That would be perfect, although getting a single-line prompt that's
>> understandable by random users is a challenge.
> We're displaying a help screen, so we can blather on at length there and
> keep the prompt shortish...

Great.


        Stefan

Re: Changing the default for `send-mail-function'

[Ken Brown]

On 8/7/2011 12:25 PM, Stefan Monnier wrote:
>>> That would be perfect, although getting a single-line prompt that's
>>> understandable by random users is a challenge.
>> We're displaying a help screen, so we can blather on at length there and
>> keep the prompt shortish...
>
> Great.

This seems to have been forgotten about again.  Windows and Mac are 
still the only platforms on which users are offered the choice of using 
mailclient-send-it if they haven't configured mail.

Are there plans to implement this soon, or should I file a bug report to 
make sure it doesn't get forgotten?

Ken

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Ken Brown <kbrown@cornell.edu> writes:

> This seems to have been forgotten about again.  Windows and Mac are
> still the only platforms on which users are offered the choice of
> using mailclient-send-it if they haven't configured mail.

`mailclient-send-it' has never been a possible default on Linux systems,
so I think it's quite likely to break in unexpected ways.  (I mean,
since it probably hasn't gotten much usage.)

So I think that, at this point, it's probably counter-productive to add
it as a choice.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

>> This seems to have been forgotten about again.  Windows and Mac are
>> still the only platforms on which users are offered the choice of
>> using mailclient-send-it if they haven't configured mail.
> `mailclient-send-it' has never been a possible default on Linux systems,

Hmm....

(defun message-send-mail-function ()
  "Return suitable value for the variable `message-send-mail-function'."
  (cond ((and (require 'sendmail)
	      (boundp 'sendmail-program)
	      sendmail-program
	      (executable-find sendmail-program))
	 'message-send-mail-with-sendmail)
	((and (locate-library "smtpmail")
	      (boundp 'smtpmail-default-smtp-server)
	      smtpmail-default-smtp-server)
	 'message-smtpmail-send-it)
	((locate-library "mailclient")
	 'message-send-mail-with-mailclient)
	(t
	 (error "Don't know how to send mail.  Please customize `message-send-mail-function'"))))

Maybe you think it's not a possible default, but message.el does use it
by default in some cases, without paying attention to the host OS.
And I think that's right.

> so I think it's quite likely to break in unexpected ways.  (I mean,
> since it probably hasn't gotten much usage.)

In my experience it works, and if it doesn't we should fix it.

> So I think that, at this point, it's probably counter-productive to add
> it as a choice.

I disagree,


        Stefan

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Sat, Oct 8, 2011 at 12:12 AM, Stefan Monnier
<monnier@iro.umontreal.ca> wrote:
>>> This seems to have been forgotten about again.  Windows and Mac are
>>> still the only platforms on which users are offered the choice of
>>> using mailclient-send-it if they haven't configured mail.
>> `mailclient-send-it' has never been a possible default on Linux systems,
>
> Hmm....
>
> (defun message-send-mail-function ()
>  "Return suitable value for the variable `message-send-mail-function'."
>  (cond ((and (require 'sendmail)
>              (boundp 'sendmail-program)
>              sendmail-program
>              (executable-find sendmail-program))
>         'message-send-mail-with-sendmail)
>        ((and (locate-library "smtpmail")
>              (boundp 'smtpmail-default-smtp-server)
>              smtpmail-default-smtp-server)
>         'message-smtpmail-send-it)
>        ((locate-library "mailclient")
>         'message-send-mail-with-mailclient)
>        (t
>         (error "Don't know how to send mail.  Please customize `message-send-mail-function'"))))
>
> Maybe you think it's not a possible default, but message.el does use it
> by default in some cases, without paying attention to the host OS.
> And I think that's right.
>
>> so I think it's quite likely to break in unexpected ways.  (I mean,
>> since it probably hasn't gotten much usage.)
>
> In my experience it works, and if it doesn't we should fix it.
>
>> So I think that, at this point, it's probably counter-productive to add
>> it as a choice.
>
> I disagree,
>
>
>        Stefan
>
>

1+

-- 
Tim Cross

Re: Changing the default for `send-mail-function'

[Richard Stallman]

I looked for a way to determine whether Exim is configured so that it
can send remote mail, but it looks complex.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    What I find is still unclear is whether those who will benefit will
    outweigh those who will be disadvantaged.

If we do what I suggest, some will gain and none will lose.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Tim Cross]

On Fri, Jul 8, 2011 at 10:06 AM, Richard Stallman <rms@gnu.org> wrote:
>    What I find is still unclear is whether those who will benefit will
>    outweigh those who will be disadvantaged.
>
> If we do what I suggest, some will gain and none will lose.
>

Hi Richard,

yes, if we are talking about your suggestion to test and determine if
the local MTA is configured, in theory that would possibly work. Your
suggestion of using make install is unlikely to work though. Most
emacs users seem to work from pre-built binaries and never run make.
While we could then say this is a distro packaging issue, I doubt any
distros will implement it due to the complexity involved.

I think the main problems with determining if a local MTA is installed
and configured are

     - There are many different MTAs. While most have at least partial
compatibility with 'sendmail', they don't always and they do things in
different ways. Reliably determining which MTA is installed can impact
on how you will test for a working config - some have good support for
this, others do not.

     - Testing for things in the local spool file is not reliable.
Some MTAs don't use the traditional mbox spool file for user's mail,
some use maildir, some can offer either etc.

     - The location used to store users mail can vary a lot. While
/var/mail is now common, it use to be /var/spool/mail and I've seen
many configurations that put it in the users home directory directly.

With respect to the question of why bounced messages would end up in
root (or postmaster's mailbox), the reasons are many. In the years
where I was responsible for administering mail servers for large
numbers of users, the three most common reasons were

     - Misconfigure 'from' address i.e. wrong username or domain. This
is surprisingly common. Mail servers never silently throw mail away.
If it cannot be delivered or bounced back to the sender, it is given
to postmaster, which is often aliased to root (at least on smaller and
single user systems). This can be somewhat mitigated with local MTAs
depending on the configuration. In the 'old' days, sendmail only
allowed 'trusted' users to set a from address different to the
username or owner of the process injecting the message for delivery.
You could use the -f (from memory) switch to sendmail to alter the
from address, but if yhou were not a trusted user, this was not
allowed. There also use to be restrictions on being able to modify the
envelop-from address. This greatly reduced the number of messages that
could not be returned to the sender as the from address had to be
correct i.e. had to have a real account on that system. However, newer
MTAs don't seem to have this restriction and many systems don't appear
to enforce this sort of restriction any more. It is now much easier to
get the from address settings wrong and have your mail appear to be
sent when in fact it was bounced, but the bounce never got to you.


     - Quota issues. Sometimes, the users mbox is past quota and won't
allow any additional messages to be put into their mailbox.  Usually,
the mail will be bounced back to the sender, but if that sender is
also the recipient, we end up with a looping problem. The loop is
broken by delivering the message to postmaster.

     - Misconfigured procmail or other delivery scripts. Sometimes
this is caused by the user and sometimes it is just mistakes made by
the sys admin. The types of problems are almost limitless and there is
no way to be certain about what will happen or where the mail will go.
Usually, when all else fails, the message will be delivered to
postmaster, but it is possible it will just be logged and never
delivered anywhere (though this is the extreme and rare case).

While with a lot of effort, I believe someone could probably implement
something that would get us 80% of the way, I think the effort would
be significant and maintenance would be very high. I also suspect we
are trying to solve a problem from an emacs perspective which should
really be solved from outside. Imagine the benefits to many packages
if there was a standard mechanism that could be used to determine if
the client should use a local MTA or a remote server and in the case
of the remote server, provide details, such as server name, port etc.
Maybe something similar to what the xdg group have done in other
areas.

Despite my negative assessment of your suggestion, I think the
underlying assumptions/principals are spot on. The new default should
have an overall beneficial result. I was previously focusing on the
impact for GNU/Linux users and thought the change was potentially
positive given that email environments have moved on and many people
do use remote smtp servers, especially those using laptops. However, I
didn't want those who currently get the functionality with little to
no effort to lose, especially when emacs is not their primary email
client. Maybe I expect too much!

At any rate, I've tried to express my concerns and feel its time to
let the decison be whatever it will be. In all likelihood, most people
will just accept the change and move on. There doesn't seem to be much
more to gain from this back and forth. At least the new default being
proposed is an improvement on the original suggestion of just setting
it to smtpmail.

Tim

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Your
    suggestion of using make install is unlikely to work though. Most
    emacs users seem to work from pre-built binaries and never run make.

We discussed that issue several days ago.

    While we could then say this is a distro packaging issue, I doubt any
    distros will implement it due to the complexity involved.

Why do you think that the work to be done in a distro would be
complex?

What I have in mind is a single line, in the script to set up the
Emacs package.  This would run a script included in Emacs, the same
script that `make install' would run.  It would be easy for distros to
include this one line.

Heuristics are sufficient for detecting functioning MTAs,
because they don't need to recognize all cases.
If they recognize some cases of functioning MTAs,
they can DTRT in those cases.  In any other cases
which they don't recognize, Emacs will fall back
on the code that asks the user.


Some of the problem cases you mentioned are not our problem.

	 - Misconfigure 'from' address i.e. wrong username or domain. This
    is surprisingly common.

It is not our problem.  A user who overrides the From address
is making himself responsible for whatever goes wrong.  So we
don't need to try to cope with any mistakes he might make.

    Imagine the benefits to many packages
    if there was a standard mechanism that could be used to determine if
    the client should use a local MTA or a remote server and in the case
    of the remote server, provide details, such as server name, port etc.

I agree.  At the system design level, that is the right solution.
sendmail should check this, and if there is no MTA functioning,
it should ask the user to configure to use SMTP.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    > The other way Emacs gets installed is from a package, via apt-get
    > or something like it.  That always gets done as root.
    >
    > Thus, we can assume this test is done running as root.

    Except apt-get doesn't call make install, so it's a package maintainer
    gotcha.

Yes, Emacs package maintainers will need to DTRT.
However, if they forget to do so, the result won't be any worse
than if this feature were not implemented.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Andreas Rottmann]

Antoine Levitt <antoine.levitt@gmail.com> writes:

> 05/07/11 23:05, Richard Stallman
>>     FWIW, I almost never run make install as root for Emacs.
>>
>> That susprises me -- could you explain why not?
>
> For instance, I used to use school servers that use an antiquated
> version of emacs, and I wanted the last one, so I just reinstalled emacs
> + the libs it needed in ~/new_root, with PREFIX=$HOME/new_root
> ./configure.
>
>> Anyway, the crucial case here is a multi-user machine where the person
>> using Emacs does not understand system maintenance and did not install
>> Emacs himself.  This is the case where there is probably a functioning
>> MTA plus users who use it without knowing what it is.  In that case,
>> `make install' was run as root.
>>
>> Thus, when `make install' is run not as root, it would be ok
>> not to decide, and let Emacs ask the user whatr to do.
>>
>> The other way Emacs gets installed is from a package, via apt-get
>> or something like it.  That always gets done as root.
>>
>> Thus, we can assume this test is done running as root.
>
> Except apt-get doesn't call make install, so it's a package maintainer
> gotcha.
>
Yeah, I guess distribution maintainers for Emacs packages would get
pretty annoyed if "make install" starts to send email (and I would be,
too).  In fact, on Debian buildservers, there is commonly no network
connectivity available.  

Frankly, I think "make install" doing such things is a quite horrible
violation of user expectations (i.e. of the users that run "make
install", not those necessarily those ending up using the installed
emacs): the results of "make install" should not depend on the
circumstances under which it is run (i.e. be reproducible for a given
build).  Provide a "fancy-install" target if you must, but don't make a
"make install" run inspect its environment.

Regards, Rotty
-- 
Andreas Rottmann -- <http://rotty.yi.org/>

Re: Changing the default for `send-mail-function'

[Per Starbäck]

> Since `make install' is run as root, it could look for a bounce
> message in root's local inbox too.

Or rather, is often (most of the time?) run as root.

Anyway, clearly this wouldn't be used for evil, but still I don't like
the idea of Emacs "reporting home"
when installed by sending mail. I like to think that programs I use
don't communicate over the net behind
my back, but only when I ask them to.

But install options that makes it possible for a root that installs
Emacs on a multi-user system to state
how mail is used there sounds like a good idea. No need for testing
then -- I think that roots that explicitly
choose to use these options should know what mail option is the
correct one for their system.

I guess then Emacs packages for most GNU distributions would use these
options, and then I guess
most Emacs users would use Emacses configured in such a way.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Anyway, clearly this wouldn't be used for evil, but still I don't like
    the idea of Emacs "reporting home"
    when installed by sending mail.

I agree.  I do not propose to send a real message but rather a dummy
message to determine whether a MTA is functioning.

    But install options that makes it possible for a root that installs
    Emacs on a multi-user system to state
    how mail is used there sounds like a good idea.

If it requires the user to specify it, I think users will mostly
forget to specify that option, so it won't be right.

    I guess then Emacs packages for most GNU distributions would use these
    options, and then I guess
    most Emacs users would use Emacses configured in such a way.

If the package presupposes one setting or the other, it will be wrong
a lot of the time.  The packages would need to do this same test
to determine, at package install time, whether an MTA is configured.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Antoine Levitt]

05/07/11 14:45, Richard Stallman
> Since there are problems with having Emacs check at run time
> whether there is a configured MTA, I propose doing it inside
> `make install'.

make install isn't necessarilly run at all.

>
> It could run a program that tries sending messages to certain chosen
> addresses, and detects the results.  This program could afford to wait
> a certain amount of time for the results to show up, with no danger it
> will be terminated by a user who does not realize it is still working
> on the problem.

Please, please, please don't do that. That looks like the kind of bloat
that rarely ends well, especially with mail being such a huge mess with
god knows how many programs interfering in various ways with mail
delivery. Also, I really don't want my "make install" to send mail (or
indeed, to do anything except, well, install).

> Since `make install' is run as root

Not always.

Anyways, this seems like a lot of trouble (and would complicate the task
of distro packagers) for a minor feature. More generally, this whole
thread screams "overengineering" IMO. Just set a default (smtp or not),
and users will figure out how to fix it if it doesn't work. Automagic is
nice, but more likely to have bugs, and can be confusing (many people
share their emacs configuration between machines, and might be baffled
as to why one emacs sends via SMTP and some other via sendmail).

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Anyways, this seems like a lot of trouble (and would complicate the task
    of distro packagers) for a minor feature. More generally, this whole
    thread screams "overengineering" IMO.

It certainly is -- the simple solution is for sendmail to fail if it
cannot really send mail anywhere, but that simple solution is not
implemented.

But this is better than the alternative that is proposed, which is
to require every user to understand something about how to mail should
be sent on that machine.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Richard Riley]

Richard Stallman <rms@gnu.org> writes:

> Since there are problems with having Emacs check at run time
> whether there is a configured MTA, I propose doing it inside
> `make install'.
>
> It could run a program that tries sending messages to certain chosen
> addresses, and detects the results.  This program could afford to wait
> a certain amount of time for the results to show up, with no danger it
> will be terminated by a user who does not realize it is still working
> on the problem.
>
> Since `make install' is run as root, it could look for a bounce
> message in root's local inbox too.

I don't think that's a valid assumption. I always install
"locally". Primarily because of the somewhat hairy Debian setup.

Re: Changing the default for `send-mail-function'

[Peter Münster]

On Tue, Jul 05 2011, Richard Stallman wrote:

> Since there are problems with having Emacs check at run time
> whether there is a configured MTA, I propose doing it inside
> `make install'.

Why not just print a big message on standard output at the end of
"make install" (or also at the beginning), like for example:

"Attention: the default value of send-mail-function is now smtp.
 If you want emacs to make use of your local MTA, then please change
 line X in file Y."

- If root runs "make install" on the target system, then he can change
  the default value of the variable.
- If "make install" runs on a special build host, the package maintainer
  can just DTRT, at build time with a patch or at install time with a
  post-install script.

-- 
           Peter

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    Why not just print a big message on standard output at the end of
    "make install" (or also at the beginning), like for example:

    "Attention: the default value of send-mail-function is now smtp.
     If you want emacs to make use of your local MTA, then please change
     line X in file Y."

`make install' produces lots of output; people are likely not to read
it closely.

To make this solution work, it would be necessary to make this
message stand out so people would notice it.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Peter Münster]

On Fri, Jul 08 2011, Richard Stallman wrote:

>     Why not just print a big message on standard output at the end of
>     "make install" (or also at the beginning), like for example:
>
>     "Attention: the default value of send-mail-function is now smtp.
>      If you want emacs to make use of your local MTA, then please change
>      line X in file Y."
>
> `make install' produces lots of output; people are likely not to read
> it closely.
>
> To make this solution work, it would be necessary to make this
> message stand out so people would notice it.

Indeed. What about something like:

--8<---------------cut here---------------start------------->8---
echo $BIG_OUTSTANDING_MESSAGE
read -t100 -p"Sleeping 100 seconds. Press enter to continue immediately!"
--8<---------------cut here---------------end--------------->8---
?

And you can put the message into the NEWS and the INSTALL files.

The benefit would be, that you won't need any heuristics: you would
delegate this configuration issue to the system administrator who knows
well his MTA setup or to the package maintainer who knows well his
distro.

-- 
           Peter

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Richard Stallman writes:

 >     Perhaps we can make progress if you permit me to ask you a question.
 >     What's wrong with a two-protocol system?
 > 
 > I don't know whether something is wrong with that, because the
 > question is not entirely clear.

What was unclear about the description of the initial bug submission
process that followed?  It was somewhat elliptical, but I thought the
general idea was clear enough.

 > I have nothing against making Emacs able to use multiple
 > protocols for sending mail.  Having those capabilities
 > can't hurt anyone.  The question is how Emacs decides which
 > method to use.

No, it's not about using multiple protocols for sending mail.  It's
about using a non-mail protocol for the initial submission only, it
doesn't apply to later correspondence about a particular issue, and
certainly not to general mail.  The process for deciding was described
there and is further elaborated below.

 > I want to avoid asking users to configure SMTP on systems where
 > sendmail works.

I'm suggesting that report-emacs-bug need not ask users to configure
SMTP ever.  Ask them if they know "for sure" that Emacs can send mail.
If the answer is "yes", (1) send the mail, and (2) configure Emacs to
always use mail.

If the answer is "no", then (1') ask for the user's mail address
(report-emacs-bug may default to something bogus), (1'') submit it via
an HTTP-to-mail gateway, and (2') if submission succeeds, configure
Emacs to always use the HTTP-to-mail gateway, otherwise (3) report the
failure to the user and attempt to send by mail, notifying the user
that mail submission is unreliable and suggesting they check the bg
database using a web browser or debbugs.el (in the future Emacs will
ask for preferred submission method again).

There are possible interactive variations on (2) and (2'), but this is
the minimal interaction setup.

 > You don't have to discuss it, but that doesn't convince me that
 > making sendmail reliably fail when the MTA is not really set up
 > is too difficult.

Let's put it this way: my opinion is that extensive discussion will
profit nobody involved, unless you desire to learn a huge amount of
detail about the mail system and the packaging and installation
procedures for MTAs for their own sake.  From their posts, I suspect
that Stefan and Yidong agree.

 > Also, your footnote [1] suggests we are not talking about the same
 > question.  What I have in mind does not match what you say there.

I understand that; my opinion is that (a) what you have in mind as
described is not sufficient to meet your own requirements, unless you
relax the reliability requirement to a point unacceptable to people
using the system, and (b) it is not possible without the cooperation
of many third parties, who will demand many adjustments before they
are willing to implement.  Not to mention that changes to the mail
system take time measured in decades to implement fully.  It's not
something that should concern the Emacs project, IMO YMMV.

I think that the amount of work, especially on your part, to come to a
meeting of minds (the lead maintainers and yours) on "fixing sendmail"
far exceeds the costs of "Just Do It"-ing the HTTP gateway proposal,
which is a very simple design and therefore easy to evaluate, both
cost and benefit, and requires no cooperation from anyone outside of
the GNU Project.

Ted Z proposed a method[1], which I have elaborated above, that
satisfies all of your requirements, except that it isn't 100% email
for *all* users.  (Most users can still use email for all of their bug
reporting, however.)  The costs are:

(1) A one-time yes-no configuration for all users of report-emacs-bug,
    requiring about 10-15 lines including the prompt strings.
(2) An estimated 10-15 lines of hacking using url.el for the HTTP
    submission, including the status report strings.  The message
    generation part of report-emacs-bug needs no change -- the report
    will be submitted as an RFC-822-formatted message, and gatewayed
    verbatim to a standard MTA.
(3) Installation of a very simple CGI at a gnu.org host.

(1) could be avoided by simply having report-emacs-bug always use the
HTTP gateway, but this may not work in firewall configurations where
users gateway SMTP mail via a smarthost and otherwise may have no
connection to the net from personal workstations, (2) is on offer from
Ted IIUC, but I can do it in late August if nobody else does it by
then, and (3) will require somebody with privileges on an appropriate
host.  I won't do (3) -- there are security implications on a
sensitive host, and I am familiar with neither security policy nor the
infrastructure there -- so somebody would need to volunteer for that.

Footnotes: 
[1]  Lars proposed a very similar method using the same design but a
different submission protocol.  However, Ted and I agree that it is
inferior in terms of robustness to intervening firewalls.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    No, it's not about using multiple protocols for sending mail.  It's
    about using a non-mail protocol for the initial submission only,

It looks like we are failing to communicate.  I'm talking about how
to configure Emacs for sending mail.  I think you are responding to
other suggestions that were solely about sending Emacs bug reports.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Richard Stallman writes:

 >     No, it's not about using multiple protocols for sending mail.  It's
 >     about using a non-mail protocol for the initial submission only,
 > 
 > It looks like we are failing to communicate.  I'm talking about how
 > to configure Emacs for sending mail.

You're right.  I have no interest in that, because IMHO given the mail
system as it is and will be for the next 5 years, it's a waste of time
to try to do better than the current system.  I'm surprised that you
could think otherwise (though thinking about it calmly I suppose I
could have anticipated this), but "de gustibus non est disputandum."

FWIW, I agree with the several reasons given by Antoine why attempting
to detect the mailer at "make install" time is even less likely to
work well than trying to configure when the user actually wants to
send mail.  And one more: "make install" is very likely to be executed
on a build host at debian.org or redhat.com, rather than on the target
host.  (Maybe I'm missing something, but if so, I'll let somebody else
figure out what.)

 > I think you are responding to other suggestions that were solely
 > about sending Emacs bug reports.

Yup.  If somebody doesn't already have their host set up for sending
email from Emacs, then M-x report-emacs-bug is IMHO the only good
reason so far proposed for worrying about sending mail from Emacs.
Since you think otherwise, I apologize for wasting your time rebutting
mistargeted criticism.

Sincerely,

Re: Changing the default for `send-mail-function'

[Stefan Monnier]

> My proposal is that Emacs should check whether a bounce message
> arrives in the local mailbox.  For this purpose, it doesn't matter
> whether the human user thinks of looking in that local mailbox.

But there's a delay between the time sendmail terminates and the bounce
shows up (if it does) in some local mailbox.
It's just not going to work reliably.


        Stefan

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    But there's a delay between the time sendmail terminates and the bounce
    shows up (if it does) in some local mailbox.

Allowing 5 minutes should be enough.


-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Richard Stallman <rms@gnu.org> writes:

>     But there's a delay between the time sendmail terminates and the bounce
>     shows up (if it does) in some local mailbox.
>
> Allowing 5 minutes should be enough.

There's no guarantee that Emacs will still be running in 5 minutes.

I agree with Stefan: there's no way to make this work reliably.

Re: Changing the default for `send-mail-function'

[Jason Rumney]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Eli Zaretskii <eliz@gnu.org> writes:
>
>> We don't want that to get in the way of an Emacs user who wants to
>> report a bug, for example.
>
> Another random idea: If that's the main problem, then if the gnu.org MTA
> would open a listening port on port 587, and accept mail to gnu.org on
> that port, then we could just have the bug reports use
> `smtpmail-send-it' to eggs.gnu.org:587 directly...

Why port 587?  Port 25 is the standard for incoming mail to the mail
server's domain.

Re: Changing the default for `send-mail-function'

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 539 bytes --]

On Mon, Jun 27 2011, Jason Rumney wrote:

>> Another random idea: If that's the main problem, then if the gnu.org MTA
>> would open a listening port on port 587, and accept mail to gnu.org on
>> that port, then we could just have the bug reports use
>> `smtpmail-send-it' to eggs.gnu.org:587 directly...
>
> Why port 587?  Port 25 is the standard for incoming mail to the mail
> server's domain.

Port 587 is for submission. But it's not meant for such a use, in
theory. :)

-- 
Julien Danjou
❱ http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Julien Danjou writes:
 > On Mon, Jun 27 2011, Jason Rumney wrote:
 > 
 > >> Another random idea: If that's the main problem, then if the gnu.org MTA
 > >> would open a listening port on port 587, and accept mail to gnu.org on
 > >> that port, then we could just have the bug reports use
 > >> `smtpmail-send-it' to eggs.gnu.org:587 directly...
 > >
 > > Why port 587?  Port 25 is the standard for incoming mail to the mail
 > > server's domain.
 > 
 > Port 587 is for submission. But it's not meant for such a use, in
 > theory. :)

Actually, this clearly fails to conform to the RFC.  Authentication is
mandated because a submission agent must know the identity of
submitting entity, while the whole point of this discussion is (more
or less :-) that some users don't know their own identities.

An equally reliable way to handle this would be to format the message
using report-emacs-bug as usual, then ship it via HTTP/HTTPS post to a
CGI that mails it to the bug address.  Or use NNTP to inject it via,
say, Gmane.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

"Stephen J. Turnbull" <stephen@xemacs.org> writes:

> Actually, this clearly fails to conform to the RFC.

Well, so what?

> Authentication is mandated because a submission agent must know the
> identity of submitting entity, while the whole point of this
> discussion is (more or less :-) that some users don't know their own
> identities.

We can use whatever protocol we want to submit bug reports.  Whether we
use SMTP, NNTP or HTTP, that up to us.

And the result is the same: We get bug reports from users who don't know
their own identities.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Mon, 27 Jun 2011 20:56:50 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> And the result is the same: We get bug reports from users who don't know
LMI> their own identities.

Do we want to accept anonymous bug reports?

I don't mean spam; the reports can be authenticated in some other way to
ensure they are genuine.  But if the submitter doesn't want to be known
or contacted, should that be grounds for dismissal of the bug report?

For instance it could contain embarassing or sensitive (if the sender's
identity were public) information.

Ted

Re: Changing the default for `send-mail-function'

[Stephen J. Turnbull]

Ted Zlatanov writes:
 > On Mon, 27 Jun 2011 20:56:50 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
 > 
 > LMI> And the result is the same: We get bug reports from users who don't know
 > LMI> their own identities.
 > 
 > Do we want to accept anonymous bug reports?

Why not?  It's true that if there's no return path, you have to depend
on the sender to get in touch if you need more information, and that
slows/stops progress on the bug in some significant number of cases.
But at the very least, it provides a historical record of a particular
symptom at a point in time.

OTOH, many issues are easy to resolve based on the information in the
initial report.

 > I don't mean spam; the reports can be authenticated in some other way to
 > ensure they are genuine.

The reporters cannot be identified, let alone authenticated; you don't
even know that the submitter isn't a dog.  The best you can do is
ensure a channel for requesting more information, but that is easy
enough to ignore or abandon (and often is ignored).

Filtering spam and the occasional crank (eg, who keeps opening new bug
numbers for the same not-a-bug) will be done on the basis of content,
as usual.

Re: Changing the default for `send-mail-function'

[Chong Yidong]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> We can use whatever protocol we want to submit bug reports.  Whether we
> use SMTP, NNTP or HTTP, that up to us.
>
> And the result is the same: We get bug reports from users who don't know
> their own identities.

We need to reply to the user, e.g. to ask for details.

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Chong Yidong <cyd@stupidchicken.com> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> We can use whatever protocol we want to submit bug reports.  Whether we
>> use SMTP, NNTP or HTTP, that up to us.
>>
>> And the result is the same: We get bug reports from users who don't know
>> their own identities.
>
> We need to reply to the user, e.g. to ask for details.

The From: addresses will remain the same as they do now, so there won't
be a significant difference there.

There will be some -- people who are running on systems that are totally
un/misconfigured will have From: addresses that are so mangled that
previously an MTA on the route between the user and the gnu.org MTA
would have bounced it, but I'm not sure whether that counts for or
against the idea of not using the normal mail channels to submit bug
reports.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    We can use whatever protocol we want to submit bug reports.  Whether we
    use SMTP, NNTP or HTTP, that up to us.

You need to talk with the GNU sysadmins about this -- They may have
security concerns.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Ted Zlatanov]

On Tue, 28 Jun 2011 02:10:29 +0900 "Stephen J. Turnbull" <stephen@xemacs.org> wrote: 

SJT> An equally reliable way to handle this would be to format the message
SJT> using report-emacs-bug as usual, then ship it via HTTP/HTTPS post to a
SJT> CGI that mails it to the bug address.  

s/equally/more/ since the user does not need to configure nearly as many
things and most firewalls do not block port 80 but do block other ports.
Really, this is the best solution *today* for *bug reporting* as opposed
to general sending of mail.  No ports, SMTP conversations, defcustoms,
or hand-holding are needed.

SJT> Or use NNTP to inject it via, say, Gmane.

No, please, no.

Ted

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Jason Rumney <jasonr@gnu.org> writes:

> Why port 587?  Port 25 is the standard for incoming mail to the mail
> server's domain.

Port 25 is commonly blocked by ISPs.  Port 587 is the new standard port
for submitting email from MUAs to MTAs that are aware of the relay
problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Jason Rumney]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Port 25 is commonly blocked by ISPs.

Indeed. And all outgoing ports are commonly blocked by corporate
firewalls, which I'm sure is not an insignificant proportion of Emacs
users.  So direct connections to gnu servers are not really an
improvement over misconfigured local mail servers and clients.

Re: Changing the default for `send-mail-function'

[Richard Stallman]

    As previously discussed, the default value of `send-mail-function' would
    change to `smtpmail-send-it' (on all architectures) after the smtpmail.el
    rework was complete.

The current setting is sendmail-send-it.
That works by calling the system's mailer, which (we can presume)
has already been configured for the sake of other mail programs.

    At least on Windows, the system mailer (Outlook or whatever) is
    normally already configured, and many times by a person other than the
    one who invoked Emacs.  The latter might not know the address or the
    credentials, or both.

This is often the case on GNU/Linux, too.  

When this isn't the case, the user will have to configure mail sending
somehow.  In that case, what advantage is there in configuring mail in
Emacs, rather than doing it at the system level so that every program
or script can send mail?

Is there a way Emacs can determine whether mail sending has been
configured at the system level, and if not, ask the user
to do so either in Emacs (with smtpmail) or else at system level
(and use sendmail-send-it)?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Richard Stallman <rms@gnu.org> writes:

> That works by calling the system's mailer, which (we can presume)
> has already been configured for the sake of other mail programs.

The other modern mail programs do not use the system's mailer.  If you
send mail via (for instance) Thunderbird, you have to configure the
outgoing SMTP server in Thunderbird.

That is, the other modern mail programs reached the conclusion a few
years back that doing SMTP themselves is easier than relying on the
system being set up to do outgoing SMTP, and I think it's time that
Emacs followed.

> Is there a way Emacs can determine whether mail sending has been
> configured at the system level, and if not, ask the user
> to do so either in Emacs (with smtpmail) or else at system level
> (and use sendmail-send-it)?

Emacs today (on Linux) sees whether there is a sendmail (exim, etc)
installed, and if there is, it assumes that it's capable of doing
outgoing SMTP.  This hasn't actually been true on major Linux
distributions like Ubuntu for a while, so, no, I don't think it's
realistic to find a way to parse the exim (sendmail, postfix) setup to
see whether it actually is able to deliver email to the world.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: Changing the default for `send-mail-function'

[Richard Stallman]

      This hasn't actually been true on major Linux
    distributions like Ubuntu for a while,

The term "Linux distributions" is a misnomer -- Ubuntu is a GNU/Linux
distribution.  To call it a "Linux distribution" is unfair to the GNU
Project, which includes all of us here, and impedes our work today.

Please call it a "GNU/Linux distribution", and please say you're
doing this on the GNU/Linux system.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org, www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use free telephony http://directory.fsf.org/category/tel/

Re: Changing the default for `send-mail-function'

[Andrey Paramonov]

 > Yes, but like I said, we would then, in the future, be unable to
 > change the default again, since all users would by then have a setting
 > explicitly in .emacs.

Now send-mail-function can be one of mailclient-send-it, 
smtpmail-send-it, etc.

One way to keep the ability to change the defaults is to introduce 
special promptonce-send-it as another possible value for 
send-mail-function. It would prompt user for configuration once and 
store actual mail-sending function in separate customizable variable, 
i.e. promptonce-send-mail-function. This promptonce-send-it should be 
made the default value for send-mail-function. This way, users' .emacs 
will have customized promptonce-send-mail-function, but not 
send-mail-function.

Not sure it's worth the complication though, just my 2 cents.

Best wishes,
Andrey Paramonov

Re: Changing the default for `send-mail-function'

[Lars Magne Ingebrigtsen]

Andrey Paramonov <paramon@acdlabs.ru> writes:

> One way to keep the ability to change the defaults is to introduce
> special promptonce-send-it as another possible value for
> send-mail-function. It would prompt user for configuration once and
> store actual mail-sending function in separate customizable variable,
> i.e. promptonce-send-mail-function. This promptonce-send-it should be
> made the default value for send-mail-function. This way, users' .emacs
> will have customized promptonce-send-mail-function, but not
> send-mail-function.

I think that's an excellent solution, and I'll code it up.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/