Rant - Emacs mail is not user friendly

Rant - Emacs mail is not user friendly

[Kelly Dean]

Yesterday I decided to try using Emacs for email. For sending messages to my mail server, I'll use a separate program, so I need Emacs just for composing messages, inserting headers including autogenerated Date and Message-ID headers, encoding file attachments, and moving completed messages from my drafts folder to my outbound queue. Section 32.4.1 (Mail Sending) of the manual says to read feedmail.el to learn how to do this.

feedmail.el says:
;; After a long list of options below, you will find the function
;; feedmail-send-it. Hers's the best way to use the stuff in this
;; file:
;;
;; Save this file as feedmail.el somewhere on your elisp loadpath;
;; byte-compile it.  Put the following lines in your init file:
;;
;;     (setq send-mail-function 'feedmail-send-it)
;;     (autoload 'feedmail-send-it "feedmail")
;;
;; If you plan to use the queue stuff, also use this:
;;
;;     (setq feedmail-enable-queue t)
;;     (autoload 'feedmail-run-the-queue "feedmail")
;;     (autoload 'feedmail-run-the-queue-no-prompts "feedmail")
;;     (setq auto-mode-alist (cons '("\\.fqm$" . mail-mode) auto-mode-alist))
;;
;; though VM users might find it more comfortable to use this instead of
;; the above example's last line:
;;
;;     (setq auto-mode-alist (cons '("\\.fqm$" . feedmail-vm-mail-mode) auto-mode-alist))

Um, ok. Done. Though it's already in Emacs's loadpath by default, and already byte-compiled, so I skipped that part. I don't know what a VM is in this context, or why the alternative setting might be more comfortable, so I'm ignoring that part and just hoping it won't be a problem. I searched for ⌜feedmail-vm-mail-mode⌝ and found:
⌜If you are a VM user, you might like feedmail-vm-mail-mode, though you
really don't need that (and it's not particularly well-tested).⌝

Next:
;; If you are using the desktop.el library to restore your sessions, you might
;; like to add the suffix ".fqm" to the list of non-saved things via the variable
;; desktop-files-not-to-save.

I found ⌜FQM stands for feedmail queued message⌝, so why wouldn't I want those to be saved? No explanation. Skipping this part.

Next:
;; If you are using message-mode to compose and send mail, feedmail will
;; probably work fine with that (someone else tested it and told me it worked).
;; Follow the directions above, but make these adjustments instead:
;;
;;     (setq message-send-mail-function 'feedmail-send-it)
;;     (add-hook 'message-mail-send-hook 'feedmail-mail-send-hook-splitter)

Am I using message-mode? Section 32 (Sending Mail, as opposed to Mail Sending in section 32.4.1) of the manual says ⌜To send an email message from Emacs, type C-x m⌝. That runs compose-mail, which creates a buffer in message-mode, so apparently I am. Ok, adjustments made.
Except there doesn't appear to be a message-mail-send-hook, so I omitted that part. The docstring for ⌜feedmail-mail-send-hook-splitter⌝ says ⌜Facilitate dividing `mail-send-hook' things into queued and immediate cases⌝, which isn't a feature I need since I'll only be queuing (no immediate sending), so I guess it won't be a problem that there's no hook to put it on.

Next:
;; If you use message-mode and you make use of feedmail's queueing
;; stuff, you might also like to adjust these variables to appropriate
;; values for message-mode:
;;
;;     feedmail-queue-runner-mode-setter
;;     feedmail-queue-runner-message-sender

The docstring for the first var says ⌜A function to set the proper mode of a message file⌝. It also says ⌜Most people want `mail-mode'⌝, and that's the default, so I guess I don't want to adjust it. Similarly for the second var.

Ok, C-x m, write a test message, C-x C-s, C-c C-s RET C-x k RET. It adds the appropriate headers, and in ~/mail/q/ I get a fqm file, yay. Except, it still has the ⌜--text follows this line--⌝ from the draft, so it isn't ready for sending. Why? Dunno.
And the draft is still in ~/Mail/drafts (capitalized ⌜Mail⌝, not lowercase ⌜mail⌝ like for the queue, for no apparent reason), which is illogical, since the message has already been completed and queued for sending, and leaving a copy in drafts will just confuse me later about which messages have or haven't already been sent. To decide whether I want to move the draft message to ~/mail/sent or delete the draft message and copy the queued message to ~/mail/sent, I re-open the draft to compare it to the queued version. I notice the draft isn't in message-mode anymore. What if I wanted to re-send it (assuming it should still be in drafts in the first place)? Ok, M-x message-mode. C-c C-s RET C-x k RET, and it queues another.
Then I notice: re-open the draft, M-x message-mode, C-c C-s RET, but don't kill the buffer this time. C-c C-s RET again, and this time it asks for confirmation to resend, which makes sense because from Emacs's point of view, the message was already sent, but why didn't it ask for confirmation for the previous resend? Why did it forget that the message was already sent? I discover that I don't even have to kill the buffer to make it forget; I can just do M-x message-mode again. A mail program that can't remember whether it has already sent a message is demented. I didn't even modify the default locations for saving messages; it simply failed to save that flag anywhere or to record it implicitly e.g. by moving sent messages to ~/mail/sent.

I've already queued enough copies of this test message, so this time I answer ‟no” to the ‟resend” question. It dumps me into the debugger (I have debug-on-error set to t to try to help debug all the Emacs bugs that keep biting me) because it's using error in a place where in modern Emacs it should now be using user-error, so now I have to add yet another patch to my growing pile of private fixes (including for actual bugs, not just user-error updates) that I've been asked not to submit for Emacs because saying my work is in the public domain isn't good enough, even though it apparently was good enough for the author of feedmail (see the statement ⌜This code is hereby released into the public domain⌝ in feedmail.el--which I never would have bothered to read in the first place if feedmail's documentation were in the manual).

The Message-ID header always includes ⌜i-did-not-set--mail-host-address--so-tickle-me⌝, so I do
(setq mail-host-address "localhost")
and try again. No luck, so let's try something else. I find the feedmail-message-id-generator option, which says this:

⌜Specifies the creation of a Message-Id: header field.

If nil, nothing is done about Message-Id:.

If t, a Message-Id: header of a predetermined format is produced, but
only if there is not already a Message-Id: in the message.  A value of
t is equivalent to using the function feedmail-default-message-id-generator.

If neither nil nor t, it may be a string, a fiddle-plex, or a function
which returns either nil, t, a string, or a fiddle-plex (or, in fact,
another function, but let's not be ridiculous).  If a string, it
should be just the contents of the header, not the name of the header
itself nor the trailing newline.  If a function, it will be called
with one argument: the possibly-nil name of the file associated with
the message buffer.⌝

I try that. Still no luck; I still get the standard Message-ID header. Oh, but after reading the definition of the function feedmail-fiddle-message-id, I discover that despite the docstring, feedmail-message-id-generator _can't_ be function, but it can be a symbol fbound to a function. Ok, I change my lambda to a named function.
Still no luck. Digging further, I discover that feedmail-fiddle-message-id is only called from feedmail-send-it-immediately, which I don't use, because I'm only queuing, not sending immediately. Did the docs say that feedmail-message-id-generator wouldn't be used when just queuing rather than sending immediately? Of course not.

The documentation is ridiculous, it's in the wrong place, it's outdated, the code doesn't work, it's outdated too, and there's no point in trying to fix any of it because my work is unwanted anyway. Even the worst webmail service I've ever used was more user-friendly than this, which is why I keep using them even though they routinely betray me by locking me out of my account for some stupid little thing like lying about my niece's name because it's none of Yahoo's business or logging in from a new IP address and failing to tell Google about it in advance, forcing me to create new account and lose any mail stored in the previous one that I didn't paranoidly download an offline copy of after every mail session in anticipation of their inevitable betrayal. At least Google let me back in after I traveled back to where I could log in from the previous IP address again. I'm telling you all this in order to waste your time from reading this message so you'll feel my pain, or will at least tell me you do and I'll vote for you or something.

Re: Rant - Emacs mail is not user friendly

[Alexis]

Kelly Dean writes:

> Yesterday I decided to try using Emacs for email.

Perhaps mu4e might fit your needs:

http://www.djcbsoftware.nl/code/mu/mu4e.html

mu4e was the first Emacs-based MUA i found practical, such that i now
use mu4e instead of Mutt. :-)


Alexis.

Re: Rant - Emacs mail is not user friendly

[Eli Zaretskii]

> From: Kelly Dean <kelly@prtime.org>
> Date: Sat, 15 Nov 2014 06:54:07 +0000
> 
> The documentation is ridiculous, it's in the wrong place, it's outdated, the code doesn't work, it's outdated too, and there's no point in trying to fix any of it because my work is unwanted anyway. Even the worst webmail service I've ever used was more user-friendly than this, which is why I keep using them even though they routinely betray me by locking me out of my account for some stupid little thing like lying about my niece's name because it's none of Yahoo's business or logging in from a new IP address and failing to tell Google about it in advance, forcing me to create new account and lose any mail stored in the previous one that I didn't paranoidly download an offline copy of after every mail session in anticipation of their inevitable betrayal. At least Google let me back in af
 ter I traveled back to where I could log in from the previous IP address again. I'm telling you all this in order to waste your time from reading this message so you'll feel my pain, or will at leas

You are right.  The reason is probably that feedmail is used by only a
few who already know from long experience how to set it up.

I'd encourage you to file a bug report about this, and suggest there
changes for the parts that you've succeeded to unlock.  Thanks in
advance.

Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Kelly Dean writes:

 > Even the worst webmail service I've ever used was more
 > user-friendly than this,

Except that webmail doesn't provide access to configure the message
submission process the way feedmail does.  And even hackers mostly
don't use feedmail.

I understand your frustration, but I don't think the source is Emacs.
It's that Internet mail is insanely complicated (and powerful to
match).  You are trying to manually implement some of the complexity
that is normally hidden inside the MUA and MTA.  In particular, mail
queuing is about as complex as things get.  MUAs normally delegate
queuing to the MTA (that's how I handle disconnected operation on my
laptop, for example, and that's the *only* reason I run Postfix on my
laptop), and in that sense feedmail.el is a hack.

 > Yesterday I decided to try using Emacs for email. For sending
 > messages to my mail server, I'll use a separate program,

I'm sure you have good reason for doing that, but that is a very
unusual use case.  All Emacs MUAs are highly configurable so that they
can submit directly to servers (except maybe MH-E, which delegates
that function to MH IIRC), and much effort goes into getting that
right.  The idea of depending a on program that processes a shared
queue directory doesn't get the same attention (to say the least).

 > so I need Emacs just for composing messages, inserting headers
 > including autogenerated Date and Message-ID headers, encoding file
 > attachments, and moving completed messages from my drafts folder to
 > my outbound queue.

And you chose Emacs[sic] over which other MUAs capable of being
configured to do this?

By [sic] I mean that Emacs, in fact, is not an MUA.  Rather, it
supports at least 5 major ones written in Lisp, plus MH-E which is an
Emacs front-end to the MH family of text-based MUAs.  To understand
the operation of feedmail (or at least the description in the comments
in the library) and the baroque suite of interface/configuration
variables, it helps to be aware of that background.

One issue you seem to have misunderstood is that feedmail.el's queuing
feature does *not* seem to be designed for your use case, where the
queue will be processed by an external program.  Rather, I read the
notes to assume that feedmail.el intercepts the message by inserting
itself into the send-mail-function hook, then later *reads the queued
message into a buffer*, where it invokes the normal function on the
message in the queue at user request.  That is, the queue is intended
to be private to feedmail.  That mode of operation could contribute to
explaining several of the anomolies you observe (eg, the presence of
the header/body separator in the queued message).

Bottom line: I think the task you were trying to accomplish is far
more complex that you are admitting, and has little to do with "user
friendliness" of Emacs MUAs.

Regards,

Rant - Emacs mail is not user friendly

[Kelly Dean]

> In particular, mail
> queuing is about as complex as things get.  MUAs normally delegate
> queuing to the MTA (that's how I handle disconnected operation on my
> laptop, for example, and that's the *only* reason I run Postfix on my
> laptop), and in that sense feedmail.el is a hack.
It doesn't seem very complex, at least on the user side of things. You might want to consider msmtp instead of Postfix on your laptop.

> I'm sure you have good reason for doing that,
Yes. Emacs on my machine doesn't have Internet access.

> but that is a very unusual use case.
I certainly hope not! The usual case is that a person's instance of Emacs has access to his data (otherwise, Emacs wouldn't be very useful); if my use case is unusual, that means the usual case is that his Emacs has access not only to his data, but also to the Internet. With a program like Emacs serving as a bridge, that means his data isn't his data, but instead belongs to some Russian or Chinese hacker, whoever crosses the bridge first, and the original owner is permitted to retain a copy of the data just due to the hacker's mercifulness, or maybe just as bait so the hacker can get even more data later. The years when normal people weren't targets are long gone.

> Bottom line: I think the task you were trying to accomplish is far
> more complex that you are admitting, and has little to do with "user
> friendliness" of Emacs MUAs.
My previous, manual workflow was:
0. Compose a message in Emacs, note a subject and recipient, and note any files to attach.
1. Log in to webmail, and click ‟compose message”.
2. Copy the message, subject, and recipient from Emacs to the web browser, and attach the noted files.
3. Click ‟send”.
4. Save the message in Emacs to my file of sent messages (in anticipation of the webmail service's betrayal).

That was inconvenient, so I decided a better workflow would be:
0. Compose a message in Emacs in standard format, with Subject and To headers separated from the body by a blank line.
1. Add the From, Date, and Message-ID headers.
2. Save the message to a file foo in an outbox directory.
3. Run ⌜msmtp -t < outbox/foo⌝
4. If #3 succeeds, then run ⌜mv outbox/foo sent/foo⌝

The Emacs mail-sending functionality I wanted was simply automation of steps #1 (plus let me attach files) and #2 of the better workflow, without also doing steps #3 and #4. That's hardly complex, and in fact is simply _omission_ of the last two steps that MUAs ordinarily do (or equivalents thereof). I looked in the Emacs manual, and it said Emacs can do what I wanted, and said to read feedmail.el to learn how. Nowhere did the manual warn ‟Here be dragons”; had I known to avoid feedmail, that would have saved me the day that I wasted on it.

For me, Emacs assists with step #0 except that the ⌜--text follows this line--⌝ is left in, it automates step #1 except that the Message-ID is broken, and it automates step #2 except that it leaves the draft in place after queuing the message.

I eventually figured out that I don't need feedmail at all. The broken Message-ID is just because Emacs (in message.el; not feedmail's fault) rejects my ‟localhost” hostname (the machine has no network connection, so it needs no hostname), and the other things are fixable with just a few lines:

(defun delete-mail-header-separator ()
  "Delete `mail-header-separator'. This function copied from top of `message-send-mail-partially' in Emacs's message.el."
  (goto-char (point-min))
  (re-search-forward
   (concat "^" (regexp-quote mail-header-separator) "\n"))
  (replace-match "\n"))

(defcustom message-outbox-directory
  (file-name-as-directory (expand-file-name "outbox" message-directory))
  :type 'directory)

; mv-rename copied from http://code.activestate.com/recipes/578116-move-files-with-rename-if-required/
(defun queue-message-to-outbox ()
  (let ((filename (expand-file-name "out.mail" message-outbox-directory)))
    (delete-mail-header-separator)
    (write-file filename)
    (shell-command (concat "mv-rename " filename " " message-outbox-directory))))

(defun discard-draft ()
  (delete-file (buffer-file-name)))

(setq message-send-mail-function 'queue-message-to-outbox)
(add-hook 'message-sent-hook 'discard-draft)

Also I had to add ⌜(pushnew '(utf-8 . 8bit) mm-body-charset-encoding-alist)⌝ to my init file to make Emacs stop mangling the text. (Yes I know, the mangling is intended as a preemptive surrender to the administrators of broken MTAs that used to dominate the Internet, but those seem to have been mostly vanquished now. At least my messages appear to be getting to the mailing list with no problem.)

And Emacs doesn't like it if I open sent/foo that has the ⌜--text follows this line--⌝ deleted and switch to message-mode and try to resend it, so I have to put that back in this case. I don't know what the point of it is, but it's a minor inconvenience and I've already wasted enough time on Emacs email, so I'll live with it.

Last problem that I do care about is that Emacs keeps inserting line breaks after 72 characters, so I have to switch out of message-mode or copy/paste from another buffer to make it stop doing that because I can't find the setting to turn it off. Why is it turned on by default? Modern email readers do have word-wrap, after all. Even Emacs.

Re: Rant - Emacs mail is not user friendly

[David Caldwell]

[-- Attachment #1: Type: text/plain, Size: 1282 bytes --]

On 11/16/14 4:18 AM, Kelly Dean wrote:
> Last problem that I do care about is that Emacs keeps inserting line
> breaks after 72 characters, so I have to switch out of message-mode
> or copy/paste from another buffer to make it stop doing that because
> I can't find the setting to turn it off.

Looks like it runs "auto-fill-mode". You can turn it off, or you can
adjust the column that it wraps with the "fill-column" variable.

> Why is it turned on by default? Modern email readers do have
> word-wrap, after all. Even Emacs.

Probably because of tradition. It's generally considered polite to wrap
your column somewhere between 72 to 78. The SMTP spec (RFC 821) says:

> The maximum total length of a text line including the <CRLF> is 1000
> characters (but not counting the leading dot duplicated for
> transparency).

So there is a maximum, though it's about a 13 line paragraph (wrapped at
72).

Many email clients (including Thunderbird, which I'm using) use the
quotable-printable encoding to encode longer lines (which escapes the
line with an "=" if it's not meant to end). I guess this idea is to make
the raw text mostly readable but still unwrappable. I think it's quite
ugly, but it appears to be a pretty universal standard.

-David



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4219 bytes --]

Re: Rant - Emacs mail is not user friendly

[Stephen Berman]

On Sun, 16 Nov 2014 12:18:41 +0000 "Kelly Dean" <kelly@prtime.org> wrote:

> Last problem that I do care about is that Emacs keeps inserting line
> breaks after 72 characters, so I have to switch out of message-mode or
> copy/paste from another buffer to make it stop doing that because I
> can't find the setting to turn it off. Why is it turned on by default?
> Modern email readers do have word-wrap, after all. Even Emacs.

Does customizing message-fill-column do what you want?  (If you have a
message-mode buffer when you change the value, it seems you have to kill
the buffer and open it afresh to get the effect.)

Steve Berman

Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Kelly Dean writes:

 > > In particular, mail
 > > queuing is about as complex as things get.  MUAs normally delegate
 > > queuing to the MTA (that's how I handle disconnected operation on my
 > > laptop, for example, and that's the *only* reason I run Postfix on my
 > > laptop), and in that sense feedmail.el is a hack.

 > It doesn't seem very complex, at least on the user side of
 > things.

Of course the user doesn't perceive the complexity!  It's hidden in
the abstractions of "MUA" and "MTA".  You're breaking the abstraction
by taking on the queuing function of the MTA outside of an MTA.

 > You might want to consider msmtp instead of Postfix on your laptop.

Anything is possible.  But why in the world would I want to do that?
On the one hand, Postfix was near zero effort to install and
configure, and on the other, no problems since.  Furthermore, I use it
on several other machines, besides needing to be able to support it
when Mailman site admins ask questions about Mailman/Postfix
configurations.  My laptop certainly can spare the cycles.

 > > I'm sure you have good reason for doing that,

 > Yes. Emacs on my machine doesn't have Internet access.

I don't care about your reason, that's what I said.  I only care about
what you did, which I have reason to believe was asking for exactly
the kind of trouble you experienced.

 > > but that is a very unusual use case.

 > I certainly hope not! The usual case is that a person's instance of
 > Emacs has access to his data (otherwise, Emacs wouldn't be very
 > useful); if my use case is unusual, that means the usual case is
 > that his Emacs has access not only to his data, but also to the
 > Internet.

Indeed, it does.  That is the usual case, no matter what you hope.
You don't have to like it, but it nevertheless informs the design of
programs like feedmail.

 > With a program like Emacs serving as a bridge, that means
 > his data isn't his data, but instead belongs to some Russian or
 > Chinese hacker, whoever crosses the bridge first, and the original
 > owner is permitted to retain a copy of the data just due to the
 > hacker's mercifulness, or maybe just as bait so the hacker can get
 > even more data later. The years when normal people weren't targets
 > are long gone.

You have evidence for Emacs being used in that way by crackers?  (I
agree with you that Emacs that has an attack surface that amounts to
the whole world, and practically, that securing it is too hard to
think about succeeding, but that's not a popular view on this list.
And it's just theory.)

 > > Bottom line: I think the task you were trying to accomplish is far
 > > more complex that you are admitting, and has little to do with "user
 > > friendliness" of Emacs MUAs.

 > My previous, manual workflow was:

Indeed.  *Your* *manual* workflow, *mediated by two very complex
programs*, the webmail program and the MTA behind it, which have a
traditional and well-defined, but fundamentally arbitrary division of
labor -- which you are not respecting in your redesigned workflow.
Similarly with message-mode and the send-mail module in Emacs.

 > That was inconvenient, so I decided a better workflow would be:
 > 0. Compose a message in Emacs in standard format, with Subject and To headers separated from the body by a blank line.
 > 1. Add the From, Date, and Message-ID headers.
 > 2. Save the message to a file foo in an outbox directory.
 > 3. Run ⌜msmtp -t < outbox/foo⌝
 > 4. If #3 succeeds, then run ⌜mv outbox/foo sent/foo⌝

An excellent analysis, indeed.  So why did you choose an excessively
complex program apparently designed for a different workflow, aka
feedmail.el, to do step 2?  Just save the message using write-file,
with a little extra Lisp to construct an appropriate queuefile name
and to remove MUA artifacts like the header delimiter line.  Add a
tiny shell function to do 3 and 4.  This amount of Lisp would probably
cost less keystrokes and thinking overall, although a bit more design,
than configuring feedmail.

Re: Rant - Emacs mail is not user friendly

[Kelly Dean]

Stephen Berman wrote:
>> Last problem that I do care about is that Emacs keeps inserting line
>> breaks after 72 characters, so I have to switch out of message-mode or
>> copy/paste from another buffer to make it stop doing that because I
>> can't find the setting to turn it off. Why is it turned on by default?
>> Modern email readers do have word-wrap, after all. Even Emacs.
>
> Does customizing message-fill-column do what you want?
Yes, thanks, and David Caldwell pointed out that I can just disable auto-fill-mode. I remember reading about Emacs's hard-wrap ‟filling” feature a couple years ago, but I knew it was a feature I'd never want to use, so I ignored it and forgot what it was called.

Rant - Emacs mail is not user friendly

[Kelly Dean]

Stephen J. Turnbull wrote:
> You have evidence for Emacs being used in that way by crackers?
No.

> agree with you that Emacs that has an attack surface that amounts to
> the whole world, and practically, that securing it is too hard to
> think about succeeding, but that's not a popular view on this list.
Securing Emacs isn't necessary. Just don't connect it to an untrusted network, and don't use it to interpret any untrusted data much more complex than plain text. This doesn't unreasonably limit its functionality; mail delivery can still be done using a separate program that only has access to an outbox directory that Emacs writes to, mail receipt can still be done using a separate program that only has access to an inbox directory that Emacs reads from, web browser bookmark and workspace management can be done in Emacs by just feeding URLs and commands to a browser running in a separate virtual machine that does have Internet access, etc.

> An excellent analysis, indeed.  So why did you choose an excessively
> complex program apparently designed for a different workflow, aka
> feedmail.el, to do step 2?
Because I thought feedmail was responsible for everything that happens when I tell Emacs to send a message, including adding headers and encoding attachments and making the message ready for sending to an SMTP server, leaving just steps #3 and #4 for me to do separately.
So I went to set up it, and it became progressively more frustrating until it goaded me into rant-spamming emacs-devel, though my lack of sleep and overconsumption of pumpkin pie may have had some influence too. Then I discovered that the headers are added and attachments are encoded before the message gets to feedmail, so I gladly dumped feedmail and just wrote a bit of glue to delete the mail-header-separator, save the message to my outbox, and delete the draft.

> Just save the message using write-file,
> with a little extra Lisp to construct an appropriate queuefile name
> and to remove MUA artifacts like the header delimiter line.  Add a
> tiny shell function to do 3 and 4.  This amount of Lisp would probably
> cost less keystrokes and thinking overall, although a bit more design,
> than configuring feedmail.
Er, yes, that's exactly what I did, and I included the code in my message that you just replied to.

My request to the Emacs devs about this now just is: please banish feedmail from mention in the manual, or at least add a ‟here be dragons” warning.

Re: Rant - Emacs mail is not user friendly

[Ted Zlatanov]

On Mon, 17 Nov 2014 10:06:35 +0000 "Kelly Dean" <kelly@prtime.org> wrote: 

KD> Securing Emacs isn't necessary. Just don't connect it to an untrusted
KD> network, and don't use it to interpret any untrusted data much more
KD> complex than plain text.

You realize this is completely unrealistic in today's computing world?

Ted

Re: Rant - Emacs mail is not user friendly

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > (I
  > agree with you that Emacs that has an attack surface that amounts to
  > the whole world, and practically, that securing it is too hard to
  > think about succeeding, but that's not a popular view on this list.
  > And it's just theory.)

We have done substantial work to make Emacs secure against just
visiting a malicious file.  Has a specific flaw or bug been found?  If
so, what precisely?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Richard Stallman writes:

 >   > (I agree with you that Emacs that has an attack surface that
 >   > amounts to the whole world, and practically, that securing it
 >   > is too hard to think about succeeding, but that's not a popular
 >   > view on this list.  And it's just theory.)
 > 
 > We have done substantial work to make Emacs secure against just
 > visiting a malicious file.

Yes.  But Emacs nowadays depends on a large number of external
libraries, many of which are known to have had security flaws.
(Specifically, being able to crash a program is considered a security
flaw, because crashes usually involve trying to execute unexecutable
data -- but if that data happened to be valid machine code, "anything"
can happen).

Emacs is also capable of handling almost any data known to man "out of
the box", which makes it a perfect instrument for "social
engineering".  Emacs users regularly share Lisp code, for example,
including with people who don't know Lisp.  I've done it myself,
though nothing nastier than a time bomb that pops up "Happy Birthday!" 
on the victim's birthday.  I could have had it mail me the contents of
.ssh (and I have a pretty good idea of the individual's preferences in
passphrases, and they are obviously short).

 > Has a specific flaw or bug been found?

Aside from the application/x-patch MIME type used by Gnus, I know of
none.  That one's mostly pedantic, as AFAIK noone proposes to do what
is implied by the "application" MIME type, namely, automatically apply
the patch.  (There's no good reason for diffs sent by mail to be
anything but a "text" MIME type.)

 > If so, what precisely?

As above.  I don't expect you to agree, nor am I perfectly consistent
in following the implications of the analysis.  But I don't install
Emacsen on machines storing other people's data unless they are
isolated from the Internet, and I don't install them on servers I care
about (I use TRAMP instead).

Re: Rant - Emacs mail is not user friendly

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes.  But Emacs nowadays depends on a large number of external
  > libraries, many of which are known to have had security flaws.

We need to get those problems fixed.

I have recruited someone to make a JPG-validator that will
thoroughly check for anything incorrect in a JPG file.
We need to do this for all the formats.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Rant - Emacs mail is not user friendly

[Stefan Monnier]

> I have recruited someone to make a JPG-validator that will
> thoroughly check for anything incorrect in a JPG file.

That's nice, but some security bugs strike even when faced with
a correct/valid JPG file.  What we need to validate is the libjpeg code,
not the JPG files.


        Stefan

Emacs dependencies vs. security

[Ivan Shmakov]

>>>>> Stephen J Turnbull <stephen@xemacs.org> writes:
>>>>> Richard Stallman writes:

 >>> (I agree with you that Emacs that has an attack surface that
 >>> amounts to the whole world, and practically, that securing it is
 >>> too hard to think about succeeding, but that's not a popular view
 >>> on this list.  And it's just theory.)

 >> We have done substantial work to make Emacs secure against just
 >> visiting a malicious file.

 > Yes.  But Emacs nowadays depends on a large number of external
 > libraries, many of which are known to have had security flaws.

	Fortunately, most (if not all) of these libraries are entirely
	optional.  FWIW, the build I use for Emacs development is linked
	against GnuTLS, libxml, the compression libraries (Libz,
	Liblzma), and what seems to be their respective dependencies
	(Glib, libgcrypt, libtasn1, etc.)

[…]

-- 
FSF associate member #7257  http://boycottsystemd.org/  … 3013 B6A0 230E 334A

Re: Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Richard Stallman writes:

 >   > Yes.  But Emacs nowadays depends on a large number of external
 >   > libraries, many of which are known to have had security flaws.
 > 
 > We need to get those problems fixed.

You will never get all of those problems fixed; there's an unending
supply of them, as new ones are being created every day.

That means that in practice complex applications like Emacs are
continuously vulnerable to some attack, although those attacks change
over time.

As somebody (probably Kelly) put it a few messages back, it's no
longer the case that we ordinary users can consider ourselves safe
just because there's millions of us.  The botnets have millions of
CPUs with which to seek out new victims in massively parallel fashion.

Re: Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Stefan Monnier writes:

 > That's nice, but some security bugs strike even when faced with
 > a correct/valid JPG file.  What we need to validate is the libjpeg code,
 > not the JPG files.

No, you need to validate both.  If the input is invalid, anything can
happen.  Viz, X.org, whose libraries do no validation at all (and
whose restrictions are typically undocumented)!

Re: Rant - Emacs mail is not user friendly

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > That's nice, but some security bugs strike even when faced with
  > a correct/valid JPG file.

Is this known to happen now, or is it a speculation, or what?

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Rant - Emacs mail is not user friendly

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > You will never get all of those problems fixed; there's an unending
  > supply of them, as new ones are being created every day.

We still need to fix them.

-- 
Dr Richard Stallman
President, Free Software Foundation
51 Franklin St
Boston MA 02110
USA
www.fsf.org  www.gnu.org
Skype: No way! That's nonfree (freedom-denying) software.
  Use Ekiga or an ordinary phone call.

Re: Rant - Emacs mail is not user friendly

[Stefan Monnier]

>> That's nice, but some security bugs strike even when faced with
>> a correct/valid JPG file.  What we need to validate is the libjpeg code,
>> not the JPG files.
> No, you need to validate both.

Depends on what you mean by "validate the code".  In my line of work,
a code is not validated if it can fail on some input.  I.e. validation
of the code includes checking that the code does the proper validation
of the data it receives.


        Stefan

Re: Rant - Emacs mail is not user friendly

[Stefan Monnier]

>> That's nice, but some security bugs strike even when faced with
>> a correct/valid JPG file.
> Is this known to happen now, or is it a speculation, or what?

I don't keep track of specific security bugs nowadays, but just based on
the size of the code, the fact that it's written in C and it's history,
I don't think I'd take much risk to say that there is such a bug
right now.


        Stefan

Re: Rant - Emacs mail is not user friendly

[Stephen J. Turnbull]

Stefan Monnier writes:

 > Depends on what you mean by "validate the code".  In my line of work,
 > a code is not validated if it can fail on some input.  I.e. validation
 > of the code includes checking that the code does the proper validation
 > of the data it receives.

I was assuming you were talking about the validating the JPEG library
code, because that is what you need to do to determine how much
validation Emacs needs to do of data it's about to hand to libjpeg.

The point of my X.org comment is that there is plenty of code out
there that doesn't validate input well (and some that prides itself on
not validating at all), so Emacs has to do some validation of data.

application/x-patch MIME type used by Gnus? (was: Rant - Emacs mail is not user friendly)

[Reiner Steib]

On Tue, Nov 18 2014, Stephen J. Turnbull wrote:

> Richard Stallman writes:
>  > Has a specific flaw or bug been found?
>
> Aside from the application/x-patch MIME type used by Gnus, I know of
> none.  That one's mostly pedantic, as AFAIK noone proposes to do what
> is implied by the "application" MIME type, namely, automatically apply
> the patch.  (There's no good reason for diffs sent by mail to be
> anything but a "text" MIME type.)

I cannot reproduce this.

emacs -Q / M-x message-mail RET / C-c C-a /tmp/foo.patch RET
results in type="text/x-diff" here.  Maybe your /etc/mime.types
specifies "application/x-patch"?

The only match for "application/x-patch" in Gnus is this one, that
only specifies the coding *if* the type is "application/x-patch":

| grep -nH -e application/x-patch *.el
| mm-encode.el:38:    ("application/x-patch" qp-or-base64)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

application/x-patch MIME type used by Gnus? (was: Rant - Emacs mail is not user friendly)

[Stephen J. Turnbull]

Reiner Steib writes:

 > I cannot reproduce this.
 > 
 > emacs -Q / M-x message-mail RET / C-c C-a /tmp/foo.patch RET
 > results in type="text/x-diff" here.

It might have been application/x-diff; it was a while back.

 > Maybe your /etc/mime.types specifies "application/x-patch"?

Mine is irrelevant.  These were patches received by Mailman at
xemacs.org, which got stripped because we deny application/* MIME
types.  If Gnus has stopped doing that, I'm glad, but at the time
people were trying to tell me that it's a good idea.