Help with sending mail from notmuch-emacs

Help with sending mail from notmuch-emacs

[hgv]

Dear Notmuch list,

I have used notmuch (very happily) for a number of years with alot as my client. I'm now trying out the emacs client (with doom emacs), which is very cool so far. But I am struggling with two particular points around sending emails and cannot solve them on my own. I am also new to emacs, so please forgive my struggles that stem from being a newbie there.

1. Mail gets sent but doesn't save in the correct directory via FCC. I use msmtpq as my sending program. Upon sending, I receive this message: "message-send-mail-with-sendmail: Sending...failed to   mail for [ -oi -f hgv@fastmail.com -t ] : send was successful;" The mail is sent but is not saved in the fcc dir nor entered into the notmuch database. I have enabled emacs debugger (on entry with different functions related to sending) and stepped through the process but didn't see any pertinent information in the messages. My fcc line reads `(setq notmuch-fcc-dirs "fastmail/Sent +sent +inbox +sent-new")` (fuller notmuch-emacs config below). Any guidance on fixing or further troubleshooting (what should I look for on debugger) is appreciated.

2. I have my emacs setup to prefer utf-8 encoding in basically all situations. Emails I send with emacs seem to be correctly encoded. But the Content-Type header is set only to "text/plain" rather than "text/plain; charset=utf-8" (as it is in alot and as it should be). Further, there is no Content-Transfer-Encoding header set. Based on my reading of the emacs message mode docs, Content-Transfer-Encoding should be set on the fly depending on the Content-Type and other factors (to either "quoted-printable" or "base64" for "text/plain"). Perhaps I should not worry about these headers; but it seems like they affect whether mail gets delivered at certain providers and certainly how it displays. Any advice on getting both Content-Type and Content-Transfer-Encoding to set correctly? (Without my settings that prefer utf-8, my emails are ascii encoded and the Content-Type is set as "text/plain; charset=us-ascii" but still no Content-Transfer-Encoding.)

I'm on macos 10.15.7, with notmuch .35 (via homebrew), and GNU emacs 28.1 (from https://github.com/d12frosted/homebrew-emacs-plus via homebrew).

Thank you for your help and for the tremendous software!
hgv

```
;;; $DOOMDIR/config.el -*- lexical-binding: t; -*-

---snip---

;; mainly force utf-8
(prefer-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(set-selection-coding-system 'utf-8)
(set-file-name-coding-system 'utf-8)
(set-clipboard-coding-system 'utf-8)
(set-buffer-file-coding-system 'utf-8)

---snip---

;; gnus-alias
(autoload 'gnus-alias-determine-identity "gnus-alias" "" t)
(add-hook 'message-setup-hook 'gnus-alias-determine-identity)
(setq gnus-alias-identity-alist
  '(("fastmail"
   ---snip---
   ))
;; Use "fastmail" identity by default
(setq gnus-alias-default-identity "fastmail")
;; Define rules to match work identity
(setq gnus-alias-identity-rules
  ---snip---
  ))
;; sorting out regex and replying
(setq gnus-alias-override-user-mail-address t)
(setq gnus-alias-unknown-identity-rule 'default)

;; auto save
(setq message-auto-save-directory "~/.config/emacs/.local/cache/notmuch/unsaved-drafts")
;; supposedly outdated
;; now sourced from language
(setq message-default-charset "utf-8")
(setq mm-coding-system-priorities '(utf-8))

;; notmuch
(after! notmuch
  (setq +notmuch-sync-backend "notmuch new"
    +notmuch-delete-tags '("+ztrash")
    +notmuch-spam-tags '("+spam")
    notmuch-draft-tags '("+draft" "-new")
    notmuch-archive-tags '("-inbox" "-unread"))
  (setq notmuch-mua-compose-in 'new-window)
  (setq notmuch-mua-user-agent-function 'notmuch-mua-user-agent-notmuch)
  (setq message-kill-buffer-on-exit t)
  (setq message-send-mail-function 'message-send-mail-with-sendmail)
  (setq sendmail-program "msmtpq"
    mail-specify-envelope-from t
    message-sendmail-f-is-evil nil
    mail-envelope-from 'header
    message-sendmail-envelope-from 'header)
  (setq notmuch-address-save-filename "~/.config/emacs/.local/notmuch/address-cache")
  (setq notmuch-draft-folder "fastmail/Drafts")
  ;; overwritten from ~/.config/emacs/.local/straight/repos/notmuch/emacs/notmuch-draft.el
  (defun notmuch-draft--mark-deleted ()
    (when notmuch-draft-id
      (notmuch-tag notmuch-draft-id '("+ztrash"))))
  (setq notmuch-fcc-dirs "fastmail/Sent +sent +inbox +sent-new")
  (setq notmuch-identities
  ---snip---
  (setq notmuch-mua-send-hook '(notmuch-mua-attachment-check))
  (setq notmuch-show-empty-saved-searches nil)
  (setq notmuch-multipart/alternative-discouraged '("text/plain" "text/html"))
  (setq notmuch-saved-searches
     ---snip--- 
      )))
```

Re: Help with sending mail from notmuch-emacs

[David Bremner]

hgv <hgv@fastmail.com> writes:


> 1. Mail gets sent but doesn't save in the correct directory via FCC. I
> use msmtpq as my sending program. Upon sending, I receive this
> message: "message-send-mail-with-sendmail: Sending...failed to mail
> for [ -oi -f hgv@fastmail.com -t ] : send was successful;" The mail is
> sent but is not saved in the fcc dir nor entered into the notmuch
> database. I have enabled emacs debugger (on entry with different
> functions related to sending) and stepped through the process but
> didn't see any pertinent information in the messages. My fcc line
> reads `(setq notmuch-fcc-dirs "fastmail/Sent +sent +inbox +sent-new")`
> (fuller notmuch-emacs config below). Any guidance on fixing or further
> troubleshooting (what should I look for on debugger) is appreciated.

What is your setting for the variable

     notmuch-maildir-use-notmuch-insert

It should be t (the default) for that style of notmuch-fcc-dirs setting.

Re: Help with sending mail from notmuch-emacs

[hgv]

Quoting David Bremner <david@tethera.net> (2022-05-09 18: 13:31)

> What is your setting for the variable
> 
>      notmuch-maildir-use-notmuch-insert
> 
> It should be t (the default) for that style of notmuch-fcc-dirs setting.
>

It is set to the default value (t). AFAIK, everything notmuch in my emacs is default except for the values in my config. Doom emacs (https://github.com/doomemacs/doomemacs/blob/master/modules/email/notmuch/config.el) sets some variables; but I have overwritten the pertinent ones (namely, notmuch-fcc-dirs).

Re: Help with sending mail from notmuch-emacs

[David Bremner]

hgv <hgv@fastmail.com> writes:

> 1. Mail gets sent but doesn't save in the correct directory via FCC. I
> use msmtpq as my sending program. Upon sending, I receive this
> message: "message-send-mail-with-sendmail: Sending...failed to mail
> for [ -oi -f hgv@fastmail.com -t ] : send was successful;" The mail is
> sent but is not saved in the fcc dir nor entered into the notmuch
> database. I have enabled emacs debugger (on entry with different
> functions related to sending) and stepped through the process but
> didn't see any pertinent information in the messages. My fcc line
> reads `(setq notmuch-fcc-dirs "fastmail/Sent +sent +inbox +sent-new")`
> (fuller notmuch-emacs config below). Any guidance on fixing or further
> troubleshooting (what should I look for on debugger) is appreciated.

Did you try setting a break on notmuch-maildir-fcc-with-notmuch-insert?
I'm guessing that it doesn't get called because the output of msmtpq is
misinterpreted as an error. I think this is a bug in msmtpq (or at least
a way in which it differs from a traditional sendmail). For me (using
nullmailer) "/usr/lib/sendmail -oi -f david@tethera.net -t < foo.eml"
generates no output on success. If you can change / replace the msmtpq
script, you could try commenting out the log line [1] that is printing
"send was successful".

Another test (which would be worthwhile for your second issue as well),
is to test with M-x message-mail (however one calls a command in doom).
This will eliminate the possibility of some setting of notmuch confusing
things. Of course it still won't insert into the notmuch database, but
if it is working you should not see a message about "Sending...failed"

David

[1] Maybe line 503?

Re: Help with sending mail from notmuch-emacs

[hgv]

Quoting David Bremner <david@tethera.net> (2022-05-12 17:47:23)
> hgv <hgv@fastmail.com> writes:
> 
> Did you try setting a break on notmuch-maildir-fcc-with-notmuch-insert?
> I'm guessing that it doesn't get called because the output of msmtpq is
> misinterpreted as an error. I think this is a bug in msmtpq (or at least
> a way in which it differs from a traditional sendmail). For me (using
> nullmailer) "/usr/lib/sendmail -oi -f david@tethera.net -t < foo.eml"
> generates no output on success. If you can change / replace the msmtpq
> script, you could try commenting out the log line [1] that is printing
> "send was successful".
> 
> Another test (which would be worthwhile for your second issue as well),
> is to test with M-x message-mail (however one calls a command in doom).
> This will eliminate the possibility of some setting of notmuch confusing
> things. Of course it still won't insert into the notmuch database, but
> if it is working you should not see a message about "Sending...failed"
> 
> David
> 
> [1] Maybe line 503?
> 

Thank you, David, for the continued help. You were right to focus on the msmtpq script! And I'm sorry I didn't look there before. There is a simple option to set the script to quiet, which solves the issue.

I'm still struggling with the Content-Transfer-Encoding header and the "charset=utf-8" half of the Content-Type header, both of which I cannot get set in my emails, if anyone has any thoughts!

Re: Help with sending mail from notmuch-emacs

[David Bremner]

hgv <hgv@fastmail.com> writes:
>
> I'm still struggling with the Content-Transfer-Encoding header and the
> "charset=utf-8" half of the Content-Type header, both of which I
> cannot get set in my emails, if anyone has any thoughts!

My only (not so helpful) thought is that this almost certainly an issue
outside notmuch in message-mode. If you can duplicate it there (M-x
message-mail) you can get help from the wider community of message-mode
users.

Re: Help with sending mail from notmuch-emacs

[HGV]

Thanks, David, for your continued help. I believe I have sorted this out and just
wanted to note what I found here for posterity in case others looking at the
archives have my same question.

Once I determined that indeed my issues with Content-Transfer-Encoding and
Content-Type headers happened with message-mode outside of notmuch, I read more
about how message-mode, MIME, and the rest of the sending chain all work
together. The problem seems to be that I was sending simple test emails to
myself that had only ASCII characters. Despite my own utf-8 encoding,
message-mode recognized that the files only had ASCII characters and set the
Content-Type header as just "text/plain" or "text/plain; charset=ascii" (still
not sure about the charset aspect). And with only one ASCII-only plain-text
part, Content-Transfer-Encoding was not necessary and therefore not set. When I
use a utf-8 character (e.g., Süßwasser), both the Content-Type and
Content-Transfer-Encoding were correctly set.

Thanks again for the tremendous software and help.

Yours kindly,
HGV\r

Re: Help with sending mail from notmuch-emacs

[David Bremner]

HGV <hgv@fastmail.com> writes:

> When I use a utf-8 character (e.g., Süßwasser), both the Content-Type
> and Content-Transfer-Encoding were correctly set.

Glad you got it working. Debugging encoding issues in Emacs is not very
fun in my experience.

d
\r