unofficial mirror of notmuch@notmuchmail.org
 help / color / mirror / code / Atom feed
* [RFC] writing HTML email with notmuch
@ 2019-02-25  1:52 Antoine Beaupré
  2019-02-25 18:22 ` Istvan Marko
                   ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-25  1:52 UTC (permalink / raw)
  To: notmuch

[-- Attachment #1: Type: text/plain, Size: 2619 bytes --]

Hi,

TL;DR: magic recipe to include an HTML version when writing plaintext.

I know, I know, HTML email is "evil"[1]. I mostly never ever use it, in
fact, I don't remember the last time I consciously sent HTML. Maybe I
did so back when I was using Netscape Communicator[2][3], but whatever.

The reason I thought about this again is I have been doing more
photography these days and, well, being allergic to social media, I have
very few ways of sharing those photographs with families and friends. I
have tried creating a gallery website with an RSS feed but I'm sure no
one here will be surprised that the uptake is minimal, if
non-existent. People expect to have stuff *pushed* to them, like
Instagram, Facebook, Twitter or Spam does.

So I thought[4] of Email again: the original social network! I figured I
would just make a mailing list, and write to my people once in a while
to let them new about my new pictures. And while writing the first
email, I realized it was pretty silly to not include images, or at least
*links* to images in the email.

I'm sure you can see where this is going. A link in the email: who's
going to click that. Who clicks now anyways, with all the tapping[5]
going on. So the answer comes naturally: just write frigging HTML
email. Don't be a rms^Wreligious zealot and do the right thing, what
works basically everywhere[6] (even notmuch!).

So I started Thunderbird and thought "what the heck am I doing! there
must be a better way!" After searching for "message mode emacs html
email ktxbye", I found some people already thought about this problem
and came up with somewhat elegant solutions[7]. I built on that by
trying to come up with a pure elisp solution, which goes a little like
this:

(defun anarcat/notmuch-html-convert ()
  """create an HTML part from a Markdown body

This will not work if there are *any* attachments of any form, those should be added after."""
  (interactive)
  (save-excursion
    ;; wrap signature in a <pre>
    (message-goto-signature)
    (setq signature-position (point))
    (forward-line -1)
    ;; GFM markers for pre, used because easier to undo than the
    ;; "prefix by 4 characters" standard
    (insert "```")
    (end-of-buffer)
    (insert "```")
    ;; set region to top of body then end of buffer
    (end-of-buffer)
    (message-goto-body)
    (narrow-to-region (point) (mark))
    ;; run markdown on region
    (setq output-buffer-name "*notmuch-markdown-output*")
    (markdown output-buffer-name)
    (widen)
    (save-excursion
      (set-buffer output-buffer-name)
      (markdown-add-xhtml-header-and-footer ""))
    (insert "

[-- Attachment #2.1: Type: text/plain, Size: 38 bytes --]

\n")
    (end-of-buffer)
    (insert "

[-- Attachment #2.2: Type: text/html, Size: 57 bytes --]

[-- Attachment #3: Type: text/plain, Size: 2201 bytes --]

    ;; remove Markdown <pre> markings
    (goto-char signature-position)
    (while (re-search-forward "^```" nil t)
      (replace-match ""))))

For those who can't read elisp for breakfast, this does the following:

 1. parse the current email body as markdown, in a separate buffer
 2. make the current email multipart/alternative
 3. add an HTML part
 4. inject the HTML version in the HTML part

There's some nasty business with formatting the signature correctly by
wrapping it in a <pre> that's going on there - I took that from
Thunderbird as well.

(For those who *do* read elisp for breakfast, improvements and comments
on the coding style are very welcome.)

The idea is that you write your email normally, but in markdown. When
you're done writing that email, you launch the above function (carefully
bound to "M-x anarcat/notmuch-html-convert" here) which takes that email
and adds an equivalent HTML part to it. You can then even tweak that
part to screw around with the raw HTML if you feel depressed or
nostalgic.

What do people think? Am I insane? Could this work? Does this belong in
notmuch? Or maybe in the tips section? Should I seek therapy? Do you
hate markdown? Expand on the relationship between your parents and text
editors.

Thanks for any feedback,

A.

PS: the above, naturally, could be adapted to parse the body as RST,
asciidoc, texinfo, latex or whatever insanity you think would be more
appropriate, I don't care. The idea is the same.

PPS: I remember reading about someone wanting to declare a text/markdown
mimetype for email, and remembering it was all backwards and weird and I
can't find the reference anymore. If some lazyweb magic person could
forward the link to me I would be grateful.

 [1]: one of so many: https://www.georgedillon.com/web/html_email_is_evil_still.shtml
 [2]: https://en.wikipedia.org/wiki/Netscape_Communicator
 [3]: yes my age is showing
 [4]: to be fair, this article encouraged me quite a bit:
 https://blog.chaddickerson.com/2019/01/09/replacing-facebook/
 [5]: not the bass guitar one, unfortunately
 [6]: https://en.wikipedia.org/wiki/HTML_email#Adoption
 [7]: https://trey-jackson.blogspot.com/2008/01/emacs-tip-8-markdown.html

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
  2019-02-25  1:52 [RFC] writing HTML email with notmuch Antoine Beaupré
@ 2019-02-25 18:22 ` Istvan Marko
  2019-02-25 18:56 ` Brian Sniffen
  2019-02-25 21:18 ` Daniel Kahn Gillmor
  2 siblings, 0 replies; 12+ messages in thread
From: Istvan Marko @ 2019-02-25 18:22 UTC (permalink / raw)
  To: Antoine Beaupré, notmuch

Antoine Beaupré <anarcat@orangeseeds.org> writes:

> (defun anarcat/notmuch-html-convert ()
>   """create an HTML part from a Markdown body

Nice. There is also https://github.com/org-mime/org-mime (in melpa as
"org-mime") which does something similar but with Org instead of
Markdown. 

-- 
	Istvan

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
  2019-02-25  1:52 [RFC] writing HTML email with notmuch Antoine Beaupré
  2019-02-25 18:22 ` Istvan Marko
@ 2019-02-25 18:56 ` Brian Sniffen
  2019-02-25 22:58   ` Antoine Beaupré
  2019-02-25 21:18 ` Daniel Kahn Gillmor
  2 siblings, 1 reply; 12+ messages in thread
From: Brian Sniffen @ 2019-02-25 18:56 UTC (permalink / raw)
  To: Antoine Beaupré; +Cc: notmuch

[-- Attachment #1: Type: text/plain, Size: 5603 bytes --]

I appreciate the signature handling. But perhaps you used it on this message and it ate part of this line?

> (insert "
>    ;; remove Markdown <pre> markings

-- 
Brian Sniffen

> On Feb 24, 2019, at 8:52 PM, Antoine Beaupré <anarcat@orangeseeds.org> wrote:
> 
> Hi,
> 
> TL;DR: magic recipe to include an HTML version when writing plaintext.
> 
> I know, I know, HTML email is "evil"[1]. I mostly never ever use it, in
> fact, I don't remember the last time I consciously sent HTML. Maybe I
> did so back when I was using Netscape Communicator[2][3], but whatever.
> 
> The reason I thought about this again is I have been doing more
> photography these days and, well, being allergic to social media, I have
> very few ways of sharing those photographs with families and friends. I
> have tried creating a gallery website with an RSS feed but I'm sure no
> one here will be surprised that the uptake is minimal, if
> non-existent. People expect to have stuff *pushed* to them, like
> Instagram, Facebook, Twitter or Spam does.
> 
> So I thought[4] of Email again: the original social network! I figured I
> would just make a mailing list, and write to my people once in a while
> to let them new about my new pictures. And while writing the first
> email, I realized it was pretty silly to not include images, or at least
> *links* to images in the email.
> 
> I'm sure you can see where this is going. A link in the email: who's
> going to click that. Who clicks now anyways, with all the tapping[5]
> going on. So the answer comes naturally: just write frigging HTML
> email. Don't be a rms^Wreligious zealot and do the right thing, what
> works basically everywhere[6] (even notmuch!).
> 
> So I started Thunderbird and thought "what the heck am I doing! there
> must be a better way!" After searching for "message mode emacs html
> email ktxbye", I found some people already thought about this problem
> and came up with somewhat elegant solutions[7]. I built on that by
> trying to come up with a pure elisp solution, which goes a little like
> this:
> 
> (defun anarcat/notmuch-html-convert ()
>  """create an HTML part from a Markdown body
> 
> This will not work if there are *any* attachments of any form, those should be added after."""
>  (interactive)
>  (save-excursion
>    ;; wrap signature in a <pre>
>    (message-goto-signature)
>    (setq signature-position (point))
>    (forward-line -1)
>    ;; GFM markers for pre, used because easier to undo than the
>    ;; "prefix by 4 characters" standard
>    (insert "```")
>    (end-of-buffer)
>    (insert "```")
>    ;; set region to top of body then end of buffer
>    (end-of-buffer)
>    (message-goto-body)
>    (narrow-to-region (point) (mark))
>    ;; run markdown on region
>    (setq output-buffer-name "*notmuch-markdown-output*")
>    (markdown output-buffer-name)
>    (widen)
>    (save-excursion
>      (set-buffer output-buffer-name)
>      (markdown-add-xhtml-header-and-footer ""))
>    (insert "
> \n") (insert-buffer output-buffer-name) (insert "
>    ;; remove Markdown <pre> markings
>    (goto-char signature-position)
>    (while (re-search-forward "^```" nil t)
>      (replace-match ""))))
> 
> For those who can't read elisp for breakfast, this does the following:
> 
> 1. parse the current email body as markdown, in a separate buffer
> 2. make the current email multipart/alternative
> 3. add an HTML part
> 4. inject the HTML version in the HTML part
> 
> There's some nasty business with formatting the signature correctly by
> wrapping it in a <pre> that's going on there - I took that from
> Thunderbird as well.
> 
> (For those who *do* read elisp for breakfast, improvements and comments
> on the coding style are very welcome.)
> 
> The idea is that you write your email normally, but in markdown. When
> you're done writing that email, you launch the above function (carefully
> bound to "M-x anarcat/notmuch-html-convert" here) which takes that email
> and adds an equivalent HTML part to it. You can then even tweak that
> part to screw around with the raw HTML if you feel depressed or
> nostalgic.
> 
> What do people think? Am I insane? Could this work? Does this belong in
> notmuch? Or maybe in the tips section? Should I seek therapy? Do you
> hate markdown? Expand on the relationship between your parents and text
> editors.
> 
> Thanks for any feedback,
> 
> A.
> 
> PS: the above, naturally, could be adapted to parse the body as RST,
> asciidoc, texinfo, latex or whatever insanity you think would be more
> appropriate, I don't care. The idea is the same.
> 
> PPS: I remember reading about someone wanting to declare a text/markdown
> mimetype for email, and remembering it was all backwards and weird and I
> can't find the reference anymore. If some lazyweb magic person could
> forward the link to me I would be grateful.
> 
> [1]: one of so many: https://www.georgedillon.com/web/html_email_is_evil_still.shtml
> [2]: https://en.wikipedia.org/wiki/Netscape_Communicator
> [3]: yes my age is showing
> [4]: to be fair, this article encouraged me quite a bit:
> https://blog.chaddickerson.com/2019/01/09/replacing-facebook/
> [5]: not the bass guitar one, unfortunately
> [6]: https://en.wikipedia.org/wiki/HTML_email#Adoption
> [7]: https://trey-jackson.blogspot.com/2008/01/emacs-tip-8-markdown.html
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

[-- Attachment #2: Type: text/html, Size: 8962 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
  2019-02-25  1:52 [RFC] writing HTML email with notmuch Antoine Beaupré
  2019-02-25 18:22 ` Istvan Marko
  2019-02-25 18:56 ` Brian Sniffen
@ 2019-02-25 21:18 ` Daniel Kahn Gillmor
       [not found]   ` <87lg23ldrl.fsf@curie.anarc.at>
  2 siblings, 1 reply; 12+ messages in thread
From: Daniel Kahn Gillmor @ 2019-02-25 21:18 UTC (permalink / raw)
  To: Antoine Beaupré, notmuch

[-- Attachment #1: Type: text/plain, Size: 1670 bytes --]

Hi Antoine--

thanks for an entertaining and thoughtful read.

Your elisp rendered really strangely for me in notmuch mode in emacs:

     (widen)
     (save-excursion
       (set-buffer output-buffer-name)
       (markdown-add-xhtml-header-and-footer ""))
     (insert "
 [ multipart/alternative ]
  [ text/plain ]
  \n")
      (end-of-buffer)
      (insert "
  [ text/html (hidden) ]
\n") (insert-buffer output-buffer-name) (insert "
 [ text/plain ]
     ;; remove Markdown <pre> markings
     (goto-char signature-position)

(note the []-delimited emacs buttons above)

The issue here appears to be that the MIME boundaries in your message (i
won't repeat them here because i don't want to break my own message) are
getting mixed up/confused somehow with the mime boundaries you include
in the elisp.

I'd be happy to compare raw files with you at some point if you want to
try to make sense of this.  I'm hoping that the error is in the message
generation (that your sending MUA garbled your elisp) rather than in the
receipt, because if the receiving side is at fault it's a much worse
security risk.

On Sun 2019-02-24 20:52:40 -0500, Antoine Beaupré wrote:
> PPS: I remember reading about someone wanting to declare a text/markdown
> mimetype for email, and remembering it was all backwards and weird and I
> can't find the reference anymore. If some lazyweb magic person could
> forward the link to me I would be grateful.

https://tools.ietf.org/html/rfc7763 -- I'm sure Sean Leonard (the author
of this informational RFC) would be open to discussion about what's
missing or what could be improved.

        --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
  2019-02-25 18:56 ` Brian Sniffen
@ 2019-02-25 22:58   ` Antoine Beaupré
  0 siblings, 0 replies; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-25 22:58 UTC (permalink / raw)
  To: Brian Sniffen; +Cc: notmuch

[-- Attachment #1: Type: text/plain, Size: 821 bytes --]

On 2019-02-25 13:56:31, Brian Sniffen wrote:
> I appreciate the signature handling. But perhaps you used it on this message and it ate part of this line?
>
>> (insert "
>>    ;; remove Markdown <pre> markings

Hehe... I think what happened is what Daniel identified in another
reply, that Emacs noticed the `#part` markers and interpreted it as
such, which is rather silly (shouldn't those be matched only on empty
lines?)

Anyways, here's another version which also includes the proper <title>
(based on the subject) in the HTML document (not that that really
matters) and removes extraneous sig handling that was leftover.

A.

-- 
For once you have tasted flight,
You will walk the earth with your eyes turned skyward;
For there you have been,
And there you long to return.
                        - Leonardo da Vinci

[-- Attachment #2: notmuch-config.el --]
[-- Type: application/emacs-lisp, Size: 8820 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
       [not found]   ` <87lg23ldrl.fsf@curie.anarc.at>
@ 2019-02-25 23:01     ` Antoine Beaupré
  2019-02-25 23:15       ` Daniel Kahn Gillmor
  0 siblings, 1 reply; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-25 23:01 UTC (permalink / raw)
  To: Daniel Kahn Gillmor; +Cc: notmuch

I seem to have forgotten to CC the list in my reply, sorry for the
noise.

I elided the attachment here because I sent a newer version in
87r2bvjx02.fsf@curie.anarc.at.

A.

On 2019-02-25 17:11:26, Antoine Beaupré wrote:
> On 2019-02-25 16:18:03, Daniel Kahn Gillmor wrote:
>> Hi Antoine--
>>
>> thanks for an entertaining and thoughtful read.
>>
>> Your elisp rendered really strangely for me in notmuch mode in emacs:
>>
>>      (widen)
>>      (save-excursion
>>        (set-buffer output-buffer-name)
>>        (markdown-add-xhtml-header-and-footer ""))
>>      (insert "
>>  [ multipart/alternative ]
>>   [ text/plain ]
>>   \n")
>>       (end-of-buffer)
>>       (insert "
>>   [ text/html (hidden) ]
>> \n") (insert-buffer output-buffer-name) (insert "
>>  [ text/plain ]
>>      ;; remove Markdown <pre> markings
>>      (goto-char signature-position)
>>
>> (note the []-delimited emacs buttons above)
>>
>> The issue here appears to be that the MIME boundaries in your message (i
>> won't repeat them here because i don't want to break my own message) are
>> getting mixed up/confused somehow with the mime boundaries you include
>> in the elisp.
>>
>> I'd be happy to compare raw files with you at some point if you want to
>> try to make sense of this.  I'm hoping that the error is in the message
>> generation (that your sending MUA garbled your elisp) rather than in the
>> receipt, because if the receiving side is at fault it's a much worse
>> security risk.
>
> Haha... That's actually kind of hilarious.
>
> I can confirm the badness occured on send here. I've attached my
> precious notmuch-config.el, which has that function (and many more!) as
> a real attachement which should workaround those issues.
>
> ;; autocompletion
> (eval-after-load "notmuch-address"
>   '(progn
>      (notmuch-address-message-insinuate)))
>
> ; don't remember what that is
> (add-hook 'notmuch-show-hook 'visual-line-mode)
>
> (defun notmuch-load-bug (bug)
>   "download and show a Debian bug report with notmuch-slurp-debbug"
>   (interactive "MDebian bug number: ")
>   (message "downloading debian bug %s..." bug)
>   (call-process "notmuch-slurp-debbug" nil nil nil bug)
>   (message "running notmuch-poll...")
>   (notmuch-poll)
>   (message "searching for bug %s..." bug)
>   (notmuch-search bug)
>   (message "bug %s loaded, but really you should use upstream notmuch-slurp-debbug" bug))
>
> ;; attachment checks
> ;;
> ;; should be sent upstream, but needs unit tests in test/T310-emacs.sh
> (defcustom notmuch-message-attach-regex
>   "\\b\\(attache\?ment\\|attached\\|attach\\|pi[¨e]ce\s+jointe?\\)\\b"
>   "Pattern of text announcing there should be an attachment.
>
> This is used by `notmuch-message-check-attach' to check email
> bodies for words that might indicate the email should have an
> attachement. If the pattern matches and there is no attachment (a
> `<#part ...>' magic block), notmuch will show a confirmation
> prompt before sending the email.
>
> The default regular expression is deliberately liberal: we prefer
> false positive than forgotten attachments. This should be
> customized for non-english languages and notmuch welcomes
> additions to the pattern for your native language, unless it
> conflicts with common words in other languages."
>   :type '(regexp)
>   :group 'notmuch-send)
>
> (defun notmuch-message-check-attach ()
>   """Check for missing attachments.
>
> This is normally added to `message-send-hook' and is configured
> through `notmuch-message-attach-regex'."""
>   (save-excursion ;; XXX: this fails somehow: point is at the end of the buffer on error
>     (goto-char (point-min))
>     (if (re-search-forward notmuch-message-attach-regex nil t)
>         (progn
>           (goto-char (point-min))
>           (unless (re-search-forward "<#part [^>]*filename=[^>]*>" nil t)
>             (or (y-or-n-p "Email seem to refer to attachment, but nothing attached, send anyways?")
>                 (error "No attachment found, aborting")))))))
>
> (defcustom notmuch-mua-attachment-regexp
>   "\\b\\(attache\?ment\\|attached\\|attach\\|pi[¨e]ce\s+jointe?\\)\\b"
>   "Message body text indicating that an attachment is expected.
>
> This is not used unless `notmuch-mua-attachment-check' is added
> to `notmuch-mua-send-hook'.")
>
> (defun notmuch-mua-attachment-check ()
>   "Signal an error if the message text indicates that an
> attachment is expected but no MML referencing an attachment is
> found.
>
> Typically this is added to `notmuch-mua-send-hook'."
>   (when (and
> 	 ;; When the message mentions attachment...
> 	 (save-excursion
> 	   (message-goto-body)
> 	   (loop while (re-search-forward notmuch-mua-attachment-regexp (point-max) t)
> 		 ;; For every instance of the "attachment" string
> 		 ;; found, examine the text properties. If the text
> 		 ;; has either a `face' or `syntax-table' property
> 		 ;; then it is quoted text and should *not* cause the
> 		 ;; user to be asked about a missing attachment.
> 		 if (let ((props (text-properties-at (match-beginning 0))))
> 		      (not (or (memq 'syntax-table props)
> 			       (memq 'face props))))
> 		 return t
> 		 finally return nil))
> 	 ;; ...but doesn't have a part with a filename...
> 	 (save-excursion
> 	   (message-goto-body)
> 	   (not (re-search-forward "^<#part [^>]*filename=" nil t)))
> 	 ;; ...and that's not okay...
> 	 (not (y-or-n-p "Attachment mentioned, but no attachment - is that okay?")))
>     ;; ...signal an error.
>     (error "Missing attachment")))
>
> (add-hook 'message-send-hook 'notmuch-mua-attachment-check)
> (add-hook 'message-send-hook 'notmuch-draft--mark-deleted)
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> ;;; keymappings
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> (define-key notmuch-show-mode-map "S"
>   (lambda ()
>     "mark message as spam and advance"
>     (interactive)
>     (notmuch-show-tag '("+spam" "-inbox" "-ham" "-unread" "-flagged"))
>     (notmuch-show-next-open-message t)))
>
> (define-key notmuch-search-mode-map "S"
>   (lambda (&optional beg end)
>     "mark message as spam and advance"
>     (interactive (notmuch-search-interactive-region))
>     (notmuch-search-tag (list "+spam" "-inbox" "-ham" "-unread" "-flagged") beg end t)
>     (anarcat/notmuch-search-next-thread)))
>
> (define-key notmuch-show-mode-map "H"
>   (lambda ()
>     "mark message as ham and advance"
>     (interactive)
>     (notmuch-show-tag '("-spam" "-greyspam" "+inbox" "+ham" "+flagged"))
>     (notmuch-show-next-open-message t)))
>
> (define-key notmuch-search-mode-map "H"
>   (lambda (&optional beg end)
>     "mark message as ham and advance"
>     (interactive (notmuch-search-interactive-region))
>     (notmuch-search-tag (list "-spam" "-greyspam" "+inbox" "+ham" "+flagged") beg end t)
>     (anarcat/notmuch-search-next-thread)))
>
> (define-key notmuch-search-mode-map "u"
>   (lambda (&optional beg end)
>     "undelete and advance"
>     (interactive (notmuch-search-interactive-region))
>     (notmuch-search-tag (list "-deleted") beg end t)
>     (anarcat/notmuch-search-next-thread)))
>
> (define-key notmuch-search-mode-map "d"
>   (lambda (&optional beg end)
>     "delete and advance"
>     (interactive (notmuch-search-interactive-region))
>     (notmuch-search-tag (list "+deleted" "-unread") beg end t)
>     (anarcat/notmuch-search-next-thread)))
>
> (define-key notmuch-show-mode-map "d"
>   (lambda ()
>     "delete current message and advance"
>     (interactive)
>     (notmuch-show-tag '("+deleted" "-unread"))
>     (notmuch-show-next-open-message t)))
>
> ;; https://notmuchmail.org/emacstips/#index17h2
> (define-key notmuch-show-mode-map "b"
>   (lambda (&optional address)
>     "Bounce the current message."
>     (interactive "sBounce To: ")
>     (notmuch-show-view-raw-message)
>     (message-resend address)
>     (kill-buffer)))
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> ;;; my custom notmuch functions
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> (defun anarcat/notmuch-search-next-thread ()
>   "Skip to next message from region or point
>
> This is necessary because notmuch-search-next-thread just starts
> from point, whereas it seems to me more logical to start from the
> end of the region."
>   ;; move line before the end of region if there is one
>   (unless (= beg end)
>     (goto-char (- end 1)))
>   (notmuch-search-next-thread))
>
> ;; Linking to notmuch messages from org-mode
> ;; https://notmuchmail.org/emacstips/#index23h2
> (require 'org-notmuch nil t)
> (autoload 'gnus-alias-determine-identity "gnus-alias" "" t)
> (add-hook 'message-setup-hook 'gnus-alias-determine-identity)
>
> (defun anarcat/notmuch-html-convert ()
>   """create an HTML part from a Markdown body
>
> This will not work if there are *any* attachments of any form, those should be added after."""
>   (interactive)
>   (save-excursion
>     ;; wrap signature in a <pre>
>     (message-goto-signature)
>     (forward-line -1)
>     ;; save and delete signature which requires special formatting
>     (setq signature (buffer-substring (point) (point-max)))
>     (delete-region (point) (point-max))
>     ;; set region to top of body then end of buffer
>     (end-of-buffer)
>     (message-goto-body)
>     (narrow-to-region (point) (mark))
>     ;; run markdown on region
>     (setq output-buffer-name "*notmuch-markdown-output*")
>     (markdown output-buffer-name)
>     (widen)
>     (save-excursion
>       (set-buffer output-buffer-name)
>       (end-of-buffer)
>       ;; add signature formatted as <pre>
>       (insert "\n<pre>")
>       (insert signature)
>       (insert "</pre>\n")
>       (markdown-add-xhtml-header-and-footer ""))
>     ;; restore signature
>     (message-goto-signature)
>     (insert signature)
>     (message-goto-body)
>     (insert "<#multipart type=alternative>\n")
>     (end-of-buffer)
>     (insert "<#part type=text/html>\n")
>     (insert-buffer output-buffer-name)
>     (end-of-buffer)
>     (insert "<#/multipart>\n")
>     ;; remove Markdown <pre> markings
>     (goto-char signature-position)
>     (while (re-search-forward "^```" nil t)
>       (replace-match ""))))
>
> (message "anarcat's custom notmuch config loaded")
>
>> On Sun 2019-02-24 20:52:40 -0500, Antoine Beaupré wrote:
>>> PPS: I remember reading about someone wanting to declare a text/markdown
>>> mimetype for email, and remembering it was all backwards and weird and I
>>> can't find the reference anymore. If some lazyweb magic person could
>>> forward the link to me I would be grateful.
>>
>> https://tools.ietf.org/html/rfc7763 -- I'm sure Sean Leonard (the author
>> of this informational RFC) would be open to discussion about what's
>> missing or what could be improved.
>
> I remember reading that RFC and not understanding the point of it, to be
> honest. I mean it's great to mark certain content as markdown - I
> sometimes did that when attaching drafts to my editor, but really I
> would be worried about marking (say) this email as text/markdown because
> I'd be afraid most MUAs wouldn't display it correctly.
>
> But this wasn't the article I had in mind - it was something about doing
> the opposite conversion, if my memory is correct.
>
> A.
> -- 
> I've got to design so you can put it together out of garbage cans. In
> part because that's what I started from, but mostly because I don\x19t
> trust the industrial structure\x14they might decide to suppress us
> weirdos and try to deny us the parts we need.
>                        - Lee Felsenstein

-- 
My passionate sense of social justice and social responsibility has
always contrasted oddly with my pronounced lack of need for direct
contact with other human beings and communities. I am truly a "lone
traveler" and have never belonged to my country, my home, my friends,
or even my immediate family, with my whole heart; in the face of all
these ties, I have never lost a sense of distance and a need for
solitude.
                       - Albert Einstein

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: [RFC] writing HTML email with notmuch
  2019-02-25 23:01     ` Antoine Beaupré
@ 2019-02-25 23:15       ` Daniel Kahn Gillmor
  2019-02-26  5:33         ` non-ascii email forwarding failures Antoine Beaupré
  0 siblings, 1 reply; 12+ messages in thread
From: Daniel Kahn Gillmor @ 2019-02-25 23:15 UTC (permalink / raw)
  To: Antoine Beaupré; +Cc: notmuch

[-- Attachment #1: Type: text/plain, Size: 1117 bytes --]

On Mon 2019-02-25 18:01:55 -0500, Antoine Beaupré wrote:
> I seem to have forgotten to CC the list in my reply, sorry for the
> noise.
>
> I elided the attachment here because I sent a newer version in
> 87r2bvjx02.fsf@curie.anarc.at.

thanks!

> On 2019-02-25 17:11:26, Antoine Beaupr wrote:

Interesting, your MUA is relying on the default charset=iso-8859-1
(latin-1) or is it charset=iso-8859-15 (latin-7)?

it's sending me octal 0o351 as the last letter of your last name, which
my notmuch-emacs renders as a raw octet ("Beaupr\351"), maybe because
i'm in a UTF-8 locale?

wtf, why are we still failing at this in 2019, with updated MUAs?  i
guess it might be my own fault for having this line in my
custom-set-variables in ~/.emacs:

'(message-default-charset (quote utf-8))

but the docs for message-default-charset say:

  This variable is obsolete since 26.1;
  The default charset comes from the language environment

shouldn't emacs mml mode explicitly mark the charset in the Content-Type
header when generating a text/plain part no matter what?

grumpily,

     --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

^ permalink raw reply	[flat|nested] 12+ messages in thread

* non-ascii email forwarding failures
  2019-02-25 23:15       ` Daniel Kahn Gillmor
@ 2019-02-26  5:33         ` Antoine Beaupré
  2019-02-26  5:56           ` Daniel Kahn Gillmor
  2019-02-26 10:56           ` David Edmondson
  0 siblings, 2 replies; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-26  5:33 UTC (permalink / raw)
  To: Daniel Kahn Gillmor; +Cc: notmuch

On 2019-02-25 18:15:21, Daniel Kahn Gillmor wrote:
> On Mon 2019-02-25 18:01:55 -0500, Antoine Beaupré wrote:
>> I seem to have forgotten to CC the list in my reply, sorry for the
>> noise.
>>
>> I elided the attachment here because I sent a newer version in
>> 87r2bvjx02.fsf@curie.anarc.at.
>
> thanks!
>
>> On 2019-02-25 17:11:26, Antoine Beaupr wrote:
>
> Interesting, your MUA is relying on the default charset=iso-8859-1
> (latin-1) or is it charset=iso-8859-15 (latin-7)?
>
> it's sending me octal 0o351 as the last letter of your last name, which
> my notmuch-emacs renders as a raw octet ("Beaupr\351"), maybe because
> i'm in a UTF-8 locale?
>
> wtf, why are we still failing at this in 2019, with updated MUAs?  i
> guess it might be my own fault for having this line in my
> custom-set-variables in ~/.emacs:
>
> '(message-default-charset (quote utf-8))
>
> but the docs for message-default-charset say:
>
>   This variable is obsolete since 26.1;
>   The default charset comes from the language environment
>
> shouldn't emacs mml mode explicitly mark the charset in the Content-Type
> header when generating a text/plain part no matter what?

So I have no idea what's going on with mail forwards, but this is the
kind of stuff that happens to me all the time and I can't describe
correctly enough to file a bug.

The gist of it is that, under some weird circumstances, notmuch-emacs
(message-mode.el?) will screw up email forwards in a big time. I just
forwarded (or did I just reply? not sure) that private email to the list
to trigger that bug...

I get that from time to time. I think the key is that it's an email I
*sent* not *received*, which are somewhat different in their storage
mechanism...

The first symptom will be that, when sending, I'd get a prompt like
this:

Non-printable characters found.  Continue sending? (delete, replace, send, edit, ?): 

I never know what to answer to that, i usually just hit "send" or
"PLOKTA". Then I get two more annoying prompts, just to make sure I
really know about the bug. It also reminds me of how I speak a weird
language with weird characters and I don't belong on the american
Internet (capital I):

Message contains characters with unknown encoding.  Really send? (y or n) y
Use ASCII as charset? (y or n) y

ASSCII indeed. Anyways, what you saw is the result of that *amazing*
user experience.

I wish I knew how to fix that or make that more useful to folks who have
more of a clue than me. ;)

(and sorry for the cussin'...)

A.

-- 
Man really attains the state of complete humanity when he produces,
without being forced by physical need to sell himself as a commodity.
                        - Ernesto "Che" Guevara

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: non-ascii email forwarding failures
  2019-02-26  5:33         ` non-ascii email forwarding failures Antoine Beaupré
@ 2019-02-26  5:56           ` Daniel Kahn Gillmor
  2019-02-26  7:51             ` Antoine Beaupré
  2019-02-26 10:56           ` David Edmondson
  1 sibling, 1 reply; 12+ messages in thread
From: Daniel Kahn Gillmor @ 2019-02-26  5:56 UTC (permalink / raw)
  To: Antoine Beaupré; +Cc: notmuch

On Tue 2019-02-26 00:33:33 -0500, Antoine Beaupré wrote:
> Message contains characters with unknown encoding.  Really send? (y or n) y
> Use ASCII as charset? (y or n) y

The right answer in 2019 is to use UTF-8 as a charset, everywhere.  How
to convince message mode to do that, i don't know.

Presumably your locale is configured as $something.UTF-8 (fr_CA.UTF-8
perhaps?).  Maybe there's some other setting that would also be useful,
though.  hopefully someone with deeper emacs-fu can weigh in here.

        --dkg

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: non-ascii email forwarding failures
  2019-02-26  5:56           ` Daniel Kahn Gillmor
@ 2019-02-26  7:51             ` Antoine Beaupré
  0 siblings, 0 replies; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-26  7:51 UTC (permalink / raw)
  To: Daniel Kahn Gillmor; +Cc: notmuch

On 2019-02-26 00:56:53, Daniel Kahn Gillmor wrote:
> On Tue 2019-02-26 00:33:33 -0500, Antoine Beaupré wrote:
>> Message contains characters with unknown encoding.  Really send? (y or n) y
>> Use ASCII as charset? (y or n) y
>
> The right answer in 2019 is to use UTF-8 as a charset, everywhere.  How
> to convince message mode to do that, i don't know.
>
> Presumably your locale is configured as $something.UTF-8 (fr_CA.UTF-8
> perhaps?).

It is.

> Maybe there's some other setting that would also be useful,
> though.  hopefully someone with deeper emacs-fu can weigh in here.

I have no idea. :)

a.

-- 
Non qui parum habet, sed qui plus cupit, pauper est.
It is not the man who has too little, but the man who craves more,
that is poor.            - Lucius Annaeus Seneca (65 AD)

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: non-ascii email forwarding failures
  2019-02-26  5:33         ` non-ascii email forwarding failures Antoine Beaupré
  2019-02-26  5:56           ` Daniel Kahn Gillmor
@ 2019-02-26 10:56           ` David Edmondson
  2019-02-26 15:43             ` Antoine Beaupré
  1 sibling, 1 reply; 12+ messages in thread
From: David Edmondson @ 2019-02-26 10:56 UTC (permalink / raw)
  To: Antoine Beaupré, Daniel Kahn Gillmor; +Cc: notmuch

On Tuesday, 2019-02-26 at 00:33:33 -05, Antoine Beaupré wrote:

> So I have no idea what's going on with mail forwards, but this is the
> kind of stuff that happens to me all the time and I can't describe
> correctly enough to file a bug.
>
> The gist of it is that, under some weird circumstances, notmuch-emacs
> (message-mode.el?) will screw up email forwards in a big time. I just
> forwarded (or did I just reply? not sure) that private email to the list
> to trigger that bug...
>
> I get that from time to time. I think the key is that it's an email I
> *sent* not *received*, which are somewhat different in their storage
> mechanism...
>
> The first symptom will be that, when sending, I'd get a prompt like
> this:
>
> Non-printable characters found.  Continue sending? (delete, replace, send, edit, ?): 

After looking at something related but not the same, I ended up with:

 ;; Do forward as MIME (the default), but don't show me the
 ;; MML.
 ;;
 ;; Showing the MML causes decoding and recoding problems, where the
 ;; headers of an message/rfc822 part end up being very long and
 ;; opensmtpd decides to wrap them via QP encoding. Unfortunately it
 ;; QP encodes the MIME separators as well. No idea who is actually in
 ;; the wrong here, but this fixes it so far.
 message-forward-as-mime t
 message-forward-show-mml nil

No idea if this will help with the problem you see.

dme.
-- 
I've still got sand in my shoes.

^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: non-ascii email forwarding failures
  2019-02-26 10:56           ` David Edmondson
@ 2019-02-26 15:43             ` Antoine Beaupré
  0 siblings, 0 replies; 12+ messages in thread
From: Antoine Beaupré @ 2019-02-26 15:43 UTC (permalink / raw)
  To: David Edmondson, Daniel Kahn Gillmor; +Cc: notmuch

On 2019-02-26 10:56:34, David Edmondson wrote:
> On Tuesday, 2019-02-26 at 00:33:33 -05, Antoine Beaupré wrote:
>
>> So I have no idea what's going on with mail forwards, but this is the
>> kind of stuff that happens to me all the time and I can't describe
>> correctly enough to file a bug.
>>
>> The gist of it is that, under some weird circumstances, notmuch-emacs
>> (message-mode.el?) will screw up email forwards in a big time. I just
>> forwarded (or did I just reply? not sure) that private email to the list
>> to trigger that bug...
>>
>> I get that from time to time. I think the key is that it's an email I
>> *sent* not *received*, which are somewhat different in their storage
>> mechanism...
>>
>> The first symptom will be that, when sending, I'd get a prompt like
>> this:
>>
>> Non-printable characters found.  Continue sending? (delete, replace, send, edit, ?): 
>
> After looking at something related but not the same, I ended up with:
>
>  ;; Do forward as MIME (the default), but don't show me the
>  ;; MML.
>  ;;
>  ;; Showing the MML causes decoding and recoding problems, where the
>  ;; headers of an message/rfc822 part end up being very long and
>  ;; opensmtpd decides to wrap them via QP encoding. Unfortunately it
>  ;; QP encodes the MIME separators as well. No idea who is actually in
>  ;; the wrong here, but this fixes it so far.
>  message-forward-as-mime t

I have that same setting here...

>  message-forward-show-mml nil

... but this is set to "best". Setting it to "nil" might solve some of
the problems I'm seeing (specifically with forwarding) but do note the
same problems occur when I reply to an email that has a patch with my
last name in it. For example this message construct will yield the same
error as described earlier because the attached patch has "Beaupré" in
it which will freak out message-mode (or whatever is going on here).

Content-Type: multipart/mixed;
	boundary="_004_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_"
MIME-Version: 1.0
--_004_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_
Content-Type: multipart/alternative;
	boundary="_000_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_"

--_000_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Salut Antoine,

[...]

--_000_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_
Content-Type: text/html; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

[...]
--_000_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_--

--_004_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_
Content-Type: application/octet-stream; name="CVE-2018-16864.patch"
Content-Description: CVE-2018-16864.patch
Content-Disposition: attachment; filename="CVE-2018-16864.patch"; size=1796;
	creation-date="Tue, 26 Feb 2019 13:23:19 GMT";
	modification-date="Tue, 26 Feb 2019 13:23:19 GMT"
Content-Transfer-Encoding: base64

RGVzY3JpcHRpb246IGpvdXJuYWxkOiBkbyBub3Qgc3RvcmUgdGhlIGlvdmVjIGVudHJ5IGZvciBw
cm9jZXNzIGNvbW1hbmRsaW5lIG9uIHN0YWNrCiBUaGlzIGZpeGVzIGEgY3Jhc2ggKENWRS0yMDE4
LTE2ODY0KSB3aGVyZSB3ZQogd291bGQgcmVhZCB0aGUgY29tbWFuZGxpbmUsIHdob3NlIGxlbmd0
aCBpcyB1bmRlciBjb250cm9sIG9mIHRoZQogc2VuZGluZyBwcm9ncmFtLCBhbmQgdGhlbiBjcmFz
aCB3aGVuIHRyeWluZyB0byBjcmVhdGUgYSBzdGFjawogYWxsb2NhdGlvbiBmb3IgaXQuCiAuCiBU
aGlzIGlzIGEgYmFja3BvcnQgb2YgaHR0cHM6Ly9naXRodWIuY29tL3N5c3RlbWQvc3lzdGVtZC9j
b21taXQvMDg0ZWViODY1Y2E2Mzg4NzA5OGUwOTQ1ZmI0ZTkzYzg1MmI5MWIwZgpBdXRob3I6IEFu
dG9pbmUgQmVhdXByw6kgPGFuYXJjYXRAZGViaWFuLm9yZz4KQnVnLURlYmlhbjogaHR0cHM6Ly9i
dWdzLmRlYmlhbi5vcmcvOTE4ODQxCk9yaWdpbjogRGViaWFuCkJ1ZzogaHR0cHM6Ly9idWd6aWxs
YS5yZWRoYXQuY29tL3Nob3dfYnVnLmNnaT9pZD0xNjUzODU1CkZvcndhcmRlZDogbm90LW5lZWRl
ZApMYXN0LVVwZGF0ZTogMjAxOS0wMS0yMgoKLS0tIHN5c3RlbWQtMjE1Lm9yaWcvc3JjL2pvdXJu
YWwvam91cm5hbGQtc2VydmVyLmMKKysrIHN5c3RlbWQtMjE1L3NyYy9qb3VybmFsL2pvdXJuYWxk
LXNlcnZlci5jCkBAIC02MDIsNyArNjAyLDEwIEBAIHN0YXRpYyB2b2lkIGRpc3BhdGNoX21lc3Nh
Z2VfcmVhbCgKIAogICAgICAgICAgICAgICAgIHIgPSBnZXRfcHJvY2Vzc19jbWRsaW5lKHVjcmVk
LT5waWQsIDAsIGZhbHNlLCAmdCk7CiAgICAgICAgICAgICAgICAgaWYgKHIgPj0gMCkgewotICAg
ICAgICAgICAgICAgICAgICAgICAgeCA9IHN0cmFwcGVuZGEoIl9DTURMSU5FPSIsIHQpOworICAg
ICAgICAgICAgICAgICAgICAgICAgLyogQXQgbW9zdCBfU0NfQVJHX01BWCAoMk1CIHVzdWFsbHkp
LCB3aGljaCBpcworICAgICAgICAgICAgICAgICAgICAgICAgICogdG9vIG11Y2ggdG8gcHV0IG9u
IHN0YWNrLiBMZXQncyB1c2UgYSBoZWFwCisgICAgICAgICAgICAgICAgICAgICAgICAgKiBhbGxv
Y2F0aW9uIGZvciB0aGlzIG9uZS4gKi8gCisgICAgICAgICAgICAgICAgICAgICAgICB4ID0gc3Ry
YXBwZW5kKCJfQ01ETElORT0iLCB0KTsKICAgICAgICAgICAgICAgICAgICAgICAgIGZyZWUodCk7
CiAgICAgICAgICAgICAgICAgICAgICAgICBJT1ZFQ19TRVRfU1RSSU5HKGlvdmVjW24rK10sIHgp
OwogICAgICAgICAgICAgICAgIH0KQEAgLTcxNiw3ICs3MTksOSBAQCBzdGF0aWMgdm9pZCBkaXNw
YXRjaF9tZXNzYWdlX3JlYWwoCiAKICAgICAgICAgICAgICAgICByID0gZ2V0X3Byb2Nlc3NfY29t
bShvYmplY3RfcGlkLCAmdCk7CiAgICAgICAgICAgICAgICAgaWYgKHIgPj0gMCkgewotICAgICAg
ICAgICAgICAgICAgICAgICAgeCA9IHN0cmFwcGVuZGEoIk9CSkVDVF9DT01NPSIsIHQpOworICAg
ICAgICAgICAgICAgICAgICAgICAgLyogU2VlIGFib3ZlIGZvciBzaXplIGxpbWl0cywgb25seSAt
PmNtZGxpbmUKKyAgICAgICAgICAgICAgICAgICAgICAgICAqIG1heSBiZSBsYXJnZSwgc28gdXNl
IGEgaGVhcCBhbGxvY2F0aW9uIGZvciBpdC4gKi8KKyAgICAgICAgICAgICAgICAgICAgICAgIHgg
PSBzdHJhcHBlbmQoIk9CSkVDVF9DT01NPSIsIHQpOwogICAgICAgICAgICAgICAgICAgICAgICAg
ZnJlZSh0KTsKICAgICAgICAgICAgICAgICAgICAgICAgIElPVkVDX1NFVF9TVFJJTkcoaW92ZWNb
bisrXSwgeCk7CiAgICAgICAgICAgICAgICAgfQo=

--_004_eb33b55506b743279ad0320cb4a1f036XBOX01axiscom_--

So it's not just forwarding, unfortunately.

A.

-- 
In god we trust, others pay cash.
                        - Richard Desjardins, Miami

^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2019-02-26 15:43 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-25  1:52 [RFC] writing HTML email with notmuch Antoine Beaupré
2019-02-25 18:22 ` Istvan Marko
2019-02-25 18:56 ` Brian Sniffen
2019-02-25 22:58   ` Antoine Beaupré
2019-02-25 21:18 ` Daniel Kahn Gillmor
     [not found]   ` <87lg23ldrl.fsf@curie.anarc.at>
2019-02-25 23:01     ` Antoine Beaupré
2019-02-25 23:15       ` Daniel Kahn Gillmor
2019-02-26  5:33         ` non-ascii email forwarding failures Antoine Beaupré
2019-02-26  5:56           ` Daniel Kahn Gillmor
2019-02-26  7:51             ` Antoine Beaupré
2019-02-26 10:56           ` David Edmondson
2019-02-26 15:43             ` Antoine Beaupré

Code repositories for project(s) associated with this public inbox

	https://yhetil.org/notmuch.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).