rendering and generating text/markdown parts

rendering and generating text/markdown parts

[Daniel Kahn Gillmor]

Using `Content-Type: text/markdown`
-----------------------------------

[Over on LWN, some discussion came up about rendering and generating
text/markdown variants of e-mails](https://lwn.net/Articles/730277/)

This is similar to (but maybe simpler than) [a proposal that was
discussed on the alot bug tracker earlier this
year](https://github.com/pazz/alot/issues/1051).

I thought i'd drop this mail as a marker for the start of a longer
discussion if anyone wants to try to take it further.

Next Steps
----------

I think this mail is (hopefully) actually composed correctly as
text/markdown.

Does anyone's MUA render it properly?  Anyone want to post screenshots
or other renderings?

Anyone have suggestions about message composition in markdown?

           --dkg

Re: rendering and generating text/markdown parts

[Dylan Baker]

[-- Attachment #1.1: Type: text/plain, Size: 1758 bytes --]

One of the alot devs here,

Natively no. I have a fairly minor patch to alot that would make it work if the
type was text/markdown instead of text/plain.

I have another patch that makes this render correctly as markdown, but it breaks
lots of other text/plain emails that use characters in a way that they look like
markdown to cmark, but aren't actually markdown.

I think if we're going to have a message format we should use text/markdown or
text/commonmark, or text/x-(markdown|commonmark) rather than text/plain as the
format. While it's safe to read markdown as plain text (it was originally used
that way anyway), it's not safe to assume that all text is valid markdown.

elinks with "-dump-color-mode 1" did a pretty good job of formatting the html that
came out of cmark. I've attached that if anyone wants to see it.

Dylan

Quoting Daniel Kahn Gillmor (2017-08-10 21:41:52)
> Using `Content-Type: text/markdown`
> -----------------------------------
> 
> [Over on LWN, some discussion came up about rendering and generating
> text/markdown variants of e-mails](https://lwn.net/Articles/730277/)
> 
> This is similar to (but maybe simpler than) [a proposal that was
> discussed on the alot bug tracker earlier this
> year](https://github.com/pazz/alot/issues/1051).
> 
> I thought i'd drop this mail as a marker for the start of a longer
> discussion if anyone wants to try to take it further.
> 
> Next Steps
> ----------
> 
> I think this mail is (hopefully) actually composed correctly as
> text/markdown.
> 
> Does anyone's MUA render it properly?  Anyone want to post screenshots
> or other renderings?
> 
> Anyone have suggestions about message composition in markdown?
> 
>            --dkg
> 

[-- Attachment #1.2: elinks-out.txt --]
[-- Type: text/plain, Size: 2143 bytes --]

^[[0;30m^[[0;37mUsing Content-Type: text/markdown                                               
^[[0;37m                                                                                
^[[0;37m   ^[[0;34m[1]Over on LWN, some discussion came up about rendering and generating       
^[[0;34m^[[0;37m   ^[[0;34mtext/markdown variants of e-mails                                            
^[[0;34m                                                                                
^[[0;34m^[[0;37m   This is similar to (but maybe simpler than) ^[[0;34m[2]a proposal that was           
^[[0;34m^[[0;37m   ^[[0;34mdiscussed on the alot bug tracker earlier this year^[[0;37m.                         
^[[0;37m                                                                                
^[[0;37m   I thought i'd drop this mail as a marker for the start of a longer           
^[[0;37m   discussion if anyone wants to try to take it further.                        
^[[0;37m                                                                                
^[[0;37mNext Steps                                                                      
^[[0;37m                                                                                
^[[0;37m   I think this mail is (hopefully) actually composed correctly as              
^[[0;37m   text/markdown.                                                               
^[[0;37m                                                                                
^[[0;37m   Does anyone's MUA render it properly? Anyone want to post screenshots or     
^[[0;37m   other renderings?                                                            
^[[0;37m                                                                                
^[[0;37m   Anyone have suggestions about message composition in markdown?               
^[[0;37m                                                                                
^[[0;37m        --dkg                                                                   

References

   Visible links
   1. https://lwn.net/Articles/730277/
   2. https://github.com/pazz/alot/issues/1051

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEUwPMqo/+5aFHLzU4CJ4WlhQGiO8FAlmOKYQACgkQCJ4WlhQG
iO98agf/aT1YWEGt5Rh2P0guPqzQuuNN9HxIapiTNybR0bTRZSliWjJk24VurZTP
hKzrD0vA24e2dvFFAdjsc5aUjCr0kwShKKOPZ+IdJtnD1zARD512dqG0UU0ZL0xq
GB0BXW2UIxJwheG4dcCLihYre48U/givbtvFNGn+REmimm1rIJ3UAcFDuP380cdU
LkFgp1phvSO/cocEpe0MD8gsbIrokvXTUQfbkVgjSf0r6pU0Ass30UmMbUSyL4No
PyDgReLjAnGxFhFu033pZpmkYmO/zIbXUIV9QGpb32ud5d5egzlBKzpApZMYq6AD
fOknAhp11pRcZL3z3Pq4EtX2OwsyAw==
=6FRy
-----END PGP SIGNATURE-----

Re: rendering and generating text/markdown parts

[Daniel Kahn Gillmor]

[-- Attachment #1: Type: text/plain, Size: 1409 bytes --]

On Fri 2017-08-11 15:02:44 -0700, Dylan Baker wrote:

> Natively no. I have a fairly minor patch to alot that would make it work if the
> type was text/markdown instead of text/plain.

huh, i tried to set Content-Type: text/markdown in emacs mml-mode by
just fiddling with the header but mml apparently believed that it knew
better than i did what Content-Type header to send and sent out
text/plain anyway :/

> I have another patch that makes this render correctly as markdown, but it breaks
> lots of other text/plain emails that use characters in a way that they look like
> markdown to cmark, but aren't actually markdown.

yeah, i agree we should *not* try to apply these filters to anything not
marked explicitly as text/markdown.

> I think if we're going to have a message format we should use text/markdown or
> text/commonmark, or text/x-(markdown|commonmark) rather than text/plain as the
> format. While it's safe to read markdown as plain text (it was originally used
> that way anyway), it's not safe to assume that all text is valid markdown.

Absolutely agreed.

> elinks with "-dump-color-mode 1" did a pretty good job of formatting the html that
> came out of cmark. I've attached that if anyone wants to see it.

Interesting, it does look plausible, though i confess i don't like the
idea of needing a two-stage pipeline. seems like a lot of attack surface
and moving parts :/

     --dkg

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 832 bytes --]

Re: rendering and generating text/markdown parts

[Dylan Baker]

[-- Attachment #1: Type: text/plain, Size: 2318 bytes --]

Quoting Daniel Kahn Gillmor (2017-08-11 16:05:56)
> On Fri 2017-08-11 15:02:44 -0700, Dylan Baker wrote:
> 
> > Natively no. I have a fairly minor patch to alot that would make it work if the
> > type was text/markdown instead of text/plain.
> 
> huh, i tried to set Content-Type: text/markdown in emacs mml-mode by
> just fiddling with the header but mml apparently believed that it knew
> better than i did what Content-Type header to send and sent out
> text/plain anyway :/
> 
> > I have another patch that makes this render correctly as markdown, but it breaks
> > lots of other text/plain emails that use characters in a way that they look like
> > markdown to cmark, but aren't actually markdown.
> 
> yeah, i agree we should *not* try to apply these filters to anything not
> marked explicitly as text/markdown.
> 
> > I think if we're going to have a message format we should use text/markdown or
> > text/commonmark, or text/x-(markdown|commonmark) rather than text/plain as the
> > format. While it's safe to read markdown as plain text (it was originally used
> > that way anyway), it's not safe to assume that all text is valid markdown.

Someone on LWN noted that "x-" has gone out of style (officially with an RFC and
everything), so I think "text/commonmark", to distinguish it from other flavors
of markdown.

> 
> Absolutely agreed.
> 
> > elinks with "-dump-color-mode 1" did a pretty good job of formatting the html that
> > came out of cmark. I've attached that if anyone wants to see it.
> 
> Interesting, it does look plausible, though i confess i don't like the
> idea of needing a two-stage pipeline. seems like a lot of attack surface
> and moving parts :/
> 
>      --dkg

cmark does have a mode to disable a bunch of "dangerous" stuff, like javascript
and css, and some other things. I think it would be better if there was native
support to avoid the multipart pipeline.

For alot, there is a python library for common mark I haven't looked at too
closely, but it's ported from JavaScript, and libraries that are ports from
other languages usually feel clunky. I'm hoping to be pleasantly surprised,
otherwise I may look into writing python bindings for the C library.

For emacs, it would probably mean writing a parser in elisp.

Dylan

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEUwPMqo/+5aFHLzU4CJ4WlhQGiO8FAlmOP4sACgkQCJ4WlhQG
iO8irQgAgRxsygL6v7cE7/Lb82GoQRcgGypmV7hm/Z7ZxLgwE0L4pdamELSuLfYk
yX1HHh4QxvOlS/gpD6D6C8G2fjDsu/pKhyOxhGTqjBm9AgLpysQltHba4Ee41DKt
9yTihCQV7ESFR9rgP+LCtCdWs6wr0DETB6DCaQtJo2vthfWGbo9IKpltdRB/1BBT
NSn509jQ5+p76/jtmhCGm+AsHrRls7z4Fxc1/z/fYRE/7WvhWiwM827C/I+5EXOR
m6EUi0wRlDJM57ZndRdYYZ9TUPAeDbXOjhUpe3/UyPsMQykwepKItvIlg5Kmzr/k
CuAdeEw8SAVNRriWE7IRdgR86SAP/g==
=MfB9
-----END PGP SIGNATURE-----