[PATCH] devel/nmweb: read mail files in binary mode.

[PATCH] devel/nmweb: read mail files in binary mode.

[David Bremner]

"ju" reported on IRC that browsing

    https://nmbug.notmuchmail.org/nmweb/show/20160719094205.qmf5sjnja6crt5t3%40gotlib

crashed. The underlying issue is that python3 defaults to utf8
decoding files unless they are opened in binary mode. The file in
question (in the nmbug archive; it depends a bit on the routing the
message took) has

        Content-Type: text/plain; charset=iso-8859-1
        Content-Transfer-Encoding: 8bit

and some of it is not valid utf8.
---

I have already deployed this as a hotfix, but not applied it to master or release. 


 devel/notmuch-web/nmweb.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devel/notmuch-web/nmweb.py b/devel/notmuch-web/nmweb.py
index b0d4d5cd..e0e87b49 100755
--- a/devel/notmuch-web/nmweb.py
+++ b/devel/notmuch-web/nmweb.py
@@ -207,7 +207,7 @@ env.globals['thread_nav'] = thread_nav
 
 def format_message(nm_msg, mid):
   fn = list(nm_msg.filenames())[0]
-  msg = MaildirMessage(open(fn))
+  msg = MaildirMessage(open(fn, 'rb'))
   return format_message_walk(msg, mid)
 
 def decodeAnyway(txt, charset='ascii'):
-- 
2.40.1

Re: [PATCH] devel/nmweb: read mail files in binary mode.

[Tomi Ollila]

On Sat, Sep 16 2023, David Bremner wrote:

> "ju" reported on IRC that browsing
>
>     https://nmbug.notmuchmail.org/nmweb/show/20160719094205.qmf5sjnja6crt5t3%40gotlib
>
> crashed. The underlying issue is that python3 defaults to utf8
> decoding files unless they are opened in binary mode. The file in
> question (in the nmbug archive; it depends a bit on the routing the
> message took) has
>
>         Content-Type: text/plain; charset=iso-8859-1
>         Content-Transfer-Encoding: 8bit
>
> and some of it is not valid utf8.
> ---

Looks good to me. I've been experiencing "text" mode defaulting to 'ascii'
(and had to include encoding='utf-8') lately. Another problem one may
encounter when there are \r chars in the file, text mode may remove those
(In the same case I had a file using '\r\r' as list content separator,
and failed until changing open to binary mode)

Tomi

>
> I have already deployed this as a hotfix, but not applied it to master or release. 
>
>
>  devel/notmuch-web/nmweb.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/devel/notmuch-web/nmweb.py b/devel/notmuch-web/nmweb.py
> index b0d4d5cd..e0e87b49 100755
> --- a/devel/notmuch-web/nmweb.py
> +++ b/devel/notmuch-web/nmweb.py
> @@ -207,7 +207,7 @@ env.globals['thread_nav'] = thread_nav
>  
>  def format_message(nm_msg, mid):
>    fn = list(nm_msg.filenames())[0]
> -  msg = MaildirMessage(open(fn))
> +  msg = MaildirMessage(open(fn, 'rb'))
>    return format_message_walk(msg, mid)
>  
>  def decodeAnyway(txt, charset='ascii'):
> -- 
> 2.40.1
>
> _______________________________________________
> notmuch mailing list -- notmuch@notmuchmail.org
> To unsubscribe send an email to notmuch-leave@notmuchmail.org

Re: [PATCH] devel/nmweb: read mail files in binary mode.

[David Bremner]

Tomi Ollila <tomi.ollila@iki.fi> writes:

>
> Looks good to me. I've been experiencing "text" mode defaulting to 'ascii'
> (and had to include encoding='utf-8') lately. Another problem one may
> encounter when there are \r chars in the file, text mode may remove those
> (In the same case I had a file using '\r\r' as list content separator,
> and failed until changing open to binary mode)
>
> Tomi
>

thanks for the review. Applied to master and deployed that (again) to
nmbug.notmuchmail.org. Probably I should think of a deployment method
that does not involve git merge :(.


d