From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from localhost (localhost [127.0.0.1]) by arlo.cworth.org (Postfix) with ESMTP id 22B676DE0C66 for ; Tue, 31 Oct 2017 10:13:38 -0700 (PDT) X-Virus-Scanned: Debian amavisd-new at cworth.org X-Spam-Flag: NO X-Spam-Score: -0.718 X-Spam-Level: X-Spam-Status: No, score=-0.718 tagged_above=-999 required=5 tests=[AWL=-0.000, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, WEIRD_PORT=0.001] autolearn=disabled Received: from arlo.cworth.org ([127.0.0.1]) by localhost (arlo.cworth.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id eSExzNwgO33t for ; Tue, 31 Oct 2017 10:13:34 -0700 (PDT) Received: from avasout05.plus.net (avasout05.plus.net [84.93.230.250]) by arlo.cworth.org (Postfix) with ESMTPS id 05D466DE0BCA for ; Tue, 31 Oct 2017 10:13:33 -0700 (PDT) Received: from mail.bubblegen.co.uk ([80.229.236.194]) by smtp with ESMTP id 9a6SeYYfL2du79a6TeRtEd; Tue, 31 Oct 2017 17:13:31 +0000 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.2 cv=a6FAzQaF c=1 sm=1 tr=0 a=G4bc5lkgapKKm1P+Twxy3Q==:117 a=G4bc5lkgapKKm1P+Twxy3Q==:17 a=02M-m0pO-4AA:10 a=J6cydZkJAAAA:8 a=skbIlk6SAAAA:8 a=UqCG9HQmAAAA:8 a=a-mfcSPfP4ITL9zra1sA:9 a=QEXdDO2ut3YA:10 a=LRLYrf9nAAAA:8 a=lPe1Dr-4r-yoS0iev40A:9 a=LVdCw0lTeNFPc9uF:21 a=WJkbv6Bpw4gA:10 a=qiWi7Edwir4A:10 a=aaXl3FVqvGIA:10 a=NWVoK91CQyQA:10 a=pzn1nOZqnrksBt4yenaW:22 a=L0coiOcRLHDvGY_RdqVO:22 a=P5SZwATmzn4K3VfFWgKo:22 a=ctNd1-se8QBkj9sPkfQg:22 Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com [209.85.215.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: matt) by mail.bubblegen.co.uk (Postfix) with ESMTPSA id D25654622017 for ; Tue, 31 Oct 2017 17:13:27 +0000 (GMT) Received: by mail-lf0-f50.google.com with SMTP id e143so7790991lfg.12 for ; Tue, 31 Oct 2017 10:13:27 -0700 (PDT) X-Gm-Message-State: AMCzsaVDGBIYZwaqiE5ymnpwyAaN6Dihd06fefDnBZN+z29MOgDgN9xe DocjhNdT/z/f96yZepfgWkCMlGDKRZSmWzJV2Ho= X-Google-Smtp-Source: ABhQp+QIVWMpbZg/LA5uWYFy/chMt6OmljkwUouE/qDi/yGfnmUDOY0btdqyjYhhR6o4eye6eaPUsaPAv+9imjxGZWE= X-Received: by 10.25.178.205 with SMTP id t74mr1000262lfk.152.1509470006933; Tue, 31 Oct 2017 10:13:26 -0700 (PDT) MIME-Version: 1.0 References: <87tvyvp4f2.fsf@istari.evenmere.org> <87376f13ho.fsf@fifthhorseman.net> <87r2tww9tr.fsf@nikula.org> <87wp3ow39i.fsf@fifthhorseman.net> <27e53def-32b4-45ab-1192-77cc0e837a93@gmail.com> <87zi8eopgq.fsf@istari.evenmere.org> <877evhy53k.fsf@fifthhorseman.net> <87she5nsmy.fsf@istari.evenmere.org> <87inf1gm7l.fsf@fifthhorseman.net> <87mv4co4vz.fsf@istari.evenmere.org> In-Reply-To: <87mv4co4vz.fsf@istari.evenmere.org> From: Matthew Lear Date: Tue, 31 Oct 2017 17:13:16 +0000 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: web interface to notmuch To: Brian Sniffen Cc: Daniel Kahn Gillmor , Jani Nikula , Vladimir Panteleev , notmuch@notmuchmail.org Content-Type: multipart/alternative; boundary="f403045f56b88ffd73055cdadfe0" X-CMAE-Envelope: MS4wfJ6QUPCS0V3dP5RHIWtyL/VEjBz8sPDThoRh+qNkzcuSZUaTa7CCbpbc2XWobfBUGcwT8eZb9Bk+fwOFlnQjzKAx7HNUccZ2wZQ54sn0tBemq1cmUxsJ A3ZSKhDI0IAtu1v43TWiSKyuMrTiI8SByVPg0hWJOb1aXONb9mTYi2lwIolx9Ri4N/9PTRLgNGQWZw== X-BeenThere: notmuch@notmuchmail.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Use and development of the notmuch mail system." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Oct 2017 17:13:38 -0000 --f403045f56b88ffd73055cdadfe0 Content-Type: text/plain; charset="UTF-8" On Fri, 27 Oct 2017, 18:52 Brian Sniffen, wrote: > Daniel Kahn Gillmor writes: > > > On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote: > >> With bleach integrated (all of five lines), I think this is safe enough > >> to let random notmuch users run it. > > > > hm, bleach might be a little too aggressive. > > > > jrollins just pointed toward: > > > > https://nmweb.evenmere.org/show/87innmvvam.fsf%40ligo.caltech.edu > > That's fixed in 53403ecd, and there's some examples of bleach on a rope > at > https://nmweb.evenmere.org/show/20141107190321.GL23609%40odin.tremily.us > > The mbox URL is linkified, the many other link-like texts aren't. > > > Next/prev links are at the bottom, and a thread listing. I haven't > thought through how to get the body delivered immediately, but speed > seems acceptable. Next up, some style revisions---and I'd love > proposals for something that looks less awful, or at least makes the > interface more clear. UI design is a strong anti-specialty for me. > I've been running this today - standalone on localhost port 80 with the built in CherryPy web server on my mail store. First impressions are it's terrific :-) As my intended 'target' mail store will be geared towards a 'internal work stuff knowledge collection', lots of emails contain html links to intranet pages and sites. I can adapt the bleach usage to suit (or just remove it), but along the way of searching and viewing mail, I've encountered quite a few occurrences of failing to UnicodeEncode. An example backtrace looks like this: Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/web/application.py", line 239, in process return self.handle() File "/usr/lib/python2.7/dist-packages/web/application.py", line 230, in handle return self._delegate(fn, self.fvars, args) File "/usr/lib/python2.7/dist-packages/web/application.py", line 420, in _delegate return handle_class(cls) File "/usr/lib/python2.7/dist-packages/web/application.py", line 396, in handle_class return tocall(*args) File "/b/git/notmuch-brians.git/contrib/notmuch-web/nmweb.py", line 153, in GET sprefix=webprefix) File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 989, in render return self.environment.handle_exception(exc_info, True) File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 754, in handle_exception reraise(exc_type, exc_value, tb) File "templates/show.html", line 1, in top-level template code {% extends "base.html" %} File "templates/base.html", line 32, in top-level template code {% block content %} File "templates/show.html", line 12, in block "content" {% for part in format_message(m.get_filename(),mid): %}{{ part|safe }}{% endfor %} File "/b/git/notmuch-brians.git/contrib/notmuch-web/nmweb.py", line 245, in format_message_walk tags=safe_tags).encode(part.get_content_charset('ascii'))) UnicodeEncodeError: 'latin-1' codec can't encode character u'\u201c' in position 1141: ordinal not in range(256) 127.0.0.1:60968 - - [31/Oct/2017 17:00:02] "HTTP/1.1 GET /show/ 665d8c5c2b024898ae21951c4b8b4f93@CO2PR05MB747.namprd05.prod.outlook.com" - 500 Internal Server Error I'm no Python expert, but from a quick google it would seem like the cause of such an exception is related to not using utf-8. Brian - do you think something needs modifying in nmweb.py to cater for this type of thing, or is this somehow related my own mailstore (not sure why that would be as my messages haven't been modified). Cheers, -- Matt --f403045f56b88ffd73055cdadfe0 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
On Fri, 27 Oct 2017, 18:52 Brian Sniffen, <bts@evenmere.org> wro= te:
Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote:
>> With bleach integrated (all of five lines), I think this is safe e= nough
>> to let random notmuch users run it.
>
> hm, bleach might be a little too aggressive.
>
> jrollins just pointed toward:
>
> https://nmweb.evenmere.org/sho= w/87innmvvam.fsf%40ligo.caltech.edu

That's fixed in 53403ecd, and there's some examples of bleach on a = rope
at
https://nmweb.evenmere.org/s= how/20141107190321.GL23609%40odin.tremily.us

The mbox URL is linkified, the many other link-like texts aren't.


Next/prev links are at the bottom, and a thread listing.=C2=A0 I haven'= t
thought through how to get the body delivered immediately, but speed
seems acceptable.=C2=A0 Next up, some style revisions---and I'd love proposals for something that looks less awful, or at least makes the
interface more clear.=C2=A0 UI design is a strong anti-specialty for me.

I've been running this today - standalone = on localhost port 80 with the built in CherryPy web server on my mail store= . First impressions are it's terrific :-) As my intended 'target= 9;=C2=A0mail store will be geared towards a = 'internal work stuff knowledge collection', lots of emails contain = html links to intranet pages and sites. I can adapt the bleach usage to sui= t (or just remove it), but along the way of searching and viewing mail, I&#= 39;ve encountered quite a few occurrences of failing to UnicodeEncode. An e= xample backtrace looks like this:

Trace= back (most recent call last):
=C2=A0 File "/usr/lib/python2.= 7/dist-packages/web/application.py", line 239, in process
= =C2=A0 =C2=A0 return self.handle()
=C2=A0 File "/usr/lib/pyt= hon2.7/dist-packages/web/application.py", line 230, in handle
=C2=A0 =C2=A0 return self._delegate(fn, self.fvars, args)
=C2= =A0 File "/usr/lib/python2.7/dist-packages/web/application.py", l= ine 420, in _delegate
=C2=A0 =C2=A0 return handle_class(cls)
=C2=A0 File "/usr/lib/python2.7/dist-packages/web/application.py= ", line 396, in handle_class
=C2=A0 =C2=A0 return tocall(*ar= gs)
=C2=A0 File "/b/git/notmuch-brians.git/contrib/notmuch-w= eb/nmweb.py", line 153, in GET
=C2=A0 =C2=A0 sprefix=3Dwebpr= efix)
=C2=A0 File "/usr/lib/python2.7/dist-packages/jinja2/e= nvironment.py", line 989, in render
=C2=A0 =C2=A0 return sel= f.environment.handle_exception(exc_info, True)
=C2=A0 File "= /usr/lib/python2.7/dist-packages/jinja2/environment.py", line 754, in = handle_exception
=C2=A0 =C2=A0 reraise(exc_type, exc_value, tb)
=C2=A0 File "templates/show.html", line 1, in top-level = template code
=C2=A0 =C2=A0 {% extends "base.html" %}
=C2=A0 File "templates/base.html", line 32, in top-level= template code
=C2=A0 =C2=A0 {% block content %}
=C2=A0= File "templates/show.html", line 12, in block "content"= ;
=C2=A0 =C2=A0 {% for part in format_message(m.get_filename(),mi= d): %}{{ part|safe }}{% endfor %}
=C2=A0 File "/b/git/notmuc= h-brians.git/contrib/notmuch-web/nmweb.py", line 245, in format_messag= e_walk
=C2=A0 =C2=A0 tags=3Dsafe_tags).encode(part.get_content_ch= arset('ascii')))
UnicodeEncodeError: 'latin-1' co= dec can't encode character u'\u201c' in position 1141: ordinal = not in range(256)

127.0.0.1:60968 - - [31/Oct/2017 17:00:02] "HTTP/1.1 GET /show/= 665d8c5c2b024898ae21951c4b8b4f93@CO2PR05MB747.namprd05.prod= .outlook.com" - 500 Internal Server Error
I'm no Python expert, but from a quick google it would see= m like the cause of such an exception is related to not using utf-8.
<= div>
Brian - do you think something needs modifying in nmweb.= py to cater for this type of thing, or is this somehow related my own mails= tore (not sure why that would be as my messages haven't been modified).=
Cheers,
--=C2=A0 Matt

=
--f403045f56b88ffd73055cdadfe0--