* clean up for nmweb/btsmail
@ 2018-02-05 4:19 Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 01/11] nmweb: (pylint) use __future__.absolute_import Daniel Kahn Gillmor
` (11 more replies)
0 siblings, 12 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
The patches that follow are some proposed cleanups for
contrib/notmuch-web (aka nmweb, aka btsmail), which was originally
proposed by Brian Sniffen.
These are pretty mechanical changes -- nothing fancy added or removed,
just trying to make the code a little more maintainable and moving it
to python3.
You can also find this on the "nmweb" branch at
https://gitlab.com/dkg/notmuch
Regards,
--dkg
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH 01/11] nmweb: (pylint) use __future__.absolute_import
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 02/11] nmweb: (pylint) use replace() function directly on string object Daniel Kahn Gillmor
` (10 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
pylint --py3k suggests always including this in code that might run in
either python2 or python3. it selects the python3 default. (see
"pylint --help-msg=no-absolute-import" for more details)
---
contrib/notmuch-web/nmweb.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index add0d631..c6d437c5 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -1,5 +1,7 @@
#!/usr/bin/env python
+from __future__ import absolute_import
+
import web
from notmuch import Database,Query,Message
import urllib
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 02/11] nmweb: (pylint) use replace() function directly on string object
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 01/11] nmweb: (pylint) use __future__.absolute_import Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 03/11] nmweb: (pylint) use spaces instead of tabs Daniel Kahn Gillmor
` (9 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
Use of string.replace() is deprecated, better to invoke replace()
directly on the string object.
---
contrib/notmuch-web/nmweb.py | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index c6d437c5..1f6e0145 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -10,7 +10,6 @@ from mailbox import MaildirMessage
import os
import mimetypes
import email
-import string
import re
from jinja2 import Environment, FileSystemLoader # FIXME to PackageLoader
from jinja2 import Markup
@@ -166,8 +165,7 @@ def format_message_walk(msg,mid):
yield '<ul>'
for subpart in part.get_payload():
yield ('<li><a href="#%s">%s</a></li>' %
- (string.replace(subpart.get_content_type(),
- '/', '-'),
+ (subpart.get_content_type().replace('/', '-'),
subpart.get_content_type()))
yield '</ul>'
elif part.get_content_type() == 'message/rfc822':
@@ -191,7 +189,7 @@ def format_message_walk(msg,mid):
yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
yield '</div>'
else:
- yield '<div id="%s">' % string.replace(part.get_content_type(),'/','-')
+ yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
(filename,cid) = link_to_cached_file(part,mid,counter)
counter += 1
yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
@@ -222,7 +220,7 @@ def format_message_walk(msg,mid):
env.globals['format_message'] = format_message
def replace_cids(body,mid):
- return string.replace(body,'cid:',os.path.join(prefix,cachedir,mid)+'/')
+ return body.replace('cid:',os.path.join(prefix,cachedir,mid)+'/')
def find_cids(body):
return re.findall(r'cid:([^ "\'>]*)', body)
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 03/11] nmweb: (pylint) use spaces instead of tabs
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 01/11] nmweb: (pylint) use __future__.absolute_import Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 02/11] nmweb: (pylint) use replace() function directly on string object Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 04/11] nmweb: (pylint) normalize whitespace Daniel Kahn Gillmor
` (8 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
---
contrib/notmuch-web/nmweb.py | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 1f6e0145..b56c6b12 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -47,7 +47,7 @@ class index:
tags = db.get_all_tags()
return template.render(tags=tags,
title="Notmuch webmail",
- prefix=prefix,
+ prefix=prefix,
sprefix=webprefix)
class search:
@@ -80,7 +80,7 @@ class search:
ts=ts,
title=terms,
prefix=prefix,
- sprefix=webprefix)
+ sprefix=webprefix)
def format_time_range(start,end):
if end-start < (60*60*24):
@@ -137,7 +137,7 @@ class show:
mid=mid,
title=m.get_header('Subject'),
prefix=prefix,
- sprefix=webprefix)
+ sprefix=webprefix)
def format_message(fn,mid):
msg = MaildirMessage(open(fn))
@@ -184,9 +184,9 @@ def format_message_walk(msg,mid):
decoded = decodeAnyway(unb64,part.get_content_charset('ascii'))
cid_refd += find_cids(decoded)
part.set_payload(replace_cids(decoded,mid).encode(part.get_content_charset('ascii')))
- (filename,cid) = link_to_cached_file(part,mid,counter)
- counter +=1
- yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
+ (filename,cid) = link_to_cached_file(part,mid,counter)
+ counter +=1
+ yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
yield '</div>'
else:
yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 04/11] nmweb: (pylint) normalize whitespace
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (2 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 03/11] nmweb: (pylint) use spaces instead of tabs Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-03-17 12:37 ` David Bremner
2018-02-05 4:19 ` [PATCH 05/11] nmweb: (pylint) import built-in modules before third-party modules Daniel Kahn Gillmor
` (7 subsequent siblings)
11 siblings, 1 reply; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
I've settled on:
* 2-space indent
* one space after comma
* one space on either side of assignment and comparison operators
* no trailing whitespace
This now satisfies the indentation/whitespace checks of:
pylint --indent-string=' ' --indent-after-paren=2 nmweb.py
---
contrib/notmuch-web/nmweb.py | 248 ++++++++++++++++++++++---------------------
1 file changed, 125 insertions(+), 123 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index b56c6b12..bee4896e 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -3,7 +3,7 @@
from __future__ import absolute_import
import web
-from notmuch import Database,Query,Message
+from notmuch import Database, Query, Message
import urllib
from datetime import datetime
from mailbox import MaildirMessage
@@ -27,14 +27,14 @@ urls = (
'/', 'index',
'/search/(.*)', 'search',
'/show/(.*)', 'show',
- )
+)
def urlencode_filter(s):
- if type(s) == 'Markup':
- s = s.unescape()
- s = s.encode('utf8')
- s = urllib.quote_plus(s)
- return Markup(s)
+ if type(s) == 'Markup':
+ s = s.unescape()
+ s = s.encode('utf8')
+ s = urllib.quote_plus(s)
+ return Markup(s)
env.filters['url'] = urlencode_filter
class index:
@@ -51,7 +51,7 @@ class index:
sprefix=webprefix)
class search:
- def GET(self,terms):
+ def GET(self, terms):
redir = False
if web.input(terms=None).terms:
redir = True
@@ -59,20 +59,20 @@ class search:
if web.input(afters=None).afters:
afters = web.input(afters=None).afters[:-3]
else:
- afters='0'
+ afters = '0'
if web.input(befores=None).befores:
befores = web.input(befores=None).befores
else:
befores = '4294967296' # 2^32
if int(afters) > 0 or int(befores) < 4294967296:
redir = True
- terms += ' %s..%s' % (afters,befores)
+ terms += ' %s..%s' % (afters, befores)
if redir:
raise web.seeother('/search/%s' % urllib.quote_plus(terms))
web.header('Content-type', 'text/html')
web.header('Transfer-Encoding', 'chunked')
db = Database()
- q = Query(db,terms)
+ q = Query(db, terms)
q.set_sort(Query.SORT.NEWEST_FIRST)
ts = q.search_threads()
template = env.get_template('search.html')
@@ -82,19 +82,19 @@ class search:
prefix=prefix,
sprefix=webprefix)
-def format_time_range(start,end):
+def format_time_range(start, end):
if end-start < (60*60*24):
time = datetime.fromtimestamp(start).strftime('%Y %b %d %H:%M')
else:
start = datetime.fromtimestamp(start).strftime("%Y %b %d")
end = datetime.fromtimestamp(end).strftime("%Y %b %d")
- time = "%s through %s" % (start,end)
+ time = "%s through %s" % (start, end)
return time
env.globals['format_time_range'] = format_time_range
def mailto_addrs(frm):
- frm = email.utils.getaddresses([frm])
- return ','.join(['<a href="mailto:%s">%s</a> ' % ((l,p) if p else (l,l)) for (p,l) in frm])
+ frm = email.utils.getaddresses([frm])
+ return ','.join(['<a href="mailto:%s">%s</a> ' % ((l, p) if p else (l, l)) for (p, l) in frm])
env.globals['mailto_addrs'] = mailto_addrs
def show_msgs(msgs):
@@ -102,32 +102,33 @@ def show_msgs(msgs):
for msg in msgs:
red = 'black'
flag = msg.get_flag(Message.FLAG.MATCH)
- if flag: red='red'
+ if flag: red = 'red'
frm = msg.get_header('From')
frm = mailto_addrs(frm)
subj = msg.get_header('Subject')
lnk = urllib.quote_plus(msg.get_message_id())
rs = show_msgs(msg.get_replies())
- r += '<li><font color=%s>%s—<a href="%s/show/%s">%s</a></font> %s</li>' % (red,frm,prefix,lnk,subj,rs)
+ r += '<li><font color=%s>%s—<a href="%s/show/%s">%s</a></font> %s</li>' % (
+ red, frm, prefix, lnk, subj, rs)
r += '</ul>'
return r
env.globals['show_msgs'] = show_msgs
# As email.message.walk, but showing close tags as well
def mywalk(self):
- yield self
- if self.is_multipart():
- for subpart in self.get_payload():
- for subsubpart in mywalk(subpart):
- yield subsubpart
- yield 'close-div'
+ yield self
+ if self.is_multipart():
+ for subpart in self.get_payload():
+ for subsubpart in mywalk(subpart):
+ yield subsubpart
+ yield 'close-div'
class show:
- def GET(self,mid):
+ def GET(self, mid):
web.header('Content-type', 'text/html')
web.header('Transfer-Encoding', 'chunked')
db = Database()
- q = Query(db,'id:'+mid)
+ q = Query(db, 'id:'+mid)
m = list(q.search_messages())[0]
template = env.get_template('show.html')
# FIXME add reply-all link with email.urils.getaddresses
@@ -139,11 +140,11 @@ class show:
prefix=prefix,
sprefix=webprefix)
-def format_message(fn,mid):
- msg = MaildirMessage(open(fn))
- return format_message_walk(msg,mid)
+def format_message(fn, mid):
+ msg = MaildirMessage(open(fn))
+ return format_message_walk(msg, mid)
-def decodeAnyway(txt,charset='ascii'):
+def decodeAnyway(txt, charset='ascii'):
try:
out = txt.decode(charset)
except UnicodeDecodeError:
@@ -153,63 +154,45 @@ def decodeAnyway(txt,charset='ascii'):
out = txt.decode('latin1')
return out
-def format_message_walk(msg,mid):
- counter = 0
- cid_refd = []
- for part in mywalk(msg):
- if part=='close-div':
- yield '</div>'
- elif part.get_content_maintype() == 'multipart':
- yield '<div class="multipart-%s">' % part.get_content_subtype()
- if part.get_content_subtype() == 'alternative':
- yield '<ul>'
- for subpart in part.get_payload():
- yield ('<li><a href="#%s">%s</a></li>' %
- (subpart.get_content_type().replace('/', '-'),
- subpart.get_content_type()))
- yield '</ul>'
- elif part.get_content_type() == 'message/rfc822':
- # FIXME extract subject, date, to/cc/from into a separate template and use it here
- yield '<div class="message-rfc822">'
- elif part.get_content_maintype() == 'text':
- if part.get_content_subtype() == 'plain':
- yield '<div id="text-plain"><pre>'
- out = part.get_payload(decode=True)
- out = decodeAnyway(out,part.get_content_charset('ascii'))
- yield out
- yield '</pre></div>'
- elif part.get_content_subtype() == 'html':
- yield '<div id="text-html">'
- unb64 = part.get_payload(decode=True)
- decoded = decodeAnyway(unb64,part.get_content_charset('ascii'))
- cid_refd += find_cids(decoded)
- part.set_payload(replace_cids(decoded,mid).encode(part.get_content_charset('ascii')))
- (filename,cid) = link_to_cached_file(part,mid,counter)
- counter +=1
- yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
- yield '</div>'
- else:
- yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
- (filename,cid) = link_to_cached_file(part,mid,counter)
- counter += 1
- yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
- cachedir,
- mid,
- filename),
- filename,
- part.get_content_type())
- yield '</div>'
- elif part.get_content_maintype() == 'image':
- (filename,cid) = link_to_cached_file(part,mid,counter)
- if cid not in cid_refd:
- counter += 1
- yield '<img src="%s" alt="%s">' % (os.path.join(prefix,
- cachedir,
- mid,
- filename),
- filename)
+def format_message_walk(msg, mid):
+ counter = 0
+ cid_refd = []
+ for part in mywalk(msg):
+ if part == 'close-div':
+ yield '</div>'
+ elif part.get_content_maintype() == 'multipart':
+ yield '<div class="multipart-%s">' % part.get_content_subtype()
+ if part.get_content_subtype() == 'alternative':
+ yield '<ul>'
+ for subpart in part.get_payload():
+ yield ('<li><a href="#%s">%s</a></li>' %
+ (subpart.get_content_type().replace('/', '-'),
+ subpart.get_content_type()))
+ yield '</ul>'
+ elif part.get_content_type() == 'message/rfc822':
+ # FIXME extract subject, date, to/cc/from into a separate template and use it here
+ yield '<div class="message-rfc822">'
+ elif part.get_content_maintype() == 'text':
+ if part.get_content_subtype() == 'plain':
+ yield '<div id="text-plain"><pre>'
+ out = part.get_payload(decode=True)
+ out = decodeAnyway(out, part.get_content_charset('ascii'))
+ yield out
+ yield '</pre></div>'
+ elif part.get_content_subtype() == 'html':
+ yield '<div id="text-html">'
+ unb64 = part.get_payload(decode=True)
+ decoded = decodeAnyway(unb64, part.get_content_charset('ascii'))
+ cid_refd += find_cids(decoded)
+ part.set_payload(replace_cids(decoded, mid).encode(part.get_content_charset('ascii')))
+ (filename, cid) = link_to_cached_file(part, mid, counter)
+ counter += 1
+ yield '<iframe class="embedded-html" src="%s">' % (
+ os.path.join(prefix, cachedir, mid, filename),)
+ yield '</div>'
else:
- (filename,cid) = link_to_cached_file(part,mid,counter)
+ yield '<div id="%s">' % (part.get_content_type().replace('/', '-'),)
+ (filename, cid) = link_to_cached_file(part, mid, counter)
counter += 1
yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
cachedir,
@@ -217,50 +200,69 @@ def format_message_walk(msg,mid):
filename),
filename,
part.get_content_type())
+ yield '</div>'
+ elif part.get_content_maintype() == 'image':
+ (filename, cid) = link_to_cached_file(part, mid, counter)
+ if cid not in cid_refd:
+ counter += 1
+ yield '<img src="%s" alt="%s">' % (os.path.join(prefix,
+ cachedir,
+ mid,
+ filename),
+ filename)
+ else:
+ (filename, cid) = link_to_cached_file(part, mid, counter)
+ counter += 1
+ yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
+ cachedir,
+ mid,
+ filename),
+ filename,
+ part.get_content_type())
env.globals['format_message'] = format_message
-def replace_cids(body,mid):
- return body.replace('cid:',os.path.join(prefix,cachedir,mid)+'/')
+def replace_cids(body, mid):
+ return body.replace('cid:', os.path.join(prefix, cachedir, mid)+'/')
def find_cids(body):
- return re.findall(r'cid:([^ "\'>]*)', body)
+ return re.findall(r'cid:([^ "\'>]*)', body)
-def link_to_cached_file(part,mid,counter):
- filename = part.get_filename()
- if not filename:
- ext = mimetypes.guess_extension(part.get_content_type())
- if not ext:
- ext = '.bin'
- filename = 'part-%03d%s' % (counter, ext)
+def link_to_cached_file(part, mid, counter):
+ filename = part.get_filename()
+ if not filename:
+ ext = mimetypes.guess_extension(part.get_content_type())
+ if not ext:
+ ext = '.bin'
+ filename = 'part-%03d%s' % (counter, ext)
+ try:
+ os.makedirs(os.path.join(cachedir, mid))
+ except OSError:
+ pass
+ fn = os.path.join(cachedir, mid, filename) # FIXME escape mid,filename
+ fp = open(fn, 'wb')
+ if part.get_content_maintype() == 'text':
+ data = part.get_payload(decode=True)
+ data = decodeAnyway(data, part.get_content_charset('ascii')).encode('utf-8')
+ else:
try:
- os.makedirs(os.path.join(cachedir,mid))
+ data = part.get_payload(decode=True)
+ except:
+ data = part.get_payload(decode=False)
+ if data: fp.write(data)
+ fp.close()
+ if 'Content-ID' in part:
+ cid = part['Content-ID']
+ if cid[0] == '<' and cid[-1] == '>': cid = cid[1:-1]
+ cid_fn = os.path.join(cachedir, mid, cid) # FIXME escape mid,cid
+ try:
+ os.unlink(cid_fn)
except OSError:
pass
- fn = os.path.join(cachedir, mid, filename) # FIXME escape mid,filename
- fp = open(fn, 'wb')
- if part.get_content_maintype()=='text':
- data = part.get_payload(decode=True)
- data = decodeAnyway(data,part.get_content_charset('ascii')).encode('utf-8')
- else:
- try:
- data = part.get_payload(decode=True)
- except:
- data = part.get_payload(decode=False)
- if data: fp.write(data)
- fp.close()
- if 'Content-ID' in part:
- cid = part['Content-ID']
- if cid[0]=='<' and cid[-1]=='>': cid = cid[1:-1]
- cid_fn = os.path.join(cachedir, mid, cid) # FIXME escape mid,cid
- try:
- os.unlink(cid_fn)
- except OSError:
- pass
- os.link(fn,cid_fn)
- return (filename,cid)
- else:
- return (filename,None)
+ os.link(fn, cid_fn)
+ return (filename, cid)
+ else:
+ return (filename, None)
-if __name__ == '__main__':
- app = web.application(urls, globals())
- app.run()
+if __name__ == '__main__':
+ app = web.application(urls, globals())
+ app.run()
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 05/11] nmweb: (pylint) import built-in modules before third-party modules
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (3 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 04/11] nmweb: (pylint) normalize whitespace Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 06/11] nmweb: (pylint) put if clause on separate line from predicate Daniel Kahn Gillmor
` (6 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
---
contrib/notmuch-web/nmweb.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index bee4896e..6c687554 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -2,15 +2,16 @@
from __future__ import absolute_import
-import web
-from notmuch import Database, Query, Message
import urllib
from datetime import datetime
from mailbox import MaildirMessage
-import os
import mimetypes
import email
import re
+import os
+
+import web
+from notmuch import Database, Query, Message
from jinja2 import Environment, FileSystemLoader # FIXME to PackageLoader
from jinja2 import Markup
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 06/11] nmweb: (pylint) put if clause on separate line from predicate
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (4 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 05/11] nmweb: (pylint) import built-in modules before third-party modules Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 07/11] nmweb: shim for launching from gunicorn Daniel Kahn Gillmor
` (5 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
---
contrib/notmuch-web/nmweb.py | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 6c687554..0217fce0 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -249,7 +249,8 @@ def link_to_cached_file(part, mid, counter):
data = part.get_payload(decode=True)
except:
data = part.get_payload(decode=False)
- if data: fp.write(data)
+ if data:
+ fp.write(data)
fp.close()
if 'Content-ID' in part:
cid = part['Content-ID']
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 07/11] nmweb: shim for launching from gunicorn
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (5 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 06/11] nmweb: (pylint) put if clause on separate line from predicate Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 08/11] nmweb: use urllib.quote_plus() as plain quote_plus() Daniel Kahn Gillmor
` (4 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
file:///usr/share/doc/python-webpy-doc/html/deploying.html suggests
that gunicorn is an efficient way to deploy a webpy-based WSGI daemon.
This script is glue between nmweb.py and gunicorn.
---
contrib/notmuch-web/nmgunicorn.py | 11 +++++++++++
1 file changed, 11 insertions(+)
create mode 100644 contrib/notmuch-web/nmgunicorn.py
diff --git a/contrib/notmuch-web/nmgunicorn.py b/contrib/notmuch-web/nmgunicorn.py
new file mode 100644
index 00000000..e71ba12a
--- /dev/null
+++ b/contrib/notmuch-web/nmgunicorn.py
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+# to launch nmweb from gunicorn.
+
+from nmweb import urls, index, search, show
+import web
+
+app = web.application(urls, globals())
+
+# get the wsgi app from web.py application object
+wsgiapp = app.wsgifunc()
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 08/11] nmweb: use urllib.quote_plus() as plain quote_plus()
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (6 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 07/11] nmweb: shim for launching from gunicorn Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 09/11] nmweb: explicitly move to python3 Daniel Kahn Gillmor
` (3 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
This will make it easier to import quote_plus from urllib.parse, which
is where it is in python3.
---
contrib/notmuch-web/nmweb.py | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 0217fce0..31351399 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -2,7 +2,7 @@
from __future__ import absolute_import
-import urllib
+from urllib import quote_plus
from datetime import datetime
from mailbox import MaildirMessage
import mimetypes
@@ -34,7 +34,7 @@ def urlencode_filter(s):
if type(s) == 'Markup':
s = s.unescape()
s = s.encode('utf8')
- s = urllib.quote_plus(s)
+ s = quote_plus(s)
return Markup(s)
env.filters['url'] = urlencode_filter
@@ -69,7 +69,7 @@ class search:
redir = True
terms += ' %s..%s' % (afters, befores)
if redir:
- raise web.seeother('/search/%s' % urllib.quote_plus(terms))
+ raise web.seeother('/search/%s' % quote_plus(terms))
web.header('Content-type', 'text/html')
web.header('Transfer-Encoding', 'chunked')
db = Database()
@@ -107,7 +107,7 @@ def show_msgs(msgs):
frm = msg.get_header('From')
frm = mailto_addrs(frm)
subj = msg.get_header('Subject')
- lnk = urllib.quote_plus(msg.get_message_id())
+ lnk = quote_plus(msg.get_message_id())
rs = show_msgs(msg.get_replies())
r += '<li><font color=%s>%s—<a href="%s/show/%s">%s</a></font> %s</li>' % (
red, frm, prefix, lnk, subj, rs)
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 09/11] nmweb: explicitly move to python3
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (7 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 08/11] nmweb: use urllib.quote_plus() as plain quote_plus() Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 10/11] nmweb: hide numeric date fields' Daniel Kahn Gillmor
` (2 subsequent siblings)
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
---
contrib/notmuch-web/nmweb.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 31351399..eaeeb507 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -1,8 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
from __future__ import absolute_import
-from urllib import quote_plus
+from urllib.parse import quote_plus
from datetime import datetime
from mailbox import MaildirMessage
import mimetypes
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 10/11] nmweb: hide numeric date fields'
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (8 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 09/11] nmweb: explicitly move to python3 Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 11/11] nmweb: handle non-numeric timestamp inputs Daniel Kahn Gillmor
2018-03-24 23:05 ` clean up for nmweb/btsmail David Bremner
11 siblings, 0 replies; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
These are populated by javascript, but the user shouldn't see them.
---
contrib/notmuch-web/templates/base.html | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/contrib/notmuch-web/templates/base.html b/contrib/notmuch-web/templates/base.html
index 6d204562..e3743c93 100644
--- a/contrib/notmuch-web/templates/base.html
+++ b/contrib/notmuch-web/templates/base.html
@@ -20,9 +20,9 @@ src="{{sprefix}}/js/jquery-ui-1.8.21.custom.min.js"></script>
<form action="{{prefix}}/search/" method="GET" data-ajax="false">
<label for="terms">Terms</label><input id="terms" name="terms">
<label for="after">After</label><input id="after"
-name="after"><input id="afters" name="afters">
+name="after"><input type="hidden" id="afters" name="afters">
<label for="before">Before</label><input id="before"
-name="before"><input id="befores" name="befores">
+name="before"><input id="befores" type="hidden" name="befores">
<input type="submit" name="submit" id="submit" value="Search">
</form>
{% endblock searchform %}
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 11/11] nmweb: handle non-numeric timestamp inputs
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (9 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 10/11] nmweb: hide numeric date fields' Daniel Kahn Gillmor
@ 2018-02-05 4:19 ` Daniel Kahn Gillmor
2018-03-17 12:57 ` David Bremner
2018-03-24 23:05 ` clean up for nmweb/btsmail David Bremner
11 siblings, 1 reply; 15+ messages in thread
From: Daniel Kahn Gillmor @ 2018-02-05 4:19 UTC (permalink / raw)
To: Notmuch Mail
Without this check, it's trivial to crash the nmweb daemon with a
ValueError by putting a non-numeric value in befores or afters.
---
contrib/notmuch-web/nmweb.py | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index eaeeb507..21276b66 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -65,9 +65,12 @@ class search:
befores = web.input(befores=None).befores
else:
befores = '4294967296' # 2^32
- if int(afters) > 0 or int(befores) < 4294967296:
- redir = True
- terms += ' %s..%s' % (afters, befores)
+ try:
+ if int(afters) > 0 or int(befores) < 4294967296:
+ redir = True
+ terms += ' %s..%s' % (afters, befores)
+ except ValueError:
+ pass
if redir:
raise web.seeother('/search/%s' % quote_plus(terms))
web.header('Content-type', 'text/html')
--
2.15.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 04/11] nmweb: (pylint) normalize whitespace
2018-02-05 4:19 ` [PATCH 04/11] nmweb: (pylint) normalize whitespace Daniel Kahn Gillmor
@ 2018-03-17 12:37 ` David Bremner
0 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2018-03-17 12:37 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
> I've settled on:
>
> * 2-space indent
> * one space after comma
> * one space on either side of assignment and comparison operators
> * no trailing whitespace
>
> This now satisfies the indentation/whitespace checks of:
>
> pylint --indent-string=' ' --indent-after-paren=2 nmweb.py
No objection to the patch, but would it be possible to also update
.dir-locals.el (or a directory local version) for emacs?
d
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 11/11] nmweb: handle non-numeric timestamp inputs
2018-02-05 4:19 ` [PATCH 11/11] nmweb: handle non-numeric timestamp inputs Daniel Kahn Gillmor
@ 2018-03-17 12:57 ` David Bremner
0 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2018-03-17 12:57 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]
Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
> Without this check, it's trivial to crash the nmweb daemon with a
> ValueError by putting a non-numeric value in befores or afters.
I don't really understand what's going on here enough to comment on this
patch. Perhaps Brian can confirm. I notice currently the web page is
generating URL's like
https://nmbug.notmuchmail.org/btsmail/search/crash+date%3A%401516161600..%401520910000000
and those in turn are causing internal server errors (at least on the
instance on nmbug.
> ---
> contrib/notmuch-web/nmweb.py | 9 ++++++---
> 1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
> index eaeeb507..21276b66 100755
> --- a/contrib/notmuch-web/nmweb.py
> +++ b/contrib/notmuch-web/nmweb.py
> @@ -65,9 +65,12 @@ class search:
> befores = web.input(befores=None).befores
> else:
> befores = '4294967296' # 2^32
> - if int(afters) > 0 or int(befores) < 4294967296:
> - redir = True
> - terms += ' %s..%s' % (afters, befores)
> + try:
> + if int(afters) > 0 or int(befores) < 4294967296:
> + redir = True
> + terms += ' %s..%s' % (afters, befores)
> + except ValueError:
> + pass
> if redir:
> raise web.seeother('/search/%s' % quote_plus(terms))
> web.header('Content-type', 'text/html')
> --
> 2.15.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: clean up for nmweb/btsmail
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
` (10 preceding siblings ...)
2018-02-05 4:19 ` [PATCH 11/11] nmweb: handle non-numeric timestamp inputs Daniel Kahn Gillmor
@ 2018-03-24 23:05 ` David Bremner
11 siblings, 0 replies; 15+ messages in thread
From: David Bremner @ 2018-03-24 23:05 UTC (permalink / raw)
To: Daniel Kahn Gillmor, Notmuch Mail
Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:
> The patches that follow are some proposed cleanups for
> contrib/notmuch-web (aka nmweb, aka btsmail), which was originally
> proposed by Brian Sniffen.
So, the main problem with this series is that contrib/notmuch-web
doesn't exist on master.
Assuming we still want to include it, we need one or more patches to do
so.
d
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2018-03-24 23:05 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-05 4:19 clean up for nmweb/btsmail Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 01/11] nmweb: (pylint) use __future__.absolute_import Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 02/11] nmweb: (pylint) use replace() function directly on string object Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 03/11] nmweb: (pylint) use spaces instead of tabs Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 04/11] nmweb: (pylint) normalize whitespace Daniel Kahn Gillmor
2018-03-17 12:37 ` David Bremner
2018-02-05 4:19 ` [PATCH 05/11] nmweb: (pylint) import built-in modules before third-party modules Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 06/11] nmweb: (pylint) put if clause on separate line from predicate Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 07/11] nmweb: shim for launching from gunicorn Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 08/11] nmweb: use urllib.quote_plus() as plain quote_plus() Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 09/11] nmweb: explicitly move to python3 Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 10/11] nmweb: hide numeric date fields' Daniel Kahn Gillmor
2018-02-05 4:19 ` [PATCH 11/11] nmweb: handle non-numeric timestamp inputs Daniel Kahn Gillmor
2018-03-17 12:57 ` David Bremner
2018-03-24 23:05 ` clean up for nmweb/btsmail David Bremner
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).