clean up for nmweb/btsmail

clean up for nmweb/btsmail

[Daniel Kahn Gillmor]

The patches that follow are some proposed cleanups for
contrib/notmuch-web (aka nmweb, aka btsmail), which was originally
proposed by Brian Sniffen.

These are pretty mechanical changes -- nothing fancy added or removed,
just trying to make the code a little more maintainable and moving it
to python3.

You can also find this on the "nmweb" branch at
https://gitlab.com/dkg/notmuch

Regards,

        --dkg

[PATCH 01/11] nmweb: (pylint) use __future__.absolute_import

[Daniel Kahn Gillmor]

pylint --py3k suggests always including this in code that might run in
either python2 or python3.  it selects the python3 default.  (see
"pylint --help-msg=no-absolute-import" for more details)
---
 contrib/notmuch-web/nmweb.py | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index add0d631..c6d437c5 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -1,5 +1,7 @@
 #!/usr/bin/env python
 
+from __future__ import absolute_import
+
 import web
 from notmuch import Database,Query,Message
 import urllib
-- 
2.15.1

[PATCH 02/11] nmweb: (pylint) use replace() function directly on string object

[Daniel Kahn Gillmor]

Use of string.replace() is deprecated, better to invoke replace()
directly on the string object.
---
 contrib/notmuch-web/nmweb.py | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index c6d437c5..1f6e0145 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -10,7 +10,6 @@ from mailbox import MaildirMessage
 import os
 import mimetypes
 import email
-import string
 import re
 from jinja2 import Environment, FileSystemLoader # FIXME to PackageLoader
 from jinja2 import Markup
@@ -166,8 +165,7 @@ def format_message_walk(msg,mid):
           yield '<ul>'
           for subpart in part.get_payload():
             yield ('<li><a href="#%s">%s</a></li>' %
-                   (string.replace(subpart.get_content_type(),
-                                   '/', '-'),
+                   (subpart.get_content_type().replace('/', '-'),
                     subpart.get_content_type()))
           yield '</ul>'
       elif part.get_content_type() == 'message/rfc822':
@@ -191,7 +189,7 @@ def format_message_walk(msg,mid):
 	  yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
           yield '</div>'
         else:
-          yield '<div id="%s">' % string.replace(part.get_content_type(),'/','-')
+          yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
           (filename,cid) = link_to_cached_file(part,mid,counter)
           counter += 1
           yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
@@ -222,7 +220,7 @@ def format_message_walk(msg,mid):
 env.globals['format_message'] = format_message
 
 def replace_cids(body,mid):
-    return string.replace(body,'cid:',os.path.join(prefix,cachedir,mid)+'/')
+    return body.replace('cid:',os.path.join(prefix,cachedir,mid)+'/')
 
 def find_cids(body):
     return re.findall(r'cid:([^ "\'>]*)', body)
-- 
2.15.1

[PATCH 03/11] nmweb: (pylint) use spaces instead of tabs

[Daniel Kahn Gillmor]

---
 contrib/notmuch-web/nmweb.py | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 1f6e0145..b56c6b12 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -47,7 +47,7 @@ class index:
     tags = db.get_all_tags()
     return template.render(tags=tags,
                            title="Notmuch webmail",
-			   prefix=prefix,
+                           prefix=prefix,
                            sprefix=webprefix)
 
 class search:
@@ -80,7 +80,7 @@ class search:
                              ts=ts,
                              title=terms,
                              prefix=prefix,
-			     sprefix=webprefix)
+                             sprefix=webprefix)
 
 def format_time_range(start,end):
   if end-start < (60*60*24):
@@ -137,7 +137,7 @@ class show:
                            mid=mid,
                            title=m.get_header('Subject'),
                            prefix=prefix,
-			   sprefix=webprefix)
+                           sprefix=webprefix)
 
 def format_message(fn,mid):
     msg = MaildirMessage(open(fn))
@@ -184,9 +184,9 @@ def format_message_walk(msg,mid):
           decoded = decodeAnyway(unb64,part.get_content_charset('ascii'))
           cid_refd += find_cids(decoded)
           part.set_payload(replace_cids(decoded,mid).encode(part.get_content_charset('ascii')))
-	  (filename,cid) = link_to_cached_file(part,mid,counter)
-	  counter +=1
-	  yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
+          (filename,cid) = link_to_cached_file(part,mid,counter)
+          counter +=1
+          yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
           yield '</div>'
         else:
           yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
-- 
2.15.1

[PATCH 04/11] nmweb: (pylint) normalize whitespace

[Daniel Kahn Gillmor]

I've settled on:

 * 2-space indent
 * one space after comma
 * one space on either side of assignment and comparison operators
 * no trailing whitespace

This now satisfies the indentation/whitespace checks of:

    pylint --indent-string='  ' --indent-after-paren=2 nmweb.py
---
 contrib/notmuch-web/nmweb.py | 248 ++++++++++++++++++++++---------------------
 1 file changed, 125 insertions(+), 123 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index b56c6b12..bee4896e 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -3,7 +3,7 @@
 from __future__ import absolute_import
 
 import web
-from notmuch import Database,Query,Message
+from notmuch import Database, Query, Message
 import urllib
 from datetime import datetime
 from mailbox import MaildirMessage
@@ -27,14 +27,14 @@ urls = (
   '/', 'index',
   '/search/(.*)', 'search',
   '/show/(.*)', 'show',
-  )
+)
 
 def urlencode_filter(s):
-    if type(s) == 'Markup':
-        s = s.unescape()
-    s = s.encode('utf8')
-    s = urllib.quote_plus(s)
-    return Markup(s)
+  if type(s) == 'Markup':
+    s = s.unescape()
+  s = s.encode('utf8')
+  s = urllib.quote_plus(s)
+  return Markup(s)
 env.filters['url'] = urlencode_filter
 
 class index:
@@ -51,7 +51,7 @@ class index:
                            sprefix=webprefix)
 
 class search:
-  def GET(self,terms):
+  def GET(self, terms):
     redir = False
     if web.input(terms=None).terms:
       redir = True
@@ -59,20 +59,20 @@ class search:
     if web.input(afters=None).afters:
       afters = web.input(afters=None).afters[:-3]
     else:
-      afters='0'
+      afters = '0'
     if web.input(befores=None).befores:
       befores = web.input(befores=None).befores
     else:
       befores = '4294967296' # 2^32
     if int(afters) > 0 or int(befores) < 4294967296:
       redir = True
-      terms += ' %s..%s' % (afters,befores)
+      terms += ' %s..%s' % (afters, befores)
     if redir:
       raise web.seeother('/search/%s' % urllib.quote_plus(terms))
     web.header('Content-type', 'text/html')
     web.header('Transfer-Encoding', 'chunked')
     db = Database()
-    q = Query(db,terms)
+    q = Query(db, terms)
     q.set_sort(Query.SORT.NEWEST_FIRST)
     ts = q.search_threads()
     template = env.get_template('search.html')
@@ -82,19 +82,19 @@ class search:
                              prefix=prefix,
                              sprefix=webprefix)
 
-def format_time_range(start,end):
+def format_time_range(start, end):
   if end-start < (60*60*24):
     time = datetime.fromtimestamp(start).strftime('%Y %b %d %H:%M')
   else:
     start = datetime.fromtimestamp(start).strftime("%Y %b %d")
     end = datetime.fromtimestamp(end).strftime("%Y %b %d")
-    time = "%s through %s" % (start,end)
+    time = "%s through %s" % (start, end)
   return time
 env.globals['format_time_range'] = format_time_range
 
 def mailto_addrs(frm):
-    frm = email.utils.getaddresses([frm])
-    return ','.join(['<a href="mailto:%s">%s</a> ' % ((l,p) if p else (l,l)) for (p,l) in frm])
+  frm = email.utils.getaddresses([frm])
+  return ','.join(['<a href="mailto:%s">%s</a> ' % ((l, p) if p else (l, l)) for (p, l) in frm])
 env.globals['mailto_addrs'] = mailto_addrs
 
 def show_msgs(msgs):
@@ -102,32 +102,33 @@ def show_msgs(msgs):
   for msg in msgs:
     red = 'black'
     flag = msg.get_flag(Message.FLAG.MATCH)
-    if flag: red='red'
+    if flag: red = 'red'
     frm = msg.get_header('From')
     frm = mailto_addrs(frm)
     subj = msg.get_header('Subject')
     lnk = urllib.quote_plus(msg.get_message_id())
     rs = show_msgs(msg.get_replies())
-    r += '<li><font color=%s>%s&mdash;<a href="%s/show/%s">%s</a></font> %s</li>' % (red,frm,prefix,lnk,subj,rs)
+    r += '<li><font color=%s>%s&mdash;<a href="%s/show/%s">%s</a></font> %s</li>' % (
+      red, frm, prefix, lnk, subj, rs)
   r += '</ul>'
   return r
 env.globals['show_msgs'] = show_msgs
 
 # As email.message.walk, but showing close tags as well
 def mywalk(self):
-    yield self
-    if self.is_multipart():
-        for subpart in self.get_payload():
-            for subsubpart in mywalk(subpart):
-                yield subsubpart
-        yield 'close-div'
+  yield self
+  if self.is_multipart():
+    for subpart in self.get_payload():
+      for subsubpart in mywalk(subpart):
+        yield subsubpart
+    yield 'close-div'
 
 class show:
-  def GET(self,mid):
+  def GET(self, mid):
     web.header('Content-type', 'text/html')
     web.header('Transfer-Encoding', 'chunked')
     db = Database()
-    q = Query(db,'id:'+mid)
+    q = Query(db, 'id:'+mid)
     m = list(q.search_messages())[0]
     template = env.get_template('show.html')
     # FIXME add reply-all link with email.urils.getaddresses
@@ -139,11 +140,11 @@ class show:
                            prefix=prefix,
                            sprefix=webprefix)
 
-def format_message(fn,mid):
-    msg = MaildirMessage(open(fn))
-    return format_message_walk(msg,mid)
+def format_message(fn, mid):
+  msg = MaildirMessage(open(fn))
+  return format_message_walk(msg, mid)
 
-def decodeAnyway(txt,charset='ascii'):
+def decodeAnyway(txt, charset='ascii'):
   try:
     out = txt.decode(charset)
   except UnicodeDecodeError:
@@ -153,63 +154,45 @@ def decodeAnyway(txt,charset='ascii'):
       out = txt.decode('latin1')
   return out
 
-def format_message_walk(msg,mid):
-    counter = 0
-    cid_refd = []
-    for part in mywalk(msg):
-      if part=='close-div': 
-          yield '</div>'
-      elif part.get_content_maintype() == 'multipart': 
-        yield '<div class="multipart-%s">' % part.get_content_subtype()
-        if part.get_content_subtype() == 'alternative':
-          yield '<ul>'
-          for subpart in part.get_payload():
-            yield ('<li><a href="#%s">%s</a></li>' %
-                   (subpart.get_content_type().replace('/', '-'),
-                    subpart.get_content_type()))
-          yield '</ul>'
-      elif part.get_content_type() == 'message/rfc822':
-          # FIXME extract subject, date, to/cc/from into a separate template and use it here
-          yield '<div class="message-rfc822">'
-      elif part.get_content_maintype() == 'text':
-        if part.get_content_subtype() == 'plain':
-          yield '<div id="text-plain"><pre>'
-          out = part.get_payload(decode=True)
-          out = decodeAnyway(out,part.get_content_charset('ascii'))
-          yield out
-          yield '</pre></div>'
-        elif part.get_content_subtype() == 'html':
-          yield '<div id="text-html">'
-          unb64 = part.get_payload(decode=True)
-          decoded = decodeAnyway(unb64,part.get_content_charset('ascii'))
-          cid_refd += find_cids(decoded)
-          part.set_payload(replace_cids(decoded,mid).encode(part.get_content_charset('ascii')))
-          (filename,cid) = link_to_cached_file(part,mid,counter)
-          counter +=1
-          yield '<iframe class="embedded-html" src="%s">' % os.path.join(prefix,cachedir,mid,filename)
-          yield '</div>'
-        else:
-          yield '<div id="%s">' % (part.get_content_type().replace('/','-'),)
-          (filename,cid) = link_to_cached_file(part,mid,counter)
-          counter += 1
-          yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
-                                                           cachedir,
-                                                           mid,
-                                                           filename),
-                                              filename,
-                                              part.get_content_type())
-          yield '</div>'
-      elif part.get_content_maintype() == 'image':
-        (filename,cid) = link_to_cached_file(part,mid,counter)
-        if cid not in cid_refd:
-          counter += 1
-          yield '<img src="%s" alt="%s">' % (os.path.join(prefix,
-                                                          cachedir,
-                                                          mid,
-                                                          filename),
-                                             filename)
+def format_message_walk(msg, mid):
+  counter = 0
+  cid_refd = []
+  for part in mywalk(msg):
+    if part == 'close-div':
+      yield '</div>'
+    elif part.get_content_maintype() == 'multipart':
+      yield '<div class="multipart-%s">' % part.get_content_subtype()
+      if part.get_content_subtype() == 'alternative':
+        yield '<ul>'
+        for subpart in part.get_payload():
+          yield ('<li><a href="#%s">%s</a></li>' %
+                 (subpart.get_content_type().replace('/', '-'),
+                  subpart.get_content_type()))
+        yield '</ul>'
+    elif part.get_content_type() == 'message/rfc822':
+      # FIXME extract subject, date, to/cc/from into a separate template and use it here
+      yield '<div class="message-rfc822">'
+    elif part.get_content_maintype() == 'text':
+      if part.get_content_subtype() == 'plain':
+        yield '<div id="text-plain"><pre>'
+        out = part.get_payload(decode=True)
+        out = decodeAnyway(out, part.get_content_charset('ascii'))
+        yield out
+        yield '</pre></div>'
+      elif part.get_content_subtype() == 'html':
+        yield '<div id="text-html">'
+        unb64 = part.get_payload(decode=True)
+        decoded = decodeAnyway(unb64, part.get_content_charset('ascii'))
+        cid_refd += find_cids(decoded)
+        part.set_payload(replace_cids(decoded, mid).encode(part.get_content_charset('ascii')))
+        (filename, cid) = link_to_cached_file(part, mid, counter)
+        counter += 1
+        yield '<iframe class="embedded-html" src="%s">' % (
+          os.path.join(prefix, cachedir, mid, filename),)
+        yield '</div>'
       else:
-        (filename,cid) = link_to_cached_file(part,mid,counter)
+        yield '<div id="%s">' % (part.get_content_type().replace('/', '-'),)
+        (filename, cid) = link_to_cached_file(part, mid, counter)
         counter += 1
         yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
                                                          cachedir,
@@ -217,50 +200,69 @@ def format_message_walk(msg,mid):
                                                          filename),
                                             filename,
                                             part.get_content_type())
+        yield '</div>'
+    elif part.get_content_maintype() == 'image':
+      (filename, cid) = link_to_cached_file(part, mid, counter)
+      if cid not in cid_refd:
+        counter += 1
+        yield '<img src="%s" alt="%s">' % (os.path.join(prefix,
+                                                        cachedir,
+                                                        mid,
+                                                        filename),
+                                           filename)
+    else:
+      (filename, cid) = link_to_cached_file(part, mid, counter)
+      counter += 1
+      yield '<a href="%s">%s (%s)</a>' % (os.path.join(prefix,
+                                                       cachedir,
+                                                       mid,
+                                                       filename),
+                                          filename,
+                                          part.get_content_type())
 env.globals['format_message'] = format_message
 
-def replace_cids(body,mid):
-    return body.replace('cid:',os.path.join(prefix,cachedir,mid)+'/')
+def replace_cids(body, mid):
+  return body.replace('cid:', os.path.join(prefix, cachedir, mid)+'/')
 
 def find_cids(body):
-    return re.findall(r'cid:([^ "\'>]*)', body)
+  return re.findall(r'cid:([^ "\'>]*)', body)
 
-def link_to_cached_file(part,mid,counter):
-    filename = part.get_filename()
-    if not filename:
-      ext = mimetypes.guess_extension(part.get_content_type())
-      if not ext:
-        ext = '.bin'
-      filename = 'part-%03d%s' % (counter, ext)
+def link_to_cached_file(part, mid, counter):
+  filename = part.get_filename()
+  if not filename:
+    ext = mimetypes.guess_extension(part.get_content_type())
+    if not ext:
+      ext = '.bin'
+    filename = 'part-%03d%s' % (counter, ext)
+  try:
+    os.makedirs(os.path.join(cachedir, mid))
+  except OSError:
+    pass
+  fn = os.path.join(cachedir, mid, filename) # FIXME escape mid,filename
+  fp = open(fn, 'wb')
+  if part.get_content_maintype() == 'text':
+    data = part.get_payload(decode=True)
+    data = decodeAnyway(data, part.get_content_charset('ascii')).encode('utf-8')
+  else:
     try:
-      os.makedirs(os.path.join(cachedir,mid))
+      data = part.get_payload(decode=True)
+    except:
+      data = part.get_payload(decode=False)
+  if data: fp.write(data)
+  fp.close()
+  if 'Content-ID' in part:
+    cid = part['Content-ID']
+    if cid[0] == '<' and cid[-1] == '>': cid = cid[1:-1]
+    cid_fn = os.path.join(cachedir, mid, cid) # FIXME escape mid,cid
+    try:
+      os.unlink(cid_fn)
     except OSError:
       pass
-    fn = os.path.join(cachedir, mid, filename) # FIXME escape mid,filename
-    fp = open(fn, 'wb')
-    if part.get_content_maintype()=='text':
-      data = part.get_payload(decode=True)
-      data = decodeAnyway(data,part.get_content_charset('ascii')).encode('utf-8')
-    else:
-      try:
-        data = part.get_payload(decode=True)
-      except:
-        data = part.get_payload(decode=False)
-    if data: fp.write(data)
-    fp.close()
-    if 'Content-ID' in part:
-        cid = part['Content-ID']
-        if cid[0]=='<' and cid[-1]=='>': cid = cid[1:-1]
-        cid_fn = os.path.join(cachedir, mid, cid) # FIXME escape mid,cid
-        try:
-            os.unlink(cid_fn)
-        except OSError:
-            pass
-        os.link(fn,cid_fn)
-        return (filename,cid)
-    else:
-        return (filename,None)
+    os.link(fn, cid_fn)
+    return (filename, cid)
+  else:
+    return (filename, None)
 
-if __name__ == '__main__': 
-    app = web.application(urls, globals())
-    app.run()
+if __name__ == '__main__':
+  app = web.application(urls, globals())
+  app.run()
-- 
2.15.1

[PATCH 05/11] nmweb: (pylint) import built-in modules before third-party modules

[Daniel Kahn Gillmor]

---
 contrib/notmuch-web/nmweb.py | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index bee4896e..6c687554 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -2,15 +2,16 @@
 
 from __future__ import absolute_import
 
-import web
-from notmuch import Database, Query, Message
 import urllib
 from datetime import datetime
 from mailbox import MaildirMessage
-import os
 import mimetypes
 import email
 import re
+import os
+
+import web
+from notmuch import Database, Query, Message
 from jinja2 import Environment, FileSystemLoader # FIXME to PackageLoader
 from jinja2 import Markup
 
-- 
2.15.1

[PATCH 06/11] nmweb: (pylint) put if clause on separate line from predicate

[Daniel Kahn Gillmor]

---
 contrib/notmuch-web/nmweb.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 6c687554..0217fce0 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -249,7 +249,8 @@ def link_to_cached_file(part, mid, counter):
       data = part.get_payload(decode=True)
     except:
       data = part.get_payload(decode=False)
-  if data: fp.write(data)
+  if data:
+    fp.write(data)
   fp.close()
   if 'Content-ID' in part:
     cid = part['Content-ID']
-- 
2.15.1

[PATCH 07/11] nmweb: shim for launching from gunicorn

[Daniel Kahn Gillmor]

file:///usr/share/doc/python-webpy-doc/html/deploying.html suggests
that gunicorn is an efficient way to deploy a webpy-based WSGI daemon.

This script is glue between nmweb.py and gunicorn.
---
 contrib/notmuch-web/nmgunicorn.py | 11 +++++++++++
 1 file changed, 11 insertions(+)
 create mode 100644 contrib/notmuch-web/nmgunicorn.py

diff --git a/contrib/notmuch-web/nmgunicorn.py b/contrib/notmuch-web/nmgunicorn.py
new file mode 100644
index 00000000..e71ba12a
--- /dev/null
+++ b/contrib/notmuch-web/nmgunicorn.py
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+# to launch nmweb from gunicorn.
+
+from nmweb import urls, index, search, show
+import web
+
+app = web.application(urls, globals())
+
+# get the wsgi app from web.py application object
+wsgiapp = app.wsgifunc()
-- 
2.15.1

[PATCH 08/11] nmweb: use urllib.quote_plus() as plain quote_plus()

[Daniel Kahn Gillmor]

This will make it easier to import quote_plus from urllib.parse, which
is where it is in python3.
---
 contrib/notmuch-web/nmweb.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 0217fce0..31351399 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -2,7 +2,7 @@
 
 from __future__ import absolute_import
 
-import urllib
+from urllib import quote_plus
 from datetime import datetime
 from mailbox import MaildirMessage
 import mimetypes
@@ -34,7 +34,7 @@ def urlencode_filter(s):
   if type(s) == 'Markup':
     s = s.unescape()
   s = s.encode('utf8')
-  s = urllib.quote_plus(s)
+  s = quote_plus(s)
   return Markup(s)
 env.filters['url'] = urlencode_filter
 
@@ -69,7 +69,7 @@ class search:
       redir = True
       terms += ' %s..%s' % (afters, befores)
     if redir:
-      raise web.seeother('/search/%s' % urllib.quote_plus(terms))
+      raise web.seeother('/search/%s' % quote_plus(terms))
     web.header('Content-type', 'text/html')
     web.header('Transfer-Encoding', 'chunked')
     db = Database()
@@ -107,7 +107,7 @@ def show_msgs(msgs):
     frm = msg.get_header('From')
     frm = mailto_addrs(frm)
     subj = msg.get_header('Subject')
-    lnk = urllib.quote_plus(msg.get_message_id())
+    lnk = quote_plus(msg.get_message_id())
     rs = show_msgs(msg.get_replies())
     r += '<li><font color=%s>%s&mdash;<a href="%s/show/%s">%s</a></font> %s</li>' % (
       red, frm, prefix, lnk, subj, rs)
-- 
2.15.1

[PATCH 09/11] nmweb: explicitly move to python3

[Daniel Kahn Gillmor]

---
 contrib/notmuch-web/nmweb.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index 31351399..eaeeb507 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -1,8 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 from __future__ import absolute_import
 
-from urllib import quote_plus
+from urllib.parse import quote_plus
 from datetime import datetime
 from mailbox import MaildirMessage
 import mimetypes
-- 
2.15.1

[PATCH 10/11] nmweb: hide numeric date fields'

[Daniel Kahn Gillmor]

These are populated by javascript, but the user shouldn't see them.
---
 contrib/notmuch-web/templates/base.html | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/contrib/notmuch-web/templates/base.html b/contrib/notmuch-web/templates/base.html
index 6d204562..e3743c93 100644
--- a/contrib/notmuch-web/templates/base.html
+++ b/contrib/notmuch-web/templates/base.html
@@ -20,9 +20,9 @@ src="{{sprefix}}/js/jquery-ui-1.8.21.custom.min.js"></script>
 <form action="{{prefix}}/search/" method="GET" data-ajax="false">
 <label for="terms">Terms</label><input id="terms" name="terms">
 <label for="after">After</label><input id="after"
-name="after"><input id="afters" name="afters">
+name="after"><input type="hidden" id="afters" name="afters">
 <label for="before">Before</label><input id="before"
-name="before"><input id="befores" name="befores">
+name="before"><input id="befores" type="hidden" name="befores">
 <input type="submit" name="submit" id="submit" value="Search">
 </form>
 {% endblock searchform %}
-- 
2.15.1

[PATCH 11/11] nmweb: handle non-numeric timestamp inputs

[Daniel Kahn Gillmor]

Without this check, it's trivial to crash the nmweb daemon with a
ValueError by putting a non-numeric value in befores or afters.
---
 contrib/notmuch-web/nmweb.py | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
index eaeeb507..21276b66 100755
--- a/contrib/notmuch-web/nmweb.py
+++ b/contrib/notmuch-web/nmweb.py
@@ -65,9 +65,12 @@ class search:
       befores = web.input(befores=None).befores
     else:
       befores = '4294967296' # 2^32
-    if int(afters) > 0 or int(befores) < 4294967296:
-      redir = True
-      terms += ' %s..%s' % (afters, befores)
+    try:
+      if int(afters) > 0 or int(befores) < 4294967296:
+        redir = True
+        terms += ' %s..%s' % (afters, befores)
+    except ValueError:
+      pass
     if redir:
       raise web.seeother('/search/%s' % quote_plus(terms))
     web.header('Content-type', 'text/html')
-- 
2.15.1

Re: [PATCH 04/11] nmweb: (pylint) normalize whitespace

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> I've settled on:
>
>  * 2-space indent
>  * one space after comma
>  * one space on either side of assignment and comparison operators
>  * no trailing whitespace
>
> This now satisfies the indentation/whitespace checks of:
>
>     pylint --indent-string='  ' --indent-after-paren=2 nmweb.py

No objection to the patch, but would it be possible to also update
.dir-locals.el (or a directory local version) for emacs?

d

Re: [PATCH 11/11] nmweb: handle non-numeric timestamp inputs

[David Bremner]

[-- Attachment #1: Type: text/plain, Size: 1619 bytes --]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> Without this check, it's trivial to crash the nmweb daemon with a
> ValueError by putting a non-numeric value in befores or afters.

I don't really understand what's going on here enough to comment on this
patch. Perhaps Brian can confirm. I notice currently the web page is
generating URL's like

           https://nmbug.notmuchmail.org/btsmail/search/crash+date%3A%401516161600..%401520910000000

and those in turn are causing internal server errors (at least on the
instance on nmbug.

> ---
>  contrib/notmuch-web/nmweb.py | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
>
> diff --git a/contrib/notmuch-web/nmweb.py b/contrib/notmuch-web/nmweb.py
> index eaeeb507..21276b66 100755
> --- a/contrib/notmuch-web/nmweb.py
> +++ b/contrib/notmuch-web/nmweb.py
> @@ -65,9 +65,12 @@ class search:
>        befores = web.input(befores=None).befores
>      else:
>        befores = '4294967296' # 2^32
> -    if int(afters) > 0 or int(befores) < 4294967296:
> -      redir = True
> -      terms += ' %s..%s' % (afters, befores)
> +    try:
> +      if int(afters) > 0 or int(befores) < 4294967296:
> +        redir = True
> +        terms += ' %s..%s' % (afters, befores)
> +    except ValueError:
> +      pass
>      if redir:
>        raise web.seeother('/search/%s' % quote_plus(terms))
>      web.header('Content-type', 'text/html')
> -- 
> 2.15.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 658 bytes --]

Re: clean up for nmweb/btsmail

[David Bremner]

Daniel Kahn Gillmor <dkg@fifthhorseman.net> writes:

> The patches that follow are some proposed cleanups for
> contrib/notmuch-web (aka nmweb, aka btsmail), which was originally
> proposed by Brian Sniffen.

So, the main problem with this series is that contrib/notmuch-web
doesn't exist on master.

Assuming we still want to include it, we need one or more patches to do
so.

d