[PATCH v2 00/20] nmbug-status: Python-3-compatibility and general refactoring

[PATCH v2 00/20] nmbug-status: Python-3-compatibility and general refactoring

[W. Trevor King]

This is the second round of my initial series [1].  Thanks to Tomi and
David for reviewing v1.  General changes since v1:

* Python 2.6 compatibility fixes ('{0}'.format(...) and an OrderedDict
  stub) [2].

Changes to the HTML output since v1:

* Escape &, <, and > in display data [3].
* Changed color scheme to #ffd96e / #bce [4].
* Added additional spacing between messages (smaller than the existing
  spacing between threads) [5].
* Slug section titles instead of quoting them [6].

[1]: id="cover.1391424512.git.wking@tremily.us"
     http://thread.gmane.org/gmane.mail.notmuch.general/17061
[2]: id="20140204173914.GR19935@odin.tremily.us"
     http://article.gmane.org/gmane.mail.notmuch.general/17114
[3]: id="20140207221531.GA17142@odin.tremily.us"
     http://article.gmane.org/gmane.mail.notmuch.general/17141
[4]: id="20140208190923.GE14197@odin.tremily.us"
     http://article.gmane.org/gmane.mail.notmuch.general/17151
[5]: id="20140208221925.GD17142@odin.tremily.us"
     http://article.gmane.org/gmane.mail.notmuch.general/17153
[6]: id="20140208231804.GE17142@odin.tremily.us"
     http://article.gmane.org/gmane.mail.notmuch.general/17155

W. Trevor King (20):
  nmbug-status: Convert to Python-3-compatible print functions
  nmbug-status: Use email.utils instead of rfc822
  nmbug-status: Decode Popen output using the user's locale
  nmbug-status: Factor config-loading out into read_config
  nmbug-status: Add metavars for --config and --get-query
  nmbug-status: Consolidate functions and main code
  nmbug-status: Don't require write access
  nmbug-status: Consolidate HTML header printing
  nmbug-status: Add a Python-3-compatible urllib.parse.quote import
  nmbug-status: Add Page and HtmlPage for modular rendering
  nmbug-status: Add an OrderedDict stub for Python 2.6
  nmbug-status: Normalize table HTML indentation
  nmbug-status: Convert from XHTML 1.0 to HTML 5
  nmbug-status: Encode output using the user's locale
  nmbug-status: Anchor with h3 ids instead of a names
  nmbug-status: Slug the title when using it as an id
  nmbug-status: Use <code> and <p> markup where appropriate
  nmbug-status: Color threads in HTML output
  nmbug-status: Escape &, <, and > in HTML display data
  nmbug-status: Add inter-message padding

 devel/nmbug/nmbug-status | 448 +++++++++++++++++++++++++++++++----------------
 1 file changed, 297 insertions(+), 151 deletions(-)

-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 01/20] nmbug-status: Convert to Python-3-compatible print functions

[W. Trevor King]

We shouldn't require folks to install Python 2 to run nmbug-status.
---
 devel/nmbug/nmbug-status | 44 +++++++++++++++++++++++---------------------
 1 file changed, 23 insertions(+), 21 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 934c895..b1feee9 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -6,6 +6,8 @@
 #       - python 2.6 for json
 #       - argparse; either python 2.7, or install separately
 
+from __future__ import print_function
+
 import datetime
 import rfc822
 import urllib
@@ -49,12 +51,12 @@ config = json.load(fp)
 
 if args.list_views:
     for view in config['views']:
-        print view['title']
+        print(view['title'])
     sys.exit(0)
 elif args.get_query != None:
     for view in config['views']:
         if args.get_query == view['title']:
-            print ' and '.join(view['query'])
+            print(' and '.join(view['query']))
     sys.exit(0)
 else:
     # only import notmuch if needed
@@ -75,7 +77,7 @@ class Thread:
 
 def output_with_separator(threadlist, sep):
     outputs = (thread.join_utf8_with_newlines() for thread in threadlist)
-    print sep.join(outputs)
+    print(sep.join(outputs))
 
 headers = ['date', 'from', 'subject']
 
@@ -93,13 +95,13 @@ def print_view(title, query, comment):
     lines = None
 
     if output_format == 'html':
-        print '<h3><a name="%s" />%s</h3>' % (title, title)
-        print comment
-        print 'The view is generated from the following query:'
-        print '<blockquote>'
-        print query_string
-        print '</blockquote>'
-        print '<table>\n'
+        print('<h3><a name="%s" />%s</h3>' % (title, title))
+        print(comment)
+        print('The view is generated from the following query:')
+        print('<blockquote>')
+        print(query_string)
+        print('</blockquote>')
+        print('<table>\n')
 
     for m in q_new.search_messages():
 
@@ -156,7 +158,7 @@ def print_view(title, query, comment):
     if output_format == 'html':
         output_with_separator(threadlist,
                               '\n<tr><td colspan="2"><br /></td></tr>\n')
-        print '</table>'
+        print('</table>')
     else:
         output_with_separator(threadlist, '\n\n')
 
@@ -165,26 +167,26 @@ def print_view(title, query, comment):
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
 
 if output_format == 'html':
-    print '''<?xml version="1.0" encoding="utf-8" ?>
+    print('''<?xml version="1.0" encoding="utf-8" ?>
 <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>Notmuch Patches</title>
 </head>
-<body>'''
-    print '<h2>Notmuch Patches</h2>'
-    print 'Generated: %s<br />' % datetime.datetime.utcnow().date()
-    print 'For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>'
+<body>''')
+    print('<h2>Notmuch Patches</h2>')
+    print('Generated: %s<br />' % datetime.datetime.utcnow().date())
+    print('For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>')
 
-    print '<h3>Views</h3>'
-    print '<ul>'
+    print('<h3>Views</h3>')
+    print('<ul>')
     for view in config['views']:
-        print '<li><a href="#%(title)s">%(title)s</a></li>' % view
-    print '</ul>'
+        print('<li><a href="#%(title)s">%(title)s</a></li>' % view)
+    print('</ul>')
 
 for view in config['views']:
     print_view(**view)
 
 if output_format == 'html':
-    print '</body>\n</html>'
+    print('</body>\n</html>')
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 02/20] nmbug-status: Use email.utils instead of rfc822

[W. Trevor King]

rfc822 has been deprecated since Python 2.3, and it's gone in
Python 3 [1].

[1]: http://docs.python.org/2/library/rfc822.html
---
 devel/nmbug/nmbug-status | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index b1feee9..6525176 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -9,7 +9,7 @@
 from __future__ import print_function
 
 import datetime
-import rfc822
+import email.utils
 import urllib
 import json
 import argparse
@@ -128,7 +128,7 @@ def print_view(title, query, comment):
                 val = str.join(' ', val.split(None)[1:4])
                 val = str(datetime.datetime.strptime(val, '%d %b %Y').date())
             elif header == 'from':
-                (val, addr) = rfc822.parseaddr(val)
+                (val, addr) = email.utils.parseaddr(val)
                 if val == '':
                     val = addr.split('@')[0]
 
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 03/20] nmbug-status: Decode Popen output using the user's locale

[W. Trevor King]

Avoid:

  $ ./nmbug-status --list-views
  Traceback (most recent call last):
    File "./nmbug-status", line 47, in <module>
      'cat-file', 'blob', sha1+':status-config.json'],
  TypeError: can't concat bytes to str

by explicitly converting the byte-stream read from Popen into a
Unicode string.  On Python 2, this conversion is str -> unicode; on
Python 3 it is bytes -> str.

_ENCODING is derived from the user's locale (or system default) in an
attempt to match Git's output encoding.  It may be more robust to skip
the encoding/decoding by using a Python wrapper like pygit2 [1] for
Git access.  That's a fairly heavy dependency though, and using the
locale will probably work.

[1]: http://www.pygit2.org/
---
 devel/nmbug/nmbug-status | 25 ++++++++++++++++---------
 1 file changed, 16 insertions(+), 9 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 6525176..a7a391d 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -8,8 +8,10 @@
 
 from __future__ import print_function
 
+import codecs
 import datetime
 import email.utils
+import locale
 import urllib
 import json
 import argparse
@@ -17,6 +19,10 @@ import os
 import sys
 import subprocess
 
+
+_ENCODING = locale.getpreferredencoding() or sys.getdefaultencoding()
+
+
 # parse command line arguments
 
 parser = argparse.ArgumentParser()
@@ -37,15 +43,16 @@ else:
     nmbhome = os.getenv('NMBGIT', os.path.expanduser('~/.nmbug'))
 
     # read only the first line from the pipe
-    sha1 = subprocess.Popen(['git', '--git-dir', nmbhome,
-                             'show-ref', '-s', 'config'],
-                            stdout=subprocess.PIPE).stdout.readline()
-
-    sha1 = sha1.rstrip()
-
-    fp = subprocess.Popen(['git', '--git-dir', nmbhome,
-                           'cat-file', 'blob', sha1+':status-config.json'],
-                          stdout=subprocess.PIPE).stdout
+    sha1_bytes = subprocess.Popen(
+        ['git', '--git-dir', nmbhome, 'show-ref', '-s', 'config'],
+        stdout=subprocess.PIPE).stdout.readline()
+    sha1 = sha1_bytes.decode(_ENCODING).rstrip()
+
+    fp_byte_stream = subprocess.Popen(
+        ['git', '--git-dir', nmbhome, 'cat-file', 'blob',
+         sha1+':status-config.json'],
+        stdout=subprocess.PIPE).stdout
+    fp = codecs.getreader(encoding=_ENCODING)(stream=fp_byte_stream)
 
 config = json.load(fp)
 
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 04/20] nmbug-status: Factor config-loading out into read_config

[W. Trevor King]

By isolating this peripheral handling, we make the core logic of
nmbug-status easier to read.
---
 devel/nmbug/nmbug-status | 45 +++++++++++++++++++++++++--------------------
 1 file changed, 25 insertions(+), 20 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index a7a391d..6dfbe4d 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -23,6 +23,30 @@ import subprocess
 _ENCODING = locale.getpreferredencoding() or sys.getdefaultencoding()
 
 
+def read_config(path=None, encoding=None):
+    "Read config from json file"
+    if not encoding:
+        encoding = _ENCODING
+    if path:
+        fp = open(path)
+    else:
+        nmbhome = os.getenv('NMBGIT', os.path.expanduser('~/.nmbug'))
+
+        # read only the first line from the pipe
+        sha1_bytes = subprocess.Popen(
+            ['git', '--git-dir', nmbhome, 'show-ref', '-s', 'config'],
+            stdout=subprocess.PIPE).stdout.readline()
+        sha1 = sha1_bytes.decode(encoding).rstrip()
+
+        fp_byte_stream = subprocess.Popen(
+            ['git', '--git-dir', nmbhome, 'cat-file', 'blob',
+             sha1+':status-config.json'],
+            stdout=subprocess.PIPE).stdout
+        fp = codecs.getreader(encoding=encoding)(stream=fp_byte_stream)
+
+    return json.load(fp)
+
+
 # parse command line arguments
 
 parser = argparse.ArgumentParser()
@@ -35,26 +59,7 @@ parser.add_argument('--get-query', help='get query for view')
 
 args = parser.parse_args()
 
-# read config from json file
-
-if args.config != None:
-    fp = open(args.config)
-else:
-    nmbhome = os.getenv('NMBGIT', os.path.expanduser('~/.nmbug'))
-
-    # read only the first line from the pipe
-    sha1_bytes = subprocess.Popen(
-        ['git', '--git-dir', nmbhome, 'show-ref', '-s', 'config'],
-        stdout=subprocess.PIPE).stdout.readline()
-    sha1 = sha1_bytes.decode(_ENCODING).rstrip()
-
-    fp_byte_stream = subprocess.Popen(
-        ['git', '--git-dir', nmbhome, 'cat-file', 'blob',
-         sha1+':status-config.json'],
-        stdout=subprocess.PIPE).stdout
-    fp = codecs.getreader(encoding=_ENCODING)(stream=fp_byte_stream)
-
-config = json.load(fp)
+config = read_config(path=args.config)
 
 if args.list_views:
     for view in config['views']:
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 05/20] nmbug-status: Add metavars for --config and --get-query

[W. Trevor King]

Now the suggested usage (listed by 'nmbug-status --help') is:

  usage: nmbug-status [-h] [--text] [--config PATH] [--list-views]
                      [--get-query VIEW]

instead of the less obvious:

  usage: nmbug-status [-h] [--text] [--config CONFIG] [--list-views]
                      [--get-query GET_QUERY]
---
 devel/nmbug/nmbug-status | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 6dfbe4d..22eeb5c 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -52,10 +52,12 @@ def read_config(path=None, encoding=None):
 parser = argparse.ArgumentParser()
 parser.add_argument('--text', help='output plain text format',
                     action='store_true')
-parser.add_argument('--config', help='load config from given file')
+parser.add_argument('--config', help='load config from given file',
+                    metavar='PATH')
 parser.add_argument('--list-views', help='list views',
                     action='store_true')
-parser.add_argument('--get-query', help='get query for view')
+parser.add_argument('--get-query', help='get query for view',
+                    metavar='VIEW')
 
 args = parser.parse_args()
 
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 06/20] nmbug-status: Consolidate functions and main code

[W. Trevor King]

The definitions of Thread, output_with_separator, and print_view were
between the main argparse and view-printing code.  Group them together
with our existing read_config at the top of the module, which makes
for easier reading in the main section.

I also:

* Made 'headers' a print_view argument instead of a module-level
  global.  The list -> tuple conversion avoids having a mutable
  default argument, which makes some people jumpy ;).

* Made 'db' a print_view argument instead of relying on the global
  namespace to access it from print_view.
---
 devel/nmbug/nmbug-status | 78 +++++++++++++++++++++++++-----------------------
 1 file changed, 40 insertions(+), 38 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 22eeb5c..199892f 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -47,40 +47,6 @@ def read_config(path=None, encoding=None):
     return json.load(fp)
 
 
-# parse command line arguments
-
-parser = argparse.ArgumentParser()
-parser.add_argument('--text', help='output plain text format',
-                    action='store_true')
-parser.add_argument('--config', help='load config from given file',
-                    metavar='PATH')
-parser.add_argument('--list-views', help='list views',
-                    action='store_true')
-parser.add_argument('--get-query', help='get query for view',
-                    metavar='VIEW')
-
-args = parser.parse_args()
-
-config = read_config(path=args.config)
-
-if args.list_views:
-    for view in config['views']:
-        print(view['title'])
-    sys.exit(0)
-elif args.get_query != None:
-    for view in config['views']:
-        if args.get_query == view['title']:
-            print(' and '.join(view['query']))
-    sys.exit(0)
-else:
-    # only import notmuch if needed
-    import notmuch
-
-if args.text:
-    output_format = 'text'
-else:
-    output_format = 'html'
-
 class Thread:
     def __init__(self, last, lines):
         self.last = last
@@ -89,16 +55,17 @@ class Thread:
     def join_utf8_with_newlines(self):
         return '\n'.join( (line.encode('utf-8') for line in self.lines) )
 
+
 def output_with_separator(threadlist, sep):
     outputs = (thread.join_utf8_with_newlines() for thread in threadlist)
     print(sep.join(outputs))
 
-headers = ['date', 'from', 'subject']
 
-def print_view(title, query, comment):
+def print_view(database, title, query, comment,
+               headers=('date', 'from', 'subject')):
 
     query_string = ' and '.join(query)
-    q_new = notmuch.Query(db, query_string)
+    q_new = notmuch.Query(database, query_string)
     q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
 
     last_thread_id = ''
@@ -176,6 +143,41 @@ def print_view(title, query, comment):
     else:
         output_with_separator(threadlist, '\n\n')
 
+
+# parse command line arguments
+
+parser = argparse.ArgumentParser()
+parser.add_argument('--text', help='output plain text format',
+                    action='store_true')
+parser.add_argument('--config', help='load config from given file',
+                    metavar='PATH')
+parser.add_argument('--list-views', help='list views',
+                    action='store_true')
+parser.add_argument('--get-query', help='get query for view',
+                    metavar='VIEW')
+
+args = parser.parse_args()
+
+config = read_config(path=args.config)
+
+if args.list_views:
+    for view in config['views']:
+        print(view['title'])
+    sys.exit(0)
+elif args.get_query != None:
+    for view in config['views']:
+        if args.get_query == view['title']:
+            print(' and '.join(view['query']))
+    sys.exit(0)
+else:
+    # only import notmuch if needed
+    import notmuch
+
+if args.text:
+    output_format = 'text'
+else:
+    output_format = 'html'
+
 # main program
 
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
@@ -200,7 +202,7 @@ if output_format == 'html':
     print('</ul>')
 
 for view in config['views']:
-    print_view(**view)
+    print_view(database=db, **view)
 
 if output_format == 'html':
     print('</body>\n</html>')
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 07/20] nmbug-status: Don't require write access

[W. Trevor King]

The database in only used for notmuch.Query, so there's no need for
write access.  This allows nmbug-status to run while the database is
being updated, without raising:

  A Xapian exception occurred opening database: Unable to get write lock on …: already locked
  Traceback (most recent call last):
    File "./nmbug-status", line 182, in <module>
      db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
    File "/…/notmuch/database.py", line 154, in __init__
      self.open(path, mode)
    File "/…/notmuch/database.py", line 214, in open
      raise NotmuchError(status)
  notmuch.errors.XapianError
---
 devel/nmbug/nmbug-status | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 199892f..be3e28e 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -180,7 +180,7 @@ else:
 
 # main program
 
-db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)
+db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
 
 if output_format == 'html':
     print('''<?xml version="1.0" encoding="utf-8" ?>
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 08/20] nmbug-status: Consolidate HTML header printing

[W. Trevor King]

Make this all one big string, using '...{date}...'.format(date=...) to
inject the date [1].  This syntax was added in Python 2.6, and is
preferred to %-formatting in Python 3 [1].

[1]: http://docs.python.org/2/library/stdtypes.html#str.format
---
 devel/nmbug/nmbug-status | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index be3e28e..3aa83b6 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -190,13 +190,12 @@ if output_format == 'html':
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>Notmuch Patches</title>
 </head>
-<body>''')
-    print('<h2>Notmuch Patches</h2>')
-    print('Generated: %s<br />' % datetime.datetime.utcnow().date())
-    print('For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>')
-
-    print('<h3>Views</h3>')
-    print('<ul>')
+<body>
+<h2>Notmuch Patches</h2>
+Generated: {date}<br />
+For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
+<h3>Views</h3>
+<ul>'''.format(date=datetime.datetime.utcnow().date()))
     for view in config['views']:
         print('<li><a href="#%(title)s">%(title)s</a></li>' % view)
     print('</ul>')
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 09/20] nmbug-status: Add a Python-3-compatible urllib.parse.quote import

[W. Trevor King]

Python 2's urllib.quote [1] has moved to urllib.parse.quote in Python
3 [2].

[1]: http://docs.python.org/2/library/urllib.html#urllib.quote
[2]: http://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote
---
 devel/nmbug/nmbug-status | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 3aa83b6..22b6b10 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -12,7 +12,10 @@ import codecs
 import datetime
 import email.utils
 import locale
-import urllib
+try:  # Python 3
+    from urllib.parse import quote
+except ImportError:  # Python 2
+    from urllib import quote
 import json
 import argparse
 import os
@@ -124,8 +127,8 @@ def print_view(database, title, query, comment,
 
         if output_format == 'html':
 
-            out['subject'] = '<a href="http://mid.gmane.org/%s">%s</a>' \
-                % (urllib.quote(mid), out['subject'])
+            out['subject'] = '<a href="http://mid.gmane.org/%s">%s</a>' % (
+                quote(mid), out['subject'])
 
             lines.append(' <tr><td>%s' % out['date'])
             lines.append('</td><td>%s' % out['id'])
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 10/20] nmbug-status: Add Page and HtmlPage for modular rendering

[W. Trevor King]

I was having trouble understanding the logic of the longish print_view
function, so I refactored the output generation into modular bits.
The basic text rendering is handled by Page, which has enough hooks
that HtmlPage can borrow the logic and slot-in HTML generators.

By modularizing the logic it should also be easier to build other
renderers if folks want to customize the layout for other projects.

Timezones
=========

This commit has not effect on the output, except that some dates have
been converted from the sender's timezone to UTC due to:

  -            val = m.get_header(header)
  -            ...
  -            if header == 'date':
  -                val = str.join(' ', val.split(None)[1:4])
  -                val = str(datetime.datetime.strptime(val, '%d %b %Y').date())
  ...
  +                value = str(datetime.datetime.utcfromtimestamp(
  +                    message.get_date()).date())

I also tweaked the HTML header date to be utcnow instead of the local
now() to make all times independent of the generator's local time.
This matches Gmane, which converts all Date headers to UTC (although
they use a 'GMT' suffix).  Notmuch uses
g_mime_utils_header_decode_date to calculate the UTC timestamps, but
uses a NULL tz_offset which drops the information we'd need to get
back to the sender's local time [1].  With the generator's local time
arbitrarily different from the sender's and viewer's local time,
sticking with UTC seems the best bet.

[1]: https://developer.gnome.org/gmime/stable/gmime-gmime-utils.html#g-mime-utils-header-decode-date
---
 devel/nmbug/nmbug-status | 292 +++++++++++++++++++++++++++--------------------
 1 file changed, 171 insertions(+), 121 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 22b6b10..6aa2583 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -5,10 +5,13 @@
 # dependencies
 #       - python 2.6 for json
 #       - argparse; either python 2.7, or install separately
+#       - collections.OrderedDict; python 2.7
 
 from __future__ import print_function
+from __future__ import unicode_literals
 
 import codecs
+import collections
 import datetime
 import email.utils
 import locale
@@ -24,6 +27,7 @@ import subprocess
 
 
 _ENCODING = locale.getpreferredencoding() or sys.getdefaultencoding()
+_PAGES = {}
 
 
 def read_config(path=None, encoding=None):
@@ -50,104 +54,175 @@ def read_config(path=None, encoding=None):
     return json.load(fp)
 
 
-class Thread:
-    def __init__(self, last, lines):
-        self.last = last
-        self.lines = lines
-
-    def join_utf8_with_newlines(self):
-        return '\n'.join( (line.encode('utf-8') for line in self.lines) )
-
-
-def output_with_separator(threadlist, sep):
-    outputs = (thread.join_utf8_with_newlines() for thread in threadlist)
-    print(sep.join(outputs))
-
-
-def print_view(database, title, query, comment,
-               headers=('date', 'from', 'subject')):
-
-    query_string = ' and '.join(query)
-    q_new = notmuch.Query(database, query_string)
-    q_new.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
-
-    last_thread_id = ''
-    threads = {}
-    threadlist = []
-    out = {}
-    last = None
-    lines = None
-
-    if output_format == 'html':
-        print('<h3><a name="%s" />%s</h3>' % (title, title))
-        print(comment)
-        print('The view is generated from the following query:')
-        print('<blockquote>')
-        print(query_string)
-        print('</blockquote>')
-        print('<table>\n')
-
-    for m in q_new.search_messages():
-
-        thread_id = m.get_thread_id()
-
-        if thread_id != last_thread_id:
-            if threads.has_key(thread_id):
-                last = threads[thread_id].last
-                lines = threads[thread_id].lines
+class Thread (list):
+    def __init__(self):
+        self.running_data = {}
+
+
+class Page (object):
+    def __init__(self, header=None, footer=None):
+        self.header = header
+        self.footer = footer
+
+    def write(self, database, views, stream=None):
+        if not stream:
+            try:  # Python 3
+                byte_stream = sys.stdout.buffer
+            except AttributeError:  # Python 2
+                byte_stream = sys.stdout
+            stream = codecs.getwriter(encoding='UTF-8')(stream=byte_stream)
+        self._write_header(views=views, stream=stream)
+        for view in views:
+            self._write_view(database=database, view=view, stream=stream)
+        self._write_footer(views=views, stream=stream)
+
+    def _write_header(self, views, stream):
+        if self.header:
+            stream.write(self.header)
+
+    def _write_footer(self, views, stream):
+        if self.footer:
+            stream.write(self.footer)
+
+    def _write_view(self, database, view, stream):
+        if 'query-string' not in view:
+            query = view['query']
+            view['query-string'] = ' and '.join(query)
+        q = notmuch.Query(database, view['query-string'])
+        q.set_sort(notmuch.Query.SORT.OLDEST_FIRST)
+        threads = self._get_threads(messages=q.search_messages())
+        self._write_view_header(view=view, stream=stream)
+        self._write_threads(threads=threads, stream=stream)
+
+    def _get_threads(self, messages):
+        threads = collections.OrderedDict()
+        for message in messages:
+            thread_id = message.get_thread_id()
+            if thread_id in threads:
+                thread = threads[thread_id]
             else:
-                last = {}
-                lines = []
-                thread = Thread(last, lines)
+                thread = Thread()
                 threads[thread_id] = thread
-                for h in headers:
-                    last[h] = ''
-                threadlist.append(thread)
-            last_thread_id = thread_id
-
+            thread.running_data, display_data = self._message_display_data(
+                running_data=thread.running_data, message=message)
+            thread.append(display_data)
+        return list(threads.values())
+
+    def _write_view_header(self, view, stream):
+        pass
+
+    def _write_threads(self, threads, stream):
+        for thread in threads:
+            for message_display_data in thread:
+                stream.write(
+                    ('{date:10.10s} {from:20.20s} {subject:40.40s}\n'
+                     '{message-id-term:>72}\n'
+                     ).format(**message_display_data))
+            if thread != threads[-1]:
+                stream.write('\n')
+
+    def _message_display_data(self, running_data, message):
+        headers = ('thread-id', 'message-id', 'date', 'from', 'subject')
+        data = {}
         for header in headers:
-            val = m.get_header(header)
-
-            if header == 'date':
-                val = str.join(' ', val.split(None)[1:4])
-                val = str(datetime.datetime.strptime(val, '%d %b %Y').date())
-            elif header == 'from':
-                (val, addr) = email.utils.parseaddr(val)
-                if val == '':
-                    val = addr.split('@')[0]
-
-            if header != 'subject' and last[header] == val:
-                out[header] = ''
+            if header == 'thread-id':
+                value = message.get_thread_id()
+            elif header == 'message-id':
+                value = message.get_message_id()
+                data['message-id-term'] = 'id:"{0}"'.format(value)
+            elif header == 'date':
+                value = str(datetime.datetime.utcfromtimestamp(
+                    message.get_date()).date())
             else:
-                out[header] = val
-                last[header] = val
-
-        mid = m.get_message_id()
-        out['id'] = 'id:"%s"' % mid
-
-        if output_format == 'html':
-
-            out['subject'] = '<a href="http://mid.gmane.org/%s">%s</a>' % (
-                quote(mid), out['subject'])
-
-            lines.append(' <tr><td>%s' % out['date'])
-            lines.append('</td><td>%s' % out['id'])
-            lines.append('</td></tr>')
-            lines.append(' <tr><td>%s' % out['from'])
-            lines.append('</td><td>%s' % out['subject'])
-            lines.append('</td></tr>')
-        else:
-            lines.append('%(date)-10.10s %(from)-20.20s %(subject)-40.40s\n%(id)72s' % out)
-
-    if output_format == 'html':
-        output_with_separator(threadlist,
-                              '\n<tr><td colspan="2"><br /></td></tr>\n')
-        print('</table>')
-    else:
-        output_with_separator(threadlist, '\n\n')
-
+                value = message.get_header(header)
+            if header == 'from':
+                (value, addr) = email.utils.parseaddr(value)
+                if not value:
+                    value = addr.split('@')[0]
+            data[header] = value
+        next_running_data = data.copy()
+        for header, value in data.items():
+            if header in ['message-id', 'subject']:
+                continue
+            if value == running_data.get(header, None):
+                data[header] = ''
+        return (next_running_data, data)
+
+
+class HtmlPage (Page):
+    def _write_header(self, views, stream):
+        super(HtmlPage, self)._write_header(views=views, stream=stream)
+        stream.write('<ul>\n')
+        for view in views:
+            stream.write(
+                '<li><a href="#{title}">{title}</a></li>\n'.format(**view))
+        stream.write('</ul>\n')
+
+    def _write_view_header(self, view, stream):
+        stream.write('<h3><a name="{title}" />{title}</h3>\n'.format(**view))
+        if 'comment' in view:
+            stream.write(view['comment'])
+            stream.write('\n')
+        for line in [
+                'The view is generated from the following query:',
+                '<blockquote>',
+                view['query-string'],
+                '</blockquote>',
+                ]:
+            stream.write(line)
+            stream.write('\n')
+
+    def _write_threads(self, threads, stream):
+        if not threads:
+            return
+        stream.write('<table>\n')
+        for thread in threads:
+            for message_display_data in thread:
+                stream.write((
+                    '<tr><td>{date}\n'
+                    '</td><td>{message-id-term}\n'
+                    '</td></tr>\n'
+                    '<tr><td>{from}\n'
+                    '</td><td>{subject}\n'
+                    '</td></tr>\n'
+                    ).format(**message_display_data))
+            if thread != threads[-1]:
+                stream.write('<tr><td colspan="2"><br /></td></tr>\n')
+        stream.write('</table>\n')
+
+    def _message_display_data(self, *args, **kwargs):
+        running_data, display_data = super(
+            HtmlPage, self)._message_display_data(
+                *args, **kwargs)
+        if 'subject' in display_data and 'message-id' in display_data:
+            d = {
+                'message-id': quote(display_data['message-id']),
+                'subject': display_data['subject'],
+                }
+            display_data['subject'] = (
+                '<a href="http://mid.gmane.org/{message-id}">{subject}</a>'
+                ).format(**d)
+        return (running_data, display_data)
+
+
+_PAGES['text'] = Page()
+_PAGES['html'] = HtmlPage(
+    header='''<?xml version="1.0" encoding="utf-8" ?>
+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+<title>Notmuch Patches</title>
+</head>
+<body>
+<h2>Notmuch Patches</h2>
+Generated: {date}<br />
+For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
+<h3>Views</h3>
+'''.format(date=datetime.datetime.utcnow().date()),
+    footer='</body>\n</html>\n',
+    )
 
-# parse command line arguments
 
 parser = argparse.ArgumentParser()
 parser.add_argument('--text', help='output plain text format',
@@ -177,34 +252,9 @@ else:
     import notmuch
 
 if args.text:
-    output_format = 'text'
+    page = _PAGES['text']
 else:
-    output_format = 'html'
-
-# main program
+    page = _PAGES['html']
 
 db = notmuch.Database(mode=notmuch.Database.MODE.READ_ONLY)
-
-if output_format == 'html':
-    print('''<?xml version="1.0" encoding="utf-8" ?>
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
-<head>
-<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
-<title>Notmuch Patches</title>
-</head>
-<body>
-<h2>Notmuch Patches</h2>
-Generated: {date}<br />
-For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
-<h3>Views</h3>
-<ul>'''.format(date=datetime.datetime.utcnow().date()))
-    for view in config['views']:
-        print('<li><a href="#%(title)s">%(title)s</a></li>' % view)
-    print('</ul>')
-
-for view in config['views']:
-    print_view(database=db, **view)
-
-if output_format == 'html':
-    print('</body>\n</html>')
+page.write(database=db, views=config['views'])
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 11/20] nmbug-status: Add an OrderedDict stub for Python 2.6

[W. Trevor King]

Tomi Ollila and David Bremner (and presumably others) are running
Python 2.6 on their nmbug-status boxes, so it makes sense to keep
support for that version.  This commit adds a really minimal
OrderedDict stub (e.g. it doesn't handle key removal), but it gets the
job done for Page._get_threads.  Once we reach a point where Python
2.6 is no longer important (it's already out of it's security-fix
window [1]), we can pull this stub back out.

[1]: http://www.python.org/download/releases/2.6.9/
---
 devel/nmbug/nmbug-status | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 6aa2583..57f16e2 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -5,7 +5,6 @@
 # dependencies
 #       - python 2.6 for json
 #       - argparse; either python 2.7, or install separately
-#       - collections.OrderedDict; python 2.7
 
 from __future__ import print_function
 from __future__ import unicode_literals
@@ -30,6 +29,25 @@ _ENCODING = locale.getpreferredencoding() or sys.getdefaultencoding()
 _PAGES = {}
 
 
+if not hasattr(collections, 'OrderedDict'):  # Python 2.6 or earlier
+    class _OrderedDict (dict):
+        "Just enough of a stub to get through Page._get_threads"
+        def __init__(self, *args, **kwargs):
+            super(_OrderedDict, self).__init__(*args, **kwargs)
+            self._keys = []  # record key order
+
+        def __setitem__(self, key, value):
+            super(_OrderedDict, self).__setitem__(key, value)
+            self._keys.append(key)
+
+        def __values__(self):
+            for key in self._keys:
+                yield self[key]
+
+
+    collections.OrderedDict = _OrderedDict
+
+
 def read_config(path=None, encoding=None):
     "Read config from json file"
     if not encoding:
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 12/20] nmbug-status: Normalize table HTML indentation

[W. Trevor King]

I don't think I've ever seen '</td><td>{value}\n' before :p.  The new
formatting avoids mixing tag levels and content across lines.
---
 devel/nmbug/nmbug-status | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 57f16e2..20e2447 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -197,12 +197,14 @@ class HtmlPage (Page):
         for thread in threads:
             for message_display_data in thread:
                 stream.write((
-                    '<tr><td>{date}\n'
-                    '</td><td>{message-id-term}\n'
-                    '</td></tr>\n'
-                    '<tr><td>{from}\n'
-                    '</td><td>{subject}\n'
-                    '</td></tr>\n'
+                    '<tr>\n'
+                    '  <td>{date}</td>\n'
+                    '  <td>{message-id-term}</td>\n'
+                    '</tr>\n'
+                    '<tr>\n'
+                    '  <td>{from}</td>\n'
+                    '  <td>{subject}</td>\n'
+                    '</tr>\n'
                     ).format(**message_display_data))
             if thread != threads[-1]:
                 stream.write('<tr><td colspan="2"><br /></td></tr>\n')
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 13/20] nmbug-status: Convert from XHTML 1.0 to HTML 5

[W. Trevor King]

HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
stripped it.

This shouldn't break anything at tethera, which already serves the
status as text/html:

  $ wget -S http://nmbug.tethera.net/status/
  --2014-02-02 21:20:39--  http://nmbug.tethera.net/status/
  Resolving nmbug.tethera.net... 87.98.215.224
  Connecting to nmbug.tethera.net|87.98.215.224|:80... connected.
  HTTP request sent, awaiting response...
    HTTP/1.1 200 OK
    Vary: Accept-Encoding
    Content-Type: text/html
  ...

This also matches the Content-Type in the generated HTML's http-equiv
meta.

[1]: http://www.w3.org/TR/html5/dom.html#the-lang-and-xml:lang-attributes
---
 devel/nmbug/nmbug-status | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 20e2447..873a46a 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -227,9 +227,8 @@ class HtmlPage (Page):
 
 _PAGES['text'] = Page()
 _PAGES['html'] = HtmlPage(
-    header='''<?xml version="1.0" encoding="utf-8" ?>
-<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
-<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+    header='''<!DOCTYPE html>
+<html lang="en">
 <head>
 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>Notmuch Patches</title>
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[W. Trevor King]

Instead of always writing UTF-8, allow the user to configure the
output encoding using their locale.  This is useful for previewing
output in the terminal, for poor souls that don't use UTF-8 locales
;).
---
 devel/nmbug/nmbug-status | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 873a46a..040baf0 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -88,7 +88,7 @@ class Page (object):
                 byte_stream = sys.stdout.buffer
             except AttributeError:  # Python 2
                 byte_stream = sys.stdout
-            stream = codecs.getwriter(encoding='UTF-8')(stream=byte_stream)
+            stream = codecs.getwriter(encoding=_ENCODING)(stream=byte_stream)
         self._write_header(views=views, stream=stream)
         for view in views:
             self._write_view(database=database, view=view, stream=stream)
@@ -230,7 +230,7 @@ _PAGES['html'] = HtmlPage(
     header='''<!DOCTYPE html>
 <html lang="en">
 <head>
-<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
+<meta http-equiv="Content-Type" content="text/html; charset={encoding}" />
 <title>Notmuch Patches</title>
 </head>
 <body>
@@ -238,7 +238,8 @@ _PAGES['html'] = HtmlPage(
 Generated: {date}<br />
 For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
 <h3>Views</h3>
-'''.format(date=datetime.datetime.utcnow().date()),
+'''.format(date=datetime.datetime.utcnow().date(),
+           encoding=_ENCODING),
     footer='</body>\n</html>\n',
     )
 
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 15/20] nmbug-status: Anchor with h3 ids instead of a names

[W. Trevor King]

HTML 5 allows id attributes on all HTML elements [1], but restricts
names to particular cases [2].  Attaching the id attribute to the h3
element allows us to drop the anchor a element altogether.

[1]: http://www.w3.org/TR/html5/dom.html#the-id-attribute
[2]: http://www.w3.org/TR/html5/index.html#attributes-1
---
 devel/nmbug/nmbug-status | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 040baf0..2b1f364 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -177,7 +177,7 @@ class HtmlPage (Page):
         stream.write('</ul>\n')
 
     def _write_view_header(self, view, stream):
-        stream.write('<h3><a name="{title}" />{title}</h3>\n'.format(**view))
+        stream.write('<h3 id="{title}">{title}</h3>\n'.format(**view))
         if 'comment' in view:
             stream.write(view['comment'])
             stream.write('\n')
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 16/20] nmbug-status: Slug the title when using it as an id

[W. Trevor King]

Also allow manual id overrides from the JSON config.  Sluggin avoids
errors like:

   Bad value '#Possible bugs' for attribute href on element a:
   Whitespace in fragment component. Use %20 in place of spaces.

from http://validator.w3.org.

I tried just quoting the titles (e.g. 'Possible%20bugs'), but that
didn't work (at least with Firefox 24.2.0).  Slugging avoids any
ambiguity over when the quotes are expanded in the client.  The specs
are unclear about quoting, saying only [1]:

  Value: Any string, with the following restrictions:

    must be at least one character long
    must not contain any space characters

[1]: http://dev.w3.org/html5/markup/global-attributes.html#common.attrs.id
---
 devel/nmbug/nmbug-status | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 2b1f364..989d065 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -21,6 +21,7 @@ except ImportError:  # Python 2
 import json
 import argparse
 import os
+import re
 import sys
 import subprocess
 
@@ -168,16 +169,20 @@ class Page (object):
 
 
 class HtmlPage (Page):
+    _slug_regexp = re.compile('\W+')
+
     def _write_header(self, views, stream):
         super(HtmlPage, self)._write_header(views=views, stream=stream)
         stream.write('<ul>\n')
         for view in views:
+            if 'id' not in view:
+                view['id'] = self._slug(view['title'])
             stream.write(
-                '<li><a href="#{title}">{title}</a></li>\n'.format(**view))
+                '<li><a href="#{id}">{title}</a></li>\n'.format(**view))
         stream.write('</ul>\n')
 
     def _write_view_header(self, view, stream):
-        stream.write('<h3 id="{title}">{title}</h3>\n'.format(**view))
+        stream.write('<h3 id="{id}">{title}</h3>\n'.format(**view))
         if 'comment' in view:
             stream.write(view['comment'])
             stream.write('\n')
@@ -224,6 +229,9 @@ class HtmlPage (Page):
                 ).format(**d)
         return (running_data, display_data)
 
+    def _slug(self, string):
+        return self._slug_regexp.sub('-', string)
+
 
 _PAGES['text'] = Page()
 _PAGES['html'] = HtmlPage(
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 17/20] nmbug-status: Use <code> and <p> markup where appropriate

[W. Trevor King]

* Wrap free text in <p> tags.
* Convert <blockquote> to <p><code> for query strings.
* Wrap message-id-term (id:"...") in <code>.

The <code> tags get nicer default markup (smaller monospace font) for
notmuch query terms [1].  The <p> tags don't have much effect without
attached CSS, but bare text (phrasing content [2]) in <body> (which
expects flow content [3,4]) feels wrong.

[1]: http://www.w3.org/TR/html5/text-level-semantics.html#the-code-element
[2]: http://www.w3.org/TR/html5/dom.html#phrasing-content-1
[3]: http://www.w3.org/TR/html5/dom.html#flow-content-1
[4]: http://www.w3.org/TR/html5/sections.html#the-body-element
---
 devel/nmbug/nmbug-status | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 989d065..c45b763 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -183,14 +183,18 @@ class HtmlPage (Page):
 
     def _write_view_header(self, view, stream):
         stream.write('<h3 id="{id}">{title}</h3>\n'.format(**view))
+        stream.write('<p>\n')
         if 'comment' in view:
             stream.write(view['comment'])
             stream.write('\n')
         for line in [
                 'The view is generated from the following query:',
-                '<blockquote>',
+                '</p>',
+                '<p>',
+                '  <code>',
                 view['query-string'],
-                '</blockquote>',
+                '  </code>',
+                '</p>',
                 ]:
             stream.write(line)
             stream.write('\n')
@@ -204,7 +208,7 @@ class HtmlPage (Page):
                 stream.write((
                     '<tr>\n'
                     '  <td>{date}</td>\n'
-                    '  <td>{message-id-term}</td>\n'
+                    '  <td><code>{message-id-term}</code></td>\n'
                     '</tr>\n'
                     '<tr>\n'
                     '  <td>{from}</td>\n'
@@ -243,8 +247,10 @@ _PAGES['html'] = HtmlPage(
 </head>
 <body>
 <h2>Notmuch Patches</h2>
+<p>
 Generated: {date}<br />
 For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
+</p>
 <h3>Views</h3>
 '''.format(date=datetime.datetime.utcnow().date(),
            encoding=_ENCODING),
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 18/20] nmbug-status: Color threads in HTML output

[W. Trevor King]

Add tbody sections so we don't have to color every row.  Multiple
tbody sections are allowed [1].  Use CSS 3's nth-child to handle
even/odd coloring (skipping the spacer rows) [2], which is supported
on the major browsers [3].

border-spacing is from CCS 2.1 [4,5].  I'm using it to avoid
whitespace between td cells.

border-radius is from CCS 3 [6,7].  I'm using it to make the colored
sections a bit less harsh.  I tried adding rounded borders to the
tbody itself doesn't work, but I couldn't get that to work without
setting the tbody's display to 'block'.  That rounded the corners, but
collapsed the cell spacing (e.g. columns were no longer aligned).
This commit's by-corner-td approach is not particularly elegant, but
it works.  The td padding entries just ensure that the cell body is
suitably far from the edges that it doesn't fall outside of the
rounded corners.

The doubled-braces are escapes from Python's str.format.

[1]: http://www.w3.org/TR/html5/tabular-data.html#the-table-element
[2]: http://www.w3.org/TR/css3-selectors/#nth-child-pseudo
[3]: https://developer.mozilla.org/en-US/docs/Web/CSS/:nth-child#Browser_compatibility
[4]: http://www.w3.org/TR/CSS2/tables.html#propdef-border-spacing
[5]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-spacing#Browser_compatibility
[6]: http://www.w3.org/TR/css3-background/#the-border-radius
[7]: https://developer.mozilla.org/en-US/docs/Web/CSS/border-radius#Browser_compatibility
---
 devel/nmbug/nmbug-status | 55 +++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 43 insertions(+), 12 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index c45b763..1f0873a 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -204,19 +204,22 @@ class HtmlPage (Page):
             return
         stream.write('<table>\n')
         for thread in threads:
+            stream.write('  <tbody>\n')
             for message_display_data in thread:
                 stream.write((
-                    '<tr>\n'
-                    '  <td>{date}</td>\n'
-                    '  <td><code>{message-id-term}</code></td>\n'
-                    '</tr>\n'
-                    '<tr>\n'
-                    '  <td>{from}</td>\n'
-                    '  <td>{subject}</td>\n'
-                    '</tr>\n'
+                    '    <tr>\n'
+                    '      <td>{date}</td>\n'
+                    '      <td><code>{message-id-term}</code></td>\n'
+                    '    </tr>\n'
+                    '    <tr>\n'
+                    '      <td>{from}</td>\n'
+                    '      <td>{subject}</td>\n'
+                    '    </tr>\n'
                     ).format(**message_display_data))
+            stream.write('  </tbody>\n')
             if thread != threads[-1]:
-                stream.write('<tr><td colspan="2"><br /></td></tr>\n')
+                stream.write(
+                    '  <tbody><tr><td colspan="2"><br /></td></tr></tbody>\n')
         stream.write('</table>\n')
 
     def _message_display_data(self, *args, **kwargs):
@@ -242,8 +245,35 @@ _PAGES['html'] = HtmlPage(
     header='''<!DOCTYPE html>
 <html lang="en">
 <head>
-<meta http-equiv="Content-Type" content="text/html; charset={encoding}" />
-<title>Notmuch Patches</title>
+  <meta http-equiv="Content-Type" content="text/html; charset={encoding}" />
+  <title>Notmuch Patches</title>
+  <style media="screen" type="text/css">
+    table {{
+      border-spacing: 0;
+    }}
+    td {{
+      padding-left: {border_radius};
+      padding-right: {border_radius};
+    }}
+    tr:first-child td:first-child {{
+      border-top-left-radius: {border_radius};
+    }}
+    tr:first-child td:last-child {{
+      border-top-right-radius: {border_radius};
+    }}
+    tr:last-child td:first-child {{
+      border-bottom-left-radius: {border_radius};
+    }}
+    tr:last-child td:last-child {{
+      border-bottom-right-radius: {border_radius};
+    }}
+    tbody:nth-child(4n+1) tr td {{
+      background-color: #ffd96e;
+    }}
+    tbody:nth-child(4n+3) tr td {{
+      background-color: #bce;
+    }}
+  </style>
 </head>
 <body>
 <h2>Notmuch Patches</h2>
@@ -253,7 +283,8 @@ For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
 </p>
 <h3>Views</h3>
 '''.format(date=datetime.datetime.utcnow().date(),
-           encoding=_ENCODING),
+           encoding=_ENCODING,
+           border_radius='0.5em'),
     footer='</body>\n</html>\n',
     )
 
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 19/20] nmbug-status: Escape &, <, and > in HTML display data

[W. Trevor King]

'message-id' and 'from' now have sensitive characters escaped using
xml.sax.saxutils.escape [1].  The 'subject' data was already being
converted to a link into Gmane; I've escape()d that too, so it doesn't
need to be handled ain the same block as 'message-id' and 'from'.

This prevents broken HTML by if subjects etc. contain characters that
would otherwise be interpreted as HTML markup.

[1]: http://docs.python.org/3/library/xml.sax.utils.html#xml.sax.saxutils.escape
---
 devel/nmbug/nmbug-status | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 1f0873a..7209dd1 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -24,6 +24,7 @@ import os
 import re
 import sys
 import subprocess
+import xml.sax.saxutils
 
 
 _ENCODING = locale.getpreferredencoding() or sys.getdefaultencoding()
@@ -229,11 +230,14 @@ class HtmlPage (Page):
         if 'subject' in display_data and 'message-id' in display_data:
             d = {
                 'message-id': quote(display_data['message-id']),
-                'subject': display_data['subject'],
+                'subject': xml.sax.saxutils.escape(display_data['subject']),
                 }
             display_data['subject'] = (
                 '<a href="http://mid.gmane.org/{message-id}">{subject}</a>'
                 ).format(**d)
+        for key in ['message-id', 'from']:
+            if key in display_data:
+                display_data[key] = xml.sax.saxutils.escape(display_data[key])
         return (running_data, display_data)
 
     def _slug(self, string):
-- 
1.8.5.2.8.g0f6c0d1

[PATCH v2 20/20] nmbug-status: Add inter-message padding

[W. Trevor King]

We already had the tbody with a blank row separating threads (which is
not colored); this commit adds a bit of spacing to separate messages
within a thread.  It will also add a bit of colored padding above the
first message and below the final message, but the main goal is to add
padding *between* two-row message blocks.

                                               <--- new padding
  thread-1, message-1, row-1  (class="message-first")
  thread-1, message-1, row-2  (class="message-last")
                                               <--- new padding
    spacer tbody with a blank row
                                               <--- new padding
  thread-2, message-1, row-1  (class="message-first")
  thread-2, message-1, row-2  (class="message-last")
                                               <--- new padding
                                               <--- new padding
  thread-2, message-2, row-1  (class="message-first")
  thread-2, message-2, row-2  (class="message-last")
                                               <--- new padding
---
 devel/nmbug/nmbug-status | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/devel/nmbug/nmbug-status b/devel/nmbug/nmbug-status
index 7209dd1..c4532f1 100755
--- a/devel/nmbug/nmbug-status
+++ b/devel/nmbug/nmbug-status
@@ -208,11 +208,11 @@ class HtmlPage (Page):
             stream.write('  <tbody>\n')
             for message_display_data in thread:
                 stream.write((
-                    '    <tr>\n'
+                    '    <tr class="message-first">\n'
                     '      <td>{date}</td>\n'
                     '      <td><code>{message-id-term}</code></td>\n'
                     '    </tr>\n'
-                    '    <tr>\n'
+                    '    <tr class="message-last">\n'
                     '      <td>{from}</td>\n'
                     '      <td>{subject}</td>\n'
                     '    </tr>\n'
@@ -255,6 +255,12 @@ _PAGES['html'] = HtmlPage(
     table {{
       border-spacing: 0;
     }}
+    tr.message-first td {{
+      padding-top: {inter_message_padding};
+    }}
+    tr.message-last td {{
+      padding-bottom: {inter_message_padding};
+    }}
     td {{
       padding-left: {border_radius};
       padding-right: {border_radius};
@@ -288,6 +294,7 @@ For more infomation see <a href="http://notmuchmail.org/nmbug">nmbug</a>
 <h3>Views</h3>
 '''.format(date=datetime.datetime.utcnow().date(),
            encoding=_ENCODING,
+           inter_message_padding='0.25em',
            border_radius='0.5em'),
     footer='</body>\n</html>\n',
     )
-- 
1.8.5.2.8.g0f6c0d1

Re: [PATCH v2 00/20] nmbug-status: Python-3-compatibility and general refactoring

[Tomi Ollila]

On Mon, Feb 10 2014, "W. Trevor King" <wking@tremily.us> wrote:

> This is the second round of my initial series [1].  Thanks to Tomi and
> David for reviewing v1.  General changes since v1:
>
> * Python 2.6 compatibility fixes ('{0}'.format(...) and an OrderedDict
>   stub) [2].

This series looks good to me.


Tomi


>
> Changes to the HTML output since v1:
>
> * Escape &, <, and > in display data [3].
> * Changed color scheme to #ffd96e / #bce [4].
> * Added additional spacing between messages (smaller than the existing
>   spacing between threads) [5].
> * Slug section titles instead of quoting them [6].
>
> [1]: id="cover.1391424512.git.wking@tremily.us"
>      http://thread.gmane.org/gmane.mail.notmuch.general/17061
> [2]: id="20140204173914.GR19935@odin.tremily.us"
>      http://article.gmane.org/gmane.mail.notmuch.general/17114
> [3]: id="20140207221531.GA17142@odin.tremily.us"
>      http://article.gmane.org/gmane.mail.notmuch.general/17141
> [4]: id="20140208190923.GE14197@odin.tremily.us"
>      http://article.gmane.org/gmane.mail.notmuch.general/17151
> [5]: id="20140208221925.GD17142@odin.tremily.us"
>      http://article.gmane.org/gmane.mail.notmuch.general/17153
> [6]: id="20140208231804.GE17142@odin.tremily.us"
>      http://article.gmane.org/gmane.mail.notmuch.general/17155
>
> W. Trevor King (20):
>   nmbug-status: Convert to Python-3-compatible print functions
>   nmbug-status: Use email.utils instead of rfc822
>   nmbug-status: Decode Popen output using the user's locale
>   nmbug-status: Factor config-loading out into read_config
>   nmbug-status: Add metavars for --config and --get-query
>   nmbug-status: Consolidate functions and main code
>   nmbug-status: Don't require write access
>   nmbug-status: Consolidate HTML header printing
>   nmbug-status: Add a Python-3-compatible urllib.parse.quote import
>   nmbug-status: Add Page and HtmlPage for modular rendering
>   nmbug-status: Add an OrderedDict stub for Python 2.6
>   nmbug-status: Normalize table HTML indentation
>   nmbug-status: Convert from XHTML 1.0 to HTML 5
>   nmbug-status: Encode output using the user's locale
>   nmbug-status: Anchor with h3 ids instead of a names
>   nmbug-status: Slug the title when using it as an id
>   nmbug-status: Use <code> and <p> markup where appropriate
>   nmbug-status: Color threads in HTML output
>   nmbug-status: Escape &, <, and > in HTML display data
>   nmbug-status: Add inter-message padding
>
>  devel/nmbug/nmbug-status | 448 +++++++++++++++++++++++++++++++----------------
>  1 file changed, 297 insertions(+), 151 deletions(-)
>
> -- 
> 1.8.5.2.8.g0f6c0d1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:

> Instead of always writing UTF-8, allow the user to configure the
> output encoding using their locale.  This is useful for previewing
> output in the terminal, for poor souls that don't use UTF-8 locales
> ;).

This (or some other patch in the series) seems to cause some problems on
the production instance:

remote: Traceback (most recent call last):
remote:   File "/home/nmbug/tools/nmbug-status", line 336, in <module>
remote:     page.write(database=db, views=config['views'])
remote:   File "/home/nmbug/tools/nmbug-status", line 96, in write
remote:     self._write_view(database=database, view=view, stream=stream)
remote:   File "/home/nmbug/tools/nmbug-status", line 115, in _write_view
remote:     self._write_threads(threads=threads, stream=stream)
remote:   File "/home/nmbug/tools/nmbug-status", line 219, in _write_threads
remote:     ).format(**message_display_data))
remote:   File "/usr/lib/python2.6/codecs.py", line 351, in write
remote:     data, consumed = self.encode(object, self.errors)
remote: UnicodeEncodeError: 'ascii' codec can't encode character u'\u017b' in position 219: ordinal not in range(128)

possibly because of

LANG=C
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

I think it's fine to _allow_ the user to configure the output encoding. I'm
less sure about _requiring_ it.

d

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[Tomi Ollila]

On Tue, Feb 11 2014, David Bremner <david@tethera.net> wrote:

> "W. Trevor King" <wking@tremily.us> writes:
>
>> Instead of always writing UTF-8, allow the user to configure the
>> output encoding using their locale.  This is useful for previewing
>> output in the terminal, for poor souls that don't use UTF-8 locales
>> ;).
>
> This (or some other patch in the series) seems to cause some problems on
> the production instance:
>
> remote: Traceback (most recent call last):
> remote:   File "/home/nmbug/tools/nmbug-status", line 336, in <module>
> remote:     page.write(database=db, views=config['views'])
> remote:   File "/home/nmbug/tools/nmbug-status", line 96, in write
> remote:     self._write_view(database=database, view=view, stream=stream)
> remote:   File "/home/nmbug/tools/nmbug-status", line 115, in _write_view
> remote:     self._write_threads(threads=threads, stream=stream)
> remote:   File "/home/nmbug/tools/nmbug-status", line 219, in _write_threads
> remote:     ).format(**message_display_data))
> remote:   File "/usr/lib/python2.6/codecs.py", line 351, in write
> remote:     data, consumed = self.encode(object, self.errors)
> remote: UnicodeEncodeError: 'ascii' codec can't encode character u'\u017b' in position 219: ordinal not in range(128)
>
> possibly because of
>
> LANG=C
> LANGUAGE=
> LC_CTYPE="C"
> LC_NUMERIC="C"
> LC_TIME="C"
> LC_COLLATE="C"
> LC_MONETARY="C"
> LC_MESSAGES="C"
> LC_PAPER="C"
> LC_NAME="C"
> LC_ADDRESS="C"
> LC_TELEPHONE="C"
> LC_MEASUREMENT="C"
> LC_IDENTIFICATION="C"
> LC_ALL=C
>
> I think it's fine to _allow_ the user to configure the output encoding. I'm
> less sure about _requiring_ it.

That reminded me that yesterday (after review, of course) I thought that 
we probably want configuration file to be parsed as utf-8 instead of
any encoding user may have in their system...

> d

Tomi

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 2103 bytes --]

On Tue, Feb 11, 2014 at 04:14:45PM +0200, Tomi Ollila wrote:
> On Tue, Feb 11 2014, David Bremner wrote:
> > W. Trevor King writes:
> >> Instead of always writing UTF-8, allow the user to configure the
> >> output encoding using their locale.  This is useful for
> >> previewing output in the terminal, for poor souls that don't use
> >> UTF-8 locales ;).
> >
> > …
> > remote: UnicodeEncodeError: 'ascii' codec can't encode character
> >   u'\u017b' in position 219: ordinal not in range(128)
> >
> > possibly because of
> >
> > LANG=C
> > …
> >
> > I think it's fine to _allow_ the user to configure the output
> > encoding. I'm less sure about _requiring_ it.

If a user has set LANG=C, I expect that's what we should use for
output (in which case dying with an encoding error is the right thing
to do).  If you want UTF-8 output, using a UTF-8 locale seems like a
reasonable requirement.  For the HTML case, we could fall back on
numerical character references (e.g. Ż) if the requested locale
didn't support the required character directly, but I don't see an
easy solution for the text-mode output.

> That reminded me that yesterday (after review, of course) I thought
> that we probably want configuration file to be parsed as utf-8
> instead of any encoding user may have in their system...

The POSIX spec for LANG doesn't restrict the scoping to the terminal
intput / output [1], so I feel like we should also be using LANG to
read the config file as well.  I expect folks with UTF-8 LANGs will
want UTF-8 file contents.  In both cases (terminal output and
config-file input), it is easy for users to pick their preferred
encoding:

  $ LANG=en_US.UTF-8 nmbug-status …

I think we should trust what they've chosen, rather than guessing that
they actually want UTF-8 ;).

Cheers,
Trevor

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html#tag_08_02

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:

>
> If a user has set LANG=C, I expect that's what we should use for
> output (in which case dying with an encoding error is the right thing
> to do).  
>

Perhaps for an interactive tool, intended mainly to be run in a
terminal. But nmbug-status is not such a tool (at least, that was not
why I wrote it). It's run non-interactively in a git hook to generate a
web page.

I'm not sure how important the interactive use case is, but I don't want
to make nmbug-status less robust in order to conform to some abstract
ideal.

One approach would be to enable certain "interactive" features via a
command line argument.

d

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1361 bytes --]

On Tue, Feb 11, 2014 at 06:02:09PM -0400, David Bremner wrote:
> W. Trevor King writes:
> > If a user has set LANG=C, I expect that's what we should use for
> > output (in which case dying with an encoding error is the right
> > thing to do).
> 
> Perhaps for an interactive tool, intended mainly to be run in a
> terminal.

I don't understand why your choice of LANG should depend on the
interactive-ness of an invocation.

> I'm not sure how important the interactive use case is, but I don't
> want to make nmbug-status less robust in order to conform to some
> abstract ideal.

Changing from a hard-coded version that matches your preferences to a
configurable version that respects LANG may require a bit more care on
the user side (they have to ensure LANG is set in their crontab,
etc.).  However, setting your locale to match your preferences is not
that much work.  For example:

  export LANG=en_US.UTF-8

and you're done.  The upside of a configurable language is that the
user gets output in their preferred encoding (UTF-8 or not) and—with a
bit of additional gettext work—in their preferred language.  That
sounds like a fair trade to me.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2 13/20] nmbug-status: Convert from XHTML 1.0 to HTML 5

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:

> HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
> spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
> stripped it.

I pushed the first 13 patches in the series to master.

d

Re: [PATCH v2 13/20] nmbug-status: Convert from XHTML 1.0 to HTML 5

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

On Wed, Feb 12, 2014 at 07:35:19PM -0400, David Bremner wrote:
> W. Trevor King writes:
> > HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
> > spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
> > stripped it.
> 
> I pushed the first 13 patches in the series to master.

:).  If the output encoding patch (nmbug-status: Encode output using
the user's locale) is the sticking point for the remainder of the
series, I'm happy to rebase it to the end of the series and let it
cook on the list until someone needs it.  UTF-8 works for me
personally.

Cheers,
Trevor

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:
>
> I don't understand why your choice of LANG should depend on the
> interactive-ness of an invocation.

It's not the choice of LANG, but rather the acceptability of crashing
with an unhandled exception.

>  The upside of a configurable language is that the user gets output in
> their preferred encoding (UTF-8 or not) and—with a bit of additional
> gettext work—in their preferred language.  That sounds like a fair
> trade to me.

The downside is that it introduces another error condition into the
script. The data from notmuch is unicode; conversion to most non-utf8
locales is lossy and can cause crashes. That's why I don't like the
idea of enabling it without some explicit option or configuration
choice.

d

P.S. I just saw your other reply, and yeah, a rebased version of the
other 6 patches would be welcome.

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[W. Trevor King]

[-- Attachment #1: Type: text/plain, Size: 1585 bytes --]

On Wed, Feb 12, 2014 at 10:13:50PM -0400, David Bremner wrote:
> W. Trevor King writes:
> > I don't understand why your choice of LANG should depend on the
> > interactive-ness of an invocation.
> 
> It's not the choice of LANG, but rather the acceptability of
> crashing with an unhandled exception.

I'd be fine with catching the exception and dying with a more compact
error message.  The script would still be dying though.

> > The upside of a configurable language is that the user gets output
> > in their preferred encoding (UTF-8 or not) and—with a bit of
> > additional gettext work—in their preferred language.  That sounds
> > like a fair trade to me.
> 
> The downside is that it introduces another error condition into the
> script. The data from notmuch is unicode; conversion to most non-utf8
> locales is lossy and can cause crashes.

Folks on Microsoft OSes might prefer UTF-16 [1], which is a valid
preference.

> That's why I don't like the idea of enabling it without some
> explicit option or configuration choice.

I think LANG is an explicit configuration choice ;).  I'm fine punting
on this though, since UTF-8 works for me.  Should I rebase this to the
end of the remaining series and resubmit?  Then this patch can cook on
the list until we find a user that doesn't like UTF-8 ;).

Cheers,
Trevor

[1]: http://en.wikipedia.org/wiki/Unicode_in_Microsoft_Windows

-- 
This email may be signed or encrypted with GnuPG (http://www.gnupg.org).
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH v2 13/20] nmbug-status: Convert from XHTML 1.0 to HTML 5

[Tomi Ollila]

On Thu, Feb 13 2014, "W. Trevor King" <wking@tremily.us> wrote:

> On Wed, Feb 12, 2014 at 07:35:19PM -0400, David Bremner wrote:
>> W. Trevor King writes:
>> > HTML 5 for the win :).  I also de-namespaced the language; the HTML 5
>> > spec allows a vestigial xml:lang attribute, but it's a no-op [1], so I
>> > stripped it.
>> 
>> I pushed the first 13 patches in the series to master.
>
> :).  If the output encoding patch (nmbug-status: Encode output using
> the user's locale) is the sticking point for the remainder of the
> series, I'm happy to rebase it to the end of the series and let it
> cook on the list until someone needs it.  UTF-8 works for me
> personally.

No-one will need äny alternatives... ;D

Personally hardcoding UTF-8 works for me.

Next option hardcoding UTF-8 for config read (I would not want LANG=C
borke my 'Fööbär' in title ;) and use os.isatty(1) to determine
whether hardcod UTF-8 for output could be possibly tolerable to me.

> Cheers,
> Trevor

Tomi

Re: [PATCH v2 14/20] nmbug-status: Encode output using the user's locale

[David Bremner]

"W. Trevor King" <wking@tremily.us> writes:

>
> Folks on Microsoft OSes might prefer UTF-16 [1], which is a valid
> preference.
>

I see your wikipedia reference and raise you

  http://www.utf8everywhere.org/

;).

>  Should I rebase this to the end of the remaining series and resubmit?
> Then this patch can cook on the list until we find a user that doesn't
> like UTF-8 ;).
>

Sure.