* [PATCH] Output unmodified Content-Type header value for JSON format.
@ 2011-11-18 23:45 Dmitry Kurochkin
2011-11-19 1:58 ` Jameson Graef Rollins
2011-11-19 4:18 ` [PATCH v2] " Dmitry Kurochkin
0 siblings, 2 replies; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-18 23:45 UTC (permalink / raw)
To: notmuch
Before the change, notmuch used g_mime_content_type_to_string(3)
function to output Content-Type header value. Turns out it outputs
only "type/subtype" part and ignores all parameters. Also, if there
is no Content-Type header, default "text/plain" value is used.
JSON is supposed to be a "low-level" structured format and should not
add missing values or throw away information. The patch changes
notmuch show to use unmodified Content-Type value for JSON format.
Also, no default value is added if the header is missing.
Corresponding changes to Emacs UI are made to handle full Content-Type
header values. The header is parsed using MIME
`mail-header-parse-content-type' function. All message part rendering
functions have access to full Content-Type value. In particular, this
is important for `notmuch-show-mm-display-part-inline' which uses
`mm-display-part' to display parts that notmuch-show does not handle.
Expected results for the tests are updated accordingly.
---
emacs/notmuch-show.el | 28 ++++++++++++++++++----------
notmuch-show.c | 14 ++++++++++++--
test/crypto | 23 ++++++++---------------
test/json | 6 +++---
test/maildir-sync | 1 -
test/multipart | 36 ++++++++++++++++++------------------
6 files changed, 59 insertions(+), 49 deletions(-)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index d5c95d8..d2c2fa3 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -261,120 +261,120 @@ message at DEPTH in the current thread."
(if (and header-value
(not (string-equal "" header-value)))
(notmuch-show-insert-header header header-value))))
notmuch-message-headers)
(save-excursion
(save-restriction
(narrow-to-region start (point-max))
(run-hooks 'notmuch-show-markup-headers-hook)))))
(define-button-type 'notmuch-show-part-button-type
'action 'notmuch-show-part-button-action
'follow-link t
'face 'message-mml)
(defun notmuch-show-insert-part-header (nth content-type declared-type &optional name comment)
(let ((button))
(setq button
(insert-button
(concat "[ "
(if name (concat name ": ") "")
- declared-type
- (if (not (string-equal declared-type content-type))
- (concat " (as " content-type ")")
+ (car declared-type)
+ (if (not (string-equal (car declared-type) (car content-type)))
+ (concat " (as " (car content-type) ")")
"")
(or comment "")
" ]")
:type 'notmuch-show-part-button-type
:notmuch-part nth
:notmuch-filename name))
(insert "\n")
;; return button
button))
;; Functions handling particular MIME parts.
(defun notmuch-show-save-part (message-id nth &optional filename)
(let ((process-crypto notmuch-show-process-crypto))
(with-temp-buffer
(setq notmuch-show-process-crypto process-crypto)
;; Always acquires the part via `notmuch part', even if it is
;; available in the JSON output.
(insert (notmuch-show-get-bodypart-internal message-id nth))
(let ((file (read-file-name
"Filename to save as: "
(or mailcap-download-directory "~/")
nil nil
filename)))
;; Don't re-compress .gz & al. Arguably we should make
;; `file-name-handler-alist' nil, but that would chop
;; ange-ftp, which is reasonable to use here.
(mm-write-region (point-min) (point-max) file nil nil nil 'no-conversion t)))))
(defun notmuch-show-mm-display-part-inline (msg part content-type content)
"Use the mm-decode/mm-view functions to display a part in the
current buffer, if possible."
(let ((display-buffer (current-buffer)))
(with-temp-buffer
(insert content)
- (let ((handle (mm-make-handle (current-buffer) (list content-type))))
+ (let ((handle (mm-make-handle (current-buffer) content-type)))
(set-buffer display-buffer)
(if (and (mm-inlinable-p handle)
(mm-inlined-p handle))
(progn
(mm-display-part handle)
t)
nil)))))
(defvar notmuch-show-multipart/alternative-discouraged
'(
;; Avoid HTML parts.
"text/html"
;; multipart/related usually contain a text/html part and some associated graphics.
"multipart/related"
))
(defun notmuch-show-multipart/*-to-list (part)
- (mapcar '(lambda (inner-part) (plist-get inner-part :content-type))
+ (mapcar '(lambda (inner-part) (car (notmuch-show-get-content-type inner-part)))
(plist-get part :content)))
(defun notmuch-show-multipart/alternative-choose (types)
;; Based on `mm-preferred-alternative-precedence'.
(let ((seq types))
(dolist (pref (reverse notmuch-show-multipart/alternative-discouraged))
(dolist (elem (copy-sequence seq))
(when (string-match pref elem)
(setq seq (nconc (delete elem seq) (list elem))))))
seq))
(defun notmuch-show-insert-part-multipart/alternative (msg part content-type nth depth declared-type)
(notmuch-show-insert-part-header nth declared-type content-type nil)
(let ((chosen-type (car (notmuch-show-multipart/alternative-choose (notmuch-show-multipart/*-to-list part))))
(inner-parts (plist-get part :content))
(start (point)))
;; This inserts all parts of the chosen type rather than just one,
;; but it's not clear that this is the wrong thing to do - which
;; should be chosen if there are more than one that match?
(mapc (lambda (inner-part)
- (let ((inner-type (plist-get inner-part :content-type)))
+ (let ((inner-type (notmuch-show-get-content-type inner-part)))
(if (or notmuch-show-all-multipart/alternative-parts
- (string= chosen-type inner-type))
+ (string= chosen-type (car inner-type)))
(notmuch-show-insert-bodypart msg inner-part depth)
(notmuch-show-insert-part-header (plist-get inner-part :id) inner-type inner-type nil " (not shown)"))))
inner-parts)
(when notmuch-show-indent-multipart
(indent-rigidly start (point) 1)))
t)
(defun notmuch-show-setup-w3m ()
"Instruct w3m how to retrieve content from a \"related\" part of a message."
(interactive)
(if (boundp 'w3m-cid-retrieve-function-alist)
(unless (assq 'notmuch-show-mode w3m-cid-retrieve-function-alist)
(push (cons 'notmuch-show-mode 'notmuch-show-w3m-cid-retrieve)
w3m-cid-retrieve-function-alist)))
(setq mm-inline-text-html-with-images t))
(defvar w3m-current-buffer) ;; From `w3m.el'.
(defvar notmuch-show-w3m-cid-store nil)
(make-variable-buffer-local 'notmuch-show-w3m-cid-store)
@@ -557,41 +557,42 @@ current buffer, if possible."
(set-buffer (get-file-buffer file))
(setq result (buffer-substring (point-min) (point-max)))
(set-buffer-modified-p nil)
(kill-buffer (current-buffer))
(delete-file file)
result)))
t)
(defun notmuch-show-insert-part-application/octet-stream (msg part content-type nth depth declared-type)
;; If we can deduce a MIME type from the filename of the attachment,
;; do so and pass it on to the handler for that type.
(if (plist-get part :filename)
(let ((extension (file-name-extension (plist-get part :filename)))
mime-type)
(if extension
(progn
(mailcap-parse-mimetypes)
(setq mime-type (mailcap-extension-to-mime extension))
(if (and mime-type
(not (string-equal mime-type "application/octet-stream")))
- (notmuch-show-insert-bodypart-internal msg part mime-type nth depth content-type)
+ (notmuch-show-insert-bodypart-internal msg part (list mime-type)
+ nth depth content-type)
nil))
nil))))
(defun notmuch-show-insert-part-application/* (msg part content-type nth depth declared-type
)
;; do not render random "application" parts
(notmuch-show-insert-part-header nth content-type declared-type (plist-get part :filename)))
(defun notmuch-show-insert-part-*/* (msg part content-type nth depth declared-type)
;; This handler _must_ succeed - it is the handler of last resort.
(notmuch-show-insert-part-header nth content-type declared-type (plist-get part :filename))
(let ((content (notmuch-show-get-bodypart-content msg part nth)))
(if content
(notmuch-show-mm-display-part-inline msg part content-type content)))
t)
;; Functions for determining how to handle MIME parts.
(defun notmuch-show-split-content-type (content-type)
(split-string content-type "/"))
@@ -618,51 +619,51 @@ current buffer, if possible."
(defun notmuch-show-get-bodypart-internal (message-id part-number)
(let ((args '("show" "--format=raw"))
(part-arg (format "--part=%s" part-number)))
(setq args (append args (list part-arg)))
(if notmuch-show-process-crypto
(setq args (append args '("--decrypt"))))
(setq args (append args (list message-id)))
(with-temp-buffer
(let ((coding-system-for-read 'no-conversion))
(progn
(apply 'call-process (append (list notmuch-command nil (list t nil) nil) args))
(buffer-string))))))
(defun notmuch-show-get-bodypart-content (msg part nth)
(or (plist-get part :content)
(notmuch-show-get-bodypart-internal (concat "id:" (plist-get msg :id)) nth)))
;; \f
(defun notmuch-show-insert-bodypart-internal (msg part content-type nth depth declared-type)
- (let ((handlers (notmuch-show-handlers-for content-type)))
+ (let ((handlers (notmuch-show-handlers-for (car content-type))))
;; Run the content handlers until one of them returns a non-nil
;; value.
(while (and handlers
(not (funcall (car handlers) msg part content-type nth depth declared-type)))
(setq handlers (cdr handlers))))
t)
(defun notmuch-show-insert-bodypart (msg part depth)
"Insert the body part PART at depth DEPTH in the current thread."
- (let ((content-type (downcase (plist-get part :content-type)))
+ (let ((content-type (notmuch-show-get-content-type part))
(nth (plist-get part :id)))
(notmuch-show-insert-bodypart-internal msg part content-type nth depth content-type))
;; Some of the body part handlers leave point somewhere up in the
;; part, so we make sure that we're down at the end.
(goto-char (point-max))
;; Ensure that the part ends with a carriage return.
(if (not (bolp))
(insert "\n")))
(defun notmuch-show-insert-body (msg body depth)
"Insert the body BODY at depth DEPTH in the current thread."
(mapc '(lambda (part) (notmuch-show-insert-bodypart msg part depth)) body))
(defun notmuch-show-make-symbol (type)
(make-symbol (concat "notmuch-show-" type)))
(defun notmuch-show-strip-re (string)
(replace-regexp-in-string "\\([Rr]e: *\\)+" "" string))
(defvar notmuch-show-previous-subject "")
@@ -1054,40 +1055,47 @@ All currently available key bindings:
(save-excursion
(notmuch-show-move-to-message-top)
(get-text-property (point) :notmuch-message-properties)))
(defun notmuch-show-set-prop (prop val &optional props)
(let ((inhibit-read-only t)
(props (or props
(notmuch-show-get-message-properties))))
(plist-put props prop val)
(notmuch-show-set-message-properties props)))
(defun notmuch-show-get-prop (prop &optional props)
(let ((props (or props
(notmuch-show-get-message-properties))))
(plist-get props prop)))
(defun notmuch-show-get-message-id ()
"Return the message id of the current message."
(concat "id:\"" (notmuch-show-get-prop :id) "\""))
+(defun notmuch-show-get-content-type (&optional props)
+ "Return parsed Content-Type of the given message, or part, or
+current message if no `props` is given. If there is no
+Content-Type header, it defaults to \"text/plain\"."
+ (mail-header-parse-content-type (or (notmuch-show-get-prop :content-type props)
+ "text/plain")))
+
;; dme: Would it make sense to use a macro for many of these?
(defun notmuch-show-get-filename ()
"Return the filename of the current message."
(notmuch-show-get-prop :filename))
(defun notmuch-show-get-header (header)
"Return the named header of the current message, if any."
(plist-get (notmuch-show-get-prop :headers) header))
(defun notmuch-show-get-cc ()
(notmuch-show-get-header :Cc))
(defun notmuch-show-get-date ()
(notmuch-show-get-header :Date))
(defun notmuch-show-get-from ()
(notmuch-show-get-header :From))
(defun notmuch-show-get-subject ()
diff --git a/notmuch-show.c b/notmuch-show.c
index 603992a..da3e87f 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -3,40 +3,42 @@
* Copyright © 2009 Carl Worth
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see http://www.gnu.org/licenses/ .
*
* Author: Carl Worth <cworth@cworth.org>
*/
#include "notmuch-client.h"
+static const char HEADER_CONTENT_TYPE[] = "Content-Type";
+
static void
format_message_text (unused (const void *ctx),
notmuch_message_t *message,
int indent);
static void
format_headers_text (const void *ctx,
notmuch_message_t *message);
static void
format_headers_message_part_text (GMimeMessage *message);
static void
format_part_start_text (GMimeObject *part,
int *part_count);
static void
format_part_content_text (GMimeObject *part);
static void
format_part_end_text (GMimeObject *part);
@@ -640,42 +642,50 @@ format_part_sigstatus_json (const GMimeSignatureValidity* validity)
}
printf ("}");
signer = signer->next;
}
printf ("]");
talloc_free (ctx_quote);
}
static void
format_part_content_json (GMimeObject *part)
{
GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
GMimeStream *stream_memory = g_mime_stream_mem_new ();
const char *cid = g_mime_object_get_content_id (part);
void *ctx = talloc_new (NULL);
GByteArray *part_content;
- printf (", \"content-type\": %s",
- json_quote_str (ctx, g_mime_content_type_to_string (content_type)));
+ {
+ /* Output full Content-Type header value,
+ * g_mime_content_type_to_string(3) does not include
+ * parameters. Content-Type header may be missing,
+ * g_mime_object_get_content_type(3) defaults to "text/plain"
+ * in this case. */
+ const char *const h = g_mime_object_get_header (part, HEADER_CONTENT_TYPE);
+ if (h)
+ printf (", \"content-type\": %s", json_quote_str (ctx, h));
+ }
if (cid != NULL)
printf(", \"content-id\": %s", json_quote_str (ctx, cid));
if (GMIME_IS_PART (part))
{
const char *filename = g_mime_part_get_filename (GMIME_PART (part));
if (filename)
printf (", \"filename\": %s", json_quote_str (ctx, filename));
}
if (g_mime_content_type_is_type (content_type, "text", "*") &&
!g_mime_content_type_is_type (content_type, "text", "html"))
{
show_text_part_content (part, stream_memory);
part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM (stream_memory));
printf (", \"content\": %s", json_quote_chararray (ctx, (char *) part_content->data, part_content->len));
}
else if (g_mime_content_type_is_type (content_type, "multipart", "*"))
diff --git a/test/crypto b/test/crypto
index 0af4aa8..b923d22 100755
--- a/test/crypto
+++ b/test/crypto
@@ -40,111 +40,108 @@ test_expect_success 'emacs delivery of signed message' \
test_begin_subtest "signature verification"
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with full owner trust"
# give the key full owner trust
echo "${FINGERPRINT}:6:" | gpg --no-tty --import-ownertrust >>"$GNUPGHOME"/trust.log 2>&1
gpg --no-tty --check-trustdb >>"$GNUPGHOME"/trust.log 2>&1
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000,
"userid": " Notmuch Test Suite <test_suite@notmuchmail.org> (INSECURE!)"}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with signer key unavailable"
# move the gnupghome temporarily out of the way
mv "${GNUPGHOME}"{,.bak}
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "error",
"keyid": "'$(echo $FINGERPRINT | cut -c 25-)'",
"errors": 2}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
mv "${GNUPGHOME}"{.bak,}
# create a test encrypted message with attachment
cat <<EOF >TESTATTACHMENT
This is a test file.
EOF
test_expect_success 'emacs delivery of encrypted message with attachment' \
'emacs_deliver_message \
"test encrypted message 001" \
"This is a test encrypted message.\n" \
"(mml-attach-file \"TESTATTACHMENT\") (mml-secure-message-encrypt)"'
test_begin_subtest "decryption, --format=text"
@@ -181,138 +178,135 @@ test_expect_equal \
test_begin_subtest "decryption, --format=json"
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "good"}],
"sigstatus": [],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"==-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
- "content-type": "multipart/mixed",
+ "content-type": "multipart/mixed; boundary=\"=-=-=\"",
"content": [{"id": 4,
- "content-type": "text/plain",
"content": "This is a test encrypted message.\n"},
{"id": 5,
"content-type": "application/octet-stream",
"filename": "TESTATTACHMENT"}]}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "decryption, --format=json, --part=4"
output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='{"id": 4,
- "content-type": "text/plain",
"content": "This is a test encrypted message.\n"}'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "decrypt attachment (--part=5 --format=raw)"
notmuch show \
--format=raw \
--part=5 \
--decrypt \
subject:"test encrypted message 001" >OUTPUT
test_expect_equal_file OUTPUT TESTATTACHMENT
test_begin_subtest "decryption failure with missing key"
mv "${GNUPGHOME}"{,.bak}
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "bad"}],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"==-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
"content-type": "application/octet-stream"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
mv "${GNUPGHOME}"{.bak,}
test_expect_success 'emacs delivery of encrypted + signed message' \
'emacs_deliver_message \
"test encrypted message 002" \
"This is another test encrypted message.\n" \
"(mml-secure-message-sign-encrypt)"'
test_begin_subtest "decryption + signature verification"
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 002" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 002",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "good"}],
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000,
"userid": " Notmuch Test Suite <test_suite@notmuchmail.org> (INSECURE!)"}],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"=-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
- "content-type": "text/plain",
"content": "This is another test encrypted message.\n"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "reply to encrypted message"
output=$(notmuch reply --decrypt subject:"test encrypted message 002" \
| grep -v -e '^In-Reply-To:' -e '^References:')
expected='From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: test encrypted message 002
On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
> This is another test encrypted message.'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with revoked key"
# generate revocation certificate and load it to revoke key
@@ -327,32 +321,31 @@ y
| gpg --no-tty --quiet --import
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "error",
"keyid": "6D92612D94E46381",
"errors": 8}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_done
diff --git a/test/json b/test/json
index 592b068..64f35cf 100755
--- a/test/json
+++ b/test/json
@@ -1,50 +1,50 @@
#!/usr/bin/env bash
test_description="--format=json output"
. ./test-lib.sh
test_begin_subtest "Show message: json"
add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-show-message\""
output=$(notmuch show --format=json "json-show-message")
-test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content\": \"json-show-message\n\"}]}, []]]]"
test_begin_subtest "Search message: json"
add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-search-message\""
output=$(notmuch search --format=json "json-search-message" | notmuch_search_sanitize)
test_expect_equal "$output" "[{\"thread\": \"XXX\",
\"timestamp\": 946728000,
\"matched\": 1,
\"total\": 1,
\"authors\": \"Notmuch Test Suite\",
\"subject\": \"json-search-subject\",
\"tags\": [\"inbox\", \"unread\"]}]"
test_begin_subtest "Show message: json, utf-8"
add_message "[subject]=\"json-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-show-méssage\""
output=$(notmuch show --format=json "jsön-show-méssage")
-test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"jsön-show-méssage\n\"}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content\": \"jsön-show-méssage\n\"}]}, []]]]"
test_begin_subtest "Show message: json, inline attachment filename"
subject='json-show-inline-attachment-filename'
id="json-show-inline-attachment-filename@notmuchmail.org"
emacs_deliver_message \
"$subject" \
'This is a test message with inline attachment with a filename' \
"(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\")
(message-goto-eoh)
(insert \"Message-ID: <$id>\n\")"
output=$(notmuch show --format=json "id:$id")
filename=$(notmuch search --output=files "id:$id")
-test_expect_equal "$output" "[[[{\"id\": \"$id\", \"match\": true, \"filename\": \"$filename\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"test_suite@notmuchmail.org\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed\", \"content\": [{\"id\": 2, \"content-type\": \"text/plain\", \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"filename\": \"README\"}]}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"$id\", \"match\": true, \"filename\": \"$filename\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"test_suite@notmuchmail.org\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed; boundary=\\\"=-=-=\\\"\", \"content\": [{\"id\": 2, \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"filename\": \"README\"}]}]}, []]]]"
test_begin_subtest "Search message: json, utf-8"
add_message "[subject]=\"json-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-search-méssage\""
output=$(notmuch search --format=json "jsön-search-méssage" | notmuch_search_sanitize)
test_expect_equal "$output" "[{\"thread\": \"XXX\",
\"timestamp\": 946728000,
\"matched\": 1,
\"total\": 1,
\"authors\": \"Notmuch Test Suite\",
\"subject\": \"json-search-utf8-body-sübjéct\",
\"tags\": [\"inbox\", \"unread\"]}]"
test_done
diff --git a/test/maildir-sync b/test/maildir-sync
index a60854f..c7ca22f 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -41,41 +41,40 @@ add_message [subject]='"Adding replied tag"' [filename]='adding-replied-tag:2,S'
notmuch tag +replied subject:"Adding replied tag"
output=$(cd ${MAIL_DIR}/cur; ls -1 adding-replied*)
test_expect_equal "$output" "adding-replied-tag:2,RS"
test_begin_subtest "notmuch show works with renamed file (without notmuch new)"
output=$(notmuch show --format=json id:${gen_msg_id} | filter_show_json)
test_expect_equal "$output" '[[[{"id": "adding-replied-tag@notmuch-test-suite",
"match": true,
"filename": "MAIL_DIR/cur/adding-replied-tag:2,RS",
"timestamp": 978709437,
"date_relative": "2001-01-05",
"tags": ["inbox","replied"],
"headers": {"Subject": "Adding replied tag",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"Cc": "",
"Bcc": "",
"Date": "Tue,
05 Jan 2001 15:43:57 -0000"},
"body": [{"id": 1,
-"content-type": "text/plain",
"content": "This is just a test message (#3)\n"}]},
[]]]]'
test_expect_success 'notmuch reply works with renamed file (without notmuch new)' 'notmuch reply id:${gen_msg_id}'
test_begin_subtest "notmuch new detects no file rename after tag->flag synchronization"
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail."
test_begin_subtest "When read, message moved from new to cur"
add_message [subject]='"Message to move to cur"' [date]='"Sat, 01 Jan 2000 12:00:00 -0000"' [filename]='message-to-move-to-cur' [dir]=new
notmuch tag -unread subject:"Message to move to cur"
output=$(cd "$MAIL_DIR/cur"; ls message-to-move*)
test_expect_equal "$output" "message-to-move-to-cur:2,S"
test_begin_subtest "No rename should be detected by notmuch new"
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail."
# (*) If notmuch new was not run we've got "Processed 1 file in almost
# no time" here. The reason is that removing unread tag in a previous
diff --git a/test/multipart b/test/multipart
index f83526b..ca4db71 100755
--- a/test/multipart
+++ b/test/multipart
@@ -306,140 +306,140 @@ test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=text --part=9, pgp signature (unverified)"
notmuch show --format=text --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
cat <<EOF >EXPECTED
\fpart{ ID: 9, Content-type: application/pgp-signature
Non-text part: application/pgp-signature
\fpart}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_expect_success \
"--format=text --part=8, no part, expect error" \
"notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
test_begin_subtest "--format=json --part=0, full message"
notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "${MAIL_DIR}/multipart", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [
-{"id": 1, "content-type": "multipart/signed", "content": [
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 1, "content-type": "multipart/signed; boundary=\"==-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]},
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]},
{"id": 9, "content-type": "application/pgp-signature"}]}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=1, message body"
notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 1, "content-type": "multipart/signed", "content": [
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 1, "content-type": "multipart/signed; boundary=\"==-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]},
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]},
{"id": 9, "content-type": "application/pgp-signature"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=2, multipart/mixed"
notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=3, rfc822 part"
notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=4, rfc822's multipart/alternative"
notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=5, rfc822's html part"
notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 5, "content-type": "text/html"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=6, rfc822's text part"
notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=7, inline attachment"
notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=8, plain text part"
notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=9, pgp signature (unverified)"
notmuch show --format=json --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 9, "content-type": "application/pgp-signature"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_expect_success \
"--format=json --part=10, no part, expect error" \
"notmuch show --format=json --part=10 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
test_begin_subtest "--format=raw"
notmuch show --format=raw 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
test_expect_equal_file OUTPUT "${MAIL_DIR}"/multipart
--
1.7.7.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-18 23:45 [PATCH] Output unmodified Content-Type header value for JSON format Dmitry Kurochkin
@ 2011-11-19 1:58 ` Jameson Graef Rollins
2011-11-19 2:42 ` Dmitry Kurochkin
2011-11-19 4:18 ` [PATCH v2] " Dmitry Kurochkin
1 sibling, 1 reply; 24+ messages in thread
From: Jameson Graef Rollins @ 2011-11-19 1:58 UTC (permalink / raw)
To: Dmitry Kurochkin, notmuch
[-- Attachment #1: Type: text/plain, Size: 1286 bytes --]
On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> Before the change, notmuch used g_mime_content_type_to_string(3)
> function to output Content-Type header value. Turns out it outputs
> only "type/subtype" part and ignores all parameters. Also, if there
> is no Content-Type header, default "text/plain" value is used.
Hi, Dmitry. Can you explain under what circumstances you would need the
extra content-type parameters? It just seems like a lot of extra noise
in the output to me, but that's partially because I can't think of any
reason why something that is receiving pre-parsed mime content would
need it. Maybe there's a better way to handle what you're trying to get
to.
I think it would help a lot if you could submit some sort of test
modification that demonstrates the issue. This is one of the reasons we
keep emphasizing that it's good to first have tests in hand that
demonstrate issues before patches that address them.
> "content": [{"id": 2,
> - "content-type": "text/plain",
> "content": "This is a test signed message.\n"},
Without figuring out what's going on, I notice that some of the tests
have been modified to remove the content-type fields on a bunch of
parts. I think that is probably not right.
jamie.
[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 1:58 ` Jameson Graef Rollins
@ 2011-11-19 2:42 ` Dmitry Kurochkin
2011-11-19 4:59 ` Austin Clements
2011-11-19 10:49 ` Jameson Graef Rollins
0 siblings, 2 replies; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-19 2:42 UTC (permalink / raw)
To: Jameson Graef Rollins, notmuch
Hi Jamie.
On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > Before the change, notmuch used g_mime_content_type_to_string(3)
> > function to output Content-Type header value. Turns out it outputs
> > only "type/subtype" part and ignores all parameters. Also, if there
> > is no Content-Type header, default "text/plain" value is used.
>
> Hi, Dmitry. Can you explain under what circumstances you would need the
> extra content-type parameters?
Charset is an example of a parameter which you need to render a part
correctly.
> It just seems like a lot of extra noise
> in the output to me, but that's partially because I can't think of any
> reason why something that is receiving pre-parsed mime content would
> need it. Maybe there's a better way to handle what you're trying to get
> to.
>
Why extra output in JSON is an issue?
The parameters are there for a reason. They are part of the
content-type and are needed to handle the body properly. If you say
that the parameters are not needed by notmuch users, that implies that
they are handled by notmuch. Which is just not possible.
> I think it would help a lot if you could submit some sort of test
> modification that demonstrates the issue. This is one of the reasons we
> keep emphasizing that it's good to first have tests in hand that
> demonstrate issues before patches that address them.
>
The fact that this change happens to fix an issue with HTML charsets for
me is just a side effect.
The real issue is that JSON format throws away information which is
required to properly render a part. I do not think we need to add a
dedicated test to check that JSON outputs charsets with parameters,
considering that it is already present in many other tests.
I do not think it was intended that notmuch outputs stripped
Content-Type values. It was just a side effect of using
g_mime_content_type_to_string(3) which gone unnoticed.
> > "content": [{"id": 2,
> > - "content-type": "text/plain",
> > "content": "This is a test signed message.\n"},
>
> Without figuring out what's going on, I notice that some of the tests
> have been modified to remove the content-type fields on a bunch of
> parts. I think that is probably not right.
>
I tried to explain this in the preable. These parts do not have
Content-Type in the original message. So I think it is wrong for
notmuch JSON format to add it.
Regards,
Dmitry
> jamie.
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v2] Output unmodified Content-Type header value for JSON format.
2011-11-18 23:45 [PATCH] Output unmodified Content-Type header value for JSON format Dmitry Kurochkin
2011-11-19 1:58 ` Jameson Graef Rollins
@ 2011-11-19 4:18 ` Dmitry Kurochkin
2011-11-19 12:59 ` David Bremner
1 sibling, 1 reply; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-19 4:18 UTC (permalink / raw)
To: notmuch
Before the change, notmuch used g_mime_content_type_to_string(3)
function to output Content-Type header value. Turns out it outputs
only "type/subtype" part and ignores all parameters. Also, if there
is no Content-Type header, default "text/plain" value is used.
JSON is supposed to be a "low-level" structured format and should not
add missing values or throw away information. The patch changes
notmuch show to use unmodified Content-Type value for JSON format.
Also, no default value is added if the header is missing.
Corresponding changes to Emacs UI are made to handle full Content-Type
header values. The header is parsed using MIME
`mail-header-parse-content-type' function. All message part rendering
functions have access to full Content-Type value. In particular, this
is important for `notmuch-show-mm-display-part-inline' which uses
`mm-display-part' to display parts that notmuch-show does not handle.
Expected results for the tests are updated accordingly.
---
Changes in v2 since v1:
* Use "text/plain; charset=us-ascii" for default Content-Type value in
`notmuch-show-get-content-type' function as defined in RFC 2045.
emacs/notmuch-show.el | 29 +++++++++++++++++++----------
notmuch-show.c | 14 ++++++++++++--
test/crypto | 23 ++++++++---------------
test/json | 6 +++---
test/maildir-sync | 1 -
test/multipart | 36 ++++++++++++++++++------------------
6 files changed, 60 insertions(+), 49 deletions(-)
diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index d5c95d8..cb801ae 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -261,120 +261,120 @@ message at DEPTH in the current thread."
(if (and header-value
(not (string-equal "" header-value)))
(notmuch-show-insert-header header header-value))))
notmuch-message-headers)
(save-excursion
(save-restriction
(narrow-to-region start (point-max))
(run-hooks 'notmuch-show-markup-headers-hook)))))
(define-button-type 'notmuch-show-part-button-type
'action 'notmuch-show-part-button-action
'follow-link t
'face 'message-mml)
(defun notmuch-show-insert-part-header (nth content-type declared-type &optional name comment)
(let ((button))
(setq button
(insert-button
(concat "[ "
(if name (concat name ": ") "")
- declared-type
- (if (not (string-equal declared-type content-type))
- (concat " (as " content-type ")")
+ (car declared-type)
+ (if (not (string-equal (car declared-type) (car content-type)))
+ (concat " (as " (car content-type) ")")
"")
(or comment "")
" ]")
:type 'notmuch-show-part-button-type
:notmuch-part nth
:notmuch-filename name))
(insert "\n")
;; return button
button))
;; Functions handling particular MIME parts.
(defun notmuch-show-save-part (message-id nth &optional filename)
(let ((process-crypto notmuch-show-process-crypto))
(with-temp-buffer
(setq notmuch-show-process-crypto process-crypto)
;; Always acquires the part via `notmuch part', even if it is
;; available in the JSON output.
(insert (notmuch-show-get-bodypart-internal message-id nth))
(let ((file (read-file-name
"Filename to save as: "
(or mailcap-download-directory "~/")
nil nil
filename)))
;; Don't re-compress .gz & al. Arguably we should make
;; `file-name-handler-alist' nil, but that would chop
;; ange-ftp, which is reasonable to use here.
(mm-write-region (point-min) (point-max) file nil nil nil 'no-conversion t)))))
(defun notmuch-show-mm-display-part-inline (msg part content-type content)
"Use the mm-decode/mm-view functions to display a part in the
current buffer, if possible."
(let ((display-buffer (current-buffer)))
(with-temp-buffer
(insert content)
- (let ((handle (mm-make-handle (current-buffer) (list content-type))))
+ (let ((handle (mm-make-handle (current-buffer) content-type)))
(set-buffer display-buffer)
(if (and (mm-inlinable-p handle)
(mm-inlined-p handle))
(progn
(mm-display-part handle)
t)
nil)))))
(defvar notmuch-show-multipart/alternative-discouraged
'(
;; Avoid HTML parts.
"text/html"
;; multipart/related usually contain a text/html part and some associated graphics.
"multipart/related"
))
(defun notmuch-show-multipart/*-to-list (part)
- (mapcar '(lambda (inner-part) (plist-get inner-part :content-type))
+ (mapcar '(lambda (inner-part) (car (notmuch-show-get-content-type inner-part)))
(plist-get part :content)))
(defun notmuch-show-multipart/alternative-choose (types)
;; Based on `mm-preferred-alternative-precedence'.
(let ((seq types))
(dolist (pref (reverse notmuch-show-multipart/alternative-discouraged))
(dolist (elem (copy-sequence seq))
(when (string-match pref elem)
(setq seq (nconc (delete elem seq) (list elem))))))
seq))
(defun notmuch-show-insert-part-multipart/alternative (msg part content-type nth depth declared-type)
(notmuch-show-insert-part-header nth declared-type content-type nil)
(let ((chosen-type (car (notmuch-show-multipart/alternative-choose (notmuch-show-multipart/*-to-list part))))
(inner-parts (plist-get part :content))
(start (point)))
;; This inserts all parts of the chosen type rather than just one,
;; but it's not clear that this is the wrong thing to do - which
;; should be chosen if there are more than one that match?
(mapc (lambda (inner-part)
- (let ((inner-type (plist-get inner-part :content-type)))
+ (let ((inner-type (notmuch-show-get-content-type inner-part)))
(if (or notmuch-show-all-multipart/alternative-parts
- (string= chosen-type inner-type))
+ (string= chosen-type (car inner-type)))
(notmuch-show-insert-bodypart msg inner-part depth)
(notmuch-show-insert-part-header (plist-get inner-part :id) inner-type inner-type nil " (not shown)"))))
inner-parts)
(when notmuch-show-indent-multipart
(indent-rigidly start (point) 1)))
t)
(defun notmuch-show-setup-w3m ()
"Instruct w3m how to retrieve content from a \"related\" part of a message."
(interactive)
(if (boundp 'w3m-cid-retrieve-function-alist)
(unless (assq 'notmuch-show-mode w3m-cid-retrieve-function-alist)
(push (cons 'notmuch-show-mode 'notmuch-show-w3m-cid-retrieve)
w3m-cid-retrieve-function-alist)))
(setq mm-inline-text-html-with-images t))
(defvar w3m-current-buffer) ;; From `w3m.el'.
(defvar notmuch-show-w3m-cid-store nil)
(make-variable-buffer-local 'notmuch-show-w3m-cid-store)
@@ -557,41 +557,42 @@ current buffer, if possible."
(set-buffer (get-file-buffer file))
(setq result (buffer-substring (point-min) (point-max)))
(set-buffer-modified-p nil)
(kill-buffer (current-buffer))
(delete-file file)
result)))
t)
(defun notmuch-show-insert-part-application/octet-stream (msg part content-type nth depth declared-type)
;; If we can deduce a MIME type from the filename of the attachment,
;; do so and pass it on to the handler for that type.
(if (plist-get part :filename)
(let ((extension (file-name-extension (plist-get part :filename)))
mime-type)
(if extension
(progn
(mailcap-parse-mimetypes)
(setq mime-type (mailcap-extension-to-mime extension))
(if (and mime-type
(not (string-equal mime-type "application/octet-stream")))
- (notmuch-show-insert-bodypart-internal msg part mime-type nth depth content-type)
+ (notmuch-show-insert-bodypart-internal msg part (list mime-type)
+ nth depth content-type)
nil))
nil))))
(defun notmuch-show-insert-part-application/* (msg part content-type nth depth declared-type
)
;; do not render random "application" parts
(notmuch-show-insert-part-header nth content-type declared-type (plist-get part :filename)))
(defun notmuch-show-insert-part-*/* (msg part content-type nth depth declared-type)
;; This handler _must_ succeed - it is the handler of last resort.
(notmuch-show-insert-part-header nth content-type declared-type (plist-get part :filename))
(let ((content (notmuch-show-get-bodypart-content msg part nth)))
(if content
(notmuch-show-mm-display-part-inline msg part content-type content)))
t)
;; Functions for determining how to handle MIME parts.
(defun notmuch-show-split-content-type (content-type)
(split-string content-type "/"))
@@ -618,51 +619,51 @@ current buffer, if possible."
(defun notmuch-show-get-bodypart-internal (message-id part-number)
(let ((args '("show" "--format=raw"))
(part-arg (format "--part=%s" part-number)))
(setq args (append args (list part-arg)))
(if notmuch-show-process-crypto
(setq args (append args '("--decrypt"))))
(setq args (append args (list message-id)))
(with-temp-buffer
(let ((coding-system-for-read 'no-conversion))
(progn
(apply 'call-process (append (list notmuch-command nil (list t nil) nil) args))
(buffer-string))))))
(defun notmuch-show-get-bodypart-content (msg part nth)
(or (plist-get part :content)
(notmuch-show-get-bodypart-internal (concat "id:" (plist-get msg :id)) nth)))
;; \f
(defun notmuch-show-insert-bodypart-internal (msg part content-type nth depth declared-type)
- (let ((handlers (notmuch-show-handlers-for content-type)))
+ (let ((handlers (notmuch-show-handlers-for (car content-type))))
;; Run the content handlers until one of them returns a non-nil
;; value.
(while (and handlers
(not (funcall (car handlers) msg part content-type nth depth declared-type)))
(setq handlers (cdr handlers))))
t)
(defun notmuch-show-insert-bodypart (msg part depth)
"Insert the body part PART at depth DEPTH in the current thread."
- (let ((content-type (downcase (plist-get part :content-type)))
+ (let ((content-type (notmuch-show-get-content-type part))
(nth (plist-get part :id)))
(notmuch-show-insert-bodypart-internal msg part content-type nth depth content-type))
;; Some of the body part handlers leave point somewhere up in the
;; part, so we make sure that we're down at the end.
(goto-char (point-max))
;; Ensure that the part ends with a carriage return.
(if (not (bolp))
(insert "\n")))
(defun notmuch-show-insert-body (msg body depth)
"Insert the body BODY at depth DEPTH in the current thread."
(mapc '(lambda (part) (notmuch-show-insert-bodypart msg part depth)) body))
(defun notmuch-show-make-symbol (type)
(make-symbol (concat "notmuch-show-" type)))
(defun notmuch-show-strip-re (string)
(replace-regexp-in-string "\\([Rr]e: *\\)+" "" string))
(defvar notmuch-show-previous-subject "")
@@ -1054,40 +1055,48 @@ All currently available key bindings:
(save-excursion
(notmuch-show-move-to-message-top)
(get-text-property (point) :notmuch-message-properties)))
(defun notmuch-show-set-prop (prop val &optional props)
(let ((inhibit-read-only t)
(props (or props
(notmuch-show-get-message-properties))))
(plist-put props prop val)
(notmuch-show-set-message-properties props)))
(defun notmuch-show-get-prop (prop &optional props)
(let ((props (or props
(notmuch-show-get-message-properties))))
(plist-get props prop)))
(defun notmuch-show-get-message-id ()
"Return the message id of the current message."
(concat "id:\"" (notmuch-show-get-prop :id) "\""))
+(defun notmuch-show-get-content-type (&optional props)
+ "Return parsed Content-Type of the given message, or part, or
+current message if no `props` is given. If there is no
+Content-Type header, it defaults to
+\"text/plain; charset=us-ascii\" as defined in RFC 2045."
+ (mail-header-parse-content-type (or (notmuch-show-get-prop :content-type props)
+ "text/plain; charset=us-ascii")))
+
;; dme: Would it make sense to use a macro for many of these?
(defun notmuch-show-get-filename ()
"Return the filename of the current message."
(notmuch-show-get-prop :filename))
(defun notmuch-show-get-header (header)
"Return the named header of the current message, if any."
(plist-get (notmuch-show-get-prop :headers) header))
(defun notmuch-show-get-cc ()
(notmuch-show-get-header :Cc))
(defun notmuch-show-get-date ()
(notmuch-show-get-header :Date))
(defun notmuch-show-get-from ()
(notmuch-show-get-header :From))
(defun notmuch-show-get-subject ()
diff --git a/notmuch-show.c b/notmuch-show.c
index 603992a..da3e87f 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -3,40 +3,42 @@
* Copyright © 2009 Carl Worth
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program. If not, see http://www.gnu.org/licenses/ .
*
* Author: Carl Worth <cworth@cworth.org>
*/
#include "notmuch-client.h"
+static const char HEADER_CONTENT_TYPE[] = "Content-Type";
+
static void
format_message_text (unused (const void *ctx),
notmuch_message_t *message,
int indent);
static void
format_headers_text (const void *ctx,
notmuch_message_t *message);
static void
format_headers_message_part_text (GMimeMessage *message);
static void
format_part_start_text (GMimeObject *part,
int *part_count);
static void
format_part_content_text (GMimeObject *part);
static void
format_part_end_text (GMimeObject *part);
@@ -640,42 +642,50 @@ format_part_sigstatus_json (const GMimeSignatureValidity* validity)
}
printf ("}");
signer = signer->next;
}
printf ("]");
talloc_free (ctx_quote);
}
static void
format_part_content_json (GMimeObject *part)
{
GMimeContentType *content_type = g_mime_object_get_content_type (GMIME_OBJECT (part));
GMimeStream *stream_memory = g_mime_stream_mem_new ();
const char *cid = g_mime_object_get_content_id (part);
void *ctx = talloc_new (NULL);
GByteArray *part_content;
- printf (", \"content-type\": %s",
- json_quote_str (ctx, g_mime_content_type_to_string (content_type)));
+ {
+ /* Output full Content-Type header value,
+ * g_mime_content_type_to_string(3) does not include
+ * parameters. Content-Type header may be missing,
+ * g_mime_object_get_content_type(3) defaults to "text/plain"
+ * in this case. */
+ const char *const h = g_mime_object_get_header (part, HEADER_CONTENT_TYPE);
+ if (h)
+ printf (", \"content-type\": %s", json_quote_str (ctx, h));
+ }
if (cid != NULL)
printf(", \"content-id\": %s", json_quote_str (ctx, cid));
if (GMIME_IS_PART (part))
{
const char *filename = g_mime_part_get_filename (GMIME_PART (part));
if (filename)
printf (", \"filename\": %s", json_quote_str (ctx, filename));
}
if (g_mime_content_type_is_type (content_type, "text", "*") &&
!g_mime_content_type_is_type (content_type, "text", "html"))
{
show_text_part_content (part, stream_memory);
part_content = g_mime_stream_mem_get_byte_array (GMIME_STREAM_MEM (stream_memory));
printf (", \"content\": %s", json_quote_chararray (ctx, (char *) part_content->data, part_content->len));
}
else if (g_mime_content_type_is_type (content_type, "multipart", "*"))
diff --git a/test/crypto b/test/crypto
index 0af4aa8..b923d22 100755
--- a/test/crypto
+++ b/test/crypto
@@ -40,111 +40,108 @@ test_expect_success 'emacs delivery of signed message' \
test_begin_subtest "signature verification"
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with full owner trust"
# give the key full owner trust
echo "${FINGERPRINT}:6:" | gpg --no-tty --import-ownertrust >>"$GNUPGHOME"/trust.log 2>&1
gpg --no-tty --check-trustdb >>"$GNUPGHOME"/trust.log 2>&1
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000,
"userid": " Notmuch Test Suite <test_suite@notmuchmail.org> (INSECURE!)"}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with signer key unavailable"
# move the gnupghome temporarily out of the way
mv "${GNUPGHOME}"{,.bak}
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "error",
"keyid": "'$(echo $FINGERPRINT | cut -c 25-)'",
"errors": 2}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
mv "${GNUPGHOME}"{.bak,}
# create a test encrypted message with attachment
cat <<EOF >TESTATTACHMENT
This is a test file.
EOF
test_expect_success 'emacs delivery of encrypted message with attachment' \
'emacs_deliver_message \
"test encrypted message 001" \
"This is a test encrypted message.\n" \
"(mml-attach-file \"TESTATTACHMENT\") (mml-secure-message-encrypt)"'
test_begin_subtest "decryption, --format=text"
@@ -181,138 +178,135 @@ test_expect_equal \
test_begin_subtest "decryption, --format=json"
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "good"}],
"sigstatus": [],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"==-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
- "content-type": "multipart/mixed",
+ "content-type": "multipart/mixed; boundary=\"=-=-=\"",
"content": [{"id": 4,
- "content-type": "text/plain",
"content": "This is a test encrypted message.\n"},
{"id": 5,
"content-type": "application/octet-stream",
"filename": "TESTATTACHMENT"}]}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "decryption, --format=json, --part=4"
output=$(notmuch show --format=json --part=4 --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='{"id": 4,
- "content-type": "text/plain",
"content": "This is a test encrypted message.\n"}'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "decrypt attachment (--part=5 --format=raw)"
notmuch show \
--format=raw \
--part=5 \
--decrypt \
subject:"test encrypted message 001" >OUTPUT
test_expect_equal_file OUTPUT TESTATTACHMENT
test_begin_subtest "decryption failure with missing key"
mv "${GNUPGHOME}"{,.bak}
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "bad"}],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"==-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
"content-type": "application/octet-stream"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
mv "${GNUPGHOME}"{.bak,}
test_expect_success 'emacs delivery of encrypted + signed message' \
'emacs_deliver_message \
"test encrypted message 002" \
"This is another test encrypted message.\n" \
"(mml-secure-message-sign-encrypt)"'
test_begin_subtest "decryption + signature verification"
output=$(notmuch show --format=json --decrypt subject:"test encrypted message 002" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["encrypted","inbox"],
"headers": {"Subject": "test encrypted message 002",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"encstatus": [{"status": "good"}],
"sigstatus": [{"status": "good",
"fingerprint": "'$FINGERPRINT'",
"created": 946728000,
"userid": " Notmuch Test Suite <test_suite@notmuchmail.org> (INSECURE!)"}],
- "content-type": "multipart/encrypted",
+ "content-type": "multipart/encrypted; boundary=\"=-=-=\";\tprotocol=\"application/pgp-encrypted\"",
"content": [{"id": 2,
"content-type": "application/pgp-encrypted"},
{"id": 3,
- "content-type": "text/plain",
"content": "This is another test encrypted message.\n"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "reply to encrypted message"
output=$(notmuch reply --decrypt subject:"test encrypted message 002" \
| grep -v -e '^In-Reply-To:' -e '^References:')
expected='From: Notmuch Test Suite <test_suite@notmuchmail.org>
Subject: Re: test encrypted message 002
On 01 Jan 2000 12:00:00 -0000, Notmuch Test Suite <test_suite@notmuchmail.org> wrote:
> This is another test encrypted message.'
test_expect_equal \
"$output" \
"$expected"
test_begin_subtest "signature verification with revoked key"
# generate revocation certificate and load it to revoke key
@@ -327,32 +321,31 @@ y
| gpg --no-tty --quiet --import
output=$(notmuch show --format=json --verify subject:"test signed message 001" \
| notmuch_json_show_sanitize \
| sed -e 's|"created": [1234567890]*|"created": 946728000|')
expected='[[[{"id": "XXXXX",
"match": true,
"filename": "YYYYY",
"timestamp": 946728000,
"date_relative": "2000-01-01",
"tags": ["inbox","signed"],
"headers": {"Subject": "test signed message 001",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "test_suite@notmuchmail.org",
"Cc": "",
"Bcc": "",
"Date": "01 Jan 2000 12:00:00 -0000"},
"body": [{"id": 1,
"sigstatus": [{"status": "error",
"keyid": "6D92612D94E46381",
"errors": 8}],
- "content-type": "multipart/signed",
+ "content-type": "multipart/signed; boundary=\"=-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"",
"content": [{"id": 2,
- "content-type": "text/plain",
"content": "This is a test signed message.\n"},
{"id": 3,
"content-type": "application/pgp-signature"}]}]},
[]]]]'
test_expect_equal \
"$output" \
"$expected"
test_done
diff --git a/test/json b/test/json
index 592b068..64f35cf 100755
--- a/test/json
+++ b/test/json
@@ -1,50 +1,50 @@
#!/usr/bin/env bash
test_description="--format=json output"
. ./test-lib.sh
test_begin_subtest "Show message: json"
add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-show-message\""
output=$(notmuch show --format=json "json-show-message")
-test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content\": \"json-show-message\n\"}]}, []]]]"
test_begin_subtest "Search message: json"
add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-search-message\""
output=$(notmuch search --format=json "json-search-message" | notmuch_search_sanitize)
test_expect_equal "$output" "[{\"thread\": \"XXX\",
\"timestamp\": 946728000,
\"matched\": 1,
\"total\": 1,
\"authors\": \"Notmuch Test Suite\",
\"subject\": \"json-search-subject\",
\"tags\": [\"inbox\", \"unread\"]}]"
test_begin_subtest "Show message: json, utf-8"
add_message "[subject]=\"json-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-show-méssage\""
output=$(notmuch show --format=json "jsön-show-méssage")
-test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"jsön-show-méssage\n\"}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"${gen_msg_id}\", \"match\": true, \"filename\": \"${gen_msg_filename}\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content\": \"jsön-show-méssage\n\"}]}, []]]]"
test_begin_subtest "Show message: json, inline attachment filename"
subject='json-show-inline-attachment-filename'
id="json-show-inline-attachment-filename@notmuchmail.org"
emacs_deliver_message \
"$subject" \
'This is a test message with inline attachment with a filename' \
"(mml-attach-file \"$TEST_DIRECTORY/README\" nil nil \"inline\")
(message-goto-eoh)
(insert \"Message-ID: <$id>\n\")"
output=$(notmuch show --format=json "id:$id")
filename=$(notmuch search --output=files "id:$id")
-test_expect_equal "$output" "[[[{\"id\": \"$id\", \"match\": true, \"filename\": \"$filename\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"test_suite@notmuchmail.org\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed\", \"content\": [{\"id\": 2, \"content-type\": \"text/plain\", \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"filename\": \"README\"}]}]}, []]]]"
+test_expect_equal "$output" "[[[{\"id\": \"$id\", \"match\": true, \"filename\": \"$filename\", \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite <test_suite@notmuchmail.org>\", \"To\": \"test_suite@notmuchmail.org\", \"Cc\": \"\", \"Bcc\": \"\", \"Date\": \"01 Jan 2000 12:00:00 -0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed; boundary=\\\"=-=-=\\\"\", \"content\": [{\"id\": 2, \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"filename\": \"README\"}]}]}, []]]]"
test_begin_subtest "Search message: json, utf-8"
add_message "[subject]=\"json-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-search-méssage\""
output=$(notmuch search --format=json "jsön-search-méssage" | notmuch_search_sanitize)
test_expect_equal "$output" "[{\"thread\": \"XXX\",
\"timestamp\": 946728000,
\"matched\": 1,
\"total\": 1,
\"authors\": \"Notmuch Test Suite\",
\"subject\": \"json-search-utf8-body-sübjéct\",
\"tags\": [\"inbox\", \"unread\"]}]"
test_done
diff --git a/test/maildir-sync b/test/maildir-sync
index a60854f..c7ca22f 100755
--- a/test/maildir-sync
+++ b/test/maildir-sync
@@ -41,41 +41,40 @@ add_message [subject]='"Adding replied tag"' [filename]='adding-replied-tag:2,S'
notmuch tag +replied subject:"Adding replied tag"
output=$(cd ${MAIL_DIR}/cur; ls -1 adding-replied*)
test_expect_equal "$output" "adding-replied-tag:2,RS"
test_begin_subtest "notmuch show works with renamed file (without notmuch new)"
output=$(notmuch show --format=json id:${gen_msg_id} | filter_show_json)
test_expect_equal "$output" '[[[{"id": "adding-replied-tag@notmuch-test-suite",
"match": true,
"filename": "MAIL_DIR/cur/adding-replied-tag:2,RS",
"timestamp": 978709437,
"date_relative": "2001-01-05",
"tags": ["inbox","replied"],
"headers": {"Subject": "Adding replied tag",
"From": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"To": "Notmuch Test Suite <test_suite@notmuchmail.org>",
"Cc": "",
"Bcc": "",
"Date": "Tue,
05 Jan 2001 15:43:57 -0000"},
"body": [{"id": 1,
-"content-type": "text/plain",
"content": "This is just a test message (#3)\n"}]},
[]]]]'
test_expect_success 'notmuch reply works with renamed file (without notmuch new)' 'notmuch reply id:${gen_msg_id}'
test_begin_subtest "notmuch new detects no file rename after tag->flag synchronization"
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail."
test_begin_subtest "When read, message moved from new to cur"
add_message [subject]='"Message to move to cur"' [date]='"Sat, 01 Jan 2000 12:00:00 -0000"' [filename]='message-to-move-to-cur' [dir]=new
notmuch tag -unread subject:"Message to move to cur"
output=$(cd "$MAIL_DIR/cur"; ls message-to-move*)
test_expect_equal "$output" "message-to-move-to-cur:2,S"
test_begin_subtest "No rename should be detected by notmuch new"
output=$(NOTMUCH_NEW)
test_expect_equal "$output" "No new mail."
# (*) If notmuch new was not run we've got "Processed 1 file in almost
# no time" here. The reason is that removing unread tag in a previous
diff --git a/test/multipart b/test/multipart
index f83526b..ca4db71 100755
--- a/test/multipart
+++ b/test/multipart
@@ -306,140 +306,140 @@ test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=text --part=9, pgp signature (unverified)"
notmuch show --format=text --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
cat <<EOF >EXPECTED
\fpart{ ID: 9, Content-type: application/pgp-signature
Non-text part: application/pgp-signature
\fpart}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_expect_success \
"--format=text --part=8, no part, expect error" \
"notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
test_begin_subtest "--format=json --part=0, full message"
notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "match": true, "filename": "${MAIL_DIR}/multipart", "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Cc": "", "Bcc": "", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [
-{"id": 1, "content-type": "multipart/signed", "content": [
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 1, "content-type": "multipart/signed; boundary=\"==-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]},
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]},
{"id": 9, "content-type": "application/pgp-signature"}]}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=1, message body"
notmuch show --format=json --part=1 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 1, "content-type": "multipart/signed", "content": [
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 1, "content-type": "multipart/signed; boundary=\"==-=-=\";\tmicalg=pgp-sha1; protocol=\"application/pgp-signature\"", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]},
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]},
{"id": 9, "content-type": "application/pgp-signature"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=2, multipart/mixed"
notmuch show --format=json --part=2 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 2, "content-type": "multipart/mixed", "content": [
+{"id": 2, "content-type": "multipart/mixed; boundary=\"=-=-=\"", "content": [
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]},
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"},
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"},
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=3, rfc822 part"
notmuch show --format=json --part=3 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 3, "content-type": "message/rfc822", "content": [{"headers": {"From": "Carl Worth <cworth@cworth.org>", "To": "cworth@cworth.org", "Subject": "html message", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=4, rfc822's multipart/alternative"
notmuch show --format=json --part=4 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 4, "content-type": "multipart/alternative", "content": [
+{"id": 4, "content-type": "multipart/alternative; boundary=\"==-=-==\"", "content": [
{"id": 5, "content-type": "text/html"},
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=5, rfc822's html part"
notmuch show --format=json --part=5 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 5, "content-type": "text/html"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=6, rfc822's text part"
notmuch show --format=json --part=6 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=7, inline attachment"
notmuch show --format=json --part=7 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 7, "content-type": "text/plain", "filename": "attachment", "content": "This is a text attachment.\n"}
+{"id": 7, "filename": "attachment", "content": "This is a text attachment.\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=8, plain text part"
notmuch show --format=json --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
-{"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}
+{"id": 8, "content": "And this message is signed.\n\n-Carl\n"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_begin_subtest "--format=json --part=9, pgp signature (unverified)"
notmuch show --format=json --part=9 'id:87liy5ap00.fsf@yoom.home.cworth.org' | sed 's|{"id":|\n{"id":|g' >OUTPUT
echo >>OUTPUT # expect *no* newline at end of output
cat <<EOF >EXPECTED
{"id": 9, "content-type": "application/pgp-signature"}
EOF
test_expect_equal_file OUTPUT EXPECTED
test_expect_success \
"--format=json --part=10, no part, expect error" \
"notmuch show --format=json --part=10 'id:87liy5ap00.fsf@yoom.home.cworth.org'"
test_begin_subtest "--format=raw"
notmuch show --format=raw 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT
test_expect_equal_file OUTPUT "${MAIL_DIR}"/multipart
--
1.7.7.3
^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 2:42 ` Dmitry Kurochkin
@ 2011-11-19 4:59 ` Austin Clements
2011-11-19 5:26 ` Dmitry Kurochkin
2011-11-19 10:49 ` Jameson Graef Rollins
1 sibling, 1 reply; 24+ messages in thread
From: Austin Clements @ 2011-11-19 4:59 UTC (permalink / raw)
To: Dmitry Kurochkin; +Cc: notmuch
Quoth Dmitry Kurochkin on Nov 19 at 6:42 am:
> Hi Jamie.
>
> On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > function to output Content-Type header value. Turns out it outputs
> > > only "type/subtype" part and ignores all parameters. Also, if there
> > > is no Content-Type header, default "text/plain" value is used.
> >
> > Hi, Dmitry. Can you explain under what circumstances you would need the
> > extra content-type parameters?
>
> Charset is an example of a parameter which you need to render a part
> correctly.
Can notmuch convert to a common charset, given that, otherwise, every
client is going to have to implement this conversion anyway?
(And are there other examples of useful things in the content type?)
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 4:59 ` Austin Clements
@ 2011-11-19 5:26 ` Dmitry Kurochkin
2011-11-19 18:58 ` Austin Clements
0 siblings, 1 reply; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-19 5:26 UTC (permalink / raw)
To: Austin Clements; +Cc: notmuch
On Fri, 18 Nov 2011 23:59:57 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Dmitry Kurochkin on Nov 19 at 6:42 am:
> > Hi Jamie.
> >
> > On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > > On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > > function to output Content-Type header value. Turns out it outputs
> > > > only "type/subtype" part and ignores all parameters. Also, if there
> > > > is no Content-Type header, default "text/plain" value is used.
> > >
> > > Hi, Dmitry. Can you explain under what circumstances you would need the
> > > extra content-type parameters?
> >
> > Charset is an example of a parameter which you need to render a part
> > correctly.
>
> Can notmuch convert to a common charset, given that, otherwise, every
> client is going to have to implement this conversion anyway?
>
Notmuch can handle charset (and any other) parameters but only for known
media types (i.e. text/*). I think it would be useful (especially for
human-readable output formats). But it is a separate issue.
Notmuch can not convert other types it does not know how to handle.
E.g. HTML charset conversion is not as simple as for plain text.
AFAIK standard defines charset parameter just for few types. So in
general, charset parameter can have any meaning for some custom media
type.
> (And are there other examples of useful things in the content type?)
What is meant by useful? All parameters do have some use. The fact
that notmuch does not handle them does not mean they are useless. And
notmuch can not handle all parameters just because the list of
parameters is not defined. So there is no choice but to let notmuch
users see and use these parameters.
Regards,
Dmitry
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 2:42 ` Dmitry Kurochkin
2011-11-19 4:59 ` Austin Clements
@ 2011-11-19 10:49 ` Jameson Graef Rollins
2011-11-20 18:32 ` Dmitry Kurochkin
1 sibling, 1 reply; 24+ messages in thread
From: Jameson Graef Rollins @ 2011-11-19 10:49 UTC (permalink / raw)
To: Dmitry Kurochkin, notmuch
[-- Attachment #1: Type: text/plain, Size: 2734 bytes --]
On Sat, 19 Nov 2011 06:42:00 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> The parameters are there for a reason. They are part of the
> content-type and are needed to handle the body properly. If you say
> that the parameters are not needed by notmuch users, that implies that
> they are handled by notmuch. Which is just not possible.
Hey, Dimitry. At least some of the parameters in the content-type are
actually related to the mime structure itself. A good example is:
boundary=\"=-=-=\"
This parameter is there to tell the MIME parser how to parse the content
that follows. It is meant for the first level parser and no more. Once
the MIME has been separated into it's constituent parts, there's no need
for any further clients to know anything about boundary string.
I would argue that notmuch is acting as the first level parser. As far
as I can tell, most of the rest of the parameters I've seen should only
be useful to the those first-level parsers.
As Austin mentioned, is it not possible for notmuch itself to act on the
parameter to give a properly formatted output to its clients?
> The fact that this change happens to fix an issue with HTML charsets for
> me is just a side effect.
But isn't that actually a large part of the issue? If this patch fixes
something that you think notmuch is doing improperly, could there not be
a test for it?
However, based on your patches and as far as I can tell, this change
adds more than a boundary parameter to only crypto parts
(application/signed and application/encrypted). However, I don't think
any of the crypto functionality needs any of the extra information
provided in the extended output. If there was a test for the
functionality you think is missing, it would help bolster the case for
the additional output.
> > > "content": [{"id": 2,
> > > - "content-type": "text/plain",
> > > "content": "This is a test signed message.\n"},
> >
> > Without figuring out what's going on, I notice that some of the tests
> > have been modified to remove the content-type fields on a bunch of
> > parts. I think that is probably not right.
> >
>
> I tried to explain this in the preable. These parts do not have
> Content-Type in the original message. So I think it is wrong for
> notmuch JSON format to add it.
Ah, ok, I think I understand this point. I think this is actually a
separate issue than the one the rest of the patch set is for, though.
One part of the patch is that content-type parameters are also about,
and another part is that parts without content-type shouldn't be
assigned one automatically. I personally think those should be separate
patches.
jamie.
[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2] Output unmodified Content-Type header value for JSON format.
2011-11-19 4:18 ` [PATCH v2] " Dmitry Kurochkin
@ 2011-11-19 12:59 ` David Bremner
2011-11-20 18:35 ` Dmitry Kurochkin
0 siblings, 1 reply; 24+ messages in thread
From: David Bremner @ 2011-11-19 12:59 UTC (permalink / raw)
To: Dmitry Kurochkin, notmuch
On Sat, 19 Nov 2011 08:18:41 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> Before the change, notmuch used g_mime_content_type_to_string(3)
> function to output Content-Type header value. Turns out it outputs
> only "type/subtype" part and ignores all parameters. Also, if there
> is no Content-Type header, default "text/plain" value is used.
Hi Dmitry;
I haven't analyzed the substance of your patch yet, but I did have a
couple thoughts while reading your mail.
- It seems that every time we change the json format, we have a round of
suffering because people are unable to detect a mismatch between their
emacs code and the cli. Not that this is your problem necessarily, but
it would be nice if someone (TM), would come up with some version info
for the json output, and a patch to check it on the emacs side.
- The previous point is a bit of a counterargument to this, but in
general, I think I prefer patches that modify the core seperate from
those that do emacs (or python, or ...) stuff.
- I understand you want to make your patches reviewable without applying
by including lots of context, but at a certain point it has actually
the opposite effect for me. I just don't read 900+ line emails ;). Of
course, I can still apply the patch and look at it, so it's really up
to you.
d
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 5:26 ` Dmitry Kurochkin
@ 2011-11-19 18:58 ` Austin Clements
2011-11-20 18:52 ` Dmitry Kurochkin
0 siblings, 1 reply; 24+ messages in thread
From: Austin Clements @ 2011-11-19 18:58 UTC (permalink / raw)
To: Dmitry Kurochkin; +Cc: notmuch
Quoth Dmitry Kurochkin on Nov 19 at 9:26 am:
> On Fri, 18 Nov 2011 23:59:57 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > Quoth Dmitry Kurochkin on Nov 19 at 6:42 am:
> > > Hi Jamie.
> > >
> > > On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > > > On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > > > function to output Content-Type header value. Turns out it outputs
> > > > > only "type/subtype" part and ignores all parameters. Also, if there
> > > > > is no Content-Type header, default "text/plain" value is used.
> > > >
> > > > Hi, Dmitry. Can you explain under what circumstances you would need the
> > > > extra content-type parameters?
> > >
> > > Charset is an example of a parameter which you need to render a part
> > > correctly.
> >
> > Can notmuch convert to a common charset, given that, otherwise, every
> > client is going to have to implement this conversion anyway?
> >
>
> Notmuch can handle charset (and any other) parameters but only for known
> media types (i.e. text/*). I think it would be useful (especially for
> human-readable output formats). But it is a separate issue.
>
> Notmuch can not convert other types it does not know how to handle.
> E.g. HTML charset conversion is not as simple as for plain text.
>
> AFAIK standard defines charset parameter just for few types. So in
> general, charset parameter can have any meaning for some custom media
> type.
Interesting. I hadn't realized the content-type specification was so
open-ended. However, there are many things that *could* be included
in the JSON format but aren't; what's included is primarily driven by
what consumers actually need and it seems like the actual need here is
charset handling. Maybe the JSON format *shouldn't* evolve this way,
but I think it should either be driven by its needs like it is now, or
we should be taking bigger steps like providing *all* of the headers
(essentially, a JSON-ification of the MIME structure), which would
subsume more specific generalizations like exposing just the full
content-type header.
Regarding charset, specifically, though, the JSON format only includes
part bodies for text/* types and, according to RFC 2045,
For example, the "charset" parameter is applicable to any subtype of
"text", ...
Section 4.1.2 (Charset Parameter) of RFC 2046 beats around the bush,
but I think it's saying essentially the same thing in a lot more
detail. Given that, I think it does make sense for notmuch to handle
the charset parameter and re-coding.
> > (And are there other examples of useful things in the content type?)
>
> What is meant by useful? All parameters do have some use. The fact
> that notmuch does not handle them does not mean they are useless. And
> notmuch can not handle all parameters just because the list of
> parameters is not defined. So there is no choice but to let notmuch
> users see and use these parameters.
Yes, I now agree with this, modulo my statements about generality above.
> Regards,
> Dmitry
>
--
Austin Clements MIT/'06/PhD/CSAIL
amdragon@mit.edu http://web.mit.edu/amdragon
Somewhere in the dream we call reality you will find me,
searching for the reality we call dreams.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 10:49 ` Jameson Graef Rollins
@ 2011-11-20 18:32 ` Dmitry Kurochkin
2011-11-20 20:10 ` Jameson Graef Rollins
0 siblings, 1 reply; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-20 18:32 UTC (permalink / raw)
To: Jameson Graef Rollins, notmuch
Hi Jameson.
On Sat, 19 Nov 2011 02:49:43 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> On Sat, 19 Nov 2011 06:42:00 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > The parameters are there for a reason. They are part of the
> > content-type and are needed to handle the body properly. If you say
> > that the parameters are not needed by notmuch users, that implies that
> > they are handled by notmuch. Which is just not possible.
>
> Hey, Dimitry. At least some of the parameters in the content-type are
> actually related to the mime structure itself. A good example is:
>
> boundary=\"=-=-=\"
>
> This parameter is there to tell the MIME parser how to parse the content
> that follows. It is meant for the first level parser and no more. Once
> the MIME has been separated into it's constituent parts, there's no need
> for any further clients to know anything about boundary string.
>
Yes, at least in most cases. On the other hand, if you can make notmuch
show raw multipart part (you can, right?), then it seems natural that
notmuch provides enough information to parse it.
> I would argue that notmuch is acting as the first level parser. As far
> as I can tell, most of the rest of the parameters I've seen should only
> be useful to the those first-level parsers.
>
I do not think first-level parser is a standard term. As I understand,
you mean that notmuch parses MIME recursively until the leaf non-MIME
parts. Correct?
I do not know what parameters you have seen. The most common example of
a useful parameter for second-level parsers is a charset.
I do not understand why do we try to come up with excuses for not
providing useful information to users. Current assumption that all
parameters that notmuch does not handle are useless is plain invalid.
> As Austin mentioned, is it not possible for notmuch itself to act on the
> parameter to give a properly formatted output to its clients?
>
Please see my answer to Austin. I explained why this is not an option
in general case.
As for parameters that notmuch already handles, like "boundary", I just
do not understand why we should invent some artificial exceptions and
decide for our users what parameters are useful or useless for them
instead of implementing a simple and kind of expected approach:
content-type in JSON is original Content-Type header value. It makes
both the code and the format simpler.
> > The fact that this change happens to fix an issue with HTML charsets for
> > me is just a side effect.
>
> But isn't that actually a large part of the issue? If this patch fixes
> something that you think notmuch is doing improperly, could there not be
> a test for it?
>
No. It just happens to be how I found the problem. The issue is:
notmuch JSON format mangles Content-Type header value by throwing away
useful information in some cases and adding info that was not there in
others. Note that I do not mention any single parameter name here. It
is a general issue, not a "charset" or "boundary" parameter issue.
> However, based on your patches and as far as I can tell, this change
> adds more than a boundary parameter to only crypto parts
> (application/signed and application/encrypted). However, I don't think
> any of the crypto functionality needs any of the extra information
> provided in the extended output. If there was a test for the
> functionality you think is missing, it would help bolster the case for
> the additional output.
>
Again, the patch is not about "add boundary and other useless crypto
parameters". The patch is about stop throwing away useless
information.
> > > > "content": [{"id": 2,
> > > > - "content-type": "text/plain",
> > > > "content": "This is a test signed message.\n"},
> > >
> > > Without figuring out what's going on, I notice that some of the tests
> > > have been modified to remove the content-type fields on a bunch of
> > > parts. I think that is probably not right.
> > >
> >
> > I tried to explain this in the preable. These parts do not have
> > Content-Type in the original message. So I think it is wrong for
> > notmuch JSON format to add it.
>
> Ah, ok, I think I understand this point. I think this is actually a
> separate issue than the one the rest of the patch set is for, though.
> One part of the patch is that content-type parameters are also about,
> and another part is that parts without content-type shouldn't be
> assigned one automatically. I personally think those should be separate
> patches.
>
The implementation makes it not practical to separate these changes.
They come as a result of the same code change.
Regards,
Dmitry
> jamie.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2] Output unmodified Content-Type header value for JSON format.
2011-11-19 12:59 ` David Bremner
@ 2011-11-20 18:35 ` Dmitry Kurochkin
2012-01-12 17:08 ` Pieter Praet
0 siblings, 1 reply; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-20 18:35 UTC (permalink / raw)
To: David Bremner, notmuch
On Sat, 19 Nov 2011 08:59:29 -0400, David Bremner <david@tethera.net> wrote:
> On Sat, 19 Nov 2011 08:18:41 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > Before the change, notmuch used g_mime_content_type_to_string(3)
> > function to output Content-Type header value. Turns out it outputs
> > only "type/subtype" part and ignores all parameters. Also, if there
> > is no Content-Type header, default "text/plain" value is used.
>
> Hi Dmitry;
>
> I haven't analyzed the substance of your patch yet, but I did have a
> couple thoughts while reading your mail.
>
> - It seems that every time we change the json format, we have a round of
> suffering because people are unable to detect a mismatch between their
> emacs code and the cli. Not that this is your problem necessarily, but
> it would be nice if someone (TM), would come up with some version info
> for the json output, and a patch to check it on the emacs side.
>
IMO this is a good idea.
> - The previous point is a bit of a counterargument to this, but in
> general, I think I prefer patches that modify the core seperate from
> those that do emacs (or python, or ...) stuff.
>
I couls separate it. I made is a single patch to avoid having a
revision with broken emacs UI (and tests).
Regards,
Dmitry
> - I understand you want to make your patches reviewable without applying
> by including lots of context, but at a certain point it has actually
> the opposite effect for me. I just don't read 900+ line emails ;). Of
> course, I can still apply the patch and look at it, so it's really up
> to you.
>
> d
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-19 18:58 ` Austin Clements
@ 2011-11-20 18:52 ` Dmitry Kurochkin
0 siblings, 0 replies; 24+ messages in thread
From: Dmitry Kurochkin @ 2011-11-20 18:52 UTC (permalink / raw)
To: Austin Clements; +Cc: notmuch
On Sat, 19 Nov 2011 13:58:18 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Dmitry Kurochkin on Nov 19 at 9:26 am:
> > On Fri, 18 Nov 2011 23:59:57 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > > Quoth Dmitry Kurochkin on Nov 19 at 6:42 am:
> > > > Hi Jamie.
> > > >
> > > > On Fri, 18 Nov 2011 17:58:52 -0800, Jameson Graef Rollins <jrollins@finestructure.net> wrote:
> > > > > On Sat, 19 Nov 2011 03:45:05 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > > > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > > > > function to output Content-Type header value. Turns out it outputs
> > > > > > only "type/subtype" part and ignores all parameters. Also, if there
> > > > > > is no Content-Type header, default "text/plain" value is used.
> > > > >
> > > > > Hi, Dmitry. Can you explain under what circumstances you would need the
> > > > > extra content-type parameters?
> > > >
> > > > Charset is an example of a parameter which you need to render a part
> > > > correctly.
> > >
> > > Can notmuch convert to a common charset, given that, otherwise, every
> > > client is going to have to implement this conversion anyway?
> > >
> >
> > Notmuch can handle charset (and any other) parameters but only for known
> > media types (i.e. text/*). I think it would be useful (especially for
> > human-readable output formats). But it is a separate issue.
> >
> > Notmuch can not convert other types it does not know how to handle.
> > E.g. HTML charset conversion is not as simple as for plain text.
> >
> > AFAIK standard defines charset parameter just for few types. So in
> > general, charset parameter can have any meaning for some custom media
> > type.
>
> Interesting. I hadn't realized the content-type specification was so
> open-ended. However, there are many things that *could* be included
> in the JSON format but aren't; what's included is primarily driven by
> what consumers actually need and it seems like the actual need here is
> charset handling. Maybe the JSON format *shouldn't* evolve this way,
> but I think it should either be driven by its needs like it is now, or
> we should be taking bigger steps like providing *all* of the headers
> (essentially, a JSON-ification of the MIME structure), which would
> subsume more specific generalizations like exposing just the full
> content-type header.
>
I think it is a good idea to provide all headers in JSON output. Still
I believe this patch is still valid. It is a simple change, which makes
the JSON format simpler and we have consumers that need it. Providing
all headers would be a bigger change (and I expect it to be much more
difficult to get accepted).
What I definately do not like, is adding an exception for charset
parameter and inventing complex rules for JSON format instead of keeping
it simple.
> Regarding charset, specifically, though, the JSON format only includes
> part bodies for text/* types and, according to RFC 2045,
>
> For example, the "charset" parameter is applicable to any subtype of
> "text", ...
>
> Section 4.1.2 (Charset Parameter) of RFC 2046 beats around the bush,
> but I think it's saying essentially the same thing in a lot more
> detail. Given that, I think it does make sense for notmuch to handle
> the charset parameter and re-coding.
>
I think it may be a good idea but it is not trivial to do right. We
should not just convert all text parts unconditionally to locale or
UTF-8.
> > > (And are there other examples of useful things in the content type?)
> >
> > What is meant by useful? All parameters do have some use. The fact
> > that notmuch does not handle them does not mean they are useless. And
> > notmuch can not handle all parameters just because the list of
> > parameters is not defined. So there is no choice but to let notmuch
> > users see and use these parameters.
>
> Yes, I now agree with this, modulo my statements about generality above.
>
Thanks.
Regards,
Dmitry
> > Regards,
> > Dmitry
> >
>
> --
> Austin Clements MIT/'06/PhD/CSAIL
> amdragon@mit.edu http://web.mit.edu/amdragon
> Somewhere in the dream we call reality you will find me,
> searching for the reality we call dreams.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-20 18:32 ` Dmitry Kurochkin
@ 2011-11-20 20:10 ` Jameson Graef Rollins
2011-11-23 3:40 ` Austin Clements
0 siblings, 1 reply; 24+ messages in thread
From: Jameson Graef Rollins @ 2011-11-20 20:10 UTC (permalink / raw)
To: Dmitry Kurochkin, notmuch
[-- Attachment #1: Type: text/plain, Size: 2811 bytes --]
On Sun, 20 Nov 2011 22:32:53 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> Yes, at least in most cases. On the other hand, if you can make notmuch
> show raw multipart part (you can, right?), then it seems natural that
> notmuch provides enough information to parse it.
This is kind of an unresolved issue, actually. Currently headers are
only included in the "raw" format output of an entire message.
Otherwise, when raw output is requested of an individual part only the
body is output. For multipart parts, the raw format output includes all
body parts concatenated together, still without any headers.
This "raw" multipart output clearly doesn't really make much sense and
we need to figure that out. dkg wrote a good breakdown of the issue
here:
id:"4E09072A.7040906@fifthhorseman.net"
However, this only for "raw" output. It's definitely not the same as
the json output. For json the parts are all parsed by notmuch and
placed into separate json elements. The receiver is not going to do any
further parsing since all the mime structure parsing has been done.
We need to keep clear the distinction between "parsing" the mime
structure, and "encoding" the content of the part. Confusion seems to
be coming from the fact that the Content-Type header includes
information needed for both mime parsing and content encoding. However,
I don't think that means that we need to just include everything in the
output. Parameters that have to do with mime parsing should be dropped,
since that information has already been used in the mime parsing and
can't is no longer useful to the consumer. It's just noise, and I don't
think notmuch should be outputting useless noise.
The open question seems to be how we handle the content encoding
parameters. My argument is that those should either be used by notmuch
to properly encode the content for the consumer. If that's not
possible, then just those parameters needed by the consumer to decode
the content should be output.
> > But isn't that actually a large part of the issue? If this patch fixes
> > something that you think notmuch is doing improperly, could there not be
> > a test for it?
>
> No. It just happens to be how I found the problem. The issue is:
> notmuch JSON format mangles Content-Type header value by throwing away
> useful information in some cases and adding info that was not there in
> others. Note that I do not mention any single parameter name here. It
> is a general issue, not a "charset" or "boundary" parameter issue.
I'm sorry, but I still don't believe it's not possible to test for this
issue. If there's a problem that you're seeing, then you must of
identified it somehow, and therefore there must be a way to test for it.
jamie.
[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-20 20:10 ` Jameson Graef Rollins
@ 2011-11-23 3:40 ` Austin Clements
2012-01-12 17:07 ` Pieter Praet
0 siblings, 1 reply; 24+ messages in thread
From: Austin Clements @ 2011-11-23 3:40 UTC (permalink / raw)
To: Jameson Graef Rollins; +Cc: notmuch
Quoth Jameson Graef Rollins on Nov 20 at 12:10 pm:
> The open question seems to be how we handle the content encoding
> parameters. My argument is that those should either be used by notmuch
> to properly encode the content for the consumer. If that's not
> possible, then just those parameters needed by the consumer to decode
> the content should be output.
If notmuch is going to include part content in the JSON output (which
perhaps it shouldn't, as per recent IRC discussions), then it must
handle content encodings because JSON must be Unicode and therefore
the content strings in the JSON must be Unicode.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2011-11-23 3:40 ` Austin Clements
@ 2012-01-12 17:07 ` Pieter Praet
2012-01-12 17:28 ` Austin Clements
0 siblings, 1 reply; 24+ messages in thread
From: Pieter Praet @ 2012-01-12 17:07 UTC (permalink / raw)
To: Austin Clements, Jameson Graef Rollins; +Cc: notmuch
On Tue, 22 Nov 2011 22:40:21 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Jameson Graef Rollins on Nov 20 at 12:10 pm:
> > The open question seems to be how we handle the content encoding
> > parameters. My argument is that those should either be used by notmuch
> > to properly encode the content for the consumer. If that's not
> > possible, then just those parameters needed by the consumer to decode
> > the content should be output.
>
> If notmuch is going to include part content in the JSON output (which
> perhaps it shouldn't, as per recent IRC discussions), then it must
> handle content encodings because JSON must be Unicode and therefore
> the content strings in the JSON must be Unicode.
Having missed the IRC discussions: what is the rationale for not
including (specific types of?) part content in the JSON output ?
Eg. how about inline attached text/x-patch ?
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
Peace
--
Pieter
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2] Output unmodified Content-Type header value for JSON format.
2011-11-20 18:35 ` Dmitry Kurochkin
@ 2012-01-12 17:08 ` Pieter Praet
2012-01-13 3:16 ` David Bremner
0 siblings, 1 reply; 24+ messages in thread
From: Pieter Praet @ 2012-01-12 17:08 UTC (permalink / raw)
To: Dmitry Kurochkin, David Bremner, notmuch
On Sun, 20 Nov 2011 22:35:42 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> On Sat, 19 Nov 2011 08:59:29 -0400, David Bremner <david@tethera.net> wrote:
> > On Sat, 19 Nov 2011 08:18:41 +0400, Dmitry Kurochkin <dmitry.kurochkin@gmail.com> wrote:
> > > Before the change, notmuch used g_mime_content_type_to_string(3)
> > > function to output Content-Type header value. Turns out it outputs
> > > only "type/subtype" part and ignores all parameters. Also, if there
> > > is no Content-Type header, default "text/plain" value is used.
> >
> > Hi Dmitry;
> >
> > I haven't analyzed the substance of your patch yet, but I did have a
> > couple thoughts while reading your mail.
> >
> > - It seems that every time we change the json format, we have a round of
> > suffering because people are unable to detect a mismatch between their
> > emacs code and the cli. Not that this is your problem necessarily, but
> > it would be nice if someone (TM), would come up with some version info
> > for the json output, and a patch to check it on the emacs side.
> >
>
> IMO this is a good idea.
>
> > - The previous point is a bit of a counterargument to this, but in
> > general, I think I prefer patches that modify the core seperate from
> > those that do emacs (or python, or ...) stuff.
> >
>
> I couls separate it. I made is a single patch to avoid having a
> revision with broken emacs UI (and tests).
>
I'd like to propose to always apply patch series on a *topic* branch
which would then be merged back into 'master', thus avoiding this issue
altogether whilst making it more obvious which patches belong together
(eg. for easier cross-referencing with the ML).
> Regards,
> Dmitry
>
> > - I understand you want to make your patches reviewable without applying
> > by including lots of context, but at a certain point it has actually
> > the opposite effect for me. I just don't read 900+ line emails ;). Of
> > course, I can still apply the patch and look at it, so it's really up
> > to you.
> >
> > d
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
Peace
--
Pieter
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-12 17:07 ` Pieter Praet
@ 2012-01-12 17:28 ` Austin Clements
2012-01-14 9:19 ` Pieter Praet
0 siblings, 1 reply; 24+ messages in thread
From: Austin Clements @ 2012-01-12 17:28 UTC (permalink / raw)
To: Pieter Praet; +Cc: notmuch
Quoth Pieter Praet on Jan 12 at 6:07 pm:
> On Tue, 22 Nov 2011 22:40:21 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > Quoth Jameson Graef Rollins on Nov 20 at 12:10 pm:
> > > The open question seems to be how we handle the content encoding
> > > parameters. My argument is that those should either be used by notmuch
> > > to properly encode the content for the consumer. If that's not
> > > possible, then just those parameters needed by the consumer to decode
> > > the content should be output.
> >
> > If notmuch is going to include part content in the JSON output (which
> > perhaps it shouldn't, as per recent IRC discussions), then it must
> > handle content encodings because JSON must be Unicode and therefore
> > the content strings in the JSON must be Unicode.
>
> Having missed the IRC discussions: what is the rationale for not
> including (specific types of?) part content in the JSON output ?
> Eg. how about inline attached text/x-patch ?
Technically the IRC discussion was about not including *any* part
content in the JSON output, and always using show --format=raw or
similar to retrieve desired parts. Currently, notmuch includes part
content in the JSON only for text/*, *except* when it's text/html. I
assume non-text parts are omitted because binary data is hard to
represent in JSON and text/html is omitted because some people don't
need it. However, this leads to some peculiar asymmetry in the Emacs
code where sometimes it pulls part content out of the JSON and
sometimes it retrieves it using show --format=raw. This in turn leads
to asymmetry in content encoding handling, since notmuch handles
content encoding for parts included in the JSON (and there's no good
way around that since JSON is Unicode), but not for parts retrieved as
raw.
The idea discussed on IRC was to remove all part content from the JSON
output and to always use show to retrieve it, possibly beefing up
show's support for content decoding (and possibly introducing a way to
retrieve multiple raw parts at once to avoid re-parsing). This would
get the JSON format out of the business of guessing what consumers
need, simplify the Emacs code, and normalize content encoding
handling.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2] Output unmodified Content-Type header value for JSON format.
2012-01-12 17:08 ` Pieter Praet
@ 2012-01-13 3:16 ` David Bremner
2012-01-14 8:59 ` Pieter Praet
0 siblings, 1 reply; 24+ messages in thread
From: David Bremner @ 2012-01-13 3:16 UTC (permalink / raw)
To: Pieter Praet, notmuch
On Thu, 12 Jan 2012 18:08:08 +0100, Pieter Praet <pieter@praet.org> wrote:
> > > - The previous point is a bit of a counterargument to this, but in
> > I couls separate it. I made is a single patch to avoid having a
> > revision with broken emacs UI (and tests).
> >
> I'd like to propose to always apply patch series on a *topic* branch
> which would then be merged back into 'master', thus avoiding this issue
> altogether whilst making it more obvious which patches belong together
> (eg. for easier cross-referencing with the ML).
Hi Pieter;
Sorry, I must have lost the thread somewhere. What application are you
thinking about, and what issue do you think that topic branches avoid?
d
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v2] Output unmodified Content-Type header value for JSON format.
2012-01-13 3:16 ` David Bremner
@ 2012-01-14 8:59 ` Pieter Praet
0 siblings, 0 replies; 24+ messages in thread
From: Pieter Praet @ 2012-01-14 8:59 UTC (permalink / raw)
To: David Bremner, notmuch
On Thu, 12 Jan 2012 23:16:29 -0400, David Bremner <david@tethera.net> wrote:
> On Thu, 12 Jan 2012 18:08:08 +0100, Pieter Praet <pieter@praet.org> wrote:
> > > > - The previous point is a bit of a counterargument to this, but in
>
> > > I couls separate it. I made is a single patch to avoid having a
> > > revision with broken emacs UI (and tests).
> > >
>
> > I'd like to propose to always apply patch series on a *topic* branch
> > which would then be merged back into 'master', thus avoiding this issue
> > altogether whilst making it more obvious which patches belong together
> > (eg. for easier cross-referencing with the ML).
>
> Hi Pieter;
>
> Sorry, I must have lost the thread somewhere. What application are you
> thinking about, and what issue do you think that topic branches avoid?
>
No specific application;
In the case of patch series, it may occur that an individual commit
leaves the code in a broken state. One might argue that devs need to
ensure that the project builds cleanly at every commit, but this can be
prohibitively time-consuming or even simply impossible due to the need
to keep changes atomic and area-specific (eg. changes to the binary and
the Emacs UI should be separate commits irrespective of their relation).
So it would make sense to consistently apply (certain, or perhaps all?)
patch series on an ad-hoc branch which would then be merged back into
'master', instead of being directly applied on 'master'.
Aside from preventing broken builds (as was Dmitry's reason to consolidate
a series touching on multiple distinct "spheres" of the code into a single
patch), series of related commits would also be clearly delineated/clustered,
facilitating source archaeology. (and all without making the commit log exude
the odor of entropy incarnate)
> d
>
Peace
--
Pieter
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-12 17:28 ` Austin Clements
@ 2012-01-14 9:19 ` Pieter Praet
2012-01-15 11:52 ` David Edmondson
0 siblings, 1 reply; 24+ messages in thread
From: Pieter Praet @ 2012-01-14 9:19 UTC (permalink / raw)
To: Austin Clements; +Cc: notmuch
On Thu, 12 Jan 2012 12:28:40 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Quoth Pieter Praet on Jan 12 at 6:07 pm:
> > On Tue, 22 Nov 2011 22:40:21 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> > > Quoth Jameson Graef Rollins on Nov 20 at 12:10 pm:
> > > > The open question seems to be how we handle the content encoding
> > > > parameters. My argument is that those should either be used by notmuch
> > > > to properly encode the content for the consumer. If that's not
> > > > possible, then just those parameters needed by the consumer to decode
> > > > the content should be output.
> > >
> > > If notmuch is going to include part content in the JSON output (which
> > > perhaps it shouldn't, as per recent IRC discussions), then it must
> > > handle content encodings because JSON must be Unicode and therefore
> > > the content strings in the JSON must be Unicode.
> >
> > Having missed the IRC discussions: what is the rationale for not
> > including (specific types of?) part content in the JSON output ?
> > Eg. how about inline attached text/x-patch ?
>
> Technically the IRC discussion was about not including *any* part
> content in the JSON output, and always using show --format=raw or
> similar to retrieve desired parts. Currently, notmuch includes part
> content in the JSON only for text/*, *except* when it's text/html. I
> assume non-text parts are omitted because binary data is hard to
> represent in JSON and text/html is omitted because some people don't
> need it. However, this leads to some peculiar asymmetry in the Emacs
> code where sometimes it pulls part content out of the JSON and
> sometimes it retrieves it using show --format=raw. This in turn leads
> to asymmetry in content encoding handling, since notmuch handles
> content encoding for parts included in the JSON (and there's no good
> way around that since JSON is Unicode), but not for parts retrieved as
> raw.
>
> The idea discussed on IRC was to remove all part content from the JSON
> output and to always use show to retrieve it, possibly beefing up
> show's support for content decoding (and possibly introducing a way to
> retrieve multiple raw parts at once to avoid re-parsing). This would
> get the JSON format out of the business of guessing what consumers
> need, simplify the Emacs code, and normalize content encoding
> handling.
Full ACK.
One concern though (IIUC): Due to the prevalence of retarded MUA's, not
outputting 'text/plain' and/or 'text/html' parts is unfortunately all
too often equivalent to not outputting anything at all, so wouldn't we,
in essence, be reducing `show --format=json' to an ever-so-slightly
augmented `search --format=json' ?
Peace
--
Pieter
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-14 9:19 ` Pieter Praet
@ 2012-01-15 11:52 ` David Edmondson
2012-01-15 17:58 ` Austin Clements
0 siblings, 1 reply; 24+ messages in thread
From: David Edmondson @ 2012-01-15 11:52 UTC (permalink / raw)
To: Pieter Praet, Austin Clements; +Cc: notmuch
[-- Attachment #1: Type: text/plain, Size: 1680 bytes --]
> Technically the IRC discussion was about not including *any* part
> content in the JSON output, and always using show --format=raw or
> similar to retrieve desired parts. Currently, notmuch includes part
> content in the JSON only for text/*, *except* when it's text/html. I
> assume non-text parts are omitted because binary data is hard to
> represent in JSON and text/html is omitted because some people don't
> need it. However, this leads to some peculiar asymmetry in the Emacs
> code where sometimes it pulls part content out of the JSON and
> sometimes it retrieves it using show --format=raw. This in turn leads
> to asymmetry in content encoding handling, since notmuch handles
> content encoding for parts included in the JSON (and there's no good
> way around that since JSON is Unicode), but not for parts retrieved as
> raw.
Including the text output in the JSON results in significantly fewer
calls to 'notmuch' during the building of a typical `notmuch-show-mode'
buffer. Someone with one of those older, crankier computers could easily
test how much effect this has by changing
`notmuch-show-get-bodypart-content' slightly.
> The idea discussed on IRC was to remove all part content from the JSON
> output and to always use show to retrieve it, possibly beefing up
> show's support for content decoding (and possibly introducing a way to
> retrieve multiple raw parts at once to avoid re-parsing). This would
> get the JSON format out of the business of guessing what consumers
> need, simplify the Emacs code, and normalize content encoding
> handling.
Is there a real problem being solved here? Having a clean structure is
nice, except when it's not.
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-15 11:52 ` David Edmondson
@ 2012-01-15 17:58 ` Austin Clements
2012-01-16 8:49 ` David Edmondson
0 siblings, 1 reply; 24+ messages in thread
From: Austin Clements @ 2012-01-15 17:58 UTC (permalink / raw)
To: David Edmondson, Pieter Praet; +Cc: notmuch
On Sun, 15 Jan 2012 11:52:40 +0000, David Edmondson <dme@dme.org> wrote:
> > Technically the IRC discussion was about not including *any* part
> > content in the JSON output, and always using show --format=raw or
> > similar to retrieve desired parts. Currently, notmuch includes part
> > content in the JSON only for text/*, *except* when it's text/html. I
> > assume non-text parts are omitted because binary data is hard to
> > represent in JSON and text/html is omitted because some people don't
> > need it. However, this leads to some peculiar asymmetry in the Emacs
> > code where sometimes it pulls part content out of the JSON and
> > sometimes it retrieves it using show --format=raw. This in turn leads
> > to asymmetry in content encoding handling, since notmuch handles
> > content encoding for parts included in the JSON (and there's no good
> > way around that since JSON is Unicode), but not for parts retrieved as
> > raw.
>
> Including the text output in the JSON results in significantly fewer
> calls to 'notmuch' during the building of a typical `notmuch-show-mode'
> buffer. Someone with one of those older, crankier computers could easily
> test how much effect this has by changing
> `notmuch-show-get-bodypart-content' slightly.
Yes. I was mostly reiterating the IRC discussion for Pieter. Since
this discussion, I've stabilized on the pre-fetching notion I described
in id:"20120115003617.GH1801@mit.edu", though I do think we should make
this clear in the code: that the rule for whether the JSON includes a
"content" key for a leaf part is internal to the CLI and that consumers
should be prepared to use it if it's there and to retrieve the content
separately if it's not. This is exactly how the Emacs code happens to
work, it just hasn't been codified anywhere. Looking at it this way
gives us more flexibility than the current code takes advantage of; for
example we could omit content from the JSON if it's over some size
threshold since the cost of sending that to a client that doesn't need
it is high while the cost of having the client retrieve it for itself is
relatively low.
> > The idea discussed on IRC was to remove all part content from the JSON
> > output and to always use show to retrieve it, possibly beefing up
> > show's support for content decoding (and possibly introducing a way to
> > retrieve multiple raw parts at once to avoid re-parsing). This would
> > get the JSON format out of the business of guessing what consumers
> > need, simplify the Emacs code, and normalize content encoding
> > handling.
>
> Is there a real problem being solved here? Having a clean structure is
> nice, except when it's not.
The "real" problem is the asymmetry in encoding handling that started
this discussion. Content included in the JSON is re-encoded by the CLI,
while content retrieved via raw needs to be re-encoded by the client.
OTOH, I don't understand the encoding story for HTML, since the encoding
can come from either a header or from the body of the HTML. Does this
make it strictly necessary for the client to handle the encoding?
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-15 17:58 ` Austin Clements
@ 2012-01-16 8:49 ` David Edmondson
2012-01-16 13:18 ` Tomi Ollila
0 siblings, 1 reply; 24+ messages in thread
From: David Edmondson @ 2012-01-16 8:49 UTC (permalink / raw)
To: Austin Clements, Pieter Praet; +Cc: notmuch
[-- Attachment #1: Type: text/plain, Size: 1912 bytes --]
On Sun, 15 Jan 2012 12:58:40 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
> Yes. I was mostly reiterating the IRC discussion for Pieter. Since
> this discussion, I've stabilized on the pre-fetching notion I described
> in id:"20120115003617.GH1801@mit.edu",
Will read when I get there.
> though I do think we should make this clear in the code: that the rule
> for whether the JSON includes a "content" key for a leaf part is
> internal to the CLI and that consumers should be prepared to use it if
> it's there and to retrieve the content separately if it's not. This
> is exactly how the Emacs code happens to work, it just hasn't been
> codified anywhere.
It's a bit more than 'happens to work' :-)
> Looking at it this way gives us more flexibility than the current code
> takes advantage of; for example we could omit content from the JSON if
> it's over some size threshold since the cost of sending that to a
> client that doesn't need it is high while the cost of having the
> client retrieve it for itself is relatively low.
This, I suspect, brings us back to what may have been Dmitry's original
concern. If we codify the current behaviour then we're actually
_forcing_ clients to use inline content if it's present, because
otherwise they have no way to discover the charset/encoding used for the
raw part.
That seems as though it could be a problem for some clients.
> OTOH, I don't understand the encoding story for HTML, since the encoding
> can come from either a header or from the body of the HTML. Does this
> make it strictly necessary for the client to handle the encoding?
If it can be specified by the content of a part rather than the part
headers, then I think that the client will have to be prepared to handle
it.
Even if not, it might still be more effective to choose that approach -
it would remove the need to add arbitrary encoding support to the CLI
application.
[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] Output unmodified Content-Type header value for JSON format.
2012-01-16 8:49 ` David Edmondson
@ 2012-01-16 13:18 ` Tomi Ollila
0 siblings, 0 replies; 24+ messages in thread
From: Tomi Ollila @ 2012-01-16 13:18 UTC (permalink / raw)
To: David Edmondson, Austin Clements, Pieter Praet; +Cc: notmuch
On Mon, 16 Jan 2012 08:49:03 +0000, David Edmondson <dme@dme.org> wrote:
> On Sun, 15 Jan 2012 12:58:40 -0500, Austin Clements <amdragon@MIT.EDU> wrote:
>
> This, I suspect, brings us back to what may have been Dmitry's original
> concern. If we codify the current behaviour then we're actually
> _forcing_ clients to use inline content if it's present, because
> otherwise they have no way to discover the charset/encoding used for the
> raw part.
>
> That seems as though it could be a problem for some clients.
>
> > OTOH, I don't understand the encoding story for HTML, since the encoding
> > can come from either a header or from the body of the HTML. Does this
> > make it strictly necessary for the client to handle the encoding?
Either from a header or from the body of the *HTTP*. Html meta tag is part
of the header of the HTML document, body of the HTTP message.
> If it can be specified by the content of a part rather than the part
> headers, then I think that the client will have to be prepared to handle
> it.
>
> Even if not, it might still be more effective to choose that approach -
> it would remove the need to add arbitrary encoding support to the CLI
> application.
In case of w3m interface if charset is not told it defaults
to iso-8859-1 as both input and output encoding.
That is no problem in w3m -- it just doesn't do any conversion.
But emacs thinks that it gets input in iso-8859-1 format and will
attempt to convert that to whatever charset the window user has
is using.
If input was utf8 but emacs thinks input was latin1 then we get
this infamous 'double-utf8'ied output (a subset of wtf-8 charset ;)
(in case window charset is utf8)
In case of w3m the content feed to w3m could be pre-encoded to utf-8
and w3m interface is told that charset is utf-8 -- w3m will now
called using utf-8 as input and output encoding -- and at the end
emacs does conversion from utf-8 to the window encoding (if needed).
As mentioned in IRC: 2012-01-10 11:46 (UTC) xxxXX indeed, the headers
should take precedence to meta tag, as defined in HTML4
So, complying clients takes the precedence from command line options
(which, in case of command line clients makes perfect sense).
Tomi
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2012-01-16 13:18 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-11-18 23:45 [PATCH] Output unmodified Content-Type header value for JSON format Dmitry Kurochkin
2011-11-19 1:58 ` Jameson Graef Rollins
2011-11-19 2:42 ` Dmitry Kurochkin
2011-11-19 4:59 ` Austin Clements
2011-11-19 5:26 ` Dmitry Kurochkin
2011-11-19 18:58 ` Austin Clements
2011-11-20 18:52 ` Dmitry Kurochkin
2011-11-19 10:49 ` Jameson Graef Rollins
2011-11-20 18:32 ` Dmitry Kurochkin
2011-11-20 20:10 ` Jameson Graef Rollins
2011-11-23 3:40 ` Austin Clements
2012-01-12 17:07 ` Pieter Praet
2012-01-12 17:28 ` Austin Clements
2012-01-14 9:19 ` Pieter Praet
2012-01-15 11:52 ` David Edmondson
2012-01-15 17:58 ` Austin Clements
2012-01-16 8:49 ` David Edmondson
2012-01-16 13:18 ` Tomi Ollila
2011-11-19 4:18 ` [PATCH v2] " Dmitry Kurochkin
2011-11-19 12:59 ` David Bremner
2011-11-20 18:35 ` Dmitry Kurochkin
2012-01-12 17:08 ` Pieter Praet
2012-01-13 3:16 ` David Bremner
2012-01-14 8:59 ` Pieter Praet
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).