JSON readtable error when replying

JSON readtable error when replying

[Michal Sojka]

Hi list,

when I try to reply within emacs to a particular message, I get "JSON
readtable error". This happens with with the current git master as well
as with a version from about a month ago. Is this a known problem or
shall I dig into it and find what wrong?

Thanks,
-Michal

Re: JSON readtable error when replying

[Adam Wolfe Gordon]

Hi Michal,

On Mon, Apr 30, 2012 at 09:00, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> when I try to reply within emacs to a particular message, I get "JSON
> readtable error". This happens with with the current git master as well
> as with a version from about a month ago. Is this a known problem or
> shall I dig into it and find what wrong?

Sounds like notmuch reply is producing bad JSON for that particular
message. Could you try the following, replacing <message id> with the
message ID of the message you're replying to:

notmuch reply --format=json id:"<message id>" | json_xs -t json-pretty

and see if it complains? The json_xs tool is in the libjson-xs-perl
package in Ubuntu. If you can't get it, run just the notmuch reply
command, and see what it does - the JSON will be hard to read, but if
it's producing an error of some sort you'll see it.

-- Adam

Re: JSON readtable error when replying

[James Vasile]

[-- Attachment #1: Type: text/plain, Size: 612 bytes --]

It's a JSON parsing error.  I don't know if it's a known error, but it's
not one I've seen chatter about on this list.  I've gotten it too and
haven't really looked into it.  I suspect it is a bug in the Emacs JSON
parsing function. but that I can only support that suspicion with
hunches and slander.  

Just to round out my "me too" report: I sometimes get that error on
opening a thread from the search view.  It seems to be intermittent in
that a thread will open just fine and then I'll select it again and I'll
get the error.  But that observation might be faulty memory.

Best regards,
James

[-- Attachment #2: Type: application/pgp-signature, Size: 489 bytes --]

Re: JSON readtable error when replying

[Michal Sojka]

Hi Adam,

Adam Wolfe Gordon <awg+notmuch@xvx.ca> writes:
> On Mon, Apr 30, 2012 at 09:00, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
>> when I try to reply within emacs to a particular message, I get "JSON
>> readtable error". This happens with with the current git master as well
>> as with a version from about a month ago. Is this a known problem or
>> shall I dig into it and find what wrong?
>
> Sounds like notmuch reply is producing bad JSON for that particular
> message. Could you try the following, replacing <message id> with the
> message ID of the message you're replying to:
>
> notmuch reply --format=json id:"<message id>" | json_xs -t json-pretty
>
> and see if it complains? 

json_xs didn't complain, but I've found that notmuch outputs
  Failed to verify signed part: Cannot verify multipart/signed part: unsupported signature protocol 'application/x-pkcs7-signature'.
to stderr before the json output and emacs tries to parse
stderr. Emacs then complains about not finding "F" in json-readtable.

The following patch fixes the problem for me, but I do not know if it is
a good thing to throw away the stderr output of notmuch.

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..49db603 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -113,7 +113,7 @@ list."
 
     ;; Get the reply object as JSON, and parse it into an elisp object.
     (with-temp-buffer
-      (apply 'call-process (append (list notmuch-command nil (list t t) nil) args))
+      (apply 'call-process (append (list notmuch-command nil (list t nil) nil) args))
       (goto-char (point-min))
       (let ((json-object-type 'plist)
            (json-array-type 'list)

Comments?

-Michal

Re: JSON readtable error when replying

[Adam Wolfe Gordon]

On Mon, Apr 30, 2012 at 10:50, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> json_xs didn't complain, but I've found that notmuch outputs
>  Failed to verify signed part: Cannot verify multipart/signed part: unsupported signature protocol 'application/x-pkcs7-signature'.
> to stderr before the json output and emacs tries to parse
> stderr. Emacs then complains about not finding "F" in json-readtable.
>
> The following patch fixes the problem for me, but I do not know if it is
> a good thing to throw away the stderr output of notmuch.
>
> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
> index 87bd88d..49db603 100644
> --- a/emacs/notmuch-mua.el
> +++ b/emacs/notmuch-mua.el
> @@ -113,7 +113,7 @@ list."
>
>     ;; Get the reply object as JSON, and parse it into an elisp object.
>     (with-temp-buffer
> -      (apply 'call-process (append (list notmuch-command nil (list t t) nil) args))
> +      (apply 'call-process (append (list notmuch-command nil (list t nil) nil) args))
>       (goto-char (point-min))
>       (let ((json-object-type 'plist)
>            (json-array-type 'list)
>
> Comments?

Good catch. I'm not sure what (if anything) we want to do with the
stderr output, but putting it in the buffer with the JSON is
definitely not the right thing. Your fix looks fine to me - would you
mind doing a commit and sending a patch to the list?

David, it would probably be good to get this (pretty trivial) fix in
before 0.13, since it will include the JSON reply stuff.

-- Adam

[PATCH] emacs: Do not pass stderr of notmuch reply to JSON parser

[Michal Sojka]

Sometimes, notmuch reply outputs something to stderr, for example:
"Failed to verify signed part: Cannot verify multipart/signed part:
unsupported signature protocol". When this happens, replying in emacs
fails, because emacs cannot parse the error message as JSON.

This patch causes emacs to ignore stderr when reading reply from
notmuch.
---
 emacs/notmuch-mua.el |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 87bd88d..49db603 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -113,7 +113,7 @@ list."
 
     ;; Get the reply object as JSON, and parse it into an elisp object.
     (with-temp-buffer
-      (apply 'call-process (append (list notmuch-command nil (list t t) nil) args))
+      (apply 'call-process (append (list notmuch-command nil (list t nil) nil) args))
       (goto-char (point-min))
       (let ((json-object-type 'plist)
 	    (json-array-type 'list)
-- 
1.7.10

Re: [PATCH] emacs: Do not pass stderr of notmuch reply to JSON parser

[Adam Wolfe Gordon]

On Tue, May 1, 2012 at 3:10 PM, Michal Sojka <sojkam1@fel.cvut.cz> wrote:
> Sometimes, notmuch reply outputs something to stderr, for example:
> "Failed to verify signed part: Cannot verify multipart/signed part:
> unsupported signature protocol". When this happens, replying in emacs
> fails, because emacs cannot parse the error message as JSON.
>
> This patch causes emacs to ignore stderr when reading reply from
> notmuch.

LGTM, as I said in the other thread. Thanks for the fix.

-- Adam

Re: [PATCH] emacs: Do not pass stderr of notmuch reply to JSON parser

[David Bremner]

Michal Sojka <sojkam1@fel.cvut.cz> writes:

> Sometimes, notmuch reply outputs something to stderr, for example:
> "Failed to verify signed part: Cannot verify multipart/signed part:
> unsupported signature protocol". When this happens, replying in emacs
> fails, because emacs cannot parse the error message as JSON.

pushed.

d