* [PATCH v4 0/3] split async json parser into utility function
@ 2012-10-27 9:34 Mark Walters
2012-10-27 9:34 ` [PATCH v4 1/3] emacs: Split out the incremental json parser into its own function Mark Walters
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Mark Walters @ 2012-10-27 9:34 UTC (permalink / raw)
To: notmuch
This is version 4 of this patch set (v3 is at
id:"1351037602-11157-1-git-send-email-markwalters1009@gmail.com"). It
splits the async json parser into its own function and moves it to
lib.
This version is very similar to v3 except that it finished by moving
the utility function to lib rather than leaving it notmuch.el. It also
improves some of the docstrings to clarify how the function and
variables work. I have not gone quite as far as suggested by Ethan as
these variables are purely for internal use and Austin's code is well
documented.
Best wishes
Mark
Mark Walters (3):
emacs: Split out the incremental json parser into its own function
emacs: Rename incremental JSON internal variables
emacs: Move the incremental JSON parser to notmuch-lib.el
emacs/notmuch-lib.el | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++
emacs/notmuch.el | 48 +++---------------------------------
2 files changed, 70 insertions(+), 44 deletions(-)
--
1.7.9.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 1/3] emacs: Split out the incremental json parser into its own function
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
@ 2012-10-27 9:34 ` Mark Walters
2012-10-27 9:34 ` [PATCH v4 2/3] emacs: Rename incremental JSON internal variables Mark Walters
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Mark Walters @ 2012-10-27 9:34 UTC (permalink / raw)
To: notmuch
This patch splits out the incremental json parser into its own
function.
It moves the main logic of the parser to happen inside the parse
buffer rather than inside the results buffer, but makes sure all
results and all errors are displayed in the results buffer.
It also changes the local parser variables from being buffer
local to the results buffer to being buffer local to the parse buffer,
and sets them up automatically so the caller does not need to.
Finally to keep the diff small this patch does not fix the whitespace,
nor complete the code movement (these are done in subsequent patches)
but it should contain all the functional changes.
---
emacs/notmuch.el | 39 +++++++++++++++++++++++++++------------
1 files changed, 27 insertions(+), 12 deletions(-)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index a8a85ce..799c621 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -838,8 +838,28 @@ non-authors is found, assume that all of the authors match."
;; Insert new data
(save-excursion
(goto-char (point-max))
- (insert string)))
- (with-current-buffer results-buf
+ (insert string))
+ (notmuch-json-parse-partial-list 'notmuch-search-show-result
+ 'notmuch-search-show-error
+ results-buf)))))
+
+(defun notmuch-json-parse-partial-list (result-function error-function results-buf)
+ "Parse a partial JSON list from current buffer.
+
+This function consumes a JSON list from the current buffer,
+applying RESULT-FUNCTION in buffer RESULT-BUFFER to each complete
+value in the list. It operates incrementally and should be
+called whenever the buffer has been extended with additional
+data.
+
+If there is a syntax error, this will attempt to resynchronize
+with the input and will apply ERROR-FUNCTION in buffer
+RESULT-BUFFER to any input that was skipped."
+ (let (done)
+ (unless (local-variable-p 'notmuch-search-json-parser)
+ (set (make-local-variable 'notmuch-search-json-parser)
+ (notmuch-json-create-parser (current-buffer)))
+ (set (make-local-variable 'notmuch-search-process-state) 'begin))
(while (not done)
(condition-case nil
(case notmuch-search-process-state
@@ -855,7 +875,8 @@ non-authors is found, assume that all of the authors match."
(case result
((retry) (setq done t))
((end) (setq notmuch-search-process-state 'end))
- (otherwise (notmuch-search-show-result result)))))
+ (otherwise (with-current-buffer results-buf
+ (funcall result-function result))))))
((end)
;; Any trailing data is unexpected
(notmuch-json-eof notmuch-search-json-parser)
@@ -863,16 +884,13 @@ non-authors is found, assume that all of the authors match."
(json-error
;; Do our best to resynchronize and ensure forward
;; progress
- (notmuch-search-show-error
- "%s"
- (with-current-buffer parse-buf
(let ((bad (buffer-substring (line-beginning-position)
(line-end-position))))
(forward-line)
- bad))))))
+ (with-current-buffer results-buf
+ (funcall error-function "%s" bad))))))
;; Clear out what we've parsed
- (with-current-buffer parse-buf
- (delete-region (point-min) (point)))))))
+ (delete-region (point-min) (point))))
(defun notmuch-search-tag-all (&optional tag-changes)
"Add/remove tags from all messages in current search buffer.
@@ -984,9 +1002,6 @@ Other optional parameters are used as follows:
;; This buffer will be killed by the sentinel, which
;; should be called no matter how the process dies.
(parse-buf (generate-new-buffer " *notmuch search parse*")))
- (set (make-local-variable 'notmuch-search-process-state) 'begin)
- (set (make-local-variable 'notmuch-search-json-parser)
- (notmuch-json-create-parser parse-buf))
(process-put proc 'parse-buf parse-buf)
(set-process-sentinel proc 'notmuch-search-process-sentinel)
(set-process-filter proc 'notmuch-search-process-filter)
--
1.7.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 2/3] emacs: Rename incremental JSON internal variables
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
2012-10-27 9:34 ` [PATCH v4 1/3] emacs: Split out the incremental json parser into its own function Mark Walters
@ 2012-10-27 9:34 ` Mark Walters
2012-10-27 9:34 ` [PATCH v4 3/3] emacs: Move the incremental JSON parser to notmuch-lib.el Mark Walters
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Mark Walters @ 2012-10-27 9:34 UTC (permalink / raw)
To: notmuch
This patch just renames the internal variables for the JSON parser now
it is no longer specific to search mode. It also fixes up the white
space after the previous patch. There should be no functional changes.
---
emacs/notmuch.el | 56 ++++++++++++++++++++++++++++++-----------------------
1 files changed, 32 insertions(+), 24 deletions(-)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 799c621..e3610d6 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -820,11 +820,15 @@ non-authors is found, assume that all of the authors match."
(insert (apply #'format string objects))
(insert "\n")))
-(defvar notmuch-search-process-state nil
- "Parsing state of the search process filter.")
+;; These two variables are internal variables to the parsing
+;; routines. They are always used buffer local but need to be declared
+;; globally to avoid compiler warnings.
-(defvar notmuch-search-json-parser nil
- "Incremental JSON parser for the search process filter.")
+(defvar notmuch-json-state nil
+ "Internal incremental JSON parser object: local to the buffer being parsed.")
+
+(defvar notmuch-json-parser nil
+ "State of the internal JSON parser: local to the buffer being parsed.")
(defun notmuch-search-process-filter (proc string)
"Process and filter the output of \"notmuch search\""
@@ -854,42 +858,46 @@ data.
If there is a syntax error, this will attempt to resynchronize
with the input and will apply ERROR-FUNCTION in buffer
-RESULT-BUFFER to any input that was skipped."
+RESULT-BUFFER to any input that was skipped.
+
+It sets up all the needed internal variables: the caller just
+needs to call it with point in the same place that the parser
+left it."
(let (done)
- (unless (local-variable-p 'notmuch-search-json-parser)
- (set (make-local-variable 'notmuch-search-json-parser)
+ (unless (local-variable-p 'notmuch-json-parser)
+ (set (make-local-variable 'notmuch-json-parser)
(notmuch-json-create-parser (current-buffer)))
- (set (make-local-variable 'notmuch-search-process-state) 'begin))
- (while (not done)
- (condition-case nil
- (case notmuch-search-process-state
+ (set (make-local-variable 'notmuch-json-state) 'begin))
+ (while (not done)
+ (condition-case nil
+ (case notmuch-json-state
((begin)
;; Enter the results list
(if (eq (notmuch-json-begin-compound
- notmuch-search-json-parser) 'retry)
+ notmuch-json-parser) 'retry)
(setq done t)
- (setq notmuch-search-process-state 'result)))
+ (setq notmuch-json-state 'result)))
((result)
;; Parse a result
- (let ((result (notmuch-json-read notmuch-search-json-parser)))
+ (let ((result (notmuch-json-read notmuch-json-parser)))
(case result
- ((retry) (setq done t))
- ((end) (setq notmuch-search-process-state 'end))
+ ((retry) (setq done t))
+ ((end) (setq notmuch-json-state 'end))
(otherwise (with-current-buffer results-buf
(funcall result-function result))))))
((end)
;; Any trailing data is unexpected
- (notmuch-json-eof notmuch-search-json-parser)
+ (notmuch-json-eof notmuch-json-parser)
(setq done t)))
- (json-error
- ;; Do our best to resynchronize and ensure forward
- ;; progress
- (let ((bad (buffer-substring (line-beginning-position)
- (line-end-position))))
- (forward-line)
+ (json-error
+ ;; Do our best to resynchronize and ensure forward
+ ;; progress
+ (let ((bad (buffer-substring (line-beginning-position)
+ (line-end-position))))
+ (forward-line)
(with-current-buffer results-buf
(funcall error-function "%s" bad))))))
- ;; Clear out what we've parsed
+ ;; Clear out what we've parsed
(delete-region (point-min) (point))))
(defun notmuch-search-tag-all (&optional tag-changes)
--
1.7.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 3/3] emacs: Move the incremental JSON parser to notmuch-lib.el
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
2012-10-27 9:34 ` [PATCH v4 1/3] emacs: Split out the incremental json parser into its own function Mark Walters
2012-10-27 9:34 ` [PATCH v4 2/3] emacs: Rename incremental JSON internal variables Mark Walters
@ 2012-10-27 9:34 ` Mark Walters
2012-10-27 12:12 ` [PATCH v4 0/3] split async json parser into utility function Tomi Ollila
2012-10-28 15:18 ` David Bremner
4 siblings, 0 replies; 6+ messages in thread
From: Mark Walters @ 2012-10-27 9:34 UTC (permalink / raw)
To: notmuch
This just moves the newly split out incremental json parser (together
with its state variables) to lib.
There should be no functional change.
---
emacs/notmuch-lib.el | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++
emacs/notmuch.el | 63 -----------------------------------------------
2 files changed, 66 insertions(+), 63 deletions(-)
diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
index 69867ad..8b6115b 100644
--- a/emacs/notmuch-lib.el
+++ b/emacs/notmuch-lib.el
@@ -341,6 +341,16 @@ was called."
;; Incremental JSON parsing
+;; These two variables are internal variables to the parsing
+;; routines. They are always used buffer local but need to be declared
+;; globally to avoid compiler warnings.
+
+(defvar notmuch-json-parser nil
+ "Internal incremental JSON parser object: local to the buffer being parsed.")
+
+(defvar notmuch-json-state nil
+ "State of the internal JSON parser: local to the buffer being parsed.")
+
(defun notmuch-json-create-parser (buffer)
"Return a streaming JSON parser that consumes input from BUFFER.
@@ -535,6 +545,62 @@ of the buffer if there is only trailing whitespace."
(unless (eobp)
(signal 'json-error (list "Trailing garbage following JSON data")))))
+(defun notmuch-json-parse-partial-list (result-function error-function results-buf)
+ "Parse a partial JSON list from current buffer.
+
+This function consumes a JSON list from the current buffer,
+applying RESULT-FUNCTION in buffer RESULT-BUFFER to each complete
+value in the list. It operates incrementally and should be
+called whenever the buffer has been extended with additional
+data.
+
+If there is a syntax error, this will attempt to resynchronize
+with the input and will apply ERROR-FUNCTION in buffer
+RESULT-BUFFER to any input that was skipped.
+
+It sets up all the needed internal variables: the caller just
+needs to call it with point in the same place that the parser
+left it."
+ (let (done)
+ (unless (local-variable-p 'notmuch-json-parser)
+ (set (make-local-variable 'notmuch-json-parser)
+ (notmuch-json-create-parser (current-buffer)))
+ (set (make-local-variable 'notmuch-json-state) 'begin))
+ (while (not done)
+ (condition-case nil
+ (case notmuch-json-state
+ ((begin)
+ ;; Enter the results list
+ (if (eq (notmuch-json-begin-compound
+ notmuch-json-parser) 'retry)
+ (setq done t)
+ (setq notmuch-json-state 'result)))
+ ((result)
+ ;; Parse a result
+ (let ((result (notmuch-json-read notmuch-json-parser)))
+ (case result
+ ((retry) (setq done t))
+ ((end) (setq notmuch-json-state 'end))
+ (otherwise (with-current-buffer results-buf
+ (funcall result-function result))))))
+ ((end)
+ ;; Any trailing data is unexpected
+ (notmuch-json-eof notmuch-json-parser)
+ (setq done t)))
+ (json-error
+ ;; Do our best to resynchronize and ensure forward
+ ;; progress
+ (let ((bad (buffer-substring (line-beginning-position)
+ (line-end-position))))
+ (forward-line)
+ (with-current-buffer results-buf
+ (funcall error-function "%s" bad))))))
+ ;; Clear out what we've parsed
+ (delete-region (point-min) (point))))
+
+
+
+
(provide 'notmuch-lib)
;; Local Variables:
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index e3610d6..e304dde 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -820,16 +820,6 @@ non-authors is found, assume that all of the authors match."
(insert (apply #'format string objects))
(insert "\n")))
-;; These two variables are internal variables to the parsing
-;; routines. They are always used buffer local but need to be declared
-;; globally to avoid compiler warnings.
-
-(defvar notmuch-json-state nil
- "Internal incremental JSON parser object: local to the buffer being parsed.")
-
-(defvar notmuch-json-parser nil
- "State of the internal JSON parser: local to the buffer being parsed.")
-
(defun notmuch-search-process-filter (proc string)
"Process and filter the output of \"notmuch search\""
(let ((results-buf (process-buffer proc))
@@ -847,59 +837,6 @@ non-authors is found, assume that all of the authors match."
'notmuch-search-show-error
results-buf)))))
-(defun notmuch-json-parse-partial-list (result-function error-function results-buf)
- "Parse a partial JSON list from current buffer.
-
-This function consumes a JSON list from the current buffer,
-applying RESULT-FUNCTION in buffer RESULT-BUFFER to each complete
-value in the list. It operates incrementally and should be
-called whenever the buffer has been extended with additional
-data.
-
-If there is a syntax error, this will attempt to resynchronize
-with the input and will apply ERROR-FUNCTION in buffer
-RESULT-BUFFER to any input that was skipped.
-
-It sets up all the needed internal variables: the caller just
-needs to call it with point in the same place that the parser
-left it."
- (let (done)
- (unless (local-variable-p 'notmuch-json-parser)
- (set (make-local-variable 'notmuch-json-parser)
- (notmuch-json-create-parser (current-buffer)))
- (set (make-local-variable 'notmuch-json-state) 'begin))
- (while (not done)
- (condition-case nil
- (case notmuch-json-state
- ((begin)
- ;; Enter the results list
- (if (eq (notmuch-json-begin-compound
- notmuch-json-parser) 'retry)
- (setq done t)
- (setq notmuch-json-state 'result)))
- ((result)
- ;; Parse a result
- (let ((result (notmuch-json-read notmuch-json-parser)))
- (case result
- ((retry) (setq done t))
- ((end) (setq notmuch-json-state 'end))
- (otherwise (with-current-buffer results-buf
- (funcall result-function result))))))
- ((end)
- ;; Any trailing data is unexpected
- (notmuch-json-eof notmuch-json-parser)
- (setq done t)))
- (json-error
- ;; Do our best to resynchronize and ensure forward
- ;; progress
- (let ((bad (buffer-substring (line-beginning-position)
- (line-end-position))))
- (forward-line)
- (with-current-buffer results-buf
- (funcall error-function "%s" bad))))))
- ;; Clear out what we've parsed
- (delete-region (point-min) (point))))
-
(defun notmuch-search-tag-all (&optional tag-changes)
"Add/remove tags from all messages in current search buffer.
--
1.7.9.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/3] split async json parser into utility function
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
` (2 preceding siblings ...)
2012-10-27 9:34 ` [PATCH v4 3/3] emacs: Move the incremental JSON parser to notmuch-lib.el Mark Walters
@ 2012-10-27 12:12 ` Tomi Ollila
2012-10-28 15:18 ` David Bremner
4 siblings, 0 replies; 6+ messages in thread
From: Tomi Ollila @ 2012-10-27 12:12 UTC (permalink / raw)
To: Mark Walters, notmuch
On Sat, Oct 27 2012, Mark Walters <markwalters1009@gmail.com> wrote:
> This is version 4 of this patch set (v3 is at
> id:"1351037602-11157-1-git-send-email-markwalters1009@gmail.com"). It
> splits the async json parser into its own function and moves it to
> lib.
>
> This version is very similar to v3 except that it finished by moving
> the utility function to lib rather than leaving it notmuch.el. It also
> improves some of the docstrings to clarify how the function and
> variables work. I have not gone quite as far as suggested by Ethan as
> these variables are purely for internal use and Austin's code is well
> documented.
Changes look good and tests pass.
Tomi
> Best wishes
>
> Mark
>
>
> Mark Walters (3):
> emacs: Split out the incremental json parser into its own function
> emacs: Rename incremental JSON internal variables
> emacs: Move the incremental JSON parser to notmuch-lib.el
>
> emacs/notmuch-lib.el | 66 ++++++++++++++++++++++++++++++++++++++++++++++++++
> emacs/notmuch.el | 48 +++---------------------------------
> 2 files changed, 70 insertions(+), 44 deletions(-)
>
> --
> 1.7.9.1
>
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v4 0/3] split async json parser into utility function
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
` (3 preceding siblings ...)
2012-10-27 12:12 ` [PATCH v4 0/3] split async json parser into utility function Tomi Ollila
@ 2012-10-28 15:18 ` David Bremner
4 siblings, 0 replies; 6+ messages in thread
From: David Bremner @ 2012-10-28 15:18 UTC (permalink / raw)
To: Mark Walters, notmuch
Mark Walters <markwalters1009@gmail.com> writes:
> This is version 4 of this patch set (v3 is at
> id:"1351037602-11157-1-git-send-email-markwalters1009@gmail.com"). It
> splits the async json parser into its own function and moves it to
> lib.
Series pushed,
d
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2012-10-28 15:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-10-27 9:34 [PATCH v4 0/3] split async json parser into utility function Mark Walters
2012-10-27 9:34 ` [PATCH v4 1/3] emacs: Split out the incremental json parser into its own function Mark Walters
2012-10-27 9:34 ` [PATCH v4 2/3] emacs: Rename incremental JSON internal variables Mark Walters
2012-10-27 9:34 ` [PATCH v4 3/3] emacs: Move the incremental JSON parser to notmuch-lib.el Mark Walters
2012-10-27 12:12 ` [PATCH v4 0/3] split async json parser into utility function Tomi Ollila
2012-10-28 15:18 ` David Bremner
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).