From: Mark Walters <markwalters1009@gmail.com>
To: Jani Nikula <jani@nikula.org>, Austin Clements <amdragon@MIT.EDU>,
notmuch@notmuchmail.org
Subject: Re: [PATCH 07/11] emacs: Use notmuch tag --batch for large tag queries
Date: Wed, 09 Oct 2013 08:38:36 +0100 [thread overview]
Message-ID: <87d2ne28sj.fsf@qmul.ac.uk> (raw)
In-Reply-To: <87iox6ap58.fsf@nikula.org>
On Wed, 09 Oct 2013, Jani Nikula <jani@nikula.org> wrote:
> On Tue, 08 Oct 2013, Austin Clements <amdragon@MIT.EDU> wrote:
>> (Unfortunately, it's difficult to first demonstrate this problem with
>> a known-broken test because modern Linux kernels have argument length
>> limits in the megabytes, which makes Emacs really slow!)
>> ---
>> emacs/notmuch-lib.el | 8 ++++++++
>> emacs/notmuch-tag.el | 12 ++++++++++--
>> test/emacs | 8 ++++++++
>> 3 files changed, 26 insertions(+), 2 deletions(-)
>>
>> diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el
>> index 22156f1..348112b 100644
>> --- a/emacs/notmuch-lib.el
>> +++ b/emacs/notmuch-lib.el
>> @@ -261,6 +261,14 @@ user-friendly queries."
>> "Return a query that matches the message with id ID."
>> (concat "id:" (notmuch-escape-boolean-term id)))
>>
>> +(defun notmuch-hex-encode (str)
>> + "Hex-encode STR (e.g., as used by batch tagging).
>> +
>> +This replaces spaces, percents, and double quotes in STR with
>> +%NN where NN is the hexadecimal value of the character."
>> + (replace-regexp-in-string
>> + "[ %\"]" (lambda (match) (format "%%%02x" (aref match 0))) str))
>> +
>> ;;
>>
>> (defun notmuch-common-do-stash (text)
>> diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
>> index 064cfa8..a4eec14 100644
>> --- a/emacs/notmuch-tag.el
>> +++ b/emacs/notmuch-tag.el
>> @@ -242,6 +242,8 @@ from TAGS if present."
>> (error "Changed tag must be of the form `+this_tag' or `-that_tag'")))))
>> (sort result-tags 'string<)))
>>
>> +(defconst notmuch-tag-argument-limit 1000)
>> +
>> (defun notmuch-tag (query &optional tag-changes)
>> "Add/remove tags in TAG-CHANGES to messages matching QUERY.
>>
>> @@ -268,8 +270,14 @@ notmuch-after-tag-hook will be run."
>> tag-changes)
>> (unless (null tag-changes)
>> (run-hooks 'notmuch-before-tag-hook)
>> - (apply 'notmuch-call-notmuch-process "tag"
>> - (append tag-changes (list "--" query)))
>> + (if (<= (length query) notmuch-tag-argument-limit)
>> + (apply 'notmuch-call-notmuch-process "tag"
>> + (append tag-changes (list "--" query)))
>> + ;; Use batch tag mode to avoid argument length limitations
>> + (let ((batch-op (concat (mapconcat #'notmuch-hex-encode tag-changes " ")
>> + " -- " query)))
>> + (message "Batch tagging with %s" batch-op)
>
> Why the message?
>
> Jani.
I had missed that: this message is presumably for debugging as it
includes the (possibly megabyte sized) query. I think some message would
be sensible as this could be a slow operation so it's probably worth
telling the user why the interface has locked.
Best wishes
Mark
>
>> + (notmuch-call-notmuch-process :stdin-string batch-op "tag" "--batch")))
>> (run-hooks 'notmuch-after-tag-hook))
>> ;; in all cases we return tag-changes as a list
>> tag-changes)
>> diff --git a/test/emacs b/test/emacs
>> index 05295af..2917189 100755
>> --- a/test/emacs
>> +++ b/test/emacs
>> @@ -122,6 +122,14 @@ test_emacs "(notmuch-search \"$os_x_darwin_thread\")
>> output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
>> test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)"
>>
>> +test_begin_subtest "Add tag (large query)"
>> +# We use a long query to force us into batch mode and use a funny tag
>> +# that requires escaping for batch tagging.
>> +test_emacs "(notmuch-tag (concat \"$os_x_darwin_thread\" \" or \" (make-string notmuch-tag-argument-limit ?x)) \"+tag-from-%-large-query\")"
>> +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize)
>> +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-%-large-query unread)"
>> +notmuch tag -tag-from-%-large-query $os_x_darwin_thread
>> +
>> test_begin_subtest "notmuch-show: add single tag to single message"
>> test_emacs "(notmuch-show \"$os_x_darwin_thread\")
>> (execute-kbd-macro \"+tag-from-show-view\")"
>> --
>> 1.8.4.rc3
>>
>> _______________________________________________
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> http://notmuchmail.org/mailman/listinfo/notmuch
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
next prev parent reply other threads:[~2013-10-09 7:38 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-10-07 22:33 [PATCH 00/11] Fix search tagging races Austin Clements
2013-10-07 22:33 ` [PATCH 01/11] schemata: Disambiguate non-terminal names Austin Clements
2013-10-07 22:33 ` [PATCH 02/11] cli: Separate current and deprecated format version Austin Clements
2013-10-08 6:48 ` Mark Walters
2013-10-09 14:08 ` Austin Clements
2013-10-07 22:33 ` [PATCH 03/11] lib: Document extent of some return values Austin Clements
2013-10-07 22:33 ` [PATCH 04/11] test: Fix missing erase-buffer in emacs test Austin Clements
2013-10-07 22:33 ` [PATCH 05/11] emacs: Move `notmuch-call-notmuch-process' to notmuch-lib Austin Clements
2013-10-07 22:33 ` [PATCH 06/11] emacs: Support passing input via `notmuch-call-notmuch-*' Austin Clements
2013-10-07 22:33 ` [PATCH 07/11] emacs: Use notmuch tag --batch for large tag queries Austin Clements
2013-10-08 7:27 ` Mark Walters
2013-10-09 14:11 ` Austin Clements
2013-10-09 7:18 ` Jani Nikula
2013-10-09 7:38 ` Mark Walters [this message]
2013-10-09 14:14 ` Austin Clements
2013-10-07 22:33 ` [PATCH 08/11] search: Add stable queries to thread search results Austin Clements
2013-10-08 16:37 ` Mark Walters
2013-10-09 7:41 ` Jani Nikula
2013-10-09 14:36 ` Austin Clements
2013-10-09 18:25 ` Jani Nikula
2013-10-07 22:33 ` [PATCH 09/11] Add TODO about more efficient stable thread queries Austin Clements
2013-10-07 22:33 ` [PATCH 10/11] emacs: Add known-broken tests for search tagging races Austin Clements
2013-10-08 16:47 ` Mark Walters
2013-10-07 22:33 ` [PATCH 11/11] emacs: Fix " Austin Clements
2013-10-08 7:56 ` [PATCH 00/11] " Mark Walters
2013-10-09 16:19 ` Austin Clements
2013-10-09 7:43 ` Mark Walters
2013-10-09 16:11 ` Austin Clements
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://notmuchmail.org/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87d2ne28sj.fsf@qmul.ac.uk \
--to=markwalters1009@gmail.com \
--cc=amdragon@MIT.EDU \
--cc=jani@nikula.org \
--cc=notmuch@notmuchmail.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhetil.org/notmuch.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).