Bulk message tagging

Bulk message tagging

[Xavier Maillard]

Hi,

Is there an easy way to mark a whole bunch of message (restricted
in a region, result of a search, ...) ?

I can't find it.

Thank you

Xavier

Re: Bulk message tagging

[Jason White]

Xavier Maillard <xma@gnu.org> wrote:
> Is there an easy way to mark a whole bunch of message (restricted
> in a region, result of a search, ...) ?

notmuch tag +<tag>|-<tag> [...] [--] <search-terms> [...]
does this from the shell. If you want to do it from one of the user interfaces, it depends on which one you're using.

Re: Bulk message tagging

[Xavier Maillard]

On Sun, 4 Apr 2010 14:56:07 +1000, Jason White <jason@jasonjgw.net> wrote:
> Xavier Maillard <xma@gnu.org> wrote:
> > Is there an easy way to mark a whole bunch of message (restricted
> > in a region, result of a search, ...) ?
> 
> notmuch tag +<tag>|-<tag> [...] [--] <search-terms> [...]  does
> this from the shell. If you want to do it from one of the user
> interfaces, it depends on which one you're using.

You are right I forgot to mention I am using the GNU Emacs
interface exclusively.

Xavier

Re: Bulk message tagging

[Jason White]

Xavier Maillard <xma@gnu.org> wrote:
 
> You are right I forgot to mention I am using the GNU Emacs
> interface exclusively.

Then it's the * command from the buffer with the mail threads dislayed.
See also the ?h command for further help.

Re: Bulk message tagging

[Jesse Rosenthal]

On Sun, 04 Apr 2010 06:37:53 +0200, Xavier Maillard <xma@gnu.org> wrote:
> Is there an easy way to mark a whole bunch of message (restricted
> in a region, result of a search, ...) ?

In addition to the "*" command that was mentioned, there is a patch I
wrote to tag messages in search view by region in emacs. You can find it
here:

id:87sk90ragj.fsf@jhu.edu

It has also been integrated into dme's tree, if you are using that.

Best,
Jesse

Re: Bulk message tagging

[Xavier Maillard]

On Sun, 04 Apr 2010 07:38:03 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> On Sun, 04 Apr 2010 06:37:53 +0200, Xavier Maillard <xma@gnu.org> wrote:
> > Is there an easy way to mark a whole bunch of message (restricted
> > in a region, result of a search, ...) ?
> 
> In addition to the "*" command that was mentioned, there is a patch I
> wrote to tag messages in search view by region in emacs. You can find it

Sounds interesting. * is good when your search criteria is
perfect but you proposed patch is a good companion for all the
rest.

Sadly, git is not really something I know wll enough to play with
all this stuff :(

Xavier

Re: Bulk message tagging

[Xavier Maillard]

On Sun, 4 Apr 2010 15:46:40 +1000, Jason White <jason@jasonjgw.net> wrote:
> Xavier Maillard <xma@gnu.org> wrote:
>  
> > You are right I forgot to mention I am using the GNU Emacs
> > interface exclusively.
> 
> Then it's the * command from the buffer with the mail threads dislayed.
> See also the ?h command for further help.

Thank you. It worked perfectly though it is not mentionned that
you have to do the search *first* and press the '*' then after. I
would have thought to do a "all-in-one-operation" -i.e press the
* key, enter a search criteria and enter a list of tags -.

Regards,

Xavier

Re: Bulk message tagging

[Jason White]

Xavier Maillard <xma@gnu.org> wrote:
 
> Sadly, git is not really something I know wll enough to play with
> all this stuff :(

http://progit.org/book/

Re: Bulk message tagging

[Mark Anderson]

On Mon, 5 Apr 2010 01:15:39 -0500, Xavier Maillard <xma@gnu.org> wrote:
> On Sun, 04 Apr 2010 07:38:03 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> > On Sun, 04 Apr 2010 06:37:53 +0200, Xavier Maillard <xma@gnu.org> wrote:
> > > Is there an easy way to mark a whole bunch of message (restricted
> > > in a region, result of a search, ...) ?
> > 
> > In addition to the "*" command that was mentioned, there is a patch I
> > wrote to tag messages in search view by region in emacs. You can find it
> 
> Sounds interesting. * is good when your search criteria is
> perfect but you proposed patch is a good companion for all the
> rest.
> 
> Sadly, git is not really something I know wll enough to play with
> all this stuff :(
> 
> Xavier

Also, you will want to be careful with '*' in emacs.

It runs notmuch tag +<tag> <search terms from buffer>,
but the notmuch-search buffer can be woefully out of date.

I like to go through and add tags, archive, etc, lots of email.  Then I
'q' back to the search buffer, and I may realize that I want to tag
everything I can see.  '*' isn't really that command, it applies a tag
to the results of a search that you are looking at.  But it will use a
fresh version of the search, not the version in the buffer, which could
be minutes, hours, days or weeks old, depending on your Emacs buffer
management habits (or lack of them).

I think that '*' is definitely an awesome command, but I wonder if we
shouldn't have another command for the notmuch-search buffer which means
'tag all the threads that I can see in this buffer'.

If you have a crontab running 'notmuch new', you could end up tagging a
lot of things you would rather have a chance to review first.  I often
use '*' to tag:deleted a large number of mails once I have a query I
like, so deleting things sight unseen is an unpleasant thought.

I know that there is an emacs-region patch, which could probably be
extended to implement the behavior I'm talking about, but I have a hard
enough time adding keys to the keymaps, and alas, the time thing...

Maybe someone else will decide it's important enough to do, or has
already done it.

Enjoy,
-Mark

> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

Re: Bulk message tagging

[Xavier Maillard]

Hi,

On Tue, 6 Apr 2010 13:51:01 -0600, Mark Anderson <MarkR.Anderson@amd.com> wrote:
> 
> I think that '*' is definitely an awesome command, but I wonder if we
> shouldn't have another command for the notmuch-search buffer which means
> 'tag all the threads that I can see in this buffer'.

This is exactly what my initial post asked for. '*' is not
totally satisfying for me due to the "limitations" you
exposed. Though It is a good and acceptable workaround for me.
As said, I just have to pay attention to redo my search query
before pressing the '*' key.

Xavier

Re: Bulk message tagging

[Mark Anderson]

On Sat, 10 Apr 2010 08:56:48 -0500, Xavier Maillard <xma@gnu.org> wrote:
> Hi,
> 
> On Tue, 6 Apr 2010 13:51:01 -0600, Mark Anderson <MarkR.Anderson@amd.com> wrote:
> > 
> > I think that '*' is definitely an awesome command, but I wonder if we
> > shouldn't have another command for the notmuch-search buffer which means
> > 'tag all the threads that I can see in this buffer'.
> 
> This is exactly what my initial post asked for. '*' is not
> totally satisfying for me due to the "limitations" you
> exposed. Though It is a good and acceptable workaround for me.
> As said, I just have to pay attention to redo my search query
> before pressing the '*' key.

Another problem I have is that I often _don't_ want to refresh my
search.  Some of my mail processing, while not visible in the search
window, since we don't have a way to refresh tags yet, will remove some of
the current search results from matching the search query.

For example, I like to have notmuch folder definitions with "tag:unread"
in them.  For those search views, once I've looked at any of the mails,
they no longer match the query.  Sometimes I want to refresh the search
so that those mails are no longer visible, sometimed I want to apply an
action on all the visible messages which I've just processed.

When I visit a folder view, what intent do I have?  Am I returning to a
'moment-ago' processing view that was interrupted?  Or am I wanting to
do the search again on the current contents of the database?  I could
easily see mental models that match either way.

I think I read that Carl plans to update the tags in a search view at
some point, without removing threads automatically.

Perhaps there ought to be a way to colorize threads which are displayed
but no longer match the search criterion?

-Mark

Re: Bulk message tagging

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 3555 bytes --]

On Sat, 10 Apr 2010 15:56:48 +0200, Xavier Maillard <xma@gnu.org> wrote:
> On Tue, 6 Apr 2010 13:51:01 -0600, Mark Anderson <MarkR.Anderson@amd.com> wrote:
> > 
> > I think that '*' is definitely an awesome command, but I wonder if we
> > shouldn't have another command for the notmuch-search buffer which means
> > 'tag all the threads that I can see in this buffer'.
> 
> This is exactly what my initial post asked for. '*' is not
> totally satisfying for me due to the "limitations" you
> exposed. Though It is a good and acceptable workaround for me.
> As said, I just have to pay attention to redo my search query
> before pressing the '*' key.

The other bug with "*" is that it doesn't give any visual feedback, (the
tag addition/deletion doesn't show up).

We could fix all[*] the bugs of "*" by changing it to simply call the
new region-based tagging function. The only concern I have with that is
that it might be significantly slower, (it will execute N "notmuch tag"
commands to tag the N threads in the current buffer, rather than just
one "notmuch tag" command with the same search).

But the command-line based "notmuch tag +/-<tag> <search>" will always
still be available for true "bulk" tagging, so maybe having "*" in emacs
be implemented the slower way would be fine.

I think the biggest concern I have with the performance is that if it
*is* too slow, then the user might interrupt it with emacs, and with the
current implementation that would lead to inconsistent display. That's
not specific to "*" though---that's a bug with the current region-based
implementation---it's just that "*" might make it much easier to hit.

-Carl

[*] Most all the bugs, at least. Even just a simple 'a' (or '+' or '-')
on a single thread is already doing something subtly different than it
should. It's tagging all messages in the thread, but that might be more
messages than existed when the thread-summary was created. So you might
see:

	[1/1] A. Bozo				Boring topic...

and decide to archive the thread, and never realize that what you
archived away was several messages that would have been displayed as:

	[3/3] A Bozo, B Wizard, C Wizard	Boring topic... [**]

if you had only refreshed first.

So we really should fix that bug and only manipulate messages that were
actually present when the search was performed. Note that 'a' inside of
the show view does not have this bug---it does only affect messages
displayed.

I'm not currently afflicted by this bug only because I'm running
"notmuch new" manually, before mail-reading sessions as opposed to
inside a cron-job or similar.

[**] This is similar to the "thread pattern" idea described by Joey Hess
here:

	http://kitenet.net/~joey/blog/entry/thread_patterns/

Our current one-line presentation of threads does a really lousy job of
letting the user see thread patterns like this. We should perhaps come
up with something better.

One "something better" I have in mind would be the ability to write
searches that could identify and tag threads according to these
patterns. Our current search syntax isn't powerful enough to express
these kinds of relations about messages within threads, but I would be
very interested in coming up with something that does.

The parts that Xapian can't easily support here probably wouldn't have
to be fast either---they could operate on the results of threads, which
are generally "small". At least, I hope nobody has threads with hundreds
of thousands of messages in them.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Bulk message tagging

[Jesse Rosenthal]

On Wed, 14 Apr 2010 17:59:01 -0700, Carl Worth <cworth@cworth.org> wrote:
> We could fix all[*] the bugs of "*" by changing it to simply call the
> new region-based tagging function. The only concern I have with that is
> that it might be significantly slower, (it will execute N "notmuch tag"
> commands to tag the N threads in the current buffer, rather than just
> one "notmuch tag" command with the same search).

Not quite true: the region command only executes one "notmuch tag"
command over "id:X or id:Y or id:Z or ...". 

It does have to iterate over the messages to get the ids, and again to
set the tags (i.e., write them in the buffer), but that's all in the
buffer -- there's only one xapian command. 

And anything that updates the buffer in real time would have to iterate
through the messages to rewrite the tag representation, so I'm not sure
if there's that much being lost here.

That being said, there might be a bit of redundancy (one save-excursion
trip through the buffer to get the tags for completion, another to get
the ids, and one more to rewrite the parenthetical tag
representation). But I'm not quite sure how to avoid that, or how
expensive it actually is.

Best,
Jesse

Re: Bulk message tagging

[Jesse Rosenthal]

On Thu, 15 Apr 2010 16:04:38 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> Not quite true: the region command only executes one "notmuch tag"
> command over "id:X or id:Y or id:Z or ...". 

Sorry -- I meant, of course: over "thread:X or thread:Y or thread:Z or ..."

Re: Bulk message tagging

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 937 bytes --]

On Thu, 15 Apr 2010 16:04:38 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> On Wed, 14 Apr 2010 17:59:01 -0700, Carl Worth <cworth@cworth.org> wrote:
> > We could fix all[*] the bugs of "*" by changing it to simply call the
> > new region-based tagging function. The only concern I have with that is
> > that it might be significantly slower, (it will execute N "notmuch tag"
> > commands to tag the N threads in the current buffer, rather than just
> > one "notmuch tag" command with the same search).
> 
> Not quite true: the region command only executes one "notmuch tag"
> command over "id:X or id:Y or id:Z or ...". 

Oh, I see. Sorry for my misunderstanding.

So that is more efficient in that regard.

But it also does mean that this operation is all set up to run into
"argument list too long" errors.

We'll probably need to arrange for notmuch to accept search
specifications on stdin or so.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Bulk message tagging

[Jesse Rosenthal]

On Thu, 15 Apr 2010 18:46:56 -0700, Carl Worth <cworth@cworth.org> wrote:
> On Thu, 15 Apr 2010 16:04:38 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> > the region command only executes one "notmuch tag" command over
> > "id:X or id:Y or id:Z or ...".
>
> ...this operation is all set up to run into "argument list too long"
> errors.

I've never run into this error. Is there a specific length that triggers
it? If so, we could chunk the tagging command. Or does the max length
depend on the machine and system?

Re: Bulk message tagging

[Servilio Afre Puentes]

On 15 April 2010 21:46, Carl Worth <cworth@cworth.org> wrote:
[...]
> We'll probably need to arrange for notmuch to accept search
> specifications on stdin or so.

Or a daemon mode with a pipe or DBus interface.

Servilio

Re: Bulk message tagging

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 900 bytes --]

On Fri, 16 Apr 2010 07:47:45 -0400, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> I've never run into this error.

I usually run into this with things like "rm * */*" or so.

> Is there a specific length that triggers
> it? If so, we could chunk the tagging command. Or does the max length
> depend on the machine and system?

It is system dependent. Here are a couple of things I found:

From "man errno":

	E2BIG           Argument list too long (POSIX.1)

and from "man sysconf":

      ARG_MAX - _SC_ARG_MAX
              The  maximum  length  of  the arguments to the exec(3) family of
              functions.  Must not be less than _POSIX_ARG_MAX (4096).

So one could query with sysconf and break things up into multiple
commands as needed.

In fact, until we have some sort of daemon that we can feed
arbitrarily-long lists to, that's what we should do.

-Carl


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

RE: Bulk message tagging

[Arian Kuschki]

Hi
-----Original Message-----
From: Carl Worth <cworth@cworth.org>
Sent: 17 April 2010 17:43

So one could query with sysconf and break things up into multiple
commands as needed.

Doesn't xargs do exactly this?

Re: Bulk message tagging

[David Edmondson]

On Sat, 17 Apr 2010 08:43:19 -0700, Carl Worth <cworth@cworth.org> wrote:
> In fact, until we have some sort of daemon that we can feed
> arbitrarily-long lists to, that's what we should do.

Thinking about loud... What if the sub-commands which accept potentially
long argument lists (most of them?) sprouted a '--stdin' option, which
caused them to read the (remaining) argument list from stdin rather than
looking at argv? It seems that this would be simpler change than full
daemon support.

So:
        notmuch show id:foo@bar.baz
could become:
	echo id:foo@bar.baz | notmuch show --stdin

dme.
-- 
David Edmondson, http://dme.org

RE: Bulk message tagging

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 543 bytes --]

On Sat, 17 Apr 2010 20:32:27 +0200, Arian Kuschki <arian.kuschki@googlemail.com> wrote:
> So one could query with sysconf and break things up into multiple
> commands as needed.
> 
> Doesn't xargs do exactly this?

Almost.

The arguments being passed to the "notmuch tag" command in this case
look like:

	notmuch tag -inbox thread:foo or thread:bar or ...

To break that up, we'd have to be careful to neither leave a trailing
'or' at the end of a command line nor to have an 'or' at the beginning
of a command line.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

RE: Bulk message tagging

[Mark Anderson]

On Wed, 21 Apr 2010 18:02:59 -0500, Carl Worth <cworth@cworth.org> wrote:
> On Sat, 17 Apr 2010 20:32:27 +0200, Arian Kuschki <arian.kuschki@googlemail.com> wrote:
> > So one could query with sysconf and break things up into multiple
> > commands as needed.
> > 
> > Doesn't xargs do exactly this?
> 
> Almost.
> 
> The arguments being passed to the "notmuch tag" command in this case
> look like:
> 
> 	notmuch tag -inbox thread:foo or thread:bar or ...
> 
> To break that up, we'd have to be careful to neither leave a trailing
> 'or' at the end of a command line nor to have an 'or' at the beginning
> of a command line.

Perhaps this hints at an opportunity to create a new operator, that you
can pass as part of the notmuch commandline.

Something like:
cat idlist.txt | xargs notmuch tag -inbox tag:inbox and oneof: 

Then the list of arguments can break anywhere it wants. It's not as
general as having notmuch take search terms from stdin or a file, but it
seems like a long list of ID's is going to be a common case.

Another problem with passing options to xarg is that any parentheses are
going to break easily.

Actually, looking at this random proposal, you can see the invented
operator has an implicit grouping semantic, which leads to all kinds of
confusion.

-Mark