hack to retag a directory

hack to retag a directory

[david]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]


I think this will be obsolete pretty soon when the equivalent is
built-in to notmuch, but in the mean time, here is a script that
somebody might find useful: retag a whole directory (recursively). I
don't claim it is nice in any way, but it seems usable for me, taking
about 5 seconds to retag a directory containing 1000 messages.

I use it like 

  tagdir Maildir/.list.debian-devel -inbox +debian +devel


[-- Attachment #2: tagdir --]
[-- Type: application/octet-stream, Size: 681 bytes --]

#!/usr/bin/perl
# Copyright © David Bremner <david@tethera.net>
# Distributed under the WTFPL <http://sam.zoy.org/wtfpl/>

# usage: $0 directory tags

use strict;

my $batchsize=200;
my $count=0;
my $dir=$ARGV[0];
shift(@ARGV);

my @tags=@ARGV;
my @msg_ids=();

open (ID, "grep -h -R '^Message-I[dD]' $dir|") || die "$!";
while(<ID>){
  chomp();
   s/[\<\>]//g;
   s/([;<>\*\|`&\$!#\(\)\[\]\{\}:'"])/\\$1/g;
   s/^Message-I[dD]: /id:/;
   push(@msg_ids,$_);
   $count++;
   if ($count>=$batchsize){
     $count=0;
     system("notmuch tag ".join(" ",@tags)." ".join(" or ", @msg_ids));
     @msg_ids=();
   }
 }
system("notmuch tag ".join(" ",@tags)." ".join(" or ", @msg_ids));


[-- Attachment #3: Type: text/plain, Size: 3 bytes --]


  

Re: hack to retag a directory

[david]

At Thu, 03 Dec 2009 09:39:32 -0400,
david@tethera.net wrote:
> I think this will be obsolete pretty soon when the equivalent is
> built-in to notmuch, but in the mean time, here is a script that
> somebody might find useful: retag a whole directory (recursively). I
> don't claim it is nice in any way, but it seems usable for me, taking
> about 5 seconds to retag a directory containing 1000 messages.

Sigh. And of course the version I posted was broken. I put a fixed version at 

      http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=tagdir;hb=HEAD

You might, or might not also be interested in 

    http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=gitmuch;hb=HEAD

which is the beginnings of how to keep tags in git (for syncing
between machines).  Right now the notmuch restore step is the
bottleneck, but Carl apparently knows how to speed 'notmuch restore'
up.

  d

Re: hack to retag a directory

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2854 bytes --]

On Thu, 03 Dec 2009 15:26:56 -0400, david@tethera.net wrote:
> > I think this will be obsolete pretty soon when the equivalent is
> > built-in to notmuch, but in the mean time, here is a script that
> > somebody might find useful: retag a whole directory (recursively). I
> > don't claim it is nice in any way, but it seems usable for me, taking
> > about 5 seconds to retag a directory containing 1000 messages.
> 
> Sigh. And of course the version I posted was broken. I put a fixed version at 
> 
>       http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=tagdir;hb=HEAD

Thanks for sharing that, David.

Obviously, we now have outstanding patches to provide search
capabilities based on the folder containing messages. So once that gets
merged, you shouldn't need this script anymore.

> You might, or might not also be interested in 
> 
>     http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=gitmuch;hb=HEAD
> 
> which is the beginnings of how to keep tags in git (for syncing
> between machines).

But this one looks quite interesting. Obviously, it's not a complex
script, but it looks pretty handy to me. I might start using this to
have a little history of my tag changes, (rather than just including the
dump output in occasional backups like I have been doing).

And it's interesting that this script might be just good enough for the
synchronization needs of some people. It's not integrated, and might
require manual fixup of any resulting git conflicts, but it might be
handy for some.

The biggest problem I see is that if I were to read some messages
locally, and then run "gitmuch restore" then this would wipe out the
local changes I had made. So we'll definitely want a more integrated
solution to eliminate the chance of problems like this.

>                     Right now the notmuch restore step is the
> bottleneck, but Carl apparently knows how to speed 'notmuch restore'
> up.

One easy answer is to just make "notmuch restore" do nothing for
messages where the existing tags are the same as the tags mentioned in
the input file. I just pushed a change to implement this, (along with
new tests for "notmuch dump" and "notmuch restore" of course).

For me, this takes a "notmuch restore" right after a "notmuch dump" from
about 10 minutes down to 1 minute, (and it was about 2 hours before the
Xapian Defect #250 fix).

The other idea that I didn't do yet is to change "notmuch restore" to do
a single search for all messages rather than N searches each resulting
in 1 message. But the 1-minute time I'm getting for "notmuch restore"
now is basically the same time required for a "notmuch dump", (which is
already doing a single global search). So perhaps Xapian is just plain
fast enough that a change like that won't help at all.

-Carl

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: hack to retag a directory

[David Bremner]

On Fri, 05 Feb 2010 17:41:48 -0800, Carl Worth <cworth@cworth.org> wrote:

[talking about gitmuch, a simple wrapper around notmuch dump && git commit]
> 
> And it's interesting that this script might be just good enough for the
> synchronization needs of some people. It's not integrated, and might
> require manual fixup of any resulting git conflicts, but it might be
> handy for some.
> 

I have to say that merge conflicts are not very much fun. I tend to do a
certain amount of oh, take all the changes from the server.  I wonder if
the approach that someone else mentioned of keeping a file
tags/message-id with the appropriate tags in it might make merging less
painful.

> The biggest problem I see is that if I were to read some messages
> locally, and then run "gitmuch restore" then this would wipe out the
> local changes I had made. So we'll definitely want a more integrated
> solution to eliminate the chance of problems like this.

Yeah, the footgun potential is definitely there.

> One easy answer is to just make "notmuch restore" do nothing for
> messages where the existing tags are the same as the tags mentioned in
> the input file. I just pushed a change to implement this, (along with
> new tests for "notmuch dump" and "notmuch restore" of course).

Heh, I think I later posted a patch to do that as well.

d

Re: hack to retag a directory

[martin f krafft]

[-- Attachment #1: Type: text/plain, Size: 830 bytes --]

also sprach David Bremner <david@tethera.net> [2010.02.07.1238 +1300]:
> I have to say that merge conflicts are not very much fun. I tend
> to do a certain amount of oh, take all the changes from the
> server.  I wonder if the approach that someone else mentioned of
> keeping a file tags/message-id with the appropriate tags in it
> might make merging less painful.

Merge conflicts at the file level are even less fun. This reminds me
of my plan to introduce .gitignore.d/*:

http://kerneltrap.org/mailarchive/git/2007/10/1/326425

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
"there was silence for a moment, and then out of the scrambled mess
 of arthur's brain crawled some words."
                                 -- hitchhiker's guide to the galaxy
 
spamtraps: madduck.bogus@madduck.net

[-- Attachment #2: Digital signature (see http://martin-krafft.net/gpg/) --]
[-- Type: application/pgp-signature, Size: 198 bytes --]