nomuch_addresses.py

nomuch_addresses.py

[Philippe LeCavalier]

Hi again.

I'm trying to setup tab completion for addresses. I appeared to me that
the simplest solution was notmuch_addresses so I grabbed a copy from
git. When I hit tab I get this[1] Was I suppose to do something more?

-- 
Thanks,
Phil

ref.
[1]
~/notmuch/notmuch_addresses/notmuch_addresses.py exited with status 1

Re: nomuch_addresses.py

[Jesse Rosenthal]

On Thu, 16 Feb 2012 14:12:36 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> I'm trying to setup tab completion for addresses. I appeared to me that
> the simplest solution was notmuch_addresses so I grabbed a copy from
> git. When I hit tab I get this[1] Was I suppose to do something more?
...
> ref.
> [1]
> ~/notmuch/notmuch_addresses/notmuch_addresses.py exited with status 1

I'd hope so. What happens when you run it from the command line?

Best,
Jesse

Re: nomuch_addresses.py

[Philippe LeCavalier]

On Thu, 16 Feb 2012 14:46:35 -0500, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> On Thu, 16 Feb 2012 14:12:36 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> > I'm trying to setup tab completion for addresses. I appeared to me that
> > the simplest solution was notmuch_addresses so I grabbed a copy from
> > git. When I hit tab I get this[1] Was I suppose to do something more?
> ...
> > ref.
> > [1]
> > ~/notmuch/notmuch_addresses/notmuch_addresses.py exited with status 1
> 
> I'd hope so. What happens when you run it from the command line?
> 
> Best,
> Jesse

$ ~/notmuch/notmuch_addresses/notmuch_addresses.py
Traceback (most recent call last):
  File "/home/plecavalier/notmuch/notmuch_addresses/notmuch_addresses.py", line 19, in <module>
    import notmuch
ImportError: No module named notmuch

-- 
Thanks,
Phil

Re: nomuch_addresses.py

[Jesse Rosenthal]

On Thu, 16 Feb 2012 14:51:59 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> $ ~/notmuch/notmuch_addresses/notmuch_addresses.py
> Traceback (most recent call last):
>   File "/home/plecavalier/notmuch/notmuch_addresses/notmuch_addresses.py", line 19, in <module>
>     import notmuch
> ImportError: No module named notmuch

Well, there's your problem. You need the notmuch python bindings. They
should be available from your distro repository, or in the bindings/
directory if you installed from source.

Best,
Jesse

Re: nomuch_addresses.py

[Daniel Kahn Gillmor]

On 02/16/2012 02:51 PM, Philippe LeCavalier wrote:
> $ ~/notmuch/notmuch_addresses/notmuch_addresses.py
> Traceback (most recent call last):
>   File "/home/plecavalier/notmuch/notmuch_addresses/notmuch_addresses.py", line 19, in <module>
>     import notmuch
> ImportError: No module named notmuch

Assuming you're using packaged versions:

  apt-get install python-notmuch

or whatever the equivalent is for redhat systems (maybe "yum install
python-notmuch"?)

According to your User-Agent string, though, you're using notmuch 0.5
--i don't know if the python bindings from 0.5 are sufficient for
notmuch_addresses.py.  please report back and let us know what you try
and what works (or doesn't) for you !

hth,

	--dkg

Re: nomuch_addresses.py

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 688 bytes --]

On Thu, 16 Feb 2012 14:12:36 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> Hi again.
> 
> I'm trying to setup tab completion for addresses. I appeared to me that
> the simplest solution was notmuch_addresses so I grabbed a copy from
> git. When I hit tab I get this[1] Was I suppose to do something more?

Just for completeness: I'm using the nice nottoomuch-addresses.pl script
[1] by Tomi Ollila which doesn't require any bindings and is incredibly
fast (after generating an initial address database).

Disclaimer: I never tried the python script, so I can't really give a
useful comparison.

[1] id:"m2d3c8e22v.fsf@guru.guru-group.fi"

Cheers,
Daniel

[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: nomuch_addresses.py

[Tomi Ollila]

On Fri, 17 Feb 2012 02:28:52 +0100, Daniel Schoepe <daniel@schoepe.org> wrote:
> On Thu, 16 Feb 2012 14:12:36 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> > Hi again.
> > 
> > I'm trying to setup tab completion for addresses. I appeared to me that
> > the simplest solution was notmuch_addresses so I grabbed a copy from
> > git. When I hit tab I get this[1] Was I suppose to do something more?
> 
> Just for completeness: I'm using the nice nottoomuch-addresses.pl script
> [1] by Tomi Ollila which doesn't require any bindings and is incredibly
> fast (after generating an initial address database).

Thank you for promoting my stuff.

Latest version is 2.0 and name is now nottoomuch-addresses.sh
It is available at:

http://www.iki.fi/too/nottoomuch/

> Disclaimer: I never tried the python script, so I can't really give a
> useful comparison.

Speedwise it is hard to beat the fgrep(1) execution for finding matches... :)

> 
> Cheers,
> Daniel

Tomi

Re: nomuch_addresses.py

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 393 bytes --]

On Fri, 17 Feb 2012 20:58:50 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> Thank you for promoting my stuff.
> 
> Latest version is 2.0 and name is now nottoomuch-addresses.sh
> It is available at:
> 
> http://www.iki.fi/too/nottoomuch/

It would be great to add these things to the notmuch wiki so that people
can actually find it without searching the email lists.

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: nomuch_addresses.py

[Tomi Ollila]

On Fri, 17 Feb 2012 21:33:41 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> On Fri, 17 Feb 2012 20:58:50 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > Thank you for promoting my stuff.
> > 
> > Latest version is 2.0 and name is now nottoomuch-addresses.sh
> > It is available at:
> > 
> > http://www.iki.fi/too/nottoomuch/
> 
> It would be great to add these things to the notmuch wiki so that people
> can actually find it without searching the email lists.

All of these things have been in notmuch wiki for a while :)

Check http://notmuchmail.org/emacstips/#index12h2


> Sebastian

Tomi

Re: nomuch_addresses.py

[Philippe LeCavalier]

On Fri, 17 Feb 2012 22:46:49 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> On Fri, 17 Feb 2012 21:33:41 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> > On Fri, 17 Feb 2012 20:58:50 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > > Thank you for promoting my stuff.
> > > 
> > > Latest version is 2.0 and name is now nottoomuch-addresses.sh
> > > It is available at:
> > > 
> > > http://www.iki.fi/too/nottoomuch/
> > 
> > It would be great to add these things to the notmuch wiki so that people
> > can actually find it without searching the email lists.
> 
> All of these things have been in notmuch wiki for a while :)
> 
> Check http://notmuchmail.org/emacstips/#index12h2

I think Sebastien is refering to more details on making these things
actually work.

> 
> 
> > Sebastian
> 
> Tomi

-- 
Thanks,
Phil

Re: nomuch_addresses.py

[Tomi Ollila]

On Fri, 17 Feb 2012 22:04:12 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> On Fri, 17 Feb 2012 22:46:49 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > On Fri, 17 Feb 2012 21:33:41 +0100, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> > > On Fri, 17 Feb 2012 20:58:50 +0200, Tomi Ollila <tomi.ollila@iki.fi> wrote:
> > > > Thank you for promoting my stuff.
> > > > 
> > > > Latest version is 2.0 and name is now nottoomuch-addresses.sh
> > > > It is available at:
> > > > 
> > > > http://www.iki.fi/too/nottoomuch/
> > > 
> > > It would be great to add these things to the notmuch wiki so that people
> > > can actually find it without searching the email lists.
> > 
> > All of these things have been in notmuch wiki for a while :)
> > 
> > Check http://notmuchmail.org/emacstips/#index12h2
> 
> I think Sebastien is refering to more details on making these things
> actually work.

Good point !

> > > Sebastian
> > 
> > Tomi
> 
> -- 
> Thanks,
> Phil

Tomi

Re: nomuch_addresses.py

[Philippe LeCavalier]

On Fri, 17 Feb 2012 02:28:52 +0100, Daniel Schoepe <daniel@schoepe.org> wrote:
> On Thu, 16 Feb 2012 14:12:36 -0500, Philippe LeCavalier <support@plecavalier.com> wrote:
> > Hi again.
> > 
> > I'm trying to setup tab completion for addresses. I appeared to me that
> > the simplest solution was notmuch_addresses so I grabbed a copy from
> > git. When I hit tab I get this[1] Was I suppose to do something more?
> 
> Just for completeness: I'm using the nice nottoomuch-addresses.pl script
> [1] by Tomi Ollila which doesn't require any bindings and is incredibly
> fast (after generating an initial address database).
> 
> Disclaimer: I never tried the python script, so I can't really give a
> useful comparison.
> 
> [1] id:"m2d3c8e22v.fsf@guru.guru-group.fi"
> 
> Cheers,
> Daniel
Yes Daniel. Much better indeed. This should also should be mentioned in the wiki ;)

-- 
Thanks,
Phil

Re: nomuch_addresses.py

[Justus Winter]

Quoting Daniel Schoepe (2012-02-17 02:28:52) [emphasis mine]:
>Just for completeness: I'm using the nice nottoomuch-addresses.pl script
>[1] by Tomi Ollila *which doesn't require any bindings* and is incredibly
>fast (after generating an initial address database).

I don't get it. The perl script isn't using any library bindings,
mainly because there are no libnotmuch bindings for perl. *But* it
does call the notmuch binary which is worse:

* incredibly high overhead (fork&exec) compared to a simple function
  call (plus maybe some kind of ffi)
* manual and error prone serialization of ''function arguments''
* manual and error prone deserialization of ''return values''
* very limited error reporting and handling capabilities
* any kind of resource (think handle to a xapian database) is lost if
  the process exists resulting in further overhead if the binary is
  called multiple times

I do get the feeling that it is perceived as desirable not to require
any kind of notmuch bindings (David once said something similar about
nmbug).

If that's the case I'd love to hear why and if there's anything we can
do about it.

Justus

Re: nomuch_addresses.py

[Tomi Ollila]

On Tue, 21 Feb 2012 09:15:09 -0000, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:
> Quoting Daniel Schoepe (2012-02-17 02:28:52) [emphasis mine]:
> >Just for completeness: I'm using the nice nottoomuch-addresses.pl script
> >[1] by Tomi Ollila *which doesn't require any bindings* and is incredibly
> >fast (after generating an initial address database).
> 
> I don't get it. The perl script isn't using any library bindings,
> mainly because there are no libnotmuch bindings for perl. *But* it
> does call the notmuch binary which is worse:
> 
> * incredibly high overhead (fork&exec) compared to a simple function
>   call (plus maybe some kind of ffi)
> * manual and error prone serialization of ''function arguments''
> * manual and error prone deserialization of ''return values''
> * very limited error reporting and handling capabilities
> * any kind of resource (think handle to a xapian database) is lost if
>   the process exists resulting in further overhead if the binary is
>   called multiple times

Shortly answering nottoomuch-addresses.sh's (version 2.0) point of view.

Searching from "database" is done using fgrep. that means 2 forks, 2 execs
from command line (it used to be one fork but now we handle fgrep return
value) (fork, exec /bin/sh, fork, exec fgrep). Anyway, usually matches are
printed out by fgrep before ENTER key has risen (or TAB when run from
emacs).

The "database" is created by perl program ("embedded" in
nottoomuch-addresses.sh), it executes notmuch once to get all email
filenames (since beginning of time or since last update) -- the perl code
then parses email addresses out of those files, cleans those up, removes
duplicates and writes those to the "database" files (all and one with
ignored removed)... So that newest are in top of the file, returned first
in fgrep(1) execution.

> I do get the feeling that it is perceived as desirable not to require
> any kind of notmuch bindings (David once said something similar about
> nmbug).
> 
> If that's the case I'd love to hear why and if there's anything we can
> do about it.

It would be nice if notmuch(1) itself had command to be used for
address completion. I wonder how feasible it is to get a good implementation
there...

> 
> Justus

Tomi

Re: nomuch_addresses.py

[Daniel Schoepe]

[-- Attachment #1: Type: text/plain, Size: 2364 bytes --]

On Tue, 21 Feb 2012 09:15:09 -0000, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:
> Quoting Daniel Schoepe (2012-02-17 02:28:52) [emphasis mine]:
> >Just for completeness: I'm using the nice nottoomuch-addresses.pl script
> >[1] by Tomi Ollila *which doesn't require any bindings* and is incredibly
> >fast (after generating an initial address database).
> 
> I don't get it. The perl script isn't using any library bindings,
> mainly because there are no libnotmuch bindings for perl. *But* it
> does call the notmuch binary which is worse:

I think that emphasis you added may be a bit misleading:

I specifically mentioned that it doesn't need bindings, because the
initial problem in this thread had to do with missing bindings, not
because I think it's inherently a good thing.

The reason I mentioned nottoomuch-addresses at all, is that completion
itself is _a lot_ faster (at least for me), compared to
addrlookup. According to the wiki, notmuch-addresses.py is even slower
than addrlookup, so I thought (and still think) that it was worth
mentioning. Of course, one could rewrite the database-generation part in
python using the bindings, but I personally don't think it's that
necessary.

> * incredibly high overhead (fork&exec) compared to a simple function
>   call (plus maybe some kind of ffi)

I think nottoomuch-addresses.sh is a good example how such performance
considerations can be outweighed by choosing a good "algorithm" instead
(extracting all addresses beforehand).

> * manual and error prone serialization of ''function arguments''
> * manual and error prone deserialization of ''return values''
> * very limited error reporting and handling capabilities
> * any kind of resource (think handle to a xapian database) is lost if
>   the process exists resulting in further overhead if the binary is
>   called multiple times
> 
> I do get the feeling that it is perceived as desirable not to require
> any kind of notmuch bindings (David once said something similar about
> nmbug).

While I don't think that using bindings is a bad thing, the fact that we
can do non-trivial things like nmbug without them, seems to be a good
indicator for the quality of the command line interface though. (And
hence notmuch lends itself well to being used in shell scripts).

Cheers,
Daniel


[-- Attachment #2: Type: application/pgp-signature, Size: 835 bytes --]

Re: nomuch_addresses.py

[Jesse Rosenthal]

On Tue, 21 Feb 2012 14:53:06 +0100, Daniel Schoepe <daniel@schoepe.org> wrote:
> On Tue, 21 Feb 2012 09:15:09 -0000, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:
> The reason I mentioned nottoomuch-addresses at all, is that completion
> itself is _a lot_ faster (at least for me), compared to
> addrlookup. According to the wiki, notmuch-addresses.py is even slower
> than addrlookup, so I thought (and still think) that it was worth
> mentioning. Of course, one could rewrite the database-generation part in
> python using the bindings, but I personally don't think it's that
> necessary.

I'm not sure what speed comparisons were being used -- I think it was
Sebastian comparing vala to python. In any case, using
notmuch_addresses.py to look up a common prefix ("Jes") on a slowish
computer takes 0.2 seconds. So I'm not sure if the speed is all that
much of an issue. It might be a question of cache temperature, though --
it'll probably take longer the first time you run it. Still, even trying
something out on a cold cache, it seems to be about a second.

I guess the question is whether having a second database is worth those
extra fractions of a second. I could totally see how it would be for
some.

Re: nomuch_addresses.py

[Jani Nikula]

On Tue, 21 Feb 2012 11:33:38 -0500, Jesse Rosenthal <jrosenthal@jhu.edu> wrote:
> On Tue, 21 Feb 2012 14:53:06 +0100, Daniel Schoepe <daniel@schoepe.org> wrote:
> > On Tue, 21 Feb 2012 09:15:09 -0000, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:
> > The reason I mentioned nottoomuch-addresses at all, is that completion
> > itself is _a lot_ faster (at least for me), compared to
> > addrlookup. According to the wiki, notmuch-addresses.py is even slower
> > than addrlookup, so I thought (and still think) that it was worth
> > mentioning. Of course, one could rewrite the database-generation part in
> > python using the bindings, but I personally don't think it's that
> > necessary.
> 
> I'm not sure what speed comparisons were being used -- I think it was
> Sebastian comparing vala to python. In any case, using
> notmuch_addresses.py to look up a common prefix ("Jes") on a slowish
> computer takes 0.2 seconds. So I'm not sure if the speed is all that
> much of an issue. It might be a question of cache temperature, though --
> it'll probably take longer the first time you run it. Still, even trying
> something out on a cold cache, it seems to be about a second.

The speed comparisons between vanilla notmuch_addresses.py and
nottoomuch-addresses.sh are going to be flawed in that they do different
things. It's comparing apples and oranges.

notmuch_addresses.py looks for matches in the recipients of mails the
user has sent. Nothing else. notmuch_addresses.py filters out multiple
names for one email address using a popularity contest.

AFAICT nottoomuch-addresses.sh scans all the addresses in all the
mails. It has no logic for filtering out multiple names for one email
address, and just returns all matches.

Personally I would like to have best of both worlds, and I'm using a
modified notmuch_addresses.py that matches all the mails I have, and
cleans up the duplicate results. Unfortunately that does have a toll on
performance, taking about a second on my system for typical searches,
cache hot, while nottoomuch-addresses.sh takes less than a tenth of a
second. It is enough to be annoying, I'm afraid. Even so, it's not a
fair comparison because notmuch_addresses.py wasn't designed with this
in mind, and nottoomuch-addresses.sh maintains its own database and does
less.

One just needs to pick the tool that fits the needs best.


BR,
Jani.

Re: nomuch_addresses.py

[David Bremner]

On Tue, 21 Feb 2012 09:15:09 -0000, Justus Winter <4winter@informatik.uni-hamburg.de> wrote:

> I do get the feeling that it is perceived as desirable not to require
> any kind of notmuch bindings (David once said something similar about
> nmbug).

I think nmbug is a different case from an optional utility. And yes, I
still think it is better not to need the python bindings installed (and
working) in order to report a bug, or develop the core/cli.  Of course
it isn't the only factor, who knows what will happen in the future.

d