Python updates

Python updates

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 2836 bytes --]

Hi all,
I just wanted to keep everyone updated on the python API changes that
are in the master branch. With one exception everything is backwards
compatible. Feel free to stuff this into the .6 release notes :).

- Messages() can now be "listified" with a simple: msglist = list(...)
  in case you plan to keep doing stuff with the contained messages. For
  this to work, the API had to be broken: __len__ function was
  implicitly invoked by list(), exhausting the iterator. By removing
  __len__ (and implementing __nonzero__ instead), the list() function
  works now fine on a Messages() instance. If you need the length of
  your Messages() object, you can use:

    - Query(...).count_messages() #for a pretty close approximation
    - len(list(Messages())) #for the precise count (exhausting the
      iterator)
    - bool(Messages()) to see if still at least one more Message() is
      contained in the iterator.

- Message() now has a get_filenames() method which returns an iterator
  (generator function to be precise), pointing to all filenames that are
  recorded for that message ID. list(msg1.get_filenames()) will return
  all recorded names as a list.

- Message() now implements _cmp_ and _hash_. This enables a) comparison
  of Messages: msgs1 == msgs2 is True if they both 1) contain a Message
  object with the same Message-ID and 2) if the list of contained file
  names is identical.

  Another implication is that set() arithmetic on Messages() is now
  possible:
  db= notmuch.Database()
  msgs1= set(notmuch.Query(db, 'from:Sebastian').search_messages())
  msgs2= set(notmuch.Query(db, 'not to:notmuch').search_messages())

  msgs3 = msgs1.union(msgs2)
  msgs3 = msgs1.difference(msgs2)
  etc...

  I don't need to mention that the performance of set arithmetic (being
  completely done in python) is going to be abysmal compared to redoing
  proper Querys, yet, it might come in handy here or there.

  It also means you can do things like:

  msgs1= list(notmuch.Query(db, 'from:Sebastian').search_messages())
  msg= list(notmuch.Query(db, 'not to:notmuch').search_messages())[0]
  
  if msg in msgs1:
     #Yes!!!

- Both Message() and Messages() implement now __nonzero__() which is
  used for boolean testing. To see whether a Message() actually contains
  a Message, you can now do: bool(msg) or if msg: ...

  More useful, for Messages() you can check if your iterator still
  contains at least one more Message():

  msgs = notmuch.Query(db, 'from:Sebastian').search_messages()
  while msgs:
     msg = msgs.next() #yes, this will work we still had one!


  and also:

  msgs = notmuch.Query(db, 'from:quackydiquack').search_messages()
  if not msgs:
    print "Oh my, no search result for this weird query"


Have fun
Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Python updates

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 692 bytes --]

On Thu, 16 Jun 2011 12:59:47 +0200, Sebastian Spaeth  wrote:

> I just wanted to keep everyone updated on the python API changes that
> are in the master branch. With one exception everything is backwards
> compatible. Feel free to stuff this into the .6 release notes :).

P.S. Once I get started.... I was teased on IRC, so I also implemented:

Message().maildir_flags_to_tags and tags_to_maildir_flags, which are now
by default invoked on Database().add_message and on any operation that
modifies notmuch tags. See the updated API docs at:

http://packages.python.org/notmuch/#notmuch.Message.tags_to_maildir_flags

and the docs for freeze() as to how to use those efficiently.

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Python updates

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 963 bytes --]

On Thu, 16 Jun 2011 15:58:39 +0200, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> P.S. Once I get started.... I was teased on IRC, so I also implemented:
> 
> Message().maildir_flags_to_tags and tags_to_maildir_flags, which are now
> by default invoked on Database().add_message and on any operation that
> modifies notmuch tags. See the updated API docs at:

I'm not a fan of the above.

I would greatly prefer that any language bindings for notmuch implement
the same semantics as the underlying functions.

If we should fix Database().add_message to automatically call
maildir_flags_to_tags, then we should also fix
notmuch_database_add_message to do the same thing.

I can't see any good justification for behavior like this to be
different depending on the language being used to access the library.

Please consider changing things to match the semantics of the underlying
library.

Thanks,

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Python updates

[Sebastian Spaeth]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

On Tue, 21 Jun 2011 14:46:02 -0700, Carl Worth wrote:
Non-text part: multipart/signed
> On Thu, 16 Jun 2011 15:58:39 +0200, Sebastian Spaeth wrote:
> > Message().maildir_flags_to_tags and tags_to_maildir_flags, which are now
> > by default invoked on Database().add_message and on any operation that
> > modifies notmuch tags. See the updated API docs at:
> 
> I'm not a fan of the above.

Hi Carl,

I see your point. I was approached with this by someone very
confused that tagging via notmuch binary would automatically move mails
between cur/new folders while tagging via python would do nothing of
this sort.

The automatic tag/flag synchronization can be disabled via a command
line option already now, it is just enabled by default. The behavior is
of course documented in the python API docs. It would be no problem, to
disable the syncing by default to achieve API consistency. (and allow
people to turn it on, if they wish to do so explicitly)

> If we should fix Database().add_message to automatically call
> maildir_flags_to_tags, then we should also fix
> notmuch_database_add_message to do the same thing.

 
> I can't see any good justification for behavior like this to be
> different depending on the language being used to access the library.

See above, people don't consider using the libnotmuch API, they "tag" a
message via python and it behaves differently than "tag" a message via
notmuch binary....
So we'll have some level of inconsistency in any case. :)

> Please consider changing things to match the semantics of the underlying
> library.

Would you be happy to have maildir syncing disabled by default and users
can enable it via a parameter? I do see why you want to achieve
consistency with the API. On the other hand are the python API somewhat
more highlevel than the low-level API calls, and we provide a few
convenience functions that are not available in the API at all.

I can do away with it and force people to invoke the syncing manually,
if you feel strongly about this. I don't have a problem with either way.

Sebastian

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Python updates

[Carl Worth]

[-- Attachment #1: Type: text/plain, Size: 2682 bytes --]

On Wed, 22 Jun 2011 08:57:09 +0200, Sebastian Spaeth <Sebastian@SSpaeth.de> wrote:
> I see your point. I was approached with this by someone very
> confused that tagging via notmuch binary would automatically move mails
> between cur/new folders while tagging via python would do nothing of
> this sort.

That's also a good point.

But the same confusion can happen to someone using "notmuch tag" and
then switching to using notmuch_message_add_tag at the C library
interface.

That's what I meant when I said that if there's an inconsistency here,
then we should fix it at the C library interface, and not just in a
language-specific wrapper.

> See above, people don't consider using the libnotmuch API, they "tag" a
> message via python and it behaves differently than "tag" a message via
> notmuch binary....
> So we'll have some level of inconsistency in any case. :)

Let's figure out one right answer for the library interface, (regardless
of language) and implement that.

> Would you be happy to have maildir syncing disabled by default and users
> can enable it via a parameter?

I'd be happy to hear a proposal to add a parameter to the library
interface for maildir syncing, (and I wouldn't even be opposed to having
it enabled by default).

The only thing I care about strongly here is that we have a single
library interface. I don't want one interface in C, a different
interface in python, (and different interfaces in go, ruby, etc.).

Sometimes a language will provide some convenient builtin handling for
something that's awkward at the notmuch C interface, (such as a native
interface for iterators). And I definitely don't mind a language binding
using something like that as opposed to manually binding every
iteration-supporting function that we have in the notmuch library.

But I don't want to see semantic changes to the interface that don't
have anything to do with the language itself.

> I do see why you want to achieve consistency with the API.

Thanks.

> On the other hand are the python API somewhat more highlevel than the
> low-level API calls, and we provide a few convenience functions that
> are not available in the API at all.

That's not the stance I'd like to see.

If you want convenience in the python interface that doesn't exist in
the C interface, then let's start by fixing the C interface.

If there's convenience you want in the python interface that shouldn't
exist in the C interface, then I would propose that that functionality
should be in a separate python layer (above the binding) with its own
name.

What do you think?

-Carl

-- 
carl.d.worth@intel.com

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]