Python/thaw/freeze: Different references to same message

Python/thaw/freeze: Different references to same message

[William Pettersson]

[-- Attachment #1.1: Type: text/plain, Size: 1270 bytes --]

Hey,

I've been trying to emulate something similar to what Ben Gamari shows
in https://notmuchmail.org/pipermail/notmuch/2011/003976.html and I ran
into an issue.

That script (which admittedly is from 6 years ago) freezes messages
before changing tags. But when changing tags, new searches are often
run, returning some of the same results. It seems any changes on these
"inner" messages are lost when the outer messages are thawed, and I'm
guessing this is because the "inner" and "outer" messages reference
different objects in memory which point to the same Xapian document.

I've attached a short script which demonstrates this, to test it you
need at least one message tagged with "this_is_my_test", but the script
is fairly simple.

Now If I'm correct in guessing the issue here, I can understand that
it's not the sort of thing that is fun to fix, so I'm more looking to
see if my understanding is right. In particular, I'm asking because
clearly 6 years ago Ben's scripts worked, so maybe I'm just not
understanding them/notmuch correctly.

If I am right, then I'll just leave out the freezing/thawing. As it
stands, I'm only adding ~5-10 emails at most each run (which is every 10
minutes) so performance when adding emails isn't (yet) an issue for me.

Will

[-- Attachment #1.2: test_nm.py --]
[-- Type: text/x-python, Size: 645 bytes --]

import notmuch

db = notmuch.Database(mode=notmuch.Database.MODE.READ_WRITE)

query_str = "tag:this_is_my_test"

query1 = notmuch.Query(db, query_str)

for msg in query1.search_messages():
    msg.freeze()
    print "At start tags are %s" % ", ".join(msg.get_tags())
    msg.add_tag("added_first")
    query2 = notmuch.Query(db, query_str)
    for msg2 in query2.search_messages():
        msg2.freeze()
        msg2.add_tag("added_second")
        msg2.thaw()
        print "Inside inner loop tags are %s" % ", ".join(msg2.get_tags())
    msg.thaw()
    print "At end tags are %s" % ", ".join(msg.get_tags())
    msg.remove_tag("added_first")


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Python/thaw/freeze: Different references to same message

[David Bremner]

William Pettersson <william@ewpettersson.se> writes:

> That script (which admittedly is from 6 years ago) freezes messages
> before changing tags. But when changing tags, new searches are often
> run, returning some of the same results. It seems any changes on these
> "inner" messages are lost when the outer messages are thawed, and I'm
> guessing this is because the "inner" and "outer" messages reference
> different objects in memory which point to the same Xapian document.

That sounds likely to be the case. There's no attempt to synchronize
multiple in memory caches of the same xapian document.

> Now If I'm correct in guessing the issue here, I can understand that
> it's not the sort of thing that is fun to fix, so I'm more looking to
> see if my understanding is right. In particular, I'm asking because
> clearly 6 years ago Ben's scripts worked, so maybe I'm just not
> understanding them/notmuch correctly.
>
> If I am right, then I'll just leave out the freezing/thawing. As it
> stands, I'm only adding ~5-10 emails at most each run (which is every 10
> minutes) so performance when adding emails isn't (yet) an issue for me.

You might try begin_atomic / end_atomic instead. These nest properly by
design, although I'm not so confident about the performance impact
(they're not really focussed on performance).