Enabling and disabling maildir.synchronize_flags

Enabling and disabling maildir.synchronize_flags

[dm-list-email-notmuch]

Sorry if this question is answered somewhere, but I'm wondering:  What
is the best way to enable and disable maildir.synchronize_flags?

It seems that disabling it should simply be safe.  But re-enabling, one
risks losing tags, as the next notmuch new will cause old maildir flags
to override the xapian database.  So that suggests something like:

   notmuch dump > backup
   notmuch config set maildir.synchronize_flags false
   # Do I need to run notmuch new here?
   notmuch restore < backup

Is that safe?  The man page suggests one additionally need to run
notmuch new before running notmuch restore.  All of this is pretty slow.
Is there a more efficient way?

A one or two sentence clarification in the notmuch-config man page might
be helpful to people contemplating playing with this switch.  The
default is on, to I suspect it costs a lot of performance.  I've been
afraid to turn it off for fear that I won't be able to undo this
cleanly.

Thanks,
David

Re: Enabling and disabling maildir.synchronize_flags

[David Bremner]

dm-list-email-notmuch@scs.stanford.edu writes:

> It seems that disabling it should simply be safe.  But re-enabling, one
> risks losing tags, as the next notmuch new will cause old maildir flags
> to override the xapian database.  So that suggests something like:
>
>    notmuch dump > backup
>    notmuch config set maildir.synchronize_flags false
>    # Do I need to run notmuch new here?
>    notmuch restore < backup

Hi David;

Sorry about the long delay. I'm not sure I follow the connection between
your paragraph above and the the example. The example seems safe, but as
you say, disabling synching should bs safe anyway.

d

Re: Enabling and disabling maildir.synchronize_flags

[David Mazieres]

David Bremner <david@tethera.net> writes:

> dm-list-email-notmuch@scs.stanford.edu writes:
>
>> It seems that disabling it should simply be safe.  But re-enabling, one
>> risks losing tags, as the next notmuch new will cause old maildir flags
>> to override the xapian database.  So that suggests something like:
>>
>>    notmuch dump > backup
>>    notmuch config set maildir.synchronize_flags false
>>    # Do I need to run notmuch new here?
>>    notmuch restore < backup
>
> Hi David;
>
> Sorry about the long delay. I'm not sure I follow the connection between
> your paragraph above and the the example. The example seems safe, but as
> you say, disabling synching should bs safe anyway.

It's not an example, it's kind of a worst case scenario if there's no
easy and safe way way to enable synchronize_flags.  I want to try
disabling flags, but if I change my mind and the only way to get it back
is to do a full notmuch dump/restore, that's a pretty hefty penalty.
Also, even if I do a full notmuch dump / restore, I'm not sure if the
notmuch new is necessary in the middle.

So my question remains, what's the easiest safe way to re-enable
synchronize_flags after disabling it?  (Safe meaning it won't change any
tags.)  It could be that there's a very simple answer, in which case
sticking it in the man page might be nice.

Thanks,
David

Re: Enabling and disabling maildir.synchronize_flags

[David Bremner]

David Mazieres <dm-list-email-notmuch@scs.stanford.edu> writes:


> So my question remains, what's the easiest safe way to re-enable
> synchronize_flags after disabling it?  (Safe meaning it won't change any
> tags.)  It could be that there's a very simple answer, in which case
> sticking it in the man page might be nice.

I can't think of a simple, safe, and fast answer.

As a hack, if you run notmuch new right before you enable
synchronize_flags, then next the next notmuch new will not notice the
fact that flags are out of sync [1]. I don't think this really qualifies
as safe, as the fact that the maildir is out of sync with the database
will eventually bite you. Maybe in some special circumstances this
helps.

Otherwise, two things to that might speed up the process.

1) dump and restore both take queries, so you can only worry about the
maildir relevant flags.

2) when the lastmod changes go in, it seems like you could run the first
   notmuch new after enabling tag synchonizing, and dump only the tag
   changes since a checkpoint lastmod value. This would allow rolling
   back the unwanted tag changes.


[1]: see this potential test, if for some reason we wanted to
     guarantee this behaviours.
     
test_begin_subtest "Turning on maildir.synchronize_flags w/o backup"
OLDCONFIG=$(notmuch config get maildir.synchronize_flags)

notmuch config set maildir.synchronize_flags true
add_message [subject]='"Message to test ignoring flags"' \
    [date]='"Sat, 01 Jan 2000 12:00:00 -0000"' \
    [filename]='ignoring-flags' [dir]=cur
${NOTMUCH_NEW}
notmuch dump > EXPECTED

notmuch config set maildir.synchronize_flags false
# make flags out of sync with db
mv ${MAIL_DIR}/cur/ignoring-flags ${MAIL_DIR}/cur/ignoring-flags,S
${NOTMUCH_NEW}

notmuch config set maildir.synchronize_flags true
${NOTMUCH_NEW}
notmuch dump > OUTPUT
notmuch config set maildir.synchronize_flags "${OLDCONFIG}"
test_expect_equal_file EXPECTED OUTPUT

Re: Enabling and disabling maildir.synchronize_flags

[dm-list-email-notmuch]

David Bremner <david@tethera.net> writes:

> David Mazieres <dm-list-email-notmuch@scs.stanford.edu> writes:
>> So my question remains, what's the easiest safe way to re-enable
[ 2 more citation lines. Click/Enter to show. ]
>> synchronize_flags after disabling it?  (Safe meaning it won't change any
>> tags.)  It could be that there's a very simple answer, in which case
>> sticking it in the man page might be nice.
>
> I can't think of a simple, safe, and fast answer.

Okay, thanks.  At least I wasn't missing something obvious.

> 2) when the lastmod changes go in, it seems like you could run the first
>    notmuch new after enabling tag synchonizing, and dump only the tag
>    changes since a checkpoint lastmod value. This would allow rolling
>    back the unwanted tag changes.

Indeed, one of many reasons I'm eagerly awaiting lastmod changes.

> [1]: see this potential test, if for some reason we wanted to
>      guarantee this behaviours.

If we did want this, I'm assuming it would take the form of a new option
to notmuch new (--override-flags) which says to do the synchronization
in the other direction (Xapian -> Maildir)?  There would be benefit to
having such a flag, but I don't know how hard it would be to implement,
so I can't do the cost/benefit analysis.

As a kind of aside, one reason people might want to synchronize flags is
for mobile device support.  I don't regularly access my email from my
mobile phone, but on those rare occasions when I might need to, I set up
an IMAP server and use an imap client on the phone.  I wonder if anyone
has thought about implementing an IMAP-ish server directly on top of
libnotmuch.  (I say IMAP-ish because the obvious SEARCH command
implementation wouldn't be RFC3501-compliant, but who cares when notmuch
has something better.)

Does anyone else use both notmuch but also access email from a mobile
device?  If so what do you do?

David