Re: [BUG] notmuch_message_tags_to_maildir_flags does not always invalidate filenames_list

[Eliza Velasquez <eliza@eliza.sh>]

On Tue, May 24 2022 at 09:45 -03, David Bremner <david@tethera.net> wrote:

> Eliza Velasquez <eliza@eliza.sh> writes:
>
>> Hello notmuch,
>>
>> As the subject says, it seems like the filename cache for a message is
>> not reliably invalidated when notmuch_message_tags_to_maildir_flags is
>> called, causing the following series of function calls to behave
>> unexpectedly.
>>
>> - notmuch_database_index_file
>> - notmuch_message_get_filenames
>> - notmuch_message_tags_to_maildir_flags
>> - notmuch_message_get_filenames (stale list returned)
>>
>> According to the investigator [1], this happens about 10% of the time,
>> but it's possible that this is just because there is some quirk in the
>> client-side code causing it to trigger only sometimes.
>>
>> [1] https://github.com/elizagamedev/mujmap/pull/10#issuecomment-1131587169
>>
>
> I have been looking at this code, and making some small improvements [1],
> but without a C reproducer I don't really know if it helps the
> submitter's issue.
>
> d
>
> [1]: https://nmbug.notmuchmail.org/nmweb/show/20220523233901.3506880-1-david@tethera.net

Thank you for looking into this and putting out such a quick patch, I
really appreciate it. It turns out the core issue that we were
encountering actually seems like it's due to cache propagation issues
across message instances, according to the investigator (Rob). To quote:

> I think its a fairly standard cache write propagation issue, with the
> outstanding message objects as the caches. Whether or not its a bug in
> notmuch itself I think is entirely down to whether holding and
> modifying multiple message objects for the "same" record (with
> duplicates) is supposed to work or not. I'm in two minds about it; on
> the one hand, multiple message objects is clearly a valid use case
> (queries!), on the other, playing with the mail files themselves is a
> bit of a niche feature. I'll think about it a bit more and maybe take
> it up to the notmuch list myself.

He wrote a more detailed writeup of it as well [1], which includes a
minimal reproducible test case written in C. After reviewing this, I'm
also not sure whether or not this should be considered a notmuch bug,
but it was surprising behavior.

[1] https://gist.github.com/robn/a56420062127238f57541eb47bc962f5