[FEATURE] Purge ignored messages from index

[FEATURE] Purge ignored messages from index

[Markus Weimar]

Indexed but subsequently ignored messages remain indexed. I suggest to  purge them from the index as if the files were removed.

Example:

* Create index including spam directory
* Add spam directory to ignore list
* Neither ´new´ nor ´reindex 'folder:spam'´ purges the spam messages from the index

Markus

Re: [FEATURE] Purge ignored messages from index

[David Bremner]

Markus Weimar <markus.weimar@gmail.com> writes:

> Indexed but subsequently ignored messages remain indexed. I suggest to  purge them from the index as if the files were removed.
>
> Example:
>
> * Create index including spam directory
> * Add spam directory to ignore list
> * Neither ´new´ nor ´reindex 'folder:spam'´ purges the spam messages from the index
>

My first instinct would be to leave new as is, but change reindex to pay
attention to some ignore parameter.

d

Re: [FEATURE] Purge ignored messages from index

[Jeremy Nickurak]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]

Don't they need to be in the index in order to match the ignore condition,
which then allows them to be excluded from usual activities?

What's the actual thing you're hoping to achieve by not having those
ignored messages be in the index?

On Thu, Nov 22, 2018 at 10:33 AM David Bremner <david@tethera.net> wrote:

> Markus Weimar <markus.weimar@gmail.com> writes:
>
> > Indexed but subsequently ignored messages remain indexed. I suggest to
> purge them from the index as if the files were removed.
> >
> > Example:
> >
> > * Create index including spam directory
> > * Add spam directory to ignore list
> > * Neither ´new´ nor ´reindex 'folder:spam'´ purges the spam messages
> from the index
> >
>
> My first instinct would be to leave new as is, but change reindex to pay
> attention to some ignore parameter.
>
> d
> _______________________________________________
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
>

[-- Attachment #2: Type: text/html, Size: 1564 bytes --]

Re: [FEATURE] Purge ignored messages from index

[David Bremner]

Jeremy Nickurak <not-much@trk.nickurak.ca> writes:

> Don't they need to be in the index in order to match the ignore condition,
> which then allows them to be excluded from usual activities?

new.ignore is not about searching. It is about indexing.

A similar, but different feature for searching is search.exclude_tags

d

Re: [FEATURE] Purge ignored messages from index

[Markus Weimar]

Thanks for your reply! I only installed notmuch today, so please take this into account when considering my thoughts.

On Thu, 2018-11-22 13:33:06 -0400, David Bremner wrote:
> My first instinct would be to leave new as is, but change reindex to pay
> attention to some ignore parameter.

´new´ already removes messages from the index if the files are gone. Removing those but keeping newly ignored messages seems inconsistent to me and it did confuse me.

Why would I ever want to keep an ignored email in the index? In regard to indexing, I don’t see much of a difference between deleted and ignored. I don’t want search results for either.

I’d guess setting up notmuch and creating a ´notmuch new´ cronjob is not that uncommon. If a user later adds ignores, they are likely to run into this issue. If the indexing followed the config by purging anything ignored, this would “just work.”

Best wishes,
Markus

Re: [FEATURE] Purge ignored messages from index

[David Bremner]

Markus Weimar <markus.weimar@gmail.com> writes:

> Thanks for your reply! I only installed notmuch today, so please take this into account when considering my thoughts.
>
> On Thu, 2018-11-22 13:33:06 -0400, David Bremner wrote:
>> My first instinct would be to leave new as is, but change reindex to pay
>> attention to some ignore parameter.
>
> ´new´ already removes messages from the index if the files are
> gone. Removing those but keeping newly ignored messages seems
> inconsistent to me and it did confuse me.

Notmuch new works very hard to not visit files that haven't
changed. Changing this would have a big negative performance impact, afaik.

I care less about the performance of reindex; also it already
rescans all the relevant messages, so I guess the performance impact
would not be that bad.

Sometimes it's worth giving up performance for more intuitive behaviour,
but most people seem to manage with the current behaviour.

d

Re: [FEATURE] Purge ignored messages from index

[Markus Weimar]

On Thu, 2018-11-22 16:49:56 -0400, David Bremner wrote:
> Notmuch new works very hard to not visit files that haven't
> changed. Changing this would have a big negative performance impact, afaik.

I see. I thought it might be possible to purge the entries without looking at any files by doing a cheap comparison of indexed file path against ignored patterns. If that’s not possible, I agree that ´new´ shouldn’t be slowed down for this edge case.

> I care less about the performance of reindex; also it already
> rescans all the relevant messages, so I guess the performance impact
> would not be that bad.

Makes sense to me. Short notes in the docs and config file to run ´reindex 'folder:<pattern>'´ after adding new ignore patterns would prevent any such confusion.

Take care,
Markus

Re: [FEATURE] Purge ignored messages from index

[David Bremner]

Markus Weimar <markus.weimar@gmail.com> writes:

> On Thu, 2018-11-22 16:49:56 -0400, David Bremner wrote:
>> Notmuch new works very hard to not visit files that haven't
>> changed. Changing this would have a big negative performance impact, afaik.
>
> I see. I thought it might be possible to purge the entries without
> looking at any files by doing a cheap comparison of indexed file path
> against ignored patterns. If that’s not possible, I agree that ´new´
> shouldn’t be slowed down for this edge case.
>

File paths aren't really indexed, only the directory part.

d