Re: out of memory on idle machine

[David Bremner <david@tethera.net>]

Gregor Zattler <telegraph@gmx.net> writes:

> Hi David, notmuch and xapian developers,
> * David Bremner <david@tethera.net> [03. Feb. 2021]:
>
>
>> Olly Betts mentioned in a different thread that he will build a version
>> of xapian 1.4.18 for buster backports, so trying with that is probably a
>> good step when it is available.
>
> I'll do so.
>

At this point I don't really have any good ideas, so I'm waiting for
results from the 1.4.18 trial.

>>
>> Comparing those two lists may give you some hints. Any directory that
>> shows up in the second list but not the first should have no files in it
>> (but potentially other directories) or be ignored either implicitly
>> (.notmuch, .notmuch/xapian) or explicitely by your configuration.
>
>
> I tried this after the second OOM (see above):
>
> grfz@mic:~/Mail/.notmuch$ rm -rf xapian
> grfz@mic:~/Mail/.notmuch$ cp -a xapian-3 xapian
>
> grfz@mic:~/Mail/.notmuch$ xapian-delve -1 -A XDIRECTORY ~/Mail/.notmuch/xapian | sort -u > /tmp/delve.txt
> grfz@mic:~/Mail/.notmuch$ find ~/Mail/  -type d -not -empty | sed s,/home/grfz/Mail/,XDIRECTORY, |sort -u  >/tmp/find.txt
>
> As expected the find.txt contains a huge amount of ../cur
> directories.
>
> I searched for the directories in find.txt, which are not in
> delve.txt:
>
> grfz@mic:~/Mail/.notmuch$ while read ; do grep -qF "$REPLY" /tmp/delve.txt 2>/dev/null || echo $REPLY ; done < /tmp/find.txt > /tmp/out
>
> It contains a huge amount of ../cur directories.
>
> I searched if some of these ../cur directories do not
> contain files:
>
> grfz@mic:~/Mail/.notmuch$ cat /tmp/out | grep "/cur$" | sed -e "s,XDIRECTORY,/home/grfz/Mail/," | while read ; do cd "$REPLY" ; test $(find -type f | wc -l) = 0 && echo "$REPLY" ; done
> /home/grfz/Mail/findex/cur

I don't have any /cur directories in my version. I do have a few (3 or 4) /tmp
directories that are apparently not indexed. That's a bit mysterious,
but nothing on the scale of what you are seeing.

d