Re: Improving org-contacts performance (and state of development in general)

[Ihor Radchenko <yantar92@gmail.com>]

> tho quite interested in seeing what perf enhancements you've done on large
> org files would be interesting.

That's primarily a one single enhancement - use text properties instead
of overlays to hide/fold text. Overlays are slow - every time Emacs need
to move point across hidden overlays, it takes O(N_overlays). The
problem especially affects buffers with many property drawers (every
drawer may require an individual overlay) - should be the case for
org-contacts files. 

> Primary examples would be adding a note (CTRL-z) or changing a tag on a
> person and then having org-agenda update that. I am assuming it is because
> the entire file needs to be parsed rather than say, some index of entries.

I did some benchmarks on my agendas with and without my enhancements -
my main agenda builds in around 7sec now in comparison with 18-20sec
when using overlays. Though 20sec benchmark was not on current master,
there were several commits reducing the number of overlays in some cases
after I did the benchmark.

Also, you may consider native-comp branch. It can push the speed even
further. My agenda only took 3-4 seconds on native-comp Emacs branch.

> PS> As an outside feature though, interoperability of the org-contact
> formats with other operating system address books, most notable gnome
> contacts/evolution, goog contacts, and OSX address book would be high on my
> list in terms of improving org-contacts though. (eg, raw, structued info in
> all address books, and say perhaps notes or similar maintained and synced
> in ome manner.

I guess it is a job for [future] ox-*.el packages.

Best,
Ihor

Daryl Manning <dwm+orgmode@wakatara.com> writes:

> Primary examples would be adding a note (CTRL-z) or changing a tag on a
> person and then having org-agenda update that. I am assuming it is because
> the entire file needs to be parsed rather than say, some index of entries.
>
> (so perhaps I mischaracterized org-contacts as being slow versus its
> interaction with other programs.)
>
> (for search I use swiper which is very efficient for searching the file
> whenI need it.).
>
> tho quite interested in seeing what perf enhancements you've done on large
> org files would be interesting.
>
> Daryl.
> PS> As an outside feature though, interoperability of the org-contact
> formats with other operating system address books, most notable gnome
> contacts/evolution, goog contacts, and OSX address book would be high on my
> list in terms of improving org-contacts though. (eg, raw, structued info in
> all address books, and say perhaps notes or similar maintained and synced
> in ome manner.
>
>
>
> On Mon, Sep 7, 2020 at 10:27 AM Ihor Radchenko <yantar92@gmail.com> wrote:
>
>> > However, as the file and C-z notes have grown,
>> > performance has really started to drag. I know people have used various
>> > schemes (caching) etc to try to improve performance and the like, but
>> > updates to the file are taking a solid 5 seconds now when making major
>> > updates and moving tags around.
>>
>> Could you provide some examples what exactly is being slow?
>> Maybe my WIP work on improving performance on large org files [1] might
>> help.
>>
>> Best,
>> Ihor
>>
>> [1] https://www.mail-archive.com/emacs-orgmode@gnu.org/msg127740.html
>>
>>
>> Daryl Manning <dwm+orgmode@wakatara.com> writes:
>>
>> > Strangely, I've come to rely over the last year on org-contacts as a
>> > lightweight, taggable CRM. However, as the file and C-z notes have grown,
>> > performance has really started to drag. I know people have used various
>> > schemes (caching) etc to try to improve performance and the like, but
>> > updates to the file are taking a solid 5 seconds now when making major
>> > updates and moving tags around.
>> >
>> > Is there a solid, forked branch anywhere that focuses on enhancing
>> > performance anywhere? I'm tempted to wade in and add features and
>> > improvements myself but my elisp-fu is dodgy at best (more golang these
>> > days.).
>> >
>> > I'd be interested in what people are doing to speed it up (and if it is
>> > under anything like active development for improvements. It does feel
>> super
>> > handy, and feels like it just needs a performance and more modern
>> features
>> > overhaul - more on interoperability and less on in-emacs
>> interoperability.).
>> >
>> > Would love to hear what people have done overall workflow wise if they
>> are
>> > using it seriously.
>> >
>> > thanks,
>> > Daryl.
>>