Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> I may be wrong, but from my previous experience with performance
>> benchmarks, memory allocation often takes a significant fraction of CPU
>> time. And memory allocation is a routine process on pretty much every
>> iteration of CPU-intensive code.
>
> Do you have any evidence for that which you can share?  GC indeed
> takes significant time, but memory allocation? never heard of that.

This is from my testing of Org parser.
I noticed that storing a pair of buffer positions is noticeably faster
compared to storing string copies of buffer text.

The details usually do not show up in M-x profiler reports, but I now
tried perf out of curiosity:

    14.76%  emacs         emacs                                  [.] re_match_2_internal
     9.39%  emacs         emacs                                  [.] re_compile_pattern
     4.45%  emacs         emacs                                  [.] re_search_2
     3.98%  emacs         emacs                                  [.] funcall_subr

     
     AFAIU, this is memory allocation. Taking a good one second in this case.
     3.37%  emacs         emacs                                  [.] allocate_vectorlike


     3.17%  emacs         emacs                                  [.] Ffuncall
     3.01%  emacs         emacs                                  [.] exec_byte_code
     2.90%  emacs         emacs                                  [.] buf_charpos_to_bytepos
     2.82%  emacs         emacs                                  [.] find_interval
     2.74%  emacs         emacs                                  [.] re_iswctype
     2.57%  emacs         emacs                                  [.] set_default_internal
     2.48%  emacs         emacs                                  [.] plist_get
     2.24%  emacs         emacs                                  [.] Fmemq
     1.95%  emacs         emacs                                  [.] process_mark_stack

These are just CPU cycles. I am not sure if there are any other
overheads related to memory allocation that translate into extra user time. 

>> Would it be of interest to allow locking objects for read/write using
>> semantics similar to `with-mutex'?
>
> Locking slows down software and should be avoided, I'm sure you know
> it.

I am not sure anymore.
Po Lu appears to advocate for locking instead of isolated process
approach, referring to other software.

> ... But the global lock used by the Lisp threads we have is actually
> such a lock, and the results are well known.

To be fair, global lock is an extreme worst-case scenario.
Locking specific Lisp objects is unlikely to lock normal Emacs usage,
especially when the async code is written carefully. Except when there
is a need to lock something from global and frequently used Emacs state,
like heap or obarray. Which is why I asked about memory allocation.

The need to avoid locked heap is the main reason I proposed isolated
processes.

... well, also things like lisp_eval_depth and other global interpreter
state variables.

But, AFAIU, Po Lu is advocating that trying isolated processes is not
worth the effort and it is better to byte the bullet and implement
proper locking.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>