Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> > Which variables can safely and usefully be made thread-local?
>> 
>> PT, ZV, BEGV
>
> Even that is not enough: you forgot the gap.

Am I missing something, does the gap remain intact when the just move
point? AFAIU, the gap only needs to move when we do edits.

I was thinking about thread-local variables just to move around and read
buffer. Asynchronous writing is probably a poor idea anyway - the very
idea of a gap does not work well when we need to write in multiple
far-away places in buffer.

>> and the buffer-local variables that are represented by the
>> global C variables.
>
> That's a lot!

Do you mean that "a lot" is bad?

>> > I don't see how this could be practically useful.
>> 
>> For example, `org-element-interpret-data' converts Org mode AST to
>> string. Just now, I tried it using AST of one of my large Org buffers.
>> It took 150seconds to complete, while blocking Emacs.
>
> It isn't side-effect-free, though.

It is, just not declared so.

> I don't believe any useful Lisp program in Emacs can be
> side-effect-free, for the purposes of this discussion.  Every single
> one of them accesses the global state and changes the global state.

As I said, I hope that we can convert the important parts of the global
state into thread-local state.

>> Yes. I mean... look at Haskell. There is no shortage of pure functional
>> libraries there.
>
> I cannot follow you there: I don't know Haskell.

In short, pure functions in Haskell can utilize multiple CPUs
automatically, without programmers explicitly writing code for
multi-threading support.
https://wiki.haskell.org/Parallelism

     In Haskell we provide two ways to achieve parallelism:

     - Pure parallelism, which can be used to speed up non-IO parts of the program.
     - Concurrency, which can be used for parallelising IO.
     
     Pure Parallelism (Control.Parallel): Speeding up a pure computation
     using multiple processors. Pure parallelism has these advantages:

     - Guaranteed deterministic (same result every time)
     - no race conditions or deadlocks
     
     Concurrency (Control.Concurrent): Multiple threads of control that execute "at the same time".

     - Threads are in the IO monad
     - IO operations from multiple threads are interleaved non-deterministically
     - communication between threads must be explicitly programmed
     - Threads may execute on multiple processors simultaneously
     - Dangers: race conditions and deadlocks
     
     *Rule of thumb: use Pure Parallelism if you can, Concurrency otherwise.*

>> (declare (pure t))
>
> How many of these do we have, and can useful programs be written using
> only those?

I think I did provide several examples. Po Lu also did.
One additional example: Org mode export (the most CPU-heavy part of it).

> ... More importantly, when you call some function from
> simple.el, how do you know whether all of its subroutines and
> primitives are 'pure'?

We do not, but it may be possible to add assertions that will ensure
purity in whatever sense we need.

Having pure functions is not enough by itself - async support is still
needed and not trivial. However, supporting asynchronous pure function
is easier compared to more general async support. See the above quote
from Haskell wiki.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>