Re: Concurrency via isolated process/thread

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> 1. Limit the async process memory to a small lexical subset of symbols
>>    (within a function).
>> 
>> 2. Every time the async process needs to read/write a symbol slot
>>    outside its lexical scope, query the main Emacs process.
>
> If it queries the main process, it will have to wait when the main
> process is busy.  So this is not really asynchronous.

Sure. But not every asynchronous process would need to query the main
process frequently. As a dumb example:

(defun test/F (n) "Factorial"
  (declare (pure t))
  (let ((mult 1))
    (dotimes (i n)
      (setq mult (* mult i)))
    mult))

This function, if called asynchronously, will need to query input (N)
and later return the output. The main loop will not require any
interactions and will take most of the CPU time.

If the async process have a separate garbage collector, such process
will free the main Emacs process from allocating all the memory for i
and mult values at each loop iteration.

>> More concretely, is it possible to copy internal Elisp object
>> representations between Emacs processes and arrange mutability to query
>> the right Emacs process that "owns" the object?
>
> This is software: anything's possible ;-).  But Someone™ needs to
> write the code, like marshalling and unmarshalling of such objects
> between two processes.  (We do something like that when we write then
> load the pdumper file.)  There's more than one way of skinning this
> particular cat.

As the first step, I wanted to hear if there is any blocker that
prevents memcpy between processes without going through print/read.

>> The inter-process communication does not have to be asynchronous, but
>> may work similar to the existing thread implementation.
>
> I wouldn't recommend designing anything by the example of Lisp
> threads.  'Nough said.

IMHO, the main problem with threads is that they cannot be interrupted
or fire too frequently.

In the proposed idea, this will not be such a big deal - inter-process
communication is a known bottleneck for any asynchronous code and should
be avoided anyway.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>