Re: Emacs design and architecture. How about copy-on-write?

[Ihor Radchenko <yantar92@posteo.net>]

Eli Zaretskii <eliz@gnu.org> writes:

>> AFAIU, long-lines performance is largely problematic when running Elisp.
>
> No, the slow parts are pure C.
>
>> (Also, isn't it solved already?)
>
> Not solved, but "bypassed".  The solution does have its disadvantages,
> although we found them to be a much lesser evil than the original
> problem.

May you point to the xdisp.c function that is being slow?

>> Another one you mentioned is displaying many frames. But isn't it
>> optimized by the part of the code that delays redisplay of frames that
>> are not visible? Or does Stefan (or anyone) tend to have so many
>> _visible_ frames?
>
> The latter.

Ok. But is it possible to redisplay the whole frame without ever calling Elisp?

>> > There's more than one reason, that's true.  But it doesn't change the
>> > fact that all those problems have to be resolved before we have
>> > reasonably usable threads.
>> 
>> I disagree. Yes, _all_ the problems are to be solved eventually. But
>> solving _some_ of the problems will already be an improvement. IMHO, it
>> is not all-or-nothing; we can split the problem into several
>> sub-problems and solve them one by one.
>
> You've lost context, and effectively changed the subject, so now this
> sub-thread is not useful anymore.  To recap, I was trying to explain
> why the display aspects of any concurrency cannot be disregarded if we
> want to have a workable solution that can be used in practice.

I believe that I did not lose the context. Unless I misunderstand
something.

What I suggested is to defer the redisplay concurrency and leave it
synchronous (interlocked) for now. Concurrency of Elisp that does not
trigger redisplay can also be useful.

It looks to me that we just disagree about the relative importance of
redisplay concurrency.

>> I don't forget that. However, we do not have to make each and every
>> thread _fully_ asynchronous. It is often enough to process the
>> performance bottlenecks asynchronously. After that processing, we can go
>> ahead and present the results synchronously to user.
>
> This assumes the display is always in the end.  But that is a false
> assumption in Emacs: there are progress messages, there are prompts
> and requests for confirmation, there are display updates to show the
> stuff processed so far, etc.

Again, not always. I do not think that the _initial_ goal should be
making every aspect of Elisp working asynchronously without
interlocking. We certainly want all Elisp to work in threads, possibly
with global lock. But if we get at least some part of useful Elisp to
run truly concurrently, it will already be a win.

>> Let me provide a more concrete example.
>> Consider something as simple as grepping across project files.
>> The process of grepping involves: (1) opening and searching many files;
>> (2) presenting results to the user. Stage (1) does not really involve
>> user interaction or redisplay and can greatly benefit from asynchronous
>> threads - we can simply search multiple files at the same time. Then,
>> even if stage (2) has to be synchronous, it does not matter - stage (1)
>> is what takes most of the time and makes the user wait.
>
> You describe an Emacs without async subprocesses -- this is how it
> works on MSDOS, for example.  On more capable systems we display in
> parallel with running Grep.

I think you misunderstood. I did not mean calling GNU grep. I meant
calling M-x grep - Elisp code searching across many Emacs buffers. It
would be useful if such code could run concurrently.

Another example is org-agenda that is running a large number of regexp
search across many buffers. Some real-world users search across as many
as 500 buffers, with results accumulated into agenda. It would certainly
be useful to run that regexp search concurrently, in multiple buffers at
once. Or maybe even on several regions of a single, large buffer.

>> > I'm not talking about async redisplay, I'm talking about the ability
>> > of non-main threads to display something or do something that would
>> > cause redisplay change the stuff on the glass.
>> 
>> Then, how will it be possible without first having async Elisp threads?
>
> Async Lisp threads is a much harder problem to solve.  If we solve it,
> this one will also be solved; but the opposite is not true.

I agree here. But do note that the interest (and some trial code by Po
Lu) so far has been focused on Elisp threads.

The problem with display code, even if it is easier to transform it to
concurrent, is that understanding display code is by itself is a
time-consuming task. Making changes in it (even small changes) is no
less trivial (I am referring to bug#64596).

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at <https://orgmode.org/>.
Support Org development at <https://liberapay.com/org-mode>,
or support my work at <https://liberapay.com/yantar92>