bug#57196: 28.1.90; An idea to allow background (low-priority) threads

bug#57196: 28.1.90; An idea to allow background (low-priority) threads

[Ihor Radchenko]

Hi,

Emacs does have a limited concurrency support via Threads, which is,
unfortunately, mostly a toy feature I haven't seen being used a lot.

All my attempts to implement some real functionality using threads
failed because it is very hard to create real-life threads that do not
noticeably block Emacs. Most of the time, Emacs gets quite sluggish,
even if the thread yields frequently.

Such situation sounds similar to what one may get with 1CPU and
multiple processed fighting for the CPU time. This is something that
used to be solved with nice command.

Could something like nice be implemented for Elisp threads?

Or, alternatively, could thread execution be limited in some ways?
I have the approach taken by org-element cache processing in mind.
org-element limits cache processing using the following variables:

(defvar org-element-cache-sync-idle-time 0.6
  "Length, in seconds, of idle time before syncing cache.")

(defvar org-element-cache-sync-duration 0.04
  "Maximum duration, as a time value, for a cache synchronization.
If the synchronization is not over after this delay, the process
pauses and resumes after `org-element-cache-sync-break'
seconds.")

(defvar org-element-cache-sync-break 0.3
  "Duration, as a time value, of the pause between synchronizations.
See `org-element-cache-sync-duration' for more information.")

I imagine that something similar could be done for threads.
`make-thread' could allow some kind of priority setting that will limit
its execution time to comfortable levels, so that the user in main
thread can actually interact with Emacs without noticing anything.

WDYT?

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

bug#57196: 28.1.90; An idea to allow background (low-priority) threads

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@gmail.com>
> Date: Sun, 14 Aug 2022 12:12:20 +0800
> 
> Emacs does have a limited concurrency support via Threads, which is,
> unfortunately, mostly a toy feature I haven't seen being used a lot.
> 
> All my attempts to implement some real functionality using threads
> failed because it is very hard to create real-life threads that do not
> noticeably block Emacs. Most of the time, Emacs gets quite sluggish,
> even if the thread yields frequently.
> 
> Such situation sounds similar to what one may get with 1CPU and
> multiple processed fighting for the CPU time. This is something that
> used to be solved with nice command.
> 
> Could something like nice be implemented for Elisp threads?

Our "scheduler", such as it is, is in thread.c:really_call_select.  It
basically releases the global lock and lets the first thread waiting
on the lock to acquire the lock.  If someone wants to implement a
smarter lock-grabbing logic with some kind of "nice" capability, AFAIU
that's the place to look and modify.

TBH, I'm not really sure your analysis, which basically says this is a
problem with thread "equality", is correct.  Did you try to see what
causes the sluggish operation in the cases where you saw it?

> I imagine that something similar could be done for threads.
> `make-thread' could allow some kind of priority setting that will limit
> its execution time to comfortable levels, so that the user in main
> thread can actually interact with Emacs without noticing anything.

What mechanism will stop the running thread that has exceeded its
allotted time?  In the current implementation, the thread stops itself
when it calls a small set of primitives, but with your proposal we'd
need to make the "scheduler" run in a separate thread, which would
mean a complete redesign of how threads are scheduled.

bug#57196: 28.1.90; An idea to allow background (low-priority) threads

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

> Our "scheduler", such as it is, is in thread.c:really_call_select.  It
> basically releases the global lock and lets the first thread waiting
> on the lock to acquire the lock.  If someone wants to implement a
> smarter lock-grabbing logic with some kind of "nice" capability, AFAIU
> that's the place to look and modify.

Do I understand correctly that thread.c:really_call_select simply relies
on pthread itself to select the next thread?

Then, a simple pthread_setschedparam inside
systhread.c:sys_thread_create may do the job given that we pass an extra
optional parameter to make-thread.

> TBH, I'm not really sure your analysis, which basically says this is a
> problem with thread "equality", is correct.  Did you try to see what
> causes the sluggish operation in the cases where you saw it?

I once tried to write some code for async magit status buffer and for
async agenda generation.

I am not sure how I can properly determine the cause of sluggish
operation, but AFAIU the cause was the following:

1. There is main thread and the magit/agenda thread
2. Magit/agenda thread uses a lot of CPU while the main thread is not (I
   was typing/navigating the buffer)
3. Every keypress in the main thread caused thread switch to the
   CPU-hungry magit/agenda thread
4. Frequent switches caused high typing latency because every single key
   stroke in the main thread had to wait until the magit/agenda yields.

>> I imagine that something similar could be done for threads.
>> `make-thread' could allow some kind of priority setting that will limit
>> its execution time to comfortable levels, so that the user in main
>> thread can actually interact with Emacs without noticing anything.
>
> What mechanism will stop the running thread that has exceeded its
> allotted time?  In the current implementation, the thread stops itself
> when it calls a small set of primitives, but with your proposal we'd
> need to make the "scheduler" run in a separate thread, which would
> mean a complete redesign of how threads are scheduled.

I do not suggest to stop the running thread externally.

Instead, we may accumulate the thread execution time. Let's call this
"thread 1".

Then, every time the _other_ running thread yields (stops itself) and
"thread 1" is about to be called, we do the following:
1. If "thread 1" execution time is less than "duration" thread
   property, run the thread.
2. If "thread 1" execution time is more than "duration", skip running
   the thread remembering the time now (pause time).
3. If "thread 1" execution time is more than "duration" and "pause time"
   was more than "break time" thread property ago, set execution time to
   0 and allow the thread to be running.

"idle-time" may also be used, pausing thread until that much idle-time
passed.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92

bug#57196: 28.1.90; An idea to allow background (low-priority) threads

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@gmail.com>
> Cc: 57196@debbugs.gnu.org
> Date: Sun, 14 Aug 2022 15:57:37 +0800
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > Our "scheduler", such as it is, is in thread.c:really_call_select.  It
> > basically releases the global lock and lets the first thread waiting
> > on the lock to acquire the lock.  If someone wants to implement a
> > smarter lock-grabbing logic with some kind of "nice" capability, AFAIU
> > that's the place to look and modify.
> 
> Do I understand correctly that thread.c:really_call_select simply relies
> on pthread itself to select the next thread?

What do you mean by "pthread itself"?  AFAIU, threads are scheduled
by the OS, not by pthreads.  So which thread will be the next to grab
the lock is up to the OS, the way we programmed it.

> Then, a simple pthread_setschedparam inside
> systhread.c:sys_thread_create may do the job given that we pass an extra
> optional parameter to make-thread.

You could try that and see if that helps, although I wouldn't know how
you'd select the policy in this case.

> 1. There is main thread and the magit/agenda thread
> 2. Magit/agenda thread uses a lot of CPU while the main thread is not (I
>    was typing/navigating the buffer)
> 3. Every keypress in the main thread caused thread switch to the
>    CPU-hungry magit/agenda thread
> 4. Frequent switches caused high typing latency because every single key
>    stroke in the main thread had to wait until the magit/agenda yields.

The only way of improving the UX in this case is for the other thread
to yield more frequently.

> I do not suggest to stop the running thread externally.
> 
> Instead, we may accumulate the thread execution time. Let's call this
> "thread 1".
> 
> Then, every time the _other_ running thread yields (stops itself) and
> "thread 1" is about to be called, we do the following:
> 1. If "thread 1" execution time is less than "duration" thread
>    property, run the thread.
> 2. If "thread 1" execution time is more than "duration", skip running
>    the thread remembering the time now (pause time).
> 3. If "thread 1" execution time is more than "duration" and "pause time"
>    was more than "break time" thread property ago, set execution time to
>    0 and allow the thread to be running.

You can do this in your thread function: when it gets to run, check
whether it "earned" its time slot, and if not, yield immediately
without calling the workhorse code.