all messages for Emacs-related lists mirrored at yhetil.org
 help / color / mirror / code / Atom feed
From: Pip Cet <pipcet@gmail.com>
To: Stefan Monnier <monnier@iro.umontreal.ca>
Cc: eliz@gnu.org, emacs-devel@gnu.org
Subject: Re: Concurrent GC via `fork` (was: Opportunistic GC)
Date: Mon, 8 Mar 2021 16:25:57 +0000	[thread overview]
Message-ID: <CAOqdjBd6=F==xnDvvUB73gH3D1U97VzsasxaJY=TjPs7CWYOqw@mail.gmail.com> (raw)
In-Reply-To: <jwvr1kp63t5.fsf-monnier+emacs@gnu.org>

On Mon, Mar 8, 2021 at 4:03 PM Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> > Just a random idea: What if we exploit the fact most people have more
> > than one CPU core these days, garbage-collect in a separate fork()ed
> > process, then do only the sweeping in the main process?
>
> Sounds like a good idea.

Thanks.

Sorry for derailing your thread, I should have chosen a new subject.

> A concurrent GC which relies on `fork` to take the snapshot.
> I don't think we'll be able to always use it, so the synchronous code
> path will have to stay,

Absolutely.

And I do want to mention weak hash tables: they're a problem, because
they can resurrect unreachable objects. Worse, while normal
weak-key-strong-value hash tables only resurrect objects when you
maphash, weak-value hash tables can do so for gethash as well, so that
code path would become more expensive and more difficult to translate
into native code.

> but I think it's definitely worth working on it.

I've got it working for conses, I think (as a proof of concept,
synchronously, with an extra waitpid()).

> It will probably require moving the markbits out of the objects in order
> to work well (which would probably be beneficial in the synchronous GC
> case as well), but other than that I'd expect it should be possible to
> get pretty good performance out of it in the "normal" case:

The normal case, for me, is Emacs freeing a lot of memory in a GC
cycle, and keeping only a relatively small working set. That should
work well with the way markbits are written only for objects that are
to be kept.

> - most pages should stay shared between the two processes (because the
>   GC process should basically only modify its stack and its markbit
>   pages, and the parent process should hopefully not have enough time to
>   modify a large fraction of its working set).
> - the "normal" case I envision is one where there's a fair bit of extra
>   RAM left in the system and a few CPU cores waiting to help Emacs.

Indeed.

Apart from hating threads and loving processes, I love nice levels,
and I think this would be a good use case for them. For example, I
hear there's a new CPU out there that has two separate sets of cores
for "efficiency" and "performance", and it would be great if Emacs ran
an "efficiency" garbage collection in the background rather than a
"performance" garbage collection synchronously.

> > Eli, would this qualify as a "small" GC change, and thus be vetoed?
>
> I like the idea, but its inclusion can't be discussed without
> a proof-of-concept patch.

I think that's the sensible approach, yes. I certainly wasn't hoping
to hear anything _positive_ before submitting a patch.

Pip



  reply	other threads:[~2021-03-08 16:25 UTC|newest]

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-08  3:35 Opportunistic GC Stefan Monnier
2021-03-08  7:20 ` Pip Cet
2021-03-08  8:26   ` martin rudalics
2021-03-08  9:02     ` Pip Cet
2021-03-08  9:51       ` martin rudalics
2021-03-08 10:44         ` Pip Cet
2021-03-08 11:37           ` martin rudalics
2021-03-08 12:27             ` Pip Cet
2021-03-08 18:06               ` martin rudalics
2021-03-08 18:24                 ` Concurrent GC via fork (was: Opportunistic GC) Stefan Monnier
2021-03-09  8:10                   ` Concurrent GC via fork martin rudalics
2021-03-10 20:35                 ` Opportunistic GC Pip Cet
2021-03-08 11:38           ` Stefan Kangas
2021-03-08 12:55             ` Pip Cet
2021-03-08 14:06             ` Andrea Corallo via Emacs development discussions.
2021-03-08 14:49           ` Eli Zaretskii
2021-03-08 15:02             ` Pip Cet
2021-03-08 15:23               ` Eli Zaretskii
2021-03-08 15:39                 ` Pip Cet
2021-03-08 16:27                   ` Eli Zaretskii
2021-03-08 16:35                     ` Pip Cet
2021-03-08 17:06                       ` Eli Zaretskii
2021-03-08 15:46               ` Stefan Monnier
2021-03-08 16:11                 ` Pip Cet
2021-03-08 16:37                   ` Stefan Monnier
2021-03-08 14:01   ` Andrea Corallo via Emacs development discussions.
2021-03-08 15:04     ` Pip Cet
2021-03-08 15:50       ` Stefan Monnier
2021-03-08 15:52       ` Andrea Corallo via Emacs development discussions.
2021-03-08 16:14         ` Philipp Stephani
2021-03-08 16:18           ` Andrea Corallo via Emacs development discussions.
2021-03-08 16:40             ` Philipp Stephani
2021-03-08 16:44         ` Stefan Monnier
2021-03-08 17:13           ` Eli Zaretskii
2021-03-10 20:25             ` Pip Cet
2021-03-10 20:48               ` Eli Zaretskii
2021-03-11  7:55                 ` Pip Cet
2021-03-11  8:37                   ` Eli Zaretskii
2021-03-11  9:03                     ` Pip Cet
2021-03-11  9:34                       ` Eli Zaretskii
2021-03-11 10:11                         ` Pip Cet
2021-03-11 11:35                           ` Eli Zaretskii
2021-03-09  6:22           ` Richard Stallman
2021-03-09  8:11             ` Pip Cet
2021-03-10  5:52               ` Richard Stallman
2021-03-08 14:45   ` Eli Zaretskii
2021-03-08 14:57     ` Pip Cet
2021-03-08 15:16       ` Eli Zaretskii
2021-03-09 13:01       ` Eli Zaretskii
2021-03-10 20:21         ` Pip Cet
2021-03-10 20:41           ` Eli Zaretskii
2021-03-11  8:06             ` Pip Cet
2021-03-11  9:03               ` Eli Zaretskii
2021-03-08 16:03   ` Concurrent GC via `fork` (was: Opportunistic GC) Stefan Monnier
2021-03-08 16:25     ` Pip Cet [this message]
2021-03-08 17:37       ` Concurrent GC via `fork` Stefan Monnier
2021-03-08 19:21       ` Lars Ingebrigtsen
2021-03-10 17:18   ` Opportunistic GC Matt Armstrong
2021-03-10 19:38     ` Pip Cet
2022-07-05  1:47   ` Stefan Monnier
2022-07-05 13:49     ` T.V Raman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAOqdjBd6=F==xnDvvUB73gH3D1U97VzsasxaJY=TjPs7CWYOqw@mail.gmail.com' \
    --to=pipcet@gmail.com \
    --cc=eliz@gnu.org \
    --cc=emacs-devel@gnu.org \
    --cc=monnier@iro.umontreal.ca \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this external index

	https://git.savannah.gnu.org/cgit/emacs.git
	https://git.savannah.gnu.org/cgit/emacs/org-mode.git

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.