From: Pip Cet <pipcet@gmail.com>
To: Steve Fink <sfink@mozilla.com>
Cc: emacs-devel@gnu.org
Subject: Re: [EXPERIMENT] Emacs with the SpiderMonkey garbage collector
Date: Sun, 3 Dec 2017 00:37:21 +0000 [thread overview]
Message-ID: <CAOqdjBcdoKZ=Jdv3eFCGogHp1t7dVf=DG-cYi+7g8No=6F=7rw@mail.gmail.com> (raw)
In-Reply-To: <8fa934aa-9bbd-ca8e-2468-d6370324e6a5@mozilla.com>
On Fri, Dec 1, 2017 at 5:57 PM, Steve Fink <sfink@mozilla.com> wrote:
> Very interesting! I'm surprised and impressed that someone could get this to
> work, especially in a C project. (Servo is a C project that uses
> spidermonkey, but they do fancy things with the Rust type system and a
> bindings generator to bridge the gap.)
> Within Gecko (the Firefox rendering engine) and internally to SpiderMonkey,
> it's very similar though we don't do anything special for return values. For
> a base type of JSObject (the structure actually stored in the GC heap), we
> have
>
> - JSObject* - return value
>
> - Rooted<JSObject*> - stack value or member of stack struct.
>
> Rooted is an RAII class that inserts/removes from a list of stack roots.
>
> - Heap<JSObject*> - member of struct in the heap
>
> - Handle<JSObject*> - constant parameter, just a type-safe alias for
> JSObject**
>
> - MutableHandle<JSObject*> - mutable parameter (out parameter), again an
> alias for JSObject**
>
> If you know you're doing stuff that can't GC, then you can just use plain
> JSObject*.
>
> For JS::Value, which boxes up various types of values or pointers, they
> would be Value, Rooted<Value>, Heap<Value>, Handle<Value>, and
> MutableHandle<Value>.
I'm not currently using MutableHandle<>, though I'd like to do so for
array/vector members, which appear to have only non-mutable handle
value types pre-defined.
>> 6. Calling convention
>>
>> The usual SpiderMonkey calling convention is that functions do not
>> return GC types; their arguments are "handles", essentially read-only
>> pointers to JS::Values. I decided to return JS::Value objects
>> directly (except for being wrapped in another class), which opens up
>> a race condition: If f1 and f2 are functions returning
>> ELisp_Return_Type values, it's illegal to call another function g as
>> g(f1(...), f2(...)). f1's return value will not be traced if f2
>> triggers a GC, so if it is moved there will be a segfault (or worse).
>> It should be possible to further extend my C-to-C++ script to deal
>> with this by automatically assigning ELisp_Value temporaries in this
>> situation. I also decided to pass function arguments as JS::Value
>> objects, rooting them in the callee instead. Both of these decisions
>> are open to revision.
I've since decided to go with the SpiderMonkey calling convention, and
root arguments in the caller, since that allows us to use handles and
those appear to be a performance (and readability) gain.
> A fair number of functions do return GC pointers directly, there's really
> nothing against doing that (in our code base, that is; see below). But most
> of those are allocation functions. The reason why we don't generally return
> GC pointers is not because of the precise GC, but rather because we compile
> without exceptions so the return value is reserved for a boolean status.
Thanks for the clarification!
> You are absolutely right about the hazards of g(f1(...), f2(...)). That is
> but one of the tricky patterns that we had littering our code, even after we
> supposedly went through and fixed everything up. We are absolutely dependent
> on a static "rooting hazard" analysis that we run on every checkin. The
> above is one tricky pattern we rely on it for detecting. Another is methods
> on a GC thing; the 'this' pointer will implicitly be on the stack and so
> could get invalidated anytime you GC. Probably the trickiest is when you
> have an RAII class that can GC in its destructor, and you're returning a GC
> pointer -- the pointer could get invalidated in between a 'return' statement
> and the caller resuming! The latter is obviously not an issue for a C
> embedding.
Well, I'm rewriting the C code to use (some) constructors, so it's
good to be aware of the possibility.
> The analysis is implemented as a GCC plugin and postprocessing scripts, and
> in theory can be run on an embedding project's code. In practice, it's
> generally a pain to integrate it into the build system; there are wrappers
> you can put in your path to have it automatically called in place of gcc/g++
> (the wrappers just invoke gcc with the correct -fplugin arguments), but in
> practice it tends to be a pain to get that to cooperate happily with eg
> configure scripts. I recently tried doing it for the GNOME shell, which
> embeds spidermonkey, but couldn't get their build system happy after an hour
> or two of trying. If you're interested in trying, you could find me (sfink)
> on irc.mozilla.org in the #jsapi channel. It's very straightforward to get a
> list of problems to fix, one after another, which should resolve most weird
> crashes.
That sounds awesome! As Stefan pointed out, it's really essential to
have static analysis catch at least the most obvious bugs when some
people use a different GC from others, so that definitely sounds worth
checking out.
> The one big thing that the analysis doesn't handle particularly well is
> internal pointers. For now, it sounds like you're mostly safe from these
> moving because all your data is hanging off of the JS_GetPrivate
> indirection, so you wouldn't have any pointers internal to GC things. But
> you can still run into issues keeping things alive. We mostly have this
> problem with strings, since small strings are stored inline in the GC things
> and so it's very easy to end up with a char* that will move. We tend to work
> around this by either (1) passing around the GC pointer instead of the
> char*; (2) making functions that accept such a char* to also declare that
> they won't GC by requiring a reference to an 'AutoRequireCannotGC&' token,
> which is validated by the static analysis; or (3) forcing the contents to be
> stored in the malloc heap if they aren't already, and just being careful
> with keeping the owning JSString* in a Rooted<JSString*> somewhere higher on
> the stack.
>
> Collections are also an issue, if you want to index them by GC pointer
> value.
It seems I have two options: rewrite all hashed collections whenever
something moves, or make up a hash value and store it in a private
slot for each object upon creation. My understanding is SpiderMonkey
does the former for WeakMaps, and those seem to perform okay, so that
might be the better option long-term, but I haven't given much thought
to this and the made-up hash value seems easier to implement...
> One last note: I don't know how you're doing timings, but DEBUG spidermonkey
> builds are much, much, much slower. We do a crazy amount of sanity checking,
> including redundant whole-heap scans to verify various invariants. So
> develop with DEBUG (the assertions are essential for using JSAPI correctly,
> even for experienced users) but ignore any speed estimates until you try a
> non-DEBUG, preferably optimized build.
I'm back to developing with DEBUG after trying for a few builds to see
what performance is like. (And it's caught bugs, so I'll be careful to
keep it that way.) I suspect I'll get back to you about the
performance issues, but so far the most obvious ones are my fault.
next prev parent reply other threads:[~2017-12-03 0:37 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CAOqdjBe98BpWE- Ey8fPY1DmfCiLYnB06d30Xqf_KMm_muvKbDg@mail.gmail.com>
2017-12-01 17:57 ` [EXPERIMENT] Emacs with the SpiderMonkey garbage collector Steve Fink
2017-12-03 0:37 ` Pip Cet [this message]
2017-12-03 5:24 ` Steve Fink
2017-11-23 19:01 Pip Cet
2017-11-24 8:07 ` Paul Eggert
2017-11-24 16:23 ` Pip Cet
2017-11-24 18:20 ` Paul Eggert
2017-11-24 23:27 ` Pip Cet
2017-11-25 0:21 ` Paul Eggert
2017-11-25 23:50 ` Pip Cet
2017-11-24 22:13 ` Stefan Monnier
2017-11-24 23:05 ` Pip Cet
2017-11-25 4:15 ` Stefan Monnier
2017-11-25 23:50 ` Pip Cet
2017-11-26 1:20 ` Stefan Monnier
2017-11-26 4:20 ` Paul Eggert
2017-11-26 5:11 ` Stefan Monnier
2017-11-26 10:27 ` martin rudalics
[not found] ` <jwva7z9rgqh.fsf-monnier+Inbox@gnu.org>
[not found] ` <9d7be625-85ae-54d5-3897-6f701c8ea124@cs.ucla.edu>
[not found] ` <jwvo9npprfw.fsf-monnier+emacs@gnu.org>
2017-12-01 1:03 ` Steve Fink
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://www.gnu.org/software/emacs/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAOqdjBcdoKZ=Jdv3eFCGogHp1t7dVf=DG-cYi+7g8No=6F=7rw@mail.gmail.com' \
--to=pipcet@gmail.com \
--cc=emacs-devel@gnu.org \
--cc=sfink@mozilla.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).