From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.org!.POSTED!not-for-mail From: Pip Cet Newsgroups: gmane.emacs.devel Subject: Re: [EXPERIMENT] Emacs with the SpiderMonkey garbage collector Date: Sun, 3 Dec 2017 00:37:21 +0000 Message-ID: References: <8fa934aa-9bbd-ca8e-2468-d6370324e6a5@mozilla.com> NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Trace: blaine.gmane.org 1512261488 5714 195.159.176.226 (3 Dec 2017 00:38:08 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sun, 3 Dec 2017 00:38:08 +0000 (UTC) Cc: emacs-devel@gnu.org To: Steve Fink Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Sun Dec 03 01:38:04 2017 Return-path: Envelope-to: ged-emacs-devel@m.gmane.org Original-Received: from lists.gnu.org ([208.118.235.17]) by blaine.gmane.org with esmtp (Exim 4.84_2) (envelope-from ) id 1eLIIG-0001D0-3R for ged-emacs-devel@m.gmane.org; Sun, 03 Dec 2017 01:38:04 +0100 Original-Received: from localhost ([::1]:37365 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eLIIN-0008UL-GE for ged-emacs-devel@m.gmane.org; Sat, 02 Dec 2017 19:38:11 -0500 Original-Received: from eggs.gnu.org ([2001:4830:134:3::10]:42012) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1eLIIG-0008UG-Qr for emacs-devel@gnu.org; Sat, 02 Dec 2017 19:38:06 -0500 Original-Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1eLIIF-0006at-Af for emacs-devel@gnu.org; Sat, 02 Dec 2017 19:38:04 -0500 Original-Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]:42365) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1eLIIF-0006aE-0o for emacs-devel@gnu.org; Sat, 02 Dec 2017 19:38:03 -0500 Original-Received: by mail-wm0-x22d.google.com with SMTP id l141so209288wmg.1 for ; Sat, 02 Dec 2017 16:38:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=iGt/PHloAZlgxH06loRfvCZQQWuylcw62KRRxGmcHiw=; b=mXry/+7x/gwTIEHo0ZX9mZGyx3nss1OvecxAYF6cqzPm9cW+Fiy5W5XFPub74bdvFK +3tWL0MPFfb396vEHvz7LhdRuNirHdrTIXFwCvTaSc/+D30z5CEZxmaFPcyzYKkUgOGA Z/xzZuY6JAdrEpCCNlmr7BKOJTYJjXspAvPi265QhbXWJtzH0jynk2Kgi3SkaPwtLIyp i7MCmzoNMN7lWVaa26sM0FqvxlXgQ6XyYUqrNl0B3lUaFgtH+7yV5A3UQWT6GI1WHnwF cV27OI9DdaFUwiAXzH92p7jhhnOKLnozFQkqP4aojQ/pCIpbkNKQVF6JKH9SsJckemJ6 mAYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=iGt/PHloAZlgxH06loRfvCZQQWuylcw62KRRxGmcHiw=; b=T16GAsc+f+MhWoFhFvicKJRsoY7e7WLAlqtRQ7hsTTdrHPgCgjBg5XyRxNPeH+cAka QXb9WHrrsh9vLTHOgYgDre4Z3ps+C6Y0E2l7XsE305zBgPXXSXazpf6juqrcYmzlZgvV B4kpajwrQJFADU9BoFy8n+mfnjNnpvYNOQ6tsSpnHWsy+nzPRGgtMzcH9Muqq7TR2qaE vr4I+jkvNSEtD+nhV38xfxu+tZmtUjSFpy9znx26A2lNNSfCc2RDfb2bGM452qcJSoGV zyzVIB64mkTU8bBtfdHFUx5gRxB2ejkPDpQ5IUafd+4IdMV9vrPhIT6CpXjO0Lqh7LTH nVcA== X-Gm-Message-State: AJaThX7UISCR7yPMZwxTMT1ZBKwQryJdSAbFuNrEfoMzr+5GVYOSNoyn 1OMgPeDR70eJm6a5F5rlyml0mZPsjGijP4eyIok= X-Google-Smtp-Source: AGs4zMbgqjGwnRD4SBdn+Ja3EdGrmQgBv6UrdcvNpeR0Gk5IQbCGNdoAlz7M485K3py+JAMhXq0bRR7hfK2iqLBgaWk= X-Received: by 10.80.216.11 with SMTP id o11mr23363207edj.235.1512261481595; Sat, 02 Dec 2017 16:38:01 -0800 (PST) Original-Received: by 10.80.150.6 with HTTP; Sat, 2 Dec 2017 16:37:21 -0800 (PST) In-Reply-To: <8fa934aa-9bbd-ca8e-2468-d6370324e6a5@mozilla.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22d X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane.org@gnu.org Original-Sender: "Emacs-devel" Xref: news.gmane.org gmane.emacs.devel:220639 Archived-At: On Fri, Dec 1, 2017 at 5:57 PM, Steve Fink wrote: > Very interesting! I'm surprised and impressed that someone could get this to > work, especially in a C project. (Servo is a C project that uses > spidermonkey, but they do fancy things with the Rust type system and a > bindings generator to bridge the gap.) > Within Gecko (the Firefox rendering engine) and internally to SpiderMonkey, > it's very similar though we don't do anything special for return values. For > a base type of JSObject (the structure actually stored in the GC heap), we > have > > - JSObject* - return value > > - Rooted - stack value or member of stack struct. > > Rooted is an RAII class that inserts/removes from a list of stack roots. > > - Heap - member of struct in the heap > > - Handle - constant parameter, just a type-safe alias for > JSObject** > > - MutableHandle - mutable parameter (out parameter), again an > alias for JSObject** > > If you know you're doing stuff that can't GC, then you can just use plain > JSObject*. > > For JS::Value, which boxes up various types of values or pointers, they > would be Value, Rooted, Heap, Handle, and > MutableHandle. I'm not currently using MutableHandle<>, though I'd like to do so for array/vector members, which appear to have only non-mutable handle value types pre-defined. >> 6. Calling convention >> >> The usual SpiderMonkey calling convention is that functions do not >> return GC types; their arguments are "handles", essentially read-only >> pointers to JS::Values. I decided to return JS::Value objects >> directly (except for being wrapped in another class), which opens up >> a race condition: If f1 and f2 are functions returning >> ELisp_Return_Type values, it's illegal to call another function g as >> g(f1(...), f2(...)). f1's return value will not be traced if f2 >> triggers a GC, so if it is moved there will be a segfault (or worse). >> It should be possible to further extend my C-to-C++ script to deal >> with this by automatically assigning ELisp_Value temporaries in this >> situation. I also decided to pass function arguments as JS::Value >> objects, rooting them in the callee instead. Both of these decisions >> are open to revision. I've since decided to go with the SpiderMonkey calling convention, and root arguments in the caller, since that allows us to use handles and those appear to be a performance (and readability) gain. > A fair number of functions do return GC pointers directly, there's really > nothing against doing that (in our code base, that is; see below). But most > of those are allocation functions. The reason why we don't generally return > GC pointers is not because of the precise GC, but rather because we compile > without exceptions so the return value is reserved for a boolean status. Thanks for the clarification! > You are absolutely right about the hazards of g(f1(...), f2(...)). That is > but one of the tricky patterns that we had littering our code, even after we > supposedly went through and fixed everything up. We are absolutely dependent > on a static "rooting hazard" analysis that we run on every checkin. The > above is one tricky pattern we rely on it for detecting. Another is methods > on a GC thing; the 'this' pointer will implicitly be on the stack and so > could get invalidated anytime you GC. Probably the trickiest is when you > have an RAII class that can GC in its destructor, and you're returning a GC > pointer -- the pointer could get invalidated in between a 'return' statement > and the caller resuming! The latter is obviously not an issue for a C > embedding. Well, I'm rewriting the C code to use (some) constructors, so it's good to be aware of the possibility. > The analysis is implemented as a GCC plugin and postprocessing scripts, and > in theory can be run on an embedding project's code. In practice, it's > generally a pain to integrate it into the build system; there are wrappers > you can put in your path to have it automatically called in place of gcc/g++ > (the wrappers just invoke gcc with the correct -fplugin arguments), but in > practice it tends to be a pain to get that to cooperate happily with eg > configure scripts. I recently tried doing it for the GNOME shell, which > embeds spidermonkey, but couldn't get their build system happy after an hour > or two of trying. If you're interested in trying, you could find me (sfink) > on irc.mozilla.org in the #jsapi channel. It's very straightforward to get a > list of problems to fix, one after another, which should resolve most weird > crashes. That sounds awesome! As Stefan pointed out, it's really essential to have static analysis catch at least the most obvious bugs when some people use a different GC from others, so that definitely sounds worth checking out. > The one big thing that the analysis doesn't handle particularly well is > internal pointers. For now, it sounds like you're mostly safe from these > moving because all your data is hanging off of the JS_GetPrivate > indirection, so you wouldn't have any pointers internal to GC things. But > you can still run into issues keeping things alive. We mostly have this > problem with strings, since small strings are stored inline in the GC things > and so it's very easy to end up with a char* that will move. We tend to work > around this by either (1) passing around the GC pointer instead of the > char*; (2) making functions that accept such a char* to also declare that > they won't GC by requiring a reference to an 'AutoRequireCannotGC&' token, > which is validated by the static analysis; or (3) forcing the contents to be > stored in the malloc heap if they aren't already, and just being careful > with keeping the owning JSString* in a Rooted somewhere higher on > the stack. > > Collections are also an issue, if you want to index them by GC pointer > value. It seems I have two options: rewrite all hashed collections whenever something moves, or make up a hash value and store it in a private slot for each object upon creation. My understanding is SpiderMonkey does the former for WeakMaps, and those seem to perform okay, so that might be the better option long-term, but I haven't given much thought to this and the made-up hash value seems easier to implement... > One last note: I don't know how you're doing timings, but DEBUG spidermonkey > builds are much, much, much slower. We do a crazy amount of sanity checking, > including redundant whole-heap scans to verify various invariants. So > develop with DEBUG (the assertions are essential for using JSAPI correctly, > even for experienced users) but ignore any speed estimates until you try a > non-DEBUG, preferably optimized build. I'm back to developing with DEBUG after trying for a few builds to see what performance is like. (And it's caught bugs, so I'll be careful to keep it that way.) I suspect I'll get back to you about the performance issues, but so far the most obvious ones are my fault.