From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: Re: Some experience with the igc branch Date: Fri, 27 Dec 2024 14:34:22 +0000 Message-ID: <87ldw15h6s.fsf@protonmail.com> References: <87o713wwsi.fsf@telefonica.net> <861pxx2lh7.fsf@gnu.org> <86ldw40xbo.fsf@gnu.org> <87cyhf8xw5.fsf@protonmail.com> <86y103zm4m.fsf@gnu.org> <8734ia79jq.fsf@protonmail.com> <86pllexwru.fsf@gnu.org> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="7503"; mail-complaints-to="usenet@ciao.gmane.io" Cc: gerd.moellmann@gmail.com, ofv@wanadoo.es, emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Fri Dec 27 15:47:21 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tRBcW-0001oV-Ic for ged-emacs-devel@m.gmane-mx.org; Fri, 27 Dec 2024 15:47:20 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tRBbg-0000tS-TB; Fri, 27 Dec 2024 09:46:28 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRBQ8-0007c5-Iv for emacs-devel@gnu.org; Fri, 27 Dec 2024 09:34:33 -0500 Original-Received: from mail-40133.protonmail.ch ([185.70.40.133]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tRBQ6-00050f-8C for emacs-devel@gnu.org; Fri, 27 Dec 2024 09:34:32 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1735310067; x=1735569267; bh=ZYWv6C63j1WgW0qUbxWT5C8mKbScAJOdaYu08f3NnNc=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=XngruzOOnF6eks1c/t+5tkkgpb7g4VCuMtxZycqVZoY7mtaSIRkPKQTdGQsUA0/CV YxV6iHKwXEz7hr9ZpvR3IKWi7ve8YwpT6gMOjpBCLB9C+bainNL9tHckl9rtwhRJaV 76mYh3vMc0dPwBNvXeyD+JnMNH/gycdUrhkcToMihUXghgj8ZFCoj3OpRgO6UPK31L AilpHvXOx4Wly3iMaQgCsxYYaJgkK1gc01RpsJOuOs1WRZPbLk5V3Kld/a0JsvYwAH uJ0bElxDWtDYn6SOWcu7yTa5LCO09umv9mAzAUeD3c13fwY89fNue7czr8qAO1wNJk plJ8z0xD40bXw== In-Reply-To: <86pllexwru.fsf@gnu.org> Feedback-ID: 112775352:user:proton X-Pm-Message-ID: 45559e9bb00098cb4e4c12651f788b0bb883c557 Received-SPF: pass client-ip=185.70.40.133; envelope-from=pipcet@protonmail.com; helo=mail-40133.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Fri, 27 Dec 2024 09:46:24 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327195 Archived-At: "Eli Zaretskii" writes: >> Date: Thu, 26 Dec 2024 15:24:14 +0000 >> From: Pip Cet >> Cc: gerd.moellmann@gmail.com, ofv@wanadoo.es, emacs-devel@gnu.org, eller= .helmut@gmail.com, acorallo@gnu.org >> >> "Eli Zaretskii" writes: >> >> >> Date: Wed, 25 Dec 2024 17:40:42 +0000 >> >> From: Pip Cet >> >> Cc: Eli Zaretskii , ofv@wanadoo.es, emacs-devel@gnu.org= , eller.helmut@gmail.com, acorallo@gnu.org >> >> >> >> I haven't seen a technical argument against using separate stacks for >> >> MPS and signals >> > >> > You haven't actually presented it. >> >> That's correct: we have an idea and a PoC, no design to discuss or >> anything close to a proposal, at this point. >> >> My idea was to ask for obvious problems precluding or complicating this >> approach. > > OK, but still, since you wrote the code to implement it, I guess you > have at least some initial design ideas? I hoped you could describe > those ideas, so we could better understand what you have in mind, and > provide a more useful feedback about possible problems, if any, with > those ideas. The idea is that the main thread, after initialization, never calls into MPS itself. Instead, we create an allocation thread, reacting to messages from the main thread. The allocation thread never actually does anything in parallel with the main thread: its purpose is to provide a separate stack, not parallelization. All redirected MPS calls wait synchronously for the allocation thread to respond. This includes the MPS SIGSEGV handler, which calls into MPS, so it must be directed to another thread. All this makes the previously fast allocation path very slow, and we need a workaround for that: We ensure that we allocate at least 1MB (magic number here) at a time, then split the area into MPS objects when we need to. The assumption that we can split MPS allocations is significant but justifiable, because MPS will be in the same state after two successful back-to-back allocations and a single allocation combining the two. dflt_skip must never lie to MPS about the size of an object, though. Once dflt_skip told MPS how to skip it (i.e. how large the object is), we can no longer split that object. It is another significant but justifiable assumption that this happens rarely enough. > In general, as I wrote earlier, there's nothing problematic with > adding a C thread to Emacs. But since (AFAIU) the suggestion is to > run MPS from that thread, I think we should understand in more detail > how can GC be run from a separate thread. I expect that to have at > least some impact on the Emacs code elsewhere, since the original > Emacs design assumed that GC runs synchronously, and the rest of the > Lisp machine is stopped while it does. Thanks for explaining. I don't think that's a new problem (when comparing the allocation tread code to scratch/igc), as the allocation thread does not trigger GC any more spontaneously than the main thread would. The spontaneous garbage collection you're worried about can be triggered by another thread allocating memory while the main thread is busy inspecting it, but the allocatiion thread only allocates memory while the main thread is waiting, so this cannot happen. It's safe to assume no MPS collection happens when: 1. there is no other thread which might trigger a memory barrier (the allocation thread doesn't) 2. there is no other thread which might allocate memory (the allocation thread cannot do so while the main thread is in a critical section) 3. we don't allocate memory 4. we don't trigger memory barriers In practice, (4) is very hard to guarantee, so it might be easier to decide now that code should always be written to assume spontaneous GC is possible no matter where we are, which is the third step to actually enabling fully concurrent GC. Once we have made that decision, we can actually test whether it breaks things to trigger spontaneous GCs from another thread (I've experimented with this, and IIRC I fixed a bug this uncovered, but only because that bug could also have occurred without spontaneous GC). Once we've done that, we can seriously consider whether spontaneous GCs might be good for performance or usability rather than debugging. >> I've found a few minor things; so far, nothing unfixable, and no >> significant effects on performance, but the fixes will have to become >> part of the design and discussion. > > Right, so I'm asking to describe these aspects, so that others could > consider them and possibly additional issues, and provide feedback or > raise concerns about that. The main aspect is "dflt_skip must never lie, but it can delay deciding what the truth is until it's called". We keep eating ice cream until we're asked how much we've had, at which point we answer truthfully and stop eating. >> I think rr (time-travel/reverse debugging with acceptable performance) >> support is important, but I think I'm the only one? It seems to be >> really slow on this branch, though I don't know how fast it is on >> scratch/igc. > > Well, reverse debugging currently doesn't work on Windows, so at least > for that platform we cannot rely on that. Considering that rr is in a kind of arms race anyway (rseqs, hardware lock elision, the E-core/P-core split, and spectre workarounds all broke rr when introduced, and require workarounds, and CPU manufacturers appear to be fundamentally unable to agree on when an instruction should be counted as "retired"), relying on it may be a bad idea. Unfortunately, qemu doesn't seem to be seeing very active development in this area, so the qemu instruction counting mechanism is unlikely to provide usable reverse debugging in many situations. And plain old GDB reverse debugging is unbearably slow (and has been for decades), AFAIK. So it's not a safe bet that we will continue to have usable reverse debugging. It may become even more necessary to have the compiler or the CPU assist in providing it. BTW, speaking of debugging, there's always a nuclear option for signals: Use ptrace from another process to step through MPS and resend the signal at the first possible moment. Breaks GDB, hard to fix. Pip