From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Pip Cet via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: Re: Some experience with the igc branch Date: Wed, 25 Dec 2024 10:48:48 +0000 Message-ID: <87ikr89gyp.fsf@protonmail.com> References: <87o713wwsi.fsf@telefonica.net> <87ldw6as5f.fsf@protonmail.com> <86o7112rnq.fsf@gnu.org> <867c7p2nz4.fsf@gnu.org> <87y104aih6.fsf@protonmail.com> Reply-To: Pip Cet Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="22957"; mail-complaints-to="usenet@ciao.gmane.io" Cc: Eli Zaretskii , ofv@wanadoo.es, emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org To: =?utf-8?Q?Gerd_M=C3=B6llmann?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed Dec 25 12:50:05 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tQPtt-0005nU-B5 for ged-emacs-devel@m.gmane-mx.org; Wed, 25 Dec 2024 12:50:05 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tQPst-0003qQ-KE; Wed, 25 Dec 2024 06:49:03 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQOwi-0001bH-1H for emacs-devel@gnu.org; Wed, 25 Dec 2024 05:48:56 -0500 Original-Received: from mail-4316.protonmail.ch ([185.70.43.16]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQOwg-0003VP-7I; Wed, 25 Dec 2024 05:48:55 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com; s=protonmail3; t=1735123731; x=1735382931; bh=HBKN4zcp1zlFgJEpiKSz4AOL3YvzycK35Kf1vCvG7x4=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector:List-Unsubscribe:List-Unsubscribe-Post; b=DZoUeqeHo5xizilsV2UTGA4eYYX/e1jebrIkqpaQdwOjDaB9Af72MiADxpWbofgMF X86RSAdFO0C5kFZTaCAZmDzlspW/6EXf8v1a/tsnho+J66T76+Mn7RpzztMkJ1Dtya oYEnNXoXrW5f7KA/y974nbznhQL9BkBb75H/g21dN4/QfcBDIthlylF/irpbPSrWcf WhmB0KVVKp2wEJNDeg8vV8fm04ZWUDMZEMO+PXuvTlHIqwoSaBZR7g00ENzWHtC36f //4IToV4KLg/3YRizZoxo5RFe6VZpEQ+L31TCuPS/k6r8tQNSJ2EHgn1HGIZuWODIu T9yi7Y09LruSg== In-Reply-To: Feedback-ID: 112775352:user:proton X-Pm-Message-ID: bb85750144f2e55443f2581812db9acb001a35fd Received-SPF: pass client-ip=185.70.43.16; envelope-from=pipcet@protonmail.com; helo=mail-4316.protonmail.ch X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 25 Dec 2024 06:49:01 -0500 X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327072 Archived-At: Gerd M=C3=B6llmann writes: > Pip Cet writes: > >> Gerd M=C3=B6llmann writes: >> >>> Eli Zaretskii writes: >>> We're coming from the problem that MPS uses signals for memory barriers= . >>> On platforms !=3D macOS. And I am proposing a solution for that. >> >> I don't think that's the problem. The problem is that signals can >> interrupt MPS, on all platforms. We can't have MPS-signal-MPS stacks, >> ever. The best way to ensure that is to keep signals on one stack, and >> MPS on another stack. MacOS already does that for their SIGSEGV >> equivalent, but we need to do it for all entry points into MPS. >> >> If they don't have separate stacks, and we interrupt MPS, the signal >> handler cannot look at any MPS-modifiable memory (including roots, which >> may be in an inconsistent state mid-GC), ever. This includes the >> specpdl. We can't write to MPS-known memory, ever. This includes any >> area we might want to copy the backtrace or specpdl to. > > And I don't think that's right :-). It's completely right that in the > SIGPROF handler everything can be inconsistent. That's true both for MPS > and Emacs. For example, the bindings stack (specpdl) may be inconsistent > when SIGPROF arrives. Literally everything we do in the SIGPROF runs the > risk of encountering inconsistencies. This is getting into technical details, but I think the specpdl code was, at one point, carefully written so the specpdl stack would always look consistent, making some assumptions about the compiler in use. Then compilers changed to make this impossible (automatic inlining, LTO), then they changed to make this possible again (stdatomic.h, memory ordering), and we also introduced an unfortunate feature which breaks consistency. Now, we can (and should) restore the consistency assumption, at least if we drop that unfortunate feature (as we should). Inconsistency of the specpdl stack is avoidable, because we control both the mutator and the inspection code. Inconsistency of MPS data is not, unless we take over control of the entire MPS library so we can make far-reaching assumptions there. > I think that's already true for the old GC. There is nothing > guaranteeing that the contents of the binding stack is consistent, for > example. But we get away with it well enough that the profiler is > useful. My understanding is that was true at one point, before C caught up to memory ordering between a thread and its signal handlers, but with C11, we have everything we need to ensure consistency, at least on systems that store words atomically (we don't use memcpy for modifying the specpdl stack). And about the usefulness thing: I really want SIGPROF specifically to improve MPS performance, which means we need to do something in the "we've interrupted MPS" situation. Or at least I want the option, rather than make "signal handlers can't do anything useful if igc_busy_p()" an axiom. And if we start declaring huge chunks of Emacs data (the entire specpdl, a large area of storage for storing the "backtrace", all thread stacks, why not the pdumper area while we're at it?) as ambiguous roots, we risk ending up with AMS pools everywhere and no copying. > With MPS, from my POV, the situation is pretty similar. Try to get away > with it by not triggering MPS while in a state that we must assume is > inconsistent. That's one approach. The other approach is to keep arguing about this until we get a SIGPROF that we're actually happy with, and then we can tell people interested in other signal handlers to copy that code :-) > For me, it's not about a theoretical or even practical solution that > somehow ensures a consistent state in MPS, or some future changes in MPS > or something. It's about getting away with what we do in the profiler > _now_, as we do with the old GC. which is already seeing potentially > inconsistent state in Emacs' memory. > > I think the _now_ is also important. From my POV, we could discuss > better solutions later. I misunderstood. We've got a solution that I'm convinced we can get away with, on scratch/igc, now. It's not pretty or permanent, and I don't think it's "good enough"; but I don't think splitting SIGPROF handlers improves it enough to make it "good enough", either. Pip