From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: Some experience with the igc branch Date: Tue, 24 Dec 2024 15:12:40 +0100 Message-ID: References: <87o713wwsi.fsf@telefonica.net> <87ldw7fwet.fsf@protonmail.com> <87a5cnfj8t.fsf@protonmail.com> <86seqe4j4f.fsf@gnu.org> <87ttaucub8.fsf@protonmail.com> <87pllicrpi.fsf@protonmail.com> <864j2u442i.fsf@gnu.org> <87ldw6as5f.fsf@protonmail.com> <86o7112rnq.fsf@gnu.org> <867c7p2nz4.fsf@gnu.org> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="40048"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: pipcet@protonmail.com, ofv@wanadoo.es, emacs-devel@gnu.org, eller.helmut@gmail.com, acorallo@gnu.org To: Eli Zaretskii Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Dec 24 15:13:30 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1tQ5f8-000AHV-8F for ged-emacs-devel@m.gmane-mx.org; Tue, 24 Dec 2024 15:13:30 +0100 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1tQ5eR-0001M0-Oi; Tue, 24 Dec 2024 09:12:47 -0500 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1tQ5eQ-0001Ll-NR for emacs-devel@gnu.org; Tue, 24 Dec 2024 09:12:46 -0500 Original-Received: from mail-ed1-x531.google.com ([2a00:1450:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1tQ5eO-0003su-Ue; Tue, 24 Dec 2024 09:12:46 -0500 Original-Received: by mail-ed1-x531.google.com with SMTP id 4fb4d7f45d1cf-5d3d479b1e6so6912666a12.2; Tue, 24 Dec 2024 06:12:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735049561; x=1735654361; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=6s3F9K/gtqOPW8fzW2DzskQxJaCC615Neo4JEP4HQuk=; b=RVcYItnRrk+V29m/Oqb2p+0zjpoc2uiuByEXqdrPr0009OZJM5dnhwTgmGuK6gJ/Nx f1BvYqsc+jkqLRoaXdCn1XLHfMoi12u4membpTWCM56aj6Jyl1NH0mhmWCWoBiINfDDr LIlF1sLjyFayvI+X1CyTJiyYV3Zu3OMPug1kEB+ZoERHQHy7Yi3uv5dZoVfnWs2j2qsA YPa8DhuHprwxUyMYCQGfPY3R+x3qSI0TF2+qp1Z+6OP7ZxBJ7bcHh8Ppi/vHs/Jy5M52 wXkMpXJtt0ok6YIqMNMJmBQcv3MWcJ3GVrjVEqcsuoNts+WccsQsdH6fHR1J7zzH5lJo dEUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735049561; x=1735654361; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6s3F9K/gtqOPW8fzW2DzskQxJaCC615Neo4JEP4HQuk=; b=imY75fuHrvy1LLtmhS1k+T+106CLV0l/gyJG1XyEM+dckmP/xpUV76YgyZBQfYiazi XaXyMRFJfPkg9boQWcHAzkwpMWfWDTSP5IDdq1GWQsnfEWFo8Hti0cQ8l+u0ZKcbNwdq 695OVHezHZmWOrsVK9zcT4xBj5N9tgQMdGsA3RfWjGazjg6mBQLQn5bWh9ElE6O55yug SagKq1gZPmttBrM+xNNXXq+q2BSCn+fpKTanu7aZ1Q5XXGTyrjTviTDMiA0rpylpR2IH TNAGelQi+9Z2EcVjFqlLKOw+le0YSOJiv1T4CSVCaRBNidTwtDHxPfoHAjPQubrzUnD5 54cg== X-Forwarded-Encrypted: i=1; AJvYcCWMYC+Jti5qlu7aME0/zdPVycgyfg3TLQn8+19TuXn00adYi5JEj9L66vTUtGWfv/ZOrtl413tSYiw+Pds=@gnu.org, AJvYcCXnhlw13xvSJ53CPNPG0OUKq5uWtgyjcLT3f/q68D/qmrl2eRV7B1zVS8MkA4Fx2EYL39EBeFp0xQ==@gnu.org X-Gm-Message-State: AOJu0YzNA4AHAck40ycsb1kPFj1GPp2bQMoYLtL0WMbd2yBE0xQ75TJw yeITspbIzaVw/XnfxrkXhHcEJ6bjObYdWz7Q/DU4+oCdYFfsCsRYKJQcfg== X-Gm-Gg: ASbGncumnAWnABAF9YD3Ub1iDuRUcnMQTkgYe1z0NYarOl9gMxbYz0lj1ZLFHDiZXQ6 MUXfcUhXF6IniCI/9KYOQk4o9xvQYXi301JmtMOZFec14p16LgvkfaWTVpkSdX5bQdbHJGB59M7 IlEWnIg+7vqTKcivLPWvWgoPGOJVTEyFigY8Gnaua5dAIOSuGqdxrtaCCrFplbUgFT31Dr7h9yY EeKZFLwkeEHZ0tsuPWtcI7nvNkCCoMWHhWO5fTnfqy+oHBlV5/M9ljZG19dqqhr+H7fIin7ef8M LPkW/yzhvaBc7YTyhRFmEOWe89k0BArmq5bh+gGsC+1NLULKFZxhhSuOKU/o5t9h3w== X-Google-Smtp-Source: AGHT+IFlgSowXh3dglASjFeOEh1yV5EfH9PKtwqLyJu1pEbosPfNL7BMCq5+IQZhB4bvZH4iOuvKpg== X-Received: by 2002:a05:6402:27d2:b0:5d0:c697:1f02 with SMTP id 4fb4d7f45d1cf-5d81ddc0609mr34570321a12.17.1735049561385; Tue, 24 Dec 2024 06:12:41 -0800 (PST) Original-Received: from pro2 (p200300e0b7326d00f9ed2197837c3ebd.dip0.t-ipconnect.de. [2003:e0:b732:6d00:f9ed:2197:837c:3ebd]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aac0f0159f1sm655496466b.154.2024.12.24.06.12.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Dec 2024 06:12:40 -0800 (PST) In-Reply-To: <867c7p2nz4.fsf@gnu.org> (Eli Zaretskii's message of "Tue, 24 Dec 2024 15:46:07 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::531; envelope-from=gerd.moellmann@gmail.com; helo=mail-ed1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:327016 Archived-At: Eli Zaretskii writes: >> >> I'm using SIGPROF below to make it more concrete. Similar for other >> signals. >> >> The idea is to get the backtrace in the SIGPROF handler, without >> accessing Lisp data. That can be done, as I've tried to show. >> Then place that backtrace somewhere. > > Let's be more accurate: when I said "Lisp data", I actually meant any > data that is part of the Lisp machine's global state. That's because > you cannot safely access that state while the Lisp machine runs (and > modifies the state). You need the Lisp machine stopped in its tracks. > Agreed? Ok, let's use that definition. > Now, with that definition, isn't specpdl stack part of "Lisp data"? > If so, and if we can safely access it from a signal handler, why do we > need to move it aside at all? And how would the "message handler" be > different in that aspect from a signal hanlder? We're coming from the problem that MPS uses signals for memory barriers. On platforms != macOS. And I am proposing a solution for that. The SIGPROF handler does two things: (1) get the current backtrace, which does not trip on memory barriers, and (2) build a summary, i.e. count same backtraces using a hash table. (2) trips on memory barriers. So, my proposal, is to do (1) in the signal handler and do (2) elsewhere, not in the signal handler. Where (2) is done is a matter of design. If we use Helmut's work queue, it would be the main thread, I suppose. In any case we're in "normal" multi-threading territory, with the usual restrictions and so on, but these are restrictions Emacs has. And we don't need anything from MPS, which might or might not be possible to get. > >> In an an actor model architecture, one would use a message that contains >> the backtrace and post it to a message board. I used that architecture >> just as an example, because I like it a lot. In the same architecture, >> typically a scheduler thread would then assign a thread to handle the >> message. The handler handling the profiler message would then do what >> record_backtrace today does after get_backtrace, i.e. count same >> backtraces. > > What is the purpose of delaying the part of record_backtrace after > get_backtrace to later? Is the counting it does dangerous when done > from a signal handler? That part (2) which can trip on memory barriers because it accesses MPS-managed memory like vectors and so on. > >> That's only one example architectures, of course. One can use something >> else, like queues that are handled by another thread, one doesn't need a >> scheduler thread, and so on, and so on. Pip's work queue is an >> example. > > Doing this from another thread raises the problem I describe above: we > need the Lisp thread(s) stopped, because you cannot examine the data > of the Lisp machine while the machine is running. And if we stop the > Lisp threads, why do we need the other thread at all? > > I guess we are tossing ideas without sufficient detail, so each one > understands something different from each idea (since we have > different backgrounds and experiences). My suggestion is that to > describe each idea in enough detail to make the design and its > implications clear to all. A kind of DR, if you want. Then we will > be on the same page, and can have an effective discussion of the > various ideas. I hope the above helps. Please understand that I'm not proposing a ready-made design, but mainly recommend moving (2) out of the signal handler. Sorry if that was too abstract so far, I guess that's just the way I'm thinking. If it helps, maybe we should concentrate on solving this with Helmut's work queue. Put the backtrace from (1) in the work queue, then do (2) where the work queue is processed. Something like that.