From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: MPS: User GC customizations Date: Thu, 04 Jul 2024 18:07:42 +0200 Message-ID: References: <87v81pbgzi.fsf@localhost> <87y16khvhy.fsf@localhost> <87frspqwhr.fsf@localhost> <87tth5pdqc.fsf@localhost> <87le2h47kj.fsf@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="13672"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Ihor Radchenko , Eli Zaretskii , emacs-devel@gnu.org, eller.helmut@gmail.com To: Pip Cet Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu Jul 04 18:08:35 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sPP0d-0003J4-4T for ged-emacs-devel@m.gmane-mx.org; Thu, 04 Jul 2024 18:08:35 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sPP06-0000jN-R1; Thu, 04 Jul 2024 12:08:02 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sPP05-0000iw-4X for emacs-devel@gnu.org; Thu, 04 Jul 2024 12:08:01 -0400 Original-Received: from mail-ej1-x62a.google.com ([2a00:1450:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sPOzp-0008FL-RI; Thu, 04 Jul 2024 12:08:00 -0400 Original-Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a7541fad560so94328766b.0; Thu, 04 Jul 2024 09:07:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720109263; x=1720714063; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WA9ZjqAwqlRDg8OwumHQr7Zy6NI7MBv70UsEkpZCVSQ=; b=DW6rYAJx5FnTIylJ7Rt9n7qHayxrch5lO1fPh/shpZQOh791pp1IlfB0J03/ZnDxVm ZYR8s9E9seqs50fzyD+lhrbLqorS9dBL13DACcUQHyVOHedbBXcC/1od1ZRUc+URiw3p 20UMR4A7zettBv883tgxbhIH7neFJ6qZPlryXZ0zTFLLMv+h0xg7li5VIfqBh98X5/mb b4I0ctATmlaMimLZr+WWdyttKsUbJM2/H7vQeK1sLu+x3Xhe2LiVWtZs3uejwhrJXPy9 2UWWkEoC5VQy853P7zsxSymY/tJoLX4I2DR1Ef6YQkNC7VPkom9FSSewMByIbujfL03s KkbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720109263; x=1720714063; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=WA9ZjqAwqlRDg8OwumHQr7Zy6NI7MBv70UsEkpZCVSQ=; b=PuH63sEytnkJwVNvDIhxNHsJ+kDnRu2W1Nn7LQ+Yg0+fM9TTQSrzRWHjOFmk+/ag7Q wjGFHfgeSUuoC4wPBmz5o3eoH0b6sYZgvFA4NMDjeXRtpCTcVBid8avAD0q8lAVuqP6o UKIgi/+AUZJRnYVfluKZzOXDWW/gQZfOK79BpE4RfAqEYuEXawBq/6fU5IriDCh0UBrp 4gLy5VIYQjoOi4qfPT9/dfWcngoRQUMOgj0Wa45Us6yKGIj9b5KEYbM0IGCxs/UyNY+j JiTqJv+sNXhkaS+KUYQAwrxQluouT/C75XYNq+bqXU586amtS8fznXZEHU4Yrec65uK+ i01Q== X-Forwarded-Encrypted: i=1; AJvYcCUac+WJMbM1YpLjaOQPEnZHJJ1gFuMGvVDIXvMnBNqNpHfnE8br19xUOpLm3G1ZHUOwZ2firZV9y+o6UgoLcaKr/SLI+9jJLcxSGgjClyxW7fM= X-Gm-Message-State: AOJu0Yw5FM0/vSWoN501PLZ5GjJxhteOWx7mhdk1fLU9mQeRW+07AJSN lAmDaQG428Yb8r3VqpXye+xWZd6LEc1Af6m/CyNVStnQs1vUKIGm X-Google-Smtp-Source: AGHT+IEXGDAx4zzFYSg4L/deDH0WomJ3GjaH9nCJV1J9uvh2HFtIEMZXxvtq74JGhmOdvXk0j17gHg== X-Received: by 2002:a17:907:b9c4:b0:a72:7e1e:6301 with SMTP id a640c23a62f3a-a77ba7e1a06mr153714566b.61.1720109263303; Thu, 04 Jul 2024 09:07:43 -0700 (PDT) Original-Received: from pro2.fritz.box (pd9e36fa8.dip0.t-ipconnect.de. [217.227.111.168]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a72aaae7ad5sm602610566b.0.2024.07.04.09.07.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Jul 2024 09:07:42 -0700 (PDT) In-Reply-To: (Pip Cet's message of "Thu, 04 Jul 2024 15:12:50 +0000") Received-SPF: pass client-ip=2a00:1450:4864:20::62a; envelope-from=gerd.moellmann@gmail.com; helo=mail-ej1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:321337 Archived-At: Pip Cet writes: > I think we can just set flags for "called MPS" and "in a scan > function" and look at them in the SIGPROF handler to distinguish the > four cases? Not sure. What if MPS calls these in its own thread? I guess that wouldn't be so interesting for Ihor. > My suspicion is that most problems are going to be due to large > objects creating large segments which we have to scan completely, but > that's a (micro-?) optimization for another day. > > Would it be okay to keep track of the largest object/pvec of each type > in igc.el/Figc_info? I've got a patch here which does that, and I > think the number is at least as interesting as the average :-) Good idea! That first weill showing the number of objects and average size in igc-stats. > >> > > What this variable does is give MPS notice that the client is curren= tly >> > > idle and it might be a good time to do some work. >> >=20 >> > Without understanding what effect setting this variable has on the Ema= cs >> > responsiveness, it does not seem very useful. (Exactly because MPS is >> > concurrent) >>=20 >> That was kind of my point. We can use idle time for using this variable >> and see what effect it has, especially in interactive use. Sorry if that >> wasn't clear. I'm personally trying 0.1 (100ms) here at the moment. > > I think it might very well have an effect, but being able to tune the > MPS is important, too, both for debugging and to improve performance. > And the documentation seems to be quite minimal. > >> > > > For example, my recent measurement of building agendas displayed 3= 0% of >> > > > the time spend in GC. (whatever this means in the context of our h= andling >> > > > of SIGPRF) >> > >=20 >> > > Exactly, whqt does it mean? And if we don't know, why is it an examp= le >> > > for anything? >> >=20 >> > AFAIU, on master, SIGPROF handled while our vanilla GC is running >> > will record it. In contrast, on scratch/igc, SIGPROF will put all the >> > time when igc_busy_p () is non-nil into "GC". >>=20 >> Right. And I wonder if that simply is because MPS is doing stuff in its >> own thread. > > That (or another thread calling MPS to make an allocation) would definite= ly show up as a false positive. > >> > And igc_busy_p is not only non-nil when MPS is pausing Emacs to do its >> > job, but also during object allocation. So, on master, profiler "GC" >> > field records real GC pauses, while on scratch/igc "GC" field is GC >> > pauses + new object allocation. >>=20 >> The docs say >>=20 >> -- C Function: *note mps_bool_t: 129. mps_arena_busy (mps_arena_t >> arena) >>=20 >> Return true if an *note arena: 16. is part of the way through >> execution of an operation, false otherwise. >>=20 >> =E2=80=98arena=E2=80=99 is the arena. >>=20 >> Note: This function is intended to assist with debugging fatal >> errors in the *note client program: d0. It is not expected to >> be needed in normal use. If you find yourself wanting to use >> this function other than in the use case described below, >> there may be a better way to meet your requirements: please >> *note contact us: d8. >>=20 >> What "partly through an operation" means is anyone's guess at this >> point. Someone would have to consult the sources. The docs don't say >> what you are suggesting, from my POV. > > IIRC it just checks whether the arena lock is held, whenever that > might be. Good to know, thanks! That's what I was suspecting. > >> > My figure of 30% says that igc_busy_p () is for 30% of CPU time, which >> > is a significant number. But it is not very useful unless we get some >> > idea about which part of it is memory allocation and which part of it = is >> > MPS pausing all Emacs threads. >> >=20 >> > Ideally, we should also have some way to get the allocation times on >> > master. Then, we can compare them directly. >>=20 >> Maybe it would be interesting what the meansurements look like on macOS, >> where the check for igc_busy are not needed in the SIGPROF handler. > > How sure are we of that, by the way? My understanding is there are two > ways signals can interfere with one another: SIGSEGV -> SIGPROF -> > SIGSEGV (which wouldn't happen on macOS) and alloc -> SIGPROF -> > SIGSEGV, which might. I'm not an expert and so on, so just talking about my understanding: Darwin is a kind of Mach descendant, and as such it uses Mach exceptions for hardware faults. Among those is EXC_BAD_ACCESS which one gets for protection faults. Exceptions are handled by installing a "port" which runs in its own thread and which the OS invokes when the exception occurs. Exception handling is "synchronous", which I think means the OS suspends the thread which caused the exception, invokes the port for doing what it wants, and when done let's the original thread continue, if the port wants that. Signals on Darwin are something different and used for non-hardware cases.