From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: MPS: Loaded pdump Date: Thu, 09 May 2024 15:37:00 +0200 Message-ID: References: <87le4jp3t0.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="14174"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , Emacs Devel To: Helmut Eller Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu May 09 15:38:09 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1s53yL-0003aO-NY for ged-emacs-devel@m.gmane-mx.org; Thu, 09 May 2024 15:38:09 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s53xL-0007s6-M5; Thu, 09 May 2024 09:37:07 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s53xJ-0007qB-Qi for emacs-devel@gnu.org; Thu, 09 May 2024 09:37:05 -0400 Original-Received: from mail-wm1-x333.google.com ([2a00:1450:4864:20::333]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s53xH-0003I0-Pm; Thu, 09 May 2024 09:37:05 -0400 Original-Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-41b782405d5so8920665e9.2; Thu, 09 May 2024 06:37:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1715261821; x=1715866621; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CIcs5d1DdfDc8axUecscfUo2JhRbngirPsrOtY6c4IU=; b=YFSX7CDGDW7IpKSnseARI7RpnhIP61hqsIURvgKLJaquhVI1JWJXf17bmcDengIeZH UQOJcfkNIeNeRYwF+7RdXwsYUygK8cPYCWDX6LFzS5pqxAtLLalpnesHMqcUJWR679PJ KvEN94XHidNqH79kY4aJ9D/T3UyGCtYy+lwiMS9gXlOC/hzgBNmvc7Vf0WkLGd90MZ0m GGqX7wX+2RJs1IjNzNbraKj4aLg2ZY0WblacQkaAsOSQNeVUy/cdWh/lqGlNc+6x1zTv Pp5D/7IG4xrOsq5tSfp3NE9l9n5xMEmwhYEBrCSKcMItejsGcg4pY59hcmsvh5+/sQbD S0WA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715261821; x=1715866621; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=CIcs5d1DdfDc8axUecscfUo2JhRbngirPsrOtY6c4IU=; b=kU1rfwcyagDaVk7oZ0UMX8KrkQXE2l4WjCCG8rw28sNHKriDRejixBlyDVwqLYbow3 b3RcYrz8f5SkdEDSpezjaXxereY21JtV2UlcD3+UzeB9R3mkU+Ot8A0yPJ5YT2uwXoC8 dH7IUzdHLpHHnWBRLJ4EwrPQdS6pw8Ok4VReszOcZUEY9z9jxBgzGv0Q/UD8EwcYPX8s FgZeFpRx4SJnVhznPwhOOQfjKJ5g916/4C0TxHFAZ7k8qOh392m46xOYu45QzqJ84aVC qhvtpZVDHiEy7NbqI3UMbQGAL1HrE5o5V/C8X9xqXDs1Y0mAEhKNsdzxbY9HUIIafpNZ YSSg== X-Forwarded-Encrypted: i=1; AJvYcCV4geXQTuB231Fwb0OATUHpuoqaRz8Zm1F4IiYZuvC6WxxNRfN64GzzUHCNNp8XgRMNuEZu9CHawntdygMDuBDlKFph X-Gm-Message-State: AOJu0YwpVbqaWPFSBBxhmW4CbyBLojC0p6e5QEPdljNoanptZNMISA3u 1yh2MN4qpFo5Cd6kjYyWxU6AMke/39IAZ6Uub4Byapfl4sFMV9ad0Twk6A== X-Google-Smtp-Source: AGHT+IH4F7JLkKACgYdDyzKZRocQdWaJ9GeIIU+pp3UmqM2BANx/rakZKpEiHuzWytGKbmWhra7daQ== X-Received: by 2002:adf:f0c8:0:b0:34d:7e30:947a with SMTP id ffacd0b85a97d-34fca24487amr5575837f8f.36.1715261821328; Thu, 09 May 2024 06:37:01 -0700 (PDT) Original-Received: from pro2.fritz.box (pd9e36021.dip0.t-ipconnect.de. [217.227.96.33]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3502b79be1dsm1769403f8f.10.2024.05.09.06.37.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 May 2024 06:37:00 -0700 (PDT) In-Reply-To: <87le4jp3t0.fsf@gmail.com> (Helmut Eller's message of "Thu, 09 May 2024 14:28:43 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::333; envelope-from=gerd.moellmann@gmail.com; helo=mail-wm1-x333.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319074 Archived-At: Helmut Eller writes: > On Thu, May 09 2024, Gerd M=C3=B6llmann wrote: > >> I feel more and more that the handling of the loaded pdump with MPS as >> it is now is not sustainable, and would like ask you for your ideas. >> >> What we do now is make the hot part of the dump an ambig root. I don't >> remember the exact numbers, but I think that's about 18 Mb of root. This >> has at least these problems, from my POV: >> >> - It is very large, and every time MPS scans roots, and that is all the >> time, the world is stopped until it has finished. That's not good for >> pause times. > > Maybe it would be better with the MPS_RM_PROT option. Do you know which > of the telemetry events could be used to measure this? No idea. I've only looked at telemetry to see if it had something helping me debug things. I didn't see anything obvious doing that at the time. Maybe you could just add the PROT and see what the difference is? > I recorded telemetry for the nbody benchmark. It seems that there are > 16 GC flips. From one TraceFlipBegin event to the next TraceFlipEnd > takes about 32 million cycles (I think the timestamps are cycles, as > returned by rtdsc). If we assume the clock frequency is 2.5 GHz that > would be about 13 milliseconds per GC flip. But I don't know what the > events actually mean and whether this includes scanning the pdump. In "Old Design" it says 7.4.3. The flip phase .phase.flip: The roots (see design.mps.root) are scanned. This has to be an atomic action as far as the mutator is concerned, so all threads are suspended for the duration. which probably means that between flips all roots are scanned. Unless there is something "new" meanwhile. Does telemetry show something concerning root? I initially thought we could get away with the root not the least because Emacs as an interactive program probably doesn't require a big throughput. When I run it with MPS, what I observe doesn't prove this immediately wrong at least. But the horrible workarounds like last for the CUs, and the half a dozen before that, which are only 90%, and the remaining 90% are still out there. That's so horrible. And on top of all that, I don't have debug info for elns ;-). And right now I rememberd pure space. What is with pure space... Not now. >> This has of course also consequences: >> >> - copying 18 Mb of hot objects + 12 Mb or so of leaf objects to MPS >> could be slow. No idea if it is. That could impact startup time (not >> important to me at all, but people have different preferences). > > It would certainly be interesting to know how long it takes. I could add a DEFUN that allocates a mixture of objects. I think all vectors of different sizes probably suffices. I don't think the type of object makes a difference. >> - copying the graph requires that the copying functions know the layout >> of Lisp objects so that the functions can exchange references in the >> old graph to the corresponding ones in the new graph. I'm getting >> exhausted already from thinking of writing such functions, and we >> don't have C++ templates to help. > > Do we need to know the layout or can we get away with just knowing the > size and the relocation information that the pdump already has? Does it have the size? I wondered that a while back, considering the marking in the old GC where I got the impression that there is maybe more info in the pdump. >> - AFAIK, but see admin/igc.org, there is no good way of allocating >> objects in an old generation, so they will maybe take some time to >> wander to an older generation. > > There is this ramp allocation pattern but it's not exactly what we > need. Yeah ;-) >> Enough rambling. >> >> Ideas, opinions, ...? > > Does Open Dylan use MPS in some way to dump/load a large amount of > state? No idea. Given that Dylan in a way came out of CL, I always assumed it had the same image-based development model. But I've never used Dylan myself. I'll try to find out.