From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: MPS: pdump Date: Thu, 02 May 2024 20:28:42 +0200 Message-ID: References: <87bk5tc1j6.fsf@gmail.com> <877cghc0yy.fsf@gmail.com> <86jzkhu5rv.fsf@gnu.org> <87ttjlabic.fsf@gmail.com> <87v8408wsr.fsf@gmail.com> <87o79sasl5.fsf@gmail.com> <86edanqf8x.fsf@gnu.org> <86v83wlf1l.fsf@gnu.org> <87msp86yh5.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="38818"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Helmut Eller Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Thu May 02 20:29:31 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1s2bBQ-0009rs-B6 for ged-emacs-devel@m.gmane-mx.org; Thu, 02 May 2024 20:29:28 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s2bAp-0007vQ-Op; Thu, 02 May 2024 14:28:51 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s2bAm-0007tV-5V for emacs-devel@gnu.org; Thu, 02 May 2024 14:28:48 -0400 Original-Received: from mail-ej1-x62c.google.com ([2a00:1450:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s2bAk-0006mT-FW; Thu, 02 May 2024 14:28:47 -0400 Original-Received: by mail-ej1-x62c.google.com with SMTP id a640c23a62f3a-a51addddbd4so967995766b.0; Thu, 02 May 2024 11:28:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714674524; x=1715279324; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=hYs7pQSzrPTYM9rjcT2+guMRXnZhcge7/SLQiiYTQFQ=; b=jD1By3xUudezkbPwcHOYQ2hrQd029Gwi5O0oZF0RWeEPj5aOG5fkL/GhVzfq3Mhwtg Hw3Q6N6FovJW77jXn0Xm7Qw3vebZzyNO2kt6k8X/uRP6Yg9PHMSN2CLoWuStn01yEL07 C0yqOgt/k55qTokeXRC7pANtTyX96Venm50P95dOtv/5t9ZK47qw0j1bS8a5jSK2+Jgy F4YOzYm+EN/rZhQvFvUpE7q8wU4SQtcjp1GJneEwH8AP/O5ahREHKu01U/r+3hcj04QP +ljn8nDWQmc3KPcJA/Q5e3POFhIr+z4DeBIxCYLeSExb4CQWC3U7yS8j+dv6sV7be+vV hx4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714674524; x=1715279324; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=hYs7pQSzrPTYM9rjcT2+guMRXnZhcge7/SLQiiYTQFQ=; b=s6XFJ4HsCUV2T0TcBEMnM92FkBwtkPI0jl7QfyKCtp/fHG0HbU+5CfXpBYW1njUUp8 1+QtJmNi/VfS54KoNvqVvDrZTANOFypT0PTcYh+bLQ82fB8I3i3PXpTjmlg5OBuperrZ MPrFEpQ+Rbdqae+UK6ph/O5D3ov8bPqgzs9VIyUgbtYRqA/JNlRRSBOQBBX8ym2Z6YqL /FqKlhxzH76PJb+EVARk+dMwjVTngWOGvR/eTTN8wOk/XPP+YsWFFngV/iwNDQ0tlo9d EisQO00VviF85J05yIBoa5lEtAez4BRoiG+NTs1y1dyeN759n5mQT8vMf+AFIwV/6W5Y 51aQ== X-Forwarded-Encrypted: i=1; AJvYcCVe2vS2bw4A6WBF6wcTNOsve5WrfBVGJ2FGW5UbrrpF7o3oxkXbvAL343N4k6Kmgg/ncybZOlBMDon8sPrkf9bAHhUM X-Gm-Message-State: AOJu0YwGUR52bGMBlxhOqZ//xtFiiRnZUH74e27TdYVqcGTIJ0j/o+Ar TPWyIBBSU5zRBDmdg+VYS24GFaNrzJP/tUrrSnhVzCNgxDpCATkykBsP1g== X-Google-Smtp-Source: AGHT+IEXaDYkutGMHmOB+LO9wCbEDrf2EmmMmvla3LaqMIuHsM7ntlK3ha1i+Rc9FvYdc6shwd7CFQ== X-Received: by 2002:a17:906:5591:b0:a46:f9c2:f059 with SMTP id y17-20020a170906559100b00a46f9c2f059mr230672ejp.22.1714674523892; Thu, 02 May 2024 11:28:43 -0700 (PDT) Original-Received: from pro2.fritz.box (p4fe3adac.dip0.t-ipconnect.de. [79.227.173.172]) by smtp.gmail.com with ESMTPSA id r7-20020a170906350700b00a57e2d39d56sm816893eja.223.2024.05.02.11.28.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 02 May 2024 11:28:43 -0700 (PDT) In-Reply-To: <87msp86yh5.fsf@gmail.com> (Helmut Eller's message of "Thu, 02 May 2024 17:10:46 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::62c; envelope-from=gerd.moellmann@gmail.com; helo=mail-ej1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:318605 Archived-At: Helmut Eller writes: >> @Helmut: Did we already talk about what the problem with the frame in >> the loaded pdump could be? Sorry that I don't remember. > > I never heard of that before. As the famous philosopher Manuel Manousakis says: Katastrophe! :-) Okay, I think one can understand this best when I try to describe what the pdumper does. Let's start with generating a pdump. I'll try to leave out as much details as a can. When we create a pdump, we start by allocating 3 big memory blocks which I'll call H (hot), C (cold), and R (relocs). We then traverse the graph of live objects like the old GC, starting from known roots. Each newly encountered object is copyied to H or C in binary form. C is used for leaf objects like strings and floats, H for the rest. The copying of objects is done by invoking type-specific functions, example dump_float, dump_vector, etc. We cannot use memcpy for the copying because we need more information when the pdump is loaded, namely relocation information, which goes to R. Relocation is necessary because both Emacs' DATA segment as well as H/C may end up at different addresses in a new process. Relocation info is recorded in S, and tells us where in the copied objects Lisp_Objects or pointers are that need patching when loaded. At the end, we write H, C, S to one big file. Good. Now let's load that file. We mmap the whole file and now have H', C', R' in the new process. C' and R'are good to go (In C are leaf objects). H' is patches according to the reloc info that is in S'. At the end of the relocation H' is ready to use. Some additional setup and initalizations, and we are good to go. I won't describe these. Thing is that H' now contains real Lisp objects of basically all types. Lisp objects contain references, so I make H' an ambig root. So far so good, but some Lisp objects contain not only references to other Lisp objects but also pointers to malloc'd memory. Not initially, in the dump, but during their lifetime. And finally we have reached face_cache. If initial_frame is an object in H', fix_frame won't be called for it. It cannot because the dump is not part of the MPS memory, and is instead traced ambigously as part of the big blob H'. Does that make any sense?