From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: =?utf-8?Q?Gerd_M=C3=B6llmann?= Newsgroups: gmane.emacs.devel Subject: Re: MPS: Please check if scratch/igc builds with native compilation Date: Wed, 22 May 2024 08:27:13 +0200 Message-ID: References: <87r0du5rly.fsf@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="30716"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Andrea Corallo , Emacs Devel , Eli Zaretskii To: Helmut Eller Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Wed May 22 08:31:31 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1s9fVa-0007hd-VF for ged-emacs-devel@m.gmane-mx.org; Wed, 22 May 2024 08:31:31 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s9fSf-00076k-RK; Wed, 22 May 2024 02:28:30 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s9fRa-0006Mi-8k for emacs-devel@gnu.org; Wed, 22 May 2024 02:27:24 -0400 Original-Received: from mail-ed1-x52c.google.com ([2a00:1450:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s9fRX-0006lp-LM; Wed, 22 May 2024 02:27:21 -0400 Original-Received: by mail-ed1-x52c.google.com with SMTP id 4fb4d7f45d1cf-56e69888a36so10262246a12.3; Tue, 21 May 2024 23:27:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716359235; x=1716964035; darn=gnu.org; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fpcQOUM3b5VdzutWxU5A8LaG05YPD53WY9nYNDJnJmo=; b=VQfYNYLGvIIa86pyCB1Jq0eOJFAaMuzvuyZToP2NJAECBhkavt+eQBJ31nq0MPL5bd EaxyJrTI01qOOrVdqj9IcHwQdGWUiLflOvsfSMUtJrrrACN8EmY1stPxldNzUCVtkcZ2 aw/gYx0zukfIB+RQJz2/JtHtlAxSRFBJyzv3YleyiPWiFmispzyI6lovC+VobgQlYL7U 2dx8sqSWu/dHZ3RUwk/FY+GPptH0KT1zzJdtThbPR3FYK4fJRkIH86CrTrIIyNR0myHW 2GznqxGNmi/w7T2IMcmc5NI8l4VWqGX72q74U3C/m46OklyahH386C00err31ZZAWeFN thsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716359235; x=1716964035; h=content-transfer-encoding:mime-version:user-agent:message-id:date :references:in-reply-to:subject:cc:to:from:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=fpcQOUM3b5VdzutWxU5A8LaG05YPD53WY9nYNDJnJmo=; b=N8WVy2x+ZEeHel6yyLRdI+wYpoPuKjDcXg4yUy+8bg/BEmq60f9CW4u5nb/Y8pMy+V t6nft03GTJspS8ygSKleN0eQLc6mlZZZvO+cLwNnUocZg5GFYE9fn2kH5fAO+CGDlo3Q IQxAiiu14S9dI0WVHc0qs6XC0VJjulA8RtoyslS1TrNNuCDHu7WJuK1N2v00iDKnBz87 3pPDtzjumAEZFYcOfe8K9FCgYqifSfTQeP4422PTj/M2bsa9LZ5JiPvzPf/ZTARY9Ag9 CnGXmaTCYLQ6IZrPi+fiAfCAx3BZWfYNJZp/ccGlRGLHt+2jZ8GG08ptS50tBbQnTd1y P51A== X-Forwarded-Encrypted: i=1; AJvYcCXzlHAxBHHrt5Jo/I6oGNp/n/h06HKvv3cL4GfBxe1WQ+z7kCDr6bt/EOiXzaQH93PwV3IT4BQDVqJuAedsthfZctHu4SzUyPElr8WVVfYXoSQ= X-Gm-Message-State: AOJu0YyYxxCOvU9w67psXrIh8lUsHAVO2jv//nektUF/YAZ7crgYnodr m0L9ipqw2o0FTCo1dhgpyZAti0PERdcRdipuBBIUsEUwjwHeeakSbp5piA== X-Google-Smtp-Source: AGHT+IHo0E0LaHJXt3oXhfTau0i45ZtsxuA0mwFzpcIlUrXbZjUJEJrWwsVHVFsh4iVYQnnKQEg/yg== X-Received: by 2002:a50:c005:0:b0:56d:f3a7:60e with SMTP id 4fb4d7f45d1cf-57832abc6bbmr622890a12.22.1716359235042; Tue, 21 May 2024 23:27:15 -0700 (PDT) Original-Received: from pro2.fritz.box (p4fe3a357.dip0.t-ipconnect.de. [79.227.163.87]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-57829c0eb82sm1034424a12.73.2024.05.21.23.27.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 May 2024 23:27:14 -0700 (PDT) In-Reply-To: ("Gerd =?utf-8?Q?M=C3=B6llman?= =?utf-8?Q?n=22's?= message of "Wed, 22 May 2024 08:07:15 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::52c; envelope-from=gerd.moellmann@gmail.com; helo=mail-ed1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319475 Archived-At: Gerd M=C3=B6llmann writes: > Helmut Eller writes: > >>> I've checked that d_reloc is indeed scanned by fix_comp_unit. The >>> check gives me reasonable confidence that this "should work". But as >>> an alternative, I also made all the things like d_reloc in the .elns >>> ambiguous roots, so that they cannot possibly be moved, if all works as >>> expected. >> >> Registering the dump as root happens rather late. The relocation code >> allocates a hash table and stores a reference to it in >> comp_u->lambda_gc_guard_h. By that time the dump should already be a >> root. Can we register the dump earlier? AFAIU, the dumper writes zeros >> in the cells for to-be-relocated references and the scan code will >> ignore them. So I think this could work: >> >> diff --git a/src/pdumper.c b/src/pdumper.c >> index b039e375c1f..39484a16c95 100644 >> --- a/src/pdumper.c >> +++ b/src/pdumper.c >> @@ -5958,6 +5958,7 @@ pdumper_load (const char *dump_filename, char *arg= v0) >> & ~(DUMP_ALIGNMENT - 1)); >> void *hot_start =3D (void *) (dump_base + aligned_header_size); >> void *hot_end =3D (void *) (dump_base + header->discardable_start); >> + igc_on_pdump_loaded (hot_start, hot_end); >> #endif > > AFAIU, at the point above the necessary relocations haven't yet been > done, so the dump is still in its "raw" form as it is in the file. Which > means, among other things, that Lisp_Objects haven't been changed to > point to where the dump is mmap'd or where the data segment of Emacs is > and so on. So I don't think that would work. > >> dump_do_all_dump_reloc_for_phase (header, dump_base, EARLY_RELOCS); >> @@ -6002,10 +6003,6 @@ pdumper_load (const char *dump_filename, char *ar= gv0) >> dump_private.load_time =3D timespectod (load_timespec); >> dump_private.dump_filename =3D dump_filename_copy; >> >> -# ifdef HAVE_MPS >> - igc_on_pdump_loaded (hot_start, hot_end); >> -# endif >> - >> out: >> for (int i =3D 0; i < ARRAYELTS (sections); ++i) >> dump_mmap_release (§ions[i]); > > But, what if we park MPS while we are loading the dump? WDYT? That would be something like the below. Alas, it doesn't change the asserts in the native comp build. Unstaged modified lisp/emacs-lisp/comp.el @@ -54,7 +54,7 @@ comp "Emacs Lisp native compiler." :group 'lisp) =20 -(defcustom native-comp-speed 2 +(defcustom native-comp-speed 0 "Optimization level for native compilation, a number between -1 and 3. -1 functions are kept in bytecode form and no native compilation is perfo= rmed (but *.eln files are still produced, and include the compiled code in @@ -68,7 +68,7 @@ native-comp-speed :safe #'integerp :version "28.1") =20 -(defcustom native-comp-debug 0 +(defcustom native-comp-debug 2 "Debug level for native compilation, a number between 0 and 3. This is intended for debugging the compiler itself. 0 no debug output. modified src/igc.c @@ -2281,6 +2281,18 @@ igc_park_arena (void) return count; } =20 +void +igc_park (void) +{ + mps_arena_park (global_igc->arena); +} + +void +igc_release (void) +{ + mps_arena_release (global_igc->arena); +} + static igc_root_list * root_find (void *start) { modified src/igc.h @@ -121,6 +121,8 @@ #define EMACS_IGC_H size_t igc_hash (Lisp_Object key); void igc_create_charset_root (void *table, size_t size); specpdl_ref igc_park_arena (void); +void igc_park (void); +void igc_release (void); void igc_check_vector (const struct Lisp_Vector *v); void igc_postmortem (void); void igc_on_grow_specpdl (void); modified src/pdumper.c @@ -5809,6 +5809,10 @@ pdumper_load (const char *dump_filename, char *argv0) /* We can load only one dump. */ eassert (!dump_loaded_p ()); =20 +#ifdef HAVE_MPS + igc_park (); +# endif + int err; int dump_fd =3D emacs_open_noquit (dump_filename, O_RDONLY, 0); if (dump_fd < 0) @@ -5980,6 +5984,9 @@ pdumper_load (const char *dump_filename, char *argv0) if (dump_fd >=3D 0) emacs_close (dump_fd); =20 +#ifdef HAVE_MPS + igc_release (); +#endif return err; } =20