From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Helmut Eller via "Emacs development discussions." Newsgroups: gmane.emacs.devel Subject: MPS native subrs (was: MPS make-thread) Date: Tue, 25 Jun 2024 20:42:44 +0200 Message-ID: <87ikxwamor.fsf_-_@gmail.com> References: <87v823xvq1.fsf@localhost> <87tthm3gq2.fsf@gmail.com> <87sex6ags5.fsf@localhost> <87msne3flr.fsf@gmail.com> <87frt63dvt.fsf@gmail.com> <86o77ulgk8.fsf@gnu.org> <87zfre1p3r.fsf@gmail.com> <87zfreo5u6.fsf@localhost> <87plsa1n8k.fsf@gmail.com> <87wmmio3vq.fsf@localhost> <87jzii1hbs.fsf_-_@gmail.com> <8734p61evv.fsf@gmail.com> <87iky0zvyz.fsf@gmail.com> Reply-To: Helmut Eller Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="23132"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Eli Zaretskii , emacs-devel@gnu.org To: Gerd =?utf-8?Q?M=C3=B6llmann?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Jun 25 22:53:48 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sMDAh-0005f0-4u for ged-emacs-devel@m.gmane-mx.org; Tue, 25 Jun 2024 22:53:47 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sMBjF-0003MD-FU; Tue, 25 Jun 2024 15:21:49 -0400 Original-Received: from eggs.gnu.org ([209.51.188.92]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sMBg7-0002Hk-Qi for emacs-devel@gnu.org; Tue, 25 Jun 2024 15:18:19 -0400 Original-Received: from mail-ej1-x629.google.com ([2a00:1450:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sMBf0-0002DC-Gp; Tue, 25 Jun 2024 15:17:33 -0400 Original-Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a725282b926so368413066b.0; Tue, 25 Jun 2024 12:14:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1719342873; x=1719947673; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=JovtuxDnA28GkVojuMe3yKnq++VVm6oT1uGgp22bQEg=; b=gJChH+iDYdwSpptjrvjR265VH0eD8V6d/rPa3jF3Mwf+dtcDRai6yPIsy5ubX/VP+m xxikxJR030Y0Uc32fcoxXwTR7Ghw2DDYyYlw+/vF1fpUmOd4kbf9veTUDSiKfspX8cEy 85KZZB5euK1d/dJPB+/ze+bXCAB7dSFoDkO6kF40h0aSJte6LY0qr3YwquVNhWMdnN4Q 5AcrkMp6VXYzElNLyUfIroLdETgPZp7dRgZMbUuKL1mFvkl3XrWnF3C7QMX1D6K1mKJW KjjBnwkQ8rM/1LXq31GwLSkqf6ehLEYf8rGmUnQ3+LpQgzetPBzWMf/gH45YfdO3uQfu yn8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1719342873; x=1719947673; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=JovtuxDnA28GkVojuMe3yKnq++VVm6oT1uGgp22bQEg=; b=oVHr07V+a1wEoGAxDrNr79tESVW3hd/sAX2LuaMaJMmJttEsbKF32rkiuN7PW9uVj3 uTGXE5HfVJk+S5TTtalX9FLW0S9s1kRmmsQ5VVtDQ5cnrkWUM6dOjkJ0pIin9ypPPRFl FERnv30r1Z/7T2N9YhmqEb/zsY04sNc9Qlo/d81en0iZbXyeshciJ2lwIAH5pseH2ydR VF0pBvR4MPMd3kgJybdeXk5riSmxjwEGED0otqXL2QICd+e2r0xkH5fbn+Bm/owwMLGU jC+DK/ijc5jHZqUKRBIdOHRgOjWY2dQSdQIAM8Uiy/zrJsp9NtDo0TvEBMWTPl/yeKoW 2afw== X-Forwarded-Encrypted: i=1; AJvYcCUlJD8GxOo9iLFUFgPQVMFLWScxn1sJ/OhtUVEmUK6NwpEhbS8zl0kb3dYk/SmwsQ5DQgB8D4KJrIqzwpFiPrfOdTtK X-Gm-Message-State: AOJu0YzyZ1lBjpt2C3l7ld1G4Kv5egOr/cIHTPsgy4OecumJ+JY7Rf+8 8s4q/0fQqpafua3b0TUn9sDuNTqz55RMdtDSVA0bqq79+D2/sQOS0qs+Xg== X-Google-Smtp-Source: AGHT+IHw9eqSLIemmEZf+NWLm7r6aeY+Evsv9eUsreZYFINekN/hm377uOBN9qHRgZViaVeYSCZzXg== X-Received: by 2002:a05:600c:6a1a:b0:424:ac5e:eedf with SMTP id 5b1f17b1804b1-424ac5eef8emr3296655e9.16.1719340966341; Tue, 25 Jun 2024 11:42:46 -0700 (PDT) Original-Received: from caladan (dial-187254.pool.broadband44.net. [212.46.187.254]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4247d0bea05sm220777165e9.18.2024.06.25.11.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 25 Jun 2024 11:42:45 -0700 (PDT) In-Reply-To: <87iky0zvyz.fsf@gmail.com> (Helmut Eller's message of "Sat, 22 Jun 2024 20:13:56 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::629; envelope-from=eller.helmut@gmail.com; helo=mail-ej1-x629.google.com X-Spam_score_int: -16 X-Spam_score: -1.7 X-Spam_bar: - X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1, DKIM_SIGNED=0.1, FREEMAIL_FROM=0.001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:320641 Archived-At: --=-=-= Content-Type: text/plain On Sat, Jun 22 2024, Helmut Eller wrote: >>> I thought native comp works for you? > > I haven't tried with the latest dump loading code. I think that needs > some work for native-comp-units. Here are some patches for this. I also have some questions: 1) In fix_subr, the &s->intspec.native field should only traced if the subr is a non-primitive. Because, for primitives, it's a (possibly non-aligned) char*. Right? 2) In dump_subr, the command_modes field is dumped with dump_field_lv for non-primitives but for primitives with dump_field_emacs_ptr. Is this intentional or a bug? 3) In fix_subr, why is the command_modes field only traced if HAVE_NATIVE_COMP? (Today was the first time I heard of the command-modes function; still have no clue what it's used for :-) --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-Add-some-checks-when-dumping-and-when-loading-the-du.patch >From 64f552d11cb9660d153647ba713276dd5e07ec77 Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 25 Jun 2024 16:44:35 +0200 Subject: [PATCH 1/3] Add some checks when dumping and when loading the dump * src/igc.h (igc_dump_check_object_starts): New. * src/igc.c (igc_dump_check_object_starts): Implementation. (check_dump): New. (igc_on_pdump_loaded): Use it. * src/pdumper.c (dump_igc_check_object_starts): New. (Fdump_emacs_portable): Use it. --- src/igc.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++++ src/igc.h | 3 +++ src/pdumper.c | 16 +++++++++++++++ 3 files changed, 76 insertions(+) diff --git a/src/igc.c b/src/igc.c index 4c9860c35d9..49abc136a0c 100644 --- a/src/igc.c +++ b/src/igc.c @@ -4093,6 +4093,62 @@ igc_dump_finish_obj (void *client, enum igc_obj_type type, return base + nbytes; } +void +igc_dump_check_object_starts (Lisp_Object relocs, void *dump_base, + void *hot_start, void *hot_end, + void *cold_start, void *heap_end) +{ + eassert (is_aligned (dump_base)); + eassert (is_aligned (hot_start)); + eassert (is_aligned (hot_end)); + eassert (is_aligned (cold_start)); + eassert (is_aligned (hot_end)); + struct region + { + mps_addr_t start, end; + } regions[] = { + {hot_start, hot_end}, + {cold_start, heap_end}, + }; + for (size_t i = 0; i < ARRAYELTS (regions); i++) + { + struct region region = regions[i]; + mps_addr_t p = region.start; + while (p != region.end) + { + eassert (p < region.end); + Lisp_Object r = XCAR (relocs); + relocs = XCDR (relocs); + EMACS_INT start_off = XFIXNUM (XCAR (r)); + EMACS_INT end_off = XFIXNUM (XCAR (XCDR (r))); + mps_addr_t start = (uint8_t *) dump_base + start_off; + mps_addr_t end = (uint8_t *) dump_base + end_off; + eassert (start == p); + p = dflt_skip (p); + eassert (end == p); + } + } + eassert (NILP (relocs)); +} + +static bool +check_dump (mps_addr_t start, mps_addr_t end) +{ + struct pdumper_object_it it = { 0 }; + for (mps_addr_t p = start; p != end; p = dflt_skip (p)) + { + eassert (p < end); + struct igc_header *h = p; + if (h->obj_type != IGC_OBJ_PAD) + { + mps_addr_t obj = pdumper_next_object (&it); + eassert (p == obj); + } + } + eassert (pdumper_next_object (&it) == NULL); + return true; +} + static mps_addr_t pinned_objects_in_dump[3]; /* Called from pdumper_load. [START, END) is the hot section of the @@ -4129,6 +4185,7 @@ igc_on_pdump_loaded (void *dump_base, void *hot_start, void *hot_end, /* Ignore relocs */ set_header (heap_end, IGC_OBJ_PAD, relocs_size, 0); + eassert (check_dump (h, cold_end)); /* Pin some stuff in the dump */ mps_addr_t pinned_roots[] = { charset_table, diff --git a/src/igc.h b/src/igc.h index 8beb4fd2dce..5e59a706af7 100644 --- a/src/igc.h +++ b/src/igc.h @@ -153,6 +153,9 @@ #define EMACS_IGC_H size_t igc_header_size (void); char *igc_dump_finish_obj (void *client, enum igc_obj_type type, char *base, char *end); +void igc_dump_check_object_starts (Lisp_Object relocs, void *dump_base, + void *hot_start, void *hot_end, + void *cold_start, void *heap_end); void *igc_alloc_dump (size_t nbytes); bool igc_busy_p (void); Lisp_Object igc_discard_killed_buffers (Lisp_Object list); diff --git a/src/pdumper.c b/src/pdumper.c index e0ff08c9d3d..a9d67fe203a 100644 --- a/src/pdumper.c +++ b/src/pdumper.c @@ -882,6 +882,21 @@ dump_align_output (struct dump_context *ctx, int alignment) } # ifdef HAVE_MPS +static void +dump_igc_check_object_starts (struct dump_context *ctx) +{ + Lisp_Object relocs = CALLN (Fsort, Freverse (ctx->igc_object_starts), + Qdump_emacs_portable__sort_predicate); + void *dump_base = (uint8_t *) ctx->buf; + size_t dump_header_size = ROUNDUP (sizeof ctx->header, sizeof (uintptr_t)); + void *hot_start = (uint8_t *) dump_base + dump_header_size; + void *hot_end = (uint8_t *) dump_base + ctx->header.discardable_start; + void *cold_start = (uint8_t *) dump_base + ctx->header.cold_start; + void *heap_end = (uint8_t *) dump_base + ctx->header.heap_end; + igc_dump_check_object_starts (relocs, dump_base, + hot_start, hot_end, cold_start, heap_end); +} + static void dump_igc_start_obj (struct dump_context *ctx, enum igc_obj_type type, const void *in) @@ -4471,6 +4486,7 @@ DEFUN ("dump-emacs-portable", # ifdef HAVE_MPS ctx->header.heap_end = ctx->offset; + dump_igc_check_object_starts (ctx); dump_igc_start_obj (ctx, IGC_OBJ_DUMPED_BYTES, &discardable_end); # endif -- 2.39.2 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Support-dumping-native-subrs.patch >From dd05fd62893d87c744c4b2b58d75812110bba19b Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 25 Jun 2024 19:47:46 +0200 Subject: [PATCH 2/3] Support dumping native subrs * src/pdumber.c (dump_cold_native_subr): Emit headers. (dump_do_dump_relocation): Copy the C strings out of the dump. (dump_igc_start_obj): Check alignment. --- src/pdumper.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/pdumper.c b/src/pdumper.c index a9d67fe203a..667f75ef6bf 100644 --- a/src/pdumper.c +++ b/src/pdumper.c @@ -903,6 +903,7 @@ dump_igc_start_obj (struct dump_context *ctx, enum igc_obj_type type, { eassert (ctx->igc_type == IGC_OBJ_INVALID); eassert (ctx->igc_obj_dumped == NULL); + eassert (ctx->offset % igc_header_size () == 0); ctx->igc_obj_dumped = (void *) in; ctx->igc_type = type; ctx->igc_base_offset = ctx->offset; @@ -3620,6 +3621,10 @@ dump_cold_native_subr (struct dump_context *ctx, Lisp_Object subr) /* Dump subr contents. */ dump_off subr_offset = dump_recall_object (ctx, subr); eassert (subr_offset > 0); +# ifdef HAVE_MPS + /* FIXME: more descriptive name? but igc_obj_type has no more free bits */ + dump_igc_start_obj (ctx, IGC_OBJ_DUMPED_BYTES, (void *)~0); +# endif dump_remember_fixup_ptr_raw (ctx, subr_offset + dump_offsetof (struct Lisp_Subr, symbol_name), @@ -3633,6 +3638,9 @@ dump_cold_native_subr (struct dump_context *ctx, Lisp_Object subr) ctx->offset); const char *c_name = XSUBR (subr)->native_c_name; dump_write (ctx, c_name, 1 + strlen (c_name)); +# ifdef HAVE_MPS + dump_igc_finish_obj (ctx); +# endif } #endif @@ -5738,7 +5746,13 @@ dump_do_dump_relocation (const uintptr_t dump_base, XNATIVE_COMP_UNIT (subr->native_comp_u); if (!comp_u->handle) error ("NULL handle in compilation unit %s", SSDATA (comp_u->file)); +#ifdef HAVE_MPS + /* FIXME: needs finalization? */ + subr->symbol_name = xstrdup (subr->symbol_name); + const char *c_name = xstrdup (subr->native_c_name); +#else const char *c_name = subr->native_c_name; +#endif eassert (c_name); void *func = dynlib_sym (comp_u->handle, c_name); if (!func) -- 2.39.2 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0003-Only-GUI-versions-have-images-caches.patch >From 8fd45d8daa5bd0587f2b6518d6ba556a33ec4212 Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 25 Jun 2024 20:20:38 +0200 Subject: [PATCH 3/3] Only GUI versions have images caches * src/igc.c (fix_frame) [HAVE_WINDOW_SYSTEM]: Avoid compilation errors in non-GUI configurations. --- src/igc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/igc.c b/src/igc.c index 49abc136a0c..f2f8a02927f 100644 --- a/src/igc.c +++ b/src/igc.c @@ -1714,10 +1714,10 @@ fix_frame (mps_ss_t ss, struct frame *f) IGC_FIX12_RAW (ss, &f->face_cache); if (f->terminal) IGC_FIX12_RAW (ss, &f->terminal); - if (f->image_cache) - IGC_FIX12_RAW (ss, &f->image_cache); #ifdef HAVE_WINDOW_SYSTEM + if (f->image_cache) + IGC_FIX12_RAW (ss, &f->image_cache); if (FRAME_WINDOW_P (f) && FRAME_OUTPUT_DATA (f)) { struct font **font_ptr = &FRAME_FONT (f); -- 2.39.2 --=-=-=--