From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.blaine.gmane.org!not-for-mail From: Helmut Eller Newsgroups: gmane.emacs.devel Subject: Re: MPS: Update Date: Tue, 11 Jun 2024 22:35:33 +0200 Message-ID: <87le3b43qi.fsf@gmail.com> References: Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" Injection-Info: ciao.gmane.io; posting-host="blaine.gmane.org:116.202.254.214"; logging-data="20413"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Gnus/5.13 (Gnus v5.13) Cc: Emacs Devel To: Gerd =?utf-8?Q?M=C3=B6llmann?= Original-X-From: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Tue Jun 11 22:36:20 2024 Return-path: Envelope-to: ged-emacs-devel@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1sH8E8-000556-4f for ged-emacs-devel@m.gmane-mx.org; Tue, 11 Jun 2024 22:36:20 +0200 Original-Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1sH8DW-0002wr-FR; Tue, 11 Jun 2024 16:35:42 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1sH8DU-0002wT-Jv for emacs-devel@gnu.org; Tue, 11 Jun 2024 16:35:40 -0400 Original-Received: from mail-lj1-x22c.google.com ([2a00:1450:4864:20::22c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1sH8DS-00064t-AE for emacs-devel@gnu.org; Tue, 11 Jun 2024 16:35:40 -0400 Original-Received: by mail-lj1-x22c.google.com with SMTP id 38308e7fff4ca-2eabd22d441so97571501fa.2 for ; Tue, 11 Jun 2024 13:35:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718138136; x=1718742936; darn=gnu.org; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=6NKCqKhDn+gr9bzuYbeQnA6SLLsks/NdOIhT1GzN9Sk=; b=iGGKSXW4B+ENMImD/vvEA45qEGjkZ6YTerxX+l5y89ptkpnSQ1adocxtpdqmlGC69A hCMm9v5Dbo+TsQ2nFOrGmZm/6H0Ipv5oRBs6+8taZpnukqMU/89mJw8dNkJIendYNlpB uV2a2lPhQ7hyWcaG9+zUZ/MpSYV1AY5EiNHd6h0hWwem+PEXOQXu00EMCC7+bZlZv8Rh eVUOVg7dlasYSLYPkSV0O/K3IWrUSBVAOgNxc6loXuOh0mEEaP1QzLoOtRmDn2pW0YQq UfMAsVMMIGOMkhn6t5GyR5I3r2E5rNPjNQI1PI1cCDicmuuTUEy6Q+wxDdAHtYS8FlRF W3DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718138136; x=1718742936; h=mime-version:user-agent:message-id:date:references:in-reply-to :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=6NKCqKhDn+gr9bzuYbeQnA6SLLsks/NdOIhT1GzN9Sk=; b=TkDhnGGVFPoNl4IfCj4umXmPhzvL69SY4Q1Cn85YLvhN1/Y88HmU7OORTnX4+CR9cz TuQsAQouplouaGcbwPbaMcx313hDDaPNa2UuDYvTtYPCb+cYilb9Z+VfGo3OMrEIlc9G s5WSWnNjyPsVq1Gbmqdx3D+I2lDIpPo9W390hJz+ugBZlvbFxOpeRdVc9zg/8CQkzx9c GEnTMtoR7bgSbXPmSOqvnLq99H3PP4cqZIbSsj5SJRSiI6xWB2/Uq5HnrB5kUG7152YL 4H48UsQUYDtfMZCPIlZAJnMX5SkxQLmCMjcgtMNu+QbHwftsSfz9xFckpcWlZxcdTOvT n//Q== X-Gm-Message-State: AOJu0Yz1st6YG6hfuAnobEx4Ufnrq+IGHcqXhySBfwbaCHHtuK6AKflJ 40QRJJY3wtTRXjGsbDWM1fNWSekSL2y7BBxmOg3fnYNDGSqDOBC38B4MEA== X-Google-Smtp-Source: AGHT+IGkFoq6Nh6WBC21rIZPuF7ZfALeuWUOrsbVFEwav0unKFLSs0Iolnzlx85qDwbu4dWBPTY/fw== X-Received: by 2002:a05:6512:b1c:b0:52c:80fb:ee76 with SMTP id 2adb3069b0e04-52c80fbef32mr9180442e87.58.1718138135364; Tue, 11 Jun 2024 13:35:35 -0700 (PDT) Original-Received: from caladan (dial-189091.pool.broadband44.net. [212.46.189.91]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6ef61aa6afsm574594666b.101.2024.06.11.13.35.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Jun 2024 13:35:34 -0700 (PDT) In-Reply-To: ("Gerd =?utf-8?Q?M=C3=B6llman?= =?utf-8?Q?n=22's?= message of "Mon, 10 Jun 2024 15:39:15 +0200") Received-SPF: pass client-ip=2a00:1450:4864:20::22c; envelope-from=eller.helmut@gmail.com; helo=mail-lj1-x22c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: emacs-devel@gnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Emacs development discussions." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Original-Sender: emacs-devel-bounces+ged-emacs-devel=m.gmane-mx.org@gnu.org Xref: news.gmane.io gmane.emacs.devel:319980 Archived-At: --=-=-= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On Mon, Jun 10 2024, Gerd M=C3=B6llmann wrote: > I now have code to copy the dump to MPS in my fork at > > https://github.com/gerd-moellmann/emacs-with-cl-packages/tree/igc > > The whole copying and so on takes a total of 0.032s with an optimized > build on my machine. The hot part of the dump is discarded after the > copy. > > I've not transferred that to GNU because I'm now through with it, for > the time being. I can't see it anymore. Please anyone feel free to > transfer this to GNU, but keep in mind that I don't have obarrays and > pure space in my fork, so some work will certainly have to be done to > make it work. Here are some patches for obarrays and pure space. I wrote them after cherry picking these revisions from your repo: git cherry-pick ce3380d6d8102dd0b704^..5bfc127264bf0f2e4f2fb92 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0001-WIP-some-fixes-for-pure-space-and-obarrays.patch >From 58ea72ee7e3b8c6e8f88e54c92c9dd71b9548b5b Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 11 Jun 2024 21:06:25 +0200 Subject: [PATCH 1/3] WIP some fixes for pure space and obarrays * src/igc.c (mirror_string): Don't change references to rodata. (mirror_obarray): Mirror the buckets array. (mirror_vector): Fix typo. --- src/igc.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/src/igc.c b/src/igc.c index b101c61b3c2..ce389f1d2d5 100644 --- a/src/igc.c +++ b/src/igc.c @@ -3976,12 +3976,14 @@ mirror_string (struct igc_mirror *m, struct Lisp_String *s) /* FIXME: IGC_OBJ_STRING_DATA is currently not used in the dump, which means string data has no igc_header in the dump. We could leave the string data alone. Not sure what's best. */ - igc_assert (pdumper_object_p (s->u.s.data)); - ptrdiff_t nbytes = STRING_BYTES (s); - unsigned char *data = alloc_string_data (nbytes, false); - memcpy (data, s->u.s.data, nbytes + 1); - s->u.s.data = data; - + if (s->u.s.size_byte != -2) + { + igc_assert (pdumper_object_p (s->u.s.data)); + ptrdiff_t nbytes = STRING_BYTES (s); + unsigned char *data = alloc_string_data (nbytes, false); + memcpy (data, s->u.s.data, nbytes + 1); + s->u.s.data = data; + } IGC_MIRROR_RAW (m, &s->u.s.intervals); } @@ -4103,8 +4105,7 @@ #define IGC_MIRROR_VECTORLIKE(m, v) \ static void mirror_obarray (struct igc_mirror *m, struct Lisp_Obarray *o) { - if (o->buckets) - IGC_MIRROR_NOBJS (m, o->buckets, obarray_size (o)); + IGC_MIRROR_RAW (m, &o->buckets); } #endif @@ -4304,7 +4305,7 @@ mirror_vector (struct igc_mirror *m, void *client) { #ifndef IN_MY_FORK case PVEC_OBARRAY: - mirror_obarray (c, client); + mirror_obarray (m, client); break; #endif -- 2.39.2 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0002-Dump-IGC_OBJ_STRING_DATA-with-headers.patch >From 8f9c59f8c51cd771a5baf06e1a5f309432a0dba3 Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 11 Jun 2024 21:14:21 +0200 Subject: [PATCH 2/3] Dump IGC_OBJ_STRING_DATA with headers This makes the mirroring code more uniform. * src/pdumper.c (dump_cold_string): Emit headers. * src/igc.c (igc_dump_finish_obj): Handle objects in pure space. (builtin_obj_type_and_hash): Renamed from builtin_obj_type. (pure_obj_type_and_hash): New helper. (is_builtin_obj_type): Abort for unrecognized cases. (mirror_string): Don't copy strings in the cold dump. --- src/igc.c | 68 ++++++++++++++++++++++++++++++++------------------- src/pdumper.c | 8 ++++++ 2 files changed, 51 insertions(+), 25 deletions(-) diff --git a/src/igc.c b/src/igc.c index ce389f1d2d5..713be9308dd 100644 --- a/src/igc.c +++ b/src/igc.c @@ -109,6 +109,9 @@ #define igc_assert(expr) \ #define igc_static_assert(x) verify (x) #define igc_const_cast(type, expr) ((type) (expr)) +#define NOT_IMPLEMENTED() \ + igc_assert_fail (__FILE__, __LINE__, "not implemented") + #define IGC_TAG_MASK (~VALMASK) /* Using mps_arena_has_addr is expensive. so try to do something that is @@ -3579,17 +3582,42 @@ is_builtin_subr (enum igc_obj_type type, void *client) } static enum igc_obj_type -builtin_obj_type (enum igc_obj_type type, void *client) +builtin_obj_type_and_hash (size_t *hash, enum igc_obj_type type, void *client) { if (c_symbol_p (client)) - return IGC_OBJ_BUILTIN_SYMBOL; + return *hash = igc_hash (make_lisp_symbol (client)), + IGC_OBJ_BUILTIN_SYMBOL; if (client == &main_thread) - return IGC_OBJ_BUILTIN_THREAD; + return *hash = igc_hash (make_lisp_ptr (client, Lisp_Vectorlike)), + IGC_OBJ_BUILTIN_THREAD; if (is_builtin_subr (type, client)) - return IGC_OBJ_BUILTIN_SUBR; + return *hash = igc_hash (make_lisp_ptr (client, Lisp_Vectorlike)), + IGC_OBJ_BUILTIN_SUBR; emacs_abort (); } +static enum igc_obj_type +pure_obj_type_and_hash (size_t *hash_o, enum igc_obj_type type, void *client) +{ + switch (type) + { + case IGC_OBJ_STRING: + return *hash_o = igc_hash (make_lisp_ptr (client, Lisp_String)), type; + case IGC_OBJ_VECTOR: + return *hash_o = igc_hash (make_lisp_ptr (client, Lisp_Vectorlike)), + type; + case IGC_OBJ_CONS: + return *hash_o = igc_hash (make_lisp_ptr (client, Lisp_Cons)), type; + case IGC_OBJ_STRING_DATA: + return *hash_o = (uintptr_t)client & IGC_HASH_MASK, type; + case IGC_OBJ_FLOAT: + return *hash_o = igc_hash (make_lisp_ptr (client, Lisp_Float)), type; + default: + NOT_IMPLEMENTED (); + emacs_abort (); + } +} + static bool is_builtin_obj_type (enum igc_obj_type type) { @@ -3625,16 +3653,17 @@ is_builtin_obj_type (enum igc_obj_type type) case IGC_OBJ_BUILTIN_SUBR: return true; } + emacs_abort(); } char * -igc_dump_finish_obj (void *client, enum igc_obj_type type, - char *base, char *end) +igc_dump_finish_obj (void *client, enum igc_obj_type type, char *base, + char *end) { if (client == NULL) return end; - struct igc_header *out = (struct igc_header *) base; + struct igc_header *out = (struct igc_header *)base; if (is_mps (client)) { struct igc_header *h = client_to_base (client); @@ -3643,13 +3672,15 @@ igc_dump_finish_obj (void *client, enum igc_obj_type type, *out = *h; return base + to_bytes (h->nwords); } - size_t client_size = end - base - sizeof *out; size_t nbytes = obj_size (client_size); size_t nwords = to_words (nbytes); - type = builtin_obj_type (type, client); - *out = (struct igc_header) - { .obj_type = type, .hash = igc_hash (client), .nwords = nwords }; + size_t hash; + type = is_pure (client) ? pure_obj_type_and_hash (&hash, type, client) + : builtin_obj_type_and_hash (&hash, type, client); + *out = (struct igc_header){ .obj_type = type, + .hash = hash, + .nwords = nwords }; return base + nbytes; } @@ -3973,17 +4004,7 @@ mirror_symbol (struct igc_mirror *m, struct Lisp_Symbol *sym) static void mirror_string (struct igc_mirror *m, struct Lisp_String *s) { - /* FIXME: IGC_OBJ_STRING_DATA is currently not used in the dump, which - means string data has no igc_header in the dump. We could leave - the string data alone. Not sure what's best. */ - if (s->u.s.size_byte != -2) - { - igc_assert (pdumper_object_p (s->u.s.data)); - ptrdiff_t nbytes = STRING_BYTES (s); - unsigned char *data = alloc_string_data (nbytes, false); - memcpy (data, s->u.s.data, nbytes + 1); - s->u.s.data = data; - } + IGC_MIRROR_RAW (m, &s->u.s.data); IGC_MIRROR_RAW (m, &s->u.s.intervals); } @@ -3999,9 +4020,6 @@ mirror_interval (struct igc_mirror *m, struct interval *i) IGC_MIRROR_OBJ (m, &i->plist); } -#define NOT_IMPLEMENTED() \ - igc_assert_fail (__FILE__, __LINE__, "not implemented") - static void mirror_itree_tree (struct igc_mirror *m, struct itree_tree *t) { diff --git a/src/pdumper.c b/src/pdumper.c index b487bc39b43..aecb7ab1a16 100644 --- a/src/pdumper.c +++ b/src/pdumper.c @@ -3556,11 +3556,19 @@ dump_cold_string (struct dump_context *ctx, Lisp_Object string) error ("string too large"); dump_off total_size = ptrdiff_t_to_dump_off (SBYTES (string) + 1); eassert (total_size > 0); + +# ifdef HAVE_MPS + dump_align_output (ctx, DUMP_ALIGNMENT); + dump_igc_start_obj (ctx, IGC_OBJ_STRING_DATA, XSTRING (string)->u.s.data); +# endif dump_remember_fixup_ptr_raw (ctx, string_offset + dump_offsetof (struct Lisp_String, u.s.data), ctx->offset); dump_write (ctx, XSTRING (string)->u.s.data, total_size); +# ifdef HAVE_MPS + dump_igc_finish_obj (ctx); +# endif } static void -- 2.39.2 --=-=-= Content-Type: text/x-diff Content-Disposition: attachment; filename=0003-Fix-some-compiler-warnings.patch >From c4a30a7f5446bef207d04d52fe83ca9071bc2e4c Mon Sep 17 00:00:00 2001 From: Helmut Eller Date: Tue, 11 Jun 2024 21:22:31 +0200 Subject: [PATCH 3/3] Fix some compiler warnings * src/igc.h (igc_alloc_lisp_obj_vec): Remove redundant declaration. * src/igc.c: Remove warning about HAVE_TEXT_CONVERSION. It's not helpful. --- src/igc.c | 4 ---- src/igc.h | 1 - 2 files changed, 5 deletions(-) diff --git a/src/igc.c b/src/igc.c index 713be9308dd..112167848a4 100644 --- a/src/igc.c +++ b/src/igc.c @@ -64,10 +64,6 @@ #ifndef HAVE_PDUMPER # error "HAVE_PDUMPER required" #endif -#ifdef HAVE_TEXT_CONVERSION -//# error "HAVE_TEXT_CONVERSION not supported" -# warning "HAVE_TEXT_CONVERSION not supported" -#endif struct Lisp_Weak_Ref { diff --git a/src/igc.h b/src/igc.h index a647fbca887..c0bed80503a 100644 --- a/src/igc.h +++ b/src/igc.h @@ -111,7 +111,6 @@ #define EMACS_IGC_H void *igc_grow_ptr_vec (void *v, ptrdiff_t *n, ptrdiff_t n_incr_min, ptrdiff_t n_max); void igc_grow_rdstack (struct read_stack *rs); Lisp_Object *igc_make_hash_table_vec (size_t n); -Lisp_Object *igc_alloc_lisp_obj_vec (size_t n); void *igc_alloc_bytes (size_t nbytes); struct image_cache *igc_make_image_cache (void); struct interval *igc_make_interval (void); -- 2.39.2 --=-=-=--